Sessions at PyCon US 2012 about Python in H2

Your current filters are…

Clear

Wednesday 7th March 2012

  • SQL for Python Developers

    by Brandon Rhodes

    Relational databases are often the bread-and-butter of large-scale data storage, yet they are often poorly understood by Python programmers. Organizations even split programmers into SQL and front-end teams, each of which jealously guards its turf. These tutorials will take what you already know about Python programming, and advance into a new realm: SQL programming and database design.

    The class will consist of six 25-minute lessons, each of which features a 10-minute lecture, 10 minutes of interesting exercises, and a 5-minute wrap-up in which the instructor recaps the exercises by giving his own answers. The focus will be on keeping things simple so that each building block is grasped clearly. The six lessons will be laid out something like this:

    1. Tables, INSERT, and SELECT.

    • Create a simple sqlite3 table with the DB-API interface provided by the Python Standard Library.
    • Use INSERT to fill the table with data.
    • Concatenate INSERT statements to increase the speed and reduce the number of database round-trips required during a bulk data load.
    • Read back table rows with SELECT.
    • Add dynamic expressions to the rows returned by SELECT.
    • Quote values correct to avoid SQL injection attacks.
    • Avoid “gotchya” differences between Python and SQL data types, with particular attention to Unicode, date-times, and the behavior of NULL verses None.

    2. WHERE and the importance of being indexed.

    • Run quick performance checks that demonstrate that WHERE usually requires the entire table to be read into memory and scanned.
    • Add a simple index to shortcut specific WHERE clauses and return their results more quickly.
    • Check whether an index is being used, and learn several reasons why apparently useful indexes get ignored by the database.
    • Add aggregate indexes that yield performance increases for very specific WHERE clauses.
    • Investigate how our data distrubtion — for example, whether a particular column has thousands of different values, or merely thousands of instances of a handful of values — can impact the wisdom and performance of various query plans.

    3. FOREIGN KEY and JOIN

    • Use a foreign key to relate rows in one table with rows in another.
    • Add JOIN clauses to a SELECT statement to assemble query-result rows that are built from pieces of several tables.
    • Diagnose performance problems with JOIN by observing the cost of full N×M scans that compare every row from one table with every row from another.
    • Think about the indexes that a query plan could take advantage of behind the scenes.
    • Create indexes that let the database take shortcuts when doing common JOINs.

    4. Post-processing.

    • Use ORDER BY to control the rows which are returned first by a given query.
    • Combine OFFSET and LIMIT to return "paged" results suitable for displaying on a limited display, like a web page or GUI window
    • Observe how indexes affect the performance of ORDER BY / LIMIT.
    • Use GROUP BY to support aggregate operations such as sums, averages, maxima, and minima.
    • Filter aggregate results with the HAVING clause.

    The exercises will present small Python scripts that post-process data, and ask students to write the equivalent GROUP BY / HAVING expressions to remove the need for the Python post-processing.

    5. Modifying tables.

    • Write WHERE clauses for UPDATE and DELETE using the same patterns already learned for SELECT.
    • Use transactions in combinations with UPDATE and DELETE to prevent inconsistent database states from becoming visible to other clients.

    6. ORMs, Objects, and Tables.

    • Create tables of objects using the SQLAlchemy declarative schema in combination with classes.
    • Understand the main differences between SQLAlchemy and the Django ORM, including the idea of explicit saves versus a unit-of-work pattern.
    • See how ORM query syntaxes mix down to SQL statements.
    • Determine when an ORM will be helpful, versus when straight SQL might be a better solution for a particular problem.

    Of course, mastery of these topics cannot be conveyed in a single three-hour course! The tutorial will have succeeded if students learn the main moving parts that are involved in a relationally-backed Python application, if they have gotten some practice with SQL and the kind of tasks that it seeks to simplify, and if they have a foundation upon which to build when they are next faced with writing or modifying Python code that interfaces with a SQL database.

    At 9:00am to 12:20pm, Wednesday 7th March

    In H2, Santa Clara Convention Center

    Coverage video

  • MongoDB and Python

    by Rick Copeland and Bernie Hackett

    This intermediate-level class will teach you techniques using the popular NoSQL database MongoDB, its driver PyMongo, and the object-document mapper Ming to write maintainable, high-performance, and scalable applications. We will cover everything you need to become an effective Ming/MongoDB developer from basic PyMongo queries to high-level object-document mapping setups in Ming.

    The class will begin with a brief overview of MongoDB and its Python driver PyMongo. We will cover basic operations using PyMongo, including data manipulation, querying, and GridFS. Students will install MongoDB and PyMongo as part of this section.

    We will then describe the design philosophy and setup of Ming, a SQLAlchemy-inspired object-document mapper (ODM) for MongoDB developed at SourceForge.

    Next we will cover the base-level implementation of Ming, including schema design, the session and datastore, lazy migrations, data polymorphism, and GridFS support. We will also cover effective MongoDB index design, querying, and updating techniques, and how to use these with Ming. Students will install Ming as a part of this section, and have exercises covering schema design, lazy migrations, and GridFS.

    The final segment will cover the object-document mapper portion of Ming. We will cover the unit of work design pattern, object relations, ODM-level polymorphism, and how to drop down to the base layer (or even down to pymongo) when you really need to. This section will include exercises in designing your ODM model and effectively using the unit-of-work session.

    This talk targets Python 2.6-2.7 and MongoDB 2.0. Students should have Python 2.6 or 2.7 installed on their machines prior to the class and should be comfortable using virtualenv and pip or easy_install to install packages.

    At 1:20pm to 4:40pm, Wednesday 7th March

    In H2, Santa Clara Convention Center

    Coverage video note

Thursday 8th March 2012

  • Documenting Your Project With Sphinx

    by Brandon Rhodes

    Python projects can succeed or fail because of their documentation. Thanks to Sphinx, Python now has a “documentation framework” with indexing, syntax highlighting, and integration with your code. Students will be given a small undocumented Python package, and during the exercises they will give the package a tutorial and reference manual. Plus: deployment and theming!

    Python projects can succeed or fail based on their documentation. Thanks to Sphinx, Python now has a "documentation framework" that provides convenient indexing and automatic syntax highlighting, and can also integrate your documentation with your code (your documentation can be run as a test, and your class and function docstrings can become your reference documentation). Students will be given an undocumented sample Python package, and be lead through exercises that result, by the end of the tutorial, in their giving the package a full tutorial and reference manual. Deployment and theming will also be taught.

    Besides a 15-minute introduction and 15 minutes for questions and discussion at the end, the tutorial will be organized in six 25-minute sessions which each involve a short lecture and then an interactive exercise that asks the students to apply what they have just learned. Here are the major topics covered by each of the six sessions:

    • The reStructuredText markup language and its syntax; the standard doctools; and the two different conventions that Sphinx can support for laying projects out as directories and files.
    • The Sphinx documentation build process on both Unix and Windows; how to arrange your project documentation in a way that will make sense to novice, experienced, and expert users alike; and how Sphinx supports connections between different pages of documentation.
    • Running code examples in the documentation as doctests; the pros and cons of pulling docstrings from the code as API documentation (and how to do it if it proves necessary); and including non-doctest full code listings in the documentation.
    • Referencing headings in the same document; cross-referencing between documents; making class and method names automatically link to their entry in the API documentation; and how to make code objects appear in the index.
    • Theming with custom HTML and CSS, for students who happen to know web design; plugging in pre-made Sphinx themes; and how to integrate Sphinx into an entire web site for their product.
    • Shipping documentation with your package on PyPI; installing it readthedocs.org; making sure that documentation gets included with a binary install; using a version control source browser to view documentation directly in their project trunk; and deploying Sphinx to a web site.

    The Sphinx approach will be linked to other successful documentation systems in our computing heritage, most notably in the practices it shares in common with the Unix Documenter's Workbench (DWB) of the 1970s.

    At 9:00am to 12:20pm, Thursday 8th March

    In H2, Santa Clara Convention Center

    Coverage video

  • Making and understanding music with Python and a little bit of Math

    by Pedro Kroger

    Are you interested in learning more about Music but have found most material to be kind of patronizing or to present things magically instead of logically? The good news is that much of music can be understood with programming and math, two things you're already good at! In this hands-on workshop you'll learn some elements of music from a (Python) programmer's perspective.

    There are (many) music operations that can be expressed directly in Math and code. Just to give you a taste, tranposing a set of notes to an interval i is as simple as [(n + i) % 12 for n in notes].

    In this tutorial we'll explore the primitives of Music (notes, intervals, dynamics, articulations, and duration), take a look inside them, the means of combining those primitives (motives, chords, operations such as transposition, retrograde, inversion), and the means of abstraction (phrases, harmonic functions and progressions, and form) using Python and some basic Math. At the end of this tutorial you should have a demysthified understanding of many elements of Music and be able to use Python to create short compositions.

    You can find a list of required libraries and programs, with installation instructions at https://github.com/kroger/learni....

    At 1:20pm to 4:40pm, Thursday 8th March

    In H2, Santa Clara Convention Center