Sessions at PyCon US 2012 with slides and video in E2

Your current filters are…


Friday 9th March 2012

  • Code Generation in Python: Dismantling Jinja

    by Armin Ronacher

    For many DSLs such as templating languages it's important to use code generation to achieve acceptable performance in Python. The current version of Jinja went through many different iterations to end up where it is currently. This talk walks through the design of Jinja2's compiler infrastructure and why it works the way it works and how one can use newer Python features for better results.

    Why Code Generation?
    It seems like the general consensus for code generation in many dynamic language communities is: eval is evil, do not use it. However if done properly code generation solves a lot of problems easily, securely and with much better performance than an interpreter written on top of an interpreted language like Python.

    Code generation is what powers most template languages in Python, what powers object relational mappers and more. It is also an excellent tool to simplify debugging.

    Why Codegen is no Silver Bullet
    Just because you generate code does not mean you're faster than an interpreter written in Python. This part of the talk focuses on why compiling Django templates to Python bytecode does not automatically make it fast.

    Design of Jinja2
    Jinja2 underwent multiple design iterations, most of which were made to either improve performance or debug-ability. The internals however are largely undocumented and confusing unless you're familiar with the code. In it however are a few gems hidden and interesting tricks to make code generation work in the best possible way.

    Python's Support for Code Generation
    Over the years Python's support for code generation was steadily improved with different ways to access the abstract syntax tree and to compiling it back to bytecode. This section highlights some alternative ways to do code generation that are not yet fully implemented in Jinja2 but are otherwise widely used.

    At 2:00pm to 2:40pm, Friday 9th March

    In E2, Santa Clara Convention Center

  • Apache Cassandra and Python

    by Jeremiah Jordan

    Using Apache Cassandra from Python is easy to do. This talk will cover setting up and using a local development instance of Cassandra from Python. It will cover using the low level thrift interface, as well as using the higher level pycassa library.

    • Very brief intro to Apache Cassandra
    • What is Apache Cassandra and where do I get it?
    • Using the Cassandra CLI to setup a keyspace (table) to hold our data
    • Installing the Cassandra thrift API module
    • Using Cassandra from the thrift API
    • Connecting
    • Writing
    • Reading
    • Batch operations
    • Installing the pycassa module
    • Using Cassandra from the pycassa module
    • Connecting
    • Reading
    • Writing
    • Batch operations
    • Indexing in Cassandra
    • Automatic vs Rolling your own
    • Using Composite Columns
    • Setting them up from the CLI
    • How to using them from pycassa
    • Lessons learned

    At 2:40pm to 3:20pm, Friday 9th March

    In E2, Santa Clara Convention Center

  • Putting Python in PostgreSQL

    by Frank Wiles

    PostgreSQL is pretty powerful all on it's own, but did you know you can use Python as a stored procedure language? Not only does using a familiar language make development easier, but you get the power of the standard library and PyPi to boot. Come learn the ins and outs of putting Python in your DB.

    Pushing logic in your database can be a blessing or a curse. While these techniques aren't appropriate for most users in most situations, it's good to know what kind of power you have at your disposal with PL/Python if the need ever arises.

    Learn about how to: - Use triggers to off load data processing tasks - Fire off email and log alerts based on what is happening in your tables - Create more granular and detailed constraints such as verifying email address or credit card checksums at the database level - Replace batch jobs with in database triggers - Learn when and when NOT to use these techniques - Debugging techniques - Security concerns

    At 3:20pm to 4:05pm, Friday 9th March

    In E2, Santa Clara Convention Center

Saturday 10th March 2012

  • Storm: the Hadoop of Realtime Stream Processing

    by Gabriel Grant

    Twitter's new scalable, fault-tolerant, and simple(ish) stream programming system... with Python!

    Storm is a high-volume, continuous, reliable stream processing system developed at BackType and recently open-sourced by Twitter. Though most of the system (and it's documentation) is written in Java-based languages, it is possible to use in a Python environment with Python-based analysis code. At DotCloud (our application-platform-as-a-service) we're doing just that, and we'll be showing how you can too.

    We collect a lot of data: we have tens of thousands of customers, many of whom have dozens of services running on our platform, each of which in turn produces dozens of metrics every second. All in all, we're dealing with millions of datapoints per minute. Storm will be the third iteration of our metrics system, an attempt at standardizing a number of previously-distinct pieces of our infrastructural software, to enable automated, real-time reactions to changes in the platform's state.

    We'll start by touching on what problems Storm is (and isn't) trying to solve and why it's model is so powerful, informed by our previous attempts to solve the stream processing problem. We'll then move on to a deep dive into how to get Storm up and running with the most Python and least Java-enduced-pain possible and finish up with tips to solve some of the challenges we've encountered while adopting Storm into our Python-based development process.

    The Problem:

    What is stream processing?
    High volume, Continuous, Reliable data analysis
    How do people solve this today?
    The Solution:

    Storm's overall model
    Automatic parts
    Why is this solution better?
    The Hard Part, Made Simple:

    Build a topology
    Code your processors
    The Simple Part, Made Hard (made simple):

    Java (ugh)
    Clojure (less ugh)

    At 2:15pm to 2:55pm, Saturday 10th March

    In E2, Santa Clara Convention Center

Sunday 11th March 2012

  • Improving Documentation with "Beginner's Mind" (or: Fixing the Django Tutorial)

    by Karen Rustad

    This talk evaluates a well-known free software tutorial (the official Django tutorial) from the perspective of a web development novice in order to point out omissions and common sticking points and suggest improvements. More generally, this talk is useful to anyone looking to improve their project's tutorials and other newcomer-targeted documentation by approaching them with "beginner's mind".

    Most free software projects make large, unstated assumptions about the background knowledge and experience of newcomers. Because the people who typically write tutorials or setup documentation for a project are veteran developers intimately familiar with the particulars of the project, it is unfortunately easy to gloss over material or forget the sorts of things that are likely to confuse or turn off newcomers. (How many free software projects assume that newcomers have expertise in bash, or previous experience in distributed version control, or knowledge of how to write a good bug report?) Left to their own devices, everyone designs for people like themselves--not just in software or design, but in documentation as well!

    The solution to this problem is twofold: one, ideally, to "user test" one's entry-level documentation by watching a newcomer attempt to use it and note where they get stuck; and two, more generally, to remember what it was like to be a beginning programmer and/or to make your first free software contribution and accordingly to intentionally cultivate empathy for and consciousness of that state of being.

    For this talk, we will take as an example the official Django tutorial. Django as a project has a reputation for prioritizing having high-quality and complete documentation far higher than many other FOSS projects. However, despite this high prioritization, even Django's tutorial has significant issues in terms of how well it serves new web programmers; it is primarily targeted as experienced web developers and operates better as an advertisement rather than as a how-to. As a result, it misrepresents or omits entirely whole swaths of knowledge which would be key to a newer web developer actually succeeding with Django.

    In this talk, we use the perspective of a web development novice to go through the official Django tutorial, point out omissions and common sticking points, and suggest ways to improve.

    Our hypothetical newcomer
    Imagine a novice programmer--let's say, a CS undergraduate--coming to the Django tutorial for the first time
    Fluent in Python and perhaps a couple other languages
    New to web programming
    Little experience reusing other people's code
    Lots of experience using web applications, some sense of their user-facing capabilities
    No direct experience using SQL / databases
    Critical look at the Django tutorial
    Missing concepts

    Projects vs. apps: "what's the difference, and why do I care?"
    Relatedly, reusing other people's apps -- resources and best practices
    Test driven development -- "what's this tests.py thing for?"
    Missing features

    The biggest "gotcha": handling database schema changes (aka "What do you mean I can't edit my models anymore?")
    HOWTO debug -- "what do I do when something goes wrong?"
    User-facing CRUD -- Tutorial emphasizes built-in admin, but most modern web apps focus on user-generated content instead; "how do I do that in Django?"
    Static file handling and template inheritance -- what would a "Django for designers" guide look like?
    Assumed knowledge

    HOWTO install Django safely (e.g., virtualenv)
    HOWTO deploy to an actual server (even "just" a hosted cloud solution like heroku or ep.io) -- aka "How do I show off this cool app I made to my friends?"
    Historical perspective on why the tutorial is this way
    In 2005, Django's audience was mostly intermediate-to-advanced programmers who were familiar with older, more minimal Python web frameworks
    Django was originally developed for newspapers, hence the admin app emphasis
    Remember, and be empathetic to, the kinds of mistakes you used to make before you became an experienced coder. Don't remember? Find someone who does!
    Link to third-party add-ons that improve the project (e.g., South for migrations)
    Be aware of your dependencies -- not just technical ones, but knowledge ones as well
    And link to third-party tutorials or other resources that teach those well! (e.g. debugging tools, deployment info)
    A tutorial can't cover everything, but it can at least mention nearly everything, with links for further details. A newbie doesn't know an important aspect of your framework exists unless you tell them--blind searching doesn't work very well.
    Know who you are writing for -- and say so.

    At 1:30pm to 2:10pm, Sunday 11th March

    In E2, Santa Clara Convention Center

  • Diversity in practice: How the Boston Python Meetup grew to 1000 people and over 15% women

    by Asheesh Laroia and Jessica McKellar

    How do you bring more women into programming communities with long-term, measurable results? In this talk we'll analyze our successful effort, the Boston Python Workshop, which brought over 200 women into Boston's Python community this year. We'll talk about lessons learned running the workshop, the dramatic effect it has had on the local user group, and how to run a workshop in your city.

    The Boston Python Workshop is a project-driven introduction to Python for women and their friends. It has run 6 times with the Boston Python Meetup in the last 12 months, bringing over 200 women into the local Python community. By being fully integrated into the main user group, the workshop has helped the Meetup grow to over 2**10 members and consistently draw over 15% women at its events. We'll talk about lessons learned running the workshop, the dramatic effect it has had on the Boston Python Meetup, and what it takes to run an outreach event in your city.

    At 2:10pm to 2:55pm, Sunday 11th March

    In E2, Santa Clara Convention Center