For many DSLs such as templating languages it's important to use code generation to achieve acceptable performance in Python. The current version of Jinja went through many different iterations to end up where it is currently. This talk walks through the design of Jinja2's compiler infrastructure and why it works the way it works and how one can use newer Python features for better results.
Why Code Generation?
It seems like the general consensus for code generation in many dynamic language communities is: eval is evil, do not use it. However if done properly code generation solves a lot of problems easily, securely and with much better performance than an interpreter written on top of an interpreted language like Python.
Code generation is what powers most template languages in Python, what powers object relational mappers and more. It is also an excellent tool to simplify debugging.
Why Codegen is no Silver Bullet
Just because you generate code does not mean you're faster than an interpreter written in Python. This part of the talk focuses on why compiling Django templates to Python bytecode does not automatically make it fast.
Design of Jinja2
Jinja2 underwent multiple design iterations, most of which were made to either improve performance or debug-ability. The internals however are largely undocumented and confusing unless you're familiar with the code. In it however are a few gems hidden and interesting tricks to make code generation work in the best possible way.
Python's Support for Code Generation
Over the years Python's support for code generation was steadily improved with different ways to access the abstract syntax tree and to compiling it back to bytecode. This section highlights some alternative ways to do code generation that are not yet fully implemented in Jinja2 but are otherwise widely used.
by Jeremiah Jordan
Using Apache Cassandra from Python is easy to do. This talk will cover setting up and using a local development instance of Cassandra from Python. It will cover using the low level thrift interface, as well as using the higher level pycassa library.
by Frank Wiles
PostgreSQL is pretty powerful all on it's own, but did you know you can use Python as a stored procedure language? Not only does using a familiar language make development easier, but you get the power of the standard library and PyPi to boot. Come learn the ins and outs of putting Python in your DB.
Pushing logic in your database can be a blessing or a curse. While these techniques aren't appropriate for most users in most situations, it's good to know what kind of power you have at your disposal with PL/Python if the need ever arises.
Learn about how to: - Use triggers to off load data processing tasks - Fire off email and log alerts based on what is happening in your tables - Create more granular and detailed constraints such as verifying email address or credit card checksums at the database level - Replace batch jobs with in database triggers - Learn when and when NOT to use these techniques - Debugging techniques - Security concerns
Twitter's new scalable, fault-tolerant, and simple(ish) stream programming system... with Python!
Storm is a high-volume, continuous, reliable stream processing system developed at BackType and recently open-sourced by Twitter. Though most of the system (and it's documentation) is written in Java-based languages, it is possible to use in a Python environment with Python-based analysis code. At DotCloud (our application-platform-as-a-service) we're doing just that, and we'll be showing how you can too.
We collect a lot of data: we have tens of thousands of customers, many of whom have dozens of services running on our platform, each of which in turn produces dozens of metrics every second. All in all, we're dealing with millions of datapoints per minute. Storm will be the third iteration of our metrics system, an attempt at standardizing a number of previously-distinct pieces of our infrastructural software, to enable automated, real-time reactions to changes in the platform's state.
We'll start by touching on what problems Storm is (and isn't) trying to solve and why it's model is so powerful, informed by our previous attempts to solve the stream processing problem. We'll then move on to a deep dive into how to get Storm up and running with the most Python and least Java-enduced-pain possible and finish up with tips to solve some of the challenges we've encountered while adopting Storm into our Python-based development process.
What is stream processing?
High volume, Continuous, Reliable data analysis
How do people solve this today?
Storm's overall model
Why is this solution better?
The Hard Part, Made Simple:
Build a topology
Code your processors
The Simple Part, Made Hard (made simple):
Clojure (less ugh)
by Karen Rustad
This talk evaluates a well-known free software tutorial (the official Django tutorial) from the perspective of a web development novice in order to point out omissions and common sticking points and suggest improvements. More generally, this talk is useful to anyone looking to improve their project's tutorials and other newcomer-targeted documentation by approaching them with "beginner's mind".
Most free software projects make large, unstated assumptions about the background knowledge and experience of newcomers. Because the people who typically write tutorials or setup documentation for a project are veteran developers intimately familiar with the particulars of the project, it is unfortunately easy to gloss over material or forget the sorts of things that are likely to confuse or turn off newcomers. (How many free software projects assume that newcomers have expertise in bash, or previous experience in distributed version control, or knowledge of how to write a good bug report?) Left to their own devices, everyone designs for people like themselves--not just in software or design, but in documentation as well!
The solution to this problem is twofold: one, ideally, to "user test" one's entry-level documentation by watching a newcomer attempt to use it and note where they get stuck; and two, more generally, to remember what it was like to be a beginning programmer and/or to make your first free software contribution and accordingly to intentionally cultivate empathy for and consciousness of that state of being.
For this talk, we will take as an example the official Django tutorial. Django as a project has a reputation for prioritizing having high-quality and complete documentation far higher than many other FOSS projects. However, despite this high prioritization, even Django's tutorial has significant issues in terms of how well it serves new web programmers; it is primarily targeted as experienced web developers and operates better as an advertisement rather than as a how-to. As a result, it misrepresents or omits entirely whole swaths of knowledge which would be key to a newer web developer actually succeeding with Django.
In this talk, we use the perspective of a web development novice to go through the official Django tutorial, point out omissions and common sticking points, and suggest ways to improve.
Our hypothetical newcomer
Imagine a novice programmer--let's say, a CS undergraduate--coming to the Django tutorial for the first time
Fluent in Python and perhaps a couple other languages
New to web programming
Little experience reusing other people's code
Lots of experience using web applications, some sense of their user-facing capabilities
No direct experience using SQL / databases
Critical look at the Django tutorial
Projects vs. apps: "what's the difference, and why do I care?"
Relatedly, reusing other people's apps -- resources and best practices
Test driven development -- "what's this tests.py thing for?"
The biggest "gotcha": handling database schema changes (aka "What do you mean I can't edit my models anymore?")
HOWTO debug -- "what do I do when something goes wrong?"
User-facing CRUD -- Tutorial emphasizes built-in admin, but most modern web apps focus on user-generated content instead; "how do I do that in Django?"
Static file handling and template inheritance -- what would a "Django for designers" guide look like?
HOWTO install Django safely (e.g., virtualenv)
HOWTO deploy to an actual server (even "just" a hosted cloud solution like heroku or ep.io) -- aka "How do I show off this cool app I made to my friends?"
Historical perspective on why the tutorial is this way
In 2005, Django's audience was mostly intermediate-to-advanced programmers who were familiar with older, more minimal Python web frameworks
Django was originally developed for newspapers, hence the admin app emphasis
Remember, and be empathetic to, the kinds of mistakes you used to make before you became an experienced coder. Don't remember? Find someone who does!
Link to third-party add-ons that improve the project (e.g., South for migrations)
Be aware of your dependencies -- not just technical ones, but knowledge ones as well
And link to third-party tutorials or other resources that teach those well! (e.g. debugging tools, deployment info)
A tutorial can't cover everything, but it can at least mention nearly everything, with links for further details. A newbie doesn't know an important aspect of your framework exists unless you tell them--blind searching doesn't work very well.
Know who you are writing for -- and say so.
How do you bring more women into programming communities with long-term, measurable results? In this talk we'll analyze our successful effort, the Boston Python Workshop, which brought over 200 women into Boston's Python community this year. We'll talk about lessons learned running the workshop, the dramatic effect it has had on the local user group, and how to run a workshop in your city.
The Boston Python Workshop is a project-driven introduction to Python for women and their friends. It has run 6 times with the Boston Python Meetup in the last 12 months, bringing over 200 women into the local Python community. By being fully integrated into the main user group, the workshop has helped the Meetup grow to over 2**10 members and consistently draw over 15% women at its events. We'll talk about lessons learned running the workshop, the dramatic effect it has had on the Boston Python Meetup, and what it takes to run an outreach event in your city.
7th–15th March 2012