Sessions at DjangoCon US 2011 with slides

Your current filters are…

Monday 5th September 2011

  • Test Anything, Test Everything

    by Dave Dash

    Testing in Django is easy if you're testing models against your database. What happens when you need to test other-systems, like a search engine, or an API? This tutorial will cover how I built SphinxTestCase, ESTestCase and redisutils to allow us to maintain test coverage on our web sites.

    Testing backends other than your database can be tricky. It's entirely necessary, however, otherwise you'll start getting glaring holes in terms of coverage.

    This tutorial will show you how to subclass TestCase and properly setup and teardown external systems.

    It will also show you how to be nice to your other developers who may want to opt-out of testing backend systems (like search).

    At 9:00am to 12:30pm, Monday 5th September

Tuesday 6th September 2011

  • From Designer to Django'er in Six Weeks: A Story from Solo Founder

    by Tracy Osborn

    Django gives you the tools to start a successful web app even if you're new to development. Learn how to build and launch a web startup whilst avoiding common pitalls and problems.

    You don't have to be an expert programmer to launch a web startup; designers, marketers, and novice developers can all use Django to quickly launch a web app, build a company, and start bringing in revenue. It's a common trap to get caught up in endless development — follow these tips and tricks to get launched even when you're still learning.

    At 10:30am to 11:10am, Tuesday 6th September

  • Confessions of Joe Developer

    by Daniel Greenfeld

    Over the course of 6 years of Python development at NASA, Revsys, and Eldarion; Daniel Greenfeld has picked up a lot of easy tricks stolen from the best and brightest in the Python community that make him look good in front of his peers and clients. And now, at great risk to his career and reputation, he is going to show you how he does it.

    Highlights:

    • Get the most out of the least amount of work.
    • Being forgetful is responsible for open source development.
    • Using commonly available tools to make you look like you know what you are doing.
    • All the sample code is accessible outside the slides

    At 11:20am to 12:00pm, Tuesday 6th September

  • Monkeying Around with Python at New Relic

    by Graham Dumpleton

    This session will cover the challenges of creating a production application performance monitoring system for Python. It includes an overview of the architecture of the system, as well as how it hooks into Django and captures details about web transactions, database transactions, memcache requests, exceptions and much more.

    This session provides a technical review of the challenges associated with instrumenting Python for production monitoring. How do you place monitoring hooks into code that cannot be modified directly in order to collect meaningful performance data? Concepts covered in this session will include:

    • Using context manager objects to time execution of code.
    • Using context manager objects to catch details of exceptions.
    • Applying a context manager to functions using wrappers and decorators.
    • Applying a context manager to WSGI applications and components.
    • How to go about applying the above to existing code using monkey patching.
    • Deriving the name of a wrapped object for identification purposes.
    • Using post import hooks as a means to trigger monkey patching.
    • Driving the process of monkey patching from a configuration file.

    The talk includes real-world examples of how these mechanisms have been used to add instrumentation to the Django web framework in order to collect actionable performance data.

    At 11:20am to 12:00pm, Tuesday 6th September

  • The story and tech of Read the Docs

    by Eric Holscher

    Hosting open source documentation was a mess. The best-of-class solution for the Python world as uploading a tarball of html to packages.python.org or doing similar to upload to github pages.. We set out to solve this problem using the current best of class tools that Django has to offer.

    At 1:30pm to 2:10pm, Tuesday 6th September

  • Real-Time Django

    by Adam Miskiewicz and Ben Slavin

    The web is live. APIs give us access to continuously changing data. We discuss ways to get real-time data into your app, how to handle data processing and what to do when you get thousands of updates per second.

    "Caching is the answer" can't always be the answer. We discuss how to ensure your site is up-to-date when the data behind it is continuously changing.

    We'll discuss three primary areas pertaining to real-time-data.

    WHERE IT COMES FROM – REAL-TIME DATA SOURCES

    • Streaming APIs
    • Polled APIs (and strategies for dealing with rate limits)
    • PubSubHubbub

    DATA PROCESSING

    • Denormalization
    • Workers
    • Incremental and generational MapReduce

    DATA PRESENTATION

    • Partial caching
    • Continuous caching
    • Broadcasting real-time updates
    • Failure models

    At 2:20pm to 3:00pm, Tuesday 6th September

  • Testing: The Developer Strikes Back

    by Sandy Strong

    “Code not tested is broken by design” - Jacob Kaplan-Moss Every programming project has unique testing needs. It’s not always easy to understand what those needs are, or how to write tests that satisfy them. One of our goals as developers is to write tests that minimize failures in production that can cost our companies time, money, resources, and in many cases, hours of sleep!

    Every programming project has unique testing needs. It’s not always easy to understand what those needs are, or how to write tests that satisfy them. One of our goals as developers is to write tests that minimize failures in production that can cost our companies time, money, resources, and in many cases, hours of sleep!

    At 2:20pm to 3:00pm, Tuesday 6th September

  • State of the DSF Keynote

    by Russell Keith-Magee

    At 3:30pm to 4:20pm, Tuesday 6th September

  • A summer in the wild

    by Alex Gaynor

    I've spent this past summer away from Django, working for Quora, venturing into the depths of a foreign codebase, and making it faster by running on PyPy. This talk will feature the lessons I've learned about Django, PyPy, and web development in general.

    This talk is going to cover some pretty eclectic ground. Some of the stuff I'm going to talk about includes:

    • Making stuff run on PyPy
    • Profiling and benchmarking a webapp
    • Stuff I missed when I wasn't using Django
    • Tools every Pythonista must be using
    • Good ideas from Quora that Django should steal
    • Learning a giant codebase on the quick
    • Process is no substitute for good process

    At 4:20pm to 5:10pm, Tuesday 6th September

    Coverage slide deck

  • Secrets of PostgreSQL Performance

    by Frank Wiles

    PostgreSQL is effectively the default RDBMS for Django. Learn the dark arts of optimizing this powerful database to be blazingly fast on your own hardware or in the cloud.

    PostgreSQL has been the default recommended database since Django was initially released. Learn the dark arts of optimizing this powerful database. Both in general use cases and specifically for Django.

    In this talk you'll learn:

    ->Things to avoid
    -double duty servers
    -disk contention
    -unnecessary queries

    ->The "Big 3" configuration options you should always adjust
    - shared_buffers
    - effective_cache_size
    - random_page_cost

    ->Other useful configuration options

    ->Hardware considerations
    - RAM, disks, and CPU
    - Rackspace Cloud vs Amazon EC2
    - tablespaces
    - SSDs

    -> Common Django ORM issues
    - Slow count() operations

    -> Indexing Tips and Tricks
    - Index types
    - Multi-column indexes
    - Partial indexes

    -> Performance Tools you can use

    -> The "Dark Arts"
    - Planner Statistics
    - materialized views
    - crazy replication

    At 4:20pm to 5:00pm, Tuesday 6th September

Wednesday 7th September 2011

  • Keynote: Itch Scratching, Pros & Cons

    by Brad Fitzpatrick

    At 10:30am to 11:20am, Wednesday 7th September

  • RESTful APIs: Promises & Lies

    by Tareque Hossain

    Over the last few years RESTful APIs have become an integral part of many django projects. But some of the fundamental questions still remain unanswered. How do you decouple resources from models, formatting from definitions, authorization from authentication? How should you define resources? How do you handle pagination? Deliver facets? Prevent abuse? Implement versioning? Let's have a look.

    Implementing RESTful APIs for django applications used to be the talk of the town. Eventually a few API frameworks emerged. Some were designed to provide a quick & easy path to implementation. Some focused on decoupling components as much as possible. At the same time, numerous blog posts were published and emails exchanged seeking/ pitching sound API concepts & best practices. Somehow till this date, there is arguably no single solution that addresses all of the major concepts/practices around designing & securing RESTful APIs. Why? What were the promises? What were the lies?

    In this talk, an attempt is made in identifying these concepts/practices in light of recent experiences with a PBS Education Technology project. In designing a RESTful API, it is important to establish a relatively simple resource definition that is uniform across all resources, yet powerful enough to deliver errors, pagination, facets as well as data from attributes/ methods of various models/ instances. It is also desirable to have serialization formats decoupled from these definitions and delivered to consumers according to their preferences.

    Securing the API is a complex task as well. Not all RESTful APIs are meant for public consumption. Yet it is necessary to make AJAX calls that might leave the API vulnerable and expose security patterns. This talk highlights why it is important to have decoupled authentication backends, why 2-legged-OAuth can be an excellent choice in protecting the API and how adding pre-approved tokens can help differentiate between user/ application specific calls.

    Concepts/ best practices evolve over time. So should the API. Sometimes it is necessary to perform refactoring or optimization that alters the API significantly. In any case, establishing a path to versioning API components is a prudent choice. Here, the fundamentals are explored.

    Rest of the talk presents how PBS Education Technology team enhanced django-piston to incorporate the ideas and suggestions mentioned above. Also consumer side practices and security implementations are demonstrated using code examples.

    At 11:20am to 12:00pm, Wednesday 7th September

  • Real World Django Deployment Using Chef

    by Noah Kantrowitz

    Chef, a popular configuration management tool, is gaining momentum in the Django community. Learn how to deploy Django sites using Chef, be they one server or a thousand.

    Chef, a configuration management tool, is increasingly popular in the Django community. Many people have yet to take the plunge, and are still managing production systems exclusively through tools like Fabric or Buildout. In this talk I will quickly show the basics of building Chef recipes, both in general and a tour of the Django-and-Python-specific tools available. I will then walk through a suite of cookbooks built to deploy Packaginator as a production site in the cloud. If suitable wireless is available, I would like to do a live demonstration at the end, bringing up a new site from scratch.

    Outline:

    Introduction
    - Chef basics
    - Resources
    - Actions
    - Recipes
    - Cookbooks
    - Roles
    - Search

    Python cookbook
    - Community site/cookbooks
    - pip provider
    - virtualenv provider

    Django application recipe

    Other misc cookbooks
    - Postgres
    - Redis

    Packaginator cookbooks
    - Application recipe
    - Postgres master/slave
    - Gunicorn
    - Nginx
    - HAProxy
    - Redis
    - Celery

    Demo (pending wireless)

    Q&A

    At 1:30pm to 2:10pm, Wednesday 7th September

  • Stop Tilting at Windmills - Spotting Bottlenecks

    by Yann Malet and Brandon Konkle

    Learn the questions to ask and the tools to use to spot performance problems in your application. We won't tell you how to scale your site, but we will show you how to locate the bottlenecks as they pop up.

    Performance optimisation has always been a hot topic among the Django community, and as applications become more mature and larger companies adopt Django for high-traffic projects this issue will heat up even more. This talk will not focus on resolving performance/scalability issues but rather on discovering them. It will try to answer the following questions:

    • How do you know where the worst problems are?
    • When did the problem start, and what parts of your application are affected?
    • How do you tell the difference between the giant performance monsters and the windmills that aren't really big problems at all?

    This talk will not spell out a quick and easy 5-step method to alleviating all of your performance woes. It will, however, show you ways to measure your site’s performance over time and spot the real trouble areas as they occur. It’s vital to know both your application and your user well in order to create an effective test plan. We will discuss the importance of reliable performance measurement and realistic performance tests, and walk you through the tools we use to identify and track problems.

    • Django Debug Toolbar - get internal information about what your application is doing
    • Django Debug Logging - log Debug Toolbar statistics to the database and review details and aggregated summaries.
    • Creating a cron job to run automated performance tests at regular intervals
    • Jmeter for external tool to create load tests with high concurrency
    • Integrating Jmeter with Jenkins using the performance plugin

    To close up, we’ll mention some of our favorite resources on handling the performance problems that you encounter during your testing. We’ll also provide links to other tools that you can use as part of your testing efforts. With this information at your disposal you can charge boldly through the meadow on your faithful Rocinante, attacking the real performance problems and leaving the windmills behind.

    At 3:30pm to 4:10pm, Wednesday 7th September

  • Best Practices for Front-End Django Developers

    by Christine Cheung

    Considerations, best practices, tools, and more for front-end development in Django.

    This talk will go over considerations and best practices for front-end development in Django and beyond. Ways to:

    • Best build and organize your templates
    • Tools to make your life easier and faster
    • Optimize and test code
    • Integrate and play well with the back-end
    • Deliver the most shiny of web applications

    At 4:20pm to 5:00pm, Wednesday 7th September

  • Safely deploying on the cutting edge

    by Eric Holscher

    This talk will go through the deployment system that we have built at Urban Airship. It will cover the process that allows us to automate our deployments, making it safe to deploy at any time. It's built on Fabric, Gunicorn, Virtualenv, Rsync, Supervisord, and Python. We end up with fully isolated environments for our services that are easy to manage.

    This talk will cover how we do deployments. It is the first deployment system that I've been happy with, which says a lot. It is a system that we have been evolving over the last 6 months or so, and has made our developers lives a lot easier. There are a few different points that make it successful, as well as useful.

    Interestingly, there is very little Django specific about this setup. It is how we deploy Django applications, but we also deploy flask and standard WSGI applications using the same framework. A slightly modified version of the code is also used to deploy our Java applications, so the tooling is genericly useful for more than just Django programmers

    The talk will have 4 parts, that will cover the following:
    - The process around Version Control that we use
    - The tooling to deploy code
    - The environment that the code gets deployed in to
    - The process of deploying the code across all of our servers, a subset at a time

    The first part is that we manage our source code in a specific fashion. It's a pretty standard git branching model, where we have a master branch that is the latest stable code. All on-going work is done on feature branches. Then we have a production branch that is what is currently in production. Each push to production is tagged with the date. Git allows us to keep a record of the history of development in one place.

    Next, to actually handle the deployment of the code to the servers, we're using Fabric. To show the full power of the system, we might normally run a command like fab host:web-0 deploy:airship restart:airship monitor:airship. This is mostly abstracted out from people in the day to day use of the system. The deployment system has the following features:

    • Updating source code checkouts
    • Making VCS tags
    • Deploying code
    • Migrating the database
    • Updating media files
    • Restarting services
    • Rolling back to the previous version
    • Allowing the person deploying the ability to easier monitor logs/status of services
    • One command "smart deploys"

    The python environment that things are deployed into is a fully standardized setup using supervisord and gunicorn, along with a number of other helper utilities. It integrates into the init.d system on the host, so that normal management commands work. Each service that we deploy has it's own environment, which I'll go into more depth on.

    Finally, We are also working on a system that will allow us to automatically health check deployments and automatically rollback the deployment if we are seeing elevated error rates. We handle deploying the code to a "canary" system first, then roll it out to half our servers, then the other half. This system is implemented currently at the prototype level, but should be in production use before the talk is given.

    At 4:20pm to 5:00pm, Wednesday 7th September

  • Keynote: Designers Make It Go to Eleven!

    by Idan Gazit

    The open-source world is and always has been focused on code; attention and respect doled out in exchange for patches. As a result, many open-source projects want for design love but don't know how to secure it. We scare off potential contributors from the many fields beneath the aegis of “design”: user experience, user interfaces, usability, interaction design, information architects, graphic design, typography, and other roles assumed by the authors of our markup and styling.

    Django can be a model citizen among open-source projects in changing these attitudes and coaxing contributors out of the woodwork—not just for the benefit of the framework, but for the constellation of 3rd party apps which make Django great. Other FOSS communities have made efforts that we can learn from, and the fight to change perceptions is less uphill than you might think.

    This talk will cover:

    • Misconceptions of “Design.” What is design, and what do designers do?
    • A brief trip down memory lane, and how Django came to be relatively designer-friendly.
    • “I don't get it: what is the BDesignFL role?”
    • Process and tools: How can Django better engage designers, for core's benefit and for the benefit of your projects. How can we help these contributors get over the “suck” threshold?
    • What parts of core are ripe for love from designers? What 3rd party projects?

    At 5:10pm to 6:00pm, Wednesday 7th September

Thursday 8th September 2011

  • Lightning Talks

    At 9:00am to 10:30am, Thursday 8th September

    Coverage slide deck

  • Advanced Django Form Usage

    by Miguel Araujo and Daniel Greenfeld

    Django forms are really powerful but there are edge cases that can cause a bit of anguish. This talk will go over how to handle many common solutions not currently described in the core documentation. It will also cover some useful third-party libraries and will end with clarifications about what the state of form features will be in Django

    Outline:

    Handy tips
    - Turning non-required model fields into required form fields
    - Overriding clean methods
    - Overriding save method
    - Calling forms the easy way in views
    - Custom field definitions
    - Dynamically adding fields
    - Formsets
    - Testing forms

    Third Party Libraries that make your life easier
    - django-uni-form
    - django-floppyforms

    The future
    - Forms refactor in Django 1.4
    - Documentation improvements

    Time will be left for Q&A.

    At 11:20am to 12:00pm, Thursday 8th September

  • Deployment, Daemons and Datacenters

    by Andrew Godwin

    A tour through the systems that power ep.io, the Python hosting platform, from the array of daemons powering the system, to how redundancy is set up, and also covering general best practices for hosting Django sites yourself.

    At ep.io, we spend all day finely tweaking and polishing our deployment systems, and this talk will go through some of the parts of our archtecture, show you some of our code, and also advise you on deploying Django sites on your own servers.

    Topics covered include our use of ZeroMQ for redundant communication, the difficulties of running services with transatlantic latency, our extensive use of Eventlet for asynchronous, concurrent I/O, how we've used (and modified) pip and virtualenv, and how we've leveraged class-based views to make our API part of our normal URL structure.

    In addition, there will also be some advice on how to run sites on your own servers, including what programs to use, how to lay things out, and sensible backup strategies.

    At 2:20pm to 3:00pm, Thursday 8th September

  • Y'all Wanna Scrape with Us? Content Ain't a Thing: Web Scraping With Our Favourite Python Libraries

    by Katharine Jarmul

    Love or hate them, the top python scraping libraries have some hidden gems and tricks that you can use to enhance, update and diversify your Django models. This talk will teach you more advanced techniques to aggregate content from RSS feeds, Twitter, Tumblr and normal old web sites for your Django projects.

    OUTLINE

    • lxml fu: etree vs html
    • lxml faves: iterlinks, prev/next, strip_tags, linepos
    • incorporating xpath
    • building your xml views/templates with lxml (this bullet is optional: may not have time but would love to hear if folks might find this useful)
    • learning how to build a good JSON API handler: what you can learn from some amazing api handlers when you have to build your own
    • feedparser, HTMLParser, re: the quick & dirty ways to parse when LXML isn't fast enough

    At 3:30pm to 4:10pm, Thursday 8th September

  • Cache rules everything around me

    by Noah Silas and Jacob Burch

    This talk aims to briefly introduce the core concepts of caching and covers the best practices of using Django's cache backend.

    "Are you caching?" is a question asked early on in any yarn on web scaling advice. These conversations are much better steered by asking a more open and difficult questions "What is your caching strategy?" and “How are you implementing it?” This talk aims to briefly introduce the core concepts of caching and quickly moves to cover the best practices of using Django’s cache backend. We will let the audience know what the important questions to ask are, give them advice on how to implement the right answers, and when even the built-in core backend isn’t enough, point them to more advanced techniques and the right third party tools.

    Important questions covered:

    • Why am I caching in the first place? Do I have to?
    • What is your caching strategy?
    • What are examples of good caching strategies?
    • Where do I put my cache code?
    • How dynamic is my content? Could I use a published cache?
    • Should I cache Django objects, or raw data?

    At 4:20pm to 5:00pm, Thursday 8th September

Unscheduled