Add an eventSessions at DjangoCon US 2011 about Django with slides
Monday 5th September 2011
-
Test Anything, Test Everything
by Dave Dash
Testing in Django is easy if you're testing models against your database. What happens when you need to test other-systems, like a search engine, or an API? This tutorial will cover how I built SphinxTestCase, ESTestCase and redisutils to allow us to maintain test coverage on our web sites.
Testing backends other than your database can be tricky. It's entirely necessary, however, otherwise you'll start getting glaring holes in terms of coverage.
This tutorial will show you how to subclass TestCase and properly setup and teardown external systems.
It will also show you how to be nice to your other developers who may want to opt-out of testing backend systems (like search).
Tuesday 6th September 2011
-
From Designer to Django'er in Six Weeks: A Story from Solo Founder
by Tracy Osborn
Django gives you the tools to start a successful web app even if you're new to development. Learn how to build and launch a web startup whilst avoiding common pitalls and problems.
You don't have to be an expert programmer to launch a web startup; designers, marketers, and novice developers can all use Django to quickly launch a web app, build a company, and start bringing in revenue. It's a common trap to get caught up in endless development — follow these tips and tricks to get launched even when you're still learning.
-
Monkeying Around with Python at New Relic
This session will cover the challenges of creating a production application performance monitoring system for Python. It includes an overview of the architecture of the system, as well as how it hooks into Django and captures details about web transactions, database transactions, memcache requests, exceptions and much more.
This session provides a technical review of the challenges associated with instrumenting Python for production monitoring. How do you place monitoring hooks into code that cannot be modified directly in order to collect meaningful performance data? Concepts covered in this session will include:
- Using context manager objects to time execution of code.
- Using context manager objects to catch details of exceptions.
- Applying a context manager to functions using wrappers and decorators.
- Applying a context manager to WSGI applications and components.
- How to go about applying the above to existing code using monkey patching.
- Deriving the name of a wrapped object for identification purposes.
- Using post import hooks as a means to trigger monkey patching.
- Driving the process of monkey patching from a configuration file.
The talk includes real-world examples of how these mechanisms have been used to add instrumentation to the Django web framework in order to collect actionable performance data.
-
Real-Time Django
by Ben Slavin and Adam Miskiewicz
The web is live. APIs give us access to continuously changing data. We discuss ways to get real-time data into your app, how to handle data processing and what to do when you get thousands of updates per second.
"Caching is the answer" can't always be the answer. We discuss how to ensure your site is up-to-date when the data behind it is continuously changing.
We'll discuss three primary areas pertaining to real-time-data.
WHERE IT COMES FROM – REAL-TIME DATA SOURCES
- Streaming APIs
- Polled APIs (and strategies for dealing with rate limits)
- PubSubHubbub
DATA PROCESSING
- Denormalization
- Workers
- Incremental and generational MapReduce
DATA PRESENTATION
- Partial caching
- Continuous caching
- Broadcasting real-time updates
- Failure models
-
Testing: The Developer Strikes Back
by Sandy Strong
“Code not tested is broken by design” - Jacob Kaplan-Moss Every programming project has unique testing needs. It’s not always easy to understand what those needs are, or how to write tests that satisfy them. One of our goals as developers is to write tests that minimize failures in production that can cost our companies time, money, resources, and in many cases, hours of sleep!
Every programming project has unique testing needs. It’s not always easy to understand what those needs are, or how to write tests that satisfy them. One of our goals as developers is to write tests that minimize failures in production that can cost our companies time, money, resources, and in many cases, hours of sleep!
-
State of the DSF Keynote
-
A summer in the wild
by Alex Gaynor
I've spent this past summer away from Django, working for Quora, venturing into the depths of a foreign codebase, and making it faster by running on PyPy. This talk will feature the lessons I've learned about Django, PyPy, and web development in general.
This talk is going to cover some pretty eclectic ground. Some of the stuff I'm going to talk about includes:
- Making stuff run on PyPy
- Profiling and benchmarking a webapp
- Stuff I missed when I wasn't using Django
- Tools every Pythonista must be using
- Good ideas from Quora that Django should steal
- Learning a giant codebase on the quick
- Process is no substitute for good process
-
Secrets of PostgreSQL Performance
by Frank Wiles
PostgreSQL is effectively the default RDBMS for Django. Learn the dark arts of optimizing this powerful database to be blazingly fast on your own hardware or in the cloud.
PostgreSQL has been the default recommended database since Django was initially released. Learn the dark arts of optimizing this powerful database. Both in general use cases and specifically for Django.
In this talk you'll learn:
->Things to avoid
-double duty servers
-disk contention
-unnecessary queries->The "Big 3" configuration options you should always adjust
- shared_buffers
- effective_cache_size
- random_page_cost->Other useful configuration options
->Hardware considerations
- RAM, disks, and CPU
- Rackspace Cloud vs Amazon EC2
- tablespaces
- SSDs-> Common Django ORM issues
- Slow count() operations-> Indexing Tips and Tricks
- Index types
- Multi-column indexes
- Partial indexes-> Performance Tools you can use
-> The "Dark Arts"
- Planner Statistics
- materialized views
- crazy replication
Wednesday 7th September 2011
-
RESTful APIs: Promises & Lies
Over the last few years RESTful APIs have become an integral part of many django projects. But some of the fundamental questions still remain unanswered. How do you decouple resources from models, formatting from definitions, authorization from authentication? How should you define resources? How do you handle pagination? Deliver facets? Prevent abuse? Implement versioning? Let's have a look.
Implementing RESTful APIs for django applications used to be the talk of the town. Eventually a few API frameworks emerged. Some were designed to provide a quick & easy path to implementation. Some focused on decoupling components as much as possible. At the same time, numerous blog posts were published and emails exchanged seeking/ pitching sound API concepts & best practices. Somehow till this date, there is arguably no single solution that addresses all of the major concepts/practices around designing & securing RESTful APIs. Why? What were the promises? What were the lies?
In this talk, an attempt is made in identifying these concepts/practices in light of recent experiences with a PBS Education Technology project. In designing a RESTful API, it is important to establish a relatively simple resource definition that is uniform across all resources, yet powerful enough to deliver errors, pagination, facets as well as data from attributes/ methods of various models/ instances. It is also desirable to have serialization formats decoupled from these definitions and delivered to consumers according to their preferences.
Securing the API is a complex task as well. Not all RESTful APIs are meant for public consumption. Yet it is necessary to make AJAX calls that might leave the API vulnerable and expose security patterns. This talk highlights why it is important to have decoupled authentication backends, why 2-legged-OAuth can be an excellent choice in protecting the API and how adding pre-approved tokens can help differentiate between user/ application specific calls.
Concepts/ best practices evolve over time. So should the API. Sometimes it is necessary to perform refactoring or optimization that alters the API significantly. In any case, establishing a path to versioning API components is a prudent choice. Here, the fundamentals are explored.
Rest of the talk presents how PBS Education Technology team enhanced django-piston to incorporate the ideas and suggestions mentioned above. Also consumer side practices and security implementations are demonstrated using code examples.
-
Real World Django Deployment Using Chef
Chef, a popular configuration management tool, is gaining momentum in the Django community. Learn how to deploy Django sites using Chef, be they one server or a thousand.
Chef, a configuration management tool, is increasingly popular in the Django community. Many people have yet to take the plunge, and are still managing production systems exclusively through tools like Fabric or Buildout. In this talk I will quickly show the basics of building Chef recipes, both in general and a tour of the Django-and-Python-specific tools available. I will then walk through a suite of cookbooks built to deploy Packaginator as a production site in the cloud. If suitable wireless is available, I would like to do a live demonstration at the end, bringing up a new site from scratch.
Outline:
Introduction
- Chef basics
- Resources
- Actions
- Recipes
- Cookbooks
- Roles
- SearchPython cookbook
- Community site/cookbooks
- pip provider
- virtualenv providerDjango application recipe
Other misc cookbooks
- Postgres
- RedisPackaginator cookbooks
- Application recipe
- Postgres master/slave
- Gunicorn
- Nginx
- HAProxy
- Redis
- CeleryDemo (pending wireless)
Q&A
-
Stop Tilting at Windmills - Spotting Bottlenecks
by Yann Malet and Brandon Konkle
Learn the questions to ask and the tools to use to spot performance problems in your application. We won't tell you how to scale your site, but we will show you how to locate the bottlenecks as they pop up.
Performance optimisation has always been a hot topic among the Django community, and as applications become more mature and larger companies adopt Django for high-traffic projects this issue will heat up even more. This talk will not focus on resolving performance/scalability issues but rather on discovering them. It will try to answer the following questions:
- How do you know where the worst problems are?
- When did the problem start, and what parts of your application are affected?
- How do you tell the difference between the giant performance monsters and the windmills that aren't really big problems at all?
This talk will not spell out a quick and easy 5-step method to alleviating all of your performance woes. It will, however, show you ways to measure your site’s performance over time and spot the real trouble areas as they occur. It’s vital to know both your application and your user well in order to create an effective test plan. We will discuss the importance of reliable performance measurement and realistic performance tests, and walk you through the tools we use to identify and track problems.
- Django Debug Toolbar - get internal information about what your application is doing
- Django Debug Logging - log Debug Toolbar statistics to the database and review details and aggregated summaries.
- Creating a cron job to run automated performance tests at regular intervals
- Jmeter for external tool to create load tests with high concurrency
- Integrating Jmeter with Jenkins using the performance plugin
To close up, we’ll mention some of our favorite resources on handling the performance problems that you encounter during your testing. We’ll also provide links to other tools that you can use as part of your testing efforts. With this information at your disposal you can charge boldly through the meadow on your faithful Rocinante, attacking the real performance problems and leaving the windmills behind.
-
Best Practices for Front-End Django Developers
Considerations, best practices, tools, and more for front-end development in Django.
This talk will go over considerations and best practices for front-end development in Django and beyond. Ways to:
- Best build and organize your templates
- Tools to make your life easier and faster
- Optimize and test code
- Integrate and play well with the back-end
- Deliver the most shiny of web applications
-
Safely deploying on the cutting edge
This talk will go through the deployment system that we have built at Urban Airship. It will cover the process that allows us to automate our deployments, making it safe to deploy at any time. It's built on Fabric, Gunicorn, Virtualenv, Rsync, Supervisord, and Python. We end up with fully isolated environments for our services that are easy to manage.
This talk will cover how we do deployments. It is the first deployment system that I've been happy with, which says a lot. It is a system that we have been evolving over the last 6 months or so, and has made our developers lives a lot easier. There are a few different points that make it successful, as well as useful.
Interestingly, there is very little Django specific about this setup. It is how we deploy Django applications, but we also deploy flask and standard WSGI applications using the same framework. A slightly modified version of the code is also used to deploy our Java applications, so the tooling is genericly useful for more than just Django programmers
The talk will have 4 parts, that will cover the following:
- The process around Version Control that we use
- The tooling to deploy code
- The environment that the code gets deployed in to
- The process of deploying the code across all of our servers, a subset at a timeThe first part is that we manage our source code in a specific fashion. It's a pretty standard git branching model, where we have a master branch that is the latest stable code. All on-going work is done on feature branches. Then we have a production branch that is what is currently in production. Each push to production is tagged with the date. Git allows us to keep a record of the history of development in one place.
Next, to actually handle the deployment of the code to the servers, we're using Fabric. To show the full power of the system, we might normally run a command like fab host:web-0 deploy:airship restart:airship monitor:airship. This is mostly abstracted out from people in the day to day use of the system. The deployment system has the following features:
- Updating source code checkouts
- Making VCS tags
- Deploying code
- Migrating the database
- Updating media files
- Restarting services
- Rolling back to the previous version
- Allowing the person deploying the ability to easier monitor logs/status of services
- One command "smart deploys"
The python environment that things are deployed into is a fully standardized setup using supervisord and gunicorn, along with a number of other helper utilities. It integrates into the init.d system on the host, so that normal management commands work. Each service that we deploy has it's own environment, which I'll go into more depth on.
Finally, We are also working on a system that will allow us to automatically health check deployments and automatically rollback the deployment if we are seeing elevated error rates. We handle deploying the code to a "canary" system first, then roll it out to half our servers, then the other half. This system is implemented currently at the prototype level, but should be in production use before the talk is given.
-
Keynote: Designers Make It Go to Eleven!
by Idan Gazit
The open-source world is and always has been focused on code; attention and respect doled out in exchange for patches. As a result, many open-source projects want for design love but don't know how to secure it. We scare off potential contributors from the many fields beneath the aegis of “design”: user experience, user interfaces, usability, interaction design, information architects, graphic design, typography, and other roles assumed by the authors of our markup and styling.
Django can be a model citizen among open-source projects in changing these attitudes and coaxing contributors out of the woodwork—not just for the benefit of the framework, but for the constellation of 3rd party apps which make Django great. Other FOSS communities have made efforts that we can learn from, and the fight to change perceptions is less uphill than you might think.
This talk will cover:
• Misconceptions of “Design.” What is design, and what do designers do?
• A brief trip down memory lane, and how Django came to be relatively designer-friendly.
• “I don't get it: what is the BDesignFL role?”
• Process and tools: How can Django better engage designers, for core's benefit and for the benefit of your projects. How can we help these contributors get over the “suck” threshold?
• What parts of core are ripe for love from designers? What 3rd party projects?
Thursday 8th September 2011
-
Lightning Talks
-
Advanced Django Form Usage
by Daniel Greenfeld and Miguel Araujo
Django forms are really powerful but there are edge cases that can cause a bit of anguish. This talk will go over how to handle many common solutions not currently described in the core documentation. It will also cover some useful third-party libraries and will end with clarifications about what the state of form features will be in Django
Outline:
Handy tips
- Turning non-required model fields into required form fields
- Overriding clean methods
- Overriding save method
- Calling forms the easy way in views
- Custom field definitions
- Dynamically adding fields
- Formsets
- Testing formsThird Party Libraries that make your life easier
- django-uni-form
- django-floppyformsThe future
- Forms refactor in Django 1.4
- Documentation improvementsTime will be left for Q&A.
-
Deployment, Daemons and Datacenters
A tour through the systems that power ep.io, the Python hosting platform, from the array of daemons powering the system, to how redundancy is set up, and also covering general best practices for hosting Django sites yourself.
At ep.io, we spend all day finely tweaking and polishing our deployment systems, and this talk will go through some of the parts of our archtecture, show you some of our code, and also advise you on deploying Django sites on your own servers.
Topics covered include our use of ZeroMQ for redundant communication, the difficulties of running services with transatlantic latency, our extensive use of Eventlet for asynchronous, concurrent I/O, how we've used (and modified) pip and virtualenv, and how we've leveraged class-based views to make our API part of our normal URL structure.
In addition, there will also be some advice on how to run sites on your own servers, including what programs to use, how to lay things out, and sensible backup strategies.
-
Cache rules everything around me
by Jacob Burch and Noah Silas
This talk aims to briefly introduce the core concepts of caching and covers the best practices of using Django's cache backend.
"Are you caching?" is a question asked early on in any yarn on web scaling advice. These conversations are much better steered by asking a more open and difficult questions "What is your caching strategy?" and “How are you implementing it?” This talk aims to briefly introduce the core concepts of caching and quickly moves to cover the best practices of using Django’s cache backend. We will let the audience know what the important questions to ask are, give them advice on how to implement the right answers, and when even the built-in core backend isn’t enough, point them to more advanced techniques and the right third party tools.
Important questions covered:
- Why am I caching in the first place? Do I have to?
- What is your caching strategy?
- What are examples of good caching strategies?
- Where do I put my cache code?
- How dynamic is my content? Could I use a published cache?
- Should I cache Django objects, or raw data?