The Python web framework Django can handle extremely high levels of traffic… with appropriate design and support. We’ll go over techniques to keep your Django site from being crushed under high load.
We’ll also discuss caching.
Based on our (Philipp Wassibauer and Jannis Leidel) experiences building Gidsy.com this talk will give you valuable insight as to how your infrastructure will evolve and how to set up the basic components (Loadbalancer, Webservers, DB, Caching, Celery, CDN, …) of your site.
Every application has its hotspots – small portions of code that consume considerably more resources than all of the other code combined.
Django apps are no different. Some pages, invoked with the just the right, or wrong input, can bring a server to its knees, hogging the CPU and taking many seconds, or in extreme cases even minutes to render. By keeping workers tied up, the whole system can then become slow to respond, or collapse altogether.
Many webservers have a crude built-in failsafe to prevent this. They automatically kill workers that fail to complete their requests in time. As a result, you may not fully appreciate, or indeed realize at all that you are routinely serving 500 pages, denying users access to your service, or leaving uncommitted database transactions – possibly even slowly corrupting data. Workers killed by force leave virtually no forensic traces and so even when issues are suspected, it’s hard to pin them down.
The cause behind these hotspots can be poorly generated SQL queries from the ORM, an algorithm with non-linear complexity, excessive disk or network IO, or lock contention in the database – to name just a few.
Oftentimes these problems escape a developer’s attention, as dev and test environments simply don’t have the dataset, level of concurrency or sheer size of the real thing.
In this talk we’ll address the challenges of tuning your Django app with continuous automatic runtime inspection tools, including homegrown Dogslow. We’ll uncover the pages that consume disproportionate amounts of time and cycles to complete and the pages that get killed altogether.
We’ll discuss several ways to help you identify and eliminate the hotspots, both passively through monitoring exclusively, as well as actively by selectively interrupting workers before they get killed and examine how to effectively interpret the automatically collected forensic evidence.
The concept: run through the official Django tutorial, but with full TDD
Includes browser-based testing with Selenium, using the new WebDriver API
In-depth unit-testing, including the Django Test Client and tools like Mock
Discussions of TDD philosophy: what to test, what not to test, how to test
come prepared! you’ll need git, firefox, python2.7, django1.4 and selenium installed.
This tutorial is aimed primarily at people who want to learn about one or many of:
So it’s aimed at beginners of one kind or another – if you are already well versed in all three, then you may not get much out of it.
The idea is to run through the steps covered by the official Django tutorial, covering the essentials of Django from a beginner’s point of view. However, at each stage, instead of “just” writing the Django code required to build our site, we first write tests.
And we don’t just write unit tests! We start by writing “functional” or “acceptance” tests, using Selenium - which means driving a real browser, and checking the actual behaviour of the site as it is rendered, from the user’s point of view.
Why should you listen to me, I hear you ask?
Well, I was lucky enough to get my first “proper” software development job about two years ago with a bunch of Extreme Programming fanatics, who’ve thoroughly inculcated me into their cult of Test-Driven development. Believe me when I say I’m contrary enough to have questioned every single practice, challenged every single decision, moaned about every extra minute spent doing “pointless” tests instead of writing “proper” code. But I’ve come round to the idea now, and whenever I’ve had to go back to some of my old projects which don’t have tests, boy have I ever realised the wisdom of the approach. So, I’ve learnt from some really good people, and the learning process is still fresh in my mind, and most importantly, I still have the passion of a recent convert!
Django o Pyramid, rapidi da apprendere e mettere a frutto, sono potenti strumenti di lavoro, utili a costruire siti gestiti da un solo autore o applicazioni web di servizio. Ma se avete bisogno di far collaborare un vera redazione alla pubblicazione di un portale ricco di contenuti, non troverete stoffa pythonica migliore di Plone!
Come costruire un servizio di Content e Document Management potente e professionale nel 2012?
Facciamo il punto su cosa offre il mondo Python di oggi.
Currently in development is the DISQUS realtime service. This talk will detail what it took to add real time to a truly “web scale” django app.
This will include:
I hate long talks and love open spaces. Also, I’m not the world’s leading expert in real time anything. So I’m planning on making this a 20-30 minute talk, and then converting it to an open space for 20-30 minutes which I will lead. The goal is to get a good conversation going around the topics I will have presented. So I’d prefer a 45-60 minute slot.
Suggested audience experience:
by Simone Federici and Michal Petrucha
Currently Django models only support single column primary keys.
Multi-column primary keys support would improve Django integration with legacy databases (whose schema cannot be changed and usualy have multi-column primary keys).
This project allows the usage of django ORM with Multi-Column Primary Key and includes Foreign Key support.
In this talk we will explain the problem, propose a solution and present two projects aiming to solve it.
One of them is a pluggable app which monkey-patches Django’s ORM. Its obvious benefit is you can introduce it in a project without maintaining an unofficial Django fork.
The other one is the long-term solution approved of by the Django core team. It was part of last year’s Google Summer of Code and we will talk about what is ready and what is yet to be done before it is included into mainline Django.
The talk will discuss ways to achieve reduced loading times of django web applications. The focus will be on optimizing and re-architecting the web application itself, instead of throwing more hardware at it.
Some of the topics (in a high level) of the talk will be optimizing SQL queries, caching and executing operations in the background as well as taking advantage of features that the stack provides you.
Basic knowledge of django is needed.
by Ed Crewe
This talk is about the development of the user system for a online survey application.
The goal of the talk would be to impart some knowledge of the current state of open authorisation standards and how the python web application tools that are available for them may be applied in practise. The prerequisites are some background in web development and perhaps authorisation systems - experience of Django is not necessary but may be useful.
The introduction will give background regarding the application, for context, e.g. 3 million survey responses in a Perl web application being rewritten in Django with Cassandra and PostgreSQL data storage. The need to add external access control via Shibboleth (SAML2) and OpenID.
This will be followed by a summary of the features and differences between the three main open standards for third party access control, SAML2, OAuth and OpenId.
Then I will move on to the issues involved:
Next will be an explanation of what django.contrib.auth has and its likely future (a rewrite is currently under discussion). Then a review of the various authentication and authorisation add on eggs available for Django that could help deliver elements of these requirements.
This section will end with what we chose to use and the issues that this involved.
Finally, some Python code! So a look at some of the more generically useful implementation code, e.g. development of standard object permission decorators for Django class views.
Concluding with where we are now and lessons learned.
2nd–8th July 2012