Healthy Webapps Through Continuous Introspection

A session at EuroPython 2012

Wednesday 4th July, 2012

2:30pm to 3:30pm (CET)

Every application has its hotspots – small portions of code that consume considerably more resources than all of the other code combined.

Django apps are no different. Some pages, invoked with the just the right, or wrong input, can bring a server to its knees, hogging the CPU and taking many seconds, or in extreme cases even minutes to render. By keeping workers tied up, the whole system can then become slow to respond, or collapse altogether.

Many webservers have a crude built-in failsafe to prevent this. They automatically kill workers that fail to complete their requests in time. As a result, you may not fully appreciate, or indeed realize at all that you are routinely serving 500 pages, denying users access to your service, or leaving uncommitted database transactions – possibly even slowly corrupting data. Workers killed by force leave virtually no forensic traces and so even when issues are suspected, it’s hard to pin them down.

The cause behind these hotspots can be poorly generated SQL queries from the ORM, an algorithm with non-linear complexity, excessive disk or network IO, or lock contention in the database – to name just a few.

Oftentimes these problems escape a developer’s attention, as dev and test environments simply don’t have the dataset, level of concurrency or sheer size of the real thing.

In this talk we’ll address the challenges of tuning your Django app with continuous automatic runtime inspection tools, including homegrown Dogslow. We’ll uncover the pages that consume disproportionate amounts of time and cycles to complete and the pages that get killed altogether.

We’ll discuss several ways to help you identify and eliminate the hotspots, both passively through monitoring exclusively, as well as actively by selectively interrupting workers before they get killed and examine how to effectively interpret the automatically collected forensic evidence.

About the speaker

This person is speaking at this event.
Nicolas Venegas

Next session in Spaghetti

3:30pm The Story Of Stackless Python by Christian Tismer and Armin Rigo

Coverage of this session

Sign in to add slides, notes or videos to this session

EuroPython 2012

Italy Italy, Florence

2nd8th July 2012

Tell your friends!

When

Time 2:30pm3:30pm CET

Date Wed 4th July 2012

Where

Spaghetti, Grand Hotel Mediterraneo

Short URL

lanyrd.com/swpfh

View the schedule

Share

Topics

See something wrong?

Report an issue with this session