Sessions at EuroPython 2011 about Celery and Django

Tuesday 21st June 2011

• Advanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric

  by Simon Willison

  Django benefits from a thriving third-party ecosystem. This talk will take a deep look at three of the components I have found most useful in my own projects: Haystack for full-text search, Celery for offline processing and Fabric for automated deployments. I'll discuss how we use them for Lanyrd.com, and demonstrate why they deserve a permanent place in your toolbox.

  • Celery
  • Django
  • Django Haystack
  • Fabric
  • Lanyrd

  At 2:30pm to 3:30pm, Tuesday 21st June

  Coverage slide deck video

Wednesday 22nd June 2011

• Playing tasks with Django-Celery

  by Mauro Rocco

  Celery is an open source task queueing system based on distributed message passing.

  I will talk about the tools that Celery offers for task distribution and how to monitor and manage the system using a Django web interface.
  This talk will also focus on how we use Celery at Jamendo and our real solutions to some common issues you may encounter when developing a back-office based on Celery.

  The talk will cover the following topics:

  -- A brief overview of Celery and the AMPQ protocol.
  AMPQ protocol overview
  Celery introduction
  Celery, RabbitMQ code examples

  -- The impact of Celery on the Jamendo work-flow; examples with real tasks.
  Here I will talk about the Jamendo back-office infrastructure and some of our common tasks. I will discuss the improvements made by introducing a new back-office system based on Celery.
  I will show some code snippets and go over some real scenarios.

  -- Overview of the Django Celery admin interface and some Jamendo extensions
  Let's talk about the Django-Celery interface that allows one to monitor or schedule tasks directly from the Django admin.
  I will explain which common additional features are necessary and how to add them.

  -- Common "gotchas" we encountered while working with Celery and how we solved them.
  - Global task locks
  - Centralized logging: be able to read all the logs of all celery workers on different servers and filter them for real-time debugging

  • AMPQ
  • Celery
  • Django
  • Python

  At 3:30pm to 4:30pm, Wednesday 22nd June

  Coverage video