At ep.io we built a Python hosting platform from the ground up, designed to run large numbers of web applications on a small number of physical machines both securely and in a reasonably scalable way. This talk will show you how we built our infrastructure - using Redis, eventlet, PostgreSQL and more - and what lessons we learnt from our first few thousand deploys.
See how we split services into multiple processes and greenthreads; the pains of building a cooperatively-multitasking PTY module; how Redis isn't the answer to everything, but is still very useful; how to persuade third-party software to work securely in a shared environment; and how important it is to have good logging, especially when you have more than five servers.
We will look at the limitations of available rule engines when it comes to complex pricing rules. Rob has been part of a Python team writing a pricing engine based on network graphs. We will show:
(1) Why pricing is not easy
(2) Rule engine anti-patterns, e.g. to change one price requires a
week's development time, and has to be fitted into a release cycle
(3) How to make pricing rules data driven, usable by the business
(4) How the use of graphs and sets reduces the complexity
python-graph, NetworkX, Redis
Django makes it easy to build web sites and applications on tight deadlines - but how does it deal with sites that scale up in both traffic and complexity? Using Lanyrd.com as a case study, this talk will explore tricks and techniques for taking advantage of what Django has to offer while coping with the unexpected challenges raised by a highly dynamic social web application.
Topics covered will include performance monitoring and optimisation, advanced caching tricks, denormalisation with Solr and Redis and finding the right balance between technical elegance and pragmatism.
20th–26th June 2011