As CTO of a successful high-growth startup, and in my new stealth company, I've seen my share of challenging scenarios keeping a very busy PostgreSQL-based business online and responsive during tremendous growth. EC2 + PostgreSQL + PostGIS + + Scala + Ruby + no downtime. Others can probably learn from my battle scars!
Running multiple PostgreSQL, Ruby, Solr, and Scala instances on the EC2 cloud opens up many possibilities for scaling infrastructure, but also introduces a number challenges. From unpredictable disk i/o, to RAM constraints, vanishing instances and replication challenges, this talk aims to share real-world experiences and solutions with Postgres 8.* and 9.* and all of the technologies we used to build solutions.
It covers the topic via a series of anecdotes from tiny team of developers supporting one of the biggest sites in the world (CNN.com); these stories range from humorous to horrifying, and there are valuable lessons for attendees running or contemplating a similar architecture with PostgreSQL, MongoDB, Solr, Ruby, and Scala.
by Robert Treat
Postgres has a long and stable legacy, but with that comes some steadfast design principals that don't always jive with operability in large systems. Fixing these issues in-kernel is an epic undertaking in both code complexity and social complexity, and sometimes it just proves easier to do it your damn self.
In this talk, we'll deep dive into the development and refinement of a low-level bloat removal/defragmentation system, designed entirely outside the database kernel and explore an adventure of wrong turns, bad bugs, and handicapped approaches. The adventure doesn't have a magical ending, but rather an acceptably happy one: a tool that can be used on highly concurrent, high-performance, zero-downtime environments.
28th–30th September 2011