Sessions at Postgres Open 2011 matching your filters

Wal-E is an open source tool designed to continuously archive your business's Postgres data to Amazon's S3 cloud storage service. It can help with recovery from failures even when the individual database servers are not individually very reliable, which is the case in many common cloud database deployments.

Individual systems in any environment can't be relied on to save critical data perfectly. No matter how much reundancy your database server includes, you still need to save backups elsewhere. The fact that no single machine can ever be reliable enough is particularly obvious when deploying servers onto a cloud deployment such as Amazon's EC2 service. Individual nodes are never expected to be as reliable as a traditional dedicated database server.

The architecture at cloud service provider Heroku solves this problem by reversing the normal approach to mission critical database deployment: they make the backups the most reliable part of the system. PostgreSQL's write-ahead log (WAL) mechanism can be used to make streaming, real-time backups of your database. The WAL-E tool developed there archives this data into Amazon's S3 storage, which so far boasts a perfect track record for storing all data ever committed to it. This approach is currently in use protecting over 100,000 Postgres databases, and has allowed Heroku to survive major interruptions in Amazon's cloud infrastructure with minimal data loss. In this session, learn how and why WAL-E was implemented, and how you can use it to keep your own data safe and secure.