Thursday 4th October, 2012
4:15pm to 4:55pm
The database is typically the most difficult web application component to scale horizontally. This is especially true if the application was originally built with a single relational database in mind. In this talk we will present a step-by-step description of how Box’s web application stack was transitioned from one monolithic MySQL database to a fully sharded MySQL architecture. The emphasis will be on the incremental steps and best practices that enabled the successful execution of this fundamental change, all the while continuously serving 2 billion queries per day. The techniques presented will be relevant to web application architects, DevOps and DBAs trying to make the initial transition to a sharded solution.
We will begin with an overview of our web application architecture both before and after sharding. We will then walk through the modifications we made to our ORM layer to support this new architecture, including support for cross-shard queries and online moving of data between shards. We will continue with a detailed description of the technique we developed for migrating live data to shards without downtime, which also supports table by table migration for added flexibility. We will then describe how we use a similar migration technique to move and split shards for rebalancing purposes. Finally, we will discuss some of the challenges of maintaining a sharded databases architecture, and present some tools and techniques we have found to be useful. Throughout the talk we will focus on how to make large-scale changes in an incremental fashion, without adversely affecting functionality, and most importantly without downtime.
Sign in to add slides, notes or videos to this session