by Kevin Weil
Most analytics systems rely on large offline computations, which means results come in hours or days behind. Twitter is all about realtime, but with over 160 million users producing over 90 million tweets per day, we need realtime analytics that scaled horizontally. This talk discusses the development of that infrastructure, as well as the products we are beginning to build on top of it.
The rise of sensor network data and the expectation for low latency query responses combine to obsolete available databases and storage platforms. We have built a platform for web-scale OLAP and in this talk I will cover how we made our infrastructure capable of real-time update and query performance over hundreds of terabytes of multidimensional data.
1st–3rd February 2011