Tuesday 25th September, 2012
4:30pm to 5:20pm
Over the past few years, Twitter’s Hadoop-based infrastructure has experienced explosive growth in a number of dimensions — terabytes stored, jobs processed, nodes active, and number of engineers producing and consuming data. Along the way we encountered a number of challenges despite the data-scalability of Hadoop-based technologies. In this talk, I will describe these problems and the solutions we developed as we expanded from 30 nodes and one team of 3 people to many hundreds of nodes, multiple teams, dozens of people, and
thousands of daily jobs.
Sign in to add slides, notes or videos to this session