Scaling scalability: Evolving Twitter Analytics

A session at Strange Loop 2012

Tuesday 25th September, 2012

4:30pm to 5:20pm (CST)

Over the past few years, Twitter’s Hadoop-based infrastructure has experienced explosive growth in a number of dimensions — terabytes stored, jobs processed, nodes active, and number of engineers producing and consuming data. Along the way we encountered a number of challenges despite the data-scalability of Hadoop-based technologies. In this talk, I will describe these problems and the solutions we developed as we expanded from 30 nodes and one team of 3 people to many hundreds of nodes, multiple teams, dozens of people, and
thousands of daily jobs.

About the speaker

This person is speaking at this event.
Dmitriy Ryaboy

Analytics Tech Lead at Twitter. Apache Pig committer. bio from Twitter

Sign in to add slides, notes or videos to this session

Tell your friends!


Time 4:30pm5:20pm CST

Date Tue 25th September 2012

Short URL


View the schedule



See something wrong?

Report an issue with this session