Thursday 1st March, 2012
11:30am to 12:10pm
Social data is growing, Twitter produces 250+ million tweets per day and 27 million links to news and media. Big Data can give insights into these large datasets but first the data must be curated, cleaned and quantified before it has value. We will cover how we move from unstructured to structured and how we take simple data and apply complex processes to give context to the data.
We will cover how we developed a platform that can deal with billions of items per day and perform complex analysis before handing the data onto thousands of customers in real-time. We will also walk through our platform architecture looking at our use of Hadoop, HBase, 0MQ, Kafka and many other cutting edge technologies. You will learn some of the pitfalls of running a production Hadoop cluster and the value when you make it work.
Sign in to add slides, notes or videos to this session