by Cathy O'Neil
Data management teams need strong cloud computing and database management skills, and proficiency with tools like Hadoop, mapreduce jobs and SQL queries. The analysts need to be deep thinkers and creative modelers with experience in machine learning and financial modeling—ideally both. Being model- and data-driven means overnight data-crunching to produce daily data reports, which often lead to more overnight questions. There are also inherent difficulties in talking to clients and internal stakeholders when inherently unstable data and statistics are key tools for decision-making.
From a process standpoint, we need start asking new kinds of questions that Big Data is opening up for the first time. Speaker Cathy O’Neill will use her unique experience in finance, which is the field that is the most developed in terms of modeling, to explain how she sees today’s business world as relatively unsophisticated and ””spoiled for data.”” She’ll explain various techniques that financial analysts employ to improve models, and reconsider the practice of A/B testing in a model-driven world.
The abundance of data made it easy to collect and analyze data from online user behavior and discussions. Although it is encouraging to see businesses using such data and making data-driven decisions, we often see decisions based on flawed analyses, or simply using data that measure the wrong things. Panos will illustrate cases, where simple reading of available data points into one conclusion, while a deeper study uncovers a different truth. Examples will be drawn from online reputation systems, online reviews, crowdsourcing, and other case studies.
19th September 2011