Processing Firefox Crash Reports With Python

Every day, Mozilla collects three million Firefox crash reports from around the world. The data in these reports drives the bug-fixing priorities of Firefox engineers, and is critical to understanding the stability of our platform. In this case study, I'll describe the challenges we've faced and the architecture and infrastructure we use to process, store, and analyze approximately 110TB of crash reports using Python, with Hadoop, PostgreSQL, and a few other things thrown in for good measure.