Music Data and Music Search at Insane Scale at The Echo Nest

A session at Velocity Europe 2012

Wednesday 3rd October, 2012

4:10pm to 4:50pm (GMT)

In the past few years, The Echo Nest has built the largest database of music anywhere – over 2 million artists and 30 million songs each with detailed information down to the pitch of each note in each guitar solo and every adjective ever said about your favorite new band. We’ve done it with a nimble and speedy custom infrastructure—web crawling, natural language processing, audio analysis and synthesis, audio fingerprinting and deduplication, and front ends to our massive key-value stores and text indexes. Our real time music data API handles hundreds of queries a second and powers most music discovery experiences you have on the internet today, from iHeartRadio and Spotify to eMusic, VEVO, MOG and MTV.

During this talk, the Echo Nest’s co-founder and CTO will run through the challenges and solutions needed to build music recommendation, search and identification at “severe scale,” with the constraint that most of our results are computed on the fly with little caching. It’s hard to store results when data about music changes on the internet so quickly as do the tastes and preferences of your customers’ listeners.

  • What’s next
  • Intro – the world of music data
  • “The future music platform” and APIs in the music industry
  • The Echo Nest music intelligence platform * Recommendation & playlisting * Audio fingerprinting * Music resolving and search
  • Scale challenges with a music database * Acoustic analysis * Text indexing * Key-value stores
  • Scaling solr and lucene for recommendation and playlisting * Building a Pandora with 30 million songs * Text indexing challenges with Solr over 1 billion documents * Cutting text indexes to dynamic user queries
  • API front end & workers * Scaling to hundreds of queries a second without much caching
  • Fingerprinting * Shazam for the world in Solr and Tokyo Tyrant

About the speaker

This person is speaking at this event.
Brian Whitman

co-founder and CTO of The Echo Nest, the nicest music technology company ever bio from Twitter

Sign in to add slides, notes or videos to this session

Tell your friends!


Time 4:10pm4:50pm GMT

Date Wed 3rd October 2012

Short URL


View the schedule


See something wrong?

Report an issue with this session