Real Time Recommendations using Spark Streaming

A session at QCon San Francisco 2016

  • Elliot Chow

Recommendations play a vital role in a great Netflix experience. Traditionally, these recommendations are precomputed using viewing history, scroll activity, and a variety of other signals in a near-line fashion. To be able to react more quickly to surges and dips in interest, we introduced the Trending Now row that makes use of real time data as an additional signal for generating recommendations. This allows us to not only personalize this row based on the context like time of day and day of week, but also react to sudden changes in collective interests of members, due to real-world events.

We will discuss the data pipeline that we built to process Netflix user activities in real time for the Trending Now row. We will share our experiences with developing, monitoring, and productionalizing a system that uses Kafka, Spark Streaming, and Cassandra.

About the speaker

This person is speaking at this event.
Elliot Chow

Senior Software Engineer @Netflix

Coverage of this session

Sign in to add slides, notes or videos to this session

Tell your friends!


Date Wed 9th November 2016

Session Hash Tag


Short URL


Official session page


View the schedule


See something wrong?

Report an issue with this session