Spark on Mesos - A Deep Dive

A session at Spark Summit 2015

Monday 15th June, 2015

5:30pm to 6:00pm

While Spark and Mesos emerged together from the AMPLab at Berkeley, Mesos is now one of several clustering options for Spark, along with Hadoop YARN, which is growing in popularity, and Spark's "standalone" mode. This talk describes in detail the integration between Spark and Mesos to support clustering of Spark jobs, including the sequence of events that occurs during the life cycle of a typical Spark job. We'll discuss recommendations for optimizing performance and resource utilization, and to avoid known limitations. We'll also discuss possible future work for Spark on Mesos. Along the way, we'll understand the abstractions that Spark exposes for clustering, in general. We'll also compare and contrast Spark on Mesos vs. Spark Standalone mode and Spark on YARN. We'll offer suggestions for when to choose one option vs. the others.

About the speakers

This person is speaking at this event.
Dean Wampler

Big Data, Scala, and FP expert.

This person is speaking at this event.
Timothy Chen

Distributed Systems Engineer bio from LinkedIn

Sign in to add slides, notes or videos to this session

Tell your friends!


Time 5:30pm6:00pm PST

Date Mon 15th June 2015

Short URL


Official event site


View the schedule


Books by speaker

  • Programming Scala: Scalability = Functional Programming + Objects
  • Programming Hive
  • Functional Programming for Java Developers: Tools for Better Concurrency, Abstraction, and Agility

See something wrong?

Report an issue with this session