•  

Spark on Mesos - A Deep Dive

A session at Spark Summit 2015

Monday 15th June, 2015

5:30pm to 6:00pm

While Spark and Mesos emerged together from the AMPLab at Berkeley, Mesos is now one of several clustering options for Spark, along with Hadoop YARN, which is growing in popularity, and Spark's "standalone" mode. This talk describes in detail the integration between Spark and Mesos to support clustering of Spark jobs, including the sequence of events that occurs during the life cycle of a typical Spark job. We'll discuss recommendations for optimizing performance and resource utilization, and to avoid known limitations. We'll also discuss possible future work for Spark on Mesos. Along the way, we'll understand the abstractions that Spark exposes for clustering, in general. We'll also compare and contrast Spark on Mesos vs. Spark Standalone mode and Spark on YARN. We'll offer suggestions for when to choose one option vs. the others.

About the speakers

This person is speaking at this event.
Dean Wampler

Big Data, Scala, and FP expert.

This person is speaking at this event.
Timothy Chen

Distributed Systems Engineer bio from LinkedIn

Sign in to add slides, notes or videos to this session

Tell your friends!

When

Time 5:30pm6:00pm PST

Date Mon 15th June 2015

Short URL

lanyrd.com/sdpdcr

Official event site

spark-summit.org/2015

View the schedule

Share

Books by speaker

  • Programming Scala: Scalability = Functional Programming + Objects
  • Programming Hive
  • Functional Programming for Java Developers: Tools for Better Concurrency, Abstraction, and Agility

See something wrong?

Report an issue with this session