•  

Beyond Map/Reduce: Getting Creative with Parallel Processing

A session at Strata 2012

Thursday 1st March, 2012

2:20pm to 3:00pm (PST)

While Map/Reduce is an excellent environment for some parallel computing tasks, there are many ways to use a cluster beyond Map/Reduce. Within the last year, the YARN and NextGen Map/Reduce has been contributed into the Hadoop trunk, Mesos has been released as an open source project, and a variety of new parallel programming environments have emerged such as Spark, Giraph, Golden Orb, Accumulo, and others.

We will discuss the features of YARN and Mesos, and talk about obvious yet relatively unexplored uses of these cluster schedulers as simple work queues. Examples will be provided in the context of machine learning. Next, we will provide an overview of the Bulk-Synchronous-Parallel model of computation, and compare and contrast the implementations that have emerged over the last year. We will also discuss two other alternative environments: Spark, an in-memory version of Map/Reduce which features a Scala-based interpreter; and Accumulo, a BigTable-style database that implements a novel model for parallel computation and was recently released by the NSA.

About the speaker

This person is speaking at this event.
Ed Kohlwey

Booz Allen Hamilton

Next session in Ballroom CD

4pm Petabyte Scale, Automated Support for Remote Devices by Kumar Palaniappan and Ron Bodkin

Coverage of this session

Sign in to add slides, notes or videos to this session

Strata 2012

United States United States, Santa Clara

28th February to 1st March 2012

Tell your friends!

When

Time 2:20pm3:00pm PST

Date Thu 1st March 2012

Where

Ballroom CD, Santa Clara Convention Center

Short URL

lanyrd.com/smmym

View the schedule

Share

Topics

See something wrong?

Report an issue with this session