Data Migration at Scale with RabbitMQ and Spring

A session at SpringOne2GX 2014

Thursday 11th September, 2014

8:30am to 10:00am (CST)

Data / Integration Track
Slides: http://www.slideshare.net/Spring...
In this talk we are going to learn about RabbitMQ and messaging architectures, both from a theoretical perspective and a practical one, analysing a product that runs integrated inside Gmail, Comcast, and other architectures. First we are going to set the foundations, explaining what messaging is all about, and how it can be implemented and integrated into our architectures using RabbitMQ as message broker. We'll revisit several messaging patterns and learn about the capabilities offered by RabbitMQ like cross cluster data replication, high availability and more. We will learn about various RabbitMQ features, from the basics of message routing and persistence, to more advanced ones like Federation, Dead Letter Queues and more. Then we'll provide an overview of the architecture of our migration product, which at this moment is integrated inside Comcast and Time Warner Cable among others, migrating millions of contacts per week. We'll explain the design decisions that took us to choose the stack, which is, Spring Integration for orchestration, RabbitMQ for internal messages, Redis for intermediate operations and MySQL as a resource tracker. The platform is completely decoupled, every operation is idempotent so we can retry them without side effects, which gives us a failover platform. We'll also explain how using Spring Integration and RabbitMQ helped us to make every operation idempotent. What are the attendees going to learn? The fundamentals of Messaging Architectures. How to build a resilient platform thanks to RabbitMQ and Spring Integration. The importance of designing an architecture with idempotent operations in mind and how easy is to create them with these technologies.

About the speakers

This person is speaking at this event.
Alvaro Videla

Co-Author of RabbitMQ in Action

This person is speaking at this event.
Félix López

Félix López is a software engineer at ShuttleCloud working on data migration and analysis products used by clients such as Google, Comcast, Stanford & Harvard.

Next session in Trinity 6-7

10:30am Spring XD for Real-time Hadoop Workload Analysis by Girish Lingappa, Rodrigo Meneses and Vineet Goel

Sign in to add slides, notes or videos to this session

Tell your friends!


Time 8:30am10:00am CST

Date Thu 11th September 2014


Trinity 6-7, Omni Hotel and Resorts, Dallas

Short URL


Official event site


View the schedule


Books by speaker

  • RabbitMQ in Action

See something wrong?

Report an issue with this session