by Jeff Lindsay
Concurrency is hard! At least, it seems hard. You might feel that unless you take up Erlang, you’ll never be able to express distributed systems in an easy way. I’ll let you in on a secret: the magic of Erlang, and Go, and any other language with good concurrency primitives can also be found in a library called ZeroMQ, which can be used in any language. It’s especially powerful and expressive when combined with an evented programming environment like Node.js, EventMachine, or Twisted.
I’ll let you in on another secret: there is a better way to do evented async programming than having callbacks all over the place, making your program read backwards and making your mind melt. By combining the reactor pattern of async frameworks with green threads, you get something that, so far, only exists in Python as two libraries called Gevent and Eventlet. These give you the advantages of I/O based evented programming, but without the ugly callbacks. Plus, most Python libraries are likely to work, unlike the completely incompatible and reimplemented world of Twisted or EventMachine in Ruby. Your Python code ends up feeling as powerful as Go code.
By combining these two technologies, you have a powerful toolkit for easily throwing together distributed systems in a “normal” language. Although we’ll focus on Gevent and ZeroMQ in Python, the lessons of this talk apply to any language. We’ll dive in and further explain these two technologies and then explore them together in action.
18th–20th September 2011