Add an eventSessions at Strange Loop 2011 about Distributed Systems
Your current filters are…
- Topic: Distributed Systems
Monday 19th September 2011
-
On Distributed Failures (and handling them with Doozer)
In unreliable networks, with unreliable processors (also known as “The Cloud”) one must know all the ways their thing can fail; and just as importantly, how to recover. It is even better to have the ability to recover in an automated way. I’ll go over the common failure conditions in distributed systems, and then introduce you to Doozer, the highly-available, consistant data store that makes recovering from these problems easy.
-
Dynamo is not just for datastores
by Susan Potter
Find out how to build decentralized distributed systems based on a Dynamo-esque philosophy using riak_core, which is used to abstract away some of the more complicated decentralized and distributed features of Riak KV, Riak Search and other applications. riak_core is an open source library written in Erlang that runs masterless distributed applications inside of it. Riak KV is a dynamo-inspired decentralized distributed key-value datastore that has received attention in the NoSQL world and is built
on top of riak_core.In this talk I will demonstrate how to implement in applications built running in riak_core:
- Implementing a coordinator which enforces the consistency semantics of N, R and W and performs read repair.
- Implementing a vnode which handles one partition on the consistent hashing ring.
- How to decompose your applications logic into “commands” which is central to implementing a vnode callback module.
- Discuss how to implement hinted handoff using the riak_core vnode behavior callbacks.
-
Skynet: A Scalable, Distributed Service Mesh in Go
The Go programming language is fast like C and flexible like Python. Skynet takes advantage of Go’s powerful features to create a highly scalable mesh of service processes. Skynet services self-announce and self-configure when they start so that they’re immediately available to the rest of the mesh. Creating a Skynet service couldn’t be simpler: just use the built-in generator, insert your business logic and compile.
This presentation will demonstrate creating a service in Skynet, then deploying it to several servers. We will demonstrate Skynet’s self-healing capabilities and built-in monitoring, and discuss the features of Go that make Skynet possible. With Skynet your company worry less about uptime and spend more time on your customers.
-
Distributed STM: A new programming model for the cloud
by Cyprien Noel
Imagine developing an application using the equivalent of source control for your data. Some STM (MVCC) enable this model for threads: a thread can take a snapshot of memory, run totally in isolation, and commit atomically.
Distributed STM are a recent development which enables this model for threads in different processes. In the same way a developer can check out a project on his local machine, an app. can replicate an object graph from a cloud, have local threads update it, and commit changes back to the cloud. The cloud can then broadcast changes to other clients’ replicas.
Developers can use this technology for scenarios like real-time collaboration, pub/sub, market quotes broadcasting, MMO game-state synchronization between players, or simply storing data in a cloud. The amount of code required on both client and server is an order of magnitude lower than models like REST, and performance can be much higher.
Many implementations are emerging, like the Fénix framework which is successfully running a large university campus portal, the JCell project, D²STM, Multiverse, or the EU-funded CloudTM initiative.
Our own implementation, xstm.org, started in the financial software industry for high performance price transfer and trading. It is open source (Apache 2), and has been integrated in several commercial projects over the past two years, each time adding new functionalities and tools.
We now have SDKs for Java, Android, GWT and .NET which can interoperate with each other for multi-platform apps, a high-performance store to make replicated objects persistent either locally (with offline sync to the server) or in a NoSQL store. Data can be exchanged over a socket, http (polling) or comet, and WebSocket.
We offer tools to deploy and debug a server in Amazon EC2, create secure connections and handle data versioning. We are also developing a cloud back-end, objectfabric.com, which developers will be able to use as a highly available auto-scaling service.
-
Distributed Systems with Gevent and ZeroMQ
by Jeff Lindsay
Concurrency is hard! At least, it seems hard. You might feel that unless you take up Erlang, you’ll never be able to express distributed systems in an easy way. I’ll let you in on a secret: the magic of Erlang, and Go, and any other language with good concurrency primitives can also be found in a library called ZeroMQ, which can be used in any language. It’s especially powerful and expressive when combined with an evented programming environment like Node.js, EventMachine, or Twisted.
I’ll let you in on another secret: there is a better way to do evented async programming than having callbacks all over the place, making your program read backwards and making your mind melt. By combining the reactor pattern of async frameworks with green threads, you get something that, so far, only exists in Python as two libraries called Gevent and Eventlet. These give you the advantages of I/O based evented programming, but without the ugly callbacks. Plus, most Python libraries are likely to work, unlike the completely incompatible and reimplemented world of Twisted or EventMachine in Ruby. Your Python code ends up feeling as powerful as Go code.
By combining these two technologies, you have a powerful toolkit for easily throwing together distributed systems in a “normal” language. Although we’ll focus on Gevent and ZeroMQ in Python, the lessons of this talk apply to any language. We’ll dive in and further explain these two technologies and then explore them together in action.
Tuesday 20th September 2011
-
Distributed Systems: The Stuff Nobody Told You
We at Greplin have had an exciting time learning how to scale our service. A year ago, we were storing a few million documents on just two servers – while we’re now storing billions of documents on hundreds of servers (and adding up to 5000 documents a second).
When we started designing our architecture, we gave a lot of thought to distributed data stores (Riak and HBase) and message buses (RabbitMQ and Redis). But we drastically underestimated the importance of the tools and support systems that actually make managing a large and growing cluster manageable, such as:
- Monitoring
- Analytics
- Scripts (deployment/debugging/etc)
- Logging
- Security
Some of these topics aren’t as sexy as the CAP theorem – but they are just as important to running distributed systems in the real world. I’d like to talk about some of the best practices and tools we’ve discovered or created (and open-sourced!) at Greplin to make this possible. I’ll also be sharing some examples of problems that our tools have allowed us to discover and fix quickly.
-
Building Polyglot Systems with Scalang
by Cliff Moon
Erlang and Scala have a very complementary set of strengths and weaknesses for building large scale distributed systems. Cliff will show how to integrate Erlang and Scala using scalang, a library that allows scala actors to transparently participate in Erlang/OTP based distributed systems.
-
DataMapper on Infinispan: Clustered NoSQL
by Lance Ball
TorqueBox is a Ruby application server built on top of JBoss AS7 that extends the footprint of Ruby applications to include built-in support for services such as messaging, scheduling, and daemons. Infinispan is an extremely scalable, highly available data grid platform, designed to make the most of modern multi-processor/multi-core architectures while at the same time providing distributed cache capabilities. DataMapper is an Object Relational Mapper for Ruby. TorqueBox brings the two together to provide a fast, scalable, distributed NoSQL data store for your Ruby applications. In this presentation, I will take a brief look at each of these technologies and combine them with Sinatra to create a super awesome beer catalogue.
Strange Loop 2011
United States United States, St. Louis
18th–20th September 2011
Calendar subscribe
Schedule incomplete?
Filter by Day
Filter by coverage
Filter by Topic
Filter by Venue
Filter by Space
- Gateway 3 2
- Gateway 4/5 4
- Lindbergh 2