Sandboxing Ruby Code - Lessons from the battlefield

Sometime you want to run untrusted code on your server. Our expirements with sandboxing started with the creation of a hosted continuous integration service called Goldberg Pro, and then continued on with RubyMonk, which needed to prevent users from performing potentially dangerous operations in code they submitted via the website.
Here we discuss various system level, language level, and application level techniques that we tried (and we plan to try in the future) to ensure that the user does not bring down the entire system. Some techniques we plan to speak about include LXC (Linux Containers), Chroots, Ruby's SAFE levels, Kernel level limits, SELinux and PTrace.
Some of our learnings are published in the form of the Open Source ruby gem - secure.