Prismatic delivers realtime newsfeeds for thousands of learned topics based on your interests and social networks. The Prismatic backend requires many custom abstractions, sophisticated machine learning techniques, data crawling, and graph analysis algorithms. We discuss how our small backend team (just four engineers) built this stack in record time using a unique design philosophy facilitated by Clojure. In a nutshell, our problem-solving approach prefers lightweight composable custom abstractions for problems rather than monolithic open-source frameworks (e.g., Hadoop).
We talk about how Clojure lets us achieve maximal productivity-per-person and extreme reuse, all while maintaining high performance. We highlight key elements of our stack:
At circleci.com, we're running clojure and jruby on rails in the same process. We wrote ____, (soon to be released) a library for calling jruby from clojure, and vice versa. We'll cover
Calling JRuby from Clojure
Calling Clojure from JRuby
Using the Clojure REPL to speed up Rails development
Starting multiple ruby runtimes inside the same JVM to speed up testing
Time permitting, we can also talk about testing and continuous deployment at CircleCI, a Continuous Integration SaaS written in Clojure.
by Nathan Marz
Storm makes it easy to write and scale complex realtime computations on a cluster of computers, doing for realtime processing what Hadoop did for batch processing. Storm guarantees that every message will be processed. And it’s fast — you can process millions of messages per second with a small cluster. Storm was open-sourced by Twitter in September of 2011 and has since been adopted by many companies around the world.
Storm is implemented in Clojure and ships with an excellent Clojure DSL. In this talk I'll start by discussing the basics of Storm: streams, spouts, bolts, and topologies. Then I'll show how to use Storm's Clojure DSL for a wide variety of applications, such as stream processing, distributed RPC, and continuous computations.
By now most of the community has heard about Cascalog, a querying library for Apache Hadop that leverages the full power of Clojure, instead of providing a cumbersome External DSL, like Hive and Pig do.
In this talk I want to focus first on where Cascalog comes from, so I'm going to throw in just a little of Cascading to explain some crucial concepts from which Cascalog builds on. After that I want to move on to what's awesome about Cascalog now and my real world experiences using Cascalog for doing actual production work. In this part I will explain some of the most powerful abstractions, examples of this abstractions are prepared and parametric operations and predicate macros. Finally I want to mention what are the plans for Cascalog in the future, which issues are coming in the pipeline and why you (the audience should be a part of the project).
by Colin Jones
Code quality matters: programmer happiness and efficiency depend on it, especially as codebases grow and change. The notion of simplicity is a powerful one that offers some guidance, but how, specifically, can we make our code simple?
For object-oriented languages, the SOLID principles are a group of helpful guidelines for improving code, and I claim that these principles also apply to writing simple Clojure. There are certainly others that are at least equally applicable (e.g. Don't Repeat Yourself), and yet others that are more specific to Clojure (e.g. isolating mutation), but given these principles' image as OO-specific rules, I will concentrate on their perhaps-surprising application to Clojure.
I offer some comparisons and contrasts with these ideas' applications in object-oriented programming languages, and hope that by the end of the session, it will seem obvious that these are rules for writing *software*, and not just OO. There will be plenty of code examples to make things concrete.
Just as we learn from the computing past in terms of the JVM platform, database transactions, Lisp's macro system, and the mountain of other great ideas that Clojure the language is built on, we can also learn about building systems that are easy to change from the experience of object-oriented programmers.
The Seattle Clojure group (http://seajure.github.com) has developed a unique style of collaborative interactive development I've termed "swarm coding". It involves a medium-to-small group with laptops all connected to a single host trading off on writing a small, focused piece of software.
This 80-minute unsession will begin with an explanation of swarm coding. We will discuss how to facilitate it at your own user groups, how to pick a project, and common pitfalls. Following this, we will break up into a number of smaller groups and each implement a small project.
In the beginning, trying to do number crunching using Clojure was a pain: to get any sort of performance, you had to resort to low-level, non-idiomatic constructs. Clojure 1.3 introduced enhanced primitive support that promises to make heavily numeric work in Clojure practical. This session will show you how to use these new features, help you become aware of potential pitfalls, and review some of the other tools within Clojure to help you create high-performance number-crunching applications.
by Jim Duey
Revolutions in software development are enabled by new abstractions being used to produce more software while writing less code. Domain Specific Languages are an abstraction that can help fuel the next revolution. This talk will introduce you to the concepts of DSL's, how to think about them, design them and use them to hide complexity at various levels in your software design using some common Clojure DSL's as examples. It will also touch on concepts like macros, monads and arrows to show how these can be used and why you would want to.
by Amit Rathore
Runa has been in existence since the beginning of 2008, and has had a Clojure system in production since the fall of the same year. This talk is about how a small team of Lisp engineers is disrupting the e-commerce world.
We’ll cover how Runa leverages Clojure DSLs to run the business: i.e. provide real-time dynamic pricing for online retailers to increase conversion.
We’ll also provide a look behind the magic curtain of how Runa’s DSLs facilitate communication between the analytics team, the business team, and the engineering team. Finally, we’ll give up the secret of our profit margin.
This is a relatively high-level talk, and will broadly address the following:
Clojure DSLs as requirements specifications
Training non-technical people in Clojure DSLs
How the Clojure back-end integrates with our internal Ruby/Rails applications
The Runa Dynamic Pricing/Promotions DSL
Dropping s-expressions into live systems
We’ll also briefly cover what our analytics process looks like, and what sort of algorithms we use to drive the smarts of our system.
One tough problem in debugging distributed applications is understanding what happened on a client that later caused an exception on a server. I'll discuss the nature of Clojure stack traces and demonstrate a library that can extend a stack trace from where it was thrown, back across multiple thread or server boundaries to the client that initiated it. Even with more helpful stack traces, however, testing for the handling of race conditions is still very challenging. I'll demonstrate a technique that we've used at LonoCloud in our test suite to consistently reproduce and guard against race conditions.
by Paul Stadig
One of the more interesting features of Clojure is its lazy seqs. You see uses of lazy seqs throughout the core API. Lazy seqs can be used to process data efficiently, but there are some gotchas. Learn how they work, how they work well, and what mistakes to avoid.
by Ryan Senior
The core.logic library brings logic programming to Clojure and can greatly simplify some kinds of problems. This session will cover a graph querying problem I encountered at Revelytix and how I solved it with and without core.logic. Using graph querying as a goal, I will give an intro to core.logic using live coding. We'll build up abstractions from the basic core.logic constructs and ultimately be able to run sophisticated queries over Clojure data. The focus of the intro will be practical, focusing on concepts and constructs that you need to be productive with core.logic.
16th–17th March 2012