Your current filters are…
by Alex Robbins
Cascalog is a clojure library that makes it easier to write distributed batch processing jobs. It is built on top of Hadoop, but allows computations and queries to be written at a much higher level of abstraction. Cascalog allows large, distributed processing jobs to be written in a declarative manner, greatly shortening and simplifying writing many Hadoop tasks.
This talk will provide an introduction to Cascalog, a brief tour of its features, and finish with writing a sample application using some live coding. The intent is to help people unfamiliar with Cascalog to identify places it could be of use in their systems.
by Amit Rathore
For a lot of us, Clojure is very different from anything we’ve used before. The syntax is different, the functional paradigm is different, multimethods are new, the macro system is alien, immutability, side-effects, mutating state, protocols, data-types, reify... oh my!
Even after understanding the basics of the language, there’s often uncertainty about how to go about writing programs. How to structure applications? What goes where? What kind of layers? What about I/O? What’s the scene with TDD? Mocking or stubbing? What about distributed computation? How to organize namespaces? What’s this bottom-up thing? Composition? Meta-linguistic abstraction? Really?
This talk is drawn from multiple large-scale Clojure systems, across nearly 5 years of real-world Clojure usage. You’ll see how to get started with designing non-trivial applications, how to ensure the code communicates to anyone reading it, and how to ensure you can grow the code-base and the team, with some level of sanity.
Everyone knows the value of exposing their web APIs to internal and external clients. A well written web API is a great asset to any organization and to all that use it.
Now, there’s been enough written and said about this topic, so what new ground could we cover here? The question we’ll try to answer is, “then why do most web APIs suck so bad?"
This talk is about a holistic, end-to-end approach on how to build good web APIs using Resource Oriented Architecture (ROA) and Clojure. I’ll talk about the importance of ROA, and then show to build an awesome web APIs example Clojure code. And yes, it will be Richardson Maturity Model Level 3 compliant. Don’t know what that is? You’ll learn that too!
by Steve Miner
Don't panic if you're unsure about picking up data-readers. It turns out that tagged literals are mostly harmless. Much as Douglas Adams produced "The Hitchhicker's Guide" material in multiple forms, a Clojure programmer can assign data-readers to process tagged literals with customized implementations. Clojure 1.5 adds a new feature that makes dealing with unknown tags simple and convenient. We'll also talk about the Extensible Data Notation (EDN), which aims to be the Babel fish of data transfer. Finally, we will explore a few unothordox uses for data-readers.
We built a Platform-as-a-Service written entirely in Clojure. CircleCI has run hundreds of thousands of builds for thousands of customers and every line is written in Clojure.
PaaSs tend to be written in a combination of scripting languages and bash. However, Clojure's features were uniquely well-suited for the task. Clojure allowed us to compose high level features such as DSLs, threading macros, multithreading, middleware with very low-level concepts such as EC2 machine managment, VM building, filesystem twiddling, and running arbitrary user commands over SSH.
We discuss what went and well, and problems we faced, including JVM OOMs, horizontal scaling and distributed systems headaches.
Having an expressive programming language like Clojure comes with a price: it's not always clear how programs should be structured. Without classes in the traditional OOP sense, what are the fundamental building blocks of an application? How do we divide up our source files? Where do different kinds of state and behavior "belong"? These problems become more acute as programs grow larger and more complex.
This talk will present patterns for building Clojure programs out of independent components. Topics will include:
Clojure's rich interactive development encourages developers to explore a solution space while experimenting within a problem domain. But what if those explorations could be captured as tangible specifications, and used to automatically generate tests and documentation, be applied as contracts for functions, and even used in external verification systems?
This talk dives into such a system, illustrating the advantages of unifying core.contracts, test.generative, and external systems under a single common specification, captured as a value.
Specification-as-a-value serves as a useful tool that guides developers through all phases of the software development lifecycle: from design to deployment. A single specification could ensure your conceptual design is complete, that your functions conform to the design at runtime, and the functions pass an array of automatically generated tests.
Designing hardware is a fascinating and frustrating procress–the virtually unlimited parallelism and customizability are often outweighed by the simple, relatively inexpressive languages would-be hardware programmers must use. Piplin is an open-source DSL that allows a subset of Clojure to be automatically converted into a hardware description, which can then be placed onto an FPGA or made into a silicon chip.
Piplin has several interesting features: it provides polymorphic implementation of the standard Clojure library, along with a flow-typing and inference system that allows for libraries to define new numeric and structural types. Piplin also takes an atypical approach to embedding–it can actually produce hardware descriptions or Clojure functions without any performance penalty due to an innovative internal representation.
This talk will discuss the design of Piplin, focusing on how it leverages Clojure's polymorphic capabilities to provide an extremely flexible system for simulating and synthesizing hardware modules in Clojure.
Puppet is the most widely deployed open-source configuration management tool on the planet, written in Ruby. As deployments got bigger, people began demanding more of Puppet: store more data, store it more reliably, store it more quickly, and give us better ways to access it.
Thus, PuppetDB was born. It's an open-source, centralized storage system for Puppet users; it persists object graphs, key-value data, and time-series logging information. Ruby wouldn't cut it for us. Written in Clojure, it features a bespoke query language and compiler, exploits and embeds a wide variety of JVM libraries such as ActiveMQ and Jetty, uses a whole ton of Clojure libraries for things ranging from cheshire to ring to clojureql to slingshot and so on.
In this talk we'll discuss a few major topics: what PuppetDB does and how it does it, the experience of bringing Clojure to a Ruby shop (comparing and contrasting the toolchains), and things we learned from having thousands of non-developers consuming our Clojure code.
by Dan Hammer
Deforestation accounts for 15% of annual greenhouse gas emissions, more than the entire global transportation sector. We will all die if carbon emissions continue apace. Any viable effort to curb the rate of deforestation depends on open and transparent monitoring of forest clearing activity.
We have developed an algorithm to parse terabytes of NASA satellite imagery toward a tractable estimate of deforestation at 500-meter resolution, updated every 16 days. The algorithm is now the basis for Global Forest Watch, which will be released in April 2013 by the World Resources Institute.
We rely on Clojure and Cascalog to analyze and classify the spectral time series of each pixel in the humid tropics. A few off-shoot Clojure projects support the data processing, including lein-emr to manage an Amazon Web Services cluster and cartodb-clj to interact with a large-scale spatial database directly from the Clojure environment.
by Chas Emerick
Friend is "an extensible authentication and authorization library for Clojure Ring web applications and services".
We'll take a look at Friend's overall design and how you can use it to model and implement authentication and authorization for your app within the Ring tradition. We'll also go beyond well-worn paths like interactive form login and role-based authorization to see how Friend may be extended to implement things like multi-factor authentication, phishing prevention, and arbitrary authorization schemes.
LLVM is a compiler toolkit for developing native languages. With companies from NVidia, to Apple pouring resources into this project, how can we leverage these libraries in Clojure?
In this talk, I will show my work writing a compiler in Clojure that uses LLVM for code generation. This project is known as clojure-py2. This from-the-ground re-write of clojure-py uses LLVM to create native (machine-code) Python extension modules on the fly. The resulting code is not only faster than python, but with proper type hints can approach the speed of ClojureScript and Clojure.
Running Clojure on Python opens up new areas to explore. Systems like the Raspberry Pi are often too memory constrained to run a JVM language. Python fills this gap by running closer to the metal, while also providing an extremely rich set of libraries to pull from.
Clojure's targeting of the JVM makes a vast array of libraries readily available. However, this dependency puts certain platforms, such as iOS mobile devices, out of reach. With the advent of cross-compilation approaches such as ClojureScript, these limitations are starting to lessen. This talk will focus on clojure-scheme, a recent fork of the ClojureScript compiler that translates Clojure code to Scheme code. In combination with the mature and robust Gambit Scheme compiler, it now becomes possible to compile this Schemified Clojure code to raw C. This talk will show how to use this trick to perform stunning feats never seen before, such as creating tiny and efficient executables or running Clojure code natively on iOS with full access to Objective-C libraries. We will also look behind the curtain and see the semantic similarities between Scheme and Clojure that make this technique straightforward, as well as the differences that pose more difficulty.
The relational programming model provided by core.logic frees developers from the impedance mismatch, and other limitations of Object Relational Mapping systems (ORMs). This allows for massive gains in write speed and schema flexibility, avoids the rigidity of a single entity-relationship model and artifical notions like object identity, and greatly reduces the complexities of modeling time and change.
By extending core.logic defrel/fact we were able to define a set of relations that worked both with core.logic, and also mapped naturally to SQL tables. A single data model was used to build our rule system for identifying malware behavior, as well as being the schema for our analysis and correlation database which is served up via a Clojure web app.
In this talk we will be presenting the tradeoffs involved with moving to a purely relational model, and ditching your ORM. We will also introduce some of the tools we built to facilitate this.
by Zach Tellman
A system in motion generates tremendous amounts of ephemeral data.
We put some of it our logs, trying to capture as much context as we can without drowning in it, but this gives us only glimpses of what's actually going on. We can use the logs and source code to guess, but this remains a painful, error-prone process.
This talk will discuss how to instrument and analyze running systems, using real world examples from Factual's production systems.
Macros are the classic meta-feature of lisps, enabling the implementation of nearly anything imaginable. On the other hand they're frequently overused and their overuse is frequently complained about. Against all good judgment, we will give an intuitive account of what macros are, how they work, and how to analyze their behavior; give advice for when to use them or avoid them, and how to write good ones when you need to; and show off some of the wilder possibilities.
by Ryan Senior
[This session is part of the miniKanren Confo colocated with Clojure/West.]
Traditional logic programming allows only a single constraint via equality. Until recently, this was the only option in core.logic. Constraint Logic Programming (CLP) allows the user to declare more specific information about the possible values of a variable. A variable could be declared as a number in a specific range, for example. This goes beyond a basic equality check and allows for efficient higher level operations, such as arithmetic. With features inspired by cKanren, core.logic now has support for CLP. This opens up another large set of problems that core.logic can help solve efficiently. This talk will introduce CLP using core.logic through examples and live coding. The coding and examples will build to some logic puzzles and give attendees the essential background to have some fun with core.logic and CLP.
by Ryan Neufeld
In the Clojure world, Emacs is king. From Paredit to nREPL, Emacs and Clojure are a match made in heaven. Find out how the experts use Emacs to bring joy to their Clojure. Let me help you set up an effective minimal emacs config, go beyond the basics with paredit and make nREPL free your code.
We Clojurians live a privileged life for most of our days: quick REPL prototyping, immutable data structures, abstractions, declarative statements, multi-methods, you name it, it's all fn!
But this fn stops when we are to deploy our beautifully crafted functional applications inside a larger infrastructure. Then it's all pain: setting up servers, installing the right versions of server software, updating configuration files... Ugh! No more quick REPL prototyping, abstractions, code reuse, declarative statements, immutability; all gone! It gets worse though. We now use platforms like Hadoop, HBase or Cassandra that require complex cross-node configurations. We have multiple developers wanting to test on different versions of the software and infrastructure. There are also many environments to deploy on: Integration, QA, Stage, Production, and There are different OS distros and versions to support, each with their own quirks and bugs. Oh!, and different hardware options too: cloud, virtualized, or even bare metal. You get the gist: a combinatorial explosion of complexity.
You say "Automate it with scripts!", but I say "Automation not enough!". "Scripts + database then!" you say... "Are you paying attention!?" Why should we develop in LISP-land but deploy Perl-land? Can't the same principles of functional programming make infrastructure automation as pleasant and productive as coding in Clojure? It is possible, and this talk is about what it takes to build and grow such stress-free Functional Infrastructures.
It's said that if Lisp is the ultimate high-level language, then Forth is the ultimate low-level one. The Clojure community is on the cutting edge when it comes to embracing good ideas from the full-spectrum of programming paradigms. While core.logic looks up the spectrum to import good ideas from Logic Programming, Factjor looks down the spectrum to import good ideas from Concatenative and Stack-oriented Programming.
This talk begins with a brief history and background of concatenative programming and why it matters. Then, we'll take a whirlwind tour of the Factor language. Factor is to Forth what Clojure is to Common Lisp. The remainder of the talk will be a demonstration of Factjor, a concatenative programming DSL embedded in Clojure, and DomScript, a new Browser DOM manipulation library for ClojureScript that takes full advantage of the concatenative style.
Why are you excited about Clojurescript? Is it the promise of one language to rule them all or the allure of Clojure style, repl driven development for the browser? Unfortunately this dream is not reality, yet. To port your clj libs/apps to cljs you must walk a long path, with many obstacles: host specific apis that lead to code forks and other annoyances. There is shorter path.
This talk will introduce several techniques and new libraries that will:
Simulation allows a rigorous, scalable, and reproducible approach to testing. Simulation breaks the testing process into distinct steps: modeling, defining activity streams, execution, result capture, and validation. Artifacts from each step are captured in a time-aware database, so steps can be run (and re-run, and enhanced) independently of each other.
This talk will introduce simulation testing, walking through a complete example using Simulant, an open-source simulation library.
18th–20th March 2013