Sessions at QCon San Francisco 2011 on Friday 18th November

Gilad Bracha will provide an overview of Google's new programming language for structured web programming. For more information on Dart: http://www.dartlang.org/

The premise and the promise of using web technologies for building mobile apps is well understood. But are we there yet? This presentation takes an honest look at the state of the art of mobile standards and implementations. We discuss what can be built, what has been built - and of course what will soon be possible - with these exciting tools and approaches.

Git, Gerrit, Hudson and Mylyn are increasingly popular (and open source) Java development tools. While these tools can add a lot of power to the software development process, adapting to the complexity of their workflows and learning new DVCS terms such as push, pull and fetch can be daunting. We will show how these open source tools can be integrated together in an automated manner, to remove a lot of the manual steps which would normally be required. This integration includes connecting issue tracking, code review, commits and builds so that typical (and annoying) processes such as browsing from defects to changes or managing code reviews are now seamless. You'll see how much faster all of these components are when this automation is in place, and how you stop having to think about the tools being used and can just focus on the code you're trying to write. The best part is that everything you will see and use is open source, so you'll be able to set up all of these tools in the manner described as soon as you get back to the office if you want.

In November 2009 I set out on a journey to learn Erlang. And not just learn it: I wanted to immerse myself in it, get under it's skin, feel it, get to think like Erlang. I took it for a test drive, and ended up spending 18 months in Erlang's company.

This is the story of my trip. Not so much a talk about Erlang itself, but about what I realized on the way about where things are going. The systems that we build are getting increasingly integrated, distributed, and fragile. And becoming so at a disturbing rate when I think about it.

Viewing the world from behind the steering wheel of this old beauty, I'll take you on a tour of some of the significant issues we're dealing with in today's software systems: cloud, multi-core, integration, high availability, and living with the fact that our software isn't perfect.

Marketers have long sought more effective ways to reach their audience – to show the right ad to the right person at the right time. Huge volumes of internet activity data, advances in machine learning methods, new hardware and software for large scale distributed computing, and developments in real-time decisioning have made this finally possible. Increasingly the particular advertisement that is seen on a web page is decided in a auction that takes place in a fraction of a second, while the page is loading. In this presentation I will discuss how we, at Quantcast, meet the challenges in personalizing advertising. This process involves multiple machine learning methods to evaluate of about 15 billion individual daily media events and leveraging this data to to make precise bids in almost 100,000 auctions every second.

Netflix migrated its website and streaming service from a conventional datacenter implementation to the Amazon public cloud during 2009-2010. For 2011 Netflix is making a back-end migration from SimpleDB to Apache Cassandra, cutting the connection to the Datacenter, and re-tooling the architecture to build a fully internationalized and globally distributed product. By leveraging multiple AWS regions, Netflix is able to rapidly deploy services anywhere in the world, primarily using Apache Cassandra to manage highly available wide area replication. In this talk, Netflix' Cloud Architect Adrian Cockcroft will discuss how the Netflix cloud architecture takes advantage of almost every feature of AWS, and is optimized for running in a highly automated global environment with ephemeral instances, non-deterministic performance, and agile deployment processes.

Dart is Google's new language for web programming. In this presentation, we explore the most interesting features of the Dart Programming Language in as much detail as time allows and the audience can tolerate. We'll talk about where the language is now and where it might go in the future, discuss design trade-offs, demonstrate tooling and in general have fun.

Distributed version control (DVCS) is a transformative shift in the way that software is developed. Learn how Bitbucket and other Atlassian teams have adapted their agile process to use a DVCS workflow with pull requests and forks, to improve the speed and quality of software being shipped.

Hadoop for the Enterprise Architect Panel:

What is the role of Hadoop in Enterprise Architecture? How does it interact with databases and streaming events? What does the future of Hadoop hold? This panel will bring together leading technologists and enterprise users of Hadoop including Cloudera, Hortonworks, MapR, Amazon, CBSi, and Visa.

This presentation will focus on the development of next generation realtime web apps with the use of Socket.IO. Socket.IO is a Node.JS project that makes WebSockets and realtime possible in all browsers. It also enhances WebSockets by providing built-in multiplexing, horizontal scalability, automatic JSON encoding/decoding, and more.

This talk will cover:
• HTML5 WebSocket, server sent events, long polling and how the realtime transports work
• The Socket.IO API
• The creation of an example realtime app
• It will also introduce people to server side JavaScript, and highlight its importance for the realtime web and the future of web development.

Developing and deploying applications in the cloud should be easy. It should just work, and work well. DevOps don't want to worry about things such as virtual machines, kernels, and packages, they want to worry about their applications and services. PaaS, or aPaaS (Application Platform as a Service) tries to deliver on that goal. But how does one develop a system that is architected for the largest datacenters, but can also run simply on your laptop or in a single downloadable VM? Something that will be run as a service, and distributed as a standalone product as well? We will discuss how Cloud Foundry was architected as a simple, yet scalable, and flexible distributed system that can be run on your laptop, on vSphere, Amazon Web Services, and anywhere in between.

In this session we will dig deep into Cloud Foundry's core architecture and design principles. We will discuss the challenges around scaling and operating a large scale service as we combined the PaaS and traditional IaaS layers, and how we achieve multiple updates per week to the system with no perceived downtime. Allowing user to download a single virtual machine that is a complete replica of the service presented some challenges as well, and we will discuss our approach to offering up the downloadable private cloud.

With the advent of HTML 5, designers and developers are looking for ways to work together most efficiently, and leverage their respective skills without requiring developers to become designers and designers to become developers. In this session, we explore the latest advances in HTML 5/CSS 3 tooling that facilitate the integration of design and development to fully deliver on the potential of HTML 5. We also provide a pragmatic, drama-free view of when and how Flash can complement HTML 5, and discuss the future of Flash in the new Rich Internet Application landscape.

Learn everything you need to know in order to start programming in CUDA C. CUDA C minimally extends the C programming language to allow C code to execute on the GPUs (Graphics Processing Units) present in your system. A GPU is a highly parallel, multithreaded, manycore coprocessor to the CPU with very high computational horsepower and memory bandwidth. For workloads with high degree of parallelism, GPUs can significantly outperform CPUs. GPUs are used today to accelerate applications in several industries, such as Oil and Gas, Life Sciences, Computer-Aided Engineering, Computational Physics, Computational Finance and many more.

The presentation only assumes you have basic background in C programming. It progressively introduces the few CUDA C language extensions through a number of simple code examples.

Running apps in the cloud has never been easier. Providing both low level system services like image resizing, hosting, fail-over, and high level app services like social, databases and synchronization, Sencha.io is the first next-generation mobile cloud helping developers focus on the fun of building apps for their customers. In this session, you’ll learn about what Sencha.io is, how it works, and how to exploit it in your projects.

NoSQL databases are rapidly emerging as a technology for a variety of problems. What areas are they being used in? How do they fit into the enterprise? What are the differences in focus among them? This panel brings together experts in leading NoSQL databases including MongoDB, Cassandra, HBase, Riak, and Couchbase. Alexandru Popescu will be moderating the panel.

It’s a common misconception that Amazon Web Services built its cloud services to solve use cases for the amazon.com retail web sites. In fact, Amazon’s retail web sites have faced challenges like any other business when it comes to migrating to the cloud. This talk provides an architectural history of the amazon.com web site and how its architecture has changed over the last 15 years. The presentation will explore a half-dozen detailed use cases where amazon.com has migrated traditional systems to the cloud. Special attention is given to new approaches for assuring high-availability, low-latency and scalability with cloud architectures.

For years we built web apps that far outpaced the capabilities of the browsers they ran in. Just as the browsers were catching up HTML5 came on the scene - video and audio, canvas, SVG, Application Cache, Web Storage, @font-face, ContentEdible, WebSockets, Web Workers, and more. Now the browsers are racing to stay ahead of the wave that's building as developers adopt these new capabilities. Is your HTML5 app going to ride the wave or be dashed on the rocks leaving users stranded? Learn which HTML5 features to seek out and avoid when it comes to building fast HTML5 web apps

It was always the toughest mobile decision to make … build a mobile website or a native app? Today the answer is BOTH! Appcelerator's Titanium platform allows developers to build mobile browsers and native apps using a single JavaScript-based platform, and reuse 90% or more of the code between them. Apps can be deployed to iOS, Android, and Blackberry with the same 90% code reuse.

NetApp is a fast growing leader in storage technology. Its devices phone home, sending unstructured auto-support log and configuration data back to centralized data centers. This data is used to provide timely support, to better sell, and to plan product improvements. To allow this, the data needs to be collected, organized, and analyzed. Data volumes are growing 40% per year, and are currently 5 TB of compressed data per week. NetApp was previously storing flat files on disk volumes and keeping summary data in relational databases. Now NetApp is working with Think Big Analytics, incrementally deploying Hadoop, HBase, Flume, and Solr for managing auto-support data. Key requirements include:
* Query data in seconds within 5 minutes of event occurrence.
* Execute complex ad hoc queries to investigate issues and plan accordingly.
* Build models to predict support issues and capacity limits to take action before issues arise.
* Build models for cross-sale opportunities.
In this session we look at the design and lessons learned to:
* Collect 1000 messages of 20MB compressed per minute. This uses a fan-out configuration for Flume, reusing Perl parsers, writing large data sets into HDFS, updating HBase tables for current status, and creating cases for high priority issues. It also uses Java MapReduce jobs that process data downstream.
* Store 2 PB of incoming support events by 2015.
* Provide low latency access to support information and configuration changes in HBase at scale within 5 minutes of event arrival.
* Support complex ad hoc queries that join multiple data sets, using custom User Defined Functions (UDF's) to correlate JSON data. These queries benefit from partitioning and indexing in Hive and can query tens of Terabytes of data. * Operate efficiently at scale.
* Integrate with a data warehouse in Oracle.

The North American railway industry is undergoing a renaissance as it is being seen by the market as a safer, greener, and more efficient way to ship goods across long distances. Key shifts are underway the large railways do business, including: scheduled and optimized railway operations, real-time analytics decision support, and positive train control. Canadian Pacific sees this as requiring a "leap frog" in agility to meet coming challenges, in part through the adoption of both VMware-based and Amazon EC2-based infrastructure services to accelerate the delivery of Java, .NET, and SAP application development, along with major changes to the solution lifecycle: agile delivery meets agile operations. This talk will explore CP's continuing journey from IT under-investment to a modern, agile delivery IT organization: the requirements, architectural and technology considerations, and lessons learned so far in our efforts to reduce lead-times from months to days.

This talk is a hands-on presentation of concurrent programming in Ozma, an extension of Scala that adds the concurrency features of Oz. Ozma is based on deterministic dataflow concurrency. This is extremely simple: 'val' variables in Ozma are single assignment and all operations can synchronize on this assignment. This extension has the incredible property that race conditions are impossible. Combined with lightweight threads, deterministic dataflow makes concurrent programming effortless; it is much simpler than any other model we know of. In addition, Ozma integrates both lazy execution and message passing with the dataflow concurrency. The Ozma design combines two pillars of advanced technology, namely Scala and Oz. Scala is based on Java technology and integrates object-oriented and functional features in a smooth way (see http://www.scala-lang.org). Oz is a research language that has explored language simplicity and power for over 20 years (see http://ctm.info.ucl.ac.be and http://www.mozart-oz.org). All necessary information to install and run Ozma is available on GitHub (see https://github.com/sjrd/ozma).

When was the last time you cleaned out your email inbox? Do you even worry about your Gmail or Yahoo INBOX running out storage space? You expect "infinite" storage there! Email storage has become a poster child for big-data. For Multi-Service operators offering email services to their consumers, big data translates to big dollars! Keeping the Total Cost of Ownership (TCO) under control while offering a five-9s reliable service has become extremely challenging.

In this talk, Utpal will outline how his company successfully transitioned from Oracle to Cassandra for a carrier-grade Email product, in order to achieve TCO and Reliability goals.

There has been a explosion in SaaS appications in recnt years and companies of all sizes have gotten more comfortable with storing their business data outside of the firewall. However, the mass adoption of SaaS has created an major integration challenge as companies find their data split across multiple SaaS data silos.

This session will introduce Mule iON the integration PaaS and how it changes the game for integration of SaaS applications. I will provide practical advice and demonstrations on how to unlock your data and synchronize your applications in the cloud and on premise.

To make secure mashups practical, we must address both the offensive code problem and the defensive code problem. The offensive code problem is familiar -- how to keep potentially hostile programs from exceeding their proper authority. The defensive code problem is how to write abstractions with rich APIs to be exposed to potential hostile code that can nevertheless robustly defend their own integrity. Mashing up mutually suspicious bodies of code requires a symmetric solution to both problems.

JavaScript is the language of mashups. We designed the latest JavaScript standard, EcmaScript 5, with the features needed to turn itself into Secure EcmaScript (SES) -- an object-capability subset of JavaScript adequate to solve both these problems. However, JavaScript's evolution is constrained by its history so there was only so much we could fix. Remaining unfixable irregularities of JavaScript create surprising new security hazards. We present several patterns for programming in SES that help programmers avoid these remaining traps and successfully write fully defensive abstractions.

Time permitting, we will also explain how expected features of the next EcmaScript standard will provide even better support for secure programming.

In October, Oracle announced their intention to build an enterprise public cloud for business-critical applications. This cloud will provide a Database-as-a-Service, Java-as-a-Service, and a set of sales, marketing, and collaboration applications made available in a self-service interface, elastic scalability, and monthly subscription billing model. This session will introduce the audience to the architecture and design considerations made during the design and implementation of the service. We’ll discuss the elements that went into building the datacenter, structuring the hardware and multi-tenancy model, how end users administer their services, design considerations for elasticity, and how various interoperability, identity, and security issues were addressed. This session will be ideal for those looking to understand the concerns that go into building a scalable cloud service or those looking to get a technical introduction to Oracle’s new offerings.

Agile software development, which APL users will claim has been in widespread use in the APL community since the 1970's has finally been named by the mainstream. APL is no longer alone in having data-parallel constructs or a "function-oriented" approach to programming. Many other features which were pioneered by APL implementers and users decades ago - such as (safe) dynamic typing, garbage collection, column-store databases, keyed index object stores, and "in-memory OLAP" (which made APL the curse of many a mainframe system administrator in the 1980's) have been absorbed into mainstream products.

Why then, as the APL community prepares to celebrate the 50th anniversary of Kenneth Iverson's book titled "A Programming Language" in 2012, do many APL users still claim that the main obstacle to widespread adoption of APL is that the notation is still ahead of its time? This session will offer some possible answers to this question, and also discuss the potential of APL as vehicle for putting parallel processing at the fingertips of both programmers and domain experts.