by Seth Vargo
There are two sides to monitoring – exposing problems with alerts and acting upon those alerts to find solutions to the exposed problem. For exposing problems, users can define any script for Consul to intelligently check and report the health status of all nodes in a cluster. These scripts could be as simple as returning a 200, or as complex as querying the load and query response time on a database server. Other monitoring solutions already provide such functionality, but where Consul shines is in the second half of monitoring – automatic intervention to find solutions to problems without human operators.
Since Consul has built-in health checking, it not only notifies operators of a node or service failure, but automatically routes traffic away from unhealthy nodes. Consul is also able to re-route traffic back to a troubled node, once the node reports it is healthy again. In this way Consul pushes the existing paradigms of monitoring, making it much more than a simple notification system. Rather it surfaces problems and solves them without human intervention. Don’t worry about that pager going off in the middle of the night – rest easy with Consul.
This talk will cover many of the high-level features of Consul as they relate to monitoring. While this talk may seem deeply technical on the surface, it is aimed at a very large audience. I will not discuss setup or installation of Consul, but I will demonstrate the Consul Web UI and command line. There will be some commands/code on slides, but there will be no live demos.
One of the goals of this talk is to demonstrate the Consul is more than a traditional monitoring solution - it is a runtime for the modern datacenter. Consul allows infrastructure to adapt to the changing needs of any environment.
by yaniv shalev
Finding the effectiveness of a Marketing campaign is complex. Working with Fortune 500 companies in their conversion of customers on their million dollar marketing campaigns is extremely complex. Data comes from a multitude of sources and algorithms must be built to understand their data and answer tough questions. Data must be accurate, algorithms must be novel and dashboards must provide correct insights. Dashboards give marketers the ability to make order, see what's important, make decisions and act. And they needs answers FAST. How fast?
We visualize 300TB of data in less than 5 seconds. Why, because if we can do this marketers win and if they can win we win.
This talk will explain the techniques and lessons learned of how to create interactive deep analytics dashboards of very large data sets.
Micro-services are the new kid in the block. Everyone are talking about them. Everyone wanna do them. But what makes a service into a micro-service? how large should a micro-service be? what advantages does it bring? And just as important, what are the limitations of a micro-service? In this talk we will share Wix experience with over 100 micro-services.
by Issac Goldstand
In the current technical world, SaaS providers have plenty to help them out: from public clouds, to containers. From microservices architectures, to limitless scaling potential. But when you need to deploy multiple singe-tenant applications that use these, how do you manage to share resources while keeping sensitive data apart? In this presentation I'll talk about how we did it at ironSource.
In this talk I’ll focus on how we quickly discovered growing pains when scaling up a single-tenant SaaS to multiple customers at ironSource.
On the one hand we wanted to make use of shared resources (be it code, containers or physical resources) as much as possible; on the other hand, we promised customers a single-tenant application, and so needed to segregate sensitive data.
We needed a solution that would allow us segregated data, across multiple datacenters while remaining cost-conscious regarding resource usage. And, of course, the system would need to be balanced between self-managing itself and requiring manual intervention for unlocking secure data.
In the end we built a mashup of segregated secure data stores on (as much as possible) shared architecture at AWS using some great emerging (and some more time tested) open source technologies, including Chef, Consul and Vault.
by Oleg Verhovsky and Raziel Tabib
In this session we will go over the fundamental of Docker & Docker compose.
We will show how to migrate your application to micro-service in easy and efficient way.
The session will include a live demo of Docker technology and Codefresh platform.
by Ilan Yaniv
Production Map is an open source IDE for DevOps
by Nir Sharony
With more than 75 million users who have created 27 million family trees containing over 1.8 billion individuals, and over 6 billion historical documents, records and newspapers, MyHeritage has become the world's leading destination for people to discover, preserve and share their family history.
MyHeritage's home-brewed FamilyGraph RESTful API is a key component of our growth strategy; not only is it used by us internally, but it allows us to generate additional revenues and increase users via partners who seamlessly integrate our highly accurate matching technologies. Millions of API calls are made every day by partners all over the world.
But with big data comes big challenges, and in this talk we're going to be taking a look at some of the performance, scalability and security issues we deal with, the design decisions we've made and the lessons we've learned.
If a customer changes their address, it's often not enough to update their master data record. E.g. the component processing customer orders might have to learn about the update in order to ship to the correct address. In a distributed architecture, you have to spread information about a master data update to several client apps. You can do this via REST, but if a client app is temporarily down or too busy to accept requests, it's going to miss the information. Adding a new client app requires changes to the master app. Asynchronous messaging helps to avoid such a tight coupling of components. And it offers much more than a simple action trigger: parallelizing computing-heavy tasks, load testing, or migrating existing components to new services are some of the possibilities explored in this talk. You're going to learn how to get started with asynchronous messaging, and how it helps you to keep your codebase clean and your overal l system stable as well as maintainable.
by Dor Laor
CPU core counts continue to grow, along with the raw speed of networking and storage devices available on a modern system. Software design approaches that were valid and safe even a few years ago are no longer sustainable. On new hardware, the performance of standard workloads depends more on locking and coordination across cores than on performance of an individual core.
The team behind the KVM hypervisor and the OSv unikernel is moving up the stack. The Seastar platform enables extreme high-throughput, low-latency applications on Linux, and can make existing real-world workloads run completely asynchronously.
Seastar uses shared-nothing data structures that eliminate costly locking between CPUs, and a dedicated user-space TCP implementation that runs on DPDK. These radical changes in the server design translate into 5X-10x performance gain while preserving all of the Cassandra goodies
by Lior Bar-On
Microservices recently becomes the new "default" architectural style for systems. Does using microservices guarantee eternal success, or can microservices implementations can go wrong as well?
In this talk we will get back to the foundations of Microservices architecture - what it came to solve, when to use it - and when not?
by Roy Sasson
We analyze the importance of two types of engagement signals in the domain of digital publishing: (1) content popularity, measured by page-views & click-through rates within the Outbrain network, and (2) content share-ability, measured by the proportion of Facebook shares. First, we show that counter-intuitively, the correlation between these signals is very low. Second, we characterize content which is very popular, but not shared, as “socially unacceptable” and show various examples. Third, we compare the predictive value and relative importance of each of these signals in production, as part of our recommendation system’s modeling.
The results of the research are based on more than 100,000 articles, spanning across 200 publishers worldwide, which accounted for more than 1.8 Billion visits during two different time periods (Feb. 2015 and Nov. 2015). The implications of this study are important for data scientists, publishers and content marketers.
(Joint work with Ram Meshulam, Outbrain)
by Shimon Tolts
How we transformed ironSource's BigData pipeline - ironBeast into Docker containers.
The year 2015 put containers in the spotlight, but now that everyone is using Docker it is time to put containers in production. Grid clusters are again on the rise, this time in the form of Google Kubernetes, Docker Swarm, Amazon ECS and Mesosphere just to name a few. This talk will explain the basic principles of how grids solve problems related to managing large fleets of containers, and what new problems grids introduce to application developers
by Danny Varod
As a global payment company, we in Payoneer need to make sure that we do not conduct business with any sanctioned entities. Do to the volume of our business, we need to use automated solutions, thus requiring automatic screening, then manual review of the suspicious findings.
Using NoSQL & Map Reduce together with advanced phonetic algorithms, we perform large volumes of phonetic comparison as a part of this automation.
by Anton Weiss
Configuration management tools has been evolving for the last 10 years with each newcomer pretending to cure the illnesses of their predecessors. Now that most of us agree that codifying our infrastructure is the right way to go - let's try and choose the right weapon of mass configuration.
by Aviv Laufer
How do we collect analytics at Rounds. We have build a pipeline that starts at the user mobile device and flow via our collecting servers (written in golang) until in inserted into BigQuery and Elasticsearch cluster. We will share our experience and the journey that we have made until we reached our current system
by Rotem Hermon
The Actor Model is an alternative approach to writing concurrent software without the problems introduced by multithreaded concurrency.
“Virtual Actors” are a new development of the actor model, aimed for solving distributed concurrency issues, providing better productivity, and building cloud native scalable architecture.
With the recent trend of moving into microservices based architecture, there is some confusion about Actors, microservices, and how they fit together.
This session will introduce Actors, Virtual Actors, and how they make distributed application programming a lot simpler. We will then explore the idea of building a microservices architecture on top of Actors, how they combine, and some of the lessons learnt at Gigya doing that.
by Itamar Haber
Redis - the open source, in-memory data structure store - just turned 7 years old. This makes it exactly the right time for a show and tell about its new data super powers, so in this session I'll take your data on a journey through fields made of bits, around the Earth, through the dimensions and to the Moon.
Put somewhat differently, I'll also talk about managing obscenely large numbers of huge integers, indexing n-dimensional and geospatial data, taking your Lua scripting to the next level and securing your Redis from harm.
by sagy rozman
In the presentation Sagy will demonstrate his take on the style of TDD described in the book Growing OO Systems Guided By Tests written by Nat Pryce and Steve Freeman.
Sagy will show how to apply the principles from the book as well as some additional design principles in the context of web service oriented architecture.
by Omri Fima
The 'Black Friday fail' is the greatest fear of every major online retailer. Since downtime equals money, and in Black Friday it means quite a lot of money.
But the sad truth is that a failure of a service is inevitable, especially in a large distributed system. So how can we survive a failure of a service when it inevitably fails.
* In this lecture I will show how failures in large systems differs from failures in small systems.
* Will show examples of resilience engineering.
* Why simulate failures, and how to do it in your system.
* How to use gradual rollout, circuit breakers and automatic fallback to protect your system.
* The importance of failing fast, and failing silently.
* And the misconceptions we all have on how a large scale website failure unfolds.
by Gil Levi
Deep Learning methods have become vastly popular in recent years and seem to beat the state of the art in almost every image classification benchmark. The purpose of this talk is to give a "how-to" introductory talk to deep learning. A short theoretical overview will be given following a technical deep dive on how to train deep networks with a few demos, practical examples and tips.
22nd March 2016