by Adam Carmi
Automated visual testing is a major emerging trend in the dev/test community. In this talk you will learn what visual testing is and why it should be automated. We will take a deep dive into some of the technological challenges involved with visual test automation and show how modern tools address them. We will review available Selenium-based open-source and commercial visual testing tools, demo cutting edge technologies that enable running cross browser and cross device visual tests at large scale, and show how visual test automation fits in the development/deployment lifecycle.
If you don’t know what visual testing is, if you think that Sikuli is a visual test automation tool, if you are already automating your visual tests and want to learn more on what else is out there, if you are on your way to implement Continuous Deployment or just interested in seeing how cool image processing algorithms can be, this talk is for you!
by Oleg Volodin
How many of us, C# engineers, dreamed of an extremely easy and really effective Selenium based framework/library? Like Selenide in Java. 😉 It happened that it’s not a big deal just to run Java Selenide on .NET via JNI. In this talk we will see how Selenide runs on .NET and what are the specifics based on its real usage in production. We also will get familiar with the tool that make all this possible – jni4net, and share experience of its usage to port Selenide, all the pros and cons.
by Alex Kogon
The Selenium Grid is a vital piece of infrastructure for any serious Selenium WebDriver test automation solution. Many organizations find the setup and maintenance of a Grid a difficult and expensive task. While outsourced solutions exist, they are too slow for any real professional Continuous Integration or Continuous Delivery workflow, and are expensive.
The Selenium Docker project provides a very easy to setup, deploy, maintain, and use solution for running a Selenium Grid on Docker supporting Firefox and Chrome on Linux. This talk will present how to use the Selenium Docker project, how to implement it as a PaaS solution, how to connect to Selenium Docker VMs and watch a test running for debugging, and extensions such as video recording.
Fortunately today we have really easy and effective tool for Web UI Automation – Selenide for Java. In other languages the sky is not so blue and clean. :) But world is changing and today we are meeting Selene – its alternative in Python. Selene also gathered ideas from other popular selenium framework in Java – htmlelements, which gives ability to construct your own custom elements in addition to simple WebElements. In this talk we will define the key features that make an abstract selenium wrapper really effective and will see how to use them with Selene in Python.
by Noah Sussman
Test Oracles address a mutex problem that is faced by all teams who *successfully* practice continuous deployment. The problem test oracles address is: once your tests have run, and you have the results, how do you know what that means? In other words, what are you testing? What do your tests tell you? How do you know what you know?
Command line tools provide powerful data analytics when applied to a mature CI system and deployment pipeline. Such systems are known to be opaque — but the GNU/bash command line environment is powerful enough to solve those problems (with a little help from HTML5!).
A healthy CI system and deployment pipeline will naturally accrue a wealth of filesystem artifacts which can be mined for Test Oracles using command-line tools. Topics covered in depth will include:
• JSON testing and transformation with jq.
• Jenkins monitoring tips.
• Network analysis tools for Selenium developers.
• Visualizing xUnit test runner output.
• Analyzing your captured screenshots.
• Tips on introducing new dashboards to your team.
By providing yourself and your team not just with data but with actionable oracles, you will discover new reservoirs of activity that can safely become automated-rather-than-manual work.
In this talk we will go deeper into specifics of porting Selenide from Java to other langauges using Python as a sample. We will make an overview of architecture and some tricks that make it effective and fast enough. Based on the presented information you will understand deeper how selenide-like wrappers work so you can implement it yourself for the language that still has no Selenide alternative. 😉 You also can use these ideas to enhance your current selenium based framework.
by Igor Khrol
There are various approaches to write automated tests: unit tests, API calls, Selenium of course. I want to show how to perform the same checks on various application levels and what changes are required to make it possible. We’ll write together Ruby on Rails application and adopt it to our needs. Ruby knowledge is not really required to understand the presentation: only your desire and engineering attitude.
by Alex Kogon
Selenium and its cousin Appium provide a powerful, open source solution for Test Automation for Web and Mobile applications, but what about native Windows apps, fat-client Java apps, fat-client C#/.NET apps, SAP, and other platforms?
HP’s LeanFT provides a powerful, standardized format for coding Automated Tests for Web applications, iOS and Android applications, SAP, .NET Web Forms, WPF, and Java applications in Java or C# that closely follows the Selenium/Appium paradigm: objects on the screen are identified via a locator, then can be queried for their content, type into, clicked on, etc. It is integrated into Eclipse and Visual Studio (with an IntelliJ version on the way), and provides an easy to use Object Identification Center for creating and embedding locators.
For anyone familiar with Selenium WebDriver or Appium Test Automation with Java, LeanFT provides an easy route to testing many different types of applications.
This talk will cover the key pain points of UI automation: Speed, Reliability and Monitoring!
From speed perspective it will be about the means to achieve distributed execution of UI automation using AWS and Selenium Grid aka Autoscaled Distributed Automation with the goal of “Running all the test within the time taken by the slowest test case”. This is something we implemented and currently running all our 9000+ UI tests round the clock as part of our pipeline.
I will also cover key learnings on how we achieved reliability with UI automation, how and why we are focusing on reducing the overall UI Test Cases. Moving away from having 100+ jenkins job to only very few, storing UI results into MongoDB and NodeJS based reporting for easy monitoring of results!
In this talk I want to compare typical tasks performed by test automation engineers, and their implementation in Java and Groovy. The main message of the report is to show the participants the tools and libraries that will allow test engineers to be more efficient and to solve typical tasks in a short time. Working with files, working with Json and XML files, creating customers for REST and SOAP services – all of this can be simplified by using the flexibility of Groovy and reliability of Java. This talk will be interesting and helpful not only for test automation engineers, but also for people from the manual testing, because they will learn about the simple and reliable tools that will help them to solve daily routine tasks.
Everyone knows about that one flaky test, or that one slow test, but no one ever fixes them. I will talk about how gathering metadata about your test runs can teach you about your tests, and help you test faster, smarter, and better. I will give examples from several years on the Developer Infrastructure team at Google, supporting the running of tens of millions of test cases using Selenium (and many more without) every day.
There are a lot of books and publications about the continuous integration in the world. But in my experience it’s difficult to find information about how to open quality gates between automated tests and to continuous integration practice to in your current project. After reading several articles and even a couple of books you will understand how to work with it. But what next? I will share with you practical tips and tricks on how to lift iron curtain to your automated tests before a continuous quality practice today. It is for this reason why I am pleased to share with you my acquired experience in my presentation.
by Dan Cuellar
The tools and practices refined over the last decade by web developers have not crossed over easily into the mobile community. Mobile devs have been without tools like continuous integration and automated testing, which enable delivery of high-quality apps more rapidly. A key piece of the solution for mobile is Appium, an open-source, cross-platform, language-agnostic mobile automation framework you can use to write automated UI tests for Android and iOS apps (native, hybrid, web), to be run on emulators or devices.
In this talk we will explore the philosophy that underlies the work behind Appium, including the open-source nature of Appium’s diverse, active and friendly community. Then, after a brief explanation of the technology that powers Appium, we’ll move into the technical portion of the talk, where attendees will see easy it is to write Appium scripts using the WebDriver protocol in their favorite language.
Ultimately, the talk provides an argument for moving from manual QA to automated testing for mobile, an understanding of the mobile automation landscape, and a solid introduction to Appium, which leading projects and companies around the world have chosen as their mobile automation framework of choice.
On this Q&A panel everybody will have a chance to get an answer on the most interesting questions from our top speakers. It is always the most useful and interactive part of the conference.
by Noah Sussman
xUnit test runners are one of the basic capabilities of rapid iterative development practices such as Continuous Integration and Test Driven Development.
Topics covered in depth will include:
• How to get the output you want from your test runner: JUnit.xml, JSON, TAP and even emoji!
• Instrumenting your test runner to get granular timing and debugging information.
• A real-world example of adding functionality to a test runner.
by Viktor Losiev
Your web application use Web Players for showing Video, 3D objects, Flash,Flex or Unity 3D applications in browser? You can not configure Web Driver for access to UI Controls of your online Flash or Unity 3D game? Well, in this cases the best idea is to use Computer Vision technologies, and to connect Computer Vision framework with Web Driver.
OpenCVSharp is a free .NET library which give you opportunity to detect UI elements with help of image recognition algorithm. Using OpenCVSharp and Webdriver allows you to create test automation framework which can work with UI HTML elements of your web page and UI elements of your application which loaded in Web Player – Flash, Flex, Unity 3D – etc. Also, the above solution can be used for automation testing of web applications which connected with 3D graphics, Video, GIS, Games etc.
Used technologies: C#, OpenCV, WebDirver, WinApi32.
by Oren Rubin
In this talk we will delve into the biggest challenge that Test Automation developers face, finding elements robustness i.e. finding elements while the Application Under Test keeps changing. We’ll categorise methods, and show where developer fall, where machines fall (Record/Playback), and suggest a new way for locating elements, and analyze the skill-set required to overcome those difficulties.
In this talk I want to show how to do automated testing of Web UI front-end faster, more reliable and accurate. Without back-end. With mocks. Is that possible? Let’s talk about this!
Applications are becoming more and more difficult for testing. It’s a time when we all must forget about the black box and the complex functional scenarios. It’s time to become real engineers.
In this talk I’ll share our successful experience of building reliable and scalable cross-platform automated acceptance test suite for a product called Citrix Concierge.
The product is built on a top of micro-services architecture. So, we will start from a brief introduction to a test strategy that was defined for this product. Then we’ll dig a bit more into acceptance testing and cover such topics as architecture and implementation of cross-platform acceptance test framework. Also, we I’ll show how we’re doing implementation of isolated tests, parallel execution, running at a scale using docker, etc.
We’ll be talking about technologies like: ruby, cucumber, rspec, capybara, appium, selenium webdriver, factory_girl, browsermob-proxy, faye.
When you’re facing with an open API testing, you know that the testing itself is not a single problem. The main goal of this report is to share an approach of autoscalable testing with a bonus in automatic documentation generation. Java 8, Junit, Allure report are used as a tool set.
In this talk we’ll cover deep technical questions to get better understanding of some common issues and pitfalls we could face with while functional tests scaling:
– Static context: friend or enemy?
– Accurate Selenium Grid nodes’ identification: how to run tests on particular configuration?
– Generic DataProvider: how we can safely manage test data?
– Object pool: how to distribute unique entities between tests?
– Support tools: rebooting remote services and advanced screenshots processing.
You could find more information about this talk here.
On the size of hundreds or even thousands browsers, stock Selenium grid works unstable and slow. It doesn’t scale and doesn’t provide fault tolerance.
In Yandex for many years we used client side balancing over multi-hub installation to solve these problems. It was easy when only testers did testing, when they used the same jUnit approach and ran tests in the same environment (our AQuA framework). But when developers joined testers and brought their JS and Python frameworks, we realized that client side approach doesn’t work for Yandex as a whole.
In 2015 we introduced Grid Router solution.
– It works like hub for tests.
– It provides fault tolerance and scales infinitely.
– It uses only stock Selenium packages.
– It works in a cloud and includes specific strategy for hubs in the cloud.
– It is open-source.
In almost a year of using and tuning we found out a lot about our solution, got an interesting and wide experience in setting up browser instances including IEs and mobile browsers and keep moving.
In my talk I want to dig into details of Grid Router solution, share our experience and plans.
Working in many companies as consultant, delivery manager or tech lead I have always seen the same mistakes made in test automation process. I could count successful cases on fingers of one hand. Sometimes people don’t understand the true value of test automation, sometimes just could not organize this process spending lots of money and time without any significant result. I want to share 5 top mistakes aggregated from whole my practice and solutions I recommend for them.
Everybody knows about test pyramid. It is always used on slides about test automation and presented on different conferences. But understanding test pyramid and its layers is not enough. There are more important questions: “how to find right balance for particular project?”, “how to build this pyramid from scratch for already running project?”, “what to do if you are limited in time and resources?”, “how to measure if your pyramid is good enough?”. In this talk I will bring some more “secret ingridients” on the table to make test pyramid concept really useful.
by Justin Ison
In modern times we have many different cloud testing services to choose from. These cloud services are useful and help reduce the burden of building and maintaining your own Selenium Grid environment. However, there are many scenarios in which you need your tests running locally, such you work for a government (or agency) and cannot expose your data to the cloud, or the service costs are too expensive to run all tests on every commit.
This presentation will feature getting started with setting up your own mobile device grid. Running your tests in parallel and distributed, capturing critical report data (logs, screenshots, and video), and leveraging cloud test services (such as SauceLabs). I will also talk about the challenges and lessons I learned along the way.
26th–27th February 2016