Automated E2E UI testing with Python

Introduction

Nowadays there are lots of test strategies that you can find on the internet. I’m gonna list below the most important tests to ensure your code quality in a software development cycle from top to down:

In this article I’m gonna cover only the automated E2E UI testing part.

Part 1: Persuade developers of writing tests

Trust me, you should write tests as a software developer

Test automation can improve the quality of your software. As a software engineer do yourself a favour and start to write tests. We know that applications can grow really fast. It’s very hard to test each piece of a complex application. Especially when you wanna do regression tests, which means testing the entire application with several user scenarios i.e. admin or user before each release, believe me it can cause a huge headache. If you are lucky your company has a QA team and they do all the testing stuff for you. But even though the company you are working has a Quality Assurance Team you should definitely

• write unit tests to ensure the logical part of your code is working
• write integration/service tests to ensure that your DAO (database access) and BO (Business objects) layers work as expected
• write UI tests to automate your UI

Part 2: What is what?

What is BDD (Behavior driven development)?

Behavior-driven development (or BDD) is an agile software development technique that encourages collaboration between developers, QA and non-technical or business participants in a software project.

Behavior-driven development - Wikipedia

BDD specifies that business analysts and developers should collaborate in this area and should specify behavior in terms of user stories, which are each explicitly written down in a dedicated document. Each user story should, in some way, follow the following structure:

Story: The story should have a clear, explicit title.

Scenario: Acceptance criteria or scenario description of each specific case of the narrative. Such a scenario has the following structure:

• Given: It starts by specifying the initial condition that is assumed to be true at the beginning of the scenario. This may consist of a single clause, or several.
• When: It then states which event triggers the start of the scenario.
• Then: Finally, it states the expected outcome, in one or more clauses.

BDD does not have any formal requirements for exactly how these user stories must be written down. The scenarios are ideally phrased declaratively rather than imperatively — in the business language, with no reference to elements of the UI through which the interactions take place.

Gherkin language

Gherkin is a Business Readable, Domain Specific Language created especially for behavior descriptions. It gives you the ability to remove logic details from behavior tests. Gherkin serves two purposes: serving as your project’s documentation and automated tests.

In BDD, a developer or QA engineer might clarify the requirements by breaking this down into specific examples. The language of the examples below is called Gherkin. You can use the Gherkin language to define test cases. It is designed to be non-technical and human readable, and collectively describes use cases relating to a software system. The purpose behind Gherkin’s syntax is to promote Behavior Driven Development practices across an entire development team, including business analysts, QA team and managers.

There are a few conventions.

• Single Gherkin source file contains a description of a single feature.
• Source files have .feature extension.

Part 3: Writing .feature files

Every *.feature file conventionally consists of a single feature. Lines starting with the keyword Feature: (or its localized equivalent) followed by three indented lines starts a feature. A feature usually contains a list of scenarios. You can write whatever you want up until the first scenario, which starts with Scenario: (or localized equivalent) on a new line. You can usetags to group features and scenarios together, independent of your file and directory structure.

Every scenario consists of a list of steps, which must start with one of the keywords Given, When, Then, But or And (or localized one). Behat treats them all the same, but you shouldn’t. Here is an example:

Feature: Serve coffee
  In order to earn money
  Customers should be able to
  buy coffee at all times
  Scenario: Buy last coffee
    Given there are 1 coffees left in the machine
    And I have deposited 1 dollar
    When I press the coffee button
    Then I should be served a coffee

In addition to basic scenarios, feature may contain scenario outlines and backgrounds.

Scenarios

Scenario is one of the core Gherkin structures. Every scenario starts with the Scenario: keyword (or localized one), followed by an optional scenario title. Each feature can have one or more scenarios, and every scenario consists of one or more steps.

The following scenarios each have 3 steps:

Scenario: Wilson posts to his own blog
  Given I am logged in as Wilson
  When I try to post to "Expensive Therapy"
  Then I should see "Your article was published."
Scenario: Wilson fails to post to somebody else's blog
  Given I am logged in as Wilson
  When I try to post to "Greg's anti-tax rants"
  Then I should see "Hey! That's not your blog!"
Scenario: Greg posts to a client's blog
  Given I am logged in as Greg
  When I try to post to "Expensive Therapy"
  Then I should see "Your article was published."

Scenario outlines

Copying and pasting scenarios to use different values can quickly become tedious and repetitive:

Scenario: Eat 5 out of 12
  Given there are 12 cucumbers
  When I eat 5 cucumbers
  Then I should have 7 cucumbers
Scenario: Eat 5 out of 20
  Given there are 20 cucumbers
  When I eat 5 cucumbers
  Then I should have 15 cucumbers

Scenario Outlines allow us to more concisely express these examples through the use of a template with placeholders:

Scenario Outline: Eating
  Given there are <start> cucumbers
  When I eat <eat> cucumbers
  Then I should have <left> cucumbers
  Examples:
    | start | eat | left |
    |  12   |  5  |  7   |
    |  20   |  5  |  15  |

The Scenario outline steps provide a template which is never directly run. A Scenario Outline is run once for each row in the Examples section beneath it (not counting the first row of column headers).

The Scenario Outline uses placeholders, which are contained within < > in the Scenario Outline’s steps. For example:

Given <I'm a placeholder and I'm ok>

Think of a placeholder like a variable. It is replaced with a real value from the Examples: table row, where the text between the placeholder angle brackets matches that of the table column header. The value substituted for the placeholder changes with each subsequent run of the Scenario Outline, until the end of the Examples table is reached.

You can also use placeholders in Multiline Arguments.

Your step definitions will never have to match the placeholder text itself, but rather the values replacing the placeholder.

So when running the first row of our example:

Scenario Outline: controlling order
  Given there are <start> cucumbers
  When I eat <eat> cucumbers
  Then I should have <left> cucumbers
  Examples:
    | start | eat | left |
    |  12   |  5  |  7   |

The scenario that is actually run is:

Scenario Outline: controlling order
  # <start> replaced with 12:
  Given there are 12 cucumbers
  # <eat> replaced with 5:
  When I eat 5 cucumbers
  # <left> replaced with 7:
  Then I should have 7 cucumbers

Backgrounds

Backgrounds allows you to add some context to all scenarios in a single feature. A Background is like an untitled scenario, containing a number of steps. The difference is when it is run: the background is run before each of your scenarios, but after your BeforeScenario hooks.

Feature: Multiple site support
  Background:
    Given a global administrator named "Greg"
    And a blog named "Greg's anti-tax rants"
    And a customer named "Wilson"
    And a blog named "Expensive Therapy" owned by "Wilson"
  Scenario: Wilson posts to his own blog
    Given I am logged in as Wilson
    When I try to post to "Expensive Therapy"
    Then I should see "Your article was published."
  Scenario: Greg posts to a client's blog
    Given I am logged in as Greg
    When I try to post to "Expensive Therapy"
    Then I should see "Your article was published."

For more information take a look at the following sites, from where i summarized the explanation of feature files.

Behavior Driven Development - behave 1.2.5 documentation

http://docs.behat.org/en/v2.5/guides/1.gherkin.html

Possible file structure

I’m using this file structure for my automated UI tests.

1. All your feature files go here
2. All your screenshots go to its platform folder
3. All your test steps written in python go here

Part 4: Write python files

First you need PyAutoGui

PyAutoGUI lets your Python scripts control the mouse and keyboard to automate interactions with other applications. The API is designed to be as simple. PyAutoGUI works on Windows, macOS, and Linux, and runs on Python 2 and 3.

Welcome to PyAutoGUI's documentation! - PyAutoGUI documentation

Behave

First you need behave

Installation - behave 1.2.7.dev1 documentation

To transform your feature files to python steps just run behave.

I highly reccomend the offical behave tutorial, which you can find below:

Tutorial - behave 1.2.7.dev1 documentation