Writing Good Acceptance Tests

Acceptance tests can be used to verify end to end functionality in a way that is understandable to the business. They are very broad compared with unit tests, and when automated via a tool like Specflow, are based on a more natural language. It's important that we use this natural language to keep tests simple and concise, rather than trying to write long-winded "functional" tests.

Good Gherkin

Before writing Acceptance tests, learn about Acceptance Criteria Best Practices. The ability to write good Acceptance Criteria translates directly to Acceptance testing. They both follow the same standards and they can be written in the same way.

We should attempt to apply these best practices apply to Acceptance tests as much as is practically possible, especially when it comes to grammar as this helps us stick to a standard.

The most important thing is to remember that Acceptance tests should be written with the Given, When, Then syntax where:

• Given sets up a state,
• When is an action,
• Then is an assertion.

Every test should have an action and an assertion, and most require some kind of prerequisites. This ensures that the outcome of actions are actually being tested, and maps directly to a test case.

Let's reiterate a couple of points.

Given should be used to set up pre-requisites. It should not be used for actions, for example "Given I navigate to the admin page". You can however say "Given I am on the admin page" which achieves the same thing but keeps the structure clearer.

Likewise, Then should not be used for actions, such as "Then I delete a record". Then is for assertions, for checking the state of something that was changed by the "When" directive.

Remember to be careful when using AND that you don't break these rules. For example, don't say

"WHEN I delete a record THEN I should have a record AND I click exit"

In this case, the AND is now a "When", not a "Then". In other words, AND should be the same as the preceding directive. In the above example, as it is not an assertion, it should be a "When", and you probably want to follow it with another assertion "Then".

It might sound like I'm picking on grammar here but the point is that the 3 directives serve different purposes - the point of a test is to ensure that an action has an outcome. Mixing up their use means that you may not be creating an actual test case, but rather running through a series of steps, and you may not even have any assertions.

Focus on business actions rather than functional sequences

Practically speaking, in many Acceptance tests there are usually a lot of When steps, or actions performed. However, where possible we should strive to make all End to End code as concise as possible. Writing Specflow/Cypress that does several steps on one line is OK if each of those steps make up a single functional action.

For example, rather than 

When I populate the form with the following values: | Field | Value | And I click the "Save Changes" button

We could just say

When I save the following values:| Field | Value |

and have the underlying code also click the save button/save the changes.

The best way to think about this is we should be more concerned with business actions, or feature functionality, than stepping through a sequence of button clicks. 

Avoid multiple When/Then repetitions

One test, one behaviour, where possible. Any more and you're really testing multiple criteria, and each test should be testing one scenario (hence the Specflow keyword "Scenario").

Of course with Acceptance tests having such a high overhead, you might still want to do multiple assertions from time to time, but use it sparingly and be aware of it. 

Use Background steps

A background step happens before every Scenario in a feature file. Use it for setting up data. Then your tests can be clean and start with a simple "Given I am on the Search screen".

If you need each scenario to have different data you can put the data setup Given steps in each test, but in this case you might want to consider why there is different data for each test, and perhaps you need to move the tests into a different feature file.

Avoid testing logic

It can be tempting to test lots of different scenarios with Acceptance tests. I am speaking from extensive experience when I say: this will make your life very difficult. Acceptance tests have high overhead and can be extremely flaky.

Acceptance tests are for testing high level End to End functionality. They should not be used for testing intricate business logic. This should be done by Unit and Integration tests.

If it is too hard to write Unit/Integration tests, you may need to look at making your code more testable and addressing technical debt.

A large part of avoiding testing the logic comes down to testers and developers working together. When this happens, you can avoid a situation where the testers may write a dozen Acceptance tests which just test the same code path with different variables - not valuable from a quality perspective and also adding a huge unnecessary overhead to the testing time and maintenance. 

When working together, developers and testers can come up with a strategy where the logic and edge cases are covered by unit/integration tests, while the basic Acceptance Criteria/end to end connectivity/functionality can be verified by a small number of Acceptance tests. 

Don't write too many Acceptance tests

Acceptance tests are known for being flaky and unreliable. They can also be hard to maintain. Keeping the number of tests low makes them easier to manage. 

Acceptance tests can provide tremendous value by verifying End to End functionality, but people need to be able to trust them, so keeping them easy to maintain is key to keeping them reliable.