Showing posts with label Acceptance Critiera. Show all posts
Showing posts with label Acceptance Critiera. Show all posts

Friday, October 01, 2021

Writing Good Acceptance Criteria

As discussed in my last article, Writing good Acceptance Criteria gives the Agile team clarity and helps them work towards a common goal, saving time and effort while increasing quality. Now I'd like to dig into the details of how to write good Acceptance Criteria.

Behaviour Driven vs Procedural Driven Mindsets

Many of us may be used to writing/seeing procedural driven tests; step by step instructions with actions and expected results. Procedure-driven tests are often imperative and trace a path through the system that covers multiple behaviors. As a result, they may be unnecessarily long, which can delay failure investigation, increase maintenance costs, and create confusion.

Acceptance criteria, and indeed Behaviour Driven tests, should instead be behaviour driven.

This means they should be Declarative, specifying how a system should behave in a particular scenario.

Procedural/Imperative:

Given the user opens a web browser
And the user navigates to "https://localhost/mysite/"

Behaviour Driven/Declarative:

Given a web browser is at the local home page

As a result of this declarative mindset, a behaviour should have a one to one relationship with scenarios, meaning we don't have multiple expectations of behaviour from a single scenario, which would be easy to do with the procedural mindset:

// BAD EXAMPLE - covers 2 scenarios

When the user enters "cats" into the search bar
Then links related to "cats" are shown on the results page
When the user clicks on the "Images" link at the top of the results page
Then images related to "cats" are shown on the results page

Given, When Then

Acceptance Criteria should be written in the Given, When Then format. The reason for this is that each of these steps has a specific purpose. Respect the order of these, if you move them around, the criteria may not be clear.

Given

Given defines the prerequisites for the scenario, which screen we are on, and which data already exists. As it sets up state, it should be written in the present perfect tense, eg:

Bad: Given the Administrator navigates to the User Search page

Good: Given the Administrator is on the User Search page

When

When indicates the action being performed. It should therefore be written in the present tense, eg:

Bad: When the Administrator filtered the Users by name "Smith"

Good: When the Administrator filters the Users by name "Smith"

Then

Then is the assertion, it verifies the new state that is a result of the When action.These are better written in the present tense, because using the future tense reinforces a procedure driven approach, treating the scenario as a time sequence.

Bad: Then users containing the name "Smith" will be shown on the results page

Good: Then users containing the name "Smith" are shown on the results page

Write full sentences

Subject - Predicate - Action

It may tempting to leave parts of speech out of a step line for brevity, especially when using Ands and Buts, but partial phrases make steps ambiguous. For example, consider the following:

// Bad Example 

Given the user navigates to the Google home page
When the user entered "cats" at the search bar
Then the results page shows links related to "cats"
And image links for "cats"
And video links for "cats"

Writing steps without a clear subject and predicate is not only poor English but poor communication.

Test Cases

Acceptance Criteria should cover the main scenarios which will be faced by the new functionality. They are not a full breakdown of test cases. Developers writing unit tests can cover edge cases and paths through the code, while QA can apply a broader, more exploratory approach to the new feature.

If you find yourself creating lots of different Acceptance Criteria, it might mean that the job needs to be broken down.

Summary

The Given, When, Then format may seem verbose, and all this talk of correct grammar might sound scary, but writing Acceptance Criteria this way is quite easy. Think of each Acceptance Criteria as an action with an expected result - and then add prerequisites (the "Given") only if required.

Use of the correct tenses is something that just comes with practice and is not super important, it just helps to prevent you falling back into the procedural mindset. It's more important to write full sentences and follow the Given/When/Then, or Prerequisites/Action/Expectation format.

Following this format sets a standard which all jobs can follow, bringing consistency and clarity to tasks. It ensures that the required expectations are met, and gets the team thinking about all the actions that need to be performed.

Thursday, September 23, 2021

The Importance of Good Acceptance Criteria

Acceptance Criteria can be seen as a chore, a convoluted, verbose set of prose with the sole purpose of satisfying "the business". At best, its value is underestimated and it is often written as a vague list of requirements.

Given the right attention, Acceptance Criteria can be extremely valuable to all members of a Scrum team. Before it can be done correctly, we have to understand its value.

So what exactly is the point of Acceptance Criteria?

Purpose of Acceptance Criteria

To define boundaries

Acceptance criteria help development teams define the boundaries of a user story. In other words, acceptance criteria help you confirm when the application functions as desired.

To reach consensus

Having acceptance criteria synchronises the development team with the client. The team knows exactly what conditions should be met, just as the client knows exactly what to expect from the developed functionality.

To allow for accurate planning and estimation

Acceptance criteria scenarios allow for the correct division of user stories into tasks so user stories are correctly estimated and planned.

To serve as a basis for tests

Acceptance criteria provide a good baseline for writing tests. Test cases can break down Acceptance Criteria to go into more functional detail, and to cover edge cases.

Acceptance Criteria helps the team deliver quality

Acceptance Criteria is vital for delivering quality. If done correctly, it can simplify development and guide a comprehensive testing strategy. It is the glue that holds requirements together.

Acceptance Criteria is valuable to all stakeholders; Developers, QA, Product Owners, and the team as a whole. How it helps each stakeholder, however, varies according to their role.

Good Acceptance Criteria

How it benefits Developers

Good Acceptance Criteria gives developers clarity and allows them to break down their tasks into manageable pieces of functionality. With visibility of all possible scenarios, good Acceptance Criteria actually allows optimum design of the code. It enables developers to structure their code in a way that best meets the requirements. The code can be written around each scenario, ensuring each criteria is met in the simplest way possible.

The clarity given by good Acceptance Criteria keeps the business requirements at the forefront of the developer's mind, helping them to avoid getting bogged down in technical details which may not be conducive to the task.

How it benefits QA

Good Acceptance Criteria can guide QA in generating test cases and any required test data. With clear, simple scenarios, QA will have a solid base for test cases, and can then focus on expanding the Acceptance Criteria to cover edge cases.

Without clear, broken down Acceptance Criteria, it can be hard to work out test cases. If this happens, certain scenarios might be missed, and the important expectations of the job might be overlooked.

How it benefits the team

Good Acceptance Criteria helps get all the team on the same page. It aligns everyone with common goals, and helps them work together to build out the requirements. This collaboration is important and often overlooked.

Development and QA can come together at the start of a task in order to work out what needs to be tested. This allows Development to perform some degree of Test Driven Development. Even if it isn't possible to write automated tests before the code, the developer will at least have a good idea of what those tests are going to be before writing any code.

QA can guide this process, elaborating on what they will be testing, helping to set up test data, and informing the developer of possible edge cases to watch out for.

If the Acceptance Criteria is missing anything, the Product Owner can be questioned for more details before the work commences. Without clear Acceptance Criteria, there is no way of knowing if there are any missing details.

Good Acceptance Criteria can make a difference

With good Acceptance Criteria, the team is able to see, at a glance, what is required from a task. It provides a consensus for the whole team which aids both planning and delivery.

Good Acceptance Criteria helps developers come up with cleaner code. It makes testers' lives much easier. It helps the team come up with the best Automated Testing solutions, which is vital for Agile projects. It helps the team plan the best possible approach, and avoids stumbling into questions mid-development.

Don't underestimate the value of good Acceptance Criteria. It can increase productivity, collaboration, and quality.

Tuesday, November 24, 2020

Writing Good Acceptance Tests


Acceptance tests can be used to verify end to end functionality in a way that is understandable to the business. They are very broad compared with unit tests, and when automated via a tool like Specflow, are based on a more natural language. It's important that we use this natural language to keep tests simple and concise, rather than trying to write long-winded "functional" tests.

Good Gherkin

Before writing Acceptance tests, learn about Acceptance Criteria Best Practices. The ability to write good Acceptance Criteria translates directly to Acceptance testing. They both follow the same standards and they can be written in the same way.

We should attempt to apply these best practices apply to Acceptance tests as much as is practically possible, especially when it comes to grammar as this helps us stick to a standard.

The most important thing is to remember that Acceptance tests should be written with the Given, When, Then syntax where:
  • Given sets up a state,
  • When is an action,
  • Then is an assertion.
Every test should have an action and an assertion, and most require some kind of prerequisites. This ensures that the outcome of actions are actually being tested, and maps directly to a test case.

Let's reiterate a couple of points.

Given should be used to set up pre-requisites. It should not be used for actions, for example "Given I navigate to the admin page". You can however say "Given I am on the admin page" which achieves the same thing but keeps the structure clearer.

Likewise, Then should not be used for actions, such as "Then I delete a record". Then is for assertions, for checking the state of something that was changed by the "When" directive.

Remember to be careful when using AND that you don't break these rules. For example, don't say

"WHEN I delete a record THEN I should have a record AND I click exit"

In this case, the AND is now a "When", not a "Then". In other words, AND should be the same as the preceding directive. In the above example, as it is not an assertion, it should be a "When", and you probably want to follow it with another assertion "Then".

It might sound like I'm picking on grammar here but the point is that the 3 directives serve different purposes - the point of a test is to ensure that an action has an outcome. Mixing up their use means that you may not be creating an actual test case, but rather running through a series of steps, and you may not even have any assertions.

Focus on business actions rather than functional sequences

Practically speaking, in many Acceptance tests there are usually a lot of When steps, or actions performed. However, where possible we should strive to make all End to End code as concise as possible. Writing Specflow/Cypress that does several steps on one line is OK if each of those steps make up a single functional action.

For example, rather than 
When I populate the form with the following values: | Field | Value | And I click the "Save Changes" button

We could just say
When I save the following values:| Field | Value |

and have the underlying code also click the save button/save the changes.

The best way to think about this is we should be more concerned with business actions, or feature functionality, than stepping through a sequence of button clicks. 

Avoid multiple When/Then repetitions

One test, one behaviour, where possible. Any more and you're really testing multiple criteria, and each test should be testing one scenario (hence the Specflow keyword "Scenario").

Of course with Acceptance tests having such a high overhead, you might still want to do multiple assertions from time to time, but use it sparingly and be aware of it. 

Use Background steps

A background step happens before every Scenario in a feature file. Use it for setting up data. Then your tests can be clean and start with a simple "Given I am on the Search screen".

If you need each scenario to have different data you can put the data setup Given steps in each test, but in this case you might want to consider why there is different data for each test, and perhaps you need to move the tests into a different feature file.

Avoid testing logic

It can be tempting to test lots of different scenarios with Acceptance tests. I am speaking from extensive experience when I say: this will make your life very difficult. Acceptance tests have high overhead and can be extremely flaky.

Acceptance tests are for testing high level End to End functionality. They should not be used for testing intricate business logic. This should be done by Unit and Integration tests.

If it is too hard to write Unit/Integration tests, you may need to look at making your code more testable and addressing technical debt.

A large part of avoiding testing the logic comes down to testers and developers working together. When this happens, you can avoid a situation where the testers may write a dozen Acceptance tests which just test the same code path with different variables - not valuable from a quality perspective and also adding a huge unnecessary overhead to the testing time and maintenance. 

When working together, developers and testers can come up with a strategy where the logic and edge cases are covered by unit/integration tests, while the basic Acceptance Criteria/end to end connectivity/functionality can be verified by a small number of Acceptance tests. 

Don't write too many Acceptance tests

Acceptance tests are known for being flaky and unreliable. They can also be hard to maintain. Keeping the number of tests low makes them easier to manage. 

Acceptance tests can provide tremendous value by verifying End to End functionality, but people need to be able to trust them, so keeping them easy to maintain is key to keeping them reliable.