Azure Devops Test Plans

Azure Devops

Document Source

Building Your Azure DevOps Test Plans as a Business Analyst or Test Lead

Building your documents for your Azure DevOps Test Plans as a Business Analyst or Test Lead can be a daunting task. The same thing is true for requirements that live within your Azure DevOps project, but Azure Test Plans pose a unique challenge.

In this article, you get a brief overview of what the Azure DevOps Test Plans module is, how it works, and how you can document your test outcomes easily. You will also learn about the architecture of Azure DevOps itself.

What Is Azure DevOps Test Plans?

Azure Test Plans is a test management module within Azure DevOps that lets users manage test plans, test suites, and test cases for everyone in the software development process. Using test plans, you can Azure Test Plans also provides a browser extension for exploratory testing and gathering feedback from stakeholders.

Azure DevOps (ADO) is a great place for your Quality Assurance team to build out their Test Plans for a given project. By bringing your Testing Team onto the Azure DevOps platform, you can effectively use your ADO project as your single source of truth.

You can have your developers using the repos and tying development work to requirements, you can have your business analysis teams building requirements and attaching test cases, and you can have your QA team using those Test Cases, and requirements made by the BA team, to run all of your Testing.

Now Azure DevOps Test Plans can be used for both automated and manual testing. But this article only focuses on manual testing on ADO test plans.

Test Plans is an incredible place for your team to do their manual testing needs for many reasons:

Stakeholder Feedback

The biggest advantage of Test Plans is letting your testing teamwork with your Business Analysis and development teams in the same environment. By doing this, you create a single source of truth for everyone working on a project.

By deploying your manual testing operations from here, you can directly interact with the Test Cases that were already built inside of your Azure DevOps project. So the contributions of your Business Analysis team when building test cases will easily be pulled into your testing workflow. This capability adds a great benefit for teams who have BAs that are capable of building Test Cases for the QA team – if you’re a BA who does this, you will see how it works later in the article.

Robust Manual Testing Features

Test Plans also offers some incredibly helpful manual testing features to help your testers provide meaningful input on the outcome or results of their tests beyond simply marking a specific test step as passed or failed. It also provides an interactive window for both marking the outcome of individual test steps, capturing live screenshots/recordings, and providing comments tied directly to the run of that test case.

So, when you review any Test Runs that you have completed, you can identify what your tester was experiencing when they marked an individual test step as blocked or failed. This level of insight means that even an external manual testing team can contribute thoughtfully and allow teams to identify and interpret failed results easily.

Easy Extensibility

With the right test case management tools, you can generate Smart Reports and Horizontal Traceability Matrices on the details of Test Plans, Test Suites, Test Cases, Test Runs, Test Case Runs, and Test Case Step Runs.

Tools like this can change the functionality of Test Plans to reflect how companies manage test cases through virtual linking.

How Does Test Plans Work?

Every month, Google gets thousands of searches asking how ADO testing works. One reason may be that most teams use proprietary, in-house solutions that are built for testing that are separate from ADO Test Plans.

Before learning how ADO Test Plans works, it’s worth knowing the relevant terminology. Azure Test Plans gives you three main types of test management artifacts:

  • Test Plan: It is a container made of configurations, test suites, and test cases which you can divide into shared test steps and use parameters. You can group test suites and individual test cases together. Test plans include static test suites, requirement-based suites, and query-based suites.

  • Test Suite: It is a group of test cases that investigate distinct scenarios under an overarching test plan. Grouping cases makes it easier to see which scenarios are complete.

  • Test Case: It is a step or series of steps to validate individual parts of your code or app deployment. Using a test case, you can check if your code works correctly and meets business and customer needs. You can create test cases under a test plan without needing to create a test suite.

Sometimes the terminology can be confusing, as many teams say “test plan” when they mean “test suite.”

If your team creates a Test Suite, they must choose between three different kinds of test suites, as shown below:

  • Test Suite 1: Requirements-based test suites are the simplest and most traceable. They pull in all of the Test Cases for a given Requirement.

  • Test Suite 2: Query-based test suites pull in a group of tests from your project, irrespective of what requirements the test cases are linked to.

  • Test Suite 3: Static-based test suites are used either as containers to group other test suites or to group a specific set of test cases.

Depending on which Test Suite you create, building a Test Suite allows you to pull in all the Test Cases that you want to test. Once you create your Test Suite with their respective Test Cases, you can execute the entire Suite, which is known as a Test Run. Before beginning your Test Run, you must choose between three different kinds of Test Suites to execute: Requirements-based, Query-based, and Static.

How Do I Choose a Test Suite Type?

Before starting your Test Run, you must choose between three different kinds of Test Suites to execute: Requirements-based, Query-based, and Static. Remember: Test Suites (of any type) can only include Test Cases.

  1. Requirements-based Test Suite

    • A Requirements-based Test Suite is one in which you associate your Test Cases to a requirement to define its acceptance criteria. Creating a Requirements-based Test Suite will pull in all of the Test Cases for that requirement. When building a suite, you automatically import all its associated Test Cases. You will instead simply choose the Requirement.
    • You can then run all the Tests for that given requirement and see from each Test Run and conclude if the product you’re developing fully meets that requirement.
    • As creating a suite automatically pulls in all the associated tests, your QA team doesn’t need to redo the work of populating all requirements. If your BAs and developers added the Test Cases for a requirement when they created it, the QAs can use it. This is the benefit of a single source of truth model. Your teams can benefit from each other’s work.

  2. Query-based Test Suite

    • When you make a Work Item query and select which test cases to include into your suite, you create a Query-based Suite. Azure DevOps automatically adds all test cases that meet these criteria to the suite.
    • When do you use a Query-based Test Suite? In some cases, teams run tests on all the Test Cases that are currently in each iteration or run tests on everything with a given tag, or some other criteria. These conditions are not necessarily tied directly to a given requirement, and you can only satisfy them through a specific query.

  3. Static Test Suite

    • Static suites are logical containers where you can add any tests you like. They aren’t connected to any requirements or queries.
    • They are useful when you may need to create multiple levels of nested Test Suites or when you need a place to create and experiment with ad-hoc tests.

How to Build a Test Plan

Test Plans help your team to structure how tests are run so that your project fulfills the quality standards. You will create both Test Plans and Test Suites for all the requirements for every iteration below.

Step 1:

To build a Test Plan called “Given Iteration Test Plan,” go to the Test Plan tab on the left panel of your Azure DevOps interface. Select “+ New Test Plan.”

Name this Test Plan “Test Plan for Iteration 1.”

Select if you would like this specific Test Plan to be for a given area path or iteration. Otherwise, you can leave this blank (in these example images those two fields are left blank as the default options.)

With the test plans created, it’s now time to create a suite.

Step 2: Add Test Suites

Since the aim is to test all the requirements from a given iteration of a product, you will add Requirements-based Suites.

When creating a Requirements-based Suite, you will pull in Test Cases that already exist for the given requirements. Otherwise, you will have to add new Test Cases to a requirement before adding it to a Requirements-based Test Suite.

We don’t want to bring in requirements; we want to bring in the Test cases associated with requirements. This point is just a reminder that even when you build a Requirement-based Suite, you are creating a group of Test Cases, not requirements.

So, let’s create Test Suites (groups of Test Cases) for each of the requirements in iteration 1.

Step 3: Run Query

Above you can see the query you would need to build to pull in the Iteration 1 requirements, then you can simply click “Run Query” as shown above.

When you run the query and see your Iteration 1 Work Items, select a few.

Since each Work Item above is a functional requirement and you have chosen to create Requirements-based Suites, you will create four Test Suites, one for each requirement.

Let’s look at the outcome of clicking the blue “Create Suites” button above.

In the image above, you see that Test Suites for Work Items 306 and 307 have numbers in parentheses next to them. They represent the number of Test Cases they contain, one and three respectively.

Running Your First Test Plan

Once you create your Test Plan and its respective Test Suites, you’re ready to run tests.

In the image below, you can see that we have several Test Cases available in our Requirement-based Test Suite. The next steps involve selecting the specific Test Cases to run:

  • Select the test cases you would like to run.
  • Use either the Azure DevOps Test Plans interface or a browser extension for exploratory testing to execute them.
  • Document the outcome (pass/fail) of each test case along with any relevant observations (blocker, etc.) for each test step, including screenshots or recordings.

Documenting Your Test Plan

Maintaining detailed records of test results in Azure DevOps is critical. You can achieve this by:

  • Completing a Test Run and capturing results as you progress.
  • Using the Results tab to review outcomes, including visual feedback, to help the team understand the quality of the application being tested.

Conclusion

Azure DevOps Test Plans is a powerful tool for managing and executing manual tests, fostering collaboration among teams, and ensuring that software products meet quality standards. By effectively utilizing its features, teams can streamline their testing processes, enhance communication, and achieve better alignment with project requirements.