Contents

Azure Test Plans

What is Azure Test Plans?
Get started
Navigate Test Plans
Test objects and terms
Create test plans and test suites
Create test cases
Share steps between test cases
Copy/clone test plans, suites, cases
Run manual tests
Default permissions (Security)
Manual tests
Test different configurations
Repeat a test with different data
Test from the Kanban board
Perform user acceptance testing
Collect diagnostic data
Delete test artifacts
Set permissions and access
Set test retention policies
Automated tests
Associate automated tests with test cases
Run automated tests from test plans
Request & track feedback
About feedback
Test & Feedback extension
Request stakeholder feedback
Provide stakeholder feedback
Get insights from tests
 Install the extension
Test in Connected mode
Test in Standalone mode
Explore work items
Add to existing bugs
Provide feedback without a request
Track stakeholder feedback requests
Microsoft Feedback client
Request feedback
Provide feedback
Track feedback requests
Set feedback permissions (Security)
Enable remote audio capture
Change the audio device or annotation tool
Track progress and results
Track test status
Progress report
Pipelines & test results
Code coverage
Pipeline Analytics
Test Analytics
Build and release test results
Tests and Requirements traceability
Pool consumption report
Configure a Test Results Trend widget
Test Impact Analysis (TIA)
Flaky test management and reporting
Troubleshooting
Manual testing FAQs
Unable to connect
Reference
Test case management commands
 Work item field index
Build and test integration fields
Microsoft Test Manager (deprecated)
REST API Reference
Test
Test Plans
Test Suites
Test Variables
Test Results
Resources
Azure Load Test Preview
Azure Pipelines
Azure Boards
Marketplace Test Plan extensions
Marketplace Visual Studio test extensions
Azure DevOps Blog
 What is Azure Test Plans?
6/27/2022 • 16 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Azure Test Plans provides rich and powerful tools everyone in the team can use to drive quality and
collaboration throughout the development process. The easy-to-use, browser-based test management solution
provides all the capabilities required for planned manual testing, user acceptance testing, exploratory testing,
and gathering feedback from stakeholders.

NOTE
This article applies to Azure DevOps Services and Azure DevOps Server 2020 and later versions. Most of the information
is valid for earlier on-premises versions, however, images show only examples for the latest version. Also, the user
interface changed significantly with the release of Azure DevOps Server 2020. For an overview of the new interface and
supported capabilities, see Navigate Test Plans.

How does Azure Test Plans work?

Through a combination of browser-based tools—Test plans , Progress repor t , Parameters , Configurations ,
Runs , and Test tools—and DevOps integration features, Azure Test Plans supports the following test objectives:
Manual and explorator y testing : Manual and exploratory testing which includes the following test
activities:
 Planned manual testing . Manual testing by organizing tests into test plans and test suites by
designated testers and test leads.
User acceptance testing . Testing carried out by designated user acceptance testers to verify the
value delivered meets customer requirements, while reusing the test artifacts created by engineering
teams.
Explorator y testing . Testing carried out by development teams, including developers, testers, UX
teams, product owners and more, by exploring the software systems without using test plans or test
suites.
Stakeholder feedback . Testing carried out by stakeholders outside the development team, such as
users from marketing and sales divisions.
Automated testing : Azure Test Plans is fully integrated with Azure Pipelines to support testing within
continuous integration/continuous deployment (CI/CD). Test plans and test cases can be associated with
build or release pipelines. Pipeline tasks can be added to pipeline definitions to capture and publish test
results. Test results can be reviewed via built in progress reports and pipeline test reports.
Traceability : Test cases and test suites linked to user stories, features, or requirements supports end-to-
end traceability. Tests and defects are automatically linked to the requirements and builds being tested,
which also helps tracking the quality of requirements. Users can add and run tests from the Kanban
board, or for larger teams, use the Test plans hub to define test plans and test suites. Pipeline results and
the Requirements widget provide a means to track testing of requirements.
Repor ting and analysis : Test result tracking and progress monitoring is supported through
configurable tracking charts, test-specific widgets that you can add to dashboards, and built-in reports,
such as Progress report, pipeline test result reports, and the Analytics service.

NOTE
Load and performance testing : While Azure DevOps cloud-based load testing service is deprecated, Azure Load
Testing Preview is available. Azure Load Testing Preview is a fully managed load testing service that enables you to use
existing Apache JMeter scripts to generate high-scale load. To learn more, see What is Azure Load Testing Preview?. For
more information about the deprecation of Azure DevOps load testing, see Changes to load test functionality in Visual
Studio and cloud load testing in Azure DevOps.

Key benefits
Azure Test Plans provides software development teams the following benefits.
Test on any platform : With the Test Plans web portal, you can use any supported browser to access all
the manual testing capabilities. It enables you to create and run manual tests through an easy-to-use,
browser-based interface that users can access from all major browsers on any platform.
Rich diagnostic data collection : Using the web-based Test Runner and Test Runner client you can
collect rich diagnostic data during your manual tests. This includes screenshots, an image action log,
screen recordings, code coverage, IntelliTrace traces, and test impact data for your apps under test. This
data is automatically included in all the bugs you create during test, making it easy for developers to
reproduce the issues.
End to End traceability : Azure DevOps provides end-to-end traceability of your requirements, builds,
tests and bugs. Users can track their requirement quality from cards on the Kanban board. Bugs created
while testing are automatically linked to the requirements and builds being tested, which helps you track
the quality of the requirements or builds.
Integrated analytics : The Analytics service provides data that feeds into built-in reports, configurable
dashboard widgets, and customizable reports using Power BI. Data tracks test plan progress and trends
for both manual and automated tests. Test analytics provides near real-time visibility into test data for
 builds and releases. Teams can act on this data to improve test collateral to help maintain healthy
pipelines.
Extensible platform . You can combine the tools and technologies you already know with the
development tools that work best for you to integrate with and extend Azure DevOps. Use the REST APIs
and contribution model available for the Test platform to create extensions that provide the experience
you need for your test management lifecycle.
Supported scenarios and access requirements
Access to Azure DevOps web portal features are managed through access levels assigned to users. The three
main access levels are Stakeholder , Basic , and Basic+Test plans as described in About access levels. The
following table indicates the access-level required to exercise the associated tasks with Azure Test Plans. In
addition to access levels, select features require permissions to execute. To learn more, see Manual test access
and permissions.
Scenario and tasks
Stakeholder
Basic
Basic +Test Plans

Test planning
Create test plans and test suites
Manage test plan run settings
Manage configurations
️
✔
Test execution
Run tests on any platform (Windows, Linux, Mac) with Test Runner
️
✔
️
✔
Perform explorator y testing with the Test & Feedback extension
️
✔
️
✔
️
✔
Analyze and review tests
Create charts with various pivots like priority, configuration, etc., to track test progress
Browse test results
Export test plans and test suites for review
User Acceptance Testing – Assign tests and invite by email
️
✔
️
✔

Manual and exploratory testing

To support manual and exploratory testing, Azure Test Plans uses test-specific work item types to plan and
author tests. In addition, it provides two test tools to support running tests. The Test plans , Parameters , and
Configurations hubs provide the tools to efficiently create and manage test items, their settings, and
configurations. Test suites can be dynamic—requirements-based-suites and query-based-suites—to help you
understand the quality of associated requirements under development, or static to help you cover regression
tests.
Test-specific work item types
The work item types—Test Plans , Test Suites , Test Cases , Shared Steps , and Shared Parameters —support
several explicit links to support requirements tracking and sharing test steps and data across many test cases.
Test cases can be assigned as manual or automated. For a description of each of these test items, see Test objects
and terms.

NOTE
With Azure DevOps Server 2020 and later versions, you can perform automated tests by adding test tasks to pipelines.
Defining test plans, test cases, and test suites isn't required when test tasks are used.

Define test plans and test suites

You create and manage test plans and test suites from the Test plans hub. Add one or more test suites—static,
requirement-based, or query-based—to the test plans. Export and share test plans and test suites with your
teams. To learn how, see Create test plans and test suites and Copy or clone test plans, test suites, and test cases.
Author tests using test cases
You define manual test cases by defining the test steps and optionally the test data to reference. Test suites
consist of one or more test cases. You can share test cases within test suites. The Grid view for defining test cases
supports copy, paste, insert, and delete operations. Quickly assign single or multiple testers to execute tests.
View test results and references to a test case across test suites. To learn how, see Create test cases.

Within each test case, you specify a set of test steps with their expected outcomes. Optionally, you can add
shared steps or shared parameters. For traceability, link test cases to the user stories, features, or bugs that they
test.

Manage shared parameters

Teams use the Parameters hub, to define and manage parameters shared across test cases. Shared parameters
provide support for repeating manual tests several times with different test data. For example, if your users can
add different quantities of a product to a shopping cart, then you want to check that a quantity of 200 works just
as well as a quantity of 1.

Manage test configurations and variables

With the Configurations hub, teams can define, review, and manage test configurations and variables referenced
by test plans. Test configurations provide support for testing your applications on different operating systems,
web browsers, and versions. As with shared parameters, test configurations can be shared across multiple test
plans.

Test execution and test tools

With the following tools, developers, testers, and stakeholders can initiate tests and capture rich data as they
execute tests and automatically log code defects linked to the tests. Test your application by executing tests
across desktop or web apps.
Test Runner : A browser-based tool for testing web applications and a desktop client version for testing
desktop applications that you launch from the Test plans hub to run manual tests. Test Runner supports rich
data collection while performing tests, such as image action log, video recording, code coverage, etc. It also
 allows users to create bugs and mark the status of tests.
Test & Feedback extension : A free extension to support exploratory testing that you access from Chrome,
Edge, or Firefox browsers. The extension captures interactions with the application being explored through
images or video and entering verbal or type-written comments. Information is captured in the Feedback
Response work item type to help track response data.
Test execution capability
You can perform the following tasks using the indicated tools.

T EST & F EEDB A C K

TA SK T EST P L A N S H UB T EST RUN N ER EXT EN SIO N

Bulk mark tests ️

✔

Pass or fail tests or test ️

✔ ️
✔
steps

Inline changes to tests ️

✔ ️
✔
during execution

Pause and resume tests ️

✔ ️
✔

File bugs during test ️

✔ ️
✔
execution

Capture screenshots, image ️

✔ ️
✔
action log, and screen
recording during test
execution

Update existing bugs ️

✔ ️
✔
during test execution

Verify bugs ️
✔ ️
✔

Assign a build for the test ️

✔
run

Assign test settings ️

✔

Review test runs ️

✔

Execute tests
From the Test plans hub, Execute tab, team members can initiate test execution for one or more test cases
defined for a test suite. Choices include running Test Runner for a web or desktop application. Optionally, team
members can select Run with options to choose other supported clients for manual testing, or to select a
build for automated testing. To learn more, see Run manual tests.
Test Runner
Test Runner runs tests for your web and desktop applications. Mark test steps and test outcomes as pass or
fail, and collect diagnostic data such as system information, image action logs, screen recordings, and screen
captures as you test. Bugs filed during the tests automatically include all captured diagnostic data to help your
developers reproduce the issues. To learn more, see Run tests for web apps and Run tests for desktop apps.

!
User acceptance testing
User acceptance testing (UAT) helps ensure teams deliver the the value requested by customers. You can create
UAT test plans and suites, invite several testers to execute these tests, and monitor test progress and results
using lightweight charts. To learn how, see User acceptance testing.

Exploratory testing with the Test & Feedback extension

The Test & Feedback extension is a simple browser-based extension you can use to test web apps anytime and
anywhere, and is simple enough for everyone in the team to use. It helps to improve productivity by allowing
you to spend more time finding issues, and less time filing them.

Stakeholder feedback
Seeking feedback from stakeholders outside the development team, such as marketing and sales teams, is vital
to develop good quality software. Developers can request feedback on their user stories and features.
Stakeholders can respond to feedback requests using the browser-based Test & Feedback extension - not just to
rate and send comments, but also by capturing rich diagnostic data and filing bugs and tasks directly. See more
at Request stakeholder feedback and Provide stakeholder feedback.
Automated testing
Automated testing is facilitated by running tests within Azure Pipelines. Test analytics provides near real-time
visibility into your test data for builds and releases. It helps improve pipeline efficiency by identifying repetitive,
high impact quality issues.
Azure Test Plans supports automated testing in the following ways:
Associate test plans or test cases with build or release pipelines
Specify test-enable tasks within a pipeline definition. Azure Pipelines provides several tasks, including
those listed below, that support a comprehensive test reporting and analytics experience.
Publish Test Results task: Use to publish test results to Azure Pipelines.
Visual Studio Test task: Use to run unit and functional tests (Selenium, Appium, Coded UI test, and
more) using the Visual Studio Test Runner.
.NET Core CLI task: Use to build, test, package, or publish a dotnet application.
For additional tasks, see Publish Test Results task
Provide built-in reports and configurable dashboard widgets to display results of pipeline testing.
Collect test results and associated test data into the Analytics service.

Traceability
Azure Test Plans supports linking bugs and requirements to test cases and test suites. In addition, the following
web portal, test-related tools support traceability:
View items linked to a test case : View the test plans, test suites, requirements, and bugs that a test case
links to.
Add and run tests from the Kanban board : An Azure Boards feature that supports defining test cases
from the user stories, features, or bugs from the Kanban board. Also, you can launch the Test Runner or the
Test & Feedback extension to run tests or perform exploratory testing.
 Requirements quality widget : Configurable widget used to track quality continuously from a build or
release pipeline. The widget shows the mapping between a requirement and latest test results executed
against that requirement. It provides insights into requirements traceability. For example, requirements not
meeting the quality, requirements not tested, and so on.

View items linked to a test case

From the Test plans hub, you can view and open the test suites, requirements, and bugs linked to a test case.
The Test Suites tab also indicates the test plans and projects that reference the test case. The Requirements
tab lists work items linked to the test case that belong to the requirements category. In addition, you can create a
direct-links query that lists items that link to test cases via the Tests/Tested by link type. To learn more, see
Create test cases and Use direct links to view dependencies.

Add and run tests from the Kanban board

From the Azure Boards Kanban boards, you can add tests from a user story or feature, automatically linking the
test case to the user story or feature. You can view, run, and interact with test cases directly from the Kanban
board, and progressively monitor status directly from the card. Learn more at Add, run, and update inline tests.
Requirements quality widget
The Requirements quality widget displays a list of all the requirements in scope, along with the Pass Rate for
the tests and count of Failed tests. Selecting a Failed test count opens the Tests tab for the selected build or
release. The widget also helps to track the requirements without any associated test(s). To learn more, see
Requirements traceability.

Reporting and analysis

To support reporting and analysis, Azure Test Plans supports test tracking charts, a test Runs hub, several built-
in pipeline test reports, dashboard widgets, and test-data stored in the Analytics service.
Configurable test char ts : You can gain insight into the test plan authoring and execution activity by
creating test tracking charts.
Progress repor t : Track progress of one or test plans or test suites.
Test Runs : Review the results of manual and automated test runs.
Dashboard widgets: Configurable widgets that display test results based on selected builds or releases.
Widgets include the Deployment status widget and the Test Results Trend (Advanced) widget.
Test Analytics: Gain detailed insights from built-in pipeline reports or create custom reports by querying the
 Analytics service.

Configurable test charts

Quickly configure lightweight charts to track your manual test results using the chart types of your choice, and
pin the charts to your dashboard to easily analyze these results. Choose a retention policy to control how long
your manual testing results are retained. See more at Track test status.

Progress reports
With the Progress report hub, teams can track progress of more than one test plan or test suite. This report
helps answer the following questions:
How much testing is complete?
How many tests have passed, failed or are blocked?
Is testing likely to complete in time?
What is the daily rate of execution?
Which test areas need attention?
Test runs
The Runs hub displays the results of test runs. This includes all test runs, both manual and automated.

NOTE
The Runs hub is available with Azure DevOps Server 2020 and later versions. It requires enabling the Analytics service
which is used to store and manage test run data. To learn more about the service, see What is the Analytics service?

Choose any specific run to view a summary of the test run.

Deployment status
The Deployment status widget configurable widget shows a combined view of the deployment status and test
pass rate across multiple environments for a recent set of builds. You configure the widget by specifying a build
pipeline, branch, and linked release pipelines. To view the test summary across multiple environments in a
release, the widget provides a matrix view of each environment and corresponding test pass rate.

Hover over any build summary, and you can view more details, specifically the number of tests passed and
failed.

Test results trend (Advanced)

The Test Results Trend (Advanced) widget provides near real-time visibility into test data for multiple builds and
releases. The widget shows a trend of your test results for selected pipelines. You can use it to track the daily
count of test, pass rate, and test duration. Tracking test quality over time and improving test collateral is key to
maintaining a healthy DevOps pipeline. The widget supports tracking advanced metrics for one or more build
pipelines or release pipelines. The widget also allows filtering of test results by outcome, stacking metrics, and
more. To learn more, see Configure the Test Results Trend (Advanced) widget.
Test Analytics
The built-in tests and test-supported widgets derive their data from the Analytics service. The Analytics service is
the reporting platform for Azure DevOps. Test Analytics data is available for Azure DevOps Server 2019 and
later versions. It supports the Analytics and Tests tab and drill-down reports available from the Pipelines hub.
The Test failure drill down report provides a summary of passed and failing tests. To learn more, see Test
Analytics.

In addition, you can create custom reports by querying the Analytics service. To learn more, see Overview of
sample reports using OData queries.

Next steps
Test objects and terms

Related articles
Navigate Test Plans
Copy or clone test plans, test suites, and test cases
Associate automated tests with test cases
About requesting and providing feedback
Cross-service integration and collaboration overview
Manage a virtual machine in Azure DevTest Labs
About pipeline tests

Additional resources
Unit testing
Unit test basics
Durable Functions unit testing
What is Azure Load Testing Preview?
 Navigate Test Plans
6/27/2022 • 13 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020

Learn how to navigate Test Plans to support your test goals and operations. Use the following sections of the
Test Plans page to accomplish the corresponding tasks.
Mine : View list of test plans that you created or are of interest to you.
All : View a list of all test plans defined for the project.
Test plan header : Locate, favorite, edit, copy or clone a test plan.
Test suites tree : Add, manage, export, or order test suites. Leverage this to also assign configurations and
perform user acceptance testing.
Define tab : Collate, add, and manage test cases in a test suite of choice via this tab. This tab doesn't appear
for users with Basic access.
Execute tab : Assign and execute tests via this tab or locate a test result to drill into.
Char t tab : Track test execution and status via charts, which can also be copied to dashboards.

To learn about specific test artifacts, see the following articles:

Create test plans and test suites
 Create manual test cases
Test settings

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view test-related work items or run test cases, you must have Basic access or higher. Users with
Stakeholder access have no access to Test Plans.
To exercise the full range of test-related features, you must have been granted Basic + Test Plans access level
or have one of the following subscriptions:
Enterprise
Test Professional
MSDN Platforms
To add or edit test-related artifacts, you must have the following permissions:
To add or modify test plans, test suites, test cases, or other test-based work item types, you must have
Edit work items in this node permission set to Allow under the corresponding Area Path .
To modify test plan properties such as build and test settings, you must have Manage test plans
permission set to Allow under the corresponding Area Path .
To create and delete test suites; add, and remove test cases from test suites; change test configurations
associated with test suites—you must have Manage test suites permission set to Allow under the
corresponding Area Path .
Project-level permissions that you may need include:
To view, create, or delete test runs, you must be granted the corresponding permissions: View test
runs , Create test runs , and Delete test runs .
To manage test configurations or test environments, you must be granted the corresponding
permissions: Manage test configurations and Manage test environments .
To add new tags to test-based work items, you must have the Create tag definition permission set
to Allow .
To move test-based work items out of a project, you must have the Move work items out of this
project permission set to Allow .
To delete or restore test-based work items, you must have the Delete and restore work items
permission set to Allow .
To permanently delete test-based work items, you must have the Permanently delete work items
permission set to Allow .
To learn more, see Manual test access and permissions.

NOTE
Most operations under the Define tab are only available to users with Basic + Test Plans access level or equivalent. Most
operations under the Execute tab can be exercised by users granted a Basic access level.

The "Mine" page

The Mine page shows a list of test plans that are of interest to you. This includes plans for teams you are a
member of, and test plans you have marked as favorites. You can expand and hide the plans for each team to
make it easier to locate or review specific team plans. Use the shortcut menu to edit or delete the selected plan.
Show the filter controls to help you find plans if you have a large number of plans in the list. Filter the plans by
name, team, state, or iteration.

The "All" page

The All page shows a list of all test plans. Filter test plans, and edit and delete plans using the shortcut menu, in
the same way as in the Mine page. Use this page to add any test plan to your favorites list.
You can also add a plan to your favorites list while you are viewing it.

Test plan header

Test plan tasks

Use the Test Plan header, to do one of the following tasks:
Mark or unmark a test plan as a favorite
Easily navigate among your favorite test plans
View the iteration path of the test plan, which clearly indicates if the test plan is Current or Past
Navigate to the Test Progress report via the View repor t link
Navigate back to the All/Mine Test Plans page by choosing All test plans .
Test plan context menu options

Choose the More options to open the context menu for the Test Plan to perform one of the following
actions:
Copy test plan : Quickly copy the current test plan.
Edit test plan : Edit the Test Plan work item form to define work item fields or add to the description or
discussion.
Test plan settings : Configure the Test Run settings (to associate build or release pipelines) and the Test
Outcome settings.

Configure test plan settings

Choose Test plan settings to configure test runs or test outcomes.
Copy test plan
We recommend creating a new Test Plan per sprint/release. When doing so, generally you can copy the Test Plan
for the prior cycle and with few changes the copied test plan is ready for the new cycle. Use the Copy test plan
menu option. This option lets you copy or clone test plans within a project.
Test suites header and tree

Test suites header tasks

The Test suite header allows you to perform the following tasks:

Expand/collapse : Choose the Expand All/Collapse All toolbar options to expand or collapse
the suite hierarchy tree.
Show test points from child suites : Choose the Show test points from child suites toolbar
option, which is only visible when you are in the Execute tab, to view all the test points for the given suite
and its children in one view for easier management of test points without having to navigate to individual
suites one at a time.
Order suites : Drag/drop suites to either reorder the hierarchy of suites or move them from one suite
hierarchy to another within the test plan.
Test suites context menu options
Choose the More options to open the context menu for the Test Suite and perform one of the following
actions:
Create new suites : You can create one of three different types of suites:
Choose Static suite to organize your tests within a folder suite.
Choose Requirement-based suite to directly link to the requirements/user stories for seamless
traceability.
Choose Quer y-based suite to dynamically organize test cases that meet a query criteria.
Assign configurations : Assign configurations for the suite. For example, assign Chrome, Firefox,
EdgeChromium, which then become applicable to all existing or new test cases that you add later to the suite.
Expor t : Export the test plan properties, test suite properties along with details of the test cases and test
points as either an email or print to pdf.
Open test suite work item : This option allows you to edit the Test suite work item form to manage the
work item fields.
Assign testers to run all tests : This option is very useful for User Acceptance testing (UAT) scenarios
where the same test needs to be run/executed by multiple testers, generally belonging to different
departments.
Rename/Delete : These options allow you to manage the suite name or remove the suite and its content
 from the test plan.
Impor t test suites : Use this option to import test cases present in other suites from other test plans in the
same project or even across projects. More details below.
Expor t test suite dialog

Impor t test suites

Reuse the suites you have created and import them into the current suite/plan. You can select the Project , Test
Plan , and Test Suite from which you want to import the tests. Depending upon the suite you select, the entire
hierarchy of that suite and corresponding test cases are imported into the current plan. Note that the test cases
are added as a reference and not a clone or [Link] note that we cannot import test suites from the same test
plan.
Define and organize test cases
From the Define tab, you can collate, add, and manage test cases for a test suite. Whereas you use the Execute
tab for assigning test points and executing them.

Define test case tasks

From the Define tab you can exercise the following tasks:
New Test Case :
Add Test Case : Create a new test case using the work item form and automatically add it to the suite.
Add existing test cases : Open a dialog to query for existing test cases that you can select from and
add to the test suite.
Add New test case using grid : Create one or more test cases using the test cases grid view and
automatically add them to the suite.
Order test cases : Reorder test cases by dragging/dropping one or more test cases within a given suite. The
order of test cases only applies to manual test cases and not to automated tests.
Move test cases from one suite to another : Using drag/drop, move test cases from one test suite to
another.
Add existing test cases to a test suite dialog
Define tab toolbar options

From the Define tab, toolbar, you can exercise the following options:
Expor t test cases to CSV : Export existing test cases to a CSV file and make changes using Excel.
Impor t test cases from CSV : Import changes made to test cases from a CSV file.
Grid View : Use the grid mode for viewing or editing multiple test cases along with test steps.
Toggle full screen mode : Expand the view by choosing full screen mode.
Filter : Choose Filter to toggle the filter toolbar to filter the list of test cases.
Column options : Add or remove fields to display and sort on. The list of columns available for selection are
primarily the fields from the test case work item form.
Define test case context menu options
The context menu on the Test case node within the Define tab provides the following options:
Open/edit test case work item form : This option allows you to edit a Test case using the work item form
wherein you edit the work item fields including test steps.
Edit test cases : This option allows you to bulk edit Test case work item fields. However, you cannot use this
option to bulk edit test steps.
Edit test cases in grid : This option allows you to bulk edit the selected test cases including test steps using
the grid view.
Assign configurations : This option allows you to override the suite level configurations with test case level
configurations.
Remove test cases : This option allows you to remove the test cases from the given suite. It does not change
the underlying test case work item though.
Create a copy/clone of test cases : Create a copy or clone selected test cases.
View linked items : Review items linked to a test case.
Copy or clone test cases
Choose Copy test case to copy or clone a test case. Specify the destination project, destination test plan and
destination test suite in which to create the copy/cloned test case. In addition, you can also specify whether you
want to include existing links/attachments to flow into the cloned copy.
View linked item
Use View linked items option, to review objects linked to the test case. Choose each tab to view the links listed
under the linked object type:
Test Suites
Requirements : Includes any work item that belongs to the Requirements Category, such as User Stories
(Agile), Product Backlog Items (Scrum), Requirements (CMMI), or a custom work item type.
Bugs : Includes bugs that have been filed as part of test execution and any work items that belong to the Bug
Category that have been linked to the test case.

Bulk update using the Grid view

Choose the Grid View to perform bulk updates to the test cases.
Click within a cell to edit the text in that cell. Right-click within a cell to choose an option to Inser t row , Delete
row , or Clear row . Choose Save test cases to perform a bulk save of your changes. Or, choose Refresh
to clear your changes. When complete, choose Close Grid .

Execute tests
Use the Execute tab to assign test points or run tests.

What is a test point? Test cases by themselves are not executable. When you add a test case to a test suite
then test point(s) are generated. A test point is a unique combination of test case, test suite, configuration, and
tester.
For example, if you have a test case named Test login functionality and you add two configurations for the Edge
and Chrome browsers, you have two test points. You can execute or run each of these test points. On execution,
test results are generated. Through the test results view, or execution history, you can see all executions of a test
point. The latest execution for the test point is what you see in the Execute tab.
Test cases are reusable entities. By including them in a test plan or suite, test points are generated. By executing
test points, you determine the quality of the product or service under development.
Execute tasks
From the Execute tab you can do one of the following tasks:
Bulk mark test points : Quickly mark the outcome of or more test points—Passed , Failed , Blocked or Not
Applicable —without having to run the test case via the Test runner.
Run test points :Run the test cases by individually going through each test step and marking them pass/fail
using a Test runner. Depending upon the application you are testing, you can use the Web Runner for
testing a "web application" or the Desktop Runner for testing desktop or web applications. Invoke Run
with options to specify a Build against which the testing you want to perform.
Column options : Add or remove columns. The list of columns available for selection are associated with
test points, such as Run by, Assigned Tester, Configuration, etc.
Toggle Full screen view : View the contents of the page in a full screen mode.
Filter : Filter the list of test points using the fields of a test case Title , ID , Outcome , Tester , Configuration ,
Assigned To , or State . You can also sort the list by choosing a column header.
Change column options
Choose Column options to open the Column options dialog. Add or remove columns and drag and drop
them in the order you want.
Test Points context menu options
Choose the More options to open the context menu for a Test Point node to perform one of the following
actions:
View execution histor y : View the execution history for the test case in a side pane. You can move across
the test points or open the detailed execution history from the pane.
Mark Outcome : Quickly mark the outcome of the test points—Passed , Failed , Blocked or Not
Applicable .
Run - Initiate a test runner with options to Run for web application , Run for desktop , Run with
options .
Reset test to active : Reset the test outcome to Active , ignoring the last outcome of the test point.
Edit test case : Open the work item form, optionally edit work item fields including test steps.
Assign tester : Assign the test points to a tester for test execution.
To learn more about executing tests, see Run manual tests and Run automated tests from test plans.

Chart test cases and test results

From the chart tab you can create various test case or test result charts. To learn how, see Track test status,
Charts.

Next steps
Create test plans and test suites

Related articles
Run manual tests
Run automated tests from test plans
 Track test status.
About pipeline tests
What are extensions?
Test Planning and Management Guide.
REST APIs
Test Management REST API
 Test objects and terms
6/27/2022 • 8 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Read this article to gain an understanding of the objects and terms used in manual and exploratory testing.

Test-specific work item types

To support manual and automated testing, you add and group three main types of test-specific work item types:
Test Plans , Test Suites , and Test Cases . To support sharing of various test steps and test parameters, you
define Shared Steps and Shared Parameters . These objects are stored in the work tracking data store as
specific types of work items.

The following table describes the work item types used to support the Azure DevOps test experience. Test-
specific work items link together using the link types shown in the previous image.
Work item type
Description

Test plans
Are used to group test suites and individual test cases. To define a test plan, see Create test plans and test suites.
Test suite
Group test cases into separate testing scenarios within a single test plan. Grouping test cases makes it easier to
see which scenarios are complete. When creating a test suite, you can specify one of three types:
Static test suites : Used to group test cases under a single test suite.
Requirement-based suites : Select one or more requirements from a query which are then linked to the
test suite.
Quer y-based suites : Select one or more test cases which are then linked to the test suite.

TIP
The Test Suite Type read-only field indicates the type of suite selected. To add test suites, see Create test plans and test
suites.

Test cases
Define the steps used to test code or an app for deployment. Define test cases to ensure your code works
correctly, has no errors, and meets business and customer requirements. You can add individual test cases to a
test plan without creating a test suite. More than one test suite or test plan can refer to a test case. You can
effectively reuse test cases without needing to copy or clone them for each suite or plan. There are two types of
test cases:
Manual : Test cases that define different steps that you run using Test Runner or other supported client.
Automated : Test cases that are designed to run within an Azure Pipeline.

TIP
You can create a test case that automatically links to a requirement—User Story (Agile), Product Backlog Item (Scrum),
Requirement (CMMI), or Issue (Basic)—when you create a test from the Kanban board. To learn more, see Add, run, and
update inline tests.

Shared steps
Use to share steps between multiple test cases. For example, log-in and verify steps for signing into an
application are steps that can be shared across a number of test cases. To learn how, see Share steps between
test cases.
Shared parameters
Use to specify different parameters for executing test a test step within a test case. To learn how, see Repeat a
test with different data.

Common fields to all test-specific work item types

The following fields and tabs appear in most work items. Each tab is used to track specific information, such as
history, links, or attachments. These three tabs provide a history of changes, view of linked work
items, and ability to view and attach files.
The only required field for all work item types is Title . When the work item is saved, the system assigns it a
unique ID . The form highlights required field in yellow. For information about test-related fields, see Query
based on build and test integration fields. For all other fields, see Work item field index.
Field
Usage

Title
Enter a description of 255 characters or less. You can always modify the title later.
Assigned To
Assign the work item to the team member responsible for performing the work. Depending on the context you
are working in, the drop-down menu lists only team members or contributors to the project.

NOTE
You can only assign work to a single user. If you need to assign work to more than one user, add a work item for each
user and distinguish the work to be done by title and description. The Assigned To field only accepts user accounts that
have been added to a project or team.

State
When the work item is created, the State defaults to the first state in the workflow. As work progresses, update it
to reflect the current status.
Reason
Use the default first. Update it when you change state as need. Each State is associated with a default reason.
Area (Path)
Choose the area path associated with the product or team, or leave blank until assigned during a planning
meeting. To change the dropdown list of areas, see Define area paths and assign to a team.
Iteration (Path)
Choose the sprint or iteration in which the work is to be completed, or leave it blank and assign it later during a
planning meeting. To change the drop-down list of iterations, see Define iteration paths and configure team
iterations.
Description
Provide enough detail to create shared understanding of scope and support estimation efforts. Focus on the
user, what they want to accomplish, and why. Don't describe how to develop the product. Do provide sufficient
details so that your team can write tasks and test cases to implement the item.

Common controls to all test-specific work item types

Several controls appear in several test-specific work items as described in the following table. If these controls
are not of interest, you can hide them from the work item form layout as described in Add and manage fields
(Inheritance process).
Control
Description

Deployment
Provides insight into whether a feature or user story has been deployed and to what stage. You gain visual
insight into the status of a work item as it is deployed to different release environments as well as quick
navigation to each release stage and run. This control is available from Test Plans , Test Suites , and Test
Cases .
Development
Records all Git development processes that support completion of the work item. It is typically used to drive Git
development from a requirement. This control supports traceability, providing visibility into all the branches,
commits, pull requests, and builds related to the work item. This control is available from Test Plans , Test
Suites , and Test Cases .
Related Work
Control used in Test Plans , Test Suites , and Test Cases to show or link to other work items such as
requirements and bugs, usually through the Related link type.
Test Cases
Control used in Shared Steps and Shared Parameters work items to indicate or link to Test Cases .

Customize test-specific work item types

For the Inherited process, you can customize test plans, test suites, and test cases. For the On-premises XML
process, you can customize all test-specific work item types. To learn more, see Customize work tracking objects
to support your team's processes.

Permissions required to modify work items

There are a number of permissions that control select features for viewing, modifying, or deleting work items.
These include those listed in the following table.

NOTE
The Change work item type permission doesn't apply to test-specific work items. Even if you choose this feature from
the work item form, changing the work item type is disallowed.

Permission
Level
Task
View test runs
Create test runs
Delete test runs
Project-level
To view, create, or delete test runs, you must be granted the corresponding permission.
Manage test configurations
Manage test environments
Project-level
Manage test configurations or test environments, you must be granted the corresponding permission.
Create tag definition
Project-level
Add new tags to test-based work items.
Delete and restore work items
Project-level
Delete test-specific work items and restore them from the Recycle bin.
Permanently delete work items
Project-level
Permanently delete test-specific work items from the data store.
View work items in this node
Edit work items in this node
Area Path
View or add or modify test plans, test suites, test cases, or other test-based work item types requires the
corresponding permission.
Manage test plans
Area Path
Modify test plan properties such as test run and test outcome settings.
Manage test plans
Area Path
Create and delete test suites; add, and remove test cases from test suites; change test configurations associated
with test suites; and modify a test suite hierarchy (move a test suite).
To learn more about setting these permissions, see Set permissions and access for testing and Change project-
level permissions.

Export, import, and bulk update of test-specific work items

As with other work items, you can bulk edit test-specific work items. To learn more, see the following articles:
Bulk modify work items.
Navigate Test Plans, Test suites context menu options
Bulk modify work items.
Can I export the test plan to share or review offline

Test terms
The following table describes several terms used in manual and exploratory testing.
Term
Definition

Configuration
Specifies the unique environment used to test an application or code. To define a test configuration, you first
define the configuration variables, and then define the test configuration. For details, see Test different
configurations.
Configuration variable
Specifies a single aspect of a test environment such as an operating system, processing power, web browser, or
other variation. For details, see Test different configurations.
Outcome
Outcome of a test point as marked by the tester upon executing the test. Valid options are:
Active (Unspecified)
Pass Test
Fail Test
Block Test
Not Applicable

To learn more, see Repeat a test with different data. Note that pipeline test outcomes differ as described in About
pipeline tests.
Test points
Test cases by themselves are not executable. When you add a test case to a test suite then test point(s) are
generated. A test point is a unique combination of test case, test suite, configuration, and tester. For example, if
you have a test case named Test login functionality and you add two configurations for the Edge and Chrome
browsers, you have two test points. You can execute or run each of these test points. On execution, test results
are generated. Through the test results view, or execution history, you can see all executions of a test point. The
latest execution for the test point is what you see in the Execute tab.
Test run settings
Dialog used to associate test plans with a build or release pipelines.
Test outcome settings
Dialog used to choose how test outcomes in multiple suites under the same test plans should be configured.
Traceability
Ability to trace test results with the requirements and bugs that they are linked to.

Related articles
Exploratory and manual testing scenarios and capabilities
Navigate Test Plans
About pipeline tests
 Create test plans and test suites
6/27/2022 • 8 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Create test plans and test suites to track manual testing for sprints or milestones. That way, you can see when
the testing for a specific sprint or milestone is complete. For more information about manual testing, see What
is Azure Test Plans? and Automated and Manual Testing with Azure Test Plans.

Share work items across your test experience

For example, you're building version 1.* of your product and you might create several test cases for that version.
Each of these test cases can be updated, and more added, at any time. For each development cycle and release
of your product, you create a test plan and import the existing test cases into that plan. You can also, if you wish,
divide the test cases into separate test suites within the plan to enable easier management and monitoring of
these separate sets of test cases.
After you create your test plan, you assign test configurations and assign testers to cover the required test
matrix. These testers run the tests and gauge the quality of the product. Testers continue testing until the product
meets exit criteria. For the next development cycle and release, you can create a new test plan and reuse the
same test cases. You repeat this development-test-release cycle by importing the same test cases into each new
test plan.
The great thing is that, because test plans refer to test cases, updates to a test case automatically reflect in all the
test plans and test suites that use it.
In the next version of the product, you can reuse the existing test cases. However, a better option may be to copy
or clone the test cases. A copy creates a new baseline. Changes to these new test cases don't affect your previous
test plans.

TIP
For more information about the ways you can work with test plans, test suites, and test cases, see the FAQs for manual
testing.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view test-related work items, you must have Basic access or higher and permissions to view work
items under the corresponding Area Path.
To add test plans and test suites, manual test cases, delete test artifacts, and define test configurations you
must have Basic + Test Plans access level or have one of the following Visual Studio subscriptions :
Enterprise
Test Professional
MSDN Platforms
To add or edit test-related artifacts, you must have the following permissions:
 To add or modify test plans, test suites, test cases, or other test-based work item types, you must have
Edit work items in this node permission set to Allow under the corresponding Area Path .
To modify test plan properties such as build and test settings, you must have Manage test plans
permission set to Allow under the corresponding Area Path .
To create and delete test suites, add, and remove test cases from test suites, change test configurations
associated with test suites, and modify a test suite hierarchy (move a test suite), you must have
Manage test suites permission set to Allow under the corresponding Area Path .
To learn more, see Manual test access and permissions.

NOTE
If you're new to using Azure Test Plans, review Navigate Test Plans to understand how to use the user interface to access
select functions.

Create a test plan

In general, you create test plans to test requirements. Before creating a test plan, you may want to define your
backlog of requirements.
1. From the web portal, open your project and go to Test Plans . If you already have a test plan, select Test
Plans to go to the page that lists all test plans.

2. In the Test Plans page, select New Test Plan to create a test plan for your current sprint.
3. In New Test Plan , enter a name for the test plan. Verify that the area path and iteration are set correctly,
then select Create .

Rename a test plan

To rename a test plan, use this procedure:
1. Select Test Plans to view test plans.
2. Next to the test plan name, select More Actions > Edit .
3. Change the name and then select Save & Close .
You can make other changes to the test plan here.
Delete a test plan
To delete a test plan, use this procedure:
1. Select Test Plans to view test plans.
2. Next to the test plan name, select More Actions > Delete .
3. The Permanently delete test ar tifacts dialog box explains exactly what will be deleted. Type the test
plan ID to confirm that you want to delete, and then select Permanently delete .

In general, you create test plans to test requirements. Prior to creating a test plan, therefore, you may want to
define your backlog of requirements.
1. From the web portal, open your project and go to Test Plans . If you already have a test plan, choose Test
Plans to go to the page that lists all test plans.

2. In the Test Plans page, choose New Test Plan to create a test plan for your current sprint.

3. In New Test Plan , enter a name for the test plan. Verify that the area path and iteration are set correctly,
then choose Create .

Rename a test plan

To rename a test plan, open it from the shortcut menu and rename it.
Delete a test plan
To delete a test plan, use the shortcut menu for the test plan.

Add a requirement-based test suite

Now add test suites for the backlog items that need manual tests. These tests could be user stories,
requirements, or other work items based your project.
 NOTE
Requirement tracking is supported only for test cases linked through a Requirement-based test suite . Work items
include a User Story (Agile), Product Backlog Item (Scrum), Requirement (CMMI), and Issue (Basic). The association
between a requirement work item and manual test execution is only formed when the test case is linked by using a
Requirement-based test suite .

1. To add a suite to a test plan, select More options for the test suite, and then select New Suite >
Requirement based suite .

You use requirement-based suites to group your test cases together. That way, you can track the testing
status of a backlog item. Each test case that you add to a requirement-based test suite is automatically
linked to the backlog item.
2. In Create requirement-based suites , add one or more clauses to filter your work items by the
iteration path for the sprint. Run the query to view the matching backlog items.
3. In the list of work items returned by the query, select the backlog items you want to test in this sprint.
Select Create suites to create a requirement-based suite for each one.

Now add test suites for the backlog items that need manual tests. These tests could be user stories,
requirements, or other work items based your project.

NOTE
Requirement tracking is supported only for test cases linked through a Requirement-based test suite . The association
between a requirement work item—User Story (Agile), Product Backlog Item (Scrum), Requirement (CMMI), or Issue
(Basic) and manual test execution is only formed when the test case is linked via a Requirement-based test suite .

1. To add a suite to a test plan, select the + new drop-down list and choose a type of test suite.
 You use requirement-based suites to group your test cases together. That way, you can track the testing
status of a backlog item. Each test case that you add to a requirement-based test suite is automatically
linked to the backlog item.
2. In Create requirement-based suites , add one or more clauses to filter your work items by the
iteration path for the sprint. Run the query to view the matching backlog items.

3. In the list of work items returned by the query, select the backlog items you want to test in this sprint.
Choose Create suites to create a requirement-based suite for each one.

Work with test suites

You can create a static test suite that can contain any type of test suites. Use these test suites like folders. Drag
test suites to group them in a static test plan. Drag and drop tests to reorder them.

You can track changes to test plans and test suites. Open the work item for the test plan or test suite, then view
the work item history.
For test suites, other actions are tracked in the Test Suite Audit field. For example, adding and removing test
cases from a test suite are tracked in this field.
You can export test plans, test suites, and test cases.
Select Expor t test cases to CSV .

Select the details that you want in the report. Then email or print this report for review.
Change the test case fields in the report by adding or removing columns from the list view of the test suite.

IMPORTANT
You cannot export more than 75 Test Suites in a single operation. The email supports up to 1MB of data.

Find a test case in a test plan

In Test Plans for your test plan, use the filter icon to show the search and filter list. It can help find the tests
you want.

In Test Plans for your test plan, use the filter icon (1 ) to show the search and filter lists (2 ) that help you find
the tests you want to work with. Or filter the list of tests using Outcome , Tester , and Configuration (3 ).

Next steps
Create manual test cases
Related articles
Test objects and terms
FAQs for manual testing
End-to-end traceability
 Create manual test cases
6/27/2022 • 13 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018

Create manual test cases to check that each deliverable meets your users' needs. Manual test cases define
individual steps testers perform, including steps that are shared across test cases. To test different data, you
specify parameters for the test steps. To learn more, see Share steps between test cases and Repeat a test with
different data. For an overview of test artifacts, see Test objects and terms.
Organize your test cases by adding them to test plans and test suites. Then assign testers to run the tests.
For an overview of test artifacts, see Test objects and terms.

NOTE
Test iterations are designed to support data-driven scenarios, not workflow-driven scenarios. From a best practice
perspective, if you have two test scenarios where the workflows are different, consider creating separate test cases. Also
see FAQs for manual testing.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view test-related work items, you must have Basic access or higher and permissions to view work
items under the corresponding Area Path.
To add test plans and test suites, manual test cases, delete test artifacts, and define test configurations you
must have Basic + Test Plans access level or have one of the following Visual Studio subscriptions :
Enterprise
Test Professional
MSDN Platforms
To add or edit test-related artifacts, you must have the following permissions:
To add or modify test plans, test suites, test cases, or other test-based work item types, you must have
Edit work items in this node permission set to Allow under the corresponding Area Path .
To modify test plan properties such as build and test settings, you must have Manage test plans
permission set to Allow under the corresponding Area Path .
To create and delete test suites, add, and remove test cases from test suites, change test configurations
associated with test suites, and modify a test suite hierarchy (move a test suite), you must have
Manage test suites permission set to Allow under the corresponding Area Path .
To learn more, see Manual test access and permissions.
 NOTE
If you're new to using Azure Test Plans, review Navigate Test Plans to understand how to use the user interface to access
select functions.

Create test cases

1. If you haven't already, create a test plan and requirement-based test suites.
2. Select a requirement-based test suite. To the right, select the New Test Case button.

If you have no existing test cases, the button appears in the lower right.

NOTE
The test suite shown here was created from a User Story work item in the team's backlog board. When you add a
test case to this kind of suite, the test case is automatically linked to the backlog item. To create test cases this
way, open the context menu for the work item and choose Add test .

3. In the new work item, enter a title. Select the Click or type here to add a step option.
 Add test steps with a description of the action required to carry out the test and the expected results so
that any team member can run the test. You can add attachments to a step if you want. Repeat until you
have added all the steps for the test.

NOTE
You can share steps between test cases. For more information, see Share steps.

Now you've created a test case that you can run.

You can also copy test cases. For more information, see Copy or clone stories, issues and other work items.

Assign configurations to test cases

You can specify configurations, such as different operating systems, web browsers, and other variations for your
tests.
To assign test configurations to all test cases in a test suite, select the test suite, select More options or right-
click to open the context menu, and then select Assign configurations .
In the Assign configurations to test suite dialog box, select the configurations you want.
You can also assign configurations to individual test cases. Select one or more test cases, select More options
or right-click to open the context menu, and then select Assign configuration .
The Assign configurations to selected test cases dialog box opens. Make and save your changes.
For more information about configurations, see Test different configurations.

Reorder test cases

You can reorder manual test cases in static suites, requirement-based suites, and query-based suites. Open a test
case, then use the up and down arrows to change the order.

There are also options to change order in the step context menu.

Add existing test cases to a test suite

You can add existing test cases to a test suite.
1. Select a test suite. From the New Test Case menu, select Add existing test case .
2. In the Add test cases to suite dialog box, add search clauses, as needed, and then select Run quer y .

3. After you find the test cases you want, select Add test cases .
1. If you haven't already, create a test plan and requirement-based test suites.
2. Select a requirement-based test suite. In the right-hand pane, choose + (New) and then choose New test
case .
 The test suite shown here was created from a User Story work item in the team's backlog board.
When you add a test case to this kind of suite, the test case is automatically linked to the backlog
item. To create test cases this way, open the shortcut menu for the work item and choose Add test .

3. Choose the Click or type here to add a step link and add test steps with a description of the action
required to carry out the test and the expected results so that any team member can run the test. You can
add attachments to a step if you want. Repeat until you have added all the steps for the test. Now you've
created a test case that you can run.

You can share steps between test cases. For more information, see Share steps.
How can I find an existing test case within a test plan?
You can reorder manual test cases in static suites, requirement-based suites, and query-based suites. Choose
Order tests on the tool bar, then drag and drop one or more tests. Or open the shortcut menu for a test to
move it to the top or to another position. After reordering the tests, you can sort them by the Order field and
then run them in that order with the web runner.
 TIP
You can create a test case that automatically links to a requirement—User Story (Agile), Product Backlog Item (Scrum),
Requirement (CMMI), or Issue (Basic)—when you create a test from the Kanban board. To learn more, see Add, run, and
update inline tests.

Use the Grid view to edit test cases

You can copy and paste test cases into the Grid view.
To open the Grid view, select the Grid View icon at the right.

Instead, you can select several test cases and open them for editing in the Grid view. In the context menu, select
Edit test case(s) in grid .
On the Grid context menu, you can add, delete, or clear rows.

You can use the Grid view when you add multiple test cases to the test suite:

You can edit other fields in the Grid view. In the List view, use the column options to select the fields in the test
case work item.
You can then view and edit these fields when you switch to the Grid view.
Switch between Grid and List views using the View menu at the right of the window.

On the grid shortcut menu, you can add, delete, or clear rows.

You can use the Grid View when you add multiple test cases to the test suite:
 NOTE
Do not use the Teams plugin for Excel to add or update test case work items. Excel cannot parse the format used to store
test steps, and in some cases this may affect the formatting of the test case work items.

You can edit other fields in the Grid view. In the List view, use the column options to select the fields in the test
case work item.

You can then view and edit these fields when you switch to the Grid view.
Use Excel to copy test case information into a Grid view
You can copy test cases and test steps from an existing Excel worksheet. Copy the columns from Excel that you
want to use for the title, action, and expected results fields. Copying does not copy column formatting, other
than multiline, from the worksheet. Paste these columns into the Grid view, edit if necessary, and save them.
You can copy the data from the Grid view and paste it into your Excel worksheet. Copying does not copy test
step formatting, other than multiline, into the worksheet.

NOTE
Do not use the Teams plugin for Excel to add or update test case work items. Excel cannot parse the format used to store
test steps, and in some cases this may affect the formatting of the test case work items.

You can copy test cases and test steps from an existing Excel worksheet. This is supported only with Internet
Explorer and Chrome browsers.
Copy the columns from Excel that you want to use for the title, action, and expected results fields. Copying does
not copy column formatting, other than multiline, from the worksheet. Paste these columns into the Grid view,
edit if necessary, and save them.

You can copy the data from the Grid view and paste it into your Excel worksheet. Copying does not copy test
step formatting, other than multiline, into the worksheet.

Assign testers
You can reassign test cases so that a different tester can run them. You can assign all the test cases in a test suite
to multiple testers, which is useful for acceptance testing.
Testers need Basic access to run tests from Azure Test Plans.
1. In the context menu for a test suite, select Assign testers to run all tests .
 The Select testers to run all the tests in suite dialog box opens.
2. Add or remove testers from the list. After you select the testers, select Send email and edit the message
as required so they know that tests are ready for them to run.

The email contains a link that testers can open to see the list of assigned tests.
You can assign an individual test case to a tester.
1. In the Execute tab for a test suite, select a test, and then open the context menu.
2. Select Assign tester . Search for and select a tester.
1. You can reassign test cases so that another tester can run them. Select the tests that you want to reassign.
Then open the shortcut menu (choose the "..." ellipses or right-click) and select the tester you want to run
the tests.

Or, you can assign all the test cases in a test suite to multiple testers. This is useful for acceptance testing.
Open the shortcut menu for the test plan and choose Assign testers to run all tests .
2. Add or remove testers from the list. After you select the testers, tick the Send email checkbox and edit
the message as required so they know that tests are ready for them to run.

The email contains a link that testers can open to see the list of assigned tests. Testers need just Basic
access to run tests from Azure Test Plans.

Manage test cases

You can open a test case to view it or edit it. To open a test case in a test suite, in the Define tab, double-select
the name of the test case to open. In the Execute tab, select a test case, open its context menu, and select Edit
test case .
A test case can be linked to test suites, requirements, and bugs. To see linked items, in the Define tab, open the
context menu for a test case, and select View Linked Items .

In the Linked Items dialog box, you can view Test Suites , Requirements , and Bugs .

Bulk edit test cases

You can edit more than one test case at a time. Select several test cases in a test suite and select Edit test
case(s) .
Select a Field and enter a Value . Select Add new field to add another field-value pair.
Use tags for test cases
You can tag test cases and view only the ones with specific tags. For example, tag all the tests related to signing
in so that you can rerun these tests if a bug is fixed for that page. You can filter on that tag from the Test Plans
web portal.
To add new tags to add to work items, you must have Basic access or higher and have the project-level Create
new tag definition permissions set to Allow . for more information, see Add work item tags.
You can add and edit tags when you edit a test case, or bulk edit tags in the Grid view. You can also create suites
based on queries when you use tags.

Rename or remove test cases

You can rename or remove test cases. Open the test case from its context menu.
Here you can rename the test case.

Or you can remove the test case from the test suite. From the context menu for the test case, select Remove .

To permanently delete test plans and test suites, you must be a member of the Project Administrators group or
have the Area Path node-level Manage test plans or Manage test suites permission set to Allow . To
manage or delete test artifacts, you must also have your access level set to Basic + Test Plans or Visual
Studio Enterprise . For more information, see Delete test artifacts in Azure Boards.
You can tag test cases and view test cases only with specific tags. For example, tag all the tests related to signing
in so that you can rerun these tests if a bug is fixed for that page. Then you can filter on that tag from the Test
Plans web portal.
To add new tags to add to work items, you must have Basic access or higher and have the project-level Create
new tag definition permissions set to Allow . for more information, see Add work item tags.
You can add and edit tags when you edit a test case, or bulk edit tags in the grid view. You can also create suites
based on queries when you use tags.

You can rename or delete test cases. Open the test case from its shortcut menu.

Here you can rename the test case.

Or you can permanently delete the test case.

To delete test plans and test suites, you must be a member of the Project Administrators group or have the Area
Path node-level Manage test plans or Manage test suites permission set to Allow. To manage or delete test
artifacts, you must also have your access level set to Basic + Test Plans or Visual Studio Enterprise.
To delete test plans and test suites, you must be a member of the Project Administrators group or have the Area
Path node-level Manage test plans or Manage test suites permission set to Allow . To manage or delete test
artifacts, you must also have your access level set to Basic + Test Plans or Visual Studio Enterprise . For
more information, see Delete test artifacts in Azure Boards.

Next step
Run manual tests

Related articles
Copy or clone stories, issues and other work items
Delete test artifacts in Azure Boards
FAQs for manual testing
Repeat a test with different data
Share steps between test cases
Test different configurations
Test objects and terms
 Share steps between test cases
6/27/2022 • 4 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Many manual tests require performing an identical sequence of steps and test data. For example, logging in to a
web app or saving form data are common steps performed in several test sequences. With the use of Shared
Steps and Shared Parameters work items, you can minimize the creation of test steps and data that you need
to enter and manage. Shared Steps define a sequence of steps that can be referenced by many different test
cases. Similarly, Shared Parameters define a set of test data that can be referenced by many different test
cases. These work item types are explicitly linked with the Test Cases/Shared Steps and Reference
By/References link types as shown in the following image.

This article describes how to define and use Shared Steps . To learn how to define and reference Shared
Parameters , see Repeat a test with different data.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view test-related work items, you must have Basic access or higher and permissions to view work
items under the corresponding Area Path.
To add test plans and test suites, manual test cases, delete test artifacts, and define test configurations you
must have Basic + Test Plans access level or have one of the following Visual Studio subscriptions :
Enterprise
Test Professional
MSDN Platforms
To add or edit test-related artifacts, you must have the following permissions:
To add or modify test plans, test suites, test cases, or other test-based work item types, you must have
Edit work items in this node permission set to Allow under the corresponding Area Path .
To modify test plan properties such as build and test settings, you must have Manage test plans
permission set to Allow under the corresponding Area Path .
To create and delete test suites, add, and remove test cases from test suites, change test configurations
associated with test suites, and modify a test suite hierarchy (move a test suite), you must have
Manage test suites permission set to Allow under the corresponding Area Path .
To learn more, see Manual test access and permissions.
 NOTE
If you're new to using Azure Test Plans, review Navigate Test Plans to understand how to use the user interface to access
select functions.

Create shared steps

To create shared steps, follow these steps:
1. Select Test Plans to view your test plans. Select Mine or All or Filter by title to find a particular test
plan. Select the plan to see test cases.
2. Double click a test case to edit it. Under Steps , select several steps and then select the Create shared
steps icon.
3. In the Create shared steps dialog box, type a name for these shared step, and then select Create .

Your title appears in place of the selected steps:

4. Select Save & Close to save the work item.

If you reopen the work item, you can select the shared step to open a new shared steps work item you created.
While you're editing a test case, select a sequence of steps that you want to share:

The steps you selected are replaced with a link to the new shared steps work item:

You can edit shared steps at any time. Any changes you make appear in every test case where you use the
shared steps.
You can use parameters in shared steps. For more information about parameters, see parameters.
You provide values for the parameters in the test cases that use the shared steps. You don't have to provide
values in the shared steps definition. However, you can provide one default row of values. These values are used
when you create an action recording of a standalone shared step.

Insert shared steps into other test cases

You can add shared steps to other test cases.
1. Double-click test case to edit it. Under Steps , select the Inser t shared steps icon.

2. The Inser t shared steps dialog box opens. Select Run quer y to find the steps you want to insert.

3. Select shared steps and insert them.

Now you can use the shared steps in another test case:

A query opens. Run it to find the steps you want to insert:

Mark shared steps when running a test
When you run a test, you can either mark the whole shared sequence as passed or failed, or mark each step
separately:

When you run a test, you can either mark the whole shared sequence as passed or failed, or mark each step
separately:
Related articles
Test objects and terms
Create a test case
Repeat a test with different data
Copy or clone test plans, test suites, and test cases
FAQs for manual testing
 Copy or clone test plans, test suites, and test cases
6/27/2022 • 25 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Several tools support copy, clone, or import operations of test items—such as, test plans, test suites, and test
cases. Test cases describe the steps to take to run a test and validate a feature implementation or bug fix. Test
suites group test cases, and optionally other test suites, into a particular order. Test plans define a collection of
test suites to run for a particular iteration or release.
Each test case is designed to confirm a specific behavior. Test cases may belong to one or more test suites,
however test suites can belong to one and only one test plan.
In general, you should create a test plan for every major project milestone. Copy, clone, or import operations of
test items support the following scenarios:
Define a test plan for a new sprint or release based on an existing test plan
Import test suites from one test plan to another test plan within the same or different project
Copy test cases for use in different test suites and test plans
Share test objects across projects
Move test objects from one project to another, possibly to support consolidating projects into a single
project.
For an overview of test objects and terminology, see Test objects and terms.

Supported copy, clone, and import tools

Depending on the Azure DevOps version you use, you can use the clients or tools listed in the following table to
copy, clone, or import test plans, test suites, or test cases.
Client/tool
Test Plans
Test Suites
Test Cases

Web por tal 1

️ (Copy)
✔
️ (Import)
✔
✔ (Copy)
️
️ (Bulk export/import)
✔
Web por tal (Grid)
️ (Copy and paste)
✔
Web por tal (Grid)
️ (Copy and paste)
✔
Work item form
️ (Copy)
✔
Microsoft Test Manager 1
(deprecated)
️ (Clone plan)
✔
️ (Copy a test suite)
✔
️ (Create copy)
✔
TCM CLI 2
️ (Clone)
✔
️ (Clone)
✔
️ (Import automated)
✔
REST API 3
️ (Clone)
✔
️ (Clone)
✔

NOTE
1. With the release of Azure DevOps Server 2020, the web portal for Azure Test Plans was significantly updated. Many
new features were added to support copy, clone, import, and export. For an overview, see Navigate Test Plans.
2. The Test Case Management (TCM) command-line tool is installed when you install Visual Studio 2017 or earlier
versions. Examples provided in this article reflect the options available with the Visual Studio 2017 version. Earlier
versions may support fewer options. For details, see Work with the TCM command-line tool.
3. REST API commands to support cloning of test plans and test suites were added with version 5.0, corresponding with
Azure DevOps 2019.
1. Microsoft Test Manager (MTM) was deprecated for use with Azure DevOps Services in January 2020, and isn't
supported for Azure DevOps Server 2020 and later versions. The current version of Azure Test Plans supports all
features that MTM supported and more.
2. The Test Case Management (TCM) command-line tool is installed when you install Visual Studio 2017 or earlier
versions. Examples provided in this article reflect the options available with the Visual Studio 2017 version. Earlier
versions may support fewer options. For details, see Work with the TCM command-line tool.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view test-related work items, you must have Basic access or higher and permissions to view work
items under the corresponding Area Path.
To add test plans and test suites, manual test cases, delete test artifacts, and define test configurations you
must have Basic + Test Plans access level or have one of the following Visual Studio subscriptions :
Enterprise
Test Professional
MSDN Platforms
To add or edit test-related artifacts, you must have the following permissions:
 To add or modify test plans, test suites, test cases, or other test-based work item types, you must have
Edit work items in this node permission set to Allow under the corresponding Area Path .
To modify test plan properties such as build and test settings, you must have Manage test plans
permission set to Allow under the corresponding Area Path .
To create and delete test suites, add, and remove test cases from test suites, change test configurations
associated with test suites, and modify a test suite hierarchy (move a test suite), you must have
Manage test suites permission set to Allow under the corresponding Area Path .
To learn more, see Manual test access and permissions.

Work with the TCM command-line tool

To exercise the TCM command-line tool, you must have the following in place:
Installed Visual Studio 2017 Professional or earlier version. You access TCM from the command prompt
and the following directories: %programfiles(x86)%\Microsoft Visual Studio\2017\Professional\Common7\IDE
%programfiles(x86)%\Microsoft Visual Studio\2017\Enterprise\Common7\IDE
%programfiles(x86)%\Microsoft Visual Studio 14.0\Common7\IDE

To run a TCM command, specify the /collection and /teamproject parameters, and /login as needed.

PA RA M ET ER DESC RIP T IO N

/collection :CollectionURL Required. Specifies the URI of the team project

[Link] format for the URI is as follows:
- For Azure DevOps Services:
[Link]
- For Azure DevOps Server:
[Link]
. If no virtual directory is used, then the format for the
URI is as follows:
[Link]

/teamproject : project Required. The name of the project that contains the test
objects you want to clone or import automated tests
into.

/login : username,[password] Optional. Specifies the name and password of a valid

Azure DevOps user and who has permissions to run the
command. Use this option if your Windows credentials
don't have the appropriate permissions, or you're using
basic authentication, or you're not connected to a
domain.

You must be a valid member of the project(s) you want to access and have the required permissions
based on the commands you run. To learn more, see Manual test access and permissions.
To clone or import test objects, you must have been granted the same access levels required to add test
plans and test suites as described in Prerequisites.

List test plans or test suites

You often need to know the ID assigned to a test plan or test suite to support copy, clone, or import operations.

Browser
TCM CLI
You can generate a list of test plans, test suites, or other test objects from the Boards>Queries page. For
example, by setting the Work Item Type=Test Plan , you can list all test plans defined for the team project. If
you choose the Quer y across all projects checkbox, the query will list all test plans defined for all projects. To
learn more about defining queries, see Define a work item query.

TIP
While test plans, test suites, and test cases are related to each other, you can't view the relationships through a work item
query. Link types aren't used to link test plans, test suites, and test cases. Only shared steps and shared parameters are
linked to test cases. Also, test cases are linked to user stories or other work items that they test.

Copy or clone test plans

We recommend creating a new test plan per sprint or release. When doing so, generally you can clone the test
plan for the prior cycle and, with few changes, the copied test plan is ready for the new cycle.
Cloning is useful when you want to branch your application into two versions. After cloning, tests for the two
versions can be changed without affecting each other.

Browser
TCM CLI

NOTE
If you're new to using Azure Test Plans, review Navigate Test Plans to understand how to use the user interface to access
select functions.
1. Open Test Plans>Test plans , and choose the test plan you want to copy from the Mine or All page.
Select More actions , and choose the Copy test plan menu option. This option lets you copy or clone
test plans within a project.

2. Specify the name of the new test plan, and select an Area Path and Iteration Path for the plan.

Choose one or the other radio-buttons based on the following guidance:

Reference existing test cases : Choose this option when you plan to merge the two branches
eventually. In this case, you'll want to keep the same requirements for functionality that has already
been implemented and tested.
Duplicate existing test cases : Choose this option when you want to make new user stories or
requirements that you will maintain separately. If you plan to diverge into two similar but separate
applications, you might want to change the user stories of one without changing the stories of the
other. Choosing this option creates an independent set of requirements for the new test cases.
If you duplicate existing test cases, the copied test cases will be assigned the Area Path and Iteration
Path specified for the cloned test plan.
To learn more about area and iteration paths, see About area and iteration (sprint) paths.
3. The page refreshes to display the newly copied test plan.

4. Update any query-based suites that you copied to use the new area and iteration paths.
5. Specify a build in the destination test plan if you have cloned automated test cases.

NOTE
This feature isn't supported through the web portal for Azure DevOps Server 2019 and earlier versions. The feature to
copy test plans requires Azure DevOps Server 2020 or later version.

Import or clone test suites

When you're creating the test plan for a new sprint, you often want to repeat some of the tests from the
previous sprints, to make sure that the functionality you already implemented still works.

Note the following:

When you import (Web) or clone (TCM) a test suite to a new project, the new suite will contain copies of all
test cases in the source suite. However, it won't retain any historical data like the previous test runs, related
bugs, old results.
Shared steps referenced by the test cases are also cloned and created in the destination project.
You can't clone test cases from and to the same test plan, even into a different suite. To accomplish this
scenario, you must first move the test cases to a different, temporary test plan. Then, use the temporary test
plan as the source plan and clone the test cases back to the source test plan by putting the ID into the
destination test plan place. Shared steps present in the test cases will also be duplicated in this process.
You can use the user interface to import a test suite from one test plan, within the same or different project, to
another test plan in the current project. This action essentially copies or clones the test suite, creating a new test
suite and duplicating any sub-test suites. The test cases referenced by the test suites are not duplicated, but
referenced by the cloned test suites.
You can use tcm suites /list to list all test suites defined for a project.
In order to use the same test cases in different suites and plans, copy and paste test cases using the Grid view.
To learn more, see (XLink to be provided)

Browser
 TCM CLI

1. Open Test Plans>Test plans , and choose the test plan where you want to import a test suite from the
Mine or All page. Select More actions , and choose the Copy test plan menu option. This option lets
you copy or clone test plans within a project.

2. In the dialog that opens, choose the project if the test plan resides in a different project. Otherwise, select
the test plan and test suite and then choose Create . You can only import one suite from a test plan at a
time.

3. The following message displays:

4. When the import operation completes, you'll see the following message. Choose Refresh to refresh your
browser.
 The newly added test suite then displays.

NOTE
This feature isn't supported through the web portal for Azure DevOps Server 2019 and earlier versions. The feature to
import test suites requires Azure DevOps Server 2020 or later version.

Copy test case(s)

From the web portal, you can copy test cases from within a project or another project to a test suite, or you can
use the Grid view to copy and paste test cases from one suite to another. Optionally, you can bulk import and
export test cases.

TIP
Don't copy test cases when what you really want to do is test with different configurations or different data.

You can copy test cases from within a project or another project in the organization or collection to a designated
test plan and test suite. At the same time, you can optionally copy all links and attachments.
1. Open Test Plans>Test plans , choose the Test Plan that contains the test case(s) you want to copy from
the Mine or All page. Next, choose the Test Suite that contains the test case(s) you want to copy. From the
Define page, select the check box for all test cases you want to copy.

2. Choose More actions , and choose the Copy test case menu option.
3. From the Copy test case(s) dialog, choose the Project if copying test cases from a different project.
Next, select the test plan and test suite to copy the test cases to.

Optionally select the check boxes for Include existing links and Include existing attachments . When
done, choose Create .
Depending on the number and complexity of the test cases selected, the copy operation will be
performed in the background. Once completed, you'll receive a message that the operation has
completed and a link to the test suite where the test cases have been copied to.

Copy and paste test case(s) (Grid view)

You copy and paste test cases in order to use the same tests in different suites and plans. For example, you could
have a test suite that uses a subset of the tests defined in a more exhaustive test suite. The Define>Grid view
supports editing test cases as well as copy and pasting test cases to different test suites. For editing test cases,
see Create test cases, Use the Grid view to edit test cases.

TIP
Don't copy test cases when what you really want to do is test with different configurations or different data.

From the Internet Explorer, Edge, or Chrome browsers, you can copy test cases from the Grid view for one test
plan and test suite to another test plan and test suite.
1. From the Test Plans>Test plans page, choose the test suite containing the test case(s) you want to copy.
Within the Define tab, Choose the Grid view.
2. Highlight the rows you want to copy, and then enter CTRL+C .

3. Select a different test suite from the same or different plan and paste with CTRL+V . If you don't select a
different suite, nothing happens when you paste, because each suite can only have one reference to any
test case.
4. Choose Save test cases .
The new test cases are saved with new IDs assigned.
Bulk import or export test cases
From the web portal, you can perform a bulk import or export of test cases from/to a test suite. Test cases are
defined in a comma-separated values (CSV) file.
Export test cases
1. From the Test Plans>Test plans page, choose the test plan with the test suite containing the test cases
you want to export.
2. Optional. Choose Column options to add fields to include in the download file.
3. To export all test cases for the test suite, choose Expor t test cases to CSV .
 To export a subset of test cases for the test suite, select the test cases to export, choose More options ,
and select Expor t test case(s) to CSV .

4. The exported CSV file appears in your Downloads folder.

Import test cases
1. From Test Plans>Test plans page, choose the test plan with the test suite into which you want to
import test cases.
2. Choose the file to import from the dialog that opens, and then choose Impor t .

3. Choose Confirm in the Confirm impor t dialog that displays. If you specify test cases that are already
defined in the test suite, some elements may get over written during import.

Import automated test cases (TCM)

To import automated test cases to a test suite, use tcm testcase /import . You must specify a .dll file for the test
assembly that contains your tests.

tcm testcase /import /collection:teamprojectcollectionurl /teamproject:project

/storage:path
[/maxpriority:priority]
[/minpriority:priority]
[/category:filter]
[/syncsuite:id [/include]]
[/login:username,[password]]

PA RA M ET ER DESC RIP T IO N

/storage : path Specifies the path and name of the test assembly that
contains your automated tests that you want to import.

/maxpriority : priority Optional. Specifies which tests to import based on the

maximum priority of the test method. For example, if the
parameter is /maxpriority:1 , only tests with a priority
attribute for the test method less than or equal to 1 are
imported as test cases from the assembly.
 PA RA M ET ER DESC RIP T IO N

/minpriority : priority Optional. Specifies which tests to import based on the

minimum priority of the test method. For example, if the
parameter is /minpriority:2 , only tests with a priority
attribute for the test method equal or greater than 2 are
imported as test cases from the assembly.

/categor y : filter Optional. Specifies which tests to import based on the

category of each test method in the test assembly. You can
use this parameter together with /syncsuite to import
tests with a certain category into a specific test suite.
For more information about test categories, see Run unit
tests with Test Explorer.

/syncsuite : id Optional. Specifies the suite ID for the test suite in your test
plan to which you want to add the test cases that you
import. This suite cannot be a dynamic suite or a query-
based suite. If you specify a test suite to synchronize to
update tests that have already been added, the tests that
aren't imported are removed from the test suite but not
from the test plan itself.

For descriptions of /collection , /teamproject and /login parameters, see Work with the TCM command-line
tool.

Next step
Run manual tests

Related articles
Create test plans and test suites
Create test cases
Share steps between test cases
Test different configurations
Repeat a test with different data
Test objects and terms
Create a query based on build and test integration fields
Customize and manage the test experience
Microsoft Test Manager (deprecated)
Guidance on Microsoft Test Manager usage
Copying and cloning test suites and test cases
REST APIs
You can copy and clone test plans and test suites using the following REST APIs.
Test Plan Clone - Clone Test Plan
Test Suite Clone - Clone Test Suite
 Run manual tests
6/27/2022 • 21 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Run your manual tests and record the test results for each test step using Microsoft Test Runner. You can run
tests for both web applications and desktop apps.
Test Runner lets you run all active tests as a batch or specific test cases. Modify tests while running them. You
can run tests for a specific build.
While testing, gather information like a screenshot or your testing actions. If you find an issue when testing, use
Test Runner to create a bug. Include test steps, screenshots, and comments in the bug.

NOTE
You execute test points and not test cases. When you add a test case to a test suite, test point(s) are generated. A test
point represents a unique combination of test case, test suite, configuration, and tester.
For example, a test case named Test login functionality, which has two configurations for the Edge and Chrome browsers,
generates two test points. You can execute or run each of these test points. On execution, test results are generated.
Through the test results view, or execution history, you can see all executions of a test point. The latest execution for the
test point is what you see in the Execute tab.

Supported clients and run options

In addition to running tests for a specific build, run options support various test scenarios. To see the options, in
the Execute tab, select a test, and then select Run with options .
Select test type and runner offers these options:
Manual tests using Web Browser based runner You can select a specific build to test, as described
in Run tests for a build. Select Find builds to open the Find builds dialog box and search for a build to
test against.
Manual tests using Test Runner client You can run manual tests for desktop applications by using
the Test Runner client. If necessary, install the Test Runner desktop client.
Automated tests using release stage To run tests using a release stage, select a build. Select a release
pipeline and stage to consume the build's artifacts. For more information, see Run automated tests from
test plans.
Manual tests using Microsoft Test Manager 2017 client Collects data from a local machine. Select
a build. Select one or more data collectors: Action log , Event log , Screen and voice recorder , and
System information .
Manual tests using Microsoft Test Manager 2015 or earlier client
The Test Center in the Microsoft Test Manager client is a desktop-based manual testing solution. Testers can use
it for manual testing needs. For more information, see Guidance on Microsoft Test Manager usage. To get
Microsoft Test Manager, install Visual Studio Enterprise or Visual Studio Test Professional.

Prerequisites
 You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view or run manual or automated tests, you must have Basic access or higher.
To learn more, see Manual test access and permissions.

Work with the TCM command-line tool

To exercise the TCM command-line tool, you must have the following in place:
Installed Visual Studio 2017 Professional or earlier version. You access TCM from the command prompt
and the following directories: %programfiles(x86)%\Microsoft Visual Studio\2017\Professional\Common7\IDE
%programfiles(x86)%\Microsoft Visual Studio\2017\Enterprise\Common7\IDE
%programfiles(x86)%\Microsoft Visual Studio 14.0\Common7\IDE

To run a TCM command, specify the /collection and /teamproject parameters, and /login as needed.

PA RA M ET ER DESC RIP T IO N

/collection :CollectionURL Required. Specifies the URI of the team project

[Link] format for the URI is as follows:
- For Azure DevOps Services:
[Link]
- For Azure DevOps Server:
[Link]
. If no virtual directory is used, then the format for the
URI is as follows:
[Link]

/teamproject : project Required. The name of the project that contains the test
objects you want to clone or import automated tests
into.

/login : username,[password] Optional. Specifies the name and password of a valid

Azure DevOps user and who has permissions to run the
command. Use this option if your Windows credentials
don't have the appropriate permissions, or you're using
basic authentication, or you're not connected to a
domain.

You must be a valid member of the project(s) you want to access and have the required permissions
based on the commands you run. To learn more, see Manual test access and permissions.
To clone or import test objects, you must have been granted the same access levels required to add test
plans and test suites as described in Prerequisites.

Run tests for web apps

Follow these steps to run tests for web applications.
1. From the web portal, open your project and select Test Plans > Test plans .
2. If you haven't already, create your manual tests.
3. Select Mine or All , or use Filter by title to find your test plan and select it. Select the Execute tab.
4. Select one or more tests, or all the tests from a test suite. Then select Run for web application .

Microsoft Test Runner opens and runs in a new browser.

5. Start the app that you want to test.
 Your app doesn't have to run on the same computer as Test Runner. You just use Test Runner to record
which test steps pass or fail while you manually run a test.
For example, you might run Test Runner on a desktop computer and run your store app for Windows 8
that you test on a Windows 8 tablet.
6. Mark each test step as either passed or failed based on the expected results.

If a test step fails, you can enter a comment on why it failed or collect diagnostic data for the test. You can
also Create or add to a bug.
1. If you haven't already, create your manual tests.
2. Select a test from a test suite and run it.

Microsoft Test Runner opens and runs in a new browser.

3. Start the app that you want to test. Your app doesn't have to run on the same computer as Test Runner.
You just use Test Runner to record which test steps pass or fail while you manually run a test. For
example, you might run Test Runner on a desktop computer and run your Windows 8 store app that you
are testing on a Windows 8 tablet.

4. Mark each test step as either passed or failed based on the expected results. If a test step fails, you can
enter a comment on why it failed or collect diagnostic data for the test. You can also Create or add to a
bug.
 IMPORTANT
Any test step that has expected result is called a validation test step. Testers must mark a test step with a status if it is a
validation test step. The overall result for a test case reflects the status of all the test steps that the tester marked.
Therefore, the test case will have a status of failed if the tester marked any test step as failed or not marked.

Create or add to a bug

If a test step fails, enter a comment about the behavior and collect diagnostic data. You can create a bug to
capture and track the issue. You can also update an existing bug with information about the failure.
1. When a step fails, enter a comment and select Create bug .

2. In the New bug dialog box, enter a name for the bug.
 The steps and your comments are automatically added to the bug. If Test Runner is running in a web
browser window, you can copy a screenshot from the clipboard directly into the bug.
3. You can assign the bug, enter comments, or link to other issues. Select Save & Close when done. The
test case is linked to the bug you created.
You can see any bugs reported during your test session.

Instead of creating a bug, you can update an existing bug with information about this step. Select Add to
existing bug from the Create bug drop-down menu.
1. In the Test Runner window, select Create bug .

The steps and your comments are automatically added to the bug. If Test Runner is running in a web
browser window, you can copy a screenshot from the clipboard directly into the bug.
2. You can assign the bug, enter comments, or link to other issues. Select Save & Close when done. The
test case is linked to the bug you created.
You can see any bugs that you reported during your test session.
Instead of creating a bug, you can update an existing bug with information about this step. Select the + icon for
Create bug , then Add to existing bug . Find and update an existing bug.

Save results, close the session, and review results

After you complete testing, save your results, close the session, and review test results.
1. When you've run all your tests, select Save and close . All the test results are stored in Azure Test Plans.
2. View the testing status for your test suite. You see the most recent results for each test.
 If you haven't run a test yet, its state is active. Reset the state of a test to active if you want to rerun it.
3. Open a test suite and choose the test case in the Related Work section. Then use the child links in the
Related Work section of that work item to view the bugs filed by the tester.

You can run tests offline and then import the results. For more information, see the Offline Test Execution
extension.
Run tests for desktop apps
If you want to collect more diagnostic data for your desktop application, run your tests using Test Runner client.
1. From the web portal, open your project and select Test Plans > Test plans .
2. Select Mine or All , or use Filter by title to find your test plan and select it. Select the Execute tab.

3. Launch Test Runner from Azure Test Plans by selecting Run for desktop application from the
dropdown menu.

4. If necessary, download and install the Test Runner desktop client.

 NOTE
Check that the Test Runner client is available for your platform. Currently, the Test Runner client is available only
for x64.

5. Select Launch and start testing as described in the previous section. For more information about data
collection, see Collect diagnostic data while testing.
If you want to collect more diagnostic data for your desktop application, run your tests using Test Runner client:
1. Launch the test runner client from Azure Test Plans by choosing Run for desktop application from the
Run menu.

2. Download and install the Test Runner desktop client if you haven't already set it up:
 NOTE
Check that the Test Runner client is available for your platform. Currently, the Test Runner client is available only
for x64.

3. Choose Launch and start testing the same way as described above for web apps. For more information
about data collection, see Collect diagnostic data while testing.

Run all tests

You can run all the tests in a test suite at once.
Select a test suite and select Run for web application or Run for desktop application to run all the active
tests.

Select a test suite and select Run to run all the active tests.
Run tests for a build
Choose a build to run tests against.
1. From the dropdown, select Run with options .

2. In the Run with options dialog box, select the build you want.

1. Choose Run and then select Run with options .

2. Select the build you want.

NOTE
The selected build must be from the project in which the tests are defined.

You can select a build for the following options:

Manual tests using Web Browser based runner
Automated tests using release stage
Manual tests using Microsoft Test Manager 2017 client
The fields that the dialog box offers differ depending on which option you select. For more information, see
Supported clients and run options.
Any bug filed during the run is associated with the selected build. The test outcome will be published against
that build.

Modify a test step during a test run

Fix problems with your test steps while the test is still running. Select the Edit test step icon.
You can insert, reorder, or delete steps. You can also edit the text itself.
Capture rich diagnostic data
While running your tests, you can add screenshots, capture actions as a log, and record video or voice.
Add a screenshot
Add a screenshot to the test results while running a test.
If you use Google Chrome or Firefox, use the web runner to take screenshots of the web app while testing. For
Microsoft Internet Explorer or Microsoft Edge browsers, or for desktop app testing, use the Test Runner desktop
client.

For more information, see Collect diagnostic data.

Capture actions from a test
Capture your actions on the application as a log.
If you use Google Chrome or Firefox, use the web runner capture your actions on the web app as image logs
while testing. For Microsoft Internet Explorer or Microsoft Edge browsers, or for desktop app testing, use the Test
Runner desktop client.

For more information, see Collect diagnostic data.

Capture screen recordings of your app being tested
Capture screen recordings of my app during testing.
If you use Google Chrome or Firefox, use the web runner to capture screen recordings of your web and desktop
apps while testing. For Microsoft Internet Explorer or Microsoft Edge browsers, or for desktop app testing, use
the Test Runner desktop client.
For more information, see Collect diagnostic data.

Run tests with TCM

You can run tests that are part of a test plan using the TCM command-line tool. This tool lets you create and start
a test run, and then manage all your existing test runs. Use the tcm commands documented here to accomplish
these tasks.
List test runs | Create test runs | Execute test runs | Abort test runs | Delete test runs | Export test runs | Publish
test runs

List test runs

Use tcm run /list to list the runs available in a test plan and to show their ID . The ID corresponds to the work
item ID defined when the run was created.

tcm run /list /collection:teamprojectcollectionurl /teamproject:project

[/planid:id | /querytext:query] [/login:username,[password]]

PA RA M ET ER DESC RIP T IO N

/planid : id Optional. Indicates that only those test runs associated with
the specified test plan are returned in the list.

/quer ytext : query Optional. Specifies the query to use to list a subset of test
runs.

For descriptions of /collection , /teamproject and /login parameters, see Work with the TCM command-line
tool.
Example
The following command lists the test runs defined for the Fabrikam Fiber project hosted in the fabrikamprime
organization. The ID and Title correspond to the work item ID and title defined for the test run. For example, test
run 1000052 is titled Test Plan for Cycle 1 (Manual).
 tcm run /list /collection:[Link] /teamproject:"Fabrikam Fiber"

Id Title Owner Date Completed

--------- ---------------------------------- ------------------- -----------
1000006 Sprint 2 (Manual) Thomas Margand 11/5/2021
1000032 33 : Change initial view (Manual) Danielle Brasseur 11/11/2021
1000040 Sprint 2 (Manual) Thomas Margand 11/16/2021
1000042 Sprint 3 (Manual) Thomas Margand 11/16/2021
1000046 Special testing (Manual) Nicoletta Guibord 11/18/2021
1000052 Test Plan for Cycle 1 (Manual) Bukhosi Bhengu 12/1/2021
1000060 Game Shopping (Manual) Bukhosi Bhengu 12/6/2021

Create test runs

Use tcm run /create to create a test run associated with the specified test plan. In addition to the test plan, you
also define the test suite and configuration you want to use by their corresponding ID . You can gather these IDs
using the tcm plans /list , tcm suites /list , and tcm configs /list commands.

tcm run /create /title:title /planid:id /collection:CollectionURL /teamproject:project

(suiteid:id /configid:configid | /querytext:query)
[/settingsname:name] [/owner:owner] [/builddir:directory]
[/testenvironment:name] [/login:username,[password]] [/include]

PA RA M ET ER DESC RIP T IO N

/title : title Specifies the title that you want to use for the test run that
you create.

/planid : id Specifies the test plan that where you want to create the test
run.

/suiteid : id Specifies the test suites that you want to use for your test
run.

/configid : id Specifies the test configuration you want to run for your test
suites.

/quer ytext : query Optional if you specify suiteid and configid . Specifies
the query to use to select the tests that you want to run.

Tip: You can use the /querytest parameter to run more

than one test suite. For example:
querytext:“SELECT * FROM TestPoint WHERE
(ConfigurationId=20 OR ConfigurationId=21) AND
(Suiteid=1185 OR Suiteid=1186)”
.

/settingsname : name Optional. Specifies the test settings that you want to use for
this test run. If you don't select test settings, the default test
settings in the test plan are used.

/owner : owner Optional. Specifies the owner of the test run.

 PA RA M ET ER DESC RIP T IO N

/builddir : directory Optional. Specifies the build directory to use to locate the
test assemblies for the test. If this isn't specified, the build
location is used based on the build that is currently assigned
to the test plan.

/testenvironment : name Optional. Specifies the test environment that you want to
use for this test run. If you don't select a test environment,
the default test environment in the test plan is used.

/include Optional. Specifies that all tests that are selected for the test
run are included, even if the tests are not currently set to the
Active state.

For descriptions of /collection , /teamproject and /login parameters, see Work with the TCM command-line
tool.
Example
The following command creates a test run called MyTestRun in the test plan with ID 77. The run uses the test
suite with ID 161 and the test configuration with ID 9. The run is defined for the Fabrikam Fiber project hosted
in the fabrikamprime organization.
In this example, a test run is created with an ID of 1000082.

tcm run /create /title:MyTestRun /planid:77 /collection:[Link]

/teamproject:"Fabrikam Fiber" /suiteid:161 /configid:9

Run created with ID: 1000082.

Execute test runs

Use tcm run /execute to kick off one of the runs in your test plan. The ID you specify corresponds to the work
item ID defined when the run was created. You can see a list of all test run IDs with the tcm run /list command.

tcm run /execute /id:id /collection:teamprojectcollectionurl /teamproject:project [/login:username,

[password]]

PA RA M ET ER DESC RIP T IO N

/id : id Specifies the ID for the test run that you want to run.

For descriptions of /collection , /teamproject and /login parameters, see Work with the TCM command-line
tool.
Example
The following command starts a test run for the ID 1000082 for the Fabrikam Fiber project hosted in the
fabrikamprime organization. The results are returned in your CLI window.
 tcm run /execute /id:1000082 /collection:[Link] /teamproject:"Fabrikam
Fiber"

Executing run: MyTestRun

Results
------------------------
Total: 2
Passed: 1
Failed: 1
Inconclusive: 0

Abort test runs

Use tcm run /abort to cancel a test run that is in progress. The ID you specify corresponds to the work item ID
defined when the run was created.

tcm run /abort /id:id /collection:teamprojectcollectionurl /teamproject:project [/login:username,[password]]

PA RA M ET ER DESC RIP T IO N

/id : id Specifies the ID for the test run that you want to cancel.

For descriptions of /collection , /teamproject and /login parameters, see Work with the TCM command-line
tool.
Example
The following command stops the test run with the ID 1000082 for the Fabrikam Fiber project hosted in the
fabrikamprime organization. The results confirm the ID and Title of the cancelled run.

tcm run /abort /id:1000082 /collection:[Link] /teamproject:"Fabrikam Fiber"

Run with ID [1000082] and title [MyTestRun] has been aborted.

Delete test runs

Use tcm run /delete to delete a test run from your test plan. The ID you specify corresponds to the work item
ID defined when the test run was created.

tcm run /delete /id:id [/noprompt] /collection:teamprojectcollectionurl /teamproject:project

[/login:username,[password]]

PA RA M ET ER DESC RIP T IO N

/id : id Specifies the ID for the test run that you want to delete.

/noprompt Optional. Specifies that the user isn't prompted to confirm

deletion of a test run.

For descriptions of /collection , /teamproject and /login parameters, see Work with the TCM command-line
tool.
Example
The following command deletes the test run with the ID 1000082 for the Fabrikam Fiber project hosted in the
fabrikamprime organization. The user is prompted to confirm that they want to delete the specified test run and
the result is provided.

tcm run /delete /id:1000082 /collection:[Link] /teamproject:"Fabrikam Fiber"

Are you sure you want to delete run [MyTestRun]? (Yes/No) y

Run [MyTestRun] has been deleted.

Export test runs

Use tcm run /export to export a test run to a specified location. The ID you specify corresponds to the work
item ID defined when the run was created.

tcm run /export /id:id /resultsfile:path /collection:teamprojectcollectionurl /teamproject:project

[/login:username,[password]]

PA RA M ET ER DESC RIP T IO N

/id : id Specifies the test run ID that you want to export.

/resultsfile : path Specifies a location and filename for the test run you want to
export.

For descriptions of /collection , /teamproject and /login parameters, see Work with the TCM command-line
tool.
Example
The following command specifies that the test run with the ID 1000082 for the Fabrikam Fiber project hosted in
the fabrikamprime organization is exported to c:\temp\[Link].

tcm run /export /id:1000082 /resultsfile:"c:\temp\[Link]"

/collection:[Link] /teamproject:"Fabrikam Fiber"

Publish test runs

Use tcm run /publish to publish the results from a Visual Studio test run results file for a specified test plan.

tcm run /publish /suiteid:id /configid:id /resultowner:owner /resultsfile:path

/collection:teamprojectcollectionurl /teamproject:project [/title:runtitle]
[/runowner:owner] [/build:buildnumber /builddefinition:builddefinition]
[/flavor:flavor] [/platform:platform] [/assignfailurestouser:user]
[/login:username,[password]] [/buildverification]

PA RA M ET ER DESC RIP T IO N

/suiteid : id Specifies the test suite to use when you publish a test run.

/configid : id Specifies which test configuration you want to use when you
publish a test run.
 PA RA M ET ER DESC RIP T IO N

/resultowner : owner Specifies the owner for the test results.

/resultsfile : path Specifies the location of the test run you want to publish. For
example, "c:\temp\[Link]".

/title : runtitle Optional. Specifies a title that you want to use for the test
run that you publish.

/runowner : owner Optional. Specifies the owner of the test run.

/build : buildnumber Optional. Specifies the build number to use to publish a test
run. This parameter must be used with /builddefinition .

/builddefinition : builddefinition Optional. Specifies the build definition to use to publish a

test run. This parameter must be used with /build .

/flavor : flavor Optional. Specifies the build flavor, such as Release . This
parameter can only be used if the /build parameter is
used.

/platform : platform Optional. Specifies the build platform, such as x86 . This
parameter can only be used if the /build parameter is
used.

/assignfailurestouser : user Optional. Specifies the user to whom any failed tests in the
test run are assigned.

/buildverification Optional. Specifies that this test run contains build

verification tests that check the basic functionality of your
build.

For descriptions of /collection , /teamproject and /login parameters, see Work with the TCM command-line
tool.
Example
The following command publishes a test run for the test suite with ID 161 and test configuration with ID 9 and
reassigns the owner. This updates the existing test points for the test cases in the test suite that is paired with
this configuration and publishes the results in the specified .trx file. And any failed tests in the test run are
assigned to the specified user.

tcm run /publish /suiteid:167 /configid:9 /resultowner:"Thomas Margand"

/resultsfile:"c:\temp\[Link]" /assignfailurestouser:"Bukhosi Bhengu"
/collection:[Link] /teamproject:"Fabrikam Fiber"

Frequently asked questions

Here are some common questions.
Q: How do I rerun a test?
A: Just select any test and choose Run .
Q: Can I run all the tests in a test suite together?
A: Yes, select a test suite and choose Run . This option runs all the active tests in the test suite. If you haven't run
a test yet, its state is active. You can reset the state of a test to active if you want to rerun it.

Q: Can I choose a build to run tests against?

A: Yes, Choose Run and then select Run with options .

Select the build you want from the drop-down list.

Any bug filed during the run will automatically be associated with the selected build. The test outcome will be
published against that build.
 NOTE
The selected build must be from the project in which the tests are defined.

Q: Can I fix my test steps while I'm running a test?

A: Yes, if you have Azure Test Plans for Azure DevOps. You can insert, move, or delete steps. Or you can edit the
text itself. Use the edit icon next to the test step number.

The tool to edit the test steps is shown.

Q: Can I add a screenshot to the test results when I run a test?

A: If you're using Google Chrome or Firefox, use the web runner to take screenshots of the web app while
testing. For Microsoft Internet Explorer or Microsoft Edge browsers, or for desktop app testing, you can
download and use the Test Runner desktop client.
For more information, see Collect diagnostic data.
Q: Can I capture my actions on the app as a log?
A: If you're using Google Chrome or Firefox, use the web runner capture your actions on the web app as image
logs while testing. For Microsoft Internet Explorer or Microsoft Edge browsers, or for desktop app testing, you
can download and use the Test Runner desktop client.

For more information, see Collect diagnostic data.

Q: Can I capture screen recordings of my app?
A: If you're using Google Chrome or Firefox, use the web runner to capture screen recordings of your web and
desktop apps while testing. For Microsoft Internet Explorer or Microsoft Edge browsers, or for desktop app
testing, download and use the Test Runner desktop client.

For more information, see Collect diagnostic data.

Q: How do I control how long I keep my test data?
A: For more information, see Set test retention policies.
Q: Where can I download the Test Runner client?
A: Download the Test Runner desktop client.
Q: What are the supported operating systems for the Test Runner client?
A: The Test Runner desktop client is currently supported only on Windows.
Q: Can I opt out of telemetry for the Test Runner client?
A: No. The Test Runner desktop client doesn't collect any user-identifiable data. No opt-out mechanism is
provided. For more information, see Microsoft Privacy policy.
Q: Can I run tests offline and then import the results?
A: Yes, see the Offline Test Execution extension.

Next step
View your test progress

Related articles
FAQs for manual testing
Collect diagnostic data while testing
Exploratory testing with the Test & Feedback extension in Connected mode
Run automated tests from test plans
Run manual tests with Microsoft Test Manager
 Manual test access and permissions
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Access to Azure Test Plans features are controlled by access level and permissions. To access the Test Plans web
portal, you must have a Basic access level or greater. To define and manage test plans, test suites, and test cases,
you need Basic + Test Plans , which is available from Azure DevOps.

NOTE
Users granted Stakeholder access have no access to features or functions supported through the Test Plans or Test web
portal. However, they are able to provide feedback through the Test & Feedback extension. To learn more, see
Stakeholder access quick reference.

Access and licensing

To exercise the full range of test-related features, you must have been granted Basic + Test Plans access level or
have one of the following subscriptions: - Visual Studio Enterprise - Visual Studio Test Professional - MSDN
Platform
Users who have one of the above subscriptions can exercise all Test Plans features for no additional charge. For
more information, see Compare Visual Studio subscriptions.
Non-subscribers must purchase Basic + Test Plans to use Test Plans. This can be for any specified number of
users.

IMPORTANT
Manual testers can execute tests with Basic access. -Licenses for this extension also give users rights to use Microsoft
Test Manager (a deprecated on-premises client).

The following table summarizes the license requirements required to exercise select tasks.

TA SK A C C ESS L EVEL

Provide feedback Stakeholder

Execute tests, Mark test outcomes Basic

Chart tasks, View reports Basic

Create and manage test plans and test suites Basic + Test Plans

Author test cases using a grid-like view and edit in the Test Basic + Test Plans
Runner

Assign test cases to suites, move test cases, and order test Basic + Test Plans
cases
 TA SK A C C ESS L EVEL

Prepare for execution such as assigning configurations or Basic + Test Plans

testers

Prepare User Acceptance Testing Basic + Test Plans

Permissions
In addition to having the necessary access level, you also need the necessary permissions to exercise select
tasks. Because manual testing is managed through test-specific work item types, they are subject to some of the
same permissions that manage work items.
The following table provides the default permissions assigned to the built-in security groups: Readers ,
Contributors , and Project Administrators . Permissions are assigned for Area Paths and at the project-level.
To learn how, see Set permissions and access for testing.
Permission
Level
Readers
Contributors
Project Admins
View test runs
Project-level
️
✔
️
✔
️
✔
Create test runs
Delete test runs
Project-level
️
✔
️
✔
Manage test configurations
Manage test environments
Project-level
️
✔
️
✔
Create tag definition
Delete and restore work items
Project-level
️
✔
️
✔
Permanently delete work items
Project-level
️
✔
View work items in this node
Area Path
️
✔
️
✔
️
✔
Edit work items in this node
Manage test plans
Manage test suites
Area Path
️
✔
️
✔

NOTE
The Change work item type permission doesn't apply to test-specific work items. Even if you choose this feature from
the work item form, changing the work item type is disallowed.

For a simplified view of all default permissions assigned to built-in groups, see Default permissions and access.

Related articles
Test objects and terms
Set permissions and access for manual testing
Default permissions and access
Security groups, service accounts, and permissions in Azure DevOps
 Test different configurations
6/27/2022 • 8 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Your users will probably install or run your app on a wide variety of configurations, such as different operating
systems, web browsers, and other variations. You will want to run at least some of your tests in stages that have
those different configurations.
Use your test plans to decide which tests you want to run on which configurations. You have to make sure that
when you run your tests that you have set up your stages for the configurations that you need.
You might draw a schematic matrix of the combinations that you want to test:

Then you can:

View available configurations and variables
Create configurations and variables
Assign the configurations to test plans and test suites
Run tests with each of the configurations
Track your test results for each configuration

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view or add test configurations, you must have Basic access or higher.
To add test configurations, you must have the project-level Manage test configurations permission set to
Allow . By default, this permission is granted to members of the Contributors and Project Administrator
groups.
To learn more, see Manual test access and permissions.
Work with the TCM command-line tool
To exercise the TCM command-line tool, you must have the following in place:
Installed Visual Studio 2017 Professional or earlier version. You access TCM from the command prompt
and the following directories: %programfiles(x86)%\Microsoft Visual Studio\2017\Professional\Common7\IDE
%programfiles(x86)%\Microsoft Visual Studio\2017\Enterprise\Common7\IDE
%programfiles(x86)%\Microsoft Visual Studio 14.0\Common7\IDE

To run a TCM command, specify the /collection and /teamproject parameters, and /login as needed.

PA RA M ET ER DESC RIP T IO N

/collection :CollectionURL Required. Specifies the URI of the team project

[Link] format for the URI is as follows:
- For Azure DevOps Services:
[Link]
- For Azure DevOps Server:
[Link]
. If no virtual directory is used, then the format for the
URI is as follows:
[Link]

/teamproject : project Required. The name of the project that contains the test
objects you want to clone or import automated tests
into.

/login : username,[password] Optional. Specifies the name and password of a valid

Azure DevOps user and who has permissions to run the
command. Use this option if your Windows credentials
don't have the appropriate permissions, or you're using
basic authentication, or you're not connected to a
domain.

You must be a valid member of the project(s) you want to access and have the required permissions
based on the commands you run. To learn more, see Manual test access and permissions.
To clone or import test objects, you must have been granted the same access levels required to add test
plans and test suites as described in Prerequisites.

View configurations and variables

You often want to see the configurations that are already available to run your tests.

Browser
TCM CLI

You can view a list of test configurations and configuration variables from the Configurations page.
While in your project, select Test Plans > Configurations from the left navigation area. Here you'll see all of
the test configurations and variables currently available for your test plans.
Select a configuration or variable to see more information about it in the window to the right.

Create configurations and variables

A test configuration is a combination of configuration variable values. Your configuration variables could be, for
example, operating system, browser, CPU type, database. A configuration might be "Windows 8 + 32-bit CPU" or
"Windows 10 + 64-bit CPU."
Create the configuration variables first. Then combine multiple variable values to create a configuration.
1. Open the Configurations page in the Test Plans web portal, select the + icon, and select New
configuration variable .

2. Type a name for the variable, such as Browser , and a description. Select Add new value , and then add a
value. Add as many values as you wish to the configuration variable, and then save it.
3. Repeat the steps to create any other configuration variables you need. For example, create a configuration
variable named Operating system with the names of each operating system on which you want to test.

4. Select the + icon and select New test configuration .

5. Type a name for the test configuration and add the configuration variables you created. Choose a value
for each variable for this configuration.

Be sure that Assign to new test plans is checked to make this configuration the default for all the new
test plans you create.
6. Save your new test configuration.
A test configuration is a combination of configuration variable values. Your configuration variables could be, for
example, operating system, browser, CPU type, database. A configuration might be "Windows 8 + 32-bit CPU" or
"Windows 10 + 64-bit CPU."
You must create the configuration variables first. Then combine multiple variable values to create a
configuration.
1. Open the Configurations page in the Test Plans web portal, choose the icon, and select New
configuration variable .
2. Type a name for the variable, such as Browser , and type a value. Add as many values as you wish to the
configuration variable, and then save it.

3. Repeat the steps to create any other configuration variables you need. For example, create a configuration
variable named Operating system with the names of each operating system on which you want to test.

4. Choose the icon and select New test configuration .

5. Type a name for the test configuration and add the configuration variables you created. Choose a value
for each variable for this configuration.

Ensure Assign to new test plans is checked to make this the default configuration for all the new test
plans you create.
6. Save your new test configuration.

Assign configurations to test plans and suites

You can assign configurations to a test suite or an individual test case. Configurations assigned to a test suite
apply to all tests or suites within it.
1. To assign a configuration to a test suite, in the Test plans page, select a plan. In the Test Suites area,
select a test suite and select More options or right-click to open the context menu and then select
Assign configuration .
 If you add multiple configurations to a test suite, the tests cases are repeated with each of the
configurations you have assigned.

2. If necessary, override the default configuration assigned to a test case and assign the configuration you
need. Select one or more test cases, open the context menu, and select Assign configuration .
3. Search for and select the configurations to assign to the test cases.

You can assign configurations to a test plan, a test suite, or an individual test case. Configurations assigned to a
test plan or test suite apply to all tests or suites within it.
1. To assign a configuration to a test plan, in the Test plans page, open the shortcut menu for the plan and
choose Assign configuration to test plan .

2. To assign a configuration to a test suite, open the shortcut menu for the suite and choose Assign
configuration to test suite .
 If you add multiple configurations to a test plan or suite, the tests cases are repeated in the plan or suite
with each of the configurations you have assigned.

3. If necessary, override the default configuration assigned to a test case and assign the configuration you
need. Select one or more test cases, open the shortcut menu, and choose Assign configurations .

4. Search for and select the configurations to assign to these test case(s).
Run tests with each configuration
Set up a testing platform for a particular configuration, such as testing the app using Google Chrome on
Windows 10.
1. Select and run a test that has this configuration assigned.

As you run the test, a reminder of the configuration appears in the status bar.
1. Set up a testing platform for a particular configuration, such as testing the app using Google Chrome on
Windows 10.
2. Select and run a test that has this configuration assigned.

As you run the test, a reminder of the required configuration in shown in the status bar of the Test Runner
window.
Track test results for each configuration
1. Open the Char ts page for your test suite, select New , and select New test result char t .

2. Choose a type of chart, like a pie chart or bar chart, and then select Group by > Configuration and
choose OK .

A chart is created that can help you track your tests based on configurations. You can pin this chart to
your dashboard.
If you have a test case that appears in several test suites, you can set the different configurations for each suite.
The same test case can have different configuration settings in different test suites and test plans.
1. Open the Char ts page for your test plan or test suite, choose New , and select New test result char t .

2. Choose the type of chart you require, select Configuration in the Group by list, and choose OK .

A chart is created that can help you track your tests based on configurations. You can pin this chart to
 your dashboard.

If you have a test case that appears in several test plans and test suites, you can set the different
configurations for each of these. The same test case can have different configuration settings in different test
suites and test plans.

Related articles
Overview of manual and exploratory testing
Exploratory test and submit feedback directly from your browser
 Repeat a test with different data
6/27/2022 • 7 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
You add parameters to your manual test to repeat the test with different test data. For example, you can test
adding different quantities to a shopping cart from quantities of 1, 5, 10, or 200.
You insert parameters within your test steps for a manual test case. Then, you provide a table of parameter
values. You can add shared parameters to test cases or convert parameters you've recently inserted into shared
parameters. Shared steps and shared parameters are different work item types that can be shared across
multiple test cases. They're linked to test cases through the Test Cases-Shared Steps and Referenced By-
References link types as shown in the following illustration.

With shared steps and shared parameters, you can run multiple test cases with the same data.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view test-related work items, you must have Basic access or higher and permissions to view work
items under the corresponding Area Path.
To add test plans and test suites, manual test cases, delete test artifacts, and define test configurations you
must have Basic + Test Plans access level or have one of the following Visual Studio subscriptions :
Enterprise
Test Professional
MSDN Platforms
To add or edit test-related artifacts, you must have the following permissions:
To add or modify test plans, test suites, test cases, or other test-based work item types, you must have
Edit work items in this node permission set to Allow under the corresponding Area Path .
To modify test plan properties such as build and test settings, you must have Manage test plans
permission set to Allow under the corresponding Area Path .
To create and delete test suites, add, and remove test cases from test suites, change test configurations
associated with test suites, and modify a test suite hierarchy (move a test suite), you must have
Manage test suites permission set to Allow under the corresponding Area Path .
To learn more, see Manual test access and permissions.

Add parameters to a test case

You can add parameters to your tests.
1. Create a parameter by typing a name preceded by "@ " in the actions and expected results of your test
 steps.

2. Underneath the list of steps, add combinations of parameter values. You might need to scroll down to see
them.
1. Create a parameter by typing a name preceded by "@ " in the actions and expected results of your test
steps.

2. Underneath the list of steps, add combinations of parameter values. You might need to scroll down to see
them.
Share parameters between test cases
You can convert existing parameters to shared parameters so that you can use them and the associated data in
other test cases.
1. In an open test case, select Conver t to shared parameters .

2. After you've created a shared parameter set, open another test case and add the shared parameter set to
that test case.

You can search for the shared parameter set by name.

The shared parameter set is displayed in the Parameter values section after you add it. You can now use
these parameters in your test case steps.
3. If the test case has different parameter names for these shared parameters, map the shared parameters
to the local parameters to use the shared parameter data.

When they're correctly mapped, the data associated with the shared parameter is displayed.
4. Add, edit, and rename your shared parameter sets in the Parameters page. View the test cases that
reference them in the Test cases pane.
5. Each shared parameter set is a work item. Open the Proper ties page to view or make changes to this
work item. For example, you can assign owners and track changes.
You can't add more than one shared parameter set to a single test case. If two test cases share similar data, for
example, one test case needs customer ID, name, email, and phone, and the second needs customer ID, name
and address, you might consider creating a single shared parameter set containing all of the parameters - even
though a few of the columns in the set will remain unused in each test case.
You can import parameter values from an Excel spreadsheet to your shared parameter sets. Paste the values into
your shared parameters grid. You can also copy the data from your grid back into Excel.
1. Convert existing parameters to shared parameters so that you can use them and the associated data in
other test cases.

2. After you've created a shared parameter set, open another test case and add the shared parameter set to
that test case. You can search for the shared parameter set by name.

The shared parameter set is displayed in the Parameter values section after you add it. You can now use
these parameters in your test case steps.
3. If the test case has different parameter names for these shared parameters, map the shared parameters
to the local parameters to use the shared parameter data.

When they are correctly mapped, the data associated with the shared parameter is displayed.
4. Add, edit, and rename your shared parameter sets in the Parameters page. View the test cases that
reference them in the Test cases pane.

5. Each shared parameter set is a work item. Open the Proper ties page to view or make changes to this
work item. For example, you can assign owners and track changes.
You can't add more than one shared parameter set to a single test case. If two test cases share similar data, for
example, one test case needs customer ID, name, email, and phone, and the second needs customer ID, name
and address, you might consider creating a single shared parameter set containing all of the parameters - even
though a few of the columns in the set will remain unused in each test case.
You can import parameter values from an Excel spreadsheet to your shared parameter sets. Paste the values into
your shared parameters grid. You can also copy the data from your grid back into Excel.

Run a test case with parameters

You can run a test case that uses parameters.
1. Select a test case with parameters and start running it. The Test Runner shows the first row of parameter
values.
2. When you've completed the steps, mark the test passed or failed. Then go on to the next iteration of the
test, which uses the next row of parameter values.

3. Use the menu to navigate to other iterations.

4. If any of the parameter values are incorrect, fix them without canceling the test by choosing Edit from
step's shortcut menu.
1. Select a test case with parameters and start running it. The Test Runner shows the first row of parameter
values.

2. When you've completed the steps, mark the test passed or failed. Then go on to the next iteration of the
test, which uses the next row of parameter values.
3. Use the drop-down to navigate to other iterations.

4. If any of the parameter values are incorrect, fix them without canceling the test by choosing Edit from
step's shortcut menu.

Review the test results

The outcome of the test is based on a precedence hierarchy of all the iteration outcomes. The hierarchy order is
Paused, Failed, Blocked, Passed, Not Applicable, and Unspecified (Active). For example, if you marked any
iteration as failed and all the rest as passed, the outcome of the entire test is shown as failed. This result is
different from test cases that don't have parameters, where the outcome displays the status of the last run.
To review test results, from the Execute tab, select a test point. Select More options or right-click to open the
context menu. Select View test results .
You can view the results in the Test Case Results dialog box.

The outcome of the test is based on a precedence hierarchy of all the iteration outcomes. The hierarchy order is
Paused, Failed, Blocked, Passed, Not Applicable, and Unspecified (Active). For example, if you marked any
iteration as failed and all the rest as passed, the outcome of the entire test is shown as failed. This is different
from test cases that do not have parameters, where the outcome displays the status of the last run.
1. Check the test result by opening the details pane.
2. Double-click a test result to view the test run details, and the test results for each iteration.

Speed up test iterations by using record and playback

It can be error-prone and tedious to work through a long table of parameter combinations. To speed up things,
create an action recording when you run the test with the first set of parameter values, and then play it back for
the other sets.
1. Use Azure Test Plans to run the test.
2. Select Create action recording before you choose Star t .
3. Complete the first test iteration and then move on to the next one.
4. Mark each step as passed or failed as you work. Enter parameter values in the application exactly as
displayed in the test script.
5. Choose Play to run the test with the next set of parameter values. Your actions will be played back
automatically, but you must still verify the results.
Record and playback doesn't work with all applications. For details, see Supported configurations and platforms
for coded UI tests and action recordings.
Related articles
FAQs for manual testing
Overview of manual and exploratory testing
Testing different configurations
Collect diagnostic data
Manage test results
 Add, run, and update inline tests in Azure Boards
and Azure DevOps
6/27/2022 • 5 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Similar to task checklists, you can quickly define inline tests, or a set of manual tests cases, for a backlog item
from your Kanban board. Not only can you add tests, you can run them and update their status. See Kanban
basics if you're new to working with the Kanban board. If you're new to testing, see Exploratory and manual
testing scenarios and capabilities.
In this article, you'll learn:
How to add inline tests to a backlog item from your Kanban board
How to run tests and update the status of tests
How to expand or collapse inline tests
How to reorder or reparent inline tests

Tests you create from the Kanban board are automatically linked to the user story or backlog item.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a team or project.
To add work items and exercise all board features, you must be granted Basic access or higher.
To view or modify work items, your View work items in this node and Edit work items in this node
permissions set to Allow . By default, the Contributors group has this permission set. To learn more, see Set
permissions and access for work tracking.
To view or run tests, you must have Basic access or higher. Users with Stakeholder access can't view or run
tests.
You must connect to a project. If you don't have a project yet, create one.
 You must be added to a team or project.
To add work items and exercise all board features, you must be granted Basic access or higher.
To view or modify work items, your View work items in this node and Edit work items in this node
permissions set to Allow . By default, the Contributors group has this permission set. To learn more, see Set
permissions and access for work tracking.
To view or run tests, you must have Basic access or higher. Users with Stakeholder access can't view or run
tests.

Open your Kanban board from the web portal

1. To view your Kanban board, open your project from a web browser and choose (1) Work , (2) Boards ,
and then (3) select the team's board from the selector.

To choose another team's board, open the selector and select a different team or choose the Browse
all team boards option. Or, you can enter a keyword in the search box to filter the list of team backlogs
for the project.

TIP
Choose the star icon to favorite a team board. Favorited artifacts ( favorited icon) appear at the top of the
team selector list.

1. To view your Kanban board, open your (1) project from a web browser and choose (2) Work , (3)
 Backlogs , (4) Stories , and then (5) Board .

If you don't see Work , your screen size may be reduced. Select the three dots ( , then choose Work ,
Backlogs , and then Board .

2. To choose another team, open the project/team selector and select a different team or choose the
Browse option.

3. Your Kanban board displays.

Add inline tests
1. To start adding tests, open the menu for the work item.

Adding inline tests is the same as adding test cases to a test suite. A default test plan and test suite are
automatically created under which the manual test cases are grouped.
For example, a test suite is created for each user story, and all inline tests are added to that suite. Below,
user story 152 is highlighted which has three manual tests defined with IDs of 153, 155, and 161.
 To learn more about test plans and test suites, see Plan your tests.
2. If you have many tests to add, keep typing each title and select Enter.

To add details to the test case, open it. You can select the title, double-click the inline item, or open the
context menu and choose Open.
See Create manual tests to learn more about defining tests.
Before running the test, you must add details.

Run the inline test

Run the test by selecting Run test from the actions menu for the inline test.
Microsoft Test Runner starts in a new browser instance. For details on running a test, see Run manual tests.

Update the status of an inline test from the action menu

You can update the status of the test from the actions menu.

Updating the status of tests enable you

to track test results.
Why doesn't the Kanban board show the status for test suites and plans already created in Test ?

Expand or collapse inline tests

Upon first opening the Kanban board, you'll see an unexpanded view of checklists.

Select the inline test summary to expand a collapsed set of tests. Select the same summary to collapse an
expanded list.
Copy or reparent a test
To reparent a test, drag and drop the test onto a different user story.

This action automatically changes the linked relationship of the test to point to the new user story.
To create a copy of a test to add to a different user story, select the test, press the CTRL key, and then drag and
drop the test onto the card of the user story.

Related articles
Use inline tests for lightweight traceability and to manage manual tests for user stories or other backlog items
that they support. To learn more about test case management, see Create manual tests.
If you find that you don't use this feature, you can disable it from the common configurations dialog.
Other ways you can quickly add linked items and objects to user stories from the Kanban board:
Add tasks or child items as checklists
Create a new branch, drive Git development
To start web-based exploratory testing for a user story, you need to install the Test & Feedback Marketplace
extension. For details, see Install the Test & Feedback extension.
Test status in the Kanban board
Test integration with the Kanban board makes it easy for teams to get started with manual testing and then take
advantage of the full testing capabilities in Test Manager later, when required. When test cases are created from
the Kanban board and updated afterwards in Test Manager, or the other way around, when users create
requirement-based suites with Test Manager and update them in Test Manager, the Kanban board shows the
correct status. However, the Test status in Kanban board doesn't work if the requirement-based suite has more
than one configuration assigned to it. In such scenario, the Kanban board only shows the test outcome for the
default configuration. As such, it's recommended to use Test Manager to manage/track the testing progress
across multiple configurations.
 Assign tests for user acceptance testing
6/27/2022 • 3 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Today's faster development pace requires tools that enable test teams to more easily verify value based on
business requirements, and the high quality software demanded by customers. This type of testing is often
referred to as user acceptance testing.
Typically you create a Test Suite using a formal requirement work item type. However, today's agile teams often
prefer to work from User Stories or Product Backlog items as their requirements.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To request or provide feedback, you must have Stakeholder access or higher.
To add or modify bugs or other work item types, you must have the Edit work items in this node
permission set to Allow under the corresponding Area Path .
To add new tags, you must have the Create tag definition permission set to Allow .
To learn more, see Set permissions and access for testing.
In addition, you must have already created work items and a test plan. If not, follow the steps provided in the
following articles:
Create your backlog
Create a test plan

Assign and invite testers

For user acceptance testing, you can assign multiple testers to a set of tests. You can even assign the
stakeholders who created the business requirements as testers.
1. Select Test plans to see your test suites. You can select Mine to see your favorites or All to see all test
plans. Select a test plan to open it.
2. Select a test suite and then select More options or right-click to view the context menu. Select Assign
testers to run all tests .
3. In Search users , enter a name or partial name to find the testers you want. You can select multiple
testers.
 TIP
You can select individual users who are a members of the Project Valid Users group. You can't select user groups.

4. To notify testers, select Send email .

5. Enter a message subject and note, as needed, and select Ok .

NOTE
For on-premises Azure DevOps, you must have an SMTP server configured for your deployment. For more information,
see Configure an SMTP server

If you have a large pool of potential testers, you might need to search for testers, as described above.
If you're assigning a tester to an individual test case, the search option appears in the context menu. Select the
Execute tab, then select a test case, and select More options or right-click to open the context menu.

When you select Assign tester , use the search option to find a tester.

Easily track results

A key principle of good user acceptance testing practice is to minimize the effort required to determine whether
a requirement has been achieved. There are two ways you can do this: you can focus on individual test runs and
tests to see which failed, or you can use the chart views that make it much easier to track your test results. These
views are accessible by all members of your team.
 Note: The dashboard display shown here is also used for other types of testing such as continuous testing.

If you don't see the data or information you expect in the dashboard charts, verify that the columns in your data
have been added to the Tests view. For details see this blog post.

Related articles
Test objects and terms
Collect diagnostic data while testing
Copy or clone test plans, test suites, and test cases
FAQs for manual testing
 Collect diagnostic data while testing
6/27/2022 • 3 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Collect diagnostic data while testing your apps. This data will be included in the bugs you file during the test. You
can collect diagnostic data from web apps and from desktop apps, and view it in Azure Test Plans.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To request or provide feedback, you must have Stakeholder access or higher.
To add or modify bugs or other work item types, you must have the Edit work items in this node
permission set to Allow under the corresponding Area Path .
To add new tags, you must have the Create tag definition permission set to Allow .
To learn more, see Set permissions and access for testing.

Collect diagnostic data from web and desktop apps

For web apps under test, you can use web-based Microsoft Test Runner. For desktop apps, download and install
the Test Runner desktop client to collect the following data on demand:
Screen captures
Image action log
Screen recordings

See Exploratory test and submit feedback directly from your browser.

Screen capture
Capture annotated screenshots from your app:
1. Open Test Runner and choose the Capture screenshot icon. Ensure that the app from which you want
to capture data is selected.

2. Drag to select the area of the screen you want to capture, or just capture the full screen.
3. If required, edit the title of the screenshot and add annotations and text to it using the icons in the toolbar.
4. Save your screenshot.

Image action log

Capture your interactions with the web or desktop app as an image action log that provides context:
1. Open or switch to the Test Runner and choose the Capture user actions... icon. Ensure that the app
from which you want to capture data is selected.
2. The Test Runner will now record all the actions you take on the app's browser tab or in the desktop app.

If you create a bug while recording your actions, all the data collected up to that point will be included in
the bug.
3. Finish capturing your actions by choosing the Stop button. The action log is added to the test results as
an attachment.

4. Choose the ActionLog... link at the bottom of the window to view the data captured in the action log.
 The log opens in your web browser.

Screen recording
Capture screen recordings from your apps:
1. Open or switch to the Test Runner and choose the Record screen icon.
2. Choose the entire screen, or choose an app to start recording.
 If you create a bug while recording your screen, the recording automatically stops and is added to the
bug.
3. Finish recording your actions by choosing the Stop button. The recording is added to the test results as
an attachment.

If you do not stop the recording after 10 minutes, it stops automatically and is saved as an attachment to
your test results. Restart the recording the Record screen icon if required.
4. Choose the ScreenRecording... link at the bottom of the window to view the captured recording.
View the diagnostic data
When you create a bug while capturing diagnostic data, all the data captured up to that point is included in the
bug that is created. You can view it before you save the bug.
How do I play the video recordings I created with the extension?
If you want to collect advanced diagnostic data such as code coverage, IntelliTrace, and Test Impact data in
addition to the data items listed above, you must configure the data collectors and other run settings in
Microsoft Test Manager and run your tests using Microsoft Test Manager. For more information, see Run manual
tests with Microsoft Test Manager.

NOTE
If you have an older version of Microsoft Test Manager, we recommend you upgrade to the latest version. However, if you
have Microsoft Test Manager 2015 or an earlier version installed, you can choose Microsoft Test Runner 2015 and
earlier when you launch the test runner using Run with options . You must configure the data collectors and other run
settings in Microsoft Test Manager and specify these as the default settings for the test plan. For more information, see
Run manual tests with Microsoft Test Manager.

Related articles
Exploratory test and submit feedback directly from your browser
Overview of manual and exploratory testing
Testing different configurations
Manage test results
FAQs for manual testing
 Delete test artifacts in Azure Boards
6/27/2022 • 4 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018

While test artifacts such as test plans, test suites, test cases, and so on, are all types of work items, the method
for deleting them differs from deleting non-test work items.

IMPORTANT
We only support permanent deletion of test artifacts such as test plans, test suites, test cases, shared steps and shared
parameters. Deleted test artifacts won't appear in the recycle bin and cannot be restored. Deletion of test artifacts not
only deletes the selected test artifact but also all its associated child items such as child test suites, test points across all
configurations, testers (the underlying test case work item doesn't get deleted), test results history, and other associated
history.

When you delete test artifacts, the following actions occur:

1. Removes the deleted test artifact from the test case management (TCM) data store and deletes the
underlying work item
2. Runs a job to delete all the child items both from the TCM side and the underlying work items. This action
may take time (up to a few minutes) depending on the number of artifacts to be deleted.
3. Causes all information in the work item tracking data store and TCM data store to be deleted and cannot be
reactivated nor restored.

Prerequisites
To delete test runs, you must be a member of the Project Administrators group or have the project-level
Delete test runs permission set to Allow .
To delete test plans and test suites, you must be a member of the Project Administrators group or have the
Area Path node-level Manage test plans or Manage test suites permission set to Allow .
To manage or delete test artifacts, you must also have your access level set to Basic + Test Plans or Visual
Studio Enterprise . This level provides access to the full Test Plans feature set. Users with Basic access and
with permissions to permanently delete work items and manage test artifacts can only delete orphaned test
cases. That is, they can delete test cases created from Work that aren't linked to any test plans or test suites.
To delete test runs, you must be a member of the Project Administrators group or have the project-level
Delete test runs permission set to Allow .
To delete test plans and test suites, you must be a member of the Project Administrators group or have the
Area Path node-level Manage test plans or Manage test suites permission set to Allow .
You must also have your access level set to Basic+Test Plans or Advanced, which provides access to the full
Test feature set. Users with Basic access and with permissions to permanently delete work items and manage
test artifacts can only delete orphaned test cases. That is, they can delete test cases created from Work that
aren't linked to any test plans or test suites.
To delete test artifacts, the following restrictions and operations apply:
Users with Basic access and with permissions to permanently delete work items and manage test artifacts
can only delete orphaned test cases. That is, they can delete test cases created from Work that aren't linked
to any test plans or test suites.
 When you delete a test plan, test suite, test case, shared steps, or shared parameters, you not only
permanently delete them, you also delete all associated test artifacts such as test results.
You can't bulk delete test artifacts. If test artifacts are part of a bulk selection to be deleted, all other work
items except the test artifact(s) will get deleted.

Work item types that support the test experience

The following image illustrates the set of work item types that support the test experience and work with
Microsoft Test Manager. These work item types are linked together using the link types shown.

From the web portal or Microsoft Test Manager, you can view which test cases are defined for a test suite, and
which test suites are defined for a test plan. However, these objects aren't connected to each other through link
types. For definitions of each field used in these work item types, see Query based on build and test integration
fields.

Delete a test case, test suite, or test plan

1. To delete a test case, test suite, or test plan, open it from the web portal and choose the Permanently
delete option from the actions menu. (Bulk deletion isn't supported from a query results page.)
 NOTE
You'll only see the Permanently delete option if you have the necessary permissions and access.

2. Confirm you want to actually delete the item.

3. You can also delete a test plan directly from Test Plans . To delete a test plan, open Test Plans and
choose More Actions for the plan you want to delete, and choose Delete .

3. You can also delete a test plan directly from Test or Test Plans .

4. To delete shared steps and shared parameters, you need to first manually remove all references to them
before you can delete them.
Related articles
Create a test plan
Control how long to keep test results
Set permissions and access for work tracking, Manage test artifacts
 Set permissions and access for testing
6/27/2022 • 6 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
To exercise the full features of Azure Test Plans, you must be granted Basic + Test Plans access level or have one
of the following subscriptions:
Enterprise
Test Professional
MSDN Platforms
In addition, you can grant or restrict access to various manual test features by granting users or groups specific
permissions for an object or project. Many test artifacts correspond to test-specific work item types. So, work-
tracking permissions apply to test-specific work items, such as test plans, test suites, test cases and more. You set
permissions for work items and manual testing features for area paths and at the project-level. You set
permissions to manage test controllers at the organization or collection level. Test controllers are used in
performing load tests.
Object-level, Area path level
Edit work items in this node : Add or edit test-specific work items, such as test plans, test suites, test
cases, shared steps, or shared parameters.
Manage test plans : Modify test plan properties such as build and test settings.
Manage test suites : Create and delete test suites, add, and remove test cases from test suites,
change test configurations associated with test suites, and modify a test suite hierarchy (move a test
suite).
Project-level
Manage test configurations : Add or edit test configurations and configuration variables.
Manage test environments : Add or edit test plan settings.
Create test runs : Run manual tests
Delete and restore work items : Delete test-specific work items
Delete test runs : Delete test results
Manage test configurations : Add or edit test configurations and configuration variables.
Manage test environments : Add or edit test plan settings.
Move work items out of this project : Move work items from one project to another
Permanently delete work items : Permanently delete test-specific work items
Organization or collection-level
Manage test controllers : Permission associated with a deprecated feature for TFS 2018 and later
versions. To learn more, see Overview of test agents and test controllers for running load tests and
Install test agents and test controllers.

Prerequisites
To manage access levels or organization-level permissions, you must be a member of the Project
Collection Administrators security group, or have your Edit instance-level information set to Allow .
To manage project or object-level test-related permissions, you must be a member of the Project
Administrators security group.
For more information, see the following articles:
About access levels
Add organization users and manage access
Change access levels for users or groups
Change project-level permissions
Change project collection-level permissions
To manage access-levels, you must be a member of the Azure DevOps Ser ver Administrators group.
To manage project or object-level test-related permissions, you must be a member of the Project
Administrators security group.
To manage collection-level permissions or manage access levels, you must be a member of the Project
Collection Administrators security group, or have your Edit instance-level information set to Allow .
For more information, see the following articles:
About access levels
Add a server-level administrator
Change access levels for users or groups
Change project-level permissions
Change project collection-level permissions

Grant access to manual testing features

To have full access to the Test feature set, your access level must be set to Basic + Test Plans. Users with Basic
access and with permissions to permanently delete work items and manage test artifacts can only delete
orphaned test cases.

Manage test plans and test suites under an area path

Area path permissions let you grant or restrict access to edit or modify test plans or test suites assigned to those
areas. You can restrict access to users or groups.
In addition to the project-level permissions set in the previous section, team members need permissions to
manage test artifacts which are set for an area path.
Open the Security page for area paths and choose the user or group you want to grant permissions.
Set the permissions for Manage test plans and Manage test suites to Allow .

Set permissions to create and delete test artifacts

While test artifacts such as test plans, test suites, test cases, and so on are types of work items, the method for
deleting them differs from deleting non-test work items.
 IMPORTANT
We only support permanent deletion of test artifacts such as test plans, test suites, test cases, shared steps and shared
parameters. Deleted test artifacts won't appear in the recycle bin and cannot be restored. Deletion of test artifacts not
only deletes the selected test artifact but also all its associated child items such as child test suites, test points across all
configurations, testers (the underlying test case work item doesn't get deleted), test results history, and other associated
history.

When you delete test artifacts, the following actions occur:

1. Removes the deleted test artifact from the test case management (TCM) data store and deletes the
underlying work item
2. Runs a job to delete all the child items both from the TCM side and the underlying work items. This action
may take time (up to a few minutes) depending on the number of artifacts to be deleted.
3. Causes all information in the work item tracking data store and TCM data store to be deleted and cannot be
reactivated nor restored.
You must be a member of the Project Administrators group or have the Delete test ar tifacts permission set to
Allow . You must also have your access level set to Basic + Test Plans or Visual Studio Enterprise , which
provides access to the full Test feature set. Users with Basic access and with permissions to permanently delete
work items and manage test artifacts can only delete orphaned test cases. That is, they can delete test cases
created from Work that aren't linked to any test plans or test suites.
You must be a member of the Project Administrators group or have the Delete test runs permission set to
Allow . You must also have your access level set to Basic+Test Plans or Advanced, which provides access to the
full Test feature set. Users with Basic access and with permissions to permanently delete work items and manage
test artifacts can only delete orphaned test cases. That is, they can delete test cases created from Work that
aren't linked to any test plans or test suites.
For more information, see Delete test artifacts.
As a project admin you can grant a user, team group, or other group you've created to have these permissions.
Open the Security page for the project and choose the user or group you want to grant permissions. To learn
how to access project-level Security , see Change project-level permissions.

NOTE
To enable the new user interface for the Project Permissions Settings Page, see Enable preview features.

In this example, we grant members assigned to the Team Admin group permissions to create and view test runs
and manage test configurations and environments.
Preview page
Current page
In this example, we grant members assigned to the Test Admin group permissions to delete test runs.

Set permissions to manage test controllers

Test controllers are used to perform load testing. To learn more, see Overview of test agents and test controllers
for running load tests.
To set permissions for managing test controllers, open Organization settings and choose Security or
Permissions . Choose the group you want to grant permissions. To learn how to access organization or
collection-level Security , see Change project collection-level permissions.
In this example, we grant members assigned to the Team Collection Admin group permissions to manage test
controllers.

NOTE
To enable the new user interface for Organization Permissions Settings Page, see Enable preview features.
Preview page
Current page
Related articles
Grant or restrict access
Permissions and access for work tracking
Set permissions and access for work tracking
Permissions and groups reference
Troubleshoot permissions
 Set test retention policies
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Running tests, especially automated ones, generates lots of data. To keep your test system responsive and
performing well, have a policy to clear test results that you don't need anymore. Delete automated test results
when you delete your builds. You can keep manual test results while you're still reviewing them, for example, up
to a year.

Prerequisites
To manage project-level test-related policies, you must be a member of the Project Administrators group.
To manage build pipeline object-level test retention policies, you must have the Edit build pipeline
permissions set to Allow .

Set retention period for test results

To delete manual test results after a specific number of days, set the retention limit at the project level. Azure
DevOps keeps manual test results related to builds, even after you delete those builds. That way, build policies
don't delete your test results before you can analyze the data.
1. Sign into Azure DevOps. You'll need at least project administrator permissions.
2. Go to your project and then open the project settings page.
3. In the Test>Retention page, select a limit for how long you want to keep manual test data.

Set retention policy for automated test results associated with builds
By default, Azure DevOps keeps automated test results related to builds only as long as you keep those builds.
To keep test results after you delete your builds, edit the build retention policy. If you use Git for version control,
you can specify how long to keep automated test results based on the branch.
1. Sign into Azure DevOps. You'll need at least build level permissions to edit build pipelines.
2. Go to your project. Find and edit your build pipeline.
3. Open the Retention page. Modify the default policies as required, or add new policies.

If you use Git, and have more than one branch, set the branch filter to delete test results and builds in
specific branches as required. You can keep test results in other branches, even though you delete the
builds in these branches.

Set retention policy for automated test results not associated with
builds
To clean up automated test results that are left over from deleted builds or test results that aren't related to
builds, for example, results published from external test systems, set the retention limits at the project level as
described in Set retention period for test results earlier in this article.

Related articles
Delete test artifacts
FAQs for manual testing
 Associate automated tests with test cases
6/27/2022 • 4 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Visual Studio 2022 | Visual Studio 2019 | Visual Studio 2017 | Visual Studio 2015
Consider using Visual Studio to associate automated tests with a test case when:
You created a manual test case that you later decide is a good test to automate, but you still want to be
able to run that test as part of a test plan. Tests can be run in the CI/CD pipeline by choosing the test plan
or test suite in the settings of the Visual Studio Test task. Automated tests can also be run from the Test
Plans web portal. If you are using XAML builds you can also run these automated tests by using
Microsoft Test Manager.
You want to enable end-to-end traceability of requirements. If your test cases are linked to requirements
or user stories, the results of the test execution can be used to establish the quality of those requirements.
The process to associate an automated test with a test case is:
1. Create a test project containing your automated test. What types of tests are supported?
2. Check your test project into an Azure DevOps or Team Foundation Server (TFS) repository.
3. Create a build pipeline for your project, ensuring that it contains the automated test. What are the
differences if I am still using a XAML build?
4. Use Visual Studio Enterprise or Professional 2017 or a later version to associate the automated test with a
test case as shown below. The test case must have been added to a test plan that uses the build you just
defined.
If you are using the build and release services in Azure DevOps (not a XAML build), you can run associated tests
in the build and release pipelines by using the Visual Studio Test task. You cannot run tests on-demand using
Microsoft Test Manager unless you are using a XAML build.
The parameters in a test case are not used by any automated test that you associate with a test case. Iterations of
a test case that use these parameters are for manual tests only.

For more information about checking in your test project and team build, see Add files to the server and
Continuous integration on any platform. For more information about action recordings and coded UI tests,
see Recording and Playing Back Manual Tests and Use UI Automation To Test Your Code.

Associate your test

1. Open your solution in Visual Studio Enterprise or Professional 2017 or a later version.
2. If you don't know the identifier of the work item for the test case, locate the test case in Azure Test Plans
, or query for the work item in the Work hub.
3. When you know the identifier of the work item for the test case:
If you are using Visual Studio 2017 or later version , follow these steps to associate your tests.
If the Test Explorer window is not displayed, open it from the Test | Windows menu.
 If your tests are not displayed in Test Explorer , build the solution.
In Test Explorer , select the test method you want to associate and choose Associate to Test
Case .
In the dialog that opens, type the test case identifier and choose Add Association , then choose
Save .

The dialog shows a list of test cases currently associated with the selected test method. You cannot
associate more than one test method with a test case, but you can associate a test method with more
than one test case.

FAQ
Q: What types of tests are supported?
A : These are the limitations for each type of test:
Coded UI test, Selenium tests, and unit tests written using Version 1 of the MSTest framework can be
associated with a test case.
Tests that use MSTest v2, NUnit, and xUnit frameworks can be associated with a test case workitem when
using Visual Studio 15.9 Preview 2 or later. However, these tests cannot be run using Microsoft Test Manager
and XAML builds.
Tests that use the .NET core framework can be associated with a test case workitem when using Visual Studio
15.9 Preview 2 or later. To run the .NET core tests the appropriate target framework must be specified in a
runsettings file. However, these tests cannot be run using Microsoft Test Manager and XAML builds.
Tests that use other test frameworks such as Chutzpah (for JavaScript tests such as Mocha or QUnit), or Jest
cannot be associated with a test case.
Associating generic tests may work, but running these tests is not supported.
Q: What are the differences if I am still using a XAML build?
A : If you are using a XAML build in Azure Pipelines, you can run tests that you have associated in a Build-
Deploy-Test workflow using a Lab environment. You can also run tests using Microsoft Test Manager and a Lab
environment.

Q: Can I configure work items to open in Visual Studio?

A : Yes, if you want test work items to open inside Visual Studio instead of the default Azure Pipelines UI in your
web browser, change the Work Items | General setting from the Tools | Options menu in Visual Studio.

See Also
Associate automated test results with requirements
Run automated tests from test plans
Run automated tests with Microsoft Test Manager
 Run automated tests from test plans
6/27/2022 • 13 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Automate test cases in your test plans and run them directly from Azure Test Plans . Automated tests provide
you with the following benefits:
A user-friendly process for testers who may not be well versed with running tests in Build or Release
workflows.
The flexibility to run selected tests on demand, rather than scheduled testing in Build or Release workflows
where all tests meeting the filter criteria are run.
The ability to rerun a few tests that failed due to test infrastructure issues, or you have a new build that
includes fixes for failed tests.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view or run manual or automated tests, you must have Basic access or higher.
To learn more, see Manual test access and permissions.
In addition, you'll need:
A test plan containing your automated tests, which you have associated with automated test methods using
Visual Studio 2017, or Visual Studio 2015 or earlier.
A build pipeline that generates builds containing the test binaries.
An app to test. You can deploy the app as part of the build and release workflow and also use it for on-
demand testing.
Permissions to create and manage releases, edit a release environment, and manage deployment. For more
information, see Set permissions for release pipelines and Release permissions.

Set up your environment

1. In the Test Plans page, choose your test plan, open the shortcut menu, and choose Test plan settings .
2. In the Test plan settings dialog, select the build pipeline that generates builds which contain the test
binaries. You can then select a specific build number to test, or let the system automatically use the latest
build when tests are run.

3. You will need a release pipeline that was created from the Run automated tests from Test Manager
template to run tests from test plans in Azure Test Plans . If you have an existing release pipeline that
was created using this template, select it and then select the existing stage in the release pipeline where
the tests will be executed. Otherwise, choose the Create new link in the dialog to create a new release
pipeline containing a single stage with the Visual Studio Test task already added.
 How do I pass parameters to my test code from a build or release pipeline?
4. Assign meaningful names to the release pipeline and stage as required.
5. You need the Visual Studio Test Platform to be installed on the agent computer. If Visual Studio is already
installed on the agent computer, you can skip this step. If not, you must add the Visual Studio Test
Platform Installer task to the pipeline definition.
6. Add the Visual Studio Test task to the release pipeline and configure it as follows:
Verify that version 2 of the Visual Studio Test task is selected. The version number is shown in the
drop-down list at the top left of the task settings panel.

Verify that Select tests using is set to Test run . What does this setting mean?
For the Test platform version setting, select Installed by Tools Installer .

If you have UI tests that run on physical browsers or thick clients , ensure that the agent is set
to run as an interactive process with auto-logon enabled. Setting up an agent to run interactively
must be done before queueing the build or release (setting the Test mix contains UI tests
checkbox does not configure the agent in interactive mode automatically - it is used only as a
reminder to configure the agent appropriately to avoid failures).
If you are running UI tests on a headless browser , the interactive process configuration is not
required.
Select how is the test platform is provisioned, and the version of Visual Studio or the location of
the test platform that is installed on the test machines
If your tests need input parameters such as app URLs or database connection strings, select the
relevant settings file from the build artifacts. You can use the Publish build ar tifacts tasks in
your build pipeline to publish the settings file in a drop location if this file is not included in the
artifacts. In the example shown below, the application URL is exposed in the run settings file, and is
overridden to set it to a staging URL using the Override test run parameters setting.
 For information about the option settings of the Visual Studio Test task, see Visual Studio Test task.
7. Choose the Agent job item and verify that the deployment queue is set to the one containing the
machines where you want to run the tests. If your tests require special machines from the agent pool, you
can add demands that will select these at runtime.

You may be able to minimize test times by distributing tests across multiple agents by setting
Parallelism to Multiple executions and specifying the number of agents.
 NOTE
If you are running UI tests such as CodeUI or Selenium on physical browsers such as IE, Firefox, or Chrome, the
agent on the machines must be running in interactive mode and not as a service. More details.

8. In the Pipeline page of the release pipeline, verify that the build pipeline containing the test binaries is
linked to this release pipeline as an artifact source.

9. Save the release pipeline.

10. If you chose Create new in the Test plan settings dialog in step 2 of this example, return to the browser
page containing your test plan settings. In the Test plan settings dialog, select the release pipeline and
stage you just saved.
Run the automated tests
1. In the Test Plans web portal, open the test plan and select a test suite that contains the automated tests.
2. Select the test(s) you want to run, open the Run menu, and choose Run test .

The test binaries for these tests must be available in the build artifacts generated by your build pipeline.
3. Choose OK to start the testing process. The system checks that only automated tests are selected (any
manual tests are ignored), validates the stage to ensure the Visual Studio Test task is present and has
valid settings, checks the user's permission to create a release for the selected release pipeline, creates a
test run, and then triggers the creation of a release to the selected stage.
4. Choose View test run to view the test progress and analyze the failed tests. Test results have the
relevant information for debugging failed tests such as the error message, stack trace, console logs, and
attachments.
5. After test execution is complete, the Runs page of the Azure Test Plans shows the test results. The Run
summar y page shows an overview of the run.

There is a link to the Release used to run the tests, which makes it easy to find the release that ran the
tests if you need to come back later and analyze the results. Also use this link if you want to open the
release to view the release logs.
Note : Manually attaching of files is not supported for automated test results.
What are the typical error scenarios or issues I should look out for if my tests don't run?
6. The Test results page lists the results for each test in the test run. Select a test to see debugging
information for failed tests such as the error message, stack trace, console logs, and attachments.
7. Open the Test Plans page and select the test plan to see the status of your tests if tests are updated after
test execution is complete. Select a test to see the recent test results.

FAQ
Q: What permissions do I need to run automated tests from Azure Test Plans?
You must be a Project Contributor, or have the following permissions:
Create releases
Manage releases
Edit release stage
Manage deployment
For more information, see Set permissions for release pipelines and Release permissions.
Q: Can I override the build or stage set at the test plan level for a specific instance of test run?
A: Yes, you can do this using the Run with options command. Open the shortcut menu for the test suite in the
left column and choose Run with options .
Enter the following values in the Run with options dialog and then choose OK :
Test type and runner : Select Automated tests using Release Stage .
Build : Select the build that has the test binaries. The test results will be associated this build.
Release Pipeline : Select a pipeline from the list of release pipelines that can consume the selected build
artifact.
Release Stage : Select the name of the stage configured in your release pipeline.

Q: Why use release stages to run tests?

A: Azure Pipelines offers a compelling orchestration workflow to obtain test binaries as artifacts and run tests.
This workflow shares the same concepts used in the scheduled testing workflow, meaning users running tests in
scheduled workflow will find it easy to adapt; for example, by cloning an existing scheduled testing release
pipeline.
Another major benefit is the availability of a rich set of tasks in the task catalog that enable a range of activities
to be performed before and after running tests. Examples include preparing and cleaning test data, creating and
cleaning configuration files, and more.

Q: How does selecting "Test run" in the Visual Studio Test task version 2 work?
A: The Test management sub-system uses the test run object to pass the list of tests selected for execution. The
test task looks up the test run identifier, extracts the test execution information such as the container and test
method names, runs the tests, updates the test run results, and sets the test points associated with the test
results in the test run. From an auditing perspective, the Visual Studio task provides a trace from the historical
releases and the test run identifiers to the tests that were submitted for on-demand test execution.

Q: Should the agent run in interactive mode or as a service?

A: If you are running UI tests such as coded UI or Selenium tests, the agent on the test machines must be
running in interactive mode with auto-logon enabled, not as a service, to allow the agent to launch a web
browser. If you are using a headless browser such as PhantomJS, the agent can be run as a service or in
interactive mode. See Build and release agents, Deploy an agent on Windows, and Agent pools.
Q: Where can I find detailed documentation on how to run Selenium tests?
A: See Get started with Selenium testing.
Q: What happens if I select multiple configurations for the same test?
A: Currently, the on-demand workflow is not configuration-aware.
Q: What if I need to download product binaries and test binaries from different builds? Or if I need to obtain
artifacts from a source such as Jenkins?
A: The current capability is optimized for a single team build to be tested on-demand using an Azure Pipelines
workflow. We will evaluate support for multi-artifact releases, including non-Azure Pipelines artifacts such as
Jenkins, based on user feedback.
Q: I already have a scheduled testing release pipeline. Can I reuse the same pipeline to run test on-demand,
or should I create a new pipeline as shown above?
A: We recommend you use a separate release pipeline and stage for on-demand automated testing from Azure
Test Plans because:
You may not want to deploy the app every time you want to run a few on-demand tests. Scheduled
testing stages are typically set up to deploy the product and then run tests.
New releases are triggered for every on-demand run. If you have many testers executing a few on-
demand test runs every day, your scheduled testing release pipeline could be overloaded with releases
for these runs, making it difficult to find releases that were triggered for the pipeline that contains
scheduled testing and deployment to production.
You may want to configure the Visual Studio Test task with a Test run identifier as an input so that you can
trace what triggered the release. See How does selecting "Test run (for on-demand runs)" in the Visual
Studio Test task work?.
Q: Can I trigger these runs and view the results in Microsoft Test Manager?
A: No. Microsoft Test Manager will not support running automated tests against Team Foundation builds. It only
works in the web-based interface for Azure Pipelines and TFS. All new manual and automated testing product
development investments will be in the web-based interface. No further development is planned for Microsoft
Test Manager. See Guidance on Microsoft Test Manager usage.
Q: I have multiple testers in my team. Can they run tests from different test suites or test plans in parallel
using the same release pipeline?
A: They can use the same release pipeline to trigger multiple test runs in parallel if:
The agent pool associated with the stage has sufficient agents to cater for parallel requests. If sufficient
agents are not available, runs can still be triggered but releases will be queued for processing until agents
are available.
You have sufficient jobs to enable parallel jobs. See Parallel jobs in Azure Pipelines or Parallel jobs in TFS
for more information.
Testers do not run the same tests in parallel. Doing so may cause results to be overwritten depending on
 the order of execution.
To enable multiple different test runs to execute in parallel, set the Azure Pipelines stage trigger option for
behavior when multiple releases are waiting to be deployed as follows:
If your application supports tests running in parallel from different sources, set this option to Allow
multiple releases to be deployed at the same time .
If your application does not support tests running in parallel from different sources, set this option to
Allow only one active deployment at a time .

Q: How do I pass parameters to my test code from a build or release pipeline?

A: Use a runsettings file to pass values as parameters to your test code. For example, in a release that contains
several stages, you can pass the appropriate app URL to each the test tasks in each one. The runsettings file and
matching parameters must be specified in the Visual Studio Test task.

Q: What are the typical error scenarios or issues I should look out for if my tests don't run?
A: Check and resolve issues as follows:
The release pipeline and stage in which I want to run tests are not shown after I select the build.
Make sure the build pipeline that is generating the build is linked as the primary artifact in the
Ar tifacts tab of the release pipeline.
I get an error that I don't have sufficient permission to trigger a release.
Configure Create releases and Manage deployments permissions for the user in the Security
menu of the release pipeline. See Release permissions.
I get an error that no automated tests were found.
Check the automation status of the selected tests. Do this in the work item for the test case, or use the
Column options link in Azure Test Plans to add the Automation status column to the list of tests.
See the pre-requisites section for information about automating manual tests.
My tests didn't execute, and I suspect the release pipeline is incorrect.
Use the link in the Run summar y page to access the release instance used to run the tests, and view
the release logs.
My tests go into the error state, or remain "in-progress" even after release to the stage is triggered.
Check if the release stage that you selected has the correct task and version selected. You must use
version 2 or higher of the Visual Studio Test task. Version 1 of the task, and the Run Functional
Tests task, are not supported.
See Also
Associate automated tests with test cases
Associate automated test results with requirements
Continuous testing scenarios and capabilities
 About requesting and providing feedback
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
You can request feedback using one of two tools, through the Test & Feedback extension or through the Request
feedback link you access from a dashboard.

NOTE
This article is about using Azure DevOps feedback tools. To provide feedback about Azure DevOps, see Provide product
and content feedback.

About the Test & Feedback extension

Using the Test & Feedback extension is a simple, three step process:

Capture your findings quickly and easily using the tools in the extension. Capture notes, screenshots
with annotations, and screen recordings to describe your findings and highlight issues. Additionally, in
the background the extension automatically captures rich data such as user actions as an image action
log, page load data, and system information about the browser, operating system, memory, and more
that can serve as a starting point for debugging.
Create work items such as bugs, tasks, and test cases directly from the extension. The captured findings
automatically become a part of the work item. Users can file a bug to report an issue with the product, or
create a task that indicates a new work requirement. The extension can also be used to create test cases
for scenarios discovered during exploration.
Collaborate with your team by sharing your findings. Export your session report in Standalone mode,
or connect to Azure DevOps or Team Foundation Server (2015 or later) for a fully integrated experience
including exploring user stories and backlog items, simplified tracking and triaging of bugs and tasks, and
managing feedback requests in one place.
As users perform exploratory testing, you can get insights from the sessions. View completed exploratory
sessions and derive meaningful insights across all sessions. Get end-to-end traceability such as a breakdown of
the work items created, the work items explored and not explored, session owners, and more.
To learn more, see the following articles:
Install the Test & Feedback extension
Request stakeholder feedback
Provide stakeholder feedback
Microsoft Feedback client
The Visual Studio 2015 Microsoft Feedback client is a downloadable tool that you install on your desktop. It
supports similar features for capturing findings to those provided by the Test & Feedback extension. It doesn't
support creating work items. You can download the tool from Feedback Client for Microsoft Visual Studio Team
Foundation Server 2015.
To learn more, see the following articles:
Get feedback (Work tracking)
Provide feedback with the Feedback client
Set feedback permissions

Related articles
Give reviewers permissions to provide feedback
Default permissions and access set for collaboration tools
Give us feedback, get support
 Request stakeholder feedback using the Test &
Feedback extension
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Stakeholders can respond to feedback requests for user stories and features generated in Azure DevOps using a
lightweight end-to-end flow based on the Test & Feedback extension. Only users with Basic access can request
feedback. Basic users can provide feedback using the flow described in this topic.
Request feedback
Provide feedback
Voluntary feedback
Track requests

NOTE
This lightweight end-to-end flow is applicable only for web apps and by using Azure DevOps. To get feedback for desktop
apps, or for earlier versions of TFS, use the feedback flow described in Get feedback about the Microsoft Feedback Client.

Request feedback from stakeholders

Request feedback from stakeholders directly from an Azure DevOps work item.
1. Open the work item form for the user story or feature for which you want to request feedback.
2. Open the shortcut menu from the ellipses (...) and choose Request feedback .

3. Type or select the names of the stakeholder(s) you want to send the request to, and optionally add any
instructions or notes that will help them provide meaningful feedback.
4. Choose Send to generate emails to all the selected stakeholders.

NOTE
Teams can request feedback from other team members, such as users having Basic access. Just add their names in the
feedback request form so that a Request feedback email is sent to them. Also see Can users with Basic access respond
to feedback requests?.

Related articles
Provide stakeholder feedback using the Test & Feedback extension
Voluntary stakeholder feedback using the Test & Feedback extension
Track stakeholder feedback using the Test & Feedback extension
Exploratory test and submit feedback directly from your browser
Overview of manual and exploratory testing
 Provide feedback using the Test & Feedback
extension
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Stakeholders and other users can respond to feedback requests using the Test & Feedback extension in two
ways:
From the link in a feedback request email
Directly from the Test & Feedback extension
Before you start, ensure you have installed the Test & Feedback extension. This is required in order to respond to
feedback requests.

NOTE
Any user with a Stakeholder access can use the Test & Feedback extension in Stakeholder mode. This mode is designed
to allow the widest possible range of users to assist test teams by providing feedback.

Provide feedback directly from a feedback request email

1. Open the feedback request email and choose the Provide feedback link.

2. The Azure DevOps landing page opens to confirm that the extension has been automatically configured
with the feedback request. Choose the icon in the toolbar to launch the extension.
If you are a Stakeholder , you see the Feedback requests page. Read the instructions (if any) in the
feedback form to understand how to give the feedback and what the requestor requires.

If you are a Basic user, you see the Explore work item traceability page showing details of the user
story on which feedback was requested, and the user acceptance criteria (if any).
3. Read any instructions in the email and this page to understand how to give the feedback, and on what
feature.

4. Open the application you need to provide feedback on and begin your feedback. For example, choose
Capture screenshot to take a screenshot.

You can use all the capabilities of the extension such as capturing screenshots, notes, and screen
recordings. For more details, see this topic.

Some browsers may not provide all the capture capabilities. See Supported web browsers for the
extension.

5. When you are done capturing feedback:

If you are a Stakeholder , choose Provide feedback . You can optionally choose to create bugs
and tasks when you submit your feedback. The process is the same as described in this topic.

If you are a Basic user, create a bug or a task.

6. All your feedback captured is shown in the response form, bug, or task. Type a suitable title and,
optionally, select a star rating for the feature you've been testing.
7. Save your feedback. This create a work item in Azure DevOps containing all your feedback.
8. Continue to capture more feedback if required. You can submit multiple feedback responses, bugs, and
tasks for the same feedback request.
9. If you are a Stakeholder :
When you are done providing feedback, go to the Feedback requests page and choose
Feedback requests .

In the pending feedback requests page, mark the feedback request as Completed .
10. Choose the Stop icon to end your feedback session.

Provide feedback directly from the Test & Feedback extension

1. Open the Test & Feedback extension in your browser using the icon in the toolbar.
2. In the Connection settings page, choose Connected mode.

3. Connect to the server and the project or team that is requesting feedback.
4. Open the Feedback requests page to see all your feedback requests from the project or team you
connected to.

5. Select the feedback request you want to respond to and choose View feedback .
6. Read the instructions in the feedback request details page, then choose Provide feedback .

7. Capture and submit your feedback as shown above.

Related articles
Request stakeholder feedback using the Test & Feedback extension
Voluntary stakeholder feedback using the Test & Feedback extension
Track stakeholder feedback using the Test & Feedback extension
Exploratory test and submit feedback directly from your browser
Overview of manual and exploratory testing
 Get insights across your exploratory testing sessions
6/27/2022 • 3 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
View completed exploratory testing sessions and derive meaningful insights at team or individual level, and for
a specific period.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view or run manual or automated tests, you must have Basic access or higher.
To learn more, see Manual test access and permissions.
1. Open the Recent explorator y sessions page. You can do this:
From the Test & Feedback extension by choosing the "view" icon on the Session timeline page.

From the Test Plans web portal. by opening the Runs page and choosing Recent explorator y
sessions .

2. Explore the Recent explorator y sessions page. It contains three main sections:
Summar y view - shows a graphical breakdown of the work items explored, work items created, session
owners, and the total time for these sessions.
 Pivot view - shows a collapsible nested and sortable list of items grouped in different ways.

Details view - shows the work item selected in the Pivot view or a summary of information about a
selection of items.

Get insights from your exploratory testing sessions:

Use the Recent explorator y sessions page to get insights about your app from the information collected
during your exploratory testing sessions.
1. Set the scope for the data . Summary view provides the highest level view of your test results. Use it to
get insights into the overall effort and results of the exploratory testing sessions.
Use the View filter to scope the summary view to all sessions or just your own sessions.
Use the Period filter to scope the summary view to sessions in the period from the last 7 to the last
90 days.
2. Pivot the data on the type of work item . Pivot view lets you to focus on all the work items you
created in your exploratory testing sessions, or just on bugs, tasks, or test cases; and group the results in
different ways.
Use the Pivot filter to group the work items as a nested list based on those that have been explored,
those that have not been explored (requires a query), by the session in which they were created, or by
session owner.
Use the Show filter to show all items; or just bugs, tasks, or test cases.

3. Get deep insights from Details view . Details view gives insights into the items selected in Pivot view.
Depending on the type of item you select, you see the work item as an editable form, or a series of charts.
Select a row in Pivot view to see a summary of all the related information in Details view. For example,
if you have pivoted the list based on sessions, select a session to see a summary of all the information
from the work items in just that session.
Select a child row in Pivot view to display the work item form for that individual item. For example, if
you have pivoted the list based on explored work items, expand a work item and select a child bug,
task, or test case to see the work item form for just that item.
Discover work items not yet explored
Use a query to explore the work items that users have not yet explored.
1. Create a shared query in Azure DevOps that selects work items that can be explored using the Test &
Feedback extension, such as work items in the epic category, feature category, requirement category,
requirement-based suites, or test cases.

You must use a shared query. If this query returns a mix of supported and unsupported work items,
only those in supported categories will be displayed.

2. Use the View and Period filters to scope the view to the type of session (all sessions or just your own
sessions) and the time span (from the last 7 to the last 90 days). Then open the Quer y list and choose
Select quer y .

3. In the Quer y selector dialog, choose the shared query you created earlier.
4. View all the work items returned by the query in Summary view. You see a breakdown of explored and
unexplored work items, work items filed, sessions, and total session duration.

5. Open the Pivot list and choose Unexplored Work Item .

The view now shows only the unexplored work items.

See Also
Use the Test & Feedback extension in Connected mode
Add findings to existing bugs with exploratory testing
Explore work items with exploratory testing
Use the Test & Feedback extension in Standalone mode
Exploratory testing with Microsoft Test Manager
Overview of manual and exploratory testing
 Install the Test & Feedback extension
6/27/2022 • 3 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
The Test & Feedback extension helps teams perform exploratory testing and provide feedback. Everyone in
the team, such as developers, product owners, managers, UX or UI engineers, marketing teams, early adopters,
and other stakeholders can use the extension to submit bugs or provide feedback and contribute to the quality
of your product.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To view or run manual or automated tests, you must have Basic access or higher.
To learn more, see Manual test access and permissions.

Supported web browsers for the extension

The Test & Feedback extension is currently available for Google Chrome, Microsoft Edge (Chromium Only), and
Mozilla Firefox version 50.0 and higher.
Some browser versions do not currently support all the features of the Test & Feedback extension.

F EAT URE C H RO M E M IC RO SO F T EDGE F IREF O X

Capture screenshots with Yes Yes Yes

inline annotations

Capture notes Yes Yes Yes

Capture screen recordings Yes Yes No

Capture page load data Yes Yes No

Capture user actions log Yes Yes Yes

Capture system information Yes Yes No

Create bugs Yes Yes Yes

Create tasks and test cases Yes Yes Yes

Create feedback requests Yes Yes Yes

Export session report for Yes Yes Yes

sharing
 F EAT URE C H RO M E M IC RO SO F T EDGE F IREF O X

End-to-end traceability for Yes Yes Yes

work items

Simplified bug and task Yes Yes Yes

tracking and triaging

View and get insights from Yes Yes Yes

sessions

View similar existing bugs Yes Yes Yes

Test app on devices using Yes Yes No

cloud providers such as
Perfecto

Manage feedback requests Yes Yes Yes

For more information, see Visual Studio Marketplace, Azure DevOps tab.

Install the extension

1. Check the list of supported browsers and decide which you want to use.
2. Download and install your chosen browser, if you haven't already, then open it.
3. Go to Visual Studio Marketplace > Test & Feedback and choose Install .

4. Follow the instructions shown to install the Test & Feedback extension in your browser:
If you are using Google Chrome, choose the Install link for Chrome from the above image to
open the Google Chrome web store and follow the instructions to install the extension.

If you are using Microsoft Edge (Chromium), choose the Install link for Edge from the above
image to open the Microsoft Edge Add-ons page and follow the instructions to install the
extension.

If you are using Mozilla Firefox 50.0 and higher, choose the Install link for Firefox from the above
image to open the Firefox Browser Add-ons page and follow the instructions to install the
extension.
 You need to install the extension or add-on only once. Afterwards your browser will update it automatically.

Select an exploratory testing mode

1. Open the extension you installed in your browser by choosing the icon.

2. Decide if you want to use the extension in Connected or Standalone mode.

Connected mode
Available to all users of Azure DevOps and TFS 2015 or later:
 Users with Basic access or higher: Full capture and create capabilities to submit bugs, tasks, and test
cases. Includes collaboration capabilities such as end-to-end traceability, rich insights across
completed exploratory sessions, simplified tracking and triaging for bugs and tasks, and more.
Users with Stakeholder access: Full capture and create capabilities, except for test cases, to submit
feedback and respond to feedback requests from the team.
Feedback experience is available only in Azure DevOps and TFS 2017 or later.
Standalone mode
Available to everyone. No connection to Azure DevOps is required. Take notes and screenshots with inline
annotations to capture issues. Create bugs and export a session report to share findings.
If you have problems connecting to Azure DevOps, you may find the topic TF31002: Unable to connect useful.

Related articles
FAQs for manual testing

Next step
Use the Test & Feedback extension in Connected mode
 Exploratory testing with the Test & Feedback
extension in Connected mode
6/27/2022 • 4 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
To use the Test & Feedback extension in Connected mode you must connect to an Azure DevOps project. This
automatically configures the extension based on your access level:
Users with Basic access can use the extension to perform exploratory testing, as described below.
Users with Stakeholder access can use the extension to respond to feedback requests or to provide
feedback voluntarily. More details.
Users with Basic or Stakeholder access can use extension to respond to feedback requests sent by the
team by choosing the Provide feedback link in the email. More details.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To request or provide feedback, you must have Stakeholder access or higher.
To add or modify bugs or other work item types, you must have the Edit work items in this node
permission set to Allow under the corresponding Area Path .
To add new tags, you must have the Create tag definition permission set to Allow .
To learn more, see Set permissions and access for testing.

Connect to Azure DevOps

1. If you want to use Azure DevOps, and you haven't already done so, sign up for a subscription now. Make
sure you create a project when you create your subscription.
2. If you haven't already, install the Test & Feedback extension.
3. Open the extension in your web browser and select Connected mode.
4. Enter the Azure DevOps URL you want to connect to and choose Next .

If you are connecting for the first time, you may be prompted to sign in.
5. After connecting to the server, the extension shows all the collections, projects and teams in that server.
Select the project or team you want to connect to and choose Save .
 If there are many projects or teams, use the search textbox to find the one you need.
The extension is now ready to be used in Connected mode. Depending on your access level (Basic or
Stakeholder) you will see the appropriate UI for either exploratory testing or providing feedback. The extension
remembers your selection and remains connected until the session cookies expire or you explicitly disconnect
from the server.

Create bugs or tasks

After you have connected, you are ready to begin testing your app.
1. Start your exploratory testing session.

2. Open the web application you want to test, and start exploring it.
3. When you find an area that has a bug, take a screenshot of any part of the screen, make notes, or record
your actions as a video.

Some browsers may not provide all of the capture capabilities. See Supported web browsers for the
extension.

4. When you are done exploring and capturing information, create a bug or a task.
5. The bug or task form contains all your captured information. It also contains an image action log
describing your interactions with the page (such as mouse clicks, keyboard typing events, touch gestures,
and more) and page load data. Uncheck these options if you do not want to include this data in the bug
or task.

The image action log is the sequence of steps you took that led to the issue. It can be used to
reproduce the issue and understand the context. Page load data provides preliminary information
about the time it takes to load the pages, such as the resource timings and navigation timelines.

6. Enter a title for the bug or task and add any additional notes you require to the description. Then save the
bug or task.
 You can also add your findings to an existing similar bug.

7. View a list of all your activities in reverse chronological order in the Session timeline page. It shows all
the screenshots, videos, and notes you've captured, the work items such as bugs, tasks, and test cases
you've already filed, and the work items you've explored.

You can use the extension to explore work items in Azure DevOps.

8. To view a bug or task in Azure DevOps, choose the link in the session timeline.
 This opens the work item form in Azure DevOps.

How do I play the video recordings I created with the extension?

Create test cases

The extension lets you create test cases as you explore your application.
1. When you find a scenario where you want to create a test case, choose Create test case .

2. The test case form contains a list of all your actions up to this point while exploring the app (it reads them
from the image action log).
3. Enter a title for the test case and then edit it as required. For example, uncheck the action steps you don't
want to include in the test case, edit the captured text, and add the expected result. Then save the test
case.

4. Continue exploring the application. Create more bugs, tasks, or test cases as required.

End your testing session

1. When you're done, stop your session.
2. If you are using Azure DevOps, TFS 2017, or later version, open the Session timeline page and choose
the "view" icon to see your completed exploratory sessions in Azure DevOps.

Alternatively, open the Recent explorator y sessions list directly in the Runs page of the Test Plans
web portal.

See your exploratory session results

After you file bugs, create tasks, or create test cases, all these show up in the "Recent exploratory sessions" page
in Azure Test Plans.
See how you can view your sessions and get insights.
How do I play the video recordings I created with the extension?
 Exploratory testing with the Test & Feedback
extension in Standalone mode
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
All teams can use the Test & Feedback extension in Standalone mode. Users don't need an Azure DevOps
subscription or Team Foundation Server connection to use this mode.

Start testing in Standalone mode

1. If you haven't already, install the Test & Feedback extension.
2. Open the extension in your web browser and select Standalone mode.

3. Open the web application you want to explore and start the testing session.

4. When you find an area that has a bug, take a screenshot of the entire screen or any part of it.

5. You can annotate the screenshot using the tools available in the inline annotation toolbar.
6. Make notes about the issue to share with your team, and then save the note.

Create a bug
1. When you have finished capturing information for an issue, choose Create bug .

2. The bug form contains all your captured information. Enter a title for the bug and add any additional
notes you require to the description. Then save the bug.
3. View a list of all your activities in reverse chronological order in the Session timeline page. It shows all
the screenshots and notes you've captured and the bugs you've already created.

End your testing session

1. Continue exploring the application. Create more bugs as you encounter issues with the app.
2. When you're done, stop your session.
 The extension automatically creates a session report that contains details of all the bugs created during
the session, and any attachments.

3. The report is saved in the default Downloads folder of your web browser. Share it with the rest of your
team as an email attachment, or copy it to OneNote, Word, or in any other format you prefer.
How do I play the video recordings I created with the extension?

Next step
Use the extension in Connected mode
 Explore work items with the Test & Feedback
extension
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Use the Test & Feedback extension to explore existing work items and associate them with a new or an in-
progress exploratory session. After a work item is associated with a session, all new bugs, tasks and test cases
created in the current session are automatically linked to that work item. This enables end-to-end traceability,
and simplifies tracking and management of issues.
You can explore:
Work items belonging to a requirement category, a feature category, or an epic category
Requirements-based test suites and test cases.
You can explore a work item from the Kanban board or from the extension. You can also explore multiple work
items in the same session.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To request or provide feedback, you must have Stakeholder access or higher.
To add or modify bugs or other work item types, you must have the Edit work items in this node
permission set to Allow under the corresponding Area Path .
To add new tags, you must have the Create tag definition permission set to Allow .
To learn more, see Set permissions and access for testing.

Explore work items from the Kanban board

1. In the Kanban board, open the shortcut menu of the work item you want to explore, and choose Do
explorator y testing .
2. A banner in the Work hub shows which work item is associated with your session.

3. Launch the Test & Feedback extension. If there are acceptance criteria for the work item, these are shown.

If you have not already started a session, start one now. The work item is automatically associated with
the current or new session.
4. All bugs, tasks, and test cases you create will automatically be linked to the current work item.
Explore work items from the Test & Feedback extension
1. Open the Explore work item page in the extension and search for the work item you want to explore.

You can search using the work item identifier or keywords in the work item title.
2. Select the work item in the search results and choose Explore selected work item .
3. The work item is now associated with the in-progress session. If there are acceptance criteria, these are
shown.

4. All bugs, tasks, and test cases you create will automatically be linked to the current work item.
Explore multiple work items in the same session
To explore another work item, you must first dissociate the current work item from the in-progress session.
1. Open the Explore work item page and choose Change .

2. Associate the new work item with the in-progress session as described above.

See your exploratory session results

After you file bugs, create tasks, or create test cases, all these show up in the "Recent exploratory sessions" page
in Azure Test Plans.
See how you can view your sessions and get insights.
See Also
FAQs for manual testing
Use the Test & Feedback extension in Connected mode
Add findings to existing bugs with exploratory testing
Get insights across your exploratory testing sessions
Use the Test & Feedback extension in Standalone mode
Exploratory testing with Microsoft Test Manager
Overview of manual and exploratory testing
 Add findings to existing bugs with exploratory
testing
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
To help avoid duplication, the Test & Feedback extension automatically searches for and displays existing bugs,
based on the keywords in the title, as you file a new bug. You can choose to continue creating a new bug or add
your findings to an existing bug.

Prerequisites
You must connect to a project. If you don't have a project yet, create one.
You must be added to a project. To get added, Add users to a project or team.
To request or provide feedback, you must have Stakeholder access or higher.
To add or modify bugs or other work item types, you must have the Edit work items in this node
permission set to Allow under the corresponding Area Path .
To add new tags, you must have the Create tag definition permission set to Allow .
To learn more, see Set permissions and access for testing.
1. As you type the title for a new bug, in the background the extension searches for similar bugs that might
be related to the issue you've found and displays a link to the results. Choose this link to see the results
that have similar title keywords.

The form displays 0 Similar if it does not find any matching bugs. In this case, or if you don't see a
"similar" link, you can create a new bug containing your screenshots, notes, and videos as described in
 this topic.
2. If you see a bug you want to update, instead of creating a new one:
Select it in the list and choose Edit .

The extension appends all your screenshots, notes, and videos to the existing bug.
Save the updated bug.

3. If, instead, you decide not to update an existing bug, ignore the "similar" link and:
Choose the New bug link to return to the bug details form.
 Enter the details for the new bug and save it as described in this topic.
4. Continue exploring your app, filing bugs and tasks, and creating test cases.

See your exploratory session results

After you file bugs, create tasks, or create test cases, all these show up in the "Recent exploratory sessions" page
in Azure Test Plans.
See how you can view your sessions and get insights.

See Also
Use the Test & Feedback extension in Connected mode
Explore work items with exploratory testing
Get insights across your exploratory testing sessions
Use the Test & Feedback extension in Standalone mode
Exploratory testing with Microsoft Test Manager
Overview of manual and exploratory testing
 Voluntarily provide stakeholder feedback using the
Test & Feedback extension
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Stakeholders can respond to feedback requests for user stories and features generated in Azure DevOps using a
lightweight end-to-end flow based on the Test & Feedback extension. Only users with Basic access can request
feedback. Basic users can provide feedback using the flow described in this topic.
Request feedback
Provide feedback
Voluntary feedback
Track requests

NOTE
This lightweight end-to-end flow is applicable only for web apps and by using Azure DevOps. To get feedback for desktop
apps, or for earlier versions of TFS, use the feedback flow described in Get feedback about the Microsoft Feedback Client.

Provide voluntary feedback

You can use the Test & Feedback extension to provide feedback voluntarily, even if you haven't received a
specific feedback request.
1. Open the Test & Feedback extension in your browser using the icon in the toolbar.
2. In the Connection settings page, choose Connected mode.

3. Connect to the server and the project or team that is requesting feedback.
4. Start the exploratory testing session.

5. Open the application you want to provide feedback on and begin your feedback. For example, choose
Capture screenshot to take a screenshot.

You can use all the capabilities of the extension such as capturing screenshots, notes, and screen
recordings.

Some browsers may not provide all of the capture capabilities. See Supported web browsers for the
extension.

6. When you are done capturing feedback, Choose Provide feedback .

You can optionally choose to create bugs and tasks when you submit your feedback. The process is the
same as described here.
7. All your feedback captured is shown in the response form. Type a suitable title and, optionally, select a
star rating for the feature you've been testing.
 8. Save your feedback. This create a work item in Azure DevOps containing all your feedback.
9. Continue to capture more feedback if required. You can submit multiple feedback responses, bugs, and
tasks for the same feedback request.
10. Choose the Stop icon to end your feedback session.

Related articles
Request stakeholder feedback using the Test & Feedback extension
Provide stakeholder feedback using the Test & Feedback extension
Track stakeholder feedback using the Test & Feedback extension
Exploratory test and submit feedback directly from your browser
Overview of manual and exploratory testing
 Track stakeholder feedback
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
All feedback is captured in a Feedback Response work item. You can track feedback, whether captured by the
Test & Feedback extension or the Microsoft Feedback client, through a work item query.

Prerequisites
By default, all project members and users with Stakeholder access can view and run all shared queries. You
can change the permissions set for a shared query folder or shared query. For details, see Set query
permissions.
To add and save a query under Shared queries , you must be granted Basic access or higher. Also, you must
have your Contribute permission set to Allow for the folder you want to add the query to. By default, the
Contributors group doesn't have this permission.

NOTE
Users with Stakeholder access for a public project have full access to query features just like users with Basic access. For
details, see Stakeholder access quick reference.

By default, all project members and users with Stakeholder access can view and run all shared queries. You
can change the permissions set for a shared query folder or shared query. For details, see Set query
permissions.
To add and save a query under Shared queries , you must be granted Basic access or higher. Also, you must
have your Contribute permission set to Allow for the folder you want to add the query to. By default, the
Contributors group doesn't have this permission.

Track feedback requests

To view feedback, use the Feedback shared query. Select your project and open Boards > Queries . Under
Queries , select All . In the Shared Queries, select Feedback .
This query displays a list of all the feedback responses received. To learn more, see Web portal navigation.
To create a feedback query, follow these steps:
1. Select Boards > Queries and then select New quer y .
2. In the Editor for your new query, enter the following values:

Team Project = @Project

Work Item Type In Group [Link]
State = Active
3. Select Save quer y and enter a name.
4. Select Run quer y to see a list of active feedback responses for your team project.
5. Select a response work item to see the details of the feedback.
1. Select your project and open Boards>Queries or Work>Queries . To learn how, see Web portal
navigation.
2. In the list of shared queries, select Feedback . This query displays a list of all the feedback responses
received.

Or, create a feedback query with the parameters, as shown.

3. You should see a list of all active feedback responses for your team project.
4. Open the response work item to see the details of the feedback.

Related articles
What is Azure Test Plans?
Request feedback using the Test & Feedback extension
Get feedback
Provide feedback using the Test & Feedback extension
Define a work item query
 Get feedback
6/27/2022 • 4 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Once you have working software, you're ready to get feedback from your stakeholders. You can ask reviewers to
provide videos, screenshots, type-written comments, and ratings. Their feedback is captured into work items
that you can review and use to create a bug or suggest a new backlog item.
Before requesting feedback, make sure that you provide stakeholders who'll you request feedback from the
necessary permissions.

NOTE
You can also request feedback from stakeholders for web apps using the Test & Feedback extension. For desktop apps,
you must use the feedback request form documented in this topic and stakeholders must reply using the Microsoft
Feedback Client.

Prerequisites
You must connect to a team project. If you don't have a project yet, create one in Azure DevOps Services.
You must be added to a project as a member of the Contributors or Project Administrators security
group. To get added, Add users to a project or team.
To request feedback, you must be granted Basic access or higher. For details, see About access levels.
To provide or review feedback, you must be granted Stakeholder access or higher.
To view or modify feedback responses, you must have your View work items in this node and Edit work
items in this node permissions set to Allow . By default, the Contributors group has this permission set.
To learn more, see Set permissions and access for work tracking.

NOTE
Users with Stakeholder access for a public project have full access to the Request feedback feature just like users with
Basic access. For details, see About access levels.

You must connect to a project. If you don't have a project yet, create one.
You must be added to a project as a member of the Contributors or Project Administrators security
group. To get added, Add users to a project or team.
To request feedback, you must be granted Basic access or higher. For details, see About access levels.
To provide or review feedback, you must be granted Stakeholder access or higher.
To view or modify feedback responses, you must have your View work items in this node and Edit work
items in this node permissions set to Allow . By default, the Contributors group has this permission set.
To learn more, see Set permissions and access for work tracking.
To send feedback requests, the server administrator must configure an SMTP server.

Add the Other links widget to your dashboard

Add the Other links widget to a web portal team dashboard. For details, see Add widgets to a dashboard
Request feedback
To request feedback, you fill out a form that generates an email request to your stakeholders.
1. From the dashboard, choose the Request feedback link from the Other links widget.

2. Add the feedback reviewers. If you don't see the names you want in the browse list, grant them
permissions to provide feedback.

3. Tell your reviewers how to run the app they'll be reviewing.

4. For each area of interest, decide what type of feedback you want. Set the context for the reviewers by
providing enough background information. Add up to four more areas of interest with the add
feedback item link.
5. Send the request.

1. From the dashboard, choose the Request feedback link from the Other links widget.

If the following message appears, you need to configure an SMTP server.

2. Add the feedback reviewers. If you don't see the names you want in the browse list, grant them
permissions to provide feedback.

3. Tell your reviewers how to run the app they'll be reviewing.

4. For each area of interest, decide what type of feedback you want. Set the context for the reviewers by
providing enough background information. Add up to four more areas of interest with the add
feedback item link.

5. Send the request.

Provide feedback
Reviewers launch your application and provide feedback through the free Microsoft Feedback Client.
1. Reviewers who don't have a version of Visual Studio installed can download the feedback client directly
from the feedback request they receive.

Or, they can go to the Visual Studio download site.

2. Reviewers start the feedback session.
3. They launch the app to review from the feedback tool.

4. They begin providing feedback.

5. Reviewers can add screenshots, comments, and file attachments, and even record the feedback session.
Results show up on the lower part of the screen. In this case, you can see the comment that the
stakeholder wrote after attaching the screenshot.
 NOTE
Security Note: Unless you stop recording, everything is recorded—all steps that you take as well as anything
you say. If you provide sensitive data such as user names and passwords, you will capture this information in the
recording. However, you can always delete a recording by deleting the image for the recording session that
appears in the feedback tool's text box.

6. Reviewers can modify or even delete parts of their feedback, such as a recording, before they submit
their feedback.

Review feedback
1. Open the Feedback query.
 Or, create a feedback query with the parameters, as shown.

You should see a list of all active feedback responses for your team project.

2. Open a response item and play or save a recording.

3. Or, you can create a bug or backlog item linked to the feedback.

With the feedback experience, you can engage stakeholders frequently to provide continuous feedback.
Interacting with your working apps, your stakeholders can record rich and actionable data that the
system automatically stores in the form of video or audio recordings, comments, and annotated
screenshots. You can then take action on each feedback response by assigning it to a team member or
creating bugs or backlog items to the linked feedback.

Related notes
You can change the audio device or annotation tool using the Settings icon change settings icon on the
Microsoft Feedback Client.
If you access the Microsoft Feedback Client from a remote machine, you can enable remote audio.
You can install the Test & Feedback extension from the Marketplace, Test & Feedback.
 Give feedback using Microsoft Feedback Client
6/27/2022 • 6 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
You can respond to a request for feedback using the Microsoft Feedback Client. This tool allows you to launch an
application, capture your interaction with it as video and capture your verbal or type-written comments as well.
To support traceability, your feedback is stored in the data store for Azure DevOps Services or an on-premises
Team Foundation Server (TFS).
You can install the install the Test & Feedback extension from the Marketplace, Test & Feedback.

Initiate a feedback session

When you receive an email request for feedback, it contains a link to launch Microsoft Feedback Client, and a
link to install the feedback tool if it is not already installed.
To install Microsoft Feedback Client
You can skip this procedure if you already have Microsoft Feedback Client installed or if you will access the
feedback tool on a remote machine.
1. Open your email request, and then choose the Install the feedback tool link to install Microsoft Feedback
Client.

If you are viewing the email from an internet-based email client, you might need to copy the address
behind the link and paste it into the address bar of your browser.
2. Restart your computer.
To initiate a feedback session from an email request
1. Open your email request, and then choose the Star t your local feedback session link

NOTE
If you are viewing the email from an internet-based email client, you might need to copy the address behind the
link and paste it into the address bar of your browser.

If you're accessing the feedback tool on a remote computer, open the shortcut menu for the Star t your
local feedback session link, and then choose Copy Hyperlink . On the remote computer, choose
Star t , choose Run , paste the hyperlink into the box, and then choose the OK button.
The Feedback Client launches and the Windows Security dialog box appears.
2. Enter the credentials provided to you for connecting with Team Foundation Server, and then choose the
 OK button.

Launch the application

The feedback tool opens to the Start page. Launch the application by following the instructions provided.

1. From the Start page, choose the Application link to open, start, or install the application for which you
have been requested to provide feedback.
2. Follow any additional instructions provided to sign in or access the application.
For example, you would choose the [Link] link and then enter
the user name and password provided.

Provide feedback on requested items

You can record your interactions with the application, record verbal comments, enter text, clip and annotate a
screen, or attach a file. Recordings appear as images within the feedback tool's text box.
To review descriptions of items submitted for feedback
On the Provide page, one or more items appear for you to provide feedback. For each item, you can get context
on what's being asked and then you can give feedback free form through video or audio recordings, text,
screenshot, or file attachments. When finished with one item, choose the Next button to move to the next item.

1. On the toolbar, choose the number of the ITEM that you want to review.
2. Review the information provided.
3. Interact with the application, and determine what feedback you want to provide, based on the instructions
for each item's request.
To start, stop, or delete a screen or voice recording

You can change settings defined for the audio device and annotation tool at any time. For more information, see
Change the audio device or annotation tool.
1. To star t recording: Choose one of the icons: Screen & Voice , Screen only , or Voice only .

IMPORTANT
Security Note: Unless you stop recording, all steps that you take and remarks that you make while recording
screen and voice will be recorded. If you provide sensitive data such as user names and passwords, you will
capture this information in the recording. However, you can always delete a recording by deleting the image for
the recording session that appears in the feedback tool's text box.

2. To stop recording : Choose the Stop button.

The recording stops, and an image appears in the text box that indicates the type of recording and the
length of the recording session.
3. To delete a recording : Choose the image for the recording that you want to delete, and then press the
Delete key.
To add text, capture a screenshot, or attach a file
You can add text, capture a screenshot, annotate a screenshot, or attach a file as part of your feedback. You can
perform these operations while you continue to record your feedback.
By annotating screenshots, you can indicate corrections or improvements by adding text or images to the
screenshot that you captured. By default, Microsoft Paint opens automatically when you open a screenshot
image that you captured within the feedback tool. You can also configure another annotation tool to open
automatically whenever you capture a screenshot. For more information, see Change the audio device or
annotation tool.

1. To add text : In the tool's rich-text box, enter the information that you want to include when you submit
your feedback.
 You can use the toolbar to format your comment, add a hyperlink, and insert an image.
2. To capture a screenshot : Choose the camera icon, and then select the rectangular area that you want
to capture.
The image appears in the rich-text box.
3. To annotate the screenshot : Double-click the image.
Microsoft Paint, or whichever annotation tool you have configured, opens with the image displayed. To
specify corrections or improvements, perform the following steps:
a. Draw or enter text onto the image.
b. Choose the Save icon, and close the annotation tool.
The image appears within the feedback tool with the changes that you made.
4. To attach a file : Choose the attachment icon, and in the Open dialog box, browse to the file that you
want to attach, and then choose the Open button.
To rate an item and move to the next item

1. (Optional) Choose one or more stars to indicate your overall rating of the item that you have just
reviewed.
2. Choose the Next button to move to the next item to review.

Review and submit your feedback

After you enter your feedback for each item, you can review, make corrections or additions, and then submit
your feedback.
1. Choose the Submit button, which also stops any recordings that you may have started.
2. (Optional) On the Submit page, choose the review feedback link to return to the Provide page for the
first feedback item. Review your feedback for each item and choose the Next button to advance to the
next item.
You can delete or modify any content within the text box for any item.
3. On the Submit page, choose the Submit and close button.
Microsoft Feedback Client closes, and the system creates a feedback response work item which contains
the feedback that you submit with the title, "Feedback Response from YourUserName for
RequestFeedbackItemTitle."

Related notes
You Initiate a feedback request from the home page of your web portal or using the Other links widget from
a team dashboard.
You can change the audio device or annotation tool using the Settings icon change settings icon on the
Microsoft Feedback Client.
If you access the Microsoft Feedback Client from a remote machine, you can enable remote audio.
To record audio, you must have an audio recording device configured on your computer. If you access the
feedback tool from a remote device, you might have to enable remote audio capture.
Requirements to provide feedback
To provide feedback using the Microsoft Feedback Client you must have security credentials that will allow
you to connect to Azure DevOps Services or an on-premises TFS and have the permissions described in Give
reviewers permissions to provide feedback.
Your computer must meet the system requirements for installing Microsoft Feedback Client. For more
information, see the following page on the Microsoft website: Microsoft Feedback Client Download.
 Track stakeholder feedback
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
All feedback is captured in a Feedback Response work item. You can track feedback, whether captured by the
Test & Feedback extension or the Microsoft Feedback client, through a work item query.

Prerequisites
By default, all project members and users with Stakeholder access can view and run all shared queries. You
can change the permissions set for a shared query folder or shared query. For details, see Set query
permissions.
To add and save a query under Shared queries , you must be granted Basic access or higher. Also, you must
have your Contribute permission set to Allow for the folder you want to add the query to. By default, the
Contributors group doesn't have this permission.

NOTE
Users with Stakeholder access for a public project have full access to query features just like users with Basic access. For
details, see Stakeholder access quick reference.

By default, all project members and users with Stakeholder access can view and run all shared queries. You
can change the permissions set for a shared query folder or shared query. For details, see Set query
permissions.
To add and save a query under Shared queries , you must be granted Basic access or higher. Also, you must
have your Contribute permission set to Allow for the folder you want to add the query to. By default, the
Contributors group doesn't have this permission.

Track feedback requests

To view feedback, use the Feedback shared query. Select your project and open Boards > Queries . Under
Queries , select All . In the Shared Queries, select Feedback .
This query displays a list of all the feedback responses received. To learn more, see Web portal navigation.
To create a feedback query, follow these steps:
1. Select Boards > Queries and then select New quer y .
2. In the Editor for your new query, enter the following values:

Team Project = @Project

Work Item Type In Group [Link]
State = Active
3. Select Save quer y and enter a name.
4. Select Run quer y to see a list of active feedback responses for your team project.
5. Select a response work item to see the details of the feedback.
1. Select your project and open Boards>Queries or Work>Queries . To learn how, see Web portal
navigation.
2. In the list of shared queries, select Feedback . This query displays a list of all the feedback responses
received.

Or, create a feedback query with the parameters, as shown.

3. You should see a list of all active feedback responses for your team project.
4. Open the response work item to see the details of the feedback.

Related articles
What is Azure Test Plans?
Request feedback using the Test & Feedback extension
Get feedback
Provide feedback using the Test & Feedback extension
Define a work item query
 Give reviewers permissions to provide feedback
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
You must grant permissions to provide feedback to users that you plan to request feedback from. Reviewers
who aren't members of your team require special permissions to provide feedback using the Microsoft
Feedback Client.

Add reviewers to your team project

1. From the web portal of your team project home page, open the administration context.

If you aren't a member of the Project Administrators group, get added. See Change project-level
permissions. You need to be a member in order to add users and groups to a team project, change
permissions, and grant them access to the web portal.
2. Create a group for your reviewers.

TIP
If you have a lot of reviewers, create an Azure Devops security group to help you manage permissions more
efficiently. To learn how, see Add or remove users or groups, manage security groups.

3. Name your group.

4. Add accounts to your group.

Set permissions so reviewers can provide feedback

Allow reviewers to Create test runs , View project-level information , and View test runs .
Set permissions so reviewers can modify work items
Since feedback is captured in a feedback response work item, reviewers need to be able to modify work items in
the product areas they will review.
1. Open security for the team project.

2. Add the reviews group to the VSO Groups or TFS Groups .

3. Allow reviewers to Edit work items in this node and View work items in this node.

If you want, allow reviewers the ability to modify their feedback

submissions
Sometimes additional ideas occur after reviewers submit their feedback. By providing access to the web portal,
reviewers can revisit and further annotate their feedback submissions.
Azure DevOps Ser vices: Assign the Stakeholder license to accounts that you add to your Reviewer group
On-premises Azure DevOps Ser ver : Add your Reviewer group to the Stakeholder group on the access
levels page. If you don't see this tab, get administrative permissions.
Your reviewers will be able to view and modify only those work items that they create, which includes feedback
responses. The Stakeholder group provides limited access to features and data for those members of your
organization who do not have an Azure DevOps client access license (CAL).

Related articles
Initiate a feedback request
Respond to a feedback request
Work as a stakeholder
 Enable remote audio capture
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
To record audio, you must have an audio recording device configured on your computer, or on a remote
machine if you access Microsoft Feedback Client, Test Runner, or Exploratory Testing from a remote device.
To record audio as part of a feedback or testing session on a remote machine that is running Microsoft Feedback
Client, Test Runner, or Exploratory Testing window, you must configure audio redirection settings on the remote
machine. You must also enable record from this computer when you connect to the remote machine.

IMPORTANT
We recommend that you open Microsoft Feedback Client on your local computer. Redirection will decrease audio quality.
Microsoft Feedback Client does not require you to record audio as part of the feedback session.

To set a default microphone on your local computer

To record audio from your local computer, you must set a microphone device as the default device.
1. Choose Star t, Control Panel .
2. In the Control Panel window, choose the Hardware and Sound link, and then choose Sound .
3. In the Sound window, choose the Recording tab.
4. Select a recording device from the list provided and then choose Set Default .

To enable remote audio capture

IMPORTANT
If you edit the registry incorrectly, you might severely damage your system. Before you make any changes to the registry,
you should back up any valued data on the computer.

1. Choose Star t , choose Run , enter regedit , choose the OK button, and then set the value of the following
registry key to 0 .
HKLM\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp\fDisableAudioCapture

For more information about how to set values of registry keys, see the following page on the Microsoft
website: How to add, modify, or delete registry subkeys and values by using a registration entries (.reg)
file.
2. To connect to the remote machine, choose Star t , choose Run , enter mstsc , and then choose the OK
button.
The Remote Desktop Connection dialog box opens.
3. In the Computer list, choose or enter the name of the computer to which you want to connect, and, in
the User name box, enter your user name.
4. On the Local Resources tab, choose the Settings button.
5. Under Remote audio recording , choose the Record from this computer option button, and then
choose the OK button.
6. Choose the Connect button.
7. Exit and restart whichever client tool that you are using, for example Microsoft Feedback Client, Test
Runner, or Exploratory Testing.
You must restart you client in order for it to register the audio settings.

Related articles
Exploratory testing
Run your tests
Provide feedback using the Microsoft Feedback Manager
 Change the audio device or annotation tool
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
You can change the default settings used by Microsoft Feedback Client for the annotation tool or audio device.
The annotation tool that you set automatically opens when you double-click an image within the feedback tool,
and the audio device is used when you start a Screen & Voice or a Voice only recording. You might want to
make a change if you do not have the default tools available on the computer that you use to provide feedback.
To change a setting, first launch Microsoft Feedback Client. For more information, see Provide feedback.
To change the annotation tool or annotation settings
1. On the toolbar at the top of Microsoft Feedback Client, choose the Change settings button.
2. In the Change settings dialog box, choose the Browse button.
3. In the Open dialog box, navigate to the annotation tool and executable file that you want, and then
choose the Open button.
4. (Optional) Enter any arguments that can be passed to the annotation tool.
For example, %f represents the file name to pass to the tool.
5. (Optional) Select the After taking screenshot, immediately open in the specified annotation
tool check box to enable the annotation tool to open automatic.
6. Choose the Save button.
To change the audio device
1. On the toolbar at the bottom of Microsoft Feedback Client, choose the Change settings icon.
2. In the Change settings dialog box, choose the Change button to change the default audio device.
3. In the Open dialog box, navigate to the audio device that you want, select the Default audio device
check box, and then choose the Open button.
4. Choose the Save button.

Related articles
Provide feedback
 Track test status
6/27/2022 • 7 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
View the status of planned tests or monitor testing progress by defining test case or test result charts. To learn
more about test planning, see Create test plans and test suites. For information about test result terms, see Test
objects and terms.
To track the progress of more than one test plan or test suite, open the Progress Report.

Track testing progress

Use test results charts to track how your testing is going. Choose from a fixed set of pre-populated fields related
to results.
Add your own charts for test results to visualize what's important for your team. If you already know how to add
a chart, jump to the examples below of charts that you can create.
To add a chart, follow these steps:
1. Select Test Plans to open the Test Plans page. Select a test plan to open it, then select the Char ts tab.
2. Under Char ts , select New > New test result char t .

3. Select the chart type, in this example, a pie chart. Based on the chart, configure the fields that you want to
use to group by, or for the rows and columns.
 All charts roll up the information for any child test suites of the test plan or test suite that you selected.
4. Select OK to save the chart. The chart is displayed in the Char ts page for your test plan or test suite.

By default, a pie chart is created for each test plan. This chart is grouped by the outcome field to show the latest
results for all the tests in the test plan. View this default chart from the Char ts page.
To add a chart, follow these steps:
1. Select the test plan or test suite for your chart in the Test Plans page. Then create a new chart.

2. Select the chart type. Based on the chart, configure the fields that you want to use to group by, or for the
rows and columns.
 All charts roll up the information for any child test suites of the test plan or test suite that you selected.
3. Save the chart. Now it will be displayed in the Char ts page for the test plan or test suite that you
selected.

Test results examples

This section includes examples of useful charts.
Check test status for a test suite
You can check the test status for a specific test suite.
Select Test Plans to open the Test Plans page, and then select a plan to view a test suite. Select New > New
test result char t to create a test results pie chart. Group by Outcome .
Check status for user stories per sprint
If you created requirement-based test suites in your test plan for your user stories, you can create a chart to
review status for user stories that your team is testing in the current sprint.
Create a static test suite that groups your requirement-based test suites together. Select that static test suite in
the Test Plans page. Add a test results stacked bar chart.
Select Suite as the Y-axis and Outcome as Group by .
Check tests for each tester
You can check how many tests that each tester left to run.
Select your test plan in the Test Plans page and add a test results pivot table chart. Choose Tester as Rows and
Outcome as Columns pivot.
Check quality based on the configuration
Use either a stacked bar chart or a pivot table chart. Specify Configuration and Outcome as pivots.
Track why tests are failing
You can track why tests are failing for your team.
For failure analysis, use either a stacked bar chart or a pivot table chart. Select Tester and Failure type .
 NOTE
Failure type for test results can only be set using Microsoft Test Manager.

Track resolutions for failing tests

You can track the resolution for failing tests for your team.
For resolution analysis, use either a stacked bar chart or a pivot table chart. Choose Tester and Resolution .

NOTE
Resolution type for test results can only be set using Microsoft Test Manager.

Track test case status

Use test case charts to find out the progress of your test case authoring. The charts for test cases give you the
flexibility to report on columns that you add to the tests page. By default, test case fields aren't added to the view
in the tests page.
If you already know how to add a chart, jump to the examples below of charts that you can create for test cases.
1. From the Define or Execute tabs, add any fields you want to use for your test case chart by using
Column Options . Those fields appear as choices in the lists for grouping for your test case charts.
2. Select the test plan or test suite for your chart in the Test Plans page. Under Char ts , select New > New
test case char t .

All charts roll up the information for any child test suites of the test plan or test suite that you selected.
3. Select the chart type. Based on the chart, configure the fields that you want to use to group by, for rows
and columns, or the range for trend charts.
 You can't group by test suite for the test case charts.
4. Select OK to save the chart. The chart is displayed in the Char ts page for your test plan or test suite.
1. Add any fields you want to use for your test case chart from the tests page with Column options. Then
the fields will appear as choices in the drop-down lists for grouping for your test case charts.
2. Select the test plan or test suite for your chart in the Test Plans page. Then add a test case chart.

All charts roll up the information for any child test suites of the test plan or test suite that you selected.
3. Select the chart type. Based on the chart, configure the fields that you want to use to group by, for rows
and columns, or the range (trend charts only).
 You can't group by test suite for the test case charts.
4. Save the chart. Now it will be displayed in the charts page for the test plan or test suite that you selected.

Test case examples

These examples show common ways to interact with test case results.
View recent test results
You can view the recent test results for an individual test case.
Select the Execute tab in a test suite and then select a test case. Select More options or right-click to open the
context menu. Select View test result .
Select the test case within a test suite and then choose to view the test details pane.

View the recent test results for this test case.

Track burn down by using a stacked area chart
You can track burn down for test case creation.
Create a stacked area trend chart to view the burn down for how many test cases are ready to be run. Select
State for the stack by field and Ascending for the sort field.
You can track burn down for automation status.
Use a stacked area trend chart to view the burn down for how many test cases have been automated. Select
Automation status for the Stack by field and Ascending for the Sor t field.
See test ownership and priorities
If multiple teams own test cases in your test plan, you can see how many test cases each team owns and the
priorities of the tests.
If your teams are organized by area path, use a test case pie chart. Select Area path for Group by .
If you want to know the priorities of these tests, create a stacked bar chart. Select Area path for the Y-axis and
Priority for Group by .
Track test creation status
You can track test creation status for team members by creating a chart that includes the Assigned to value.
Use a stacked bar chart or a pivot table chart. Select Assigned to for Y-axis or Rows . Select State for Group
by or Columns .

Share charts on your team's dashboard

Pin a chart to your team's dashboard for all the team to view. Use the chart's context menu.
You can configure the dashboard widget to show a range of chart types.

NOTE
You must be a team administrator to configure the dashboard widget, but team members with Stakeholder access can
view the charts on the dashboard.

Related articles
Progress report
Control how long to keep test results
FAQs for manual testing
Widget catalog
Team administration
 Progress Report
6/27/2022 • 4 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020

To track the progress of more than one test plan or test suite, use the Progress Report. It helps you track the
team's progress with respect to planned testing of your product or service by answering the following
questions:
How much testing is complete?
What is the current status how many tests have passed, failed or are blocked?
Is the testing likely to complete in time?
What is the rate of execution on a daily basis?
Get a breakdown by test plan and suite to understand where execution is going slow or has many failed or
blocked tests.
View the progress each suite has made within a test plan and identify the areas that need attention.

NOTE
The Progress Report lets you view data for one or more Test Plans in a single project. To view data across projects, use
the OData APIs. This report is also rendered using the same OData APIs.
The report, always shows you the data for the test plan you last accessed. Filter selections aren't stored.
Report data corresponds to the the current test suite hierarchy in the selected test plans. Hierarchy history isn't stored.
Report data is updated approximately every 15 minutes. Do not use this report for real-time analysis or reporting.
Expect ~15 mins of duration between a test execution and the same to show up in the report.
Data for test plans migrated from an on-premises Azure DevOps server won't show up in this report.

Prerequisites
You must be a member of a project with Basic access or higher. If you don't have a project yet, create one. For
more information, see Sign up for free.
If you haven't been added as a project member, get added now. Anyone with access to the project, except
stakeholders, can view Analytics views.
Test Plans must be enabled. If it is disabled, the Test Plans>Progress Repor t page won't display. To re-
enable Test Plans , see Turn an Azure DevOps service on or off
Have the View Analytics permission set to Allow . For more information, see Grant permissions to access
the Analytics service.
You must be a member of a project with Basic access or higher. If you haven't been added as a project
member, get added now. Anyone with access to the project, except Stakeholders, can view Analytics views.
Verify that Analytics] is installed, and if not, then enable it. You must be the Organization owner or a
member of the Project Collection Administrators group to add extensions or enable the service.
Test Plans must be enabled. If it is disabled, the Test Plans>Progress Repor t page won't display. To re-
enable Test Plans , see Turn an Azure DevOps service on or off
Have the View Analytics permission set to Allow . For more information, see Grant permissions to access
the Analytics service.

Configure the report

Open the report from the web portal by going to Test Plans>Progress Repor t . The report shows you the
status of the test plan you last accessed. However, using the filter bar, you can select one or more test plans in
the project.

Understand the report

The progress report consists of three sections:
1. Summar y : This section provides you with a consolidated view for the selected test plans. To understand
what test points are, see Navigate Test Plans.
2. Outcome trend : This graph renders a daily snapshot to give you an execution and status trendline. It can
show data for 14 days (default), 30 days, or a custom range of your choice. Data for today may or may not
appear in this chart based on the time of the day you are looking at the report.
3. Details : This section lets you drill down by each test plan and gives you summary of each test suite in it. The
section also lets you navigate to a test plan or suite of choice by double-clicking on it.

An example of a healthy outcome trend is as follows because as time is progressing, the number of not run tests
are decreasing (because tests are being executed) and the number of passed tests are on the rise.

An example of an unhealthy outcome trend is as follows because as time is progressing, there is no significant
execution occurring and the # of passed and failed tests are continuing to remain flat. In such situations, use the
details card to drill down and take suitable actions.

Filter
Use the filter barto filter by Test Suites , Configuration , Tester , Test Case Priority , and Test Case Assigned
To . For example, you can filter by configuration 'Chrome' and then 'Edge' to understand where the execution and
pass % are higher. Note: Tester is the person to whom the test point is assigned for execution whereas Assigned
to is the person who is responsible for the reusable test case.

NOTE
The test plan is considered Level 1. Its child suites are considered level 2. Their child suites in turn are considered level 3.
The Test Suites filter shows only the Level 3 test suites inside the selected test plans. To select a Level 2 test suite select all
the Level 3 test suites underneath it. This assumes the Level 2 test suites itself does not have any test points.

Related articles
Control how long to keep test results
FAQs for manual testing
 Review code coverage results
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018

Code coverage helps you determine the proportion of your project's code that is actually being tested by tests
such as unit tests. To increase your confidence of the code changes, and guard effectively against bugs, your
tests should exercise - or cover - a large proportion of your code.
Reviewing the code coverage result helps to identify code path(s) that are not covered by the tests. This
information is important to improve the test collateral over time by reducing the test debt.

NOTE
In Microsoft Team Foundation Server (TFS) 2018 and previous versions, build and release pipelines are called definitions,
runs are called builds, service connections are called service endpoints, stages are called environments, and jobs are called
phases.

Example
To view an example of publishing code coverage results for your choice of language, see the Ecosystems
section of the Pipelines topics. For example, collect and publish code coverage for JavaScript using Istanbul.

View results
The code coverage summary can be viewed in the build timeline view. The summary shows the overall
percentage of line coverage.
 NOTE
Merging code coverage results from multiple test runs is limited to .NET and .NET Core at present. This will be supported
for other formats in a future release.

The code coverage summary can be viewed on the Summar y tab on the pipeline run summary.

The results can be viewed and downloaded on the Code coverage tab.
 NOTE
In a multi-stage YAML pipeline, the code coverage results are only available after the completion of the entire pipeline.
This means that you may have to separate the build stage into a pipeline of its own if you want to review the code
coverage results prior to deploying to production.

NOTE
Merging code coverage results from multiple test runs is limited to .NET and .NET Core at present. This will be supported
for other formats in a future release.

Artifacts
The code coverage artifacts published during the build can be viewed under the Build ar tifacts published
milestone in the timeline view.

The code coverage artifacts published during the build can be viewed under the Summar y tab on the pipeline
run summary.

If you use the Visual Studio Test task to collect coverage for .NET and .NET Core apps, the artifact contains
.coverage files that can be downloaded and used for further analysis in Visual Studio.
 If you publish code coverage using Cobertura or JaCoCo coverage formats, the code coverage artifact
contains an HTML file that can be viewed offline for further analysis.

NOTE
For .NET and .NET Core, the link to download the artifact is available by choosing the code coverage milestone in the build
summary.

Tasks
Publish Code Coverage Results publishes code coverage results to Azure Pipelines or TFS, which were
produced by a build in Cobertura or JaCoCo format.
Built-in tasks such as Visual Studio Test, .NET Core, Ant, Maven, Gulp, Grunt, and Gradle provide the option to
publish code coverage data to the pipeline.

Help and support

See our troubleshooting page
Get advice on Stack Overflow, and get support via the Developer Community
 Pipeline reports
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019
Teams track their pipeline health and efficiency to ensure continuous delivery to their customers. You can gain
visibility into your team's pipeline(s) using Pipeline analytics. The source of information for pipeline analytics is
the set of runs for your pipeline. These analytics are accrued over a period of time, and form the basis of the rich
insights offered. Pipelines reports show you metrics, trends, and can help you identify insights to improve the
efficiency of your pipeline.

Prerequisites
Ensure that you have installed the Analytics Marketplace extension for Azure DevOps Server.

View pipeline reports

A summary of the pass rate can be viewed in the Analytics tab of a pipeline. To drill into the trend and insights,
click on the card to view the full report.

A summary of the pass rate and duration can be viewed in the Analytics tab of a pipeline. To drill into the trend
and insights, click on the card to view the full report.
Pipeline pass rate report
The Pipeline pass rate report provides a granular view of the pipeline pass rate and its trend over time. You
can also view which specific task failure contributes to a high number of pipeline run failures, and use that
insight to fix the top failing tasks.
The report contain the following sections:
Summar y : Provides the key metrics of pass rate of the pipeline over the specified period. The default
view shows data for 14 days, which you can modify.

Failure trend : Shows the number of failures per day. This data is divided by stages if multiple stages are
applicable for the pipeline.

Top failing tasks & their failed runs : Lists the top failing tasks, their trend and provides pointers to
their failed runs. Analyze the failures in the build to fix your failing task and improve the pass rate of the
pipeline.
Pipeline duration report
The Pipeline duration report shows how long your pipeline typically takes to complete successfully. You can
review the duration trend and analyze the top tasks by duration to optimize the duration of the pipeline.
Test failures report
The Test failures report provides a granular view of the top failing tests in the pipeline, along with the failure
details. For more information on this report, see Test failures.

Filters
Pipelines reports can be further filtered by date range or branch.
Date range : The default view shows data from the last 14 days. The filter helps change this range.
 Branch filter : View the report for a particular branch or a set of branches.

Help and support

See our troubleshooting page
Get advice on Stack Overflow, and get support via the Developer Community
 Test Analytics
6/27/2022 • 3 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019
Tracking test quality over time and improving test collateral is key to maintaining a healthy DevOps pipeline. Test
analytics provides near real-time visibility into your test data for builds and releases. It helps improve the
efficiency of your pipeline by identifying repetitive, high impact quality issues.

NOTE
Test analytics is currently available only with Azure Pipelines.

Read the glossary to understand test reports terminology.

Install the Analytics extension if required

For more information, see The Analytics Marketplace extension.

View test analytics for builds

To help teams find and fix tests that fail frequently or intermittently, use the top failing tests report. The build
summary includes the Analytics page that hosts this report. The top-level view provides a summary of the test
pass rate and results for the selected build pipeline, for the specified period. The default range is 14 days.

View test analytics for releases

For tests executing as part of release, access test analytics from the Analytics link at the top right corner. As
with build, the summary provides an aggregated view of the test pass rate and results for the specified period.
Test Failures
Open a build or release summary to view the top failing tests report. This report provides a granular view of the
top failing tests in the pipeline, along with the failure details.

The detailed view contains two sections:

Summar y : Provides key quantitative metrics for the tests executed in build or release over the specified
period. The default view shows data for 14 days.
Pass rate and results: Shows the pass percentage, along with the distribution of tests across
various outcomes.

Failing tests: Provides a distinct count of tests that failed during the specified period. In the
 example above, 986 test failures originated from 124 tests.

Chart view: A trend of the total test failures and average pass rate on each day of the specified
period.

Results : List of top failed tests based on the total number of failures. Helps to identify problematic tests
and lets you drill into a detailed summary of results.

Group test failures

The report view can be organized in several different ways using the group by option. Grouping test results can
provide deep insights into various aspects of the top failing tests. In the example below, the test results are
grouped based on the test files they belong to. It shows the test files and their respective contribution towards
the total of test failures, during the specified period to help you easily identify and prioritize your next steps.
Additionally, for each test file, it shows the tests that contribute to these failures.
Drill down to individual tests
After you have identified one or more tests in the Details section, select the individual test you want to analyze.
This provides a drill-down view of the selected test with a stacked chart of various outcomes such as passed or
failed instances of the test, for each day in the specified period. This view helps you infer hidden patterns and
take actions accordingly.
The corresponding grid view lists all instances of execution of the selected test during that period.
Failure analysis
To perform failure analysis for root causes, choose one or more instances of test execution in the drill-down
view to see failure details in context.
Infer hidden patterns
When looking at the test failures for a single instance of execution, it is often difficult to infer any pattern. In the
example below, the test failures occurred during a specific period, and knowing this can help narrow down the
scope of investigation.
Another example is tests that exhibit non-deterministic behavior (often referred to as flaky tests). Looking at an
individual instance of test execution may not provide any meaningful insights into the behavior. However,
observing test execution trends for a period can help infer hidden patterns, and help you resolve the failures.

Report information source

The source of information for test analytics is the set of published test results for the build or release pipeline.
These result are accrued over a period of time, and form the basis of the rich insights that test analytics provides.

Help and support

See our troubleshooting page
Get advice on Stack Overflow, and get support via the Developer Community
 Review test results
6/27/2022 • 10 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Automated tests can be configured to run as part of a build or release for various languages. Test reports
provide an effective and consistent way to view the tests results executed using different test frameworks, in
order to measure pipeline quality, review traceability, troubleshoot failures and drive failure ownership. In
addition, it provides many advanced reporting capabilities explored in the following sections.
You can also perform deeper analysis of test results by using the Analytics Service. For an example of using this
with your build and deploy pipelines, see Analyze test results.
Read the glossary to understand test report terminology.

NOTE
Test report is available in TFS 2015 and above, however the new experience described in this topic is currently available
only in Azure Pipelines.

NOTE
In Microsoft Team Foundation Server (TFS) 2018 and previous versions, build and release pipelines are called definitions,
runs are called builds, service connections are called service endpoints, stages are called environments, and jobs are called
phases.

Published test results can be viewed in the Tests tab in a build or release summary.

Surface test results in the Tests tab

Test results can be surfaced in the Tests tab using one of the following options:
Automatically inferred test results . By default, your pipeline can automatically infer the test output
for a few popular test runners. This is done by parsing the error logs generated during the build
operation and then checking for signatures of test failures.
Currently, Azure DevOps supports the following languages and test runners for automatically inferring
the test results:
JavaScript - Mocha, Jest and Jasmine
Python- Unittest

NOTE
This inferred test report is a limited experience. Some features available in fully-formed test reports are not
present here (more details). We recommend that you publish a fully-formed test report to get the full Test
and Insights experience in Pipelines. Also see:

Publishing fully-formed test reports for JavaScript test runners

Publishing fully-formed test reports for Python test runners
 Test execution tasks . Built-in test execution tasks such as Visual Studio Test that automatically publish
test results to the pipeline, or others such as Ant, Maven, Gulp, Grunt, and Xcode that provide this
capability as an option within the task.
Publish Test Results task . Task that publishes test results to Azure Pipelines or TFS when tests are
executed using your choice of runner, and results are available in any of the supported test result formats.
API(s) . Test results published directly by using the Test Management API(s).

Surface test information beyond the Tests tab

The Tests tab provides a detailed summary of the test execution. This is helpful in tracking the quality of the
pipeline, as well as for troubleshooting failures. Azure DevOps also provides other ways to surface the test
information:
The Dashboard provides visibility of your team's progress. Add one or more widgets that surface test
related information:
Requirements quality
Test results trend
Deployment status
Test analytics provides rich insights into test results measured over a period of time. It can help identify
problematic areas in your test by providing data such as the top failing tests, and more.

View test results in build

The build summary provides a timeline view of the key steps executed in the build. If tests were executed and
reported as part of the build, a test milestone appears in the timeline view. The test milestone provides a
summary of the test results as a measure of pass percentage along with indicators for failures and abor ts if
these exist.
View test results in release
In the pipeline view you can see all the stages and associated tests. The view provides a summary of the test
results as a measure of pass percentage along with indicators for failures and abor ts if these exist. These
indicators are same as in the build timeline view, giving a consistent experience across build and release.
Tests tab
Both the build and release summaries provide details of test execution. Choose Test summar y to view the
details in the Tests tab. This page has the following sections
Summar y : provides key quantitative metrics for the test execution such as the total test count, failed
tests, pass percentage, and more. It also provides differential indicators of change compared to the
previous execution.
Results : lists all tests executed and reported as part of the current build or release. The default view
shows only the failed and aborted tests in order to focus on tests that require attention. However, you can
choose other outcomes using the filters provided.
Details : A list of tests that you can sort, group, search, and filter to find the test results you need.
Select any test run or result to view the details pane that displays additional information required for
troubleshooting such as the error message, stack trace, attachments, work items, historical trend, and more.

TIP
If you use the Visual Studio Test task to run tests, diagnostic output logged from tests (using any of [Link],
[Link] or [Link] methods), will appear as an attachment for a failed test.

The following capabilities of the Tests tab help to improve productivity and troubleshooting experience.
Filter large test results
Over time, tests accrue and, for large applications, can easily grow to tens of thousands of tests. For these
applications with very many tests, it can be hard to navigate through the results to identify test failures,
associate root causes, or get ownership of issues. Filters make it easy to quickly navigate to the test results of
your interest. You can filter on Test Name , Outcome (failed, passed, and more), Test Files (files holding tests)
and Owner (for test files). All of the filter criteria are cumulative in nature.

Additionally, with multiple Grouping options such as Test run , Test file , Priority , Requirement , and more,
you can organize the Results view exactly as you require.

Test debt management with bugs

To manage your test debt for failing or long running tests you can create a bug or add data to existing bug and
all view all associated work items in the work item tab.
Immersive troubleshooting experience
Error messages and stack traces are lengthy in nature and need enough real estate to view the details during
troubleshooting. To provide an immersive troubleshooting experience, the Details view can be expanded to full
page view while still being able to perform the required operations in context, such as bug creation or
requirement association for the selected test result.
Troubleshooting data for Test failure
For the test failures, the error messages and stack traces are available for troubleshooting. You can also view all
attachments associated with the test failure in the Attachments tab.
Test debt management
You can create or add to an existing bug to manage test debt for failures or long running tests. The Work Items
tab details all bugs and requirements associated with a Test to help you analyze the requirement impact as well
know status and who is working on the bug.

Test trends with historical data

History of test execution can provide meaningful insights into reliability or performance of tests. When
troubleshooting a failure, it is valuable to know how a test has performed in the past. The Tests tab provides test
history in context with the test results. The test history information is exposed in a progressive manner starting
with the current build pipeline to other branches, or the current stage to other stages, for build and release
respectively.

View execution of in-progress tests

Tests, such as integration and functional tests, can run for a long time. Therefore, it is important to see the
current or near real-time status of test execution at any given time. Even for cases where tests run quickly, it's
useful to know the status of the relevant test result(s) as early as possible; especially when failures occur. The in-
progress view eliminates the need to wait for test execution to finish. Results are available in near real-time as
execution progresses, helping you to take actions faster. You can debug a failure, file a bug, or abort the pipeline.
 NOTE
The feature is currently available for both build and release, using Visual Studio Test task in a Multi Agent job. It will be
available for Single Agent jobs in a future release.

The view below shows the in-progress test summary in a release, reporting the total test count and the
number of test failures at a given point in time. The test failures are available for troubleshooting, creating
bug(s), or to take any other appropriate action.
View summarized test results
During test execution, a test might spawn multiple instances or tests that contribute to the overall outcome.
Some examples are, tests that are rerun, tests composed of an ordered combination of other tests (ordered
tests) or tests having different instances based on an input parameter (data driven tests).
As these tests are related, they must be reported together with the overall outcome derived from the individual
instances or tests. These test results are reported as a summarized test result in the Tests tab:
Rerun failed tests : The ability to rerun failed tests is available in the latest version of the Visual Studio
Test task. During a rerun, multiple attempts can be made for a failed test, and each failure could have a
different root cause due to the non-deterministic behavior of the test. Test reports provide a combined
view for all the attempts of a rerun, along with the overall test outcome as a summarized unit.
Additionally the Test Management API(s) now support the ability to publish and query summarized test
results.

Data driven tests : Similar to the rerun of failed tests, all iterations of data driven tests are reported
under that test in a summarized view. The summarized view is also available for ordered tests
(.orderedtest in Visual Studio).
 NOTE
Metrics in the test summary section, such as the total number of tests, passed, failed, or other are computed using the
root level of the summarized test result.

View aborted tests

Test execution can abort due to several reasons such as bad test code, errors in the source under test, or
environmental issues. Irrespective of the reason for the abort, it is important to be able to diagnose the behavior
and identify the root cause. The aborted tests and test runs can be viewed alongside the completed runs in the
Tests tab.
 NOTE
The feature is currently available for both build and release, using the Visual Studio Test task in a Multi Agent job or
publishing test results using the Test Management API(s). It will be available for Single Agent jobs in a future release.

Automatically inferred test results

Azure DevOps can automatically infer the output of tests that are running in your pipelines for a few supported
test frameworks. These automatically inferred test reports require no specific configuration of your pipelines,
and are a zero-effort way to get started using Test Reporting.

See the list of runners for which test results are automatically inferred.
As only limited test metadata is present in such inferred reports, they are limited in features and capabilities. The
following features are not available for inferred test reports:
Group the test results by test file, owner, priority, and other fields
Search and filter the test results
Check details of passed tests
Preview any attachments generated during the tests within the web UI itself
Associate a test failure with a new bug, or see list of associated work items for this failure
See build-on-build analytics for testing in Pipelines
 NOTE
Some runners such as Mocha have multiple built-in console reporters such as dot-matrix and progress-bar. If you have
configured a non-default console output for your test runner, or you are using a custom reporter, Azure DevOps will not
be able to infer the test results. It can only infer the results from the default reporter.

Related articles
Analyze test results
Trace test requirements
Review code coverage results

Help and support

See our troubleshooting page
Get advice on Stack Overflow, and get support via the Developer Community
 Requirements traceability
6/27/2022 • 6 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Requirements traceability is the ability to relate and document two or more phases of a development
process, which can then be traced both forward or backward from its origin. Requirements traceability help
teams to get insights into indicators such as quality of requirements or readiness to ship the
requirement . A fundamental aspect of requirements traceability is association of the requirements to test
cases, bugs and code changes.

Read the glossary to understand test report terminology.

NOTE
In Microsoft Team Foundation Server (TFS) 2018 and previous versions, build and release pipelines are called definitions,
runs are called builds, service connections are called service endpoints, stages are called environments, and jobs are called
phases.

Agile teams running automated tests

Agile teams have characteristics including, but not limited to the following
Faster release cycles
Continuous testing in a pipeline
Negligible manual testing footprint; limited to exploratory testing
High degree of automation
The following sections explore traceability from Quality , Bug and Source standpoints for Agile teams.

Quality traceability
To ensure user requirements meet the quality goals, the requirements in a project can be linked to test results,
which can then be viewed on the team's dashboard. This enables end-to-end traceability with a simple way to
monitor test results. To link automated tests with requirements, visit test report in build or release.
1. In the results section under Tests tab of a build or release summary, select the test(s) to be linked to
requirements and choose Link .
2. Choose a work item to be linked to the selected test(s) in one of the specified way:
Choose an applicable work item from the list of suggested work items. The list is based on the most
recently viewed and updated work items.
Specify a work item ID.
Search for a work item based on the title text.

The list shows only work items belonging to the Requirements category.

3. After the requirements have been linked to the test results you can view the test results grouped by
requirement. Requirement is one of the many "Group by" options provided to make it easy to navigate
the test results.
4. Teams often want to pin the summarized view of requirements traceability to a dashboard. Use the
Requirements quality widget for this.

5. Configure the Requirements quality widget with the required options and save it.
Requirements quer y : Select a work item query that captures the requirements, such as the user
stories in the current iteration.
Quality data : Specify the stage of the pipeline for which the requirements quality should be traced.
6. View the widget in the team's dashboard. It lists all the Requirements in scope, along with the Pass
Rate for the tests and count of Failed tests. Selecting a Failed test count opens the Tests tab for the
selected build or release. The widget also helps to track the requirements without any associated test(s).

To ensure user requirements meet the quality goals, the requirements in a project can be linked to test results,
which can then be viewed on the team's dashboard. This enables end-to-end traceability with a simple way to
monitor test results. To link automated tests with requirements, visit test report in build or release.
1. In the results section under Tests tab of a build or release summary, select the test(s) to be linked to
requirements and choose Link .
2. Choose a work item to be linked to the selected test(s) in one of the specified way:
Choose an applicable work item from the list of suggested work items. The list is based on the most
recently viewed and updated work items.
Specify a work item ID.
Search for a work item based on the title text.

The list shows only work items belonging to the Requirements category.

3. Teams often want to pin the summarized view of requirements traceability to a dashboard. Use the
Requirements quality widget for this.
4. Configure the Requirements quality widget with the required options and save it.
Requirements quer y : Select a work item query that captures the requirements, such as the user
stories in the current iteration.
Quality data : Specify the stage of the pipeline for which the requirements quality should be traced.

5. View the widget in the team's dashboard. It lists all the Requirements in scope, along with the Pass
Rate for the tests and count of Failed tests. Selecting a Failed test count opens the Tests tab for the
selected build or release. The widget also helps to track the requirements without any associated test(s).
Bug traceability
Testing gives a measure of the confidence to ship a change to users. A test failure signals an issues with the
change. Failures can happen for many reasons such as errors in the source under test, bad test code,
environmental issues, flaky tests, and more. Bugs provide a robust way to track test failures and drive
accountability in the team to take the required remedial actions. To associate bugs with test results, visit test
report in build or release.
1. In the results section of the Tests tab select the tests against which the bug should be created and choose
Bug . Multiple test results can be mapped to a single bug. This is typically done when the reason for the
failures is attributable to a single cause such as the unavailability of a dependent service, a database
connection failure, or similar issues.

2. Open the work item to see the bug. It captures the complete context of the test results including key
information such as the error message, stack trace, comments, and more.
3. View the bug with the test result, directly in context, within the Tests tab. The Work Items tab also lists
any linked requirements for the test result.

4. From a work item, navigate directly to the associated test results. Both the test case and the specific test
result are linked to the bug.
5. In the work item, select Test case or Test result to go directly to the Tests page for the selected build or
release. You can troubleshoot the failure, update your analysis in the bug, and make the changes required
to fix the issue as applicable. While both the links take you to the Tests tab , the default section shown are
Histor y and Debug respectively.

Source traceability
When troubleshooting test failures that occur consistently over a period of time, it is important to trace back to
the initial set of changes - where the failure originated. This can help significantly to narrow down the scope for
identifying the problematic test or source under test. To discover the first instance of test failures and trace it
back to the associated code changes, visit Tests tab in build or release.
1. In the Tests tab, select a test failure to be analyzed. Based on whether it's a build or release, choose the
Failing build or Failing release column for the test.

2. This opens another instance of the Tests tab in a new window, showing the first instance of consecutive
failures for the test.

3. Based on the build or release pipeline, you can choose the timeline or pipeline view to see what code
changes were committed. You can analyze the code changes to identify the potential root cause of the test
failure.
Traditional teams using planned testing
Teams that are moving from manual testing to continuous (automated) testing, and have a subset of tests
already automated, can execute them as part of the pipeline or on demand (see test report). Referred to as
Planned testing , automated tests can be associated to the test cases in a test plan and executed from Azure
Test Plans . Once associated, these tests contribute towards the quality metrics of the corresponding
requirements.

Help and support

See our troubleshooting page
Get advice on Stack Overflow, and get support via the Developer Community
 Historical graph for agent pools (Preview)
6/27/2022 • 4 minutes to read • Edit Online

Azure DevOps Ser vices

The pool consumption report enables you to view jobs running in your agent pools graphed with agent pool job
concurrency over a span of up to 30 days. You can use this information to help decide whether your jobs aren't
running because of concurrency limits. If you have many jobs queued or running jobs at the concurrency or
online agents limit, you may wish to purchase additional parallel jobs or provision more self-hosted agents.

Prerequisites
IMPORTANT
You must be a member of the Project Collection Administrators group to view the pool consumption reports for agent
pools in an organization, including project level reports in that organization.

Historical graph for agent pools is in preview. To enable or disable this preview feature, navigate to Preview
features, find Historical graph for agent pools , and choose the desired setting.

Pool consumption report

The pool consumption report is part of the Analytics tab for an agent pool and contains the following charts,
depending on the agent pool type.

C H A RT T Y P E DESC RIP T IO N A GEN T P O O L T Y P E

Public hosted concurrency Displays concurrency, queued jobs, and Microsoft-hosted

running jobs for public projects

Private hosted concurrency Displays concurrency, queued jobs, and Microsoft-hosted

running jobs for private projects

Agent usage Displays online agents, queued jobs, Scale set agent and self-hosted
and running jobs for self-hosted
agents

Private self-hosted concurrency Displays concurrency, queued jobs, and Scale set agent and self-hosted
running jobs for private self-hosted
projects

The charts in the pool consumption report graph the following data points:
Concurrency - The number of parallel jobs in the organization that apply to the project type (public or
private) and agent pool type (Microsoft-hosted or self-hosted). For more information, see Configure and pay
for parallel jobs.
 Online agents - The number of agents online in a self-hosted agent pool or a scale set agent pool.
Queued jobs - The number of jobs queued and waiting for an agent.
Running jobs - The number of running jobs.
Pool data is aggregated at a granularity of 10 minutes, and the number of running jobs is plotted based on the
maximum number of running jobs for the specified interval of time. Because multiple short-running jobs may
complete within the 10 minute timeline, the count of running jobs may sometimes be higher than the
concurrency or online agents during that same period.

Report scope
The pool consumption report can be displayed at organization scope, or project scope. At the organization level,
the chart is plotted using data from pipelines across any project within the organization that have run jobs in
that pool. At the project level, the chart is plotted using data from pipelines in that particular project that have
run jobs in that pool.
To view the pool consumption report at the organization level, choose Organization settings , Pipelines ,
Agent pools .
To view the pool consumption report at the project level, navigate to the desired project and choose Project
settings , Pipelines , Agent pools .
From the Agent pools view, choose the desired pool, and view the Analytics tab. The following example shows
the pool consumption report for a self-hosted agent pool.
This example shows the usage graphs for the Azure Pipelines Microsoft-hosted agent pool.

Filtering
To adjust the timeline of the graph, choose Filter , select the interval drop-down, and choose the desired
interval.
For the 1 day interval, you can view data per hour, and for the other intervals you can view it per day. Pool data
is aggregated at a granularity of 10 minutes, and the number of running jobs is plotted based on the maximum
number of running jobs for the specified interval of time. In this example there are two online agents, but in
some areas there are four running jobs due to the way the pool data is aggregated.

FAQ
Where does the pool consumption report get the data it displays?
The pool consumption report uses the Azure DevOps Analytics service and the TaskAgentRequestSnapshots
endpoint. You can query this endpoint using the following URL prefix:
[Link] .
You can retrieve the project_id for your project by navigating to the following URL:
[Link] .
The following example shows a sample query and response.
 {
"@[Link]": "[Link]
preview/$metadata#TaskAgentRequestSnapshots",
"[Link]@[Link]": "#Collection(String)",
"@[Link]": [
"VS403507: The specified query does not include a $select or $apply clause which is recommended for all
queries. Details on recommended query patterns are available here: [Link]
linkid=861060."
],
"value": [
{
"SamplingDateSK": 20201117,
"SamplingHour": 13,
"SamplingTime": "2020-11-17T13:10:00-08:00",
"QueuedDate": "2020-11-17T13:07:26.22-08:00",
"QueuedDateSK": 20201117,
"StartedDate": "2020-11-17T15:02:23.7398429-08:00",
"StartedDateSK": 20201117,
"FinishedDate": "2020-11-17T15:13:49.89-08:00",
"FinishedDateSK": 20201117,
"QueueDurationSeconds": 6897.519,
"ProjectSK": "...",
"PipelineSK": 5141,
"RequestId": 6313,
"PoolId": 28,
"PipelineType": "Build",
"IsHosted": true,
"IsRunning": false,
"IsQueued": true
},
...

For more information on query options, see Query guidelines for Analytics with OData.

NOTE
The TaskAgentRequestSnapshots endpoint is in preview and not yet documented but you can view information about
the data returned by navigating to the endpoint URL:
[Link] .

Why are there more running jobs than there are agents or concurrency?
Pool data is aggregated at a granularity of 10 minutes, and the number of running jobs is plotted based on the
maximum number of running jobs for the specified interval of time. Each running job is counted separately, and
if multiple jobs complete during the 10 minute interval they contribute to the total count of running jobs for that
interval.
What is the difference between viewing the graphs in Project settings vs Organization settings?
The scope of the data in the graph is determined based on whether the chart is accessed through Project
settings or Organization settings . At the organization level, the chart is plotted using data from pipelines
across any project within the organization that have run jobs in that pool. At the project level, the chart is plotted
using data from pipelines in that particular project that have run jobs in that pool.
 Configure the Test Results Trend (Advanced) widget
6/27/2022 • 3 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019
Teams track their test collateral health to ensure effective continuous testing in a pipeline. For example, teams
might track:
Test pass percentage
Test failures
Test duration
With the Test Results Trend widget, you can monitor test trends over a period of time, detect patterns around
test failures, test duration, and more, and generate actionable insights.
Example Test Results Trend widget showing pass percentage and test failure for last 7 days

In this article you'll learn:

The type of insights you can get by monitoring Test Results Trend charts
Prerequisites for gaining actionable insights from the Test Results Trend charts
How to install and configure the Test Results Trend widget

NOTE
The Test Results Trend (Advanced) widget is based on Analytics. Analytics is generally available for Azure DevOps Services
and in preview as an extension for Azure DevOps Server 2019. For TFS 2017 and TFS 2018, you can use the Test Results
Trend widget.
Insights supported with Test Results Trend charts
With the Test Results Trend charts, you can gain the following insights:
Identify if test health is improving over time by monitoring trends of failures and average pass rate on each
day
Identify long running tests that are impacting a pipeline's efficiency by monitoring the average test duration
on each day
Identify patterns in test outcomes. Has the test recently started to fail? Or, has the test always failed for the
selected period? Or, is the test showing non-deterministic behavior?
Obtain insights into specific areas of interest to you based on test file, branch, or stage you own by
configuring specific filters.
Obtain insights into a specific area you test by configuring the widget to focus on a test file, branch, or stage.
Test Results Trend charts require that you set up continuous testing in your build pipeline. To get deeper insights
and data you can view the Test Failure report in the pipeline. To learn more, see details Analyze test results.

Prerequisites
To configure the Test Results Trend widget:
You must be a member of a project. If you haven't been added as a project member, get added now.
Set up continuous testing for your build pipeline. For details, see Run unit tests with your builds.
To add a widget to a team dashboard, you need to be a member of the team. You must have Basic access or
greater, have dashboard permissions, or be a team admin or project admin. Default settings provide all team
members with permissions.
Enable Pipelines . If disabled, none of the Analytics widgets for pipelines will display. To re-enable it, see Turn
an Azure DevOps service on or off.
Enable Test Plans . If disabled, the Test Results Trend widget displays an "Access not supported" error
message. To enable test plans, see Turn an Azure DevOps service on or off.
You must be a member of a project. If you haven't been added as a project member, get added now.
Set up continuous testing for your build pipeline. For details, see Run unit tests with your builds.
Have installed the Analytics Marketplace extension. You must be a member of the Project Collection
Administrators group to add extensions.
To add a widget to a team dashboard, you need to be a member of the team. You must have Basic access or
greater, have dashboard permissions, or be a team admin or project admin.
Enable Pipelines . If disabled, none of the Analytics widgets for pipelines will display. To re-enable it, see Turn
an Azure DevOps service on or off.

Configure the Test Results Trend (Advanced) widget

You can configure your Test Results Trend (Advanced) widget to show results for either build or release pipelines.
1. From your team's dashboard, select the actions icon for the Test Results Trend (Advanced) widget you
want to configure and select Configure .
Modify the Title and select either Build or Release for the type of Pipelines that you'll select.
 Select the plus icon to add one or more pipelines.
2. Next, select the Period , and then choose the metrics that you want to track. Optionally, apply filters for
Branch , Test file , Owner , and Test run .
3. Select Save when you're done.
The following image shows a chart with pass rate and test results for last seven days.
Next steps
Test Analytics report.
 Speed up testing by using Test Impact Analysis (TIA)
6/27/2022 • 6 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018

NOTE
Applies only to TFS 2017 Update 1 and later, and Visual Studio 2015 Update 3 and later.

Continuous Integration (CI) is a key practice in the industry. Integrations are frequent, and verified with an
automated build that runs regression tests to detect integration errors as soon as possible. However, as the
codebase grows and matures, its regression test suite tends to grow as well - to the extent that running a full
regression test might require hours. This slows down the frequency of integrations, and ultimately defeats the
purpose of continuous integration. In order to have a CI pipeline that completes quickly, some teams defer the
execution of their longer running tests to a separate stage in the pipeline. However, this only serves to further
defeat continuous integration.
Instead, enable Test Impact Analysis (TIA) when using the Visual Studio Test task in a build pipeline. TIA performs
incremental validation by automatic test selection. It will automatically select only the subset of tests required to
validate the code being committed. For a given code commit entering the CI/CD pipeline, TIA will select and run
only the relevant tests required to validate that commit. Therefore, that test run will complete more quickly, if
there is a failure you will get to know about it sooner, and because it is all scoped by relevance, analysis will be
faster as well.

Test Impact Analysis has:

A robust test selection mechanism . It includes existing impacted tests, previously failing tests, and newly
added tests.
Safe fallback . For commits and scenarios that TIA cannot understand, it will fall back to running all tests. TIA
is currently scoped to only managed code, and single machine topology. So, for example, if the code commit
contains changes to HTML or CSS files, it cannot reason about them and will fall back to running all tests.
Configurable overrides . You can run all tests at a configured periodicity.
However, be aware of the following caveats when using TIA with Visual Studio 2015:
Running tests in parallel . In this case, tests will run serially.
Running tests with code coverage enabled . In this case, code coverage data will not get collected.

NOTE
In Microsoft Team Foundation Server (TFS) 2018 and previous versions, build and release pipelines are called definitions,
runs are called builds, service connections are called service endpoints, stages are called environments, and jobs are called
phases.

Test Impact Analysis supported scenarios

At present, TIA is supported for:
TFS 2017 Update 1 onwards, and Azure Pipelines
Version 2.* of the Visual Studio Test task in the build pipeline
Build vNext, with multiple VSTest Tasks
VS2015 Update 3 onwards on the build agent
Local and hosted build agents
CI and in PR workflows
Git, GitHub, Other Git, TFVC repos (including partially mapped TFVC repositories with a workaround)
IIS interactions (over REST, SOAP APIs), using HTTP/HTTPS protocols
Automated Tests
Single machine topology. Tests and app (SUT) must be running on the same machine.
Managed code (any .NET Framework app, any .NET service)
At present, TIA is not supported for:
Multi-machine topology (where the test is exercising an app deployed to a different machine)
Data driven tests
Test Adapter-specific parallel test execution
.NET Core
UWP
More information about TIA scope and applications

Enable Test Impact Analysis

TIA is supported through Version 2.* of the Visual Studio Test task. If your app is a single tier application, all you
need to do is to check Run only impacted tests in the task UI. The Test Impact data collector is automatically
configured. No additional steps are required.
If your application interacts with a service in the context of IIS, you must also configure the Test Impact data
collector to run in the context of IIS by using a .runsettings file. Here is a sample that creates this configuration:

<?xml version="1.0" encoding="utf-8"?>

<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<!-- This is the TestImpact data collector.-->
<DataCollector uri="datacollector://microsoft/TestImpact/1.0"
assemblyQualifiedName="[Link],
[Link], Version=[Link], Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
friendlyName="Test Impact">
<Configuration>
<!-- enable IIS data collection-->
<InstrumentIIS>True</InstrumentIIS>
<!-- file level data collection -->
<ImpactLevel>file</ImpactLevel>
<!-- any job agent related executable or any other service that the test is using needs to be
profiled. -->
<ServicesToInstrument>
<Name>TeamFoundationSshService</Name>
</ServicesToInstrument>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
View Test Impact Analysis outcome
TIA is integrated into existing test reporting at both the summary and details levels, including notification
emails.

More information about TIA and Azure Pipelines integration

Manage Test Impact Analysis behavior

You can influence the way that tests are either included or ignored during a test run:
Through the VSTest task UI . TIA can be conditioned to run all tests at a configured periodicity. Setting this
option is recommended, and is the means to regulate test selection.
 By setting a build variable . Even after TIA has been enabled in the VSTest task, it can be disabled for a
specific build by setting the variable DisableTestImpactAnalysis to true . This override will force TIA to run
all tests for that build. In subsequent builds, TIA will go back to optimized test selection.
When TIA opens a commit and sees an unknown file type, it falls back to running all tests. While this is good
from a safety perspective, tuning this behavior might be useful in some cases. For example:
Set the TI_IncludePathFilters variable to specific paths to include only these paths in a repository for which
you want TIA to apply. This is useful when teams use a shared repository. Setting this variable disables TIA for
all other paths not included in the setting.
Set the TIA_IncludePathFilters variable to specify file types that do not influence the outcome of tests and
for which changes should be ignored. For example, to ignore changes to .csproj files set the variable to the
value !**\*.csproj .

Use the minimatch pattern when setting variables, and separate multiple items with a semicolon.

To evaluate whether TIA is selecting the appropriate tests:

Manually validate the selection. A developer who knows how the SUT and tests are architected could
manually validate the test selection using the TIA reporting capabilities.
Run TIA selected tests and then all tests in sequence. In a build pipeline, use two test tasks - one that runs
only impacted Tests (T1) and one that runs all tests (T2). If T1 passes, check that T2 passes as well. If there was
a failing test in T1, check that T2 reports the same set of failures.
More information about TIA advanced configuration

Provide custom dependency mappings

TIA uses dependency maps of the following form.

TestMethod1
dependency1
dependency2
TestMethod2
dependency1
dependency3

TIA can generate such a dependencies map for managed code execution. Where such dependencies reside in .cs
and .vb files, TIA can automatically watch for commits into such files and then run tests that had these source
files in their list of dependencies.
You can extend the scope of TIA by explicitly providing the dependencies map as an XML file. For example, you
might want to support code in other languages such as JavaScript or C++, or support the scenario where tests
and product code are running on different machines. The mapping can even be approximate, and the set of tests
you want to run can be specified in terms of a test case filter such as you would typically provide in the VSTest
task parameters.
The XML file should be checked into your repository, typically at the root level. Then set the build variable
[Link] to point to it. For example, if the file is named [Link] , set the variable to
$([Link] y)/[Link] .
For an example of the XML file format, see TIA custom dependency mapping.

See Also
TIA overview and VSTS integration
 TIA scope and applications
TIA advanced configuration
TIA custom dependency mapping

Help and support

See our troubleshooting page
Get advice on Stack Overflow, and get support via the Developer Community
 Manage flaky tests
6/27/2022 • 3 minutes to read • Edit Online

Azure DevOps Ser vices

Productivity for developers relies on the ability of tests to find real problems with the code under development
or update in a timely and reliable fashion. Flaky tests present a barrier to finding real problems, since the failures
often don't relate to the changes being tested. A flaky test is a test that provides different outcomes, such as pass
or fail, even when there are no changes in the source code or execution environment. Flaky tests also impact the
quality of shipped code.

NOTE
This feature is only available on Azure DevOps Services. Typically, new features are introduced in the cloud service first,
and then made available on-premises in the next major version or update of Azure DevOps Server. To learn more, see
Azure DevOps Feature Timeline.

The goal of bringing flaky test management in-product is to reduce developer pain cause by flaky tests and cater
to the whole workflow. Flaky test management provides the following benefits.
Detection - Auto detection of flaky test with rerun or extensibility to plug in your own custom detection
method
Management of flakiness - Once a test is marked as flaky, the data is available for all pipelines for that
branch
Repor t on flaky tests - Ability to choose if you want to prevent build failures caused by flaky tests, or
use the flaky tag only for troubleshooting
Resolution - Manual bug-creation or manual marking and unmarking test as flaky based on your
analysis
Close the loop - Reset flaky test as a result of bug resolution / manual input
Enable flaky test management
To configure flaky test management, choose Project settings , and select Test management in the Pipelines
section.
Slide the On/Off button to On .
The default setting for all projects is to use flaky tests for troubleshooting.
Flaky test detection
Flaky test management supports system and custom detection.
System detection : The in-product flaky detection uses test rerun data. The detection is via VSTest task
rerunning of failed tests capability or retry of stage in the pipeline. You can select specific pipelines in the
project for which you would like to detect flaky tests.

NOTE
Once a test is marked as flaky, the data is available for all pipelines for that branch to assist with troubleshooting
in every pipeline.

Custom detection : You can integrate your own flaky detection mechanism with Azure Pipelines and use
the reporting capability. With custom detection, you need to update the test results metadata for flaky
tests. For details, see Test Results, Result Meta Data - Update REST API.
Flaky test options
The Flaky test options specify how flaky tests are available in test reporting as well as resolution capabilities,
as described in the following sections.

Flaky test management and reporting

On the Test management page under Flaky test options , you can set options for how flaky tests are included
in the Test Summary report. Flaky test data for both passed and failed test is available in Test results. The Flaky
tag helps you identify flaky tests. By default, flaky tests are included in the Test Summary. However, if you want
to ensure flaky test failures don't fail your pipeline, you can choose to not include them in your test summary
and suppress the test failure. This option ensures flaky tests (both passed and failed) are removed from the pass
percentage and shown in Tests not repor ted , as shown in the following screenshot.
 NOTE
The Test summary report is updated only for Visual Studio Test task and Publish Test Results task. You may need to add a
custom script to suppress flaky test failure for other scenarios.

Tests marked as flaky

You can mark or unmark a test as flaky based on analysis or context, by choosing Flaky (or UnFlaky , depending
on whether the test is already marked as flaky.)

When a test is marked flaky or unflaky in a pipeline, no changes are made in the current pipeline. Only on future
executions of that test is the changed flaky setting evaluated. Tests marked as flaky have the Marked flaky tag in
the user interface.
Help and support
See our troubleshooting page
Get advice on Stack Overflow, and get support via the Developer Community

Related articles
Review test results
Visual Studio Test task
Publish Test Results task
Test Results, Result Meta Data - Update REST API
 TF31002: Unable to connect
6/27/2022 • 5 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
You might receive this error when you try to connect to Azure DevOps Services or an on-premises Azure
DevOps Server from Visual Studio.

You receive this error when you try to connect to Azure DevOps
Services
P RO B L EM RESO L UT IO N

You don't have an active account or license. Check with your administrator that you're a member of the
account and have an active, valid license. See Assign licenses
to users for details.

Your Azure DevOps Services organization is connected to When your Azure DevOps Services organization is
the Azure Active Directory. connected to a directory that is associated with a Microsoft
365 or Microsoft Azure subscription, only members in the
directory can access the account.

Check with your directory administrator to have them create

an organizational account for you or add your account to
the directory as external member.

You can't switch between different organizational accounts. If you work with several organizations that connect to
different directories, such as accounts created from the
Microsoft Azure Portal, the sign-out function might not
work as expected. For example, you can't switch between
different organizational accounts to connect to multiple
accounts that are linked to directory tenants.

When this problem occurs, you see a flashing blank sign in

dialog box several times. Then, you receive either TF31002
or TF31003 error after you connect to or add a new
connection in "Connect to Team Foundation Server" dialog
box.

To resolve this problem, apply the most recent Visual Studio

update .

To learn more, see You can't switch between different

organizational accounts in Visual Studio Online.

You want to sign in to Azure DevOps Services from Visual See Connect to projects, Sign in with different credentials.
Studio using different credentials.

When you try to connect to an on-premises Azure DevOps Server

from your client computer
If you determine that you're receiving this error from one computer but not others, or others aren't receiving
this error, then check the problem resolutions that are outlined below.
 P RO B L EM RESO L UT IO N

Your password has expired. Verify that you entered your user ID and password correctly,
and that your password hasn't expired.

You've entered an incorrect server URL. Verify that you've entered the server URL correctly including
the server name, port number, and protocol (http/https).
See Connect to projects to learn more.

The TFS configuration has changed. If the configuration for the on-premises Azure DevOps
Server has changed, you must create a new connection. You
might also need to clear the client cache.

You work remotely and need to connect to a TFS Proxy Configure Visual Studio to connect to TFS Proxy.
server to check in files to Team Foundation version control.

You're connecting to a later version of TFS than your Visual Your version of Visual Studio or Team Explorer might be
Studio client version. incompatible with Team Foundation Server. You might need
to install one or more GDR packs. See Requirements and
compatibility for details.

Your firewall is blocking TFS services. See Allow a program to communicate through Windows
Firewall.

Visual Studio stops responding when you run a query in Your computer might be configured to bypass the proxy
Visual Studio. server. Verify the configuration of the BypassProxyOnLocal
setting on your computer. For more information, see
BypassProxyOnLocal Configuration.

Several users can't connect to an on-premises Azure DevOps Server

If the problem occurs on more than one computer, contact your administrator to confirm whether the server is
operational and available on the network.
As an administrator, check the event logs for the application-tier server to try to pinpoint the problem. Also, you
can use the following table to determine whether the server is misconfigured. In the table, problems that are
more likely to occur appear first. Try the resolutions in the order in which they appear, which increases the
chance that you can solve the problem quickly.

P RO B L EM RESO L UT IO N

The TFSService account password has expired or is incorrect. Many services for Team Foundation Server will stop running
when the service account for Team Foundation has expired.
For more information, see Change the service account or
password for Team Foundation Server.

The application-tier server for Team Foundation is Verify whether each required service is running. If a required
unavailable. service isn't running, you must restart it. If necessary, set it
to start automatically. For more information, see Stop and
start services, application pools, and websites.

The network is unavailable. Verify whether your network is operational.

A website identity for Team Foundation is configured Verify or correct the server binding assignments that are
incorrectly. made to websites for Team Foundation.
P RO B L EM RESO L UT IO N

Access to a website for Team Foundation has been restricted. Verify or correct restrictions that are made to those websites
that are based on IP addresses and domain names.

The firewall or ports are configured incorrectly. Verify or correct port binding assignments for websites and
port assignments for the firewall. First, you should open the
administration console for Team Foundation, display the
Application Tier page, and review the URL assignments. If
necessary, you can click Change URL to modify the URL of
a website. Next, you should verify the port assignments for
Internet Information Services (IIS) and the ports that are
allowed through the firewall. For more information, see
Review Server Status and Settings and Verify or Correct Port
Assignments.

Trust relationships between domains aren't configured If a group of users can't access Team Foundation Server, you
correctly. might have trust issues between domains.

When users connect to different versions of TFS from Visual This error can occur because the GUIDs for the TFS 2012
Studio, for example, they connect to TFS 2012 and then TFS collection are the same as TFS 2008. The local client cache
2008, they can get the TF31002 error. gets confused because it tries to maintain the same GUID-
based local cache for both the 2008 server and the new
Project Collection in 2012.

To fix, run the TFSConfig ChangeSer verID command. See

TFSConfig ChangeServerID command.
 Test case management commands ([Link])
6/27/2022 • 2 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
You can perform several tasks to list, view, copy/clone or run tests using the [Link] command-line tool. Most
commands are supported for both on-premises and hosted Azure DevOps.

Work with the TCM command-line tool

To exercise the TCM command-line tool, you must have the following in place:
Installed Visual Studio 2017 Professional or earlier version. You access [Link] from the command
prompt and the following directories:
%programfiles(x86)%\Microsoft Visual Studio\2017\Professional\Common7\IDE
%programfiles(x86)%\Microsoft Visual Studio\2017\Enterprise\Common7\IDE
%programfiles(x86)%\Microsoft Visual Studio 14.0\Common7\IDE

To run a [Link] command, specify the /collection and /teamproject parameters, and /login as
needed.

PA RA M ET ER DESC RIP T IO N

/collection :CollectionURL Required. Specifies the URI of the team project collection.
The format for the URI is as follows:
- For Azure DevOps Services:
[Link]
- For Azure DevOps Server:
[Link]
. If no virtual directory is used, then the format for the
URI is as follows:
[Link]

/teamproject : project Required. The name of the project that contains the test
objects you want to clone or import automated tests
into.

/login : username,[password] Optional. Specifies the name and password of a valid

Azure DevOps user and who has permissions to run the
command. Use this option if you're using basic
authentication, you're not connected to a domain, or
your Windows credentials don't have the appropriate
permissions.

You must be a valid member of the project you want to access and have the required permissions based
on the commands you run. To learn more, see Manual test access and permissions.

TCM commands
The following table provides links to articles that describe how to exercise the corresponding [Link]
command. Most commands are supported on both Azure DevOps Server, all versions, and Azure DevOps
Services.
Command
Azure DevOps Ser vices
Azure DevOps Ser ver

tcm configs:Lists test configurations

️
✔
️
✔
tcm fieldmapping : Imports or exports an XML file for defining the resolution types, bug type, and failure types.

NOTE
You can only exercise this command on collections that use the Online XML process model. For Azure DevOps Server
2020 and later versions that use the Inherited process model, import of XML files aren't supported.

️
✔
tcm plans : List or clone test plans.
️
✔
️
✔
tcm run: Creates, deletes, lists, aborts, publishes, exports, or runs a group of tests. Also supports importing
automated tests into a test plan.
️
✔
️
✔
tcm suites : List or clone test suites.
️
✔
️
✔
tcm testenvironments : View and list test environments.
️
✔
️
✔
tcm testcase : Import test cases from a specified assembly or a test file
️
✔
️
✔

Related articles
Copy or clone test plans, test suites, and test cases
Associate automated tests with test cases
Cross-service integration and collaboration overview
 Field descriptions for default and work item fields
used in process templates
6/27/2022 • 4 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Use this index to look up a description of each field used to track work items. This reference includes all fields
defined within the core system processes/process templates: Basic, Agile, Scrum, and CMMI. The fields and work
item types (WITs) available to you depend on the process you chose when you created your project.
To support other tracking needs, you can define your own custom work item fields.
To support other tracking needs, you can define your own custom work item fields using the Inheritance process
model, or if your project collection is configured to use the On-premises XML process model, then see Modify or
add a custom field.
To support other tracking needs, you can modify or add a custom field.

NOTE
The Analytics Service doesn't support reporting on plain text and HTML fields at this time.

Alphabetical index
Values in parenthesis indicate the following criteria:
System : Core system field assigned to all work item types for all processes
Agile : Used only by the Agile process
CMMI : Used only by the CMMI process
Scrum : Used only by the Scrum process
TCM : Used to support Test case management
A
Acceptance Criteria (Scrum)
Accepted By
Accepted Date
Activated By
Activated Date
Activity
Actual Attendee 1-8 (CMMI)
Analysis (CMMI)
Application Launch Instructions
Application Start Information
Application Type
Area ID (System)
Area Path (System)
Assigned To
Associated Context
 Associated Context Code
Associated Context Owner
Associated Context Type
Attached File Count
Authorized As (Not used)
Automated Test Id (TCM)
Automated Test Name (TCM)
Automated Test Storage (TCM)
Automated Test Type (TCM)
AutomatedTestId (TCM)
AutomatedTestName (TCM)
Automation Status (TCM)
B
Backlog Priority (Scrum)
Blocked
Board Column1
Board Column Done1
Board Lane1
Business Value
C
Called By (CMMI)
Called Date (CMMI)
Changed By (System)
Changed Date (System)
Closed By (System)
Closed Date (System)
Closed Status
Closed Status Code
Closing Comment
Comment Count2
Comments (CMMI)
Committed (CMMI)
Completed Work
Contingency Plan (CMMI)
Corrective Action Actual Resolution (CMMI)
Corrective Action Plan (CMMI)
Created By (System)
Created Date (System)
D-E-F
Discipline (CMMI)
Description (System)
Due Date
Effort
Escalate (CMMI)
External Link Count
 Finish Date
Found In Build (TCM)
Found In Environment (CMMI)
H
History (System)
How Found (CMMI)
Hyperlink Count
I
ID (System)
Impact Assessment (CMMI)
Impact on Architecture (CMMI)
Impact on Development (CMMI)
Impact on Technical Publications (CMMI)
Impact on Test (CMMI)
Impact on User Experience (CMMI)
Integrated in Build (TCM)
Issue (TCM)
Iteration ID (System)
Iteration Path (System)
J-L -M -N
Justification (CMMI)
Link Comment (System)
Link Description (System)
Local Data Source (TCM)
Meeting Type (CMMI)
Minutes (CMMI)
Mitigation Plan (CMMI)
Mitigation Triggers (CMMI)
Node Name (System)
O -P-Q
Optional Attendee 1-8 (CMMI)
Original Estimate
Parameters (TCM)
Parent3
Priority
Probability (CMMI)
Proposed Fix (CMMI)
Purpose (CMMI)
Query Text (TCM)
R
Rating
Reason (System)
Related Link Count (System)
Remaining Work
4
 Remote Link Count4 (System)
Repro Steps
Required Attendee 1-8 (CMMI)
Requirement Type (CMMI)
Requires Review (CMMI)
Requires Test (CMMI)
Resolution (Scrum)
Resolved By
Resolved Date
Resolved Reason
Reviewed By
Reviewed Date
Rev (System)
Risk (Agile)
Root Cause (CMMI)
S
Severity
Size (CMMI)
Stack Rank
Start Date
State (System)
State Change Date
State Code
Steps (TCM)
Steps to Reproduce (TCM)
Story Points (Agile)
Subject Matter Expert (CMMI)
Symptom (CMMI)
System Info (TCM)
T
Tags
Target Date
Target Resolve Date (CMMI)
Task Type (CMMI)
Team Project (System)
Test Suite Audit (TCM)
Test Suite Type (TCM)
Test Suite Type ID (TCM)
Time Criticality
Title (System)
Triage (CMMI)
U -V -W
User Acceptance Test (CMMI)
Value Area
Watermark (System)
 Work Item Type (System)

NOTE
1. These fields are available from Azure DevOps Services and TFS 2015.1 or later versions.
2. The Comment Count field is available from Azure DevOps Services and TFS 2017 or later versions.
3. These fields are available from Azure DevOps Services and Azure DevOps Server 2020.
4. These fields are available from Azure DevOps Services only.

By using the system fields or other fields you've added to your project collection, you can enable meaningful
cross-project reports and queries. Also, any non-system field that is referenced in the workflow or forms section
of the work item type definition must have a FIELD element that defines it in the FIELDS section of the work
item type definition XML file. Also, you must specify any non-system field that you might want to use to
generate a query or report in the FIELDS section.

Field reference articles

The following articles describe fields that are used in common by several WITs, or those fields that are
functionally specific to just one or a few WITs.
Fields common to many work types
Titles, IDs, and descriptive fields
History and revision changes
Areas and iterations
Assignments and account-specific fields
Planning, ranking, and priorities
Work estimates, activity, and other numeric fields
Build and test integration fields
Links and attachment related fields
Fields used by specific work item types
Code Review Request
Code Review Response
Feedback Request
Feedback Response
Shared Steps
Test Case
Fields used to track CMMI work items
Requirements
Bugs
Change Requests
Issues
Review Meetings
Risks

Related articles
About work item fields
About managed queries
Define a query
Choose a process
Reportable fields reference (on-premises Azure DevOps only)
 Create a query based on build and test integration
fields
6/27/2022 • 10 minutes to read • Edit Online

Azure DevOps Ser vices | Azure DevOps Ser ver 2020 | Azure DevOps Ser ver 2019 | TFS 2018
Work item fields that support build and test integration support the following actions:
Associate bugs with the builds where they were found or fixed
Query for bugs associated with a build
Mark test cases as either manual or automated, and store information to support automated test cases
For test cases and shared steps, define the action and validation steps and the data that are used to run tests.

Supported operators and macros

Most build and test integration fields have a data type of String, PlainText, or HTML. Query clauses that specify a
text or rich-text field can use the operators and macros listed in the following table.
Data type
Suppor ted operators and macros

Rich-text (HTML)
Contains Words, Does Not Contain Words, Is Empty1, Is Not Empty1
Multi-line text strings (PlainText)
Contains Words, Does Not Contain Words, Is Empty1, Is Not Empty1
Single text (String)
= , <> , > , < , >= , <= , =[Field], <>[Field], >[Field], <[Field], >=[Field], <=[Field], Contains, Does Not Contain,
In, Not In, In Group, Not In Group, Was Ever
Macros : [Any] , valid with the Work Item Type field
Project 2, valid with the Team Project field

NOTE
1. The Is Empty and Is Not Empty operators are supported for Azure DevOps Server 2019 RC2 and later versions
2. The @Project macro is supported for Azure Boards and TFS 2015.1 and later versions. The system automatically
defaults to filtering based on the current project. To learn more, see Query across projects.

Useful filters
Filter for
Include these quer y clauses
Automated test cases
Work Item Type = Test Case And Automation Status = Automated
Query-based test suites
Work Item Type = Test Suite And Test Suite Type = Query Based

Requirement-based test suites

Work Item Type = Test Suite And Test Suite Type = Requirement Based

List bugs and the test cases that test them

Open a new query, set the query type to Work items and direct links. Filter for bugs in the top level and add the
filter for Test Cases in the linked work items filter.

NOTE
You can't construct a query that shows a hierarchical view of Test Plans, Test Suites, and Test Cases. These items aren't
linked together using parent-child link types. You can view the hierarchy through the Test>Test Plans page.

Build and test data fields

The following table describes the fields that are defined in one or more of the test WITs. For information about
data types and field attributes, see Work item fields and attributes.
To customize a field or picklist, see Add or modify a field to support queries, reports, and workflow.
Field name
Description
Work item type

Automation Status 1
The status of a test case. You can specify the following values:
Automated
 Not Automated
Planned
To run automated tests, see Run automated tests from test plans.
Reference name=[Link], Data type=String
Test Case
Found In 2
Product build number, also known as a revision, in which a bug was found.
Reference name=[Link], Data type=String

NOTE
You can also use the Found in build link type to link a work item to a build. This link type is available from Azure
DevOps and only works with the current build processes (not XAML builds).

Bug
Integration Build 2
Product build number that incorporates the code or fixes a bug.
Reference name=[Link], Data type=String

NOTE
You can also use the Integrated in build link type to link a work item to a build. This link type is available from Azure
DevOps and only works with the current build processes (not XAML builds).

All
Issue
Indicates that the Shared Steps are associated with an expected result. Allowed values are Yes and No . Reference
name=[Link], Data type=String
Shared Steps
Parameters 3
Contains the parameters to use when running a manual test.
[Link], Data type=HTML
Shared Parameters, Shared Steps, Test Case
Steps
The action and validation steps that are required to run the test. [Link], Data type=HTML
Shared Steps, Test Case
System Info
Information about the software and system configuration that is relevant to the test.
[Link], Data type=HTML
Bug, Feedback Response
Repro Steps (or Steps to reproduce)
The steps that are required to reproduce unexpected behavior. Capture enough information so that other team
members can understand the full impact of the problem and whether they've fixed the bug. This includes actions
taken to find or reproduce the bug and expected behavior. Reference name=[Link],
Data type=HTML
Bug
Test Suite Type 1,4
The test suite category. Allowed values are:
Quer y Based : Use to group together test cases that have a particular characteristic - for example, all the
tests that have Priority=1. The suite will automatically include every test case that is returned by the query
that you define.
Static : Use to group together test cases designed to track the test status of backlog items. Each test case that
you add to a requirement-based test suite is automatically linked to the backlog item.
Requirement Based : Use to group together test cases with any characteristics or test suites.
For more information, see Create a test plan.
Reference name=[Link], Data type=String
Test Suite

NOTE
1. Do not customize the picklist for these fields. The system accepts only those values listed.
2. By adding a GLOBALLIST element to the FIELD definition, you can provide a drop-down menu of builds that users
can choose from. To learn how, see Builds and global list auto-population later in this article.
3. Requires TFS 2013.2 or later version to be installed on the application-tier server and existing projects to be updated
to support Shared Parameters. To learn more, see Configure features after a TFS upgrade.
4. Requires TFS 2013.3 or later version to be installed on the application-tier server and existing projects to be updated
to support Test Plan and Test Suite. To learn more, see Configure features after a TFS upgrade.

Other fields
The following fields don't appear on work item forms, but these fields are tracked for test cases or test suites.
You can use some of these fields to filter queries and create reports. (None of these fields are added to the data
warehouse nor indexed.)
Field name
Description
Work item type
Automated Test Storage
The assembly that contains the test that automates the test case.
Reference name=[Link], Data type=String
Test Case
Automated Test Type
The type of test that automates the test case.
Reference name=[Link], Data type=String
Test Case
AutomatedTestId
The ID of the test that automates the test case.
Reference name=[Link], Data type=String
Test Case
AutomatedTestName
The name of the test that is used to automate the test case.
Reference name=[Link], Data type=String
Test Case
LocalDataSource
The local data source that supports the test.
Reference name=[Link], Data type=HTML
Test Case
Query Text
Field used to capture the query defined for a Query-based suite type.
Reference name=[Link], Data type=PlainText
Test Suite
Test Suite Audit 1
Tracks other operations run when modifying a test suite, for example: adding tests to a test suite or changing
configurations. This field can be viewed through the History tab or through a separate query. There will be a
combined history view, including changes done to work items field and changes resulting from related artifacts
such as test points and configurations.
Reference name=[Link], Data type=PlainText
Test Suite
Test Suite Type ID 1, 2
A system assigned value that corresponds to the test suite category and only applicable to test suites. Assigned
values are:
1 (Static)
2 (Query-based)
3 (Requirement- based)
Reference name=[Link], Data type=Integer
Test Suite
 NOTE
1. Requires TFS 2013.3 or later version to be installed on the application-tier server and existing projects to be updated
to support Test Plan and Test Suite.
2. Do not customize the picklist for these fields. The system accepts only those values listed.

Fields that integrate with Team Foundation Build

Team Foundation Build is the on-premise build system you can use with Azure DevOps Server and TFS. You can
configure your build process by using Team Foundation Build, and Team Foundation Build can generate work
items when a build fails. It can also add build information to work items that were resolved in a particular build.
For this to work, Team Foundation Build requires that the following two fields be added to the work item type
definition: Found In and Integration Build .
Found In and Integrated in Build fields are defined for Bugs in the default processes. These fields associate
bugs with the builds where they were found or fixed.
You can use the following code snippet to add these fields to a WIT definition.

<FIELD name="Found In" refname="[Link]" type="String" reportable="dimension">

<HELPTEXT>Product build number (revision) in which this item was found</HELPTEXT>
<SUGGESTEDVALUES>
<LISTITEM value="&lt;None&gt;" />
</SUGGESTEDVALUES>
</FIELD>
<FIELD name="Integration Build" refname="[Link]" type="String"
reportable="dimension">
<HELPTEXT>Product build number this bug was fixed in</HELPTEXT>
<SUGGESTEDVALUES>
<LISTITEM value="&lt;None&gt;" />
</SUGGESTEDVALUES>
</FIELD>

When the Found In field is present in a WIT definition, Team Foundation Build creates a work item when a build
fails, and sets the Found In field to the build number of the build that failed. If the Found In field is missing,
Team Foundation Build doesn't create a work item for the failed build, and everything else works as expected.
When the Integration Build field is present in the WIT definition, Team Foundation Build identifies work items
that were resolved with each build and then updates those work items to set the build number in which they
were resolved in the Integration Build field. If the Integration Build field is missing, Team Foundation Build
doesn't store the build number in the work items, and everything else works as expected.

Builds and global list autopopulation

The first time you queue a build for a project using Team Foundation Build, TFS automatically adds a global list
labeled Build - ProjectName . Each time a build is run, a LISTITEM is added to this global list with the name of
the build.
By adding a GLOBALLIST element to the FIELD definition, you can provide a drop-down menu of builds that
users can choose from. For example:
 <FIELD name="Found In" refname="[Link]" type="String" reportable="dimension">
<HELPTEXT>Product build number (revision) in which this item was found</HELPTEXT>
<SUGGESTEDVALUES>
<LISTITEM value="&lt;None&gt;" />
</SUGGESTEDVALUES>
<SUGGESTEDVALUES expanditems="true" filteritems="excludegroups">
<GLOBALLIST name="Builds - TeamProjectName" />
</SUGGESTEDVALUES>
</FIELD>

Fields that Integrate with Test Plans

With Test Plans, you can automate the creation of a bug or other type of work item when a test fails. For more
information, see Add findings to existing bugs with exploratory testing.
When a work item has been created in this manner, information about the system and the steps to reproduce
the bug are captured in the System Info and Repro Steps fields.
You can add these fields to work item types that you create for tracking defects using the following code snippet.

<FIELD name="System Info" refname="[Link]" type="HTML" />

<FIELD name="Repro Steps" refname="[Link]" type="HTML" />

Fields that integrate with Team Foundation Version Control

One of the features available in Team Foundation version control (TFVC) enables you to associate or resolve
work items when you check in code. You might have worked on a particular work item when you make a code
change and you can set that association from within the source-control check-in window when you're finished
working on the code.
The ability of Team Foundation version control to resolve a work item requires that work items contain a
particular action. The source control system then queries work item tracking to determine whether the work
item supports that action, and if it does support that action, it also queries for the source and destination states
of the transition. If the action is found, the source control system can transition the work item according to the
set transition when it checks in the code.

NOTE
When you use the Checkin action, you must set appropriate from and to states to reflect the state transition that you
want.

For more information about Actions, see Automate field assignments based on State, Transition, or Reason.

Related articles
Work item field index
Drive Git development from a work item
Linking, traceability, and managing dependencies
Link and attachment queries