100% found this document useful (1 vote)

22 views70 pages

Testing Systems

Uploaded by

Wycliff Jonathan Kituri

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

100% found this document useful (1 vote)

22 views70 pages

Testing Systems

Uploaded by

Wycliff Jonathan Kituri

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

1

4 Certified Tester
5

6 Foundation Level Syllabus

8 v4.0
9
10
11
12

14 International Software Testing Qualifications Board

15
Certified Tester
Foundation Level

16 Copyright Notice
17 Copyright Notice © International Software Testing Qualifications Board (hereinafter called ISTQB®)
18 ISTQB® is a registered trademark of the International Software Testing Qualifications Board.
19 Copyright © 2022 the authors of the Foundation Level v4.0 syllabus: Renzo Cerquozzi, Wim Decoutere,
20 Klaudia Dussa-Zieger, Jean-François Riverin, Arnika Hryszko, Martin Klonk, Michaël Pilaeten, Meile
21 Posthuma, Stuart Reid, Eric Riou du Cosquer (chair), Adam Roman, Lucjan Stapp, Stephanie Ulrich (vice
22 chair), Eshraka Zakaria
23 Copyright © 2019 the authors for the update 2019 Klaus Olsen (chair), Meile Posthuma and Stephanie
24 Ulrich.
25 Copyright © 2018 the authors for the update 2018 Klaus Olsen (chair), Tauhida Parveen (vice chair), Rex
26 Black (project manager), Debra Friedenberg, Matthias Hamburg, Judy McKay, Meile Posthuma, Hans
27 Schaefer, Radoslaw Smilgin, Mike Smith, Steve Toms, Stephanie Ulrich, Marie Walsh, and Eshraka
28 Zakaria,
29 Copyright © 2011 the authors for the update 2011 Thomas Müller (chair), Debra Friedenberg, and the
30 ISTQB WG Foundation Level.
31 Copyright © 2010 the authors for the update 2010 Thomas Müller (chair), Armin Beer, Martin Klonk, and
32 Rahul Verma.
33 Copyright © 2007 the authors for the update 2007 Thomas Müller (chair), Dorothy Graham, Debra
34 Friedenberg and Erik van Veenendaal.
35 Copyright © 2005, the authors Thomas Müller (chair), Rex Black, Sigrid Eldh, Dorothy Graham, Klaus
36 Olsen, Maaret Pyhäjärvi, Geoff Thompson, and Erik van Veenendaal.
37 All rights reserved. The authors hereby transfer the copyright to the ISTQB®. The authors (as current
38 copyright holders) and ISTQB® (as the future copyright holder) have agreed to the following conditions of
39 use:
40 • Extracts, for non-commercial use, from this document may be copied if the source is acknowledged.
41 Any Accredited Training Provider may use this syllabus as the basis for a training course if the
42 authors and the ISTQB® are acknowledged as the source and copyright owners of the syllabus and
43 provided that any advertisement of such a training course may mention the syllabus only after official
44 Accreditation of the training materials has been received from an ISTQB®-recognized Member Board.

45 • Any individual or group of individuals may use this syllabus as the basis for articles and books, if the
46 authors and the ISTQB® are acknowledged as the source and copyright owners of the syllabus.

47 • Any other use of this syllabus is prohibited without first obtaining the approval in writing of the
48 ISTQB®.

49 • Any ISTQB®-recognized Member Board may translate this syllabus provided they reproduce the
50 abovementioned Copyright Notice in the translated version of the syllabus.

v4.0 Page 2 of 70 beta release

52 Revision History
53
Version Date Remarks
TBA CTFL v4.0 – General release version
CTFL v4.0 3.01.2023 CTFL v4.0 – Candidate beta version
CTFL v4.0 15.09.2022 CTFL v4.0 – Alpha review release
CTFL v3.1.1 01.07.2021 CTFL v3.1.1 – Copyright and logo update
CTFL v3.1 11.11.2019 CTFL v3.1 – Maintenance release with minor updates
ISTQB 2018 27.04.2018 CTFL v3.0 – Candidate general release version
ISTQB 2018 12.02.2018 CTFL v3.0 – Candidate beta version
ISTQB 2018 19.01.2018 Cross-review internal v3.0
ISTQB 2018 15.01.2018 Pre-cross-review internal v2.9 incorporating Core Team edits.
ISTQB 2018 9.12.2017 Alpha review v2.5 release – Technical edit of v2.0 release, no new content
added
ISTQB 2018 22.11.2017 Alpha review v2.0 release – Certified Tester Foundation Level Syllabus
Major Update 2018 – see Appendix C – Release Notes for details
ISTQB 2018 12.06.2017 Alpha review release - Certified Tester Foundation Level Syllabus Major
Update 2018 – see Appendix C – Release Notes
ISTQB 2011 1.04.2011 CTFL Syllabus Maintenance Release – see Release Notes
ISTQB 2010 30.03.2010 CTFL Syllabus Maintenance Release – see Release Notes
ISTQB 2007 01.05.2007 CTFL Syllabus Maintenance Release
ISTQB 2005 01.07.2005 Certified Tester Foundation Level Syllabus v1.0
ASQF V2.2 07.2003 ASQF Syllabus Foundation Level Version v2.2 “Lehrplan Grundlagen des
Software-testens“
ISEB V2.0 25.02.1999 ISEB Software Testing Foundation Syllabus v2.0
54

v4.0 Page 3 of 70 beta release

55 Table of Contents
56
57 Copyright Notice ....................................................................................................................................... 2
58 Revision History ....................................................................................................................................... 3
59 Table of Contents ..................................................................................................................................... 4
60 Acknowledgements .................................................................................................................................. 8
61 0. Introduction ...................................................................................................................................... 9
62 0.1. Purpose of this Syllabus .............................................................................................................. 9
63 0.2. The Certified Tester Foundation Level in Software Testing ........................................................ 9
64 0.3. Career Path for Testers ............................................................................................................... 9
65 0.4. Business Outcomes ................................................................................................................... 10
66 0.5. Examinable Learning Objectives and Cognitive Level of Knowledge ....................................... 10
67 0.6. The Foundation Level Certificate Exam .................................................................................... 11
68 0.7. Accreditation .............................................................................................................................. 11
69 0.8. Handling of Standards ............................................................................................................... 11
70 0.9. Keeping It Current ...................................................................................................................... 11
71 0.10. Level of Detail ............................................................................................................................ 11
72 0.11. How this Syllabus is Organized ................................................................................................. 12
73 1. Fundamentals of Testing – 180 minutes ....................................................................................... 13
74 1.1. What is Testing? ........................................................................................................................ 14
75 1.1.1. Objectives of Testing ............................................................................................................. 14
76 1.1.2. Testing and Debugging ......................................................................................................... 15
77 1.2. Why is Testing Necessary? ....................................................................................................... 15
78 1.2.1. Testing’s Contributions to Success ....................................................................................... 15
79 1.2.2. Testing and Quality Assurance (QA) ..................................................................................... 16
80 1.2.3. Root Causes, Errors, Defects, and Failures .......................................................................... 16
81 1.3. Testing Principles ...................................................................................................................... 16
82 1.4. Test Activities, Test Work Products and Test Roles ................................................................. 17
83 1.4.1. Test Activities and Tasks ....................................................................................................... 17
84 1.4.2. Test Process in Context ........................................................................................................ 18
85 1.4.3. Test Work Products ............................................................................................................... 19
86 1.4.4. Traceability between the Test Basis and Test Work Products .............................................. 19

v4.0 Page 4 of 70 beta release

87 1.4.5. Roles in Testing ..................................................................................................................... 20

88 1.5. Essential Skills and Good Practices .......................................................................................... 20
89 1.5.1. Generic Skills Required for Testing ....................................................................................... 20
90 1.5.2. Whole Team Approach .......................................................................................................... 21
91 1.5.3. Independence of Testing ....................................................................................................... 21
92 2. Testing Throughout the Software Development Lifecycles – 130 minutes ................................... 22
93 2.1. Testing in Context of Software Development Lifecycles ........................................................... 23
94 2.1.1. Impact of Software Development Lifecycle on Testing ......................................................... 23
95 2.1.2. Software Development Lifecycles and Good Testing Practices ........................................... 23
96 2.1.3. Testing as a Driver for Software Development ..................................................................... 24
97 2.1.4. DevOps and Testing .............................................................................................................. 24
98 2.1.5. Shift-Left Approach ................................................................................................................ 25
99 2.1.6. Retrospectives and Process Improvement ........................................................................... 25
100 2.2. Test Levels and Test Types....................................................................................................... 26
101 2.2.1. Test Levels ............................................................................................................................ 26
102 2.2.2. Test Types ............................................................................................................................. 27
103 2.2.3. Confirmation Testing and Regression Testing ...................................................................... 28
104 2.3. Maintenance Testing ................................................................................................................. 28
105 3. Static Testing – 80 minutes ........................................................................................................... 30
106 3.1. Static Testing Basics ................................................................................................................. 31
107 3.1.1. Work Products Examinable by Static Testing ....................................................................... 31
108 3.1.2. Value of Static Testing .......................................................................................................... 31
109 3.1.3. Differences between Static and Dynamic Testing ................................................................. 32
110 3.2. Feedback and Review Process ................................................................................................. 32
111 3.2.1. Benefits of Early and Frequent Customer Feedback ............................................................ 32
112 3.2.2. Review Process Activities ..................................................................................................... 33
113 3.2.3. Roles and Responsibilities in Reviews .................................................................................. 33
114 3.2.4. Review Types ........................................................................................................................ 34
115 3.2.5. Success Factors for Reviews ................................................................................................ 34
116 4. Test Analysis and Design – 390 minutes ...................................................................................... 36
117 4.1. Test Techniques Overview ........................................................................................................ 37
118 4.2. Black-Box Test Techniques ....................................................................................................... 37

v4.0 Page 5 of 70 beta release

119 4.2.1. Equivalence Partitioning ........................................................................................................ 37

120 4.2.2. Boundary Value Analysis ...................................................................................................... 38
121 4.2.3. Decision Table Testing .......................................................................................................... 39
122 4.2.4. State Transition Testing ........................................................................................................ 39
123 4.3. White-Box Testing ..................................................................................................................... 40
124 4.3.1. Statement Testing and Statement Coverage ........................................................................ 40
125 4.3.2. Branch Testing and Branch Coverage .................................................................................. 41
126 4.3.3. The Value of White-box Testing ............................................................................................ 41
127 4.4. Experience-based Testing ......................................................................................................... 41
128 4.4.1. Error Guessing ...................................................................................................................... 41
129 4.4.2. Exploratory Testing ............................................................................................................... 42
130 4.4.3. Checklist-Based Testing ........................................................................................................ 42
131 4.5. Collaboration-based Test Approaches ...................................................................................... 43
132 4.5.1. Collaborative User Story Writing ........................................................................................... 43
133 4.5.2. Acceptance Criteria ............................................................................................................... 43
134 4.5.3. Acceptance Test-driven Development (ATDD) ..................................................................... 44
135 5. Managing the Test Activities – 335 minutes .................................................................................. 45
136 5.1. Test Planning ............................................................................................................................. 46
137 5.1.1. Purpose and Content of a Test Plan ..................................................................................... 46
138 5.1.2. Tester's Contribution to Iteration and Release Planning ....................................................... 46
139 5.1.3. Entry and Exit Criteria ........................................................................................................... 47
140 5.1.4. Estimation Techniques .......................................................................................................... 47
141 5.1.5. Test Case Prioritization ......................................................................................................... 48
142 5.1.6. Test Pyramid ......................................................................................................................... 48
143 5.1.7. Testing Quadrants ................................................................................................................. 49
144 5.2. Risk Management ...................................................................................................................... 49
145 5.2.1. Risk Definition and Risk Attributes ........................................................................................ 49
146 5.2.2. Project and Product Risks ..................................................................................................... 49
147 5.2.3. Product Risk Analysis ............................................................................................................ 50
148 5.2.4. Product Risk Control ............................................................................................................. 51
149 5.3. Test Monitoring, Test Control and Test Completion .................................................................. 51
150 5.3.1. Metrics used in Testing ......................................................................................................... 52

v4.0 Page 6 of 70 beta release

151 5.3.2. Purpose, Contents and Audience for Test Reports............................................................... 52

152 5.3.3. Communicating the Status of Testing ................................................................................... 53
153 5.4. Configuration Management ....................................................................................................... 53
154 5.5. Defect Management .................................................................................................................. 54
155 6. Test Tools – 20 minutes ................................................................................................................ 56
156 6.1. Tool Support for Testing ............................................................................................................ 57
157 6.2. Benefits and Risks of Test Automation ...................................................................................... 57
158 7. References .................................................................................................................................... 59
159 8. Appendix A – Learning Objectives/Cognitive Level of Knowledge ............................................... 62
160 9. Appendix B – Business Outcomes traceability matrix with Learning Objectives .......................... 63
161 10. Appendix C – Release Notes .................................................................................................... 68
162 11. Index.......................................................................................................................................... 70
163

v4.0 Page 7 of 70 beta release

164 Acknowledgements
165 This document was formally released by the General Assembly of the ISTQB® on <date>
166 It was produced by a team from the ISTQB joint Foundation Level & Agile Working Groups: Laura Albert,
167 Renzo Cerquozzi (vice chair), Wim Decoutere, Klaudia Dussa-Zieger, Chintaka Indikadahena, Arnika
168 Hryszko, Martin Klonk, Kenji Onishi, Michaël Pilaeten (co-chair), Meile Posthuma, Gandhinee Rajkomar,
169 Stuart Reid, Eric Riou du Cosquer (co-chair), Jean-François Riverin, Adam Roman, Lucjan Stapp,
170 Stephanie Ulrich (vice chair), Eshraka Zakaria.
171 The team thanks Stuart Reid and Patricia McQuaid for their technical review and the review team and the
172 Member Boards for their suggestions and input.
173 The following persons participated in the reviewing, commenting and balloting of this syllabus: to be
174 completed
175 ISTQB Working Group Foundation Level (Edition 2018): Klaus Olsen (chair), Tauhida Parveen (vice
176 chair), Rex Black (project manager), Eshraka Zakaria, Debra Friedenberg, Ebbe Munk, Hans Schaefer,
177 Judy McKay, Marie Walsh, Meile Posthuma, Mike Smith, Radoslaw Smilgin, Stephanie Ulrich, Steve
178 Toms, Corne Kruger, Dani Almog, Eric Riou du Cosquer, Igal Levi, Johan Klintin, Kenji Onishi, Rashed
179 Karim, Stevan Zivanovic, Sunny Kwon, Thomas Müller, Vipul Kocher, Yaron Tsubery and all Member
180 Boards for their suggestions.
181 ISTQB Working Group Foundation Level (Edition 2011): Thomas Müller (chair), Debra Friedenberg. The
182 core team thanks the review team (Dan Almog, Armin Beer, Rex Black, Julie Gardiner, Judy McKay,
183 Tuula Pääkkönen, Eric Riou du Cosquier Hans Schaefer, Stephanie Ulrich, Erik van Veenendaal), and all
184 Member Boards for the suggestions for the current version of the syllabus.
185 ISTQB Working Group Foundation Level (Edition 2010): Thomas Müller (chair), Rahul Verma, Martin
186 Klonk and Armin Beer. The core team thanks the review team (Rex Black, Mette Bruhn-Pederson, Debra
187 Friedenberg, Klaus Olsen, Judy McKay, Tuula Pääkkönen, Meile Posthuma, Hans Schaefer, Stephanie
188 Ulrich, Pete Williams, Erik van Veenendaal), and all Member Boards for their suggestions.
189 ISTQB Working Group Foundation Level (Edition 2007): Thomas Müller (chair), Dorothy Graham, Debra
190 Friedenberg, and Erik van Veenendaal. The core team thanks the review team (Hans Schaefer,
191 Stephanie Ulrich, Meile Posthuma, Anders Pettersson, and Wonil Kwon) and all the Member Boards for
192 their suggestions.
193 ISTQB Working Group Foundation Level (Edition 2005): Thomas Müller (chair), Rex Black, Sigrid Eldh,
194 Dorothy Graham, Klaus Olsen, Maaret Pyhäjärvi, Geoff Thompson and Erik van Veenendaal. The core
195 team thanks the review team and all Member Boards for their suggestions.

v4.0 Page 8 of 70 beta release

196 0. Introduction

197 0.1. Purpose of this Syllabus

198 This syllabus forms the basis for the International Software Testing Qualification at the Foundation Level.
199 The ISTQB® provides this syllabus as follows:
200 1. To member boards, to translate into their local language and to accredit training providers.
201 Member boards may adapt the syllabus to their particular language needs and modify the
202 references to adapt to their local publications.
203 2. To certification bodies, to derive examination questions in their local language adapted to the
204 learning objectives for this syllabus.
205 3. To training providers, to produce courseware and determine appropriate teaching methods.
206 4. To certification candidates, to prepare for the certification exam (either as part of a training course
207 or independently).
208 To the international software and systems engineering community, to advance the profession of software
209 and systems testing, and as a basis for books and articles.

210 0.2. The Certified Tester Foundation Level in Software Testing

211 The Foundation Level qualification is aimed at anyone involved in software testing. This includes people
212 in roles such as testers, test analysts, test engineers, test consultants, test managers, software
213 developers and team members in Agile development. This Foundation Level qualification is also
214 appropriate for anyone who wants a basic understanding of software testing, such as project managers,
215 quality managers, product owners, software development managers, business analysts, IT directors and
216 management consultants. Holders of the Foundation Certificate will be able to go on to higher-level
217 software testing qualifications.

218 0.3. Career Path for Testers

219 The ISTQB® scheme provides support for testing professionals at all stages of their careers offering both
220 breadth and depth of knowledge. Individuals who achieved the ISTQB® Foundation certification may also
221 be interested in the Core Advanced Levels (Test Analyst, Technical Test Analyst, and Test Manager) and
222 thereafter Expert Level (Test Management or Improving the Test Process). Anyone seeking to develop
223 skills in testing practices in an Agile environment could consider the Agile Technical Tester or Agile Test
224 Leadership at Scale certifications. The Specialist stream offers a deep dive into areas that have specific
225 test approaches and test activities (e.g., in test automation, AI testing, model-based testing, mobile app
226 testing), that are related to specific test areas (e.g., performance testing, usability testing, acceptance
227 testing, security testing), or which cluster testing know-how for certain industry domains (e.g., automotive
228 or gaming). Please visit [Link] for the latest information on ISTQB´s Certified Tester Scheme.

v4.0 Page 9 of 70 beta release

229 0.4. Business Outcomes

230 This section lists the 14 Business Outcomes expected of a person who has achieved the Foundation
231 Level certification.
232 A Foundation Level Certified Tester can…
233 FL-BO1 Understand what testing is and why it is beneficial
234 FL-BO2 Understand fundamental concepts of software testing
235 FL-BO3 Identify the test approach and activities to be implemented depending on the context of
236 testing
237 FL-BO4 Assess and improve the quality of the documentation
238 FL-BO5 Increase the effectiveness and efficiency of testing
239 FL-BO6 Align the testing process with the software development lifecycle
240 FL-BO7 Understand test management principles
241 FL-BO8 Write and communicate clear and understandable defect reports
242 FL-BO9 Understand the factors that influence the test priorities and test efforts
243 FL-BO10 Work as part of a cross-functional team
244 FL-BO11 Know risks and benefits related to test automation
245 FL-BO12 Identify essential skills required for testing
246 FL-BO13 Understand the impact of risk on testing
247 FL-BO14 Effectively report on test progress and quality

248 0.5. Examinable Learning Objectives and Cognitive Level of Knowledge

249 Learning objectives support business outcomes and are used to create the Certified Tester Foundation
250 Level exams. In general, all contents of chapters 1-6 of this syllabus are examinable at a K1 level. That is,
251 the candidate may be asked to recognize, remember, or recall a keyword or concept mentioned in any of
252 the six chapters. The specific learning objectives levels are shown at the beginning of each chapter, and
253 classified as follows:
254 • K1: Remember
255 • K2: Understand
256 • K3: Apply
257 Further details and examples of learning objectives are given in Appendix A. All terms listed as keywords
258 just below chapter headings shall be remembered (K1), even if not explicitly mentioned in the learning
259 objectives.

v4.0 Page 10 of 70 beta release

260 0.6. The Foundation Level Certificate Exam

261 The Foundation Level Certificate exam will be based on this syllabus. Answers to exam questions may require
262 the use of material based on more than one section of this syllabus. All sections of the syllabus are
263 examinable, except for the Introduction and Appendices. Standards and books are included as references, but
264 their content is not examinable, beyond what is summarized in the syllabus itself from such standards and
265 books. Refer to Exam Structures and Rules document for the Foundation Level for further details.

266 0.7. Accreditation

267 An ISTQB® Member Board may accredit training providers whose course material follows this syllabus.
268 Training providers should obtain accreditation guidelines from the Member Board or body that performs
269 the accreditation. An accredited course is recognized as conforming to this syllabus, and is allowed to
270 have an ISTQB® exam as part of the course. The accreditation guidelines for this syllabus follow the
271 general Accreditation Guidelines published by the Processes Management and Compliance Working
272 Group.

273 0.8. Handling of Standards

274 There are standards referenced in the Foundation Syllabus (e.g., IEEE or ISO standards). The purpose of
275 these references is to provide a framework (as in the references to ISO 25010 regarding quality characteristics)
276 or to provide a source of additional information if desired by the reader. The standards documents are not
277 intended for examination. Refer to chapter 7 for more information on standards.

278 0.9. Keeping It Current

279 The software industry changes rapidly. To deal with these changes and to provide the stakeholders with access
280 to relevant and current information, the ISTQB working groups have created links on the [Link]
281 website, which refer to supporting documents and changes to standards. This information is not examinable
282 under the Foundation syllabus.

283 0.10. Level of Detail

284 The level of detail in this syllabus allows internationally consistent courses and exams. In order to achieve
285 this goal, the syllabus consists of:
286 • General instructional objectives describing the intention of the Foundation Level
287 • A list of terms (keywords) that students must be able to recall
288 • Learning objectives for each knowledge area, describing the cognitive learning outcomes to be
289 achieved
290 • A description of the key concepts, including references to recognized sources
291 The syllabus content is not a description of the entire knowledge area of software testing; it reflects the
292 level of detail to be covered in Foundation Level training courses. It focuses on test concepts and

v4.0 Page 11 of 70 beta release

293 techniques that can be applied to all software projects independent of the software development lifecycle
294 employed.

295 0.11. How this Syllabus is Organized

296 There are six chapters with examinable content. The top-level heading for each chapter specifies the
297 training time for the chapter. Timing is not provided below the chapter level. For accredited training
298 courses, the syllabus requires a minimum of 18.75 hours (18 hours and 45 minutes) of instruction,
299 distributed across the six chapters as follows:
300 • Chapter 1: Fundamentals of Testing (190 minutes)
301 o The student learns the basic principles related to testing, the reasons why testing is
302 required, and what the test objectives are.
303 o The student understands the test process, the major test activities, and work products.
304 o The student understands the essential skills for testing.
305 • Chapter 2: Testing Throughout the Software Development Lifecycles (140 minutes)
306 o The student learns how testing is incorporated into different development approaches.
307 o The student learns the concepts of test-first approaches, as well as DevOps.
308 o The student learns about the different test levels, test types, and maintenance testing.
309 • Chapter 3: Static Testing (80 minutes)
310 o The student learns the static testing basics.
311 o The student learns about the feedback and review process.
312 • Chapter 4: Test Analysis and Design (390 minutes)
313 o The student learns how to apply black-box, white-box, and experience-based test
314 techniques to derive test cases from various software work products.
315 o The student learns about the collaboration-based test approach.
316 • Chapter 5: Managing the Test Activities (305 minutes)
317 o The student learns how to plan tests in general, and how to estimate test effort.
318 o The student learns how risks can influence the scope of testing.
319 o The student learns how to monitor and control test activities.
320 o The student learns how configuration management supports testing.
321 o The student learns how to report defects in a clear and understandable way.
322 • Chapter 6: Test Tools (20 minutes)
323 o The student learns to classify tools and to understands the risks and benefits of test
324 automation.

v4.0 Page 12 of 70 beta release

325 1. Fundamentals of Testing – 180 minutes

326 Keywords
327 coverage, coverage item, debugging, defect, error, failure, quality, quality assurance, root cause, test
328 analysis, test basis, test case, test completion, test condition, test control, test data, test design, test
329 execution, test implementation, test monitoring, test object, test objective, test planning, test procedure,
330 test result, testing, testware, validation, verification
331

332 Learning Objectives for Chapter 1:

333 1.1 What is Testing?

334 FL-1.1.1 (K1) Identify typical objectives of testing
335 FL-1.1.2 (K2) Differentiate testing from debugging

336 1.2 Why is Testing Necessary?

337 FL-1.2.1 (K2) Exemplify why testing is necessary
338 FL-1.2.2 (K1) Recall the relation between testing and quality assurance
339 FL-1.2.3 (K2) Distinguish between root cause, error, defect, and failure

340 1.3 Testing Principles

341 FL-1.3.1 (K2) Explain the seven testing principles

342 1.4 Test Activities, Test Work Products and Test Roles
343 FL-1.4.1 (K2) Summarize the different test activities and tasks
344 FL-1.4.2 (K2) Explain the impact of context on the test process
345 FL-1.4.3 (K2) Differentiate the work products that support the test activities
346 FL-1.4.4 (K2) Explain the value of maintaining traceability
347 FL-1.4.5 (K2) Compare the different roles in testing

348 1.5 Essential Skills and Good Practices in Testing

349 FL-1.5.1 (K2) Give examples of the generic skills required for testing
350 FL-1.5.2 (K1) Recall the advantages of the whole team approach
351 FL-1.5.3 (K2) Distinguish the benefits and drawbacks of independence of testing

v4.0 Page 13 of 70 beta release

352 1.1. What is Testing?

353 Software systems are an integral part of our daily life. Most people have had experience with software
354 that did not work as expected. Software that does not work correctly can lead to many problems,
355 including loss of money, time or business reputation, and, in extreme cases, even injury or death.
356 Software testing assesses the quality of the software and contributes to reducing the risk of software
357 failure in operation.
358 Software testing is a set of activities conducted to facilitate the discovery of defects and the evaluation of
359 properties of software artifacts. These artifacts under test are known as test objects. A common
360 misconception about testing is that it only consists of executing tests (i.e., running the software and
361 checking the results). However, software testing includes also other activities (see chapter 2).
362 Another common misconception about testing is that testing focuses entirely on the verification of the test
363 object. While testing does involve checking whether the system meets specified requirements, it also
364 involves checking whether the system meets users’ and other stakeholders’ needs in its operational
365 environment, which is called validation.
366 Testing may be dynamic or static. Dynamic testing involves the execution of software, while static testing
367 does not. Static testing includes reviews (see chapter 3) and static analysis. Dynamic testing uses
368 different types of test techniques to derive test cases (see chapter 4).
369 Testing is not only a technical activity. It also needs to be properly planned, managed, estimated,
370 monitored and controlled (see chapter 5).
371 Testers use tools (see chapter 6), but it is important to remember that testing is largely an intellectual
372 activity, requiring the testers to have specialized knowledge, use analytical skills and apply critical
373 thinking and systems thinking (Myers 2011, Roman 2018).
374 The ISO/IEC/IEEE 29119-1 standard has further information about software testing concepts.

375 1.1.1. Objectives of Testing

376 The typical objectives of testing are:
377 • Evaluating work products such as requirements, user stories, designs, and code
378 • Identifying failures and finding defects
379 • Ensuring proper coverage of a test object
380 • Reducing the level of risk of inadequate software quality
381 • Verifying whether specified requirements have been fulfilled
382 • Verifying that a test object complies with contractual, legal, and regulatory requirements
383 • Providing information to stakeholders to allow them to make informed decisions
384 • Building confidence in the quality of the test object
385 • Validating whether the test object is complete and works as the stakeholders expect
386 The objectives of testing can vary, depending upon the context, which includes the work product being
387 tested, the test level, and the SDLC being followed.

v4.0 Page 14 of 70 beta release

388 1.1.2. Testing and Debugging

389 Testing and debugging are separate activities. Testing can show failures that are caused by defects in the
390 software (dynamic testing) or can directly find defects in the test object (static testing).
391 If dynamic testing finds a failure, debugging is concerned with finding causes of this failure (defects),
392 analyzing these causes, and eliminating them. The typical debugging process in this case involves:
393 • Reproduction of a failure
394 • Diagnosis (finding the cause)
395 • Fixing the cause
396 Subsequent confirmation testing checks whether the fixes resolved the problem. Preferably, confirmation
397 testing is done by the same person who performed the initial test. Subsequent regression testing can also
398 be performed, to check whether the fixes are causing failures in other parts of the test object (see section
399 2.2.3 for more information on confirmation testing and regression testing).
400 If static testing finds a defect, debugging is concerned with eliminating it. There is no need of reproduction
401 or diagnosis, since static testing directly finds defects, not failures (see chapter 3).

402 1.2. Why is Testing Necessary?

403 The testing of components, systems and their associated documentation supports the identification of
404 defects in software. Testing also detects gaps and other deficiencies in the specifications for the software.
405 Hence, testing can help to reduce the risk of failures occurring during operation. When defects are
406 detected and fixed, this contributes to improving the quality of the test object. In addition, software testing
407 may also be required to meet contractual or legal requirements or to comply with regulatory standards.

408 1.2.1. Testing’s Contributions to Success

409 Testing helps in achieving the agreed upon goals within the set scope, time, quality, and budget
410 standards. The success can be considered in terms of:
411 • Product quality (e.g., detecting defects allows to remove them in the debugging process,
412 therefore testing contributes to increase the quality of the system under test)
413 • Process quality (e.g., introducing test automation improves the efficiency of the release process;
414 applying risk-based testing optimizes the testing effort)
415 • Project goals (e.g., using static testing early in the project reduces the software maintenance
416 costs and improves the developers’ effectiveness by reducing time spent for fixing defects)
417 • People skills (e.g., performing code reviews increases code understanding and allows less
418 experienced developers to improve their programming and designing skills)
419 Testing’s contribution to success should not be restricted to the test team activities only. Any stakeholder
420 can use their testing skills to bring the project closer to success.

v4.0 Page 15 of 70 beta release

421 1.2.2. Testing and Quality Assurance (QA)

422 While people often use the terms “testing” and “quality assurance” (QA) interchangeably, testing and QA
423 are not the same. Testing is a form of quality control (QC). QA is typically focused on establishing,
424 introducing, monitoring, improving, and adhering to the quality-related processes. When proper processes
425 are carried out correctly, this contributes to defect prevention, and improves confidence that appropriate
426 levels of quality in the work products will be achieved. QA, when applied to software development and
427 software maintenance, should also be applied to software testing, which is part of each of these activities.
428 In addition, the use of root cause analysis to detect the causes of defects, and the application of the
429 findings of retrospective meetings to improve processes, are also important for effective QA.
430 A larger concept, quality management (QM), ties together QA and QC. QM includes all activities that
431 direct and control an organization with regard to quality. QM includes both QA and testing.

432 1.2.3. Root Causes, Errors, Defects, and Failures

433 Human beings make errors (mistakes), which produce defects (faults, bugs), which in turn may result in
434 failures. Humans make mistakes for various reasons, such as time pressure, complexity of code,
435 infrastructure or interactions, or simply because they are tired or lack adequate training.
436 The defects can be in documents, such as a requirements specification or a test script, in source code, or
437 in a supporting artifact, such as a build file. Defects in artifacts produced earlier in the lifecycle, such as
438 the requirements, if undetected, often lead to defective artifacts later in the lifecycle, such as the code. If a
439 defect in code is executed, the system may fail to do what it should do (or do something it shouldn’t),
440 causing a failure. Some defects will always result in a failure if executed, while others will only result in a
441 failure in specific circumstances, and some may never result in a failure.
442 Errors and defects are not the only cause of failures. Failures can also be caused by environmental
443 conditions, such as when radiation or electromagnetic field cause defects in firmware.
444 A root cause is a fundamental reason for the occurrence of a problem. Root cause may be a situation or
445 error that leads to a defect. Root causes are identified through root cause analysis, which is typically
446 performed when a failure occurs, and it is believed that further similar failures can be prevented or their
447 frequency reduced by addressing the root cause, such as by removing it.

448 1.3. Testing Principles

449 A number of testing principles offering general guidelines common to all testing have been suggested
450 over the past 60 years. This syllabus describes seven such principles.
451 1. Testing shows the presence, not the absence of defects. Testing can show that defects are present
452 in the test object but cannot prove that there are no defects (Buxton 1970). Testing reduces the
453 probability of undiscovered defects remaining in the test object, but, even if no defects are found, testing
454 cannot prove test object correctness.
455 2. Exhaustive testing is impossible. Testing everything is not feasible except in trivial cases (Manna
456 1978). Rather than attempting to test exhaustively, test techniques (see chapter 4), test case prioritization
457 (see section 5.1.5), and risk-based testing (see section 5.2), should be used to focus test efforts.
458 3. Early testing saves time and money. Defects that are removed early in the process will not cause
459 subsequent defects in derived work products. The total cost of quality will be reduced since fewer failures

v4.0 Page 16 of 70 beta release

460 will occur later in the lifecycle (Boehm 1981). To find defects early, both static testing (see chapter 3) and
461 dynamic test activities (see chapter 4) should be started as early as possible.
462 4. Defects cluster together. A small number of system components usually contain most of the defects
463 discovered or are responsible for most of the operational failures (Enders 1975). This phenomenon is an
464 illustration of the Pareto principle. Predicted defect clusters, and actual defect clusters observed during
465 testing or in operation, are an important input for risk-based testing (see section 5.2).
466 5. Tests wear out. If the same tests are repeated many times, they stop being effective in detecting new
467 defects (Beizer 1990). To overcome this, existing tests and test data may need changing, and new tests
468 may need to be written. However, in some cases, repeating the same tests can have the beneficial
469 outcome, e.g., in automated regression testing (see section 2.2.3).
470 6. Testing is context dependent. There is no single universally applicable approach to testing. Testing is
471 done differently in different contexts (Kaner 2011).
472 7. Absence-of-errors fallacy. It is a fallacy (i.e., a mistaken belief) to expect that software verification will
473 ensure the success of a system. Thoroughly testing all the specified requirements and fixing all the
474 defects found could still produce a system that does not fulfill the users’ needs and expectations, that
475 does not help in achieving the customer’s business goals and that is inferior compared to other competing
476 systems. In addition to verification, validation should also be carried out (Boehm 1981).

477 1.4. Test Activities, Test Work Products and Test Roles
478 Testing is context dependent, but, at a high level, there are common sets of test activities without which
479 testing is less likely to achieve its objectives. These sets of test activities form a test process. The test
480 process can be tailored for a given situation based on various factors. Which test activities are included in
481 this test process, how they are implemented, and when they occur are normally decided as part of the
482 test planning for the specific situation (see chapter 5).
483 The following sections describe general aspects of this test process in terms of test activities and tasks,
484 the impact of context, test work products, traceability between the test basis and test work products, and
485 testing roles.
486 The ISO/IEC/IEEE 29119-2 standard has further information about test processes.

487 1.4.1. Test Activities and Tasks

488 A test process usually consists of the main groups of activities described below. Although many of these
489 activities may appear to follow a logical sequence, they are often implemented iteratively or in parallel.
490 Tailoring of these test activities within the context of the system and the project is usually required.
491 Test planning includes defining the test objectives and the test approach for meeting them within the
492 constraints imposed by the context. Test planning is further explained in section 5.1.
493 Test monitoring and control. Test monitoring involves the on-going checking of all activities and the
494 comparison of actual progress against the test plan. Test control involves taking the actions necessary to
495 meet the objectives of the test plan. Test monitoring and control are further explained in section 5.3.
496 Test analysis includes analyzing the test basis to identify testable features and to define and prioritize
497 associated test conditions, together with the related risks and risk levels (see section 5.2). Test basis and
498 test objects are also evaluated to identify defects they may contain and to assess their testability. Test

v4.0 Page 17 of 70 beta release

499 analysis is often supported by the use of test techniques (see chapter 4). Test analysis answers the
500 question “what to test?” in terms of measurable coverage criteria.
501 Test design includes elaborating the test conditions into test cases and other testware (e.g., test
502 charters). This activity often involves the identification of coverage items, which serve as a guide to
503 specify test case inputs. Test techniques (see chapter 4) can be used to support this activity. Test design
504 also includes test data identification, designing the test environment and identifying any other required
505 infrastructure and tools. Test design answers the question “how to test?”.
506 Test implementation includes creating or acquiring the testware necessary for test execution (e.g., test
507 data). Test cases are organized into test procedures. Automated test scripts are created. Test procedures
508 are prioritized and arranged within a test execution schedule for efficient test execution (see section
509 5.1.5). Test environment is built and verified to be set up correctly.
510 Test execution includes running the test procedures in accordance with the test execution schedule.
511 Test execution may be manual or automatic. Test execution can take the form of continuous testing or
512 pair testing sessions. Actual test results are compared with expected results. Anomalies are analyzed to
513 identify their likely causes. Test execution outcome is logged. Defects are reported based on the failures
514 observed (see section 5.5).
515 Test completion activities occur at project milestones (e.g., release, end of iteration, test level
516 completion). Change requests or product backlog items for any unresolved defects are created. Any
517 testware that may be useful in the future is identified and archived or handed over to the appropriate
518 teams. The test environment is shut down to an agreed state. The completed test activities are analyzed
519 to identify lessons learned and identify improvements for future iterations, releases, or projects (see
520 section 2.1.6). A test completion report is created and communicated to stakeholders.

521 1.4.2. Test Process in Context

522 Testing is not performed in isolation. Testing is subservient to the development processes carried out
523 within a specific organization. Testing is also sponsored by stakeholders and its final goal is to help fulfill
524 the stakeholders’ business needs. Therefore, the way the testing is carried out will depend on a number
525 of contextual factors including:
526 • Stakeholders (needs, expectations, requirements, willingness to cooperate, etc.)
527 • Team members (skills, knowledge, level of experience, availability, training needs, etc.)
528 • Business domain (type of software, identified risks, market needs, specific legal regulations, etc.)
529 • Technical factors (product architecture, technology used, etc.)
530 • Project constraints (scope, time, budget, resources, etc.)
531 • Organizational factors (organizational structure, existing policies, practices used, etc.)
532 • SDLC (engineering techniques, development methods, etc.)
533 • Tools (availability, difficulty of use, etc.)
534 These factors will have an impact on many test-related issues, including: test strategy, test techniques
535 used, degree of test automation, required level of test coverage in relation to requirements and identified
536 risks, level of detail of test documentation, reporting etc.

v4.0 Page 18 of 70 beta release

537 1.4.3. Test Work Products

538 Test work products are created as outputs from the test activities described in section 1.4.1. There is a
539 significant variation in the work products and their naming across organizations, regarding the way they
540 are organized and managed. The following list of work products is by no means exhaustive.
541 Test planning work products include: test strategy (usually in larger projects), test plan, risk register,
542 and exit criteria (see section 5.1). Risk register and exit criteria are often a part of the test plan.
543 Test monitoring and control work products include: test progress reports (see section 5.3.2),
544 documentation of control directives (see section 5.3) and risk information (see section 5.2).
545 Test analysis work products include: (prioritized) test conditions, acceptance criteria (see section
546 4.5.2), and defect reports regarding defects in the test basis (if not fixed directly).
547 Test design work products include: test cases, coverage items, test data requirements and test
548 environment design.
549 Test implementation work products include: test procedures, automated test scripts, test suites, test
550 data, test execution schedule, and test environment elements. Examples of test environment elements
551 include: stubs, drivers, simulators, and service virtualizations.
552 Test execution work products include: test logs, documentation of the status of individual test cases,
553 defect reports (see section 5.5.1), and documentation about which test objects, test tools, and testware
554 were involved in the testing.
555 Test completion work products include: test completion report (see section 5.3.2), action items for
556 improvement of subsequent projects or iterations, and change requests (e.g., as product backlog items).

557 1.4.4. Traceability between the Test Basis and Test Work Products
558 In order to implement effective test monitoring and control, it is important to establish and maintain
559 traceability throughout the test process between the test basis elements, test work products associated
560 with these elements (e.g., test conditions, risks, test cases), test results, and detected defects.
561 Accurate traceability supports test coverage evaluation, so it is very useful if the test basis has
562 measurable coverage criteria defined. The coverage criteria can function as key performance indicators
563 to drive the activities that demonstrate the achievement of test objectives (see section 1.1.1). For
564 example, by using the traceability from:
565 • Test cases to requirements, the requirements coverage by test cases can be verified
566 • Test case results to risks, the level of residual risk in a test object can be evaluated.
567 In addition to the evaluation of coverage, good traceability allows to determine the impact of changes,
568 facilitates the auditing of testing, and supports the achievement of IT governance criteria. Good
569 traceability also improves the understandability of test progress reports and test completion reports by
570 including the status of test basis elements. This can also make the communication of technical aspects of
571 testing to stakeholders easier, in terms that they can understand. Good traceability provides information
572 used to assess product quality, process capability, and project progress against business goals.

v4.0 Page 19 of 70 beta release

573 1.4.5. Roles in Testing

574 In this syllabus, two principal roles in testing are covered: a test management role and a testing role. The
575 activities and tasks assigned to these two roles depend on factors such as the project and product
576 context, the skills of the people in the roles, and the organization.
577 The test management role takes overall responsibility for the test process, test team and leadership of the
578 test activities. The test management tasks mainly concentrate on test planning, test monitoring and
579 control and test completion activities. The testing role takes overall responsibility for the engineering
580 (technical) aspect of testing. The testing tasks mainly concentrate on test analysis, test design, test
581 implementation and test execution activities.
582 The way in which the test management role is carried out varies depending on the context. For example,
583 in Agile software development, some of the test management tasks may be handled by the Agile team.
584 Tasks that span multiple teams or the entire organization may be performed by test managers outside of
585 the development team.
586 Different people may take over these roles at different times. For example, the test management role can
587 be performed by a team leader, by a test manager, by a development manager, etc. It is also possible
588 that one person can take both the testing and test management roles at the same time.

589 1.5. Essential Skills and Good Practices

590 Skill is the ability to do something well that comes from one’s knowledge, practice and aptitude. Good
591 testers should possess some essential skills to do their job efficiently and effectively. Good testers should
592 also be the effective team players and perform testing on different levels of independence.

593 1.5.1. Generic Skills Required for Testing

594 While being generic, the following skills are particularly relevant for testers:
595 • Thoroughness, carefulness, curiosity, attention to details, being methodical (to identify different
596 types of defects, especially the ones that are difficult to find)
597 • Good communication skills, active listening, being a team player (to interact effectively with all
598 stakeholders, to convey information to others, to be understood, to report and discuss defects)
599 • Analytical thinking, critical thinking, creativity (to increase effectiveness of testing)
600 • Technical knowledge (to increase efficiency of testing, e.g., by using test tools)
601 • Knowledge of estimation techniques (to estimate the test effort more accurately)
602 • Domain knowledge (to be able to understand and to communicate with end users)
603 Testers are often the bearers of bad news. It is a common human trait to blame the bearer of bad news.
604 This makes communication skills crucial for testers. Communicating testing results may be perceived as
605 criticism of the product and of its author. Confirmation bias can make it difficult to accept information that
606 disagrees with currently held beliefs. Some people may perceive testing as a destructive activity, even
607 though it contributes greatly to project progress and product quality. To try to improve this view, information
608 about defects and failures should be communicated in a constructive way.

v4.0 Page 20 of 70 beta release

609 Defining the right set of test objectives (see section 1.1.1) can have important psychological implications
610 as most people tend to align their plans and behaviors with the set objectives.

611 1.5.2. Whole Team Approach

612 One of the important testing skills is being a team player, having the ability to work effectively in a team
613 context and to contribute positively to the team goal. The whole team approach builds upon this skill.
614 The whole team approach involves everyone with the necessary knowledge and skills to ensure project
615 success by making quality everyone’s responsibility. The team members share the same workspace, as
616 co-location facilitates communication and interaction. The whole team approach improves team
617 dynamics, enhances communication and collaboration within the team, and creates synergy by allowing
618 the various skill sets within the team to be leveraged for the benefit of the project.
619 Testers work closely with other team members to ensure that the desired quality levels are achieved. This
620 includes collaborating with business representatives to help them create suitable acceptance tests and
621 working with developers to agree on the testing strategy and decide on test automation approaches.
622 Testers can thus transfer and extend testing knowledge to other team members and influence the
623 development of the product.
624 Depending on the context, the whole team approach may be not sufficient requiring a higher level of
625 testing independence (e.g., safety-critical systems).

626 1.5.3. Independence of Testing

627 A certain degree of independence makes the tester more effective at finding defects due to differences
628 between the author’s and the tester’s cognitive biases. Independence is not, however, a replacement for
629 familiarity, and developers can efficiently find many defects in their own code.
630 Work products can be tested by its author (no independence), by the author’s peer from the same team
631 (some independence), by the testers external to the author’s team, but within the organization (high
632 independence), or by the testers external to the organization (very high independence). For most
633 projects, it is usually best to carry out testing with multiple levels of independence (e.g., developers
634 performing component and component integration testing, test team performing system and system
635 integration testing, and business representatives performing acceptance testing).
636 The main benefit of test independence is that independent testers are likely to recognize different kinds of
637 failures compared to developers because of their different backgrounds, technical perspectives, and
638 biases. Moreover, an independent tester can verify, challenge, or disprove assumptions made by
639 stakeholders during specification and implementation of the system.
640 However, there are also some drawbacks. Independent testers may be isolated from the development
641 team, which may lead to a lack of collaboration, communication problems, or an adversarial relationship
642 with the development team. Developers may lose a sense of responsibility for quality. Independent
643 testers may be seen as a bottleneck or be blamed for delays in release.

v4.0 Page 21 of 70 beta release

644 2. Testing Throughout the Software Development Lifecycles

645 – 130 minutes
646 Keywords
647 acceptance testing, component integration testing, component testing, confirmation testing, functional
648 testing, integration testing, maintenance testing, non-functional testing, operational acceptance testing,
649 regression testing, shift-left, system integration testing, system testing, test basis, test environment, test
650 level, test object, test type, user acceptance testing, white-box testing
651

652 Learning Objectives for Chapter 2:

653 2.1 Testing in Context of Software Development Lifecycles

654 FL-2.1.1 (K2) Explain the impact of the chosen software development lifecycle on testing
655 FL-2.1.2 (K1) Remember good testing practices regardless of the chosen software development
656 model
657 FL-2.1.3 (K1) Recall the examples of test-first approaches to development
658 FL-2.1.4 (K2) Summarize how DevOps might have an impact on testing
659 FL-2.1.5 (K2) Explain the shift-left approach
660 FL-2.1.6 (K2) Explain how retrospectives can be used as a mechanism for process improvement

661 2.2 Test Levels and Test Types

662 FL-2.2.1 (K2) Distinguish the different test levels
663 FL-2.2.2 (K2) Compare and contrast functional, non-functional and white-box testing
664 FL-2.2.3 (K2) Distinguish confirmation testing from regression testing

665 2.3 Maintenance Testing

666 FL-2.3.1 (K2) Summarize maintenance testing and its triggers

v4.0 Page 22 of 70 beta release

667 2.1. Testing in Context of Software Development Lifecycles

668 A software development lifecycle (SDLC) model is an abstract, high-level representation of the software
669 development process. A SDLC model defines how different development phases and types of activities
670 performed within this process relate to each other, both logically and chronologically. Examples of SDLC
671 models include: sequential models (e.g., waterfall model, V-model), iterative models (e.g., spiral model),
672 and incremental models.
673 Software development processes can be also described by more detailed models, e.g., various software
674 development methods and agile practices. Examples include: acceptance test-driven development
675 (ATDD), behavior-driven development (BDD), domain-driven design (DDD), extreme programming (XP),
676 feature-driven development (FDD), Kanban, Lean IT, Scrum, test-driven development (TDD).

677 2.1.1. Impact of Software Development Lifecycle on Testing

678 Testing must be integrated into the software lifecycle to succeed. The choice of SDLC impacts on:
679 • Scope and timing of test activities (e.g., test levels and test types)
680 • Level of detail of test documentation
681 • Choice of test techniques and test practices
682 • Extent of test automation
683 In sequential models, in initial phases testers typically participate in requirement reviews and test design.
684 The product in the executable form is usually delivered in the late phases, so typically dynamic testing
685 cannot be performed early in the lifecycle.
686 In some iterative and incremental models, it is assumed that each iteration ends up with a working
687 product increment. This implies that in each iteration testing, both static and dynamic, may be performed
688 at all test levels. Frequent delivery of increments requires fast feedback and extensive regression testing.
689 Agile development methods assume that change may occur throughout the project. Therefore, lightweight
690 work product documentation and extensive test automation to make regression testing easier to handle
691 are favored in Agile projects. Also, most of the manual testing tends to be done using experience-based
692 techniques (see Section 4.4) that do not require extensive prior planning.

693 2.1.2. Software Development Lifecycles and Good Testing Practices

694 Good testing practices independent of the chosen SDLC model, include the following:
695 • For every software development activity, there is a corresponding test activity, so that the quality
696 control can cover all the aspects
697 • Each test level (see chapter 2.2.1) has test objectives specific to the appropriate SDLC phase or
698 type of activities, so that testing can check the test object to the fullest extent possible
699 • Test analysis and design for a given test level begin during the corresponding development
700 phase of the SDLC, so that testing can adhere to the early testing principle (see section 1.3)
701 • Testers are involved in reviewing work products as soon as drafts of these documents are
702 available, so that the shift-left approach is followed (see section 2.1.5)

v4.0 Page 23 of 70 beta release

703 2.1.3. Testing as a Driver for Software Development

704 Test-driven development (TDD), acceptance test-driven development (ATDD), and behavior-driven
705 development (BDD) are similar development approaches, where tests are defined as a means of directing
706 development. Each of these approaches implements the testing principle of “Early testing saves time and
707 money” (see section 1.3) and follows a shift-left approach (see section 2.1.5), since the tests are defined
708 before the code is written. They support an iterative approach to development. Those approaches are
709 characterized as follows:
710 Test-Driven Development (TDD):
711 • TDD directs the coding through test cases (instead of extensive software design)
712 • Tests are written first, then the code is written to satisfy the tests, and then the tests and code are
713 refactored
714 Acceptance Test-Driven Development (see section 4.5.3):
715 • Derive tests from acceptance criteria as part of the design process (Gärtner 2011)
716 • Tests are written even before the part of the application is developed to satisfy the tests
717 Behavior-Driven Development (BDD):
718 • Express the desired behavior of an application by test cases written in a simple form of natural
719 language, that is easy to understand by stakeholders – usually using the given/when/then format.
720 (Chelimsky 2010)
721 • Test cases are then compiled and translated in (automatically) executable tests
722 For all the above approaches, tests may persist as automated tests to ensure the code quality in future
723 adaptions / refactoring.

724 2.1.4. DevOps and Testing

725 DevOps is an organizational transformation aiming to create synergy by getting development, testing and
726 operations to work together to achieve a set of common goals. DevOps requires a cultural shift within an
727 organization to bridge the gaps between development, testing and operations while treating their
728 functions with equal value. DevOps promotes team autonomy, fast feedback, integrated toolchains, and
729 technical practices like continuous integration (CI) or continuous delivery. This allows the teams to build,
730 test and release high-quality code faster through a DevOps delivery pipeline (Kim 2016).
731 From the testing perspective, the benefits of DevOps are:
732 • Fast feedback on the code quality, and whether changes adversely affect existing code
733 • CI creates a shift-left in testing (see section 2.1.5) by encouraging developers to submit high
734 quality code accompanied by component tests
735 • DevOps facilitates establishing stable test environments
736 • Automation through a delivery pipeline reduces the need for repetitive manual testing
737 • The risk of regression is minimized due to the scale and range of automated regression tests

v4.0 Page 24 of 70 beta release

738 DevOps is not without its risks and challenges, which include:
739 • The DevOps delivery pipeline must be defined and established
740 • CI tools have to be introduced and maintained
741 • Test automation requires additional resources and may be difficult to establish and maintain

742 2.1.5. Shift-Left Approach

743 The testing principle “Early testing saves time and money” (see section 1.3) is sometimes referred to as
744 “shift-left” because it is an approach where testing is performed earlier in the life cycle. Shift-left normally
745 suggests that testing should be done earlier (e.g., not waiting for code to be implemented or for
746 components to be integrated), but it does not mean that testing later in the life cycle should be neglected.
747 There are some good practices that illustrate how to achieve a “shift-left” in testing, which include:
748 • Review the specification from the perspective of testing. These specification review activities
749 often find potential defects, such as ambiguities, incompleteness, and inconsistencies
750 • Write tests before the code is written and have the code run against a test harness during
751 implementation
752 • Perform CI and continuous delivery as it comes with fast feedback and automated component
753 tests to accompany source code when it is submitted to the code repository
754 • Perform static analysis of source code prior to dynamic testing, or as part of an automated
755 process
756 • Perform non-functional testing at the component testing level, where possible. This is a form of
757 shift-left as these non-functional test types tend to be performed later in the SDLC when a
758 complete system and a representative test environment are available.
759 A shift-left approach might result in extra training/effort/costs earlier in the process.

760 2.1.6. Retrospectives and Process Improvement

761 Retrospectives (also known as “lessons learned meetings” or evaluations) might be held when needed,
762 often at the end of a project, release milestone or iteration. In these meetings the participants (not only
763 testers, but also e.g., developers, architects, product owner, business analysts) discuss:
764 • what was successful,
765 • what was not successful and could be improved, and
766 • how to incorporate the improvements and retain the successes in the future.
767 The results should be recorded and might be part of e.g., the test completion report (see section 5.3.2). It
768 is important that follow-up activities occur. Retrospectives are critical to the successful self-organization of
769 the development teams and the continuous improvement.
770 Typical benefits for testing include:
771 • Increased test effectiveness / productivity

v4.0 Page 25 of 70 beta release

772 • Increased test case quality

773 • Team satisfaction
774 • Improved requirements quality
775 • Better cooperation of development and testing
776 The timing and organization of the retrospectives depend on the particular SDLC model being followed.

777 2.2. Test Levels and Test Types

778 Test levels are groups of test activities that are organized and managed together. Each test level is an
779 instance of the test process, performed in relation to software at a given stage of development, from
780 individual components to complete systems or, where applicable, systems of systems.
781 Test levels are related to other activities within the SDLC. In sequential SDLC models, the test levels are
782 often defined such that the exit criteria of one level are part of the entry criteria for the next level. In some
783 iterative models, this rule may not apply. Development activities may span through multiple test levels.
784 Test levels may overlap.
785 Test types are groups of test activities related to specific characteristics and those test activities can be
786 performed at every test level.

787 2.2.1. Test Levels

788 In this syllabus, the following five test levels are described.
789 • Component testing (also known as unit testing) focuses on testing components in isolation. It
790 often requires specific support, such as test harnesses or unit testing frameworks. Component
791 testing is normally performed by developers in their development environments.
792 • Component integration testing (also known as unit integration testing) focuses on testing the
793 interfaces and interactions between integrated components. Component integration testing is
794 heavily dependent on the integration strategy.
795 • System testing focuses on the overall behavior and capabilities of an entire system or product,
796 often including functional testing of end-to-end tasks and the non-functional testing of quality
797 characteristics. For some non-functional quality characteristics, it is preferred to test them on a
798 complete system in a representative test environment (e.g., performance efficiency, security or
799 usability). Using simulations is also possible. System testing is normally performed by the
800 independent test team and relies heavily on specifications.
801 • System integration testing focuses on testing the interfaces and interactions between
802 integrated systems or external services. System integration testing requires suitable test
803 environments preferably similar to the operational environment.
804 • Acceptance testing focuses on validation and on demonstrating readiness for deployment,
805 which means that the system fulfills the user’s business needs. Ideally, acceptance testing should
806 be performed by the end users. The main forms of acceptance testing are: user acceptance

v4.0 Page 26 of 70 beta release

807 testing (UAT), operational acceptance testing (OAT), contractual/regulatory acceptance testing,
808 alpha and beta testing.
809 Test levels are characterized by the following non-exhaustive list of attributes:
810 • Test object
811 • Test objectives
812 • Test basis
813 • Defects and failures
814 • Approach and responsibilities

815 2.2.2. Test Types

816 The following test types are addressed:
817 Functional testing involves tests that evaluate the functions that a component or system should perform.
818 Functional requirements may be described in work products such as requirements specifications, user
819 stories, use cases, functional specifications, or they may be undocumented. The functions are “what” the
820 test object should do.
821 Non-functional testing evaluates attributes other than functional characteristics of systems and
822 software. The ISO/IEC 25010 standard provides the following classification of the non-functional software
823 product quality characteristics:
824 • Performance efficiency
825 • Compatibility
826 • Usability
827 • Reliability
828 • Security
829 • Maintainability
830 • Portability
831 Non-functional testing is the testing of “how well the system behaves”. Non-functional testing can and
832 often should start as early as possible. The late discovery of non-functional defects can pose a serious
833 threat to the success of a project. Non-functional testing sometimes needs a very specific test
834 environment, such as a usability lab for usability testing.
835 Similar to functional testing, different test techniques can be used to derive test conditions and test cases
836 for non-functional testing.
837 White-box testing derives tests from the system's internal structure or implementation, contrary to
838 functional and non-functional testing, where tests are derived from the requirements specifications.
839 Internal structure may include code, architecture, work flows, and data flows within the system (see
840 section 4.3). White-box test design, implementation and execution requires special skills or knowledge,
841 such as the process of building code, how data is stored, and how to use coverage tools and to correctly
842 interpret their results.

v4.0 Page 27 of 70 beta release

843 All the three above mentioned test types can be applied to all test levels, although the focus will be
844 different at each level. Every test type can also be applied using static testing. The testing quadrants
845 show the test types and test levels from different perspectives (see section 5.1.7).

846 2.2.3. Confirmation Testing and Regression Testing

847 Changes are typically made to a component or system to either enhance it by adding a new feature or to
848 fix it by removing a defect. Testing should confirm that the changes have correctly implemented the
849 functionality or corrected the defect.
850 Confirmation testing is to confirm that the original defect has been successfully fixed. Depending on the
851 risk, one can test the fixed version of the software in several ways, including:
852 • with all the test cases that previously have failed due to the defect, or
853 • adding new tests to cover any changes that were needed to fix the defect
854 However, when time or money is short, confirmation testing might be restricted to simply exercising the
855 steps that should reproduce the failure caused by the defect and checking that the failure does not occur.
856 Regression testing is to confirm that no adverse consequences have been caused by a change,
857 including a fix that has already been confirmation tested. These adverse consequences could affect the
858 same component where the change was made, other components in the same system, or even other
859 connected systems. Regression testing may not be restricted to the test object itself but can also be
860 related to the environment.
861 Confirmation and regression testing are needed on all test levels if defects are fixed and changes are
862 made on these test levels.
863 Regression test suites are run many times and generally evolve with each iteration or release, so
864 regression testing is a strong candidate for automation. Automation of these tests should start early in the
865 project (see chapter 6). Where automated builds and CI are used, such as in DevOps (see section 2.1.4),
866 it is good practice to also include automated regression testing. Depending on the situation, this may
867 include regression tests on different levels.

868 2.3. Maintenance Testing

869 Testing the changes to a system in production includes both evaluating the success of the change
870 implementation and the checking for possible regressions in parts of the system that remain unchanged
871 (which is usually most of the system). Maintenance can involve planned releases / deployments and
872 unplanned releases / deployments (hot fixes).
873 The scope of maintenance testing typically depends on:
874 • The degree of risk of the change
875 • The size of the existing system
876 • The size of the change
877 The triggers for maintenance can be classified as follows:

v4.0 Page 28 of 70 beta release

878 • Modifications, such as planned enhancements (i.e., release-based), corrective changes or hot
879 fixes
880 • Upgrades or migrations of the operational environment, such as from one platform to another,
881 which can require tests associated with the new environment as well as of the changed software,
882 or tests of data conversion when data from another application is migrated into the system being
883 maintained
884 • Retirement, such as when an application reaches the end of its life. When a system is retired, this
885 can require testing of data archiving if long data-retention periods are required. Testing of
886 restoring and retrieving procedures after archiving may also be needed.

v4.0 Page 29 of 70 beta release

887 3. Static Testing – 80 minutes

888 Keywords
889 anomaly, dynamic testing, formal review, informal review, inspection, review, static analysis, static testing,
890 technical review, walkthrough
891

892 Learning Objectives for Chapter 3:

893 3.1 Static Testing Basics

894 FL-3.1.1 (K1) Recognize types of products that can be examined by the different static testing
895 techniques
896 FL-3.1.2 (K2) Explain the value of static testing
897 FL-3.1.3 (K2) Compare and contrast static and dynamic testing

898 3.2 Feedback and Review Process

899 FL-3.2.1 (K1) Identify the benefits of early and frequent feedback
900 FL-3.2.2 (K2) Summarize the activities of the review process
901 FL-3.2.3 (K1) Recognize the different roles and responsibilities in a review
902 FL-3.2.4 (K2) Compare and contrast the different review types
903 FL-3.2.5 (K1) Recall the factors that contribute to a successful review

v4.0 Page 30 of 70 beta release

904 3.1. Static Testing Basics

905 In contrast to dynamic testing, static testing does not require the execution of the software being tested.
906 Processes, code, system architecture or other work products are evaluated through manual examination
907 (e.g., reviews) or with the help of a tool (e.g., static analysis). Goals include improving quality, detecting
908 defects and assessing characteristics like readability, completeness, correctness, testability or
909 consistency. Static testing can therefore be applied for both verification and validation.
910 Testers, business representatives and developers work together during example mappings, collaborative
911 user story writing and backlog refinement sessions to ensure user stories and related work products meet
912 certain criteria, e.g., the Definition of Ready (see section 5.1.3). Review techniques can be applied to
913 ensure user stories are complete and understandable and include testable acceptance criteria. By asking
914 the right questions, testers explore, challenge and help improve the proposed stories.
915 Static analysis (as part of static testing) can identify problems prior to dynamic testing while requiring less
916 effort, as no test cases are required, and it is typically performed using tools. Static analysis is often
917 incorporated into continuous integration (CI) frameworks (see section 2.1.4). While largely used to detect
918 specific code defects, static analysis is also used to evaluate maintainability and security.

919 3.1.1. Work Products Examinable by Static Testing

920 Almost any work product can be examined using static testing. Examples include requirement
921 specification documents, source code, test plans, test cases, test procedures, test charters, project
922 documentation, contracts, and models.
923 Any document that can be read and understood can be the subject of a review. However, for static
924 analysis, work products need a structure against which they can be checked (e.g., models, code, text with
925 a formal syntax). Work products that are not appropriate for static testing include those that are difficult to
926 interpret by human beings and that cannot be analyzed by tools (e.g., 3rd party executable code).

927 3.1.2. Value of Static Testing

928 Static testing can detect defects in the earliest phases of the SDLC, fulfilling the principle that “Early
929 testing saves time and money” (see section 1.3). It can also identify defects which cannot be detected by
930 dynamic testing (e.g., unreachable code, design patterns not followed, defects in non-executable work
931 products).
932 Static testing provides the ability to evaluate the quality of, and to build confidence in the work product
933 under review. Stakeholders can validate whether the documented requirements describe their actual
934 needs, while verifying them. Since static testing can be performed early in SDLC, a shared understanding
935 is created among those stakeholders involved in static testing. This shared understanding will also
936 improve the communication. For this reason, it is recommended to involve stakeholders from every
937 perspective.
938 Even though reviews can be expensive to implement, the overall project costs are usually much lower
939 than when no reviews are performed because less time and effort needs to be spent on fixing defects
940 later in the project. Participants in the review process also benefit from an improved shared
941 understanding of the product under review.
942 Code defects can be detected and removed using static analysis at a higher rate than dynamic testing,
943 usually resulting in both fewer defects and lower overall development effort.

v4.0 Page 31 of 70 beta release

944 3.1.3. Differences between Static and Dynamic Testing

945 Static and dynamic testing practices complement each other. They have similar objectives, such as
946 detecting defects in work products (see section 1.1.1), but there are also some differences, such as:
947 • Static testing can find different types of defects than dynamic testing
948 • Static testing finds defects directly, while dynamic testing causes failures from which the associated
949 defects are determined through subsequent analysis
950 • Static testing may more easily detect defects that lay on paths through the code that are rarely
951 executed or hard to reach using dynamic testing
952 • Static testing can be applied to non-executable work products, while dynamic testing is only
953 applicable to executable work products
954 • Dynamic testing can be used to measure characteristics (e.g., performance efficiency) that are
955 dependent on executing code
956 Typical defects that are easier and/or cheaper to find through static testing include:
957 • Defects in requirements (e.g., inconsistencies, ambiguities, contradictions, omissions, inaccuracies,
958 duplications)
959 • Design defects (e.g., inefficient database structures, high coupling, low cohesion, poor
960 modularization)
961 • Specific types of coding defects (e.g., variables with undefined values, undeclared variables,
962 unreachable or duplicated code, excessive code complexity)
963 • Deviations from standards (e.g., lack of adherence to naming conventions in coding standards)
964 • Incorrect interface specifications (e.g., mismatched number, type or order of parameters)
965 • Specific types of security vulnerabilities (e.g., susceptibility to buffer overflows)
966 • Gaps or inaccuracies in test basis coverage (e.g., missing tests for an acceptance criterion)

967 3.2. Feedback and Review Process

968 3.2.1. Benefits of Early and Frequent Customer Feedback

969 Early and frequent feedback allows for the early communication of potential quality problems. If there is
970 little stakeholder involvement during the SDLC, the product being developed might not meet the
971 stakeholder’s original, or current, vision. A failure to deliver what the stakeholder wants can result in
972 costly rework, missed deadlines, blame games, and might even lead to complete project failure.
973 Frequent stakeholder feedback throughout the SDLC can prevent misunderstandings about requirements
974 and ensure that changes to requirements are understood and implemented earlier. This helps the
975 development team to improve their understanding of what they are building. It allows them to focus on
976 those features that deliver the most value to the stakeholders and that have the most positive impact on
977 agreed risks.

v4.0 Page 32 of 70 beta release

978 3.2.2. Review Process Activities

979 The ISO/IEC 20246 standard defines a generic review process that provides a structured but flexible
980 framework from which a specific review process may be tailored for a particular situation. If the required
981 review is more formal, then more of the tasks described for the different activities will be needed.
982 The size of many work products makes them too large to be covered by a single review. In such cases,
983 the review process is typically applied multiple times to the individual parts that make up the work
984 product.
985 The activities in the review process are:
986 Planning. During the planning phase, the boundaries of the review are determined by answering the
987 who, what, where, when and why questions. The review types, review techniques, quality characteristics
988 to be evaluated, and standards to be followed are selected to answer the how question.
989 Review initiation. During review initiation, the goal is to make sure that everyone and everything
990 involved is prepared to start the actual review. This includes making sure that every participant has
991 access to the work product under review, understands their role and responsibilities and receives
992 everything needed to perform the review.
993 Individual review. Every reviewer performs an individual review to assess the quality of the work product
994 under review, and to identify anomalies by applying one or more review techniques (e.g., checklist-based
995 reviewing, scenario-based reviewing). The ISO/IEC 20246 standard provides more depth on different
996 review techniques. The reviewers log all their identified anomalies, recommendations, and questions.
997 Communication and analysis. Since the anomalies identified during a review are not necessarily
998 defects, all these anomalies need to be analyzed and discussed. For every anomaly, the decision should
999 be made on their status, ownership and required actions. This is typically done during a review meeting in
1000 which also a decision is made regarding the quality level of the work product under review and how the
1001 required actions will be followed-up. This follow-up may include another review.
1002 Fixing and reporting. For every accepted anomaly, a defect log should be created so that corrective
1003 actions can be followed-up. Once the exit criteria are reached, the work products can be accepted. All
1004 review results are reported.

1005 3.2.3. Roles and Responsibilities in Reviews

1006 Reviews involve various stakeholders, who may take on several roles. The principal roles and their
1007 responsibilities are:
1008 • Manager – decides what is to be reviewed and provides resources, such as staff and time for the
1009 review
1010 • Author – creates and fixes the work product under review
1011 • Facilitator (also known as the moderator) – ensures the effective running of review meetings,
1012 including mediation, time management, and the setting up a safe review environment
1013 • Secretary (also known as scribe or recorder) – collates anomalies from reviewers and records review
1014 information, such as decisions and new anomalies found during the review meeting. The author
1015 should not take the role of secretary to avoid biases

v4.0 Page 33 of 70 beta release

1016 • Reviewer – performs review. A reviewer may be someone working on the project, a subject matter
1017 expert, or any other stakeholder
1018 • Review leader – takes overall responsibility for the review such as deciding who will be involved, and
1019 organizing when and where the review will take place
1020 Other, more detailed roles are possible, as described in the ISO/IEC 20246 standard

1021 3.2.4. Review Types

1022 There exist many review types at various levels of formality, ranging from informal reviews to formal
1023 reviews. The required level of formality depends on factors such as the SDLC being followed, the maturity
1024 of the development process, the criticality and complexity of the work product being reviewed, any legal
1025 or regulatory requirements, and the need for an audit trail.
1026 Selecting the right review type is key to achieving the required review objectives. The selection is not only
1027 based on the objectives, but also on factors such as the project needs, available resources, work product
1028 type and risks, business domain, and company culture.
1029 Informal review (e.g., pair review). Informal reviews do not follow a defined process and have no formal
1030 documented output. The main objective is detecting potential anomalies.
1031 Walkthrough. A walkthrough, which is led by the author, can serve many objectives, like evaluating
1032 quality and building confidence in the work product, educating reviewers, gaining consensus, generating
1033 new ideas, motivating and enabling authors to improve and detecting potential defects. Reviewers might
1034 do an individual review before the walkthrough, but this is not required.
1035 Technical Review. The objectives of a technical review, performed by technically qualified reviewers, are
1036 to gain consensus and make decisions regarding a technical problem, but also to detect potential defects,
1037 evaluate quality and build confidence in the work product, generate new ideas, motivate and enable
1038 authors to improve.
1039 Inspection. As inspections are the most formal type of review, they follow the complete generic process
1040 as defined in section 3.2.2. The main objective is maximum defect yield. Other objectives are to detect
1041 potential defects, evaluate quality, build confidence in the work product and to motivate and enable
1042 authors to improve. Metrics are collected and used to improve the entire software development process,
1043 including the inspection process. In inspections, the author cannot act as the review leader, reader or
1044 recorder/scribe.

1045 3.2.5. Success Factors for Reviews

1046 There are several factors that determine the success of reviews, which include:
1047 Organizational success factors
1048 • Define clear objectives which can be used as measurable exit criteria. Evaluation of participants is
1049 never a good objective
1050 • Choose the appropriate review type for achieving the objectives, matching the type of work product
1051 and the review participants
1052 • Split large work products in small parts to make the required effort less intense

v4.0 Page 34 of 70 beta release

1053 • Provide feedback from reviews and stakeholders to authors so they can improve the product and their
1054 activities (see section 3.2.1)
1055 • Provide adequate time to participants to prepare for the review
1056 • Management should support the review process
1057 • Make reviews part of the organization culture, promoting learning and process improvement
1058 People-related success factors
1059 • Select the right participants for the review, representing different perspectives, including testers
1060 • Participants should dedicate adequate time for the review and pay attention to detail
1061 • Review meetings should be facilitated, not to waste anyone’s time
1062 • Adequate training should be provided
1063 See (Gilb 1993, Wiegers 2001) for more information on software reviews.

v4.0 Page 35 of 70 beta release

1064 4. Test Analysis and Design – 390 minutes

1065 Keywords
1066 acceptance criteria, acceptance test-driven development, black-box test technique, boundary value
1067 analysis, branch coverage, checklist-based testing, collaboration-based test approach, coverage,
1068 coverage item, decision table testing, equivalence partitioning, error guessing, experience-based test
1069 technique, exploratory testing, state transition testing, statement coverage, test technique, white-box test
1070 technique
1071

1072 Learning Objectives for Chapter 4:

1073 4.1 Test Techniques Overview

1074 FL-4.1.1 (K2) Distinguish black-box, white-box and experience-based test techniques

1075 4.2 Black-box Test Techniques

1076 FL-4.2.1 (K3) Use equivalence partitioning to derive test cases
1077 FL-4.2.2 (K3) Use boundary value analysis to derive test cases
1078 FL-4.2.3 (K3) Use decision table testing to derive test cases
1079 FL-4.2.4 (K3) Use state transition testing to derive test cases

1080 4.3 White-box Test Techniques

1081 FL-4.3.1 (K2) Explain statement testing
1082 FL-4.3.2 (K2) Explain branch testing
1083 FL-4.3.3 (K2) Explain the value of white-box testing

1084 4.4 Experience-based Test Techniques

1085 FL-4.4.1 (K2) Explain error guessing
1086 FL-4.4.2 (K2) Explain exploratory testing
1087 FL-4.4.3 (K2) Explain checklist-based testing

1088 4.5. Collaboration-based Test Approaches

1089 FL-4.5.1 (K2) Explain how to write user stories in collaboration with developers and business
1090 representatives
1091 FL-4.5.2 (K2) Classify the different options for writing acceptance criteria
1092 FL-4.5.3 (K3) Use acceptance test-driven development (ATDD) to derive test cases

v4.0 Page 36 of 70 beta release

1093 4.1. Test Techniques Overview

1094 Test techniques support the tester in test analysis (what to test) and in test design (how to test). Test
1095 techniques help to develop a relatively small, but good enough, set of test cases in a systematic way.
1096 Test techniques also help the tester to define test conditions, identify coverage item, and identify test data
1097 during the test analysis and design. Further information on test techniques and their corresponding
1098 measures can be found in the ISO/IEC/IEEE 29119-4 standard, and in (Beizer 1990, Craig 2002,
1099 Copeland 2004, Koomen 2006, Jorgensen 2014, Ammann 2016, Forgács 2019).
1100 In this syllabus, test techniques are classified as black-box, white-box, or experience-based.
1101 Black-box test techniques (also known as specification-based techniques) are based on an analysis of
1102 the specified behavior of the test object without reference to its internal structure. Hence, the test cases
1103 are independent of how the software is implemented and so if the implementation changes, but the
1104 required behavior stays the same, then the test cases are still useful.
1105 White-box test techniques (also known as structure-based techniques) are based on an analysis of the
1106 internal structure and processing within the test object. As the test cases are dependent on how the
1107 software is designed, they can only be created after the design or implementation of the test object.
1108 Experience-based test techniques leverage the knowledge and experience of testers for the design
1109 and implementation of test cases. Effectiveness of these techniques heavily depends on the tester’s
1110 skills. Experience-based test techniques can detect defects that may be missed using the black-box and
1111 white-box test techniques. Hence, experience-based techniques are complementary to the black-box and
1112 white-box test techniques.

1113 4.2. Black-Box Test Techniques

1114 Commonly used black-box test techniques discussed in the following sections are:
1115 • Equivalence Partitioning
1116 • Boundary Value Analysis
1117 • Decision Table Testing
1118 • State Transition Testing

1119 4.2.1. Equivalence Partitioning

1120 Equivalence Partitioning (EP) divides data into partitions (known as equivalence partitions) based on the
1121 expectation that all the elements of a given partition are to be processed in the same way by the test
1122 object. The theory behind this technique is that if a test case testing one value from an equivalence
1123 partition detects a defect, this defect should also be detected by test cases testing any other value from
1124 the same partition. Therefore, only one test for each partition is sufficient.
1125 Equivalence partitions can be identified for any data element related to the test object, including inputs,
1126 outputs, configuration items, internal values, time-related values, and interface parameters. The partitions
1127 may be continuous or discrete, ordered or unordered, finite or infinite.
1128 For simple test objects, EP can be easy, but in practice, understanding how the test object will treat
1129 different values is often complicated. Therefore, partitioning should be done with care.

v4.0 Page 37 of 70 beta release

1130 A partition containing valid values is called a valid partition. A partition containing invalid values is called
1131 an invalid partition. The definitions of valid and invalid values may vary among teams and organizations.
1132 For example, valid values may be interpreted as those that should be processed by the test object or as
1133 those for which the specification defines their processing. Invalid values may be interpreted as those that
1134 should be ignored or rejected by the test object or as those for which no processing is defined in the test
1135 object specification.
1136 In EP, the coverage items are the equivalence partitions. To achieve 100% coverage with this technique,
1137 test cases must exercise all identified partitions (including invalid partitions) by covering each partition at
1138 least once. Coverage is measured as the number of partitions exercised by at least one value, divided by
1139 the total number of identified partitions, normally expressed as a percentage.
1140 Many test objects include multiple sets of partitions (e.g., test objects with more than one input
1141 parameter), which means that each test case will cover partitions from different sets of partitions. The
1142 simplest coverage criterion in the case of multiple sets of partitions is called Each Choice coverage. Each
1143 Choice coverage requires test cases to exercise each equivalence partition at least once. Each Choice
1144 coverage does not take into account combinations of partitions.

1145 4.2.2. Boundary Value Analysis

1146 Boundary Value Analysis (BVA) is a technique based on exercising the boundaries of equivalence
1147 partitions. Hence, BVA can be used for ordered partitions only. The minimum and maximum values of a
1148 partition are its boundary values. In the case of BVA, if two elements belong to the same partition, all
1149 elements between them must also belong to that partition.
1150 BVA focuses on the boundary values of the partitions because developers are more prone to making
1151 mistakes with these boundary values. Typical boundary defects found by BVA are where implemented
1152 boundaries are displaced to positions above or below their intended positions or are omitted altogether.
1153 In this syllabus, two versions of the BVA are described: 2-value and 3-value BVA. They differ in terms of
1154 coverage items per boundary that need to be exercised to achieve 100% coverage.
1155 In 2-value BVA (Craig 2002, Myers 2011), for each boundary value there are two coverage items: this
1156 boundary value and its closest neighbor belonging to the adjacent partition. To achieve 100% coverage
1157 with 2-value BVA, test cases must exercise all coverage items, i.e., all identified boundary values.
1158 Coverage is measured as the number of boundary values exercised, divided by the total number of
1159 identified boundary values, normally represented as a percentage.
1160 In 3-value BVA (Koomen 2006, O’Regan 2019), for each boundary value there are three coverage items:
1161 this boundary value and both its neighbors. Therefore, in 3-value BVA some of the coverage items may
1162 not be boundary values. To achieve 100% coverage with 3-value BVA, test cases must exercise all
1163 coverage items, i.e., identified boundary values and their neighbors. Coverage is measured as the
1164 number of boundary values and their neighbors exercised, divided by the total number of identified
1165 boundary values and their neighbors, normally represented as a percentage.
1166 3-value BVA is more rigorous than 2-value BVA as it may detect defects overlooked by 2-value BVA. For
1167 example, if the decision “if (x ≤ 10) …” is incorrectly implemented as “if (x = 10) …”, no test data derived
1168 from the 2-value BVA (x = 10, x = 11) can detect the defect. However, x = 9, derived from the 3-value
1169 BVA, is likely to detect it.

v4.0 Page 38 of 70 beta release

1170 4.2.3. Decision Table Testing

1171 Decision tables are used for testing the implementation of system requirements that specify how different
1172 combinations of conditions result in different outcomes. Decision tables are an effective way of recording
1173 complex logic, such as business rules.
1174 When creating decision tables, the conditions and the resulting actions of the system are defined. These
1175 form the rows of the table. Each column corresponds to a decision rule that defines a unique combination
1176 of conditions, along with the associated actions. In limited-entry decision tables all the values of the
1177 conditions and actions (except for irrelevant or infeasible ones; see below) are shown as Boolean values
1178 (true or false). Alternatively, in extended-entry decision tables some or all the conditions and actions may
1179 also take on multiple values (e.g., ranges of numbers, equivalence classes, discrete values).
1180 The notation for conditions is as follows. “T” (true) means that the condition is satisfied. “F” (false) means
1181 that the condition is not satisfied. “–” means that the value of the condition is irrelevant for the action
1182 outcome. “N/A” means that the condition is infeasible for a given rule. For actions, “T” means that the
1183 action should occur. “N” means that the action should not occur. Other notations may also be used.
1184 A full decision table has enough columns to cover every combination of conditions. The table can be
1185 simplified by deleting columns containing infeasible combinations of conditions. The table can also be
1186 minimized by merging columns, in which some conditions do not affect the outcome, into a single column.
1187 Decision table minimization algorithm is out of scope of this syllabus.
1188 In decision table testing, the coverage items are the columns containing feasible combinations of
1189 conditions. To achieve 100% coverage with this technique, test cases must exercise all these columns.
1190 Coverage is measured as the number of columns exercised, divided by the total number of feasible
1191 columns, normally represented as a percentage.
1192 The strength of decision table testing is that it provides a systematic approach to identifying all the
1193 combinations of conditions, some of which might otherwise be overlooked. It also helps in finding any
1194 gaps or contradictions in the requirements. In the case of many conditions, exercising all the rules may be
1195 time consuming, since the number of rules grows exponentially with the number of conditions. In such a
1196 case, to reduce the number of rules exercised, a minimized decision table or a risk-based approach may
1197 be used.

1198 4.2.4. State Transition Testing

1199 State transition diagram models the behavior of a system by showing its possible states and valid state
1200 transitions. A transition is initiated by an event, which may be additionally qualified by a guard condition.
1201 The transitions are assumed to be instantaneous and may sometimes result in the software taking action.
1202 The common transition labeling syntax is as follows: “event [guard condition] / action”. Guard conditions
1203 and actions can be omitted if they do not exist or are irrelevant for the tester.
1204 A state table is a model equivalent to a state transition diagram. Its rows represent states, and its
1205 columns represent events. Table entries (cells) represent transitions, and contain the target state, as well
1206 as the guard conditions, and resulting actions, if defined. In contrast to the state transition diagram, the
1207 state table explicitly shows invalid transitions, which are represented by empty cells.
1208 A test case based on a state transition diagram or state table is usually represented as a sequence of
1209 events, which results in a sequence of state changes (and actions, if needed). One test case may, and
1210 usually will, cover several transitions between states.
1211 There exist many coverage criteria for state transition testing. This syllabus discusses three of them.

v4.0 Page 39 of 70 beta release

1212 In all states coverage, the coverage items are the states. To achieve 100% all states coverage, test
1213 cases must ensure that all the states are visited. Coverage is measured as the number of visited states
1214 divided by the total number of states, normally represented as a percentage.
1215 In valid transitions coverage (also called 0-switch coverage), the coverage items are single valid
1216 transitions. To achieve 100% valid transitions coverage, test cases must exercise all the valid transitions.
1217 Coverage is measured as the number of exercised valid transitions divided by the total number of valid
1218 transitions, normally represented as a percentage.
1219 In all transitions coverage, the coverage items are all the transitions shown in a state table. To achieve
1220 100% all transitions coverage, test cases must exercise all the valid transitions and attempt to execute
1221 invalid transitions. Testing only one invalid transition in a single test case helps to avoid fault masking,
1222 i.e., a situation in which one defect prevents the detection of another. Coverage is measured as the
1223 number of valid and invalid transitions exercised or attempted to be covered by executed test cases,
1224 divided by the total number of valid and invalid transitions, normally represented as a percentage.
1225 All states coverage is weaker than valid transitions coverage, because it can typically be achieved without
1226 exercising all the transitions. Valid transitions coverage is the most widely used coverage criterion.
1227 Achieving full all transitions coverage guarantees both full all states coverage and full valid transitions
1228 coverage and should be a minimum requirement for mission- and safety-critical software.

1229 4.3. White-Box Testing

1230 Because of their popularity and simplicity, this section focuses on two code-related white-box test
1231 techniques:
1232 • Statement testing
1233 • Branch testing
1234 There are more rigorous techniques that are used in some safety-critical, mission-critical, or high-integrity
1235 environments to achieve more thorough code coverage. There are also white-box test techniques used
1236 on higher test levels (e.g., API testing). These techniques are not discussed in this syllabus.

1237 4.3.1. Statement Testing and Statement Coverage

1238 In statement testing the coverage items are executable statements. The aim is to design test cases to
1239 exercise statements in the code until an acceptable level of coverage is achieved. Coverage is measured
1240 as the number of statements exercised by the test cases divided by the total number of executable
1241 statements in the code, normally expressed as a percentage.
1242 When 100% statement coverage is achieved, it ensures that all executable statements in the code have
1243 been exercised at least once. This means that in particular each statement with a defect will be executed,
1244 which may cause a failure, demonstrating the presence of the defect. However, exercising a statement
1245 with a test case will not detect defects in all cases. For example, it may not detect defects that are data
1246 dependent (e.g., a division by zero that only fails when a denominator is set to zero). Also, 100%
1247 statement coverage does not ensure that all the decision logic has been tested as, for instance, it may not
1248 exercise all the branches (see chapter 4.3.2) in the code.

v4.0 Page 40 of 70 beta release

1249 4.3.2. Branch Testing and Branch Coverage

1250 A branch is a transfer of control between two nodes in the control flow graph, which shows the possible
1251 sequences in which source code statements are executed in the test object. Each transfer of control can
1252 be either unconditional (i.e., straight-line code) or conditional (i.e., a decision outcome).
1253 In branch testing the coverage items are branches and the aim is to design test cases to exercise
1254 branches in the code until an acceptable level of coverage is achieved. Coverage is measured as the
1255 number of branches exercised by the test cases divided by the total number of branches, normally
1256 expressed as a percentage.
1257 When 100% branch coverage is achieved, all branches in the code, unconditional and conditional, are
1258 exercised by test cases. Conditional branches typically correspond to a true or false outcome from an
1259 “if...then” decision, an outcome from a switch/case statement, or a decision to exit or continue in a loop.
1260 Branch coverage subsumes statement coverage. This means that any set of test cases achieving 100%
1261 branch coverage also achieves 100% statement coverage (but not vice versa).

1262 4.3.3. The Value of White-box Testing

1263 A fundamental strength that all white-box techniques share is that the entire software implementation is
1264 taken into account during testing, which facilitates defect detection even when the software specification
1265 is vague or incomplete. A corresponding weakness is that if the software does not implement one or more
1266 requirements, white box testing may not detect the resultant defects of omission (Watson 1996).
1267 White-box techniques can be used in static testing (e.g., during dry runs of a code). They are well suited
1268 to reviewing code that is not yet ready for execution (Hetzel 1988), as well as the pseudocode and other
1269 high-level or top-down logic which can be modeled with a control flow graph.
1270 If solely performing black-box testing, then no measure of actual code coverage is obtained. White-box
1271 coverage measures provide an objective measure of coverage and provide the necessary information to
1272 allow additional tests to be generated to increase this coverage, and subsequently increase confidence in
1273 the code.

1274 4.4. Experience-based Testing

1275 Commonly used experience-based test techniques discussed in the following sections are:
1276 • Error guessing
1277 • Exploratory testing
1278 • Checklist-based testing

1279 4.4.1. Error Guessing

1280 Error guessing is a technique used to anticipate the occurrence of errors, defects, and failures, based on
1281 the tester’s knowledge, including:
1282 • How the application has worked in the past
1283 • The types of errors the developers tend to make and the types of defects these errors result in

v4.0 Page 41 of 70 beta release

1284 • The types of failures that have occurred in other, similar applications
1285 In general, errors, defects and failures may be related to: input (e.g., correct input not accepted,
1286 parameters wrong or missing), output (e.g., wrong format, wrong result), logic (e.g., missing cases, wrong
1287 operator), computation (e.g., incorrect operand, wrong computation), interface (e.g., parameter mismatch,
1288 incompatible types), or data (e.g., incorrect initialization, wrong type).
1289 Fault attacks are a methodical approach to the implementation of error guessing. This technique requires
1290 the tester to create or acquire a list of possible errors, defects and failures, and to design tests that will
1291 identify defects associated with the errors, expose the defects, or cause the failures. These lists can be
1292 built based on experience, defect and failure data, or from common knowledge about why software fails.
1293 See (Whittaker 2002, Whittaker 2003, Andrews 2006) for more information on error guessing and fault
1294 attacks.

1295 4.4.2. Exploratory Testing

1296 In exploratory testing, tests are simultaneously designed, executed, logged, and evaluated while the
1297 tester learns about the test object. The testing is used to learn more about the test object, to explore it
1298 more deeply with focused tests, and to create tests for untested areas.
1299 Exploratory testing is sometimes conducted using a session-based approach to structure the activity. In a
1300 session-based approach, exploratory testing is conducted within a defined time-box. The tester uses a
1301 test charter containing test objectives to guide the testing. The session is usually followed by a debrief
1302 that involves a discussion between the tester and stakeholders interested in the results of the session. In
1303 this approach test objectives may be treated as high-level test conditions. Coverage items are identified
1304 and exercised during the session. The tester may use test session sheets to document the steps followed
1305 and the discoveries made.
1306 Exploratory testing is useful when there are few or inadequate specifications or there is significant time
1307 pressure on the testing. Exploratory testing is also useful to complement other more formal testing
1308 techniques. This technique will be more effective if the tester is experienced, has domain knowledge and
1309 has a high degree of essential skills, like analytical skills, curiosity and creativeness (see section 1.5.1).
1310 Exploratory testing can incorporate the use of other test techniques. More information about exploratory
1311 testing can be found in (Kaner 1999, Whittaker 2009, Hendrickson 2013).

1312 4.4.3. Checklist-Based Testing

1313 In checklist-based testing, a tester designs, implements, and executes tests to cover test conditions from
1314 a checklist. Checklists can be built based on experience, knowledge about what is important for the user,
1315 or an understanding of why and how software fails. Checklist should not contain items that can be
1316 checked automatically, items better suited as entry/exit criteria, or items that are too general (Brykczynski
1317 1999).
1318 Checklist items are often phrased in the form of a question. It should be possible to check each item
1319 separately and directly. These items may refer to requirements, graphical interface properties, quality
1320 characteristics or other forms of test conditions. Checklists can be created to support various test types,
1321 including functional and non-functional testing (e.g., 10 heuristics for usability testing (Nielsen 1994)).
1322 Some checklist entries may gradually become less effective over time because the developers will learn
1323 to avoid making the same mistakes. New entries may also need to be added to reflect high severity

v4.0 Page 42 of 70 beta release

1324 defects found recently. Therefore, checklists should be regularly updated based on defect analysis.
1325 However, care should be taken to avoid letting the checklist become too long (Gawande 2009).
1326 In the absence of detailed test cases, checklist-based testing can provide guidelines and some degree of
1327 consistency for the testing. If the checklists are high-level, some variability in the actual testing is likely to
1328 occur, resulting in potentially greater coverage but less repeatability.

1329 4.5. Collaboration-based Test Approaches

1330 Each of the above-mentioned techniques (see sections 4.2, 4.3, 4.4) has a particular objective with
1331 respect to defect detection. Collaboration-based approaches, on the other hand, focus on defect
1332 avoidance by collaboration and communication.

1333 4.5.1. Collaborative User Story Writing

1334 A user story represents an increment that will be valuable to either a user or purchaser of a system or
1335 software. User stories are composed of three aspects (Jeffries 2000), called together the “3 C’s”:
1336 • Card – the medium describing a user story (e.g., an index card, an entry in an electronic board)
1337 • Conversation – explains how the software will be used (can be documented or verbal)
1338 • Confirmation – the acceptance criteria (see section 4.5.2)
1339 The most common format for a user story is “As a [role], I want [goal to be accomplished], so that I can
1340 [resulting business value for the role]”, followed by the acceptance criteria.
1341 The collaborative authorship of the user story can use techniques such as brainstorming and mind
1342 mapping. Good user stories should be: Independent, Negotiable, Valuable, Estimable, Small and
1343 Testable (INVEST). If a stakeholder does not know how to test a user story, this may indicate that the
1344 user story is not clear enough, or that it does not reflect something valuable to them, or that the
1345 stakeholder just needs help in testing (Wake 2003).

1346 4.5.2. Acceptance Criteria

1347 Acceptance criteria are the conditions that an implementation of a user story must meet to be accepted
1348 by stakeholders. From this perspective, acceptance criteria may be viewed as the test conditions that
1349 should be exercised by the tests. Acceptance criteria are usually a result of the conversation (see section
1350 4.5.1).
1351 Acceptance criteria are used to:
1352 • Define boundaries of a user story
1353 • Reach consensus between the stakeholders
1354 • Describe both positive and negative scenarios
1355 • Serve as a basis for the user story acceptance testing (see section 4.5.3)
1356 • Allow accurate planning and estimation
1357 There is no single way to write acceptance criteria for a user story. The two most common formats are:

v4.0 Page 43 of 70 beta release

1358 • Scenario-oriented (e.g., Given/When/Then format used in the BDD, see section 2.1.3)
1359 • Rule-oriented (e.g., bullet point verification list, or tabulate form of input-output mapping)
1360 Most acceptance criteria can be documented in one of these two formats. However, the team may use
1361 another, custom format, as long as the acceptance criteria are well-defined and unambiguous.

1362 4.5.3. Acceptance Test-driven Development (ATDD)

1363 ATDD is a test-first approach (see section 2.1.3). Test cases are created prior to implementing the user
1364 story. The test cases are created by the team members with different perspectives, e.g., customers,
1365 developers, and testers (Adzic 2009). Test cases may be manual or automated.
1366 The first step is a specification workshop where the user story and (if yet defined) its acceptance criteria
1367 are analyzed, discussed, and written by the team members. Incompleteness, ambiguities, or defects in
1368 the user story are fixed during this process. The next step is to create the tests. This can be done by the
1369 team together or by the tester individually. In any case, an independent person such as a business
1370 representative validates the tests. The tests are examples, based on the acceptance criteria, that
1371 describe the specific characteristics of the user story. These examples will help the team implement the
1372 user story correctly.
1373 Since examples and tests are the same, these terms are often used interchangeably. During the test
1374 design the test techniques described in sections 4.2, 4.3 and 4.4 may be applied.
1375 Typically, the first tests are the positive tests, confirming the correct behavior without exceptions or error
1376 conditions, comprising the sequence of activities executed if everything goes as expected. After the
1377 positive tests are done, the team should perform negative testing, and cover non-functional attributes as
1378 well (e.g., performance, usability). Tests should be expressed in a way that is understandable for the
1379 stakeholders. Typically, tests contain sentences in natural language involving the necessary
1380 preconditions (if any), the inputs, and the related outputs.
1381 The examples must cover all the characteristics of the user story and should not go beyond the story.
1382 However, the acceptance criteria may detail some of the issues described in the user story. In addition,
1383 no two examples should describe the same characteristics of the user story.
1384 When captured in a format supported by a functional test automation framework, the developers can
1385 automate the tests by writing the supporting code as they implement the feature described by a user
1386 story. The acceptance tests then become the executable requirements.

v4.0 Page 44 of 70 beta release

1387 5. Managing the Test Activities – 335 minutes

1388 Keywords
1389 defect management, defect report, entry criteria, exit criteria, product risk, project risk, risk, risk analysis,
1390 risk assessment, risk control, risk identification, risk level, risk management, risk mitigation, risk
1391 monitoring, risk-based testing, test approach, test completion report, test control, test estimation, test
1392 monitoring, test plan, test planning, test progress report, test pyramid
1393

1394 Learning Objectives for Chapter 5:

1395 5.1 Test Planning

1396 FL-5.1.1 (K2) Exemplify the purpose and content of a test plan
1397 FL-5.1.2 (K1) Recognize how a tester adds value to iteration and release planning
1398 FL-5.1.3 (K2) Compare and contrast entry and exit criteria, Definition of Ready, and Definition of Done
1399 FL-5.1.4 (K3) Use estimation techniques to calculate the required testing effort
1400 FL-5.1.5 (K3) Apply test case prioritization
1401 FL-5.1.6 (K1) Recall the concepts of the test pyramid
1402 FL-5.1.7 (K2) Summarize the testing quadrants and their relationships with test levels and test types

1403 5.2 Risk Management

1404 FL-5.2.1 (K1) Identify risk level by using likelihood and impact
1405 FL-5.2.2 (K2) Distinguish between project and product risks
1406 FL-5.2.3 (K2) Explain how product risk analysis may influence thoroughness and scope of testing
1407 FL-5.2.4 (K2) Explain what measures can be taken in response to analyzed product risks

1408 5.3 Test Monitoring, Test Control and Test Completion

1409 FL-5.3.1 (K1) Recall metrics used for testing
1410 FL-5.3.2 (K2) Summarize the purposes, contents, and audiences for test reports
1411 FL-5.3.3 (K2) Exemplify how to communicate the status of testing

1412 5.4 Configuration Management

1413 FL-5.4.1 (K2) Summarize how configuration management supports testing

1414 5.5 Defect Management

1415 FL-5.5.1 (K3) Prepare a defect report

v4.0 Page 45 of 70 beta release

1416 5.1. Test Planning

1417 5.1.1. Purpose and Content of a Test Plan

1418 A test plan describes the objectives, resources and processes for a test project. A test plan:
1419 • Documents the means and schedule for achieving test objectives
1420 • Helps to ensure that the performed test activities will meet the established criteria
1421 • Serves as a means of communication with team members and other stakeholders
1422 • Demonstrates that testing will adhere to the existing test policy and test strategy
1423 Test planning guides the testers’ thinking and forces the testers to confront the future challenges related
1424 to risks, schedules, people, tools, costs, effort, etc. The process of preparing a test plan is a useful way to
1425 think through the efforts needed to achieve the test project objectives.
1426 The typical content of a test plan includes the information about:
1427 • Context of testing (scope, objectives, constraints, test basis)
1428 • Assumptions and constraints of the test project
1429 • Stakeholders (roles, responsibilities, relevance to testing, hiring and training needs)
1430 • Communication (forms and frequency of communication, documentation templates)
1431 • Risk register (product risks, project risks)
1432 • Test approach (e.g., test levels, test types, test techniques, test deliverables, entry and exit
1433 criteria, degree of independence, metrics to be collected, test data requirements, test
1434 environment requirements, deviations from the organizational test practices)
1435 • Schedule
1436 More details about the test plan and its content can be found in the ISO/IEC/IEEE 29119-3 standard.

1437 5.1.2. Tester's Contribution to Iteration and Release Planning

1438 In iterative SDLCs, typically two kinds of planning occur: release planning and iteration planning.
1439 Release planning looks ahead to the release of a product. Release planning defines and re-defines the
1440 product backlog, and may involve refining larger user stories into a collection of smaller stories. Release
1441 planning provides the basis for a test approach and test plan spanning all iterations. Testers involved in
1442 release planning define testable user stories and acceptance criteria (see section 4.5), participate in
1443 project and quality risk analyses (see section 5.2), estimate testing effort associated with user stories (see
1444 section 5.1.4), select the necessary test levels, and plan the testing for the release.
1445 Iteration planning looks ahead to the end of a single iteration and is concerned with the iteration backlog.
1446 Testers involved in iteration planning participate in the detailed risk analysis of user stories, determine the
1447 testability of user stories, break down user stories into tasks (particularly testing tasks), estimate testing
1448 effort for all testing tasks, and identify functional and non-functional aspects of the system to be tested.

v4.0 Page 46 of 70 beta release

1449 5.1.3. Entry and Exit Criteria

1450 Entry criteria define the preconditions for undertaking a given activity. If entry criteria are not met, it is
1451 likely that the activity will prove more difficult, be more time-consuming, more costly, and riskier. Exit
1452 criteria define what must be achieved in order to declare an activity completed. Entry and exit criteria
1453 should be defined for each test level and test type, and will differ based on the test objectives.
1454 Typical entry criteria include: availability of resources (e.g., people, tools, environments, test data, budget,
1455 time), availability of testware (e.g., test basis, testable requirements, user stories, test cases), and initial
1456 quality level of a test object (e.g., all smoke tests have passed).
1457 Typical exit criteria include: measures of diligence (e.g., achieved level of coverage, number of
1458 unresolved defects, estimated defect density, number of failed test cases), and completion criteria (e.g.,
1459 planned tests have been executed, static testing has been performed, all defects found are reported, all
1460 regression tests are automated).
1461 Running out of time or budget, or pressure to bring the product to market can be also viewed as valid exit
1462 criteria. Even without other exit criteria being satisfied, it can be acceptable to end testing under such
1463 circumstances, if the stakeholders have reviewed and accepted the risk to go live without further testing.
1464 In Agile development, exit criteria applied to an increment are called Definition of Done. Entry criteria that
1465 a user story must meet to be moved from the backlog to development are called Definition of Ready.

1466 5.1.4. Estimation Techniques

1467 Testing effort estimation involves predicting the amount of test-related work needed in order to meet the
1468 objectives of a test project. It is important to make it clear to the stakeholders that the estimate is based
1469 on a number of assumptions and is always subject to estimation error. Estimation for small tasks is
1470 usually more accurate than for the large ones. Therefore, when estimating a large task, a decomposition
1471 technique called Work Breakdown Structure (WBS) can be used.
1472 In this syllabus the following four estimation techniques are described.
1473 Estimation based on ratios. In this metrics-based technique, the greatest possible amount of
1474 experience figures is collected, which makes it possible to derive “standard” ratios for similar projects. The
1475 own ratios of an organization (e.g., taken from historical data) are generally the best source to use in the
1476 estimation process. These standard ratios can then be used to estimate the testing effort for the new
1477 project. For example, if in the previous project development-to-testing effort ratio was 3:2, and in the
1478 current project the development effort is expected to be 600 person-days, the testing effort can be
1479 estimated to be 400 person-days.
1480 Extrapolation. In this metrics-based technique, measurements are made as early in the project as
1481 possible to gather real, historical data. Having enough observations, the effort required for the remaining
1482 work can be approximated by extrapolating these data. This method is very suitable in iterative SDLCs.
1483 For example, the team may extrapolate the test effort in the forthcoming iteration as the averaged effort
1484 from the last three iterations.
1485 Wideband Delphi. In this iterative, expert-based technique, experts make experience-based estimations.
1486 Each expert, in isolation, estimates the effort. The results are collected and experts discuss their current
1487 estimates. Each expert is then asked to make a new prediction based on that feedback. This process is
1488 repeated until a consensus is reached. Planning Poker is a variant of Wideband Delphi, commonly used
1489 in Agile software development. In Planning Poker, estimates are done using the cards with numbers that
1490 represent the effort size.

v4.0 Page 47 of 70 beta release

1491 Three-point estimation. In this expert-based technique, three estimations are made by the experts: most
1492 optimistic estimation (a), most likely estimation (m) and most pessimistic estimation (b). The final estimate
1493 (E) is their weighted arithmetic mean calculated as E = (a + 4*m + b) / 6. The advantage of this technique
1494 is that it allows the experts to calculate the measurement error: SD = (b – a) / 6. For example, if the
1495 estimates (in person-hours) are: a=6, m=9 and b=18, then the final estimation is 10±2 person-hours (i.e.,
1496 between 8 and 12 person-hours), because E = (6 + 4*9 + 18) / 6 = 10 and SD = (18 – 6) / 6 = 2.
1497 See (Kan 2003, Koomen 2006, Westfall 2009) for these and many other test estimation techniques.

1498 5.1.5. Test Case Prioritization

1499 Once the test cases and test procedures are produced and assembled into test suites, these test suites
1500 can be arranged in a test execution schedule that defines the order in which they are to be run. When
1501 prioritizing test cases, different factors can be taken into account. The most commonly used test case
1502 prioritization strategies are as follows.
1503 • Risk-based prioritization, where test execution order is based on the results of the risk analysis
1504 (see section 5.2.3). Test cases covering the most important risks are executed first.
1505 • Coverage-based prioritization, where test execution order is based on coverage (e.g., statement
1506 coverage). Test cases achieving the highest coverage are executed first. In another variant,
1507 called additional coverage prioritization, the test case achieving the highest coverage is executed
1508 first. Each subsequent test case is the one that achieves the highest additional coverage.
1509 • Requirements-based prioritization, where test execution order is based on the priorities of the
1510 requirements traced back to the corresponding test cases. Requirement priorities are defined by
1511 stakeholders. Test cases related to the most important requirements are executed first.
1512 Ideally, test cases would be ordered to run based on their priority levels, using, for example, one of the
1513 above-mentioned prioritization strategies. However, this practice may not work if the test cases or the
1514 features being tested have dependencies. If a test case with a higher priority is dependent on a test case
1515 with a lower priority, the lower priority test case must be executed first.
1516 Test execution order has also to take into account the availability of resources. For example, the required
1517 tools, environments or people may be available only for a specific time window.

1518 5.1.6. Test Pyramid

1519 The test pyramid is a metaphor showing that different tests may have different granularity. The test
1520 pyramid model supports the team in test automation and in test effort allocation. The pyramid layers
1521 represent groups of tests. The higher the layer, the lower the test granularity, test isolation and test
1522 execution speed. Tests in the bottom layer are small, isolated, fast, and check a small piece of
1523 functionality, so usually a lot of them are needed to achieve a reasonable coverage. The top layer
1524 represents large, high-level, end-to-end tests. These high-level tests are slower than the tests from the
1525 lower layers, and they typically check a large piece of functionality, so usually just a few of them are
1526 needed to achieve a reasonable coverage. The number and naming of the layers may differ. For
1527 example, the original test pyramid model (Cohn 2009) defines three layers: “unit tests”, “service tests” and
1528 “UI tests”. Another popular model defines unit (component), integration, and end-to-end tests.

v4.0 Page 48 of 70 beta release

1529 5.1.7. Testing Quadrants

1530 The testing quadrants, defined by Brian Marick (Marick 2003, Crispin 2008), group the test levels with the
1531 appropriate test types, activities, techniques and work products in the Agile methodology. The model
1532 supports test management in ensuring that all important test types and test levels are included in the
1533 development lifecycle and in understanding that some test types are more related to certain test levels
1534 than the others. This model also provides a way to differentiate and describe the types of tests to all
1535 stakeholders, including developers, testers, and business representatives.
1536 In this model, tests can be business facing or technology facing. Tests can also support the team or
1537 critique the product. The combination of these two characteristics determines the four quadrants:
1538 • Quadrant Q1 (technology facing, support the team). This quadrant contains component and
1539 component integration tests. These tests should be automated and included in the CI process.
1540 • Quadrant Q2 (business facing, support the team). This quadrant contains functional tests, examples,
1541 user story tests, user experience prototypes, API testing, and simulations. These tests check the
1542 acceptance criteria and can be manual or automated.
1543 • Quadrant Q3 (business facing, critique the product). This quadrant contains exploratory testing,
1544 usability tests, user acceptance testing. These tests are often manual and are user-oriented.
1545 • Quadrant Q4 (technology facing, critique the product). This quadrant contains smoke tests and non-
1546 functional tests (except usability tests). These tests are often automated.

1547 5.2. Risk Management

1548 Organizations face many internal and external factors that make it uncertain whether and when they will
1549 achieve their objectives (ISO 31000). Risk management allows the organizations to increase the
1550 likelihood of achieving objectives, improve the quality of their products and increase the stakeholders’
1551 confidence and trust.
1552 The main risk management activities are:
1553 • Risk analysis (consisting of risk identification and risk assessment; see section 5.2.3)
1554 • Risk control (consisting of risk mitigation and risk monitoring; see section 5.2.4)
1555 Test approach, in which test activities are managed, selected, and prioritized based on risk analysis and
1556 risk control, is called the risk-based testing.

1557 5.2.1. Risk Definition and Risk Attributes

1558 In this syllabus a risk is defined as the factor or event, whose potential occurrence causes an adverse
1559 effect. Risk can be characterized by two factors, which express the risk level. These factors are:
1560 • Likelihood – the probability of the factor or event occurrence
1561 • Impact (harm) – the consequences of this occurrence

1562 5.2.2. Project and Product Risks

1563 In software testing one is generally concerned by two types of risks: project risks and product risks.

v4.0 Page 49 of 70 beta release

1564 Project risks are related to the management and control of the project. Project risk factors include:
1565 • Organizational issues (e.g., delays in work products delivery, inaccurate estimates, cost-cutting)
1566 • People issues (e.g., insufficient skills, conflicts, communication problems, shortage of staff)
1567 • Technical issues (e.g., scope creep, poor tool support)
1568 • Supplier issues (e.g., third-party delivery failure, bankruptcy of the supporting company)
1569 Project risks, when they occur, may have an impact on the project schedule, budget or scope, which
1570 affects the project's ability to achieve its objectives.
1571 Product risks are related to the product quality characteristics (e.g., described in the ISO 25010 quality
1572 model). Examples of product quality risks include: missing or wrong functionality, incorrect calculations,
1573 runtime errors, poor architecture, inefficient algorithms, inadequate response time, poor user experience,
1574 security vulnerabilities. Product risks, when they occur, may result in various negative consequences,
1575 including:
1576 • User dissatisfaction
1577 • Loss of revenue
1578 • Damage to third parties
1579 • High maintenance costs
1580 • Overload of the helpdesk
1581 • Damage to the image
1582 • Loss of trust
1583 • Criminal penalties
1584 • In extreme cases, physical damage, injuries or even death

1585 5.2.3. Product Risk Analysis

1586 The goal of risk analysis is to provide an awareness of risk in order to focus the testing effort in a way that
1587 minimizes the residual level of product risk. Ideally, risk analysis begins early in the SDLC.
1588 Risk analysis consists of risk identification and risk assessment. Risk identification is about generating a
1589 comprehensive list of risks. Stakeholders can identify risks by using various techniques and tools, e.g.,
1590 brainstorming, workshops, interviews, or cause-effect diagrams. Risk assessment involves: categorization
1591 of identified risks, determining their likelihood, impact and level, prioritizing, and proposing ways to handle
1592 them. Categorization helps in assigning mitigation actions, because usually the risks falling into the same
1593 category can be mitigated using a similar approach.
1594 Risk assessment can use a quantitative or qualitative approach, or a mix of them. In the quantitative
1595 approach the risk level is calculated as the multiplication of likelihood and impact. In the qualitative
1596 approach the risk level can be calculated using a risk matrix.
1597 Product risk analysis may influence the thoroughness and scope of testing. Its results are used to:

v4.0 Page 50 of 70 beta release

1598 • Determine the scope of testing to be carried out

1599 • Determine the particular test levels and propose test types to be performed
1600 • Determine the test techniques to be employed and the coverage to be achieved
1601 • Estimate the test effort required for each task
1602 • Prioritize testing in an attempt to find the critical defects as early as possible
1603 • Determine whether any activities in addition to testing could be employed to reduce risk

1604 5.2.4. Product Risk Control

1605 Risk control comprises all measures that are taken in response to identified and assessed product risks.
1606 Risk control consists of risk mitigation and risk control. Risk mitigation involves implementing the actions
1607 proposed in risk assessment to reduce the risk level. The aim of risk monitoring is to ensure that the
1608 mitigation actions are effective, to obtain further information to improve risk assessment, and to identify
1609 emerging risks.
1610 With respect to risk control, once a risk has been analyzed, several response options to risk are possible,
1611 e.g., risk acceptance, risk transfer, contingency plan, or risk mitigation by testing (Veenendaal 2012).
1612 Actions that can be taken to mitigate the product risks by testing are as follows:
1613 • Select the testers with the right level of experience, suitable for a given risk type
1614 • Apply an appropriate level of independence of testing
1615 • Conduct reviews and perform static analysis
1616 • Apply the appropriate test design technique and coverage level
1617 • Perform dynamic testing, including regression testing

1618 5.3. Test Monitoring, Test Control and Test Completion

1619 Test monitoring and control are test management activities concerned with ensuring that the planned
1620 testing goes as smoothly as possible. Test monitoring gathers information about the testing being done,
1621 while test control is used to manage the testing to ensure it keeps to the test plan in an efficient manner.
1622 Test monitoring is concerned with gathering information, and visibility of test activities, together with
1623 feedback on them. This information is used to assess test progress and to measure whether the test exit
1624 criteria or the testing tasks associated with the Definition of Done are satisfied, such as meeting the
1625 targets for coverage of product risks, requirements, or acceptance criteria.
1626 Test control uses the information from test monitoring to provide guidance and the necessary corrective
1627 actions to achieve the most effective and efficient testing. Test control activities include:
1628 • Re-prioritize tests when an identified risk becomes an issue
1629 • Re-evaluate whether a test item meets an entry or exit criterion due to rework
1630 • Adjust the test schedule to address a delay in the delivery of the test environment
1631 • Adding new resources when and where needed

v4.0 Page 51 of 70 beta release

1632 Test completion activities collect data from completed test activities to consolidate experience, testware,
1633 and any other relevant information. Test completion activities occur at project milestones such as when a
1634 software system is released, a test project is completed (or cancelled), an Agile project iteration is
1635 finished, a test level is completed, or a maintenance release has been completed.

1636 5.3.1. Metrics used in Testing

1637 Test metrics are gathered to show progress against the planned schedule and budget, the current quality
1638 of the test object, and the effectiveness of the test activities with respect to the objectives or a sprint goal.
1639 Test monitoring gathers a variety of metrics to inform the test control activity.
1640 Common test metrics include:
1641 • Project progress metrics (e.g., task completion, resource usage, test effort)
1642 • Test progress metrics (e.g., test case implementation progress, test environment preparation
1643 progress, number of test cases run/not run, passed/failed, test execution time)
1644 • Defect metrics (e.g., number of defects found/fixed, defect density, defect detection percentage)
1645 • Risk metrics (e.g., residual risk level)
1646 • Coverage metrics (e.g., requirements coverage, code coverage)
1647 • Cost metrics (e.g., cost of testing, organizational cost of quality)

1648 5.3.2. Purpose, Contents and Audience for Test Reports

1649 Test reporting summarizes and communicates test information during and after testing. Test progress
1650 reports support the ongoing control of the testing and must provide enough information to make
1651 modifications to the testing schedule, resources, or test plan. Test completion reports summarize a
1652 specific stage of testing (e.g., test level, test cycle, iteration) and can give information for subsequent
1653 testing.
1654 During test monitoring and control, the test team generates test progress reports for stakeholders to keep
1655 them informed. Test progress reports are usually generated on a regular basis (e.g., daily, weekly, etc.)
1656 and include:
1657 • Testing period
1658 • Test status (e.g., ahead or behind schedule, perhaps using a traffic lights system)
1659 • Test progress, including any notable deviations
1660 • Factors currently impeding testing, and their workarounds
1661 • Test metrics (see section 5.3.1 for examples)
1662 • New and changed risks within testing period
1663 • Testing planned for the next period

v4.0 Page 52 of 70 beta release

1664 A test completion report is prepared during test completion, when a project, test level, or test type
1665 concludes and ideally when its exit criteria are met. This report uses test progress reports and other data.
1666 Typical test completion reports include:
1667 • Test summary
1668 • Testing and product quality evaluation based on original test plan (i.e., test objectives and exit
1669 criteria or Definition of Done)
1670 • Deviations from the test plan (e.g., differences from the planned schedule, duration, and effort).
1671 • Testing impediments and workarounds
1672 • Test metrics, based on test progress reports
1673 • Unmitigated risks
1674 • Lessons learned that are relevant to the testing
1675 Different audiences require different information in the reports, and influences the formality and frequency
1676 of reporting. Reporting on test progress to others in the same team is often frequent and informal, while
1677 reporting on testing for a complete project follows a set template and occurs only once.
1678 The ISO/IEC/IEEE 29119-3 standard includes templates and examples for the two types of test reports:
1679 test status reports and test completion reports.

1680 5.3.3. Communicating the Status of Testing

1681 The best means of communicating test status varies, depending on test management concerns,
1682 organizational test strategies, regulatory standards, or, in the case of the self-organizing teams (see
1683 section 1.5.2), by the team itself. The options include:
1684 • Verbal communication with team members and other stakeholders
1685 • Dashboards (e.g., CI/CD dashboards, task boards, and burn-down charts)
1686 • Dashboard-style emails
1687 • Online documents
1688 • Formal test reports (see section 5.3.2)
1689 One or more of these options can be used. More formal communication may be more appropriate for
1690 distributed teams where direct face-to-face communication is not always possible due to geographical or
1691 time differences.

1692 5.4. Configuration Management

1693 Configuration management (CM) provides a mechanism for identifying, controlling and tracking the work
1694 products, including test work products, and making them available as needed. These work products are
1695 known as configuration items. In the testing context, examples of these are test plans and strategies, test
1696 conditions, test cases, test results, test runs and test reports. Configuration items can consist of many
1697 other configuration items, and as they are changed their version changes. For a complex configuration

v4.0 Page 53 of 70 beta release

1698 item (e.g., a test environment), CM records the configuration items that make up this complex item, their
1699 relationships and their versions. If this complex item is approved for testing, then it becomes a baseline
1700 and can only be changed through a formal change control. Changes to configuration items are tracked
1701 and when a new baseline is created with changed configuration items, CM has a full record of the
1702 changed items and the changes that led to the current version. It should also be possible to revert to the
1703 previous baseline, e.g., when the previous test results need to be reproduced.
1704 To properly support testing, CM ensures the following:
1705 • All configuration items, including test items (individual parts of the test object), are uniquely
1706 identified, version controlled, tracked for changes, and related to other configuration items so that
1707 traceability can be maintained throughout the test process
1708 • All identified documents and software items are referenced unambiguously in test documentation
1709 Continuous integration, continuous delivery, continuous deployment and the associated testing are
1710 typically implemented as part of an automated DevOps pipeline (see section 2.1.4), in which automated
1711 CM is normally included.

1712 5.5. Defect Management

1713 Since one of the major objectives of testing is to find defects, an established defect management process
1714 is essential. Defects may be reported during any phase of the SDLC. The process must be followed by all
1715 stakeholders. At a minimum, the defect management process includes a workflow for handling individual
1716 defects from their discovery to their closure and rules for their classification. The workflow typically
1717 comprises activities to log the reported defect (in static testing) or failure (in dynamic tests), analyze and
1718 classify the issue, decide on a suitable response, such as to fix or ignore, and finally to close the defect.
1719 Typical defect reports have the following objectives:
1720 • Provide those responsible for handling and resolving reported defects with sufficient information
1721 to resolve the problem
1722 • Provide a means of tracking the quality of the work product
1723 • Provide ideas for development and test process improvement
1724 A defect report logged during dynamic testing typically includes:
1725 • Unique identifier
1726 • Title and a short summary of the defect being reported, including when it was observed
1727 • Date of the defect report, issuing organization, and author, including their role
1728 • Identification of the test object and test environment
1729 • Context of the defect (e.g., test case being run, test activity being performed, SDLC phase, and
1730 other relevant information, such as the test technique, checklist or test data being used)
1731 • Description of the defect to enable reproduction and resolution, including the steps that detected
1732 the defect, and any relevant log files, database dumps, screenshots, or recordings
1733 • Expected and actual results

v4.0 Page 54 of 70 beta release

1734 • Severity (degree of impact) of the defect on the interests of stakeholders

1735 • Urgency/priority to fix
1736 • Status of the defect (e.g., open, deferred, duplicate, waiting to be fixed, awaiting confirmation
1737 testing, re-opened, closed)
1738 • References (e.g., to the test case)
1739 Some of this data may be automatically included or managed when using defect management tools (e.g.,
1740 identifier, date, author and initial status). It is advisable to handle defects from static testing in a similar
1741 way.
1742 A document template for a defect report and example defect reports can be found in the ISO/IEC/IEEE
1743 29119-3 standard, which refers to defect reports as incident reports.

v4.0 Page 55 of 70 beta release

1744 6. Test Tools – 20 minutes

1745 Keywords
1746 test automation
1747

1748 Learning Objectives for Chapter 6:

1749 6.1 Tool Support for Testing

1750 FL-6.1.1 (K2) Explain how different types of test tools support testing

1751 6.2 Benefits and Risks of Test Automation

1752 FL-6.2.1 (K1) Recall the benefits and risks of test automation

v4.0 Page 56 of 70 beta release

1753 6.1. Tool Support for Testing

1754 Test tools support and facilitate many testing activities. Examples include, but are not limited to:
1755 • Management tools – increase the test process efficiency by facilitating management of
1756 application lifecycle, requirements, tests, defects, configuration
1757 • Static testing tools – support the tester in performing reviews and static analysis
1758 • Test design and implementation tools – facilitate generation of the test case, test data and test
1759 procedures
1760 • Test execution and coverage tools – facilitate automated test execution and coverage
1761 measurement
1762 • Non-functional testing tools – allow the tester to perform non-functional testing that is difficult or
1763 impossible to perform manually
1764 • DevOps tools – support the DevOps delivery pipeline, workflow tracking, build automation
1765 process, automated software deployment, continuous integration
1766 • Collaboration tools – facilitate communication
1767 • Tools supporting scalability and deployment standardization (e.g., virtual machines,
1768 containerization tools, etc.)
1769 Test tool is any tool that assists in testing, which supports one or more testing activities (e.g., a
1770 spreadsheet is a test tool in this context).

1771 6.2. Benefits and Risks of Test Automation

1772 Simply acquiring a tool does not guarantee success. Each new tool will require effort to achieve real and
1773 lasting benefits (e.g., for tool introduction, maintenance or training). There are also some risks, which
1774 need analysis and mitigation to avoid test automation failures.
1775 Potential benefits of using test automation tools include:
1776 • A reduction in repetitive manual work that saves time. (e.g., execute regression tests, re-enter the
1777 same test data, compare expected vs actual results, and check against coding standards)
1778 • Greater consistency and repeatability which prevents simple human errors. (e.g., tests are
1779 consistently derived from requirements, test data is created in a systematic manner, and tests are
1780 executed by a tool in the same order with the same frequency)
1781 • More objective assessment (e.g., coverage) and provides measures that are too complicated for
1782 humans to derive
1783 • Easier access to information about the testing to support test management and reporting (e.g.,
1784 statistics, graphs, and aggregated data about test progress, defect rates, and execution duration)
1785 • Reduced test execution times to provide earlier defect detection, faster feedback and faster time
1786 to market

v4.0 Page 57 of 70 beta release

1787 • Allows more time for testers to design new, deeper, more effective tests if using robust and
1788 efficient tools
1789 Potential risks of using test automation tools include:
1790 • Unrealistic expectations for the benefits of a tool (including functionality and ease of use).
1791 • Inaccurate estimations of time, costs, effort required to introduce a tool, maintain test scripts and
1792 change of the existing manual test process
1793 • Using a testing tool when manual testing is more appropriate
1794 • Relying on a tool, when human critical thinking is what is needed
1795 • The dependency on the tool vendor which may go out of business, retire the tool, sell the tool to a
1796 different vendor or provide poor support (e.g., responses to queries, upgrades, and defect fixes)
1797 • The plan of using an open-source project may be abandoned, meaning that no further updates
1798 are available, or its internal components may require quite frequent updates as a further
1799 development of the tool
1800 • Platform and the tool are not compatible
1801 • Failure to follow regulatory requirements and/or safety standards by the tool

v4.0 Page 58 of 70 beta release

1802 7. References
1803 Standards
1804 ISO/IEC/IEEE 29119-1 (2022) Software and systems engineering – Software testing – Part 1: General
1805 Concepts
1806 ISO/IEC/IEEE 29119-2 (2021) Software and systems engineering – Software testing – Part 2: Test
1807 processes
1808 ISO/IEC/IEEE 29119-3 (2021) Software and systems engineering – Software testing – Part 3: Test
1809 documentation
1810 ISO/IEC/IEEE 29119-4 (2021) Software and systems engineering – Software testing – Part 4: Test
1811 techniques
1812 ISO/IEC 25010, (2011) Systems and software engineering – Systems and software Quality Requirements
1813 and Evaluation (SQuaRE) System and software quality models
1814 ISO/IEC 20246 (2017) Software and systems engineering – Work product reviews
1815 ISO 31000 (2018) Risk management – Principles and guidelines

1816 Books
1817 Adzic, G. (2009) Bridging the Communication Gap: Specification by Example and Agile Acceptance
1818 Testing, Neuri Limited
1819 Ammann, P. and Offutt, J. (2016) Introduction to Software Testing (2e), Cambridge University Press
1820 Andrews, M. and Whittaker, J. (2006) How to Break Web Software: Functional and Security Testing of
1821 Web Applications and Web Services, Addison-Wesley Professional
1822 Beizer, B. (1990) Software Testing Techniques (2e), Van Nostrand Reinhold: Boston MA
1823 Boehm, B. (1981) Software Engineering Economics, Prentice Hall, Englewood Cliffs, NJ
1824 Buxton, J.N. and Randell B., eds (1970), Software Engineering Techniques. Report on a conference
1825 sponsored by the NATO Science Committee, Rome, Italy, 27–31 October 1969, p. 16.
1826 Chelimsky, D. et al. (2010) The Rspec Book: Behaviour Driven Development with Rspec, Cucumber, and
1827 Friends, The Pragmatic Bookshelf: Raleigh, NC
1828 Cohn, M. (2009) Succeeding with Agile: Software Development Using Scrum, Addison-Wesley
1829 Copeland, L. (2004) A Practitioner’s Guide to Software Test Design, Artech House: Norwood MA
1830 Craig, R. and Jaskiel, S. (2002) Systematic Software Testing, Artech House: Norwood MA
1831 Crispin, L. and Gregory, J. (2008) Agile Testing: A Practical Guide for Testers and Agile Teams, Pearson
1832 Education: Boston MA
1833 Forgács, I., and Kovács, A. (2019) Practical Test Design: Selection of traditional and automated test
1834 design techniques, BCS, The Chartered Institute for IT
1835 Gawande A. (2009) The Checklist Manifesto: How to Get Things Right, New York, NY: Metropolitan
1836 Books

v4.0 Page 59 of 70 beta release

1837 Gärtner, M. (2011), ATDD by Example: A Practical Guide to Acceptance Test-Driven Development,
1838 Pearson Education: Boston MA
1839 Gilb, T., Graham, D. (1993) Software Inspection, Addison Wesley
1840 Hendrickson, E. (2013) Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing, The
1841 Pragmatic Programmers
1842 Hetzel, B. (1988) The Complete Guide to Software Testing, 2nd ed., John Wiley and Sons
1843 Jeffries, R., Anderson, A., Hendrickson, C. (2000) Extreme Programming Installed, Addison-Wesley
1844 Professional
1845 Jorgensen, P. (2014) Software Testing, A Craftsman’s Approach (4e), CRC Press: Boca Raton FL
1846 Kan, S. (2003) Metrics and Models in Software Quality Engineering, 2 nd ed., Addison-Wesley
1847 Kaner, C., Falk, J., and Nguyen, H.Q. (1999) Testing Computer Software, 2nd ed., Wiley
1848 Kaner, C., Bach, J., and Pettichord, B. (2011) Lessons Learned in Software Testing: A Context-Driven
1849 Approach, 1st ed., Wiley
1850 Kim, G., Humble, J., Debois, P. and Willis, J. (2016) The DevOps Handbook, Portland, OR
1851 Koomen, T., van der Aalst, L., Broekman, B. and Vroon, M. (2006) TMap Next for result-driven testing,
1852 UTN Publishers, The Netherlands
1853 Myers, G. (2011) The Art of Software Testing, (3e), John Wiley & Sons: New York NY
1854 O’Regan, G. (2019) Concise Guide to Software Testing, Springer Nature Switzerland
1855 Pressman, R.S. (2019) Software Engineering. A Practitioner’s Approach, 9th ed., McGraw Hill
1856 Roman, A. (2018) Thinking-Driven Testing. The Most Reasonable Approach to Quality Control, Springer
1857 Nature Switzerland
1858 Van Veenendaal, E (ed.) (2012) Practical Risk-Based Testing, The PRISMA Approach, UTN Publishers:
1859 The Netherlands
1860 Watson, A.H., Wallace, D.R. and McCabe, T.J. (1996) Structured Testing: A Testing Methodology Using
1861 the Cyclomatic Complexity Metric, U.S. Dept. of Commerce, Technology Administration, NIST
1862 Westfall, L. (2009) The Certified Software Quality Engineer Handbook, ASQ Quality Press
1863 Whittaker, J. (2002) How to Break Software: A Practical Guide to Testing, Pearson
1864 Whittaker, J. (2009) Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test
1865 Design, Addison Wesley
1866 Whittaker, J. and Thompson, H. (2003) How to Break Software Security, Addison Wesley
1867 Wiegers, K. (2001) Peer Reviews in Software: A Practical Guide, Addison-Wesley Professional

1868 Articles and Web Pages

1869 Brykczynski, B. (1999) “A survey of software inspection checklists,” ACM SIGSOFT Software Engineering
1870 Notes, 24(1), pp. 82-89
1871 Enders, A. (1975) “An Analysis of Errors and Their Causes in System Programs,” IEEE Transactions on
1872 Software Engineering 1(2), pp. 140-149

v4.0 Page 60 of 70 beta release

1873 Manna, Z., Waldinger, R. (1978) “The logic of computer programming,” IEEE Transactions on Software
1874 Engineering 4(3), pp. 199-229
1875 Marick, B. (2003) Exploration through Example, [Link]
1876 blog/2003/08/[Link]#agile-testing-project-1
1877 Nielsen, J. (1994) “Enhancing the explanatory power of usability heuristics,” Proceedings of the SIGCHI
1878 Conference on Human Factors in Computing Systems: Celebrating Interdependence, pp. 152–158, ACM
1879 Press
1880 Wake, B. (2003) “INVEST in Good Stories, and SMART Tasks,” [Link]
1881 stories-and-smart-tasks/

v4.0 Page 61 of 70 beta release

1882 8. Appendix A – Learning Objectives/Cognitive Level of

1883 Knowledge
1884 The following learning objectives are defined as applying to this syllabus. Each topic in the syllabus will
1885 be examined according to the learning objective for it. The learning objectives begin with an action verb
1886 corresponding to its cognitive level of knowledge as listed below.
1887 Level 1: Remember (K1) – the candidate will remember, recognize and recall a term or concept.
1888 Action verbs: identify, recall, recognize.
1889 Examples:
1890 • “Identify typical objectives of testing.”
1891 • “Recall the concepts of the test pyramid.”
1892 • “Recognize the different roles and responsibilities in a review.”

1893 Level 2: Understand (K2) – the candidate can select the reasons or explanations for statements related
1894 to the topic, and can summarize, compare, classify and give examples for the testing concept.
1895 Action verbs: classify, compare, contrast, differentiate, distinguish, exemplify, explain, give examples,
1896 interpret, summarize.

1897 Examples:
1898 • “Classify the different options for writing acceptance criteria.”
1899 • “Compare the different roles in testing” (look for similarities, differences or both).
1900 • “Distinguish between project and product risks” (allows concepts to be differentiated).
1901 • “Exemplify the purpose and content of a test plan.”
1902 • “Explain the impact of context on the test process.”
1903 • “Summarize the activities of the review process.”
1904 Level 3: Apply (K3) – the candidate can carry out a procedure when confronted with a familiar task, or
1905 select the correct procedure and apply it to a given context.
1906 Action verbs: apply, implement, prepare, use.
1907 Examples:
1908 • “Apply test case prioritization” (should refer to a procedure, technique, process, algorithm etc.).
1909 • “Prepare a defect report.”
1910 • “Use boundary value analysis to derive test cases.”
1911 References for the cognitive levels of learning objectives:
1912 Anderson, L. W. and Krathwohl, D. R. (eds) (2001) A Taxonomy for Learning, Teaching
1913 Assessing: A Revision of Bloom's Taxonomy of Educational Objectives, Allyn & Bacon

v4.0 Page 62 of 70 beta release

1914 9. Appendix B – Business Outcomes traceability matrix with Learning Objectives

1915 This section lists the number of Foundation Level Learning Objectives related to the Business Outcomes and the traceability between Foundation
1916 Level Business Outcomes and Foundation Level Learning Objectives.
1917

FL-BO10

FL-BO11

FL-BO12

FL-BO13

FL-BO14
FL-BO1

FL-BO2

FL-BO3

FL-BO4

FL-BO5

FL-BO6

FL-BO7

FL-BO8

FL-BO9
Business Outcomes: Foundation Level

BO1 Understand what testing is and why it is beneficial 6

BO2 Understand fundamental concepts of software testing 22
Identify the test approach and activities to be implemented depending on the
BO3 6
context of testing
BO4 Assess and improve the quality of the documentation 9
BO5 Increase the effectiveness and efficiency of testing 20
BO6 Align the testing process with the software development lifecycle 6
BO7 Understand test management principles 6
BO8 Write and communicate clear and understandable defect reports 1
BO9 Understand the factors that influence the test priorities and test efforts 7

BO10 Work as part of a cross-functional team 8

BO11 Know risks and benefits related to test automation. 1

BO12 Identify essential skills required for testing 5
BO13 Understand the impact of risk on testing 4
BO14 Effectively report on test progress and quality 4
1918

v4.0 Page 63 of 70 beta release

1919
BUSINESS OUTCOMES
Chapter/
K-

FL-BO10

FL-BO11

FL-BO12

FL-BO13

FL-BO14
FL-BO1

FL-BO2

FL-BO3

FL-BO4

FL-BO5

FL-BO6

FL-BO7

FL-BO8

FL-BO9
section/ Learning objective
level
subsection

Chapter 1 Fundamentals of Testing

1.1 What is Testing?
1.1.1 Identify typical objectives of testing K1 X
1.1.2 Differentiate testing from debugging K2 X
1.2 Why is Testing Necessary?
1.2.1 Exemplify why testing is necessary K2 X
1.2.2 Recall the relation between testing and quality assurance K1 X
1.2.3 Distinguish between root cause, error, defect, and failure K2 X
1.3 Testing Principles
1.3.1 Explain the seven testing principles K2 X
1.4 Test Activities, Work Products and Roles
1.4.1 Summarize the different test activities and tasks K2 X
1.4.2 Explain the impact of context on the test process K2 X X
1.4.3 Differentiate the work products that support the test activities K2 X
1.4.4 Explain the value of maintaining traceability K2 X X
1.4.5 Compare the different roles in testing K2 X
1.5 Essential Skills and Good Practices
1.5.1 Give examples of the generic skills required for testing K2 X
1.5.2 Recall the advantages of the whole team approach K1 X
1.5.3 Distinguish the benefits and drawbacks of independence of testing K2 X
Chapter 2 Testing Throughout the Software Development Lifecycles
2.1 Testing in the Context of Software Development Lifecycles
2.1.1 Explain the impact of the chosen software development lifecycle on testing K2 X

v4.0 Page 64 of 70 beta release

Remember good testing practices regardless of the chosen software development K1

2.1.2 X
model
2.1.3 Recall the examples of test-first approaches to development K1 X
2.1.4 Summarize how DevOps might have an impact on testing K2 X X X X
2.1.5 Explain the shift-left approach K2 X X
2.1.6 Explain how retrospectives can be used as a mechanism for process improvement K2 X X
2.2 Test Levels and Test Types
2.2.1 Distinguish the different test levels K2 X X
2.2.2 Compare and contrast functional, non-functional and white-box testing K2 X
2.2.3 Distinguish confirmation testing from regression testing K2 X
2.3 Maintenance Testing
2.3.1 Summarize maintenance testing and its triggers K2 X X
Chapter 3 Static Testing
3.1 Static Testing Basics
Recognize types of products that can be examined by the different static testing K1
3.1.1 X X
techniques
3.1.2 Explain the value of static testing K2 X X X
3.1.3 Compare and contrast static and dynamic testing K2 X X
3.2 Feedback and Review Process
3.2.1 Identify the benefits of early and frequent feedback K1 X X X
3.2.2 Summarize the activities of the review process K2 X X
3.2.3 Recognize the different roles and responsibilities in a review K1 X X
3.2.4 Compare and contrast the different review types K2 X
3.2.5 Recall the factors that contribute to a successful review K1 X X
Chapter 4 Test Analysis and Design
4.1 Test Techniques Overview
4.1.1 Distinguish black-box, white-box and experience-based test techniques K2 X
4.2 Black-box Testing
4.2.1 Use equivalence partitioning to derive test cases K3 X
4.2.2 Use boundary value analysis to derive test cases K3 X

v4.0 Page 65 of 70 beta release

4.2.3 Use decision table testing to derive test cases K3 X

4.2.4 Use state transition testing to derive test cases K3 X
4.3 White-box Testing
4.3.1 Explain statement testing K2 X
4.3.2 Explain branch testing K2 X
4.3.3 Explain the value of white box testing K2 X X
4.4 Experience-based Testing
4.4.1 Explain error guessing K2 X
4.4.2 Explain exploratory testing K2 X
4.4.3 Explain checklist-based testing K2 X
4.5 Testing in an Agile context
Explain how to write user stories in collaboration with developers and business K2
4.5.1 X X
representatives
4.5.2 Classify the different options for writing acceptance criteria K2 X
4.5.3 Use acceptance test-driven development (ATDD) to derive test cases K3 X
Chapter 5 Managing the Test Activities
5.1 Test Planning
5.1.1 Exemplify the purpose and content of a test plan K2 X X
5.1.2 Recognize how a tester adds value to iteration and release planning K1 X X X
Compare and contrast entry and exit criteria, Definition of Ready, and Definition of K2
5.1.3 X X X
Done
5.1.4 Use estimation techniques to calculate the required testing effort K3 X X
5.1.5 Apply test case prioritization K3 X X
5.1.6 Recall the concepts of the test pyramid K1 X
Summarize the testing quadrants and their relationships with test levels and test types K2
5.1.7 X X

5.2 Risk Management

5.2.1 Identify risk level by using likelihood and impact K1 X X
5.2.2 Distinguish between project and product risks K2 X X
5.2.3 Explain how product risk analysis may influence thoroughness and scope of testing K2 X X X

v4.0 Page 66 of 70 beta release

5.2.4 Explain what measures can be taken in response to analyzed product risks K2 X X X
5.3 Test Monitoring and Control
5.3.1 Recall metrics used for testing K1 X X
5.3.2 Summarize the purposes, contents, and audiences for test reports K2 X X X
5.3.3 Exemplify how to communicate the status of testing K2 X X
5.4 Configuration Management
5.4.1 Summarize how configuration management supports testing K2 X X
5.5 Defect Management
5.5.1 Prepare a defect report K3 X X
Chapter 6 Test Tools
6.1 Tool Support for Testing
6.1.1 Explain how different types of test tools support testing K2 X
6.2 Benefits and Risks of Test Automation
6.2.1 Recall the benefits and risks of test automation K1 X X
1920

v4.0 Page 67 of 70 beta release

1921 10. Appendix C – Release Notes

1922
1923 ISTQB® Foundation Syllabus v4.0 is a major update based on the Foundation Level syllabus (v3.1.1) and
1924 the Agile Tester 2014 syllabus. For this reason, there are no detailed release notes per chapter and section.
1925 However, a summary of principal changes is provided below. Additionally, in a separate Release Notes
1926 document, ISTQB® provides traceability between the learning objectives (LO) in the version 3.1.1 of the
1927 Foundation Level Syllabus, 2014 version of the Agile Tester Syllabus, and the learning objectives in the
1928 new Foundation Level v4.0 Syllabus, showing which LOs have been added, updated, or removed.
1929 In 2022 more than one million people in more than 100 countries have taken the Foundation Level exam,
1930 and almost 700,000 are certified testers worldwide. With the expectation that all of them have read the
1931 Foundation Syllabus to be able to pass the exam, this makes the Foundation Syllabus likely to be the most
1932 read software testing document ever! This major update is made in respect of this heritage and to improve
1933 the views of hundreds of thousands more people on the level of quality that ISTQB ® delivers to the global
1934 testing community.
1935 In this version all LOs have been edited to make them atomic, and to create one-to-one traceability between
1936 LOs and syllabus sections, thus not having content without also having a LO. The goal is to make this
1937 version easier to read, understand, learn, and translate, focusing on increasing practical usefulness and
1938 the balance between knowledge and skills.
1939 This major release has made the following changes:
1940 • Size reduction of the overall syllabus. Syllabus is not a textbook, but a document that serves to
1941 outline the basic elements of an introductory course on software testing, including what topics
1942 should be covered and on what level. Therefore, in particular:
1943 o In most cases examples are excluded from the text. It is a task of a training provider to
1944 provide the examples, as well as the exercises, during the training
1945 o The “Syllabus writing checklist” was followed, which suggests the maximum text size for
1946 LOs at each K-level (K1 = max. 10 lines, K2 = max. 15 lines, K3 = max. 25 lines)
1947 • Reduction of the number of LOs compared to the Foundation v3.1.1 and Agile v2014 syllabi
1948 o 14 K1 LOs compared with 21 LOs in FL v3.1.1 (15) and AT 2014 (6)
1949 o 42 K2 LOs compared with 53 LOs in FL v3.1.1 (40) and AT 2014 (13)
1950 o 8 K3 LOs compared with 15 LOs in FL v3.1.1 (7) and AT 2014 (8)
1951 • More extensive references to classic and/or respected books and articles on software testing and
1952 related topics are provided
1953 • Major changes in chapter 1 (Fundamentals of Testing)
1954 o Section on test skills expanded and improved
1955 o Section on the whole team approach (K1) added
1956 o Section on the independence of testing moved to Chapter 1 from Chapter 5
1957 • Major changes in chapter 2 (Testing Throughout the SDLCs)

V4.0 Page 68 of 70 beta release

1958 o Sections 2.1.1 and 2.1.2 rewritten and improved, the corresponding LOs are modified
1959 o More focus on practices like: test-first approach (K1), shift-left (K2), retrospectives (K2)
1960 o New section on testing in the context of DevOps (K2)
1961 o Integration testing level split into two separate test levels: component integration testing
1962 and system integration testing
1963 • Major changes in chapter 3 (Static Testing)
1964 o Section on review techniques, together with the K3 LO (apply a review technique)
1965 removed
1966 • Major changes in chapter 4 (Test Analysis and Design)
1967 o Use case testing removed (but still present in the Advanced Test Analyst syllabus)
1968 o More focus on collaboration-based approach to testing: new K3 LO about using ATDD to
1969 derive test cases and two new K2 LOs about user stories and acceptance criteria
1970 o Decision testing and coverage replaced with branch testing and coverage (first, branch
1971 coverage is more commonly used in practice; second, different standards define the
1972 decision differently, as opposed to “branch”; third, this solves a subtle, but serious flaw
1973 from the old FL2018 which claims that „100% decision coverage implies 100% statement
1974 coverage” – this sentence is not true in case of programs with no decisions)
1975 o Section on the value of white-box testing improved
1976 • Major changes in chapter 5 (Managing the Test Activities)
1977 o Section on test strategies/approaches removed
1978 o New K3 LO on estimation techniques for estimating the test effort
1979 o More focus on the well-known Agile-related concepts and tools in test management:
1980 iteration and release planning (K1), test pyramid (K1), and testing quadrants (K2)
1981 o Section on risk management better structured by describing four main activities: risk
1982 identification, risk assessment, risk mitigation and risk monitoring
1983 • Major changes in chapter 6 (Test Tools)
1984 o Content on some test automation issues reduced as being too advanced for the
1985 foundation level – section on tools selection, performing pilot projects and introducing
1986 tools into organization removed
1987
1988

V4.0 Page 69 of 70 beta release

1989 11. Index

1990 All terms are defined in the ISTQB® Glossary ([Link]
1991 To be done after the beta release.

V4.0 Page 70 of 70 beta release

Certified Tester Foundation Level Syllabus
100% (1)
Certified Tester Foundation Level Syllabus
74 pages
tmp1 PDF
No ratings yet
tmp1 PDF
290 pages
Advanced Test Management Syllabus V3.0
No ratings yet
Advanced Test Management Syllabus V3.0
87 pages
ISTQB
No ratings yet
ISTQB
6 pages
TAE Sample Answers 2024
No ratings yet
TAE Sample Answers 2024
34 pages
ISTQB Software Testing Cheat Sheet
No ratings yet
ISTQB Software Testing Cheat Sheet
3 pages
AI Testing Sample Exam Questions
No ratings yet
AI Testing Sample Exam Questions
19 pages
Test Monitoring and Control Basics
No ratings yet
Test Monitoring and Control Basics
26 pages
ISTQB Foundation Level Sample Paper I
No ratings yet
ISTQB Foundation Level Sample Paper I
6 pages
Overview of Software Testing Approaches
100% (7)
Overview of Software Testing Approaches
4 pages
(Ebook) Testing IT: An Off-the-Shelf Software Testing Process by John Watkins, Simon Mills ISBN 0521148014
No ratings yet
(Ebook) Testing IT: An Off-the-Shelf Software Testing Process by John Watkins, Simon Mills ISBN 0521148014
78 pages
AI and ML in Modern Quality Engineering
No ratings yet
AI and ML in Modern Quality Engineering
22 pages
Test Design Techniques Overview
No ratings yet
Test Design Techniques Overview
6 pages
ISTQB Tananyag 2015
No ratings yet
ISTQB Tananyag 2015
257 pages
QA Tester Intern Mock Interview Guide
No ratings yet
QA Tester Intern Mock Interview Guide
5 pages
ISTQB CTFL Intro
No ratings yet
ISTQB CTFL Intro
9 pages
Next.js Testing Strategies Guide
100% (1)
Next.js Testing Strategies Guide
6 pages
Fundamentals of Software Testing Guide
No ratings yet
Fundamentals of Software Testing Guide
65 pages
ISTQB Foundation Level Exam Guide
No ratings yet
ISTQB Foundation Level Exam Guide
13 pages
CT-GenAI Syllabus Overview
No ratings yet
CT-GenAI Syllabus Overview
70 pages
ISTQB CT PT Sample Exam A Answers v1.2
No ratings yet
ISTQB CT PT Sample Exam A Answers v1.2
21 pages
Sample Istqb Quest
100% (1)
Sample Istqb Quest
39 pages
ISTQB Foundation Level Exam Summary
No ratings yet
ISTQB Foundation Level Exam Summary
4 pages
Foundation Level Sample Exam v2.9
No ratings yet
Foundation Level Sample Exam v2.9
26 pages
Black Box Testing with BVA Examples
No ratings yet
Black Box Testing with BVA Examples
8 pages
ISTQB Agile Tester Extension - Learning Material
No ratings yet
ISTQB Agile Tester Extension - Learning Material
106 pages
ISTQB Test Oracle Sample Questions
No ratings yet
ISTQB Test Oracle Sample Questions
10 pages
Software Testing Fundamentals Guide
No ratings yet
Software Testing Fundamentals Guide
74 pages
ISTQB Agile Tester Exam Structure
No ratings yet
ISTQB Agile Tester Exam Structure
10 pages
ISTQB Advanced Test Manager Exam Guide
No ratings yet
ISTQB Advanced Test Manager Exam Guide
10 pages
ISTQB Exam Study Guide & Questions
No ratings yet
ISTQB Exam Study Guide & Questions
4 pages
Key Software Testing Resources
No ratings yet
Key Software Testing Resources
1 page
ISTQB Foundation Sample Questions 6
No ratings yet
ISTQB Foundation Sample Questions 6
10 pages
Manual Test Cases for Banking App
No ratings yet
Manual Test Cases for Banking App
39 pages
Manual Testing
No ratings yet
Manual Testing
11 pages
Entry and Exit Criteria in Testing
No ratings yet
Entry and Exit Criteria in Testing
10 pages
Testing Brochure
No ratings yet
Testing Brochure
7 pages
Model-Based Software Testing
No ratings yet
Model-Based Software Testing
22 pages
ISTQB Testing Fundamentals Overview
No ratings yet
ISTQB Testing Fundamentals Overview
34 pages
Pearson VUE Authorized Test Center
No ratings yet
Pearson VUE Authorized Test Center
4 pages
Exam Questions for Chapter 2 Testing
No ratings yet
Exam Questions for Chapter 2 Testing
24 pages
10 Test Automation Frameworks For Cross-Browser Testing: A Comparison Guide For 2017
100% (1)
10 Test Automation Frameworks For Cross-Browser Testing: A Comparison Guide For 2017
26 pages
ISTQB Foundation Level Sample Exam 2
No ratings yet
ISTQB Foundation Level Sample Exam 2
4 pages
Software Testing Strategies and Techniques
100% (1)
Software Testing Strategies and Techniques
18 pages
TestManager v2.6.5 UserGuide
No ratings yet
TestManager v2.6.5 UserGuide
116 pages
ISTQB Chapter 1: Testing Fundamentals
No ratings yet
ISTQB Chapter 1: Testing Fundamentals
15 pages
Certificate Download and Quiz Testing
No ratings yet
Certificate Download and Quiz Testing
69 pages
Agile Quality Engineering Strategies
No ratings yet
Agile Quality Engineering Strategies
30 pages
Agile Testing Methodology Overview
No ratings yet
Agile Testing Methodology Overview
12 pages
Automated API Testing Overview
No ratings yet
Automated API Testing Overview
4 pages
ISTQB Foundation Level Practice Test 1
No ratings yet
ISTQB Foundation Level Practice Test 1
2 pages
ISTQB Advanced Test Manager Exam Guide
No ratings yet
ISTQB Advanced Test Manager Exam Guide
26 pages
Model-Based Testing Strategies at Spotify
No ratings yet
Model-Based Testing Strategies at Spotify
26 pages
ISTQB Foundation Sample Question Paper 27
No ratings yet
ISTQB Foundation Sample Question Paper 27
6 pages
Manual Testing Vs Automation Testing
No ratings yet
Manual Testing Vs Automation Testing
21 pages
ISTQB Performance Testing - Chapter 1
100% (1)
ISTQB Performance Testing - Chapter 1
25 pages
Understanding Test Beds in Software Testing
No ratings yet
Understanding Test Beds in Software Testing
5 pages
ISTQB 40-Question Practice Test
No ratings yet
ISTQB 40-Question Practice Test
12 pages
Software Testing Techniques Quiz
No ratings yet
Software Testing Techniques Quiz
38 pages
ISTQB Certified Tester Syllabus v4.0.1
No ratings yet
ISTQB Certified Tester Syllabus v4.0.1
149 pages
Scrum Timeboxed Events Overview
No ratings yet
Scrum Timeboxed Events Overview
56 pages
Understanding Agile Development Success
No ratings yet
Understanding Agile Development Success
19 pages
Waterfall vs. Incremental Models
No ratings yet
Waterfall vs. Incremental Models
4 pages
Scrum Reference v18.09
No ratings yet
Scrum Reference v18.09
2 pages
Android Shutdown Logs Analysis
No ratings yet
Android Shutdown Logs Analysis
2 pages
Android SettingsProvider Log Analysis
No ratings yet
Android SettingsProvider Log Analysis
30 pages
Software Testing Concepts and Practices
No ratings yet
Software Testing Concepts and Practices
2 pages
Multi-Criteria Framework for Testing Tools
No ratings yet
Multi-Criteria Framework for Testing Tools
20 pages
Coding Tools for Visual Basic Development
No ratings yet
Coding Tools for Visual Basic Development
5 pages
Web Development Skills Overview
No ratings yet
Web Development Skills Overview
3 pages
Understanding Scrum and SBOK Guide
No ratings yet
Understanding Scrum and SBOK Guide
21 pages
Evolution of Software Engineering Principles
No ratings yet
Evolution of Software Engineering Principles
61 pages
Test Bank For Systems Analysis and Design: An Object-Oriented Approach With UML, 5th Edition by Dennis, Wixom, Tegarden Resource Guide
100% (5)
Test Bank For Systems Analysis and Design: An Object-Oriented Approach With UML, 5th Edition by Dennis, Wixom, Tegarden Resource Guide
193 pages
Introduction to Blazor Framework
No ratings yet
Introduction to Blazor Framework
2 pages
QA SDET Lead with 10+ Years Experience
No ratings yet
QA SDET Lead with 10+ Years Experience
2 pages
Software Engineering MOOC Completion Report
No ratings yet
Software Engineering MOOC Completion Report
25 pages
Rancangan Peningkatan PJPK 2023/24
No ratings yet
Rancangan Peningkatan PJPK 2023/24
22 pages
Azure DevOps CI/CD for CAP Model
No ratings yet
Azure DevOps CI/CD for CAP Model
29 pages
ISTQB Foundation Level Sample Exam D
No ratings yet
ISTQB Foundation Level Sample Exam D
15 pages
Overview of Agile Software Development Models
No ratings yet
Overview of Agile Software Development Models
13 pages
Android Shutdown Thread Log Analysis
No ratings yet
Android Shutdown Thread Log Analysis
4 pages
Sandeep Kumar: SDET Profile Overview
No ratings yet
Sandeep Kumar: SDET Profile Overview
4 pages
Academic Timetable for Computer Engineering
No ratings yet
Academic Timetable for Computer Engineering
1 page
Software Engineering Principles Overview
No ratings yet
Software Engineering Principles Overview
81 pages
Java File Not Found and Security Exceptions
No ratings yet
Java File Not Found and Security Exceptions
6 pages
Understanding SAP Notes and Updates
No ratings yet
Understanding SAP Notes and Updates
3 pages
Software Engineering and Evolution Guide
No ratings yet
Software Engineering and Evolution Guide
22 pages
Venkata Ramaiah: Selenium Test Engineer
No ratings yet
Venkata Ramaiah: Selenium Test Engineer
2 pages
For The Students Software Life Cycle Model
No ratings yet
For The Students Software Life Cycle Model
44 pages
SPM - R23 - Lesson Plan
No ratings yet
SPM - R23 - Lesson Plan
4 pages