Agile Testing Methodology
DC/ SC/ Geo/Practice: IS-Life Science & Healthcare
Name of the project/support fn: Cigna
Name of the author: Gopinath Reddiagowder (147490)
Date Created: 05/01/2011
Agenda
¾ Agile in a Nutshell
¾ How do we Test?
¾ What Kind of test Should be Used?
¾ What Test Techniques are Useful?
¾ Test Conclusion
¾ Constantly Improve
Agile in a Nutshell
The aspect of "Agile Development" that comes to the front is simply "Reduce
the gap in time between doing some activity and gaining feedback”
¾ One of the big parts of being in an agile state of mind is trying to always do
the best you can with the time and resources you have.
¾ Not discovering a bug until late in the development cycle is obviously
expensive.
¾ Going overboard on building code to meet more than the feature demands is
potentially no less wasteful.
¾ In agile development we want to streamline the process to be as economical
as possible and to get feedback as early and as often as possible.
How do we test?
¾Typically testing is done manually and through automation
¾ Though you should automate as much of the testing as possible, this
does not mean that all manual testing is no longer needed
¾Be sure you are not using manual tests to do that which could be
automated
¾Be sure you are not trying to do expensive automation of tests that are
better left for human testers
¾Be sure you are not avoiding doing certain tests just because you cannot
automate them!
What kind of Test Should be Used
There is no "one-size-fits-all" strategy for Agile. Here are an assortment of test
types and the purpose/role they can play during development.
1, Unit tests are good for the developers to build up as they are coding their assigned
features. The purpose is to ensure that the basic functionality of the underlying code continues
to work as changes are made. Plus unit tests are a useful way to document how the "API" is to
be used
2, Acceptance tests are used to ensure specific functionality works. Essentially, the
acceptance test is a client-specified scenario that indicates the desired functionality is properly
implemented. By having an acceptance test, the developer can know when "good enough" is
met, and the feature is complete
What Kind of test Should be Used?
3, UI tests - these often involve some means to step through the page flow, provide
known inputs and check actual results against expected results.
4, Usability testing - this is a whole other area of testing that often involves humans. But
usability can often be "make-or-break" acceptance criteria. Some projects can also benefit
from automated tests that ensure UI standards are being followed (where they may not be
easily enforced through UI frameworks).
5, Performance tests - running a suite of tests to ensure various non-functional metrics
are met is critical for many apps. If your app requires stringent performance metrics, you need
to tackle this up front during the initial architecture and design phases. Before building the
entire app, you need to ensure the performance benchmarks are being met. Typically, you
build a thin slice of the application and conduct stress testing.
What Test Techniques are Useful ?
Smoke Tests - Essentially a handful of critical tests that ensure the basic build functionality
works. Some can be automated, but others can be manual. If it is easy to run the smoke tests, it
can help the development team know that the daily build is probably useful.
Test Harness - is frequently a good way to "institutionalize" exposing functionality of the
system (e.g., via Web Services) for coarse-grain functionality typical for acceptance tests and
major system scenarios (and even external invocation). Wiring up a test harness enables ease of
extending the breadth of the tests as new functionality is added. You can build test harnesses that
are very automated in terms of capturing the user input (using a record mode), and capturing the
output, and getting a user's (in this case typically a tester) acknowledgement that the output is
correct. Each captured test case is added to a test database for later playback orchestration
What Test Techniques are Useful ?
Automated stress test - For a layered architecture with clean separation of
concerns, you may be able to build in some testing robots. This is often especially easy
when dealing with service-based systems.
You can use XML-style config files to control the tests being run. You can build
tests to exercise each layer (e.g., persistence, business, messaging, presentation) of the
system, building up to tests that replicate actual user usages of the system (except for the
UI part). We have built such bots that can then be distributed on dozens of systems and
can even be spawned in multiples on each system. This allows for very simple concurrent
testing to "hammer" the server from all over the world. The config files control the intensity,
breadth, and duration of the tests.
What Test Techniques are Useful ?
Manual tests - these are reserved for those aspects of the system best left for
human testing. Not only is the likelihood of having errors in the testing high, but you will
probably not have time to catch the more elusive bugs. Instead, allow the testers to focus
on being more exhaustive with using the application in complex ways that may not be
easy to automate
Test Conclusion
1, Grab a well-defined feature
2, Write the acceptance test, write some code, write some unit tests and more code as
needed, get the tests to pass, check in the code
3, Is this feature performance critical?
Write a performance test to measure a critical benchmark
Ensure the test passes as you are doing development
Add to the benchmark suite
4, Does the feature have unique UI aspects?
May need to use manual testing to ensure usability and complex functionality
that is too difficult to automate
5, Add the acceptance test to the functional test suite (use an automated tool for this)
6, If the feature passes the acceptance test and the (optional) performance tests, you
can close it and move to the next feature.
Constantly Improve
¾ The tests and techniques should grow over time to meet your needs.
¾ If you get some bug reported that could be prevented in the future by a test,
add a new acceptance or unit test's
¾ If users complain about some performance aspect of the system, add a
benchmark test to your performance suite. This will be used to precisely
document the poor performance, and then to show the improvement once the
developer makes the fixes.
¾ If some tests are no longer worth executing, comment them out (and eventually
delete them).