Unit 5

Manual Testing & Need for Automated Testing tools

Adv & Disadv of Using tools
Selecting a Testing tool
When to use Automated testing tools
Metrics & Measurement : Types, Product & Process metrics , OO metrics
TESTING TOOLS AND MEASUREMENTS 12 Marks

5.1 Manual Testing and Need For Automated Testing Tools

Introduction

Now a days, rigorous application testing is a critical part of all software development life
cycles.
The need of the same is greatly increasing as the organizations are developing mission –
critical systems to support their business activities in day to day life.
Also it becomes necessary to ensure that these systems are reliable, built according to
specification and have the ability to support business processes.
Many internal and external factors are forcing the organizations to ensure a high level of
software quality and reliability.
5.1.1 Manual Testing

To ensure the completeness of software , the testers and the developers often follow a
written test plan.
This leads them through a set of important test cases.
These test cases are physically exerted upon the software to test many it for many factors.
This is what is the manual testing is!!!
Manual testing is the process of manually testing software for defects. It requires a tester
to play the role of an end user whereby they use most of the application's features to
ensure correct behaviour.
To guarantee completeness of testing, the tester often follows a written test plan that
leads them through a set of important test cases.
Definition : Manual testing is the process of using the functions and features of an
application as an end-user would in order to verify the software is working as required.
With manual testing, a tester manually conducts tests on the software by following a set
of pre-defined test cases.
• In Manual Testing , Testers manually execute test cases without using any
automation tools.
• It requires a tester to play the role of an end user.
• Any new application must be manually tested before its testing can be automated.
• Manual testing requires more effort, but is necessary to check automation feasibility.
• Manual Testing does not require knowledge of any testing tool.
• One of the Software Testing Fundamental is "100% Automation is not possible".
This makes Manual Testing imperative.
5.1.2. Advantages of manual testing
1. It is preferable for products with short life cycles.
2. It is preferable for products that have GUIs that constantly change
3. It requires less time and expense to begin productive manual testing.
4. Automation cannot replace human intuition, inference, and inductive reasoning
5. Automation cannot change course in the middle of a test run to examine something that
had not been previously considered.
6. Automation tests are more easily fooled than human testers.
7. It requires less time and expense to begin productive manual testing.
8. Batch Testing is not possible, for each and every test execution Human user
interaction is mandatory.
5.1.3 Disadvantages of manual testing. / Limitation of ManualTesting
In spite of having so many advantages of testing the software manually, this technique is
now being obsolete in the market due to certain reasons.
People need the work to be finished faster. More work has to be completed in less time
with greater efficiency.
The limitations/ disadvantages of using the manual testing techniques are as follows:
1 Manual Test Case scope is very limited.
2 Comparing large amount of data is impractical.
3 Checking relevance of search of operation is difficult
4 Processing change requests during software maintenance takes more time.
5 Manual testing is slow and costly.
6 It is very labour intensive, it takes a long time to complete tests.
7 Manual tests don‘t scale well. As the complexity of the software increases the
complexity of the testing problem grows exponentially.
This leads to an increase in total time devoted to testing as well as total cost of testing.

8 Manual testing is not consistent or repeatable.

Variations in how the tests are performed as inevitable, for various reasons. One tester may
approach and perform a certain test differently from another, resulting in different results on the
same test, because the tests are not being performed identically.
9 Lack of training is the common problem, although not unique to manual software
testing.
10 GUI objects size difference and color combinations are not easy to find in manual
testing.
11 Not suitable for large scale projects and time bound projects.
12 Batch testing is not possible, for each and every test execution Human user
interaction is
mandatory.
13 Comparing large amount of data is impractical.
14 Processing change requests during software maintenance takes more time.
Board Questions

1. State limitations of manual testing. Write any four.

2. Describe any four limitations of manual testing.

3. Write down any four limitations of manual testing.

4. State any eight limitations of manual testing.

5. State various advantages and disadvantages of using manual testing tools.

 5.1.5 Need of automated testing tools
i. An automated testing tool is able to playback pre-recorded and predefined actions,
compare the results to the expected behavior and report the success or failure of these
manual tests to a test engineer.

ii. Once automated tests are created they can easily be repeated and they can be extended
to perform tasks impossible with manual testing.

iii. Because of this, savvy managers have found that automated software testing is an
essential component of successful development projects.
Needs of automated testing tools can be listed as follows:

1. Speed
2. Efficiency
3. Accuracy and Precision
4. Resource Reduction
5. Simulation and Emulation
6. Relentlessness
 Needs of automated testing tools can be listed as follows:

1. Speed: Think about how long it would take you to manually try a few thousand test
cases for the windows Calculator. You might average a test case every five seconds or so.
Automation might be able to run 10, 100 even 1000 times that fast.

2. Efficiency: While you are busy running test cases, you can‘t be doing anything else.
If you have a test tool that reduces the time it takes for you to run your tests, you have
more time for test planning and thinking up new tests.

3. Accuracy and Precision: After trying a few hundred cases, your attention may reduce
and you will start to make mistakes .A test tool will perform the same test and check the
result perfectly, each and every time.
4. Resource Reduction: Sometimes it can be physically impossible to perform a certain
test case. The number of people or the amount of equipment required to create the test
condition could be prohibitive.
A test tool can used to simulate the real world and greatly reduce the physical resources
necessary to perform the testing.

5. Simulation and Emulation: Test tools are used to replace hardware or software that
would normally interface to your product. This ―faceǁ device or application can then be
used to drive or respond to your software in ways that you choose-and ways that might
otherwise be difficult to achieve.

6. Relentlessness: Test tool and automation never tire or give up. It will continuously
test the software.
Advantages
1. Automated testing tools Saves Time and Money
2. Testing Improves Accuracy
3. Increase Test Coverage
4. Automation Does What Manual Testing Cannot
5. Automated QA Testing Helps Developers and Testers
6. Team Morale Improves

7. Test Complete is a Powerful and Affordable Automated Tool

1. Automated testing tools Saves Time and Money
i. Software tests have to be repeated often during development cycles to ensure quality.
Every time source code is modified software tests should be repeated.
ii. For each release of the software it may be tested on all supported operating systems
and hardware configurations.
iii. Manually repeating these tests is costly and time consuming.
Once created, automated tests can be run over and over again at no additional cost and
they are much faster than manual tests.
iv. Automated can reduce the time to run repetitive tests from days to hours.
v. A time savings that translates directly into cost savings.
2. Testing Improves Accuracy
i. Even the most conscientious tester will make mistakes during monotonous manual
testing.

ii. Automated tests perform the same steps precisely every time they are executed
and

never forget to record detailed results.

3. Increase Test Coverage

i. Automated can increase the depth and scope of tests to help improve software quality.
ii. Lengthy tests that are often avoided during manual testing can be run unattended.
iii. They can even be run on multiple computers with different configurations.
iv. Automated can look inside an application and see memory contents, data tables, file
contents, and internal program states to determine if the product is behaving as
expected.
v. Automated software tests can easily execute thousands of different complex test cases
during every test run providing coverage that is impossible with manual tests.
vi. Testers freed from repetitive manual tests have more time to create new automated
software tests and deal with complex features.
4. Automation Does What Manual Testing Cannot
i. Even the largest software departments cannot perform a controlled web application test
with thousands of users.
ii. Automated testing can simulate tens, hundreds or thousands of virtual users interacting
with network or web software and applications.

5. Automated QA Testing Helps Developers and Testers

i. Shared automated tests can be used by developers to catch problems quickly before
sending to QA.
ii. Tests can run automatically whenever source code changes are checked in and notify
the team or the developer if they fail.
iii. Features like these save developers time and increase their confidence.
6. Team Morale Improves
i. This is hard to measure but we‘ve experienced it first hand, automated can
improve team morale.
ii. Automating repetitive tasks with automated gives your team time to spend on
more challenging and rewarding projects.
iii. Team members improve their skill sets and confidence and, in turn, pass those
gains on to their organization.
7. Test Complete is a Powerful and Affordable Automated Tool
i. Test Complete addresses a full range of challenges facing corporate IT departments,
product developers, QA engineers, and consultants.
ii. Test Complete enhances the process by increasing efficiency, removing complexity and
lowering costs
• Helps performing "compatibility testing" -testing the software on different configurations
• It allows the tester to perform more ad-hoc (random testing)
• It gives you the ability to run automation scenarios to perform regressions in a
shorter time
• Short term costs are reduced
• It gives you the ability to run regressions on a code that is continuously changing
• The more time tester spends testing a module the greater the odds to find real user bugs
• Can be run simultaneously on different machines thus decreasing testing time
• Long term costs are reduced
5.1.6 Why Automated testing ???

Difficult to test for multi lingual sites manually

Does not require Human intervention

Increases speed of test execution

Increase Test Coverage

Manual Testing can become boring and hence error prone

Other Advantages of AT tool
1. Saves Time 14. Overcomes Failures of Manual Testing
2. Checks Quality 15. Improves ROI
16. Repeatable
3. Early Bug Detection
17. Reusable
4. Performing Tests 24/7 18. Reliable
5. Reusability 19. Quality Software
6. Distributed Test Execution 20. Cost factor
7. Easy & Robust Reporting 21. Correct Estimation
8. Testing Capabilities
9. Improves Test Coverage
10. Manpower Utilization
11. Improves Team Motivation
12. Testing Flexibility
13. Improves Accuracy
5.2.1 Advantages of using tools.
There are many benefits of using the testing tools for supporting the software testing
process. They are :
1. It Saves Time Most of the testers find problems with the time required to write long test
scripts to perform testing especially when it is regression testing.
This takes a lot of time of the testers and the delivery of the bug-free application is
delayed.(avoid late delivery)
A delayed product delivery is not good for any business.
2. Checks Quality
Once manual testing procedures are finished, if one applies automated testing processes, it
helps to cross-check the test results. Thus, improves the quality of the manual test scripts.
3. Early Bug Detection
While developing a software, bugs are easily found when software testing is performed via
automated software testing tools. This can save a lot of time and efforts on the SDLC.
4. Performing Tests 24/7
The tests are exerted without stoppage day and night. The machines and tools never get
tired. Whatever may be the quantity if testing, can be performed without any kind of human
intervention and continuously. The test results are also produced automatically after the
tests are performed.

5. Reusability
This goes perfectly very easy to understand with automated testing. When the have test
scripts are already prepared using test automation tools, they are saved for the future
requirements. So, they can be utilized as many times as the testers want especially
for automating regression testing.
6. Distributed Test Execution
Automated testing comes with distributed testing feature. One can easily execute the test
scripts on more than one computer on a shared network or server simultaneously. So,
more
than one tool is not needed, but only one automated testing tool will be needed.

7. Easy & Robust Reporting

Automation testing tools have this amazing benefit of tracking each and every test script.
Each and every test script executed can be seen in visual logs. These reports can clearly
show the no. of test scripts already executed, scheduled, their reported bugs and how they
had been fixed.

8. Testing Capabilities
When it comes to capabilities, automated testing tools can test the web applications on the
various browsers available in the market via browser testing automation.
Also, when it comes to mobile application testing, one can test them on various devices.
This is next to impossible to achieve with manual testing.
9. Improves Test Coverage
Many a time, there are test cases with more than thousand lines of code and to write it and
test it would be very difficult with manual testing. This can be easily done with automated
testing. Also, these tools will make sure about the in and out of the application like the
databases, UI, web services, etc. works according to the requirements. Thus, improving the
overall test coverage.

10. Manpower Utilization

This is relative to the above benefits. Implementing test automation in testing processes
will require less manual efforts. Thus, it will decrease the number of people on just one
particular project and can utilize them in the various testing projects.

11. Improves Team Motivation

This is a big challenge in the testing industry. When the manpower is distributed among
various testing projects, the team can learn about new challenges, new bugs, new skill sets,
Etc.

12. Testing Flexibility

The automated testing tools are developed by the teams who have been into manual testing
for years. So, these tools are going to be flexible to match the future testing specifications.
These testing tools can be utilized for longer periods of time — say for years.
13. Improves Accuracy
Here, no tester is being blamed. A tester with more than 10 years of experience can also
make mistakes when they have to prepare the same old boring manual test scripts again and
again. When this done with automated testing, not only the results are accurate, but also
saves time.

[Link] Failures of Manual Testing

There are times when accurate test results are next to impossible with manual testing, like in
the stress testing, test automation has proved to be a benefit here.

[Link] ROI
The most important benefit is the Return on Investment. Obviously, when planning to
invest in automated testing tools, first thing needed is to figure out how it will be benefited
with those tools in terms of ROI.
The cost of manual testing includes the time, cost of manual hours and the efforts of the
testers, QA managers, etc. And, if the automated testing tools are available, testing will be
faster, easily, efficiently, accurately and would be delivering bug-free application within the
delivery time period.
[Link]
Execution of the same set of automation test cases is possible. That‘s too without any
human intervention and faster.

[Link]
Re-use of the same set of automation test cases on a different version of an
application is possible. Also the same version of an application on a different
browser can be tested with the same test cases.

[Link]
Automation is nothing but the program. If it is written as per expectation, then there are no
chance automation program will do anything which is not written. If any program is in
working condition, it will work with the same efficiency for the long time.

[Link] Software
Automation can allow us to spend more time to spend on adding the new features and
improving the quality of the software. Instead of wasting the time on testing manually, one
can think of increasing the grade of the software.
[Link] Factor
Automation in testing can replace the manual repeated tasks. Ultimately it will
reduce the costing of the project.
[Link] Estimation
After some iteration of Automation testing, person would be aware of the exact time
required for the execution. So the correct estimate with more accuracy can be given .
5.2.2 Disadvantages of using testing tools.

1. Proficiency required
2. Debugging issues
3. Test maintenance is costly
4. Maintainence of test data is issue
5. 100% test automation is not possible
6. All types of Testing not possible
7. Tools may have their own defect, limitation
8. Wrong expectation
9. Not a replacement to actual Testing
10. Programming Knowledge is required.
11. Test Tools have Environment Limitations.
12. Not suitable for dynamically changing UI designs.
13. Results False Sense of quality
14. Maintenance Time
15. Less Number of Bugs Identified
5.2.2 Disadvantages of using testing tools.

Though the automation testing has many advantages, it has its own disadvantages too.
Some of the disadvantages are:
1. Proficiency is required to write the automation test scripts.
2. Debugging the test script is major issue.
If any error is present in the test script, sometimes it may lead to deadly consequences.
3. Test maintenance is costly in case of playback methods.
Even though a minor change occurs in the GUI, the test script has to be rerecorded or
replaced by a new test script.
4. Maintenance of test data files is difficult, if the test script tests more screens.
5. Test Automation requires lot of efforts at initial stage.
In Software Testing two important tasks, one is Test Design and another is Test Execution,
For Test Design User (Tester) interaction is mandatory, Testers only create Test scripts
using Test Tool features and Programming features, It takes more time than Manual Test
Case Design.
6. 100% Test automation is impractical.
Generally it is tried to automate maximum Test cases, not all Test cases. For some Test
human user observation is required and is must. Due to some Environment limitations we
can‘t automate all testable requirements.
7. All types of Testing not possible (Ex: Usability).
Functionality tests, Performance Tests can be automated, but it is not possible to
automate
tests that verify user friendliness of the system(AUT).
8. Debugging issues
Programming syntax/logic is used to write Tests, some times locating errors in Test Script
is difficult. It is difficult to write such test cases.
9. Tools may have their own defects.
Test tool is also a Software, it may have its own defects in it, so that we may not achieve
desired benefits.
10. Programming Knowledge is required.
Every Test Tool uses any one of the Programming languages (Example UFT supports
VBScript, Selenium supports Java, Perl, PHP, C#, PHP and Ruby) to write Test scripts. So
in order to create and edit Test Scripts Programming knowledge is mandatory.
In Manual Testing, no programming knowledge is required.

11. Test Tools have Environment Limitations.

Test Tools have some compatibility issues with Operating Systems and browsers etc
example:
QTP Supports Windows operating environment only, doesn‘t support other operating
environments like UNIX, Macintosh etc...
Selenium Supports Web Application test automation only, doesn‘t support Desktop /
windows based applications.
For Manual Testing no environment limitations, we can test computer software or mobile
software on any operating system and any Browser.)

12. Not suitable for dynamically changing UI designs.

Most of the Test Tools support Test automation based on front-end objects, if User
Interface design changes dynamically then it is difficult to automate.
[Link] expectations
Team keeps a lot of expectation from the automation tool. We need to understand it is
just a program which will do what it is asked to do.

[Link] Costing
In initial days of framework development and converting manual test case in
automation test cases takes time. And as this is the first time testing team
implementing the automation, this can lead the inaccurate estimation.

[Link] False Sense of quality

Automation program will do only things which have been developed. As automation
does not have its own brain to sense the defect and test accordingly.

16. Not a replacement of Actual Testing

It is not the replacement of Actual testing, Manual testing can be more interactive
and innovative while testing. Innovation from the automation while execution cannot
be expected.
17. Maintenance Time
If an application changes, it is needed to spend separate time to update the
automation scripts.

[Link] Number of Bugs Identified

Automation will execute same test cases again and again, so over the period of time
Automation will become less effective.
Some of the above disadvantages often cause damage to the benefit gained from the
automated scripts. Though the automation testing has pros and corns, it is adapted widely
all over the world.
Guidelines for Static and Dynamic Testing Tools.
5.2.3 Features of Test Tool: Guidelines for Static and
Dynamic Testing Tools.

Test tools:
Software testing tools : Tools from a software testing context can be defined as a product
that supports one or more test activities right from planning, requirements, creating a build,
test execution, defect logging and test analysis.
Classification of Tools
Tools can be classified based on several parameters. They include:
The purpose of the tool
The Activities that are supported within the tool
The Type/level of testing it supports
The Kind of licensing (open source, freeware, commercial)
The technology used
. Static Testing Tool
. Dynamic Testing Tool
1. Static Testing Tool
i. Static analysis tools are generally used by developers as part of the development and
component testing process.
The key aspect is that the code (or other artefact) is not executed
or run but the tool itself is executed, and the source code we are interested in is the input
data to the tool.

ii. These tools are mostly used by developers.

iii. Static analysis tools are an extension of compiler technology – in fact some compilers do
offer static analysis features. It is worth checking what is available from existing compilers
or development environments before looking at purchasing a more sophisticated static
analysis tool.

iv. Other than software code, static analysis can also be carried out on things like, static
analysis of requirements or static analysis of websites (for example, to assess for proper use
of accessibility tags or the following of HTML standards).
Features or characteristics of static analysis tools are:
To calculate metrics such as cyclomatic complexity or nesting levels
(which can help to identify where more testing may be needed due to
increased risk).
To enforce coding standards.
To analyze structures and dependencies.
Help in code understanding.
To identify anomalies or defects in the code.
 2. Dynamic Testing Tool
i. Dynamic analysis tools are ̳dynamic‘ because they require the code to be in a running state.
They are ̳analysis‘ rather than̳ testing‘ tools because they analyze what is happening behind the
scenes‘ that is in the code while the software is running (whether being executed with test
cases or being used in operation).

ii. Let us take an example of a car to understand it in a better way. If you go to a showroom of a
car to buy it, you might sit in the car to see if is comfortable and see what sound the doors make
– this would be static analysis because the car is not being driven.
If you take a test drive, then you would check that how the car performs when it is in the
running mode
e.g. the car turns right when you turn the steering wheel clockwise or when you press the break
then how the car will respond and can also check the oil pressure or the brake fluid,
this would be dynamic analysis, it can only be done while the engine is running.
Features or characteristics of dynamic analysis tools are as follows:
i. To detect memory leaks
([Link]
ays-diagnose-using-onstat-commands)
ii. To identify pointer arithmetic errors such as null pointers;
iii. To identify time dependencies.
Eventually when your computer‘s response time gets slower and slower, but it
get improved after rebooting, this may be because of the ̳memory leak‘,
where the programs do not correctly release blocks of memory back to the
operating system.
Sooner or later the system will run out of memory completely and stop.
Hence,rebooting restores all of the memory that was lost, so the performance
of the system is now restored to its normal state.
These tools would typically be used by developers in component testing and
component integration testing, e.g. when testing middleware, when testing
security or when looking for robustness defects.
Dynamic analysis tool…

Another form of dynamic analysis for websites is to check

whether each link does actually link to something else (this
type of tool may be called a ̳web spider‘).
The tool does not know if you have linked to the correct page,
but at least it can find dead links, which may be helpful.
(A dead link, also known as a broken link, is a hyperlink on a web page that no longer works
because it's pointing to a webpage or resource that has been moved, deleted, or is otherwise
inaccessible.)
Static test tools
a. Flow analyzers
b. Coverage analyzers
c. Interface analyzer
d. Path tests
Similarly, data-profiling tools can be used to optimize a database.
Code-profiling tools can be used to optimize code.
Test- generators are used for generating a test plan form code.
Syntax-checking tools are used to verify correctness of code.
Static testing tools are used during static analysis of a system.
They do not involve actual input and output.

Static testing tools: are used throughout a software development life cycle, e.g. tools used
for verification purposes.
• There are many varieties of static testing tools used by different people as per the type of
system being developed.

• These tools do not involve actual input and output. Rather, they take a symbolic approach
to testing, i.e. they do not test the actual execution of the software. E.g.
a. Flow analyzers : ensure the consistency in data flow from input and output.
b. Coverage analyzers : It ensures that all logic paths are tested.
c. Interface analyzer : It examines the effects of passing variables and data
between modules.
d. Path tests: They find unused code and code with contradictions.
• Code complexity measurement tools can be used to measure the complexity of a given
code.
Similarly, data-profiling tools can be used to optimize a database.
Code-profiling tools can be used to optimize code.
Test- generators are used for generating a test plan form code.
Syntax-checking tools are used to verify correctness of code.
2. Dynamic test tools

a. Test driver
b. Test beds
c. Emulators
d. Mutation analysers
Other
Regression testing using automated tools.
Defect tracking and communication systems used by tracking & communication.
Performance, Load, stress-testing tools.
Dynamic testing tools are used at different levels of testing starting from unit testing &
which may go up to system testing & performance testing.
• These tools are generally used by tester.
• These tools test the software system with live data. e.g.
a. Test driver: It inputs data into a module –under-test (MUT)
b. Test beds : It simultaneously displays source code along with the program
under execution.
c. Emulators : The response facilities are used to emulate parts of the system not yet
developed.
d. Mutation analysers : The errors are deliberately fed into the code in order to
test fault tolerance of the system.
There are many different tools used for dynamic testing.
• Some of the areas covered by testing tools are:
1. Regression testing using automated tools.
2. Defect tracking and communication systems used by tracking & communication.
Performance, Load, stress-testing tools.
5.3 Selecting a testing tool

Automation testing success largely depends on the selection of right testing

tools.
It takes a lot of time to evaluate relevant automation tools available in the
market.
But this is a must one- time exercise that will benefit your project in long run.
Here are the criteria you need to consider before selecting any testing tool:
 Test Automation Tool Evaluation Criteria….1
1) Do you have the necessary skilled resource to allocate for automation tasks?
2) What is your budget?
3) Does the tool satisfy your testing needs? Is it suitable for the project environment and
technology you are using? Does it support all tools and objects used in the code?
Sometime you may get stuck for small tests due to inabilities of the tool to identify the
objects used in the application.
above three factors are considered as most important for selecting any tool. The factors
include :
4) Does the tool provide you the free trial version so that you can evaluate it before
making a decision?
Also, does the tool have all features available in the trial version?
5) Is the current tool version stable? Is the vendor company established with good customer
support as well as online help resources and user manual?
6) How is the tool learning curve? Is the learning time acceptable for your goals?
7) Do you want automation tool for only your project needs or you are looking for a
common tool for all projects in your company? It would be a good choice if you select a
tool that supports most of the coding languages on your projects.
8 ) Which testing types does it support? A tool which supports maximum testing types
(Unit, functional, regression etc.) is always a better choice. Don‘t go for a tool just because
it is supporting all testing types. It‘s also important that the tool should be powerful enough
to automate your complex requirements.
Test Automation Tool Evaluation Criteria….2
9) Does the tool support easy interface to create and maintain test scripts? Record and
playback tool with abilities to edit recorded scripts could be a good solution.
10) Does it provide simple interface yet powerful features to accomplish complex tasks?
11) How easy is it to provide input test data for complex or load tests? A tool supporting
test data input from various data files such as Excel, XML, text file etc. would be a big
relief for the automation the testers.
12) Does it provide the powerful reporting with graphical interface? Clear and concise
reports will always help you to conclude the test results quickly.
Test Automation Tool Evaluation Criteria….3
13) Does it integrate well with your other testing tools like project planning and test
management tools?
You may also want to consider other criteria like:
14) Tool vendor refund policy
15) Existing customer reviews for the tool
16) Is the vendor providing initial training?
i. While introducing the tool in the organization it must match a need within the
organization, and solve that need in a way that is both effective and efficient.
ii. The tool should help in building the strengths of the organization and should also
address its weaknesses.
The organization needs to be ready for the changes that will come along with the
new tool.
i. If the current testing practices are not good enough and the organization is not
mature,then it is always recommended to improve testing practices first rather
than to try to find tools to support poor practices.
Certainly, we can sometimes improve our own processes in parallel with
introducing a tool to support those practices and we can always pick up some
good ideas for improvement from the ways that the tools work.
ii. Do not depend on the tool for everything, but it should provide support
to your organization as expected.
The following factors are important during tool selection:

i. Assessment of the organization‘s maturity (e.g. readiness for change);

ii. Identification of the areas within the organization where tool support will help
to improve testing processes;
iii. Evaluation of tools against clear requirements and objective criteria;
iv. Proof-of-concept to see whether the product works as desired and meets the
requirements and objectives defined for it;
v. Evaluation of the vendor (training, support and other commercial aspects) or
open-source network of support;
vi. Identifying and planning internal implementation (including coaching and
mentoring for those new to the use of the tool).
5.4 When to Use Automated Test Tools, Testing Using Automated Tool.

5.4.1. When Does Test Automation Make Sense?

When are Automated Test Suitable and when not?

Automated tests are a very useful and impressive tool used to make testing more
efficient.
However, automated tests are not suited to all projects – this may be due to lack of
time available, or due to technical limitations.
Automated tests take time to create.
Depending on the testers, it can take 3-10 times the amount of time to create an
automated test as it takes to run the same test manually.
Therefore, automated tests will only start to be valuable when they are run more
than 3-10 times.
Automated tests are suitable for the following purposes:

– Regression testing for a stable system that will be run on a regular basis
– Fast data creation in test systems where the database must be wiped on a regular basis
i. When there are many repetitive tests.
ii. When there are frequent regression testing iterations.
iii. When you need to simulate large number of users who are using the application
resources.
iv. When AUT(Application under test) is having comparatively stable UI.
v. When you have large set of BVT cases(Build Verification Testing).
(BVT, is a set of tests run on every new build to check if the build is stable and
ready for further testing [Link]
vi. When you can‘t rely solely on manual test execution for critical functionality
Automated tests are NOT suitable for the following purposes:

– Testing new functionality – this should be done manually before automated tests are
created
– Regression testing systems that are expected to have significant user interface changes.
Large changes to the user interface require a lot of maintenance for automated tests.
When automating tests, it is wise to only automate as many tests as your team can
easily Maintain.
If some tests are becoming difficult to maintain, it may be worth considering retiring
those tests.
Above all, remember that automated tests will never detect as many bugs as a human
tester executing the same steps.
This is because the human tester‘s eyes can pick up many things, whereas the test will
only notice what it is programmed to notice.
5.4.2 Testing Using Automated Tools (Test Automation)
What is Test Automation?
Software Test automation makes use of specialized tools to control the execution of tests and
compares the actual results against the expected result.
Usually, regression tests, which are repetitive actions, are automated.
Testing Tools not only helps us to perform regression tests but also helps us to automate data set up
generation, product installation, GUI interaction, defect logging, etc.
Automation tools are used for both Functional and Non-Functional testing.
Criteria for Tool Selection:
For automating any application, the following parameters should be considered:
● Data driven capabilities
● Debugging and logging capabilities
● Platform independence
● Extensibility & Customizability
● E-mail Notifications
● Version control friendly
● Support unattended test runs
Testing using automated tools
1. Partial Automation
2. Framework Approach in Automation
1. Partial Automation:
The important aspect of test automation is the idea of partial test automation, or
automating parts of testing, but not all of the software-testing process.
In software testing, there are many processes that do not fit the need of complete
automation.
Partial automation allows the organizations to observe the benefits of automation without
the involvement of too much cost and time.
Testing tools can help to automate repetitive (iteration) tasks such as product installation,
test data creation, GUI interaction problem detection, defect logging, etc. without
necessarily automating tests in an end-to-end fashion.
This can help software testers to concentrate on more important parts and automated
parts are used repetitively where human intelligence is not required.
Conditions for Partial Test Automation :
There may be situations where a tool may pose certain limitations due to which it
may be difficult to carry out testing.
Some of the cases are :
Tool does not have the sufficient options/methods for performing automation
Sometimes automating certain cases may be more time consuming than performing
manual test.
Hence one may resort to perform manual testing in such a scenario.
2. Framework Approach in Automation
A test automation framework is an integrated system that sets the rules of automation of
a specific product.
This system integrates the function libraries, test data sources, object details and various
reusable modules.
These components act as small building blocks which need to be assembled to represent a
business process.
The framework provides the basis of test automation and simplifies the
automation effort.
The main advantage of a framework of assumptions, concepts and tools that provide
support for automated software testing is the low cost for maintenance.
If there is change to any test case then only the test case file needs to be updated and the
driver Script and startup script will remain the same. Ideally, there is no need to update
the scripts in case of changes to the application.