5.

1 MANUAL TESTING
Testing Computer Software manually without using any Automation Tools

Q] State any eight limitations of manual testing. 4M

1. Manual testing is slow and costly.

2. It is very labor intensive; it takes a long time to complete tests.
3. Manual tests don’t scale well. As the complexity of the software increases the complexity of the
testing problem grows exponentially. This leads to an increase in total time devoted to testing as well as
total cost of testing.
4. Manual testing is not consistent or repeatable. Variations in how the tests are performed as
inevitable, for various reasons. One tester may approach and perform a certain test differently from
another, resulting in different results on the same test, because the tests are not being performed
identically.
5. Lack of training is the common problem.
6. GUI objects size difference and color combinations are not easy to find in manual testing.
7. Not suitable for large scale projects and time bound projects.
8. Batch testing is not possible, for each and every test execution Human user interaction is mandatory.
9. Comparing large amount of data is impractical.
10. Processing change requests during software maintenance takes more time.

5.2 AUTOMATION TESTING

Q] What is software test automation? State types of test automation tools.
Q] What is automated testing? Write down advantages of using automated testing tools in
software testing.

Test automation is the use of special software tools to control the execution of tests comparison of
actual outcomes with predicted outcomes.
The objective of automated testing is to simplify as much of the testing effort as possible with a
minimum set of scripts.
Test automation can automate some repetitive but necessary tasks in a formalized testing process
already in place, or add additional testing that would be difficult to perform manually.

A] ADVANTAGES AND DISADVANTAGES OF USING AUTOMATED

TOOLS
Q] What are the advantages and disadvantages of using automated tools for testing? List any two
each.
Q] Elaborate the advantages (any four) of using the test automation tools.
Q] State any four advantages of using tools.
Benefits of Automation Testing / Automated Tools are:

1. Save Time /Speed:

Software tests have to be repeated often during development cycles to ensure quality. Every time
source code is modified software tests should be repeated.
Manually repeating these tests is costly and time consuming. Once created, automated tests can be run
over and over again at no additional cost and they are much faster than manual tests.

2. Reduces the tester’s involvement in executing tests: While testers are busy running test cases,
testers can't be doing anything else. If the tester have a test tool that reduces the time it takes for him to
run his tests, he has more time for test planning and thinking up new tests. It increases efficiency of
tester.

3. Repeatability/Consistency: The same tests can be re-run in exactly the same manner eliminating
the risk of human errors such as testers forgetting their exact actions, intentionally omitting steps from
the test scripts, missing out steps from the test script, all of which can result in either defects not being
identified or the reportig of invalid bugs (which can again, be time consuming for both developers and
testers to reproduce)

4. Simulated Testing: Automated tools can create many concurrent virtual users/data and effectively
test the project in the test environment before releasing the product.

5. Test case design: Automated tools can be used to design test cases also.
Through automation, better coverage can be guaranteed than if done manually.

6. Reusable: The automated tests can be reused on different versions of the software, even if the
interface changes.

7. Avoids human mistakes: Manually executing the test cases may incorporate errors. But this can be
avoided in automation testing.

8. Internal Testing: Testing may require testing for memory leakage or checking the coverage of
testing. Automation can done this easily.
9. Cost Reduction: If testing time increases cost of the software also increases. Due to testing tools time
and therefore cost is reduced.

Disadvantages of using automated tools for testing are as follows:

1. It's more expensive to automate. Initial investments are bigger than manual testing
2. High package development investment costs.
3. High manpower requirements for test preparation.
4. Considerable testing areas left uncovered.
5. You cannot automate everything; some tests still have to be done manually.
[Link] cannot rely on testing tools always.
Manual Testing vs Automated Testing
Manual testing Automated testing
For observation manual testing allow human For observation automation testing does not
factor. allow human factor.
Manual Testing is not accurate at all times due Automated testing is more reliable, as it is
to human error, hence it is less reliable performed by tools and/or scripts.
It is time consuming, because it is done by It is executed by software tools, so it is
humans significantly faster than a manual approach
In manual testing investment is required for In automation testing investment is required
humans resources because here humans test for automation tools because here testing is
the software instead of any software tools. done by some software tools.
It is only practical when the test cases are run
It is a practical option when the test cases are
once or twice, and frequent repetition is not
run repeatedly over a long time period.
required
It allows for human observation, which may be It does not entail human observation and
more useful if the goal is user-friendliness or cannot guarantee user-friendliness or positive
improved customer experience. customer experience.
It allows the tester to perform more ad-hoc Helps performing "compatibility testing" -
(random testing) testing the software on different configurations

5.3] SELECTING A TESTING TOOL

Q] Enlist factors considered for selecting a testing tool for test automation.
Q] List what are the different guidelines to be followed while selecting dynamic test tools.

The industry experts have suggested following four major criteria for selection of testing tools.
1) Meeting requirements.
2) Technology expectations.
3) Training / skills.
4) Management aspects.

1) Meeting Requirements:
There are many tools available in the market today but rarely do they meet all the requirements of
given product or a given organization. Evaluating different tools for different requirements involves lot
of effort, money and time. Huge delay is involved in selecting and implanting test tools.
A number of test tools cannot distinguish between a product failure and a test failure. This increases
analysis time and manual testing.

2) Technology Expectations:
In general, test tools may not allow test developers to extend / modify the functionality of the
framework. So, it involves going back to the tool vendor with additional cost and effort. Very few tools
available in market provide source code for extending functionality or fixing some problems.
Extensibility and customization are important expectations of a test tool.

3) Training Skills:
Test tools require plenty of training, but very few vendors provide the training to the required level.
Organization-level training is needed to deploy the test tools, as the users of the test suite are not only
the test team but also the development team and other areas like SCM (Software Configuration
Management). Test tools expect the users to learn new language/scripts and may not use standard
languages/scripts. This increases skill requirements for automation and increases the need for a
learning curve inside the organization.

4) Management Aspects:
A test tool increases the system requirement and requires the hardware and software to be upgraded.
This increases the cost of the already-expensive test tool. When selecting the test tool, it is important to
note the system requirements and the cost involved in upgrading the software and hardware needs to
be included with the cost of the tool.

OR

Guidelines for selecting a tool:

1. The tool must match its intended use. Wrong selection of a tool can lead to problems like lower
efficiency and effectiveness of testing may be lost.
2. Different phases of a life cycle have different quality-factor requirements. Tools required at each
stage may differ significantly.
3. Matching a tool with the skills of testers is also essential. If the testers do not have proper training
and skill then they may not be able to work effectively.
4. Select affordable tools. Cost and benefits of various tools must be compared before making final
decision.
5. Backdoor entry of tools must be prevented. Unauthorized entry results into failure of tool and creates
a negative environment for new tool introduction.

5.4 WHEN TO USE AUTOMMATED TESTING TOOLS

Q] Which types of test are first candidates for test automation? Why?

Stress, reliability, scalability and performance testing: These types of testing require the test case
to be run from a large number of different machines for an extended period of time, such as 24 hours,
48 hours, and so on. It is just not possible to have hundreds of users trying out the product they may be
not willing to perform the repetitive tasks, nor will it be possible to find that many people with the
required skill sets. Test cases belonging to these testing types become the first candidates for
automation.

Regression tests: Regression tests are repetitive in nature .These test cases are executed multiple
times during the product development phase. Given the repetitive nature of test cases, automation will
save significant time and effort in the long run. The time thus gained can be effectively utilized for other
tests.

Functional tests: These kinds of tests may require a complex set up and thus require specialized skill,
which may not be available on an ongoing basis. Automating these once, using the expert skill sets, can
enable using less-skilled people to run these test on an ongoing basis.
5.5 TYPES OF TOOLS
Q] What is the difference between static & dynamic testing tool?
Q] Explain static and dynamic testing tools.

1. Static Testing tools

Static testing tools are used during static analysis of a system. Static testing tools are used throughout a
software development life cycle, e.g , tools used for verification purposes.
Generally used by developers as part of development and component testing process, here code is not
executed or run but the tool itself is executed and source code we are interested in is the input data to
the tool.
Code complexity measurement tools can be used to measure the complexity of a given code.
Similarly, data-profiling tools can be used to optimize a database.
Code-profiling tools can be used to optimize code. Test-generators are used for generating a test plan
form code.
Syntax-checking tools are used to verify correctness of code.

Features / Characteristics of static analysis tools are:

 To calculate metrics, Cyclomatic complexity or nesting levels.
 To enforce coding standards
 To analyze structures and dependencies
 Help in code understanding
 To identify anomalies or defects in the code.
Examples:
 Flow analyzer
 path tests
 coverage analyzers
 Interface analyzers

2. Dynamic Testing tools

Dynamic testing tools are used at different levels of testing starting from unit testing & which may go
up to system testing & performance testing. These tools are generally used by tester.
Some of the areas covered by testing tools are:
1. Regression testing using automated tools.
2. Defect tracking and communication systems used by tracking & communication.
They require the code to be in running state They analyze rather than testing.

Features / Characteristics of dynamic analysis tools are:

 To detect memory leaks

 To identify pointer arithmetic errors such as null pointers
 To identify time dependencies.

Examples of Dynamic testing tools available:

a) Test Driver
b) Test Beds
c) Emulators
d) Mutation analyzers
Q] List what are the different guidelines to be followed while selecting dynamic test tools.
Q] Which are features for selecting static test tools? Also list any two available test tools (static).

Features for selecting static test tools:

i) Assessment of the organization‘s maturity (e.g. readiness for change);

ii) Identification of the areas within the organization where tool support will help to improve testing
processes;
iii) Evaluation of tools against clear requirements and objective criteria;
iv) Proof-of-concept to see whether the product works as desired and meets the requirements and
objectives defined for it;
v) Evaluation of the vendor (training, support and other commercial aspects) or open-source network
of support;
vi) Identifying and planning internal implementation (including coaching and mentoring for those new
to the use of the tool).
5.6 Metrics and measurement
Q] Elaborate the term metrics and measurement and write the need of software measurement.
Q] Define software metrics. Describe product Vs process & objective Vs subjective metrics.

A Metric is a measurement of the degree that any attribute belongs to a system, product or process.
For example the number of errors per person hours would be a metric.
Thus, software measurement gives rise to software metrics.
A measurement is an indication of the size, quantity, amount or dimension of a particular attribute of a
product or process. For example the number of errors in a system is a measurement.
A Metric is a quantitative measure of the degree to which a system, system component, or process
possesses a given attribute.
Metrics can be defined as “STANDARDS OF MEASUREMENT”.
Software Metrics are used to measure the quality of the project.
Simply, Metric is a unit used for describing an attribute. Metric is a scale for measurement.

Need of Software measurement:

1. Establish the quality of the current product or process.

2. To predict future qualities of the product or process.
3. To improve the quality of a product or process.
4. To determine the state of the project in relation to budget and schedule.
Types of Metrics
 Process quality
 Product quality
 Objective Metrics
 Subjective Metrics

1. Process quality: Activities related to the production of software, tasks or milestones.

1. Process metrics are collected across all projects and over long periods of time.
2. They are used for making strategic decisions.
3. The intent is to provide a set of process indicators that lead to long-term software process
improvement.
4. The only way to know how/where to improve any process is to:
 Measure specific attributes of the process.
 Develop a set of meaningful metrics based on these attributes.
 Use the metrics to provide indicators that will lead to a strategy for improvement.

2. Product quality: Explicit result of the software development activity, deliverables, products.
1. Product metrics help software engineers to better understand the attributes of models and assess the
quality of the software.
2. They help software engineers to gain insight into the design and construction of the software.
3. Focus on specific attributes of software engineering work products resulting from analysis, design,
coding, and testing.
4. Provide a systematic way to assess quality based on a set of clearly defined rules.
5. Provide an “on-the-spot” rather than “after-the-fact” insight into the software development.

3. Objective Metrics:
1. They are non-negotiable – that is the way they are defined doesn’t change with respect the type of
endeavor they are being applied to.
2. Actual cost or AC is always the total cost actually incurred in accomplishing a certain activity or a
sequence of activities.

4. Subjective Metrics:
These metrics are a relatively new precept and are more flexible than the rigid framework of the
objective metrics. Subjective metrics do deal with performance but the approach is more tailored. For
some enterprises the niche in which they function forces project management to change in order to
adapt to the demands of the workplace.
  PRODUCT METRICS

Q] Explain three types of product metrics.

Product Metrics is classified as :
1. Project Metrics: A set of metrics that indicates how the project is planned and executed.
2. Progress Metrics: A set of metrics that tracks how the different activities of the project are
progressing. It includes both development activities and testing activities. Progress
Metrics is classified as a. Test defect metrics b. Development defect metrics
a. Test defect metrics: help the testing team in analysis of product quality and testing
b. Development defect metrics: help the development team in analysis of development activities.
3. Productivity Metrics: A set of metrics that takes into account various productivity numbers that can
be collected and used for planning and tracking testing activities. These metrics help in planning and
estimating of testing activities.

 OBJECT-ORIENTED METRICS AND MEASURES

Q] Describe object oriented metrics in testing.

As object-oriented approach emerged to support major applications, the effectiveness of applying
traditional software metrics to object-oriented systems was challenged. The object-oriented design
approach gives opportunity to classify metrics naturally.
The classification captures object-oriented software features and properties hierarchically. It begins
with the high-level characteristics of an object-oriented system and moves down to the low-level
characteristics.

Source code size metrics: Traditional metrics which are applied to object oriented software give
insight into an overall system size and allow comparing systems and evaluating productivity. They can
also be used as a refactoring effectiveness indicator.

Lines of Code (LOC) metric is most common software project measure. The metric becomes a baseline
to measure the degree of work performed on a project and it is used to create time and cost estimates.

Effective Lines of Code Metric (eLOC) is a measure of all lines that are not comments, blanks or
standalone braces or parenthesis. This metric more closely represents the quantity of work performed.

Comment Line and Comment Percent (or Comment to Code Ratio) is a degree of commenting within
the source code. It measures the care taken by programmers to make the source code and algorithms
understandable. Poorly commented code makes the maintenance activities an extremely expensive.
Recommended minimum is 20%.
Blank Line and White Space Percent Metric is the number of blank lines within source code. It
indicates the readability of product. And File Count Metric counts the files processed and generates
metrics based on the file extension. It provides the distribution of the source code types, source code
types and distribution of the specifications to the implementations.

Procedural metrics: Cyclomatic Complexity is a popular procedural (called also function) software
metric equal to the number of decisions that can be taken in a procedure A decision is defined as an
occurrence of keywords such as: "while", "for", “for each”, “continue”, "if", "case", "go to", “try” and
“catch” within the function. Cyclomatic Complexity is the sum of these constructs.
That metric helps to identify software need of inspection or redesign, and also to allocate resources for
evaluation and test.
Class metrics: Class metrics describe structure of a class and relationship between classes. The volume
of a class is a basic size measure connected with the amount of information inside it. The class volume
can be measured by Number of Variables and by Number of Methods. Also Average LOC per Class and
per Method metrics can provide insight into the average module size in the system.

Method metrics are used to estimate effort for testing early. Those metrics can be measured by
Number of Parameters per Method, Weighted Methods per Class, Maximum Nesting Level, and Method
Rank. Number of Parameter per Method counts parameters of a method and also references.

Afferent Coupling and Efferent Coupling at method level are another object coupling metrics.
Afferent Coupling for a particular method is the number of methods that depends directly on it and the
Efferent Coupling for a particular method is the number of methods it directly depends on. Afferent
Coupling is an indicator for the responsibility. The higher this value is the higher is the element’s
responsibility. Efferent Coupling means that a element depends on several other implementation
details and it makes it instable.
Therefore it is good practice to keep the Efferent Coupling for all artefacts at a minimum.

Inheritance metrics: The inheritance relationships characteristic between classes and their parents
indicate to a designer where changes would improve the development. The metrics connected to
classes inheritance should take into account both the depth and breadth of the relationships. The
Height of Inheritance Tree metric is counted as the maximum number of nodes from the class node to
the root of the inheritance hierarchy. The deeper within the hierarchy, the more methods the class can
inherit, increasing its complexity.