TESTING STRATEGIES

Introduction to Testing:

 Testing is a set of activities which are decided in advance i.e before the start of development and
organized systematically.
 In the literature of software engineering various testing strategies to implement the testing are defined.
 All the strategies give a testing template.

Following are the characteristic that process the testing templates:

 The developer should conduct the successful technical reviews to perform the testing successful.
 Testing starts with the component level and work from outside toward the integration of the whole
computer based system.
 Different testing techniques are suitable at different point in time.
 Testing is organized by the developer of the software and by an independent test group.
 Debugging and testing are different activities, then also the debugging should be accommodated in any
strategy of testing.

Difference between Verification and Validation

Verification Validation
Verification is the process to find The validation process is checked
whether the software meets the whether the software meets
specified requirements for particular requirements and expectation of
phase. the customer.
It estimates an intermediate product. It estimates the final product.
The objectives of verification is to The objectives of the validation is
check whether software is to check whether the
constructed according to requirement specifications are correct and
and design specification. satisfy the business need.
It describes whether the outputs are It explains whether they are
as per the inputs or not. accepted by the user or not.
Verification is done before the It is done after the verification.
validation.
Plans, requirement, specification, Actual product or software is
code are evaluated during the tested under validation.
verifications.
It manually checks the files and It is a computer software or
document. developed program based
checking of files and document.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Software Testing Overview

Software Testing is evaluation of the software against requirements gathered from users and system
specifications. Testing is conducted at the phase level in software development life cycle or at module
level in program code. Software testing comprises of Validation and Verification.

Software Validation

Validation is process of examining whether or not the software satisfies the user requirements. It is
carried out at the end of the SDLC. If the software matches requirements for which it was made, it is
validated.

 Validation ensures the product under development is as per the user requirements.

 Validation answers the question – "Are we developing the product which attempts all that user needs
from this software ?".

 Validation emphasizes on user requirements.

Software Verification

Verification is the process of confirming if the software is meeting the business requirements, and is
developed adhering to the proper specifications and methodologies.

 Verification ensures the product being developed is according to design specifications.

 Verification answers the question– "Are we developing this product by firmly following all design
specifications ?"

 Verifications concentrates on the design and system specifications.

Target of the test are -

 Errors - These are actual coding mistakes made by developers. In addition, there is a difference in
output of software and desired output, is considered as an error.

 Fault - When error exists fault occurs. A fault, also known as a bug, is a result of an error which can
cause system to fail.

 Failure - failure is said to be the inability of the system to perform the desired task. Failure occurs when
fault exists in the system.

Manual Vs Automated Testing

Testing can either be done manually or using an automated testing tool:
 Manual - This testing is performed without taking help of automated testing tools. The software tester
prepares test cases for different sections and levels of the code, executes the tests and reports the result
to the manager.
Manual testing is time and resource consuming. The tester needs to confirm whether or not right test
cases are used. Major portion of testing involves manual testing.
 Automated This testing is a testing procedure done with aid of automated testing tools. The limitations
with manual testing can be overcome using automated test tools.
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 A test needs to check if a webpage can be opened in Internet Explorer. This can be easily done with
manual testing. But to check if the web-server can take the load of 1 million users, it is quite impossible
to test manually.
There are software and hardware tools which helps tester in conducting load testing, stress testing,
regression testing.

Strategy of Testing:

A strategy of software testing is shown in the context of spiral.

Following figure shows the testing strategy:

Software Testing Strategies

Software Testing is a type of investigation to find out if there is any default or error present in the
software so that the errors can be reduced or removed to increase the quality of the software and to
check whether it fulfills the specifies requirements or not.
According to Glen Myers, software testing has the following objectives:
 The process of investigating and checking a program to find whether there is an error or not and does it
fulfill the requirements or not is called testing.
 When the number of errors found during the testing is high, it indicates that the testing was good and is a
sign of good test case.
 Finding an unknown error that wasn’t discovered yet is a sign of a successful and a good test case.
The main objective of software testing is to design the tests in such a way that it systematically finds
different types of errors without taking much time and effort so that less time is required for the
development of the software.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 The overall strategy for testing software includes:
1. Before testing starts, it’s necessary to identify and specify the requirements of the product in a
quantifiable manner. Different characteristics quality of the software is there such as maintainability
that means the ability to update and modify, the probability that means to find and estimate any risk, and
usability that means how it can easily be used by the customers or end-users. All these characteristic
qualities should be specified in a particular order to obtain clear test results without any error.
2. Specifying the objectives of testing in a clear and detailed manner. Several objectives of testing are
there such as effectiveness that means how effectively the software can achieve the target, any failure
that means inability to fulfill the requirements and perform functions, and the cost of defects or errors
that mean the cost required to fix the error. All these objectives should be clearly mentioned in the test
plan.
3. For the software, identifying the user’s category and developing a profile for each user. Use cases
describe the interactions and communication among different classes of users and the system to achieve
the target. So as to identify the actual requirement of the users and then testing the actual use of the
product.
4. Developing a test plan to give value and focus on rapid-cycle testing. Rapid Cycle Testing is a type
of test that improves quality by identifying and measuring the any changes that need to be required for
improving the process of software. Therefore, a test plan is an important and effective document that
helps the tester to perform rapid cycle testing.
5. Robust software is developed that is designed to test itself. The software should be capable of
detecting or identifying different classes of errors. Moreover, software design should allow automated
and regression testing which tests the software to find out if there is any adverse or side effect on the
features of software due to any change in code or program.
6. Before testing, using effective formal reviews as a filter. Formal technical reviews is technique to
identify the errors that are not discovered yet. The effective technical reviews conducted before testing
reduces a significant amount of testing efforts and time duration required for testing software so that the
overall development time of software is reduced.
7. Conduct formal technical reviews to evaluate the nature, quality or ability of the test strategy and
test cases. The formal technical review helps in detecting any unfilled gap in the testing approach.
Hence, it is necessary to evaluate the ability and quality of the test strategy and test cases by technical
reviewers to improve the quality of software.
8. For the testing process, developing a approach for the continuous development. As a part of a
statistical process control approach, a test strategy that is already measured should be used for software
testing to measure and control the quality during the development of software.

Testing Approaches
Tests can be conducted based on two approaches –
 Functionality testing
 Implementation testing
When functionality is being tested without taking the actual implementation in concern it is known as
black-box testing. The other side is known as white-box testing where not only functionality is tested but
the way it is implemented is also analyzed.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Exhaustive tests are the best-desired method for a perfect testing. Every single possible value in the
range of the input and output values is tested. It is not possible to test each and every value in real world
scenario if the range of values is large.
Black box testing is a type of software testing in which the functionality of the software is not known.
The testing is done without the internal knowledge of the products.
Black box testing can be done in the following ways:
1. Syntax-Driven Testing – This type of testing is applied to systems that can be syntactically
represented by some language. For example- compilers, language that can be represented by a context-
free grammar. In this, the test cases are generated so that each grammar rule is used at least once.
2. Equivalence partitioning – It is often seen that many types of inputs work similarly so instead of
giving all of them separately we can group them and test only one input of each group. The idea is to
partition the input domain of the system into several equivalence classes such that each member of the
class works similarly, i.e., if a test case in one class results in some error, other members of the class
would also result in the same error.
The technique involves two steps:
1. Identification of equivalence class – Partition any input domain into a minimum of two sets: valid
values and invalid values. For example, if the valid range is 0 to 100 then select one valid input like
49 and one invalid like 104.
2. Generating test cases – (i) To each valid and invalid class of input assign a unique identification
number. (ii) Write a test case covering all valid and invalid test cases considering that no two invalid
inputs mask each other. To calculate the square root of a number, the equivalence classes will be: (a)
Valid inputs:
 The whole number which is a perfect square- output will be an integer.
 The whole number which is not a perfect square- output will be a decimal number.
 Positive decimals
 Negative numbers(integer or decimal).
 Characters other than numbers like “a”,”!”,”;”, etc.
3. Boundary value analysis – Boundaries are very good places for errors to occur. Hence if test cases
are designed for boundary values of the input domain then the efficiency of testing improves and the
probability of finding errors also increases. For example – If the valid range is 10 to 100 then test for
10,100 also apart from valid and invalid inputs.
4. Cause effect Graphing – This technique establishes a relationship between logical input called
causes with corresponding actions called the effect. The causes and effects are represented using
Boolean graphs. The following steps are followed:
1. Identify inputs (causes) and outputs (effect).
2. Develop a cause-effect graph.
3. Transform the graph into a decision table.
4. Convert decision table rules to test cases.
For example, in the following cause-effect graph:

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 It can be converted into a decision table like:

Each column corresponds to a rule which will become a test case for testing. So there will be 4 test
cases.
5. Requirement-based testing – It includes validating the requirements given in the SRS of a
software system.
6. Compatibility testing – The test case result not only depends on the product but is also on the
infrastructure for delivering functionality. When the infrastructure parameters are changed it is still
expected to work properly. Some parameters that generally affect the compatibility of software are:
1. Processor (Pentium 3, Pentium 4) and several processors.
2. Architecture and characteristics of machine (32-bit or 64-bit).
3. Back-end components such as database servers.
4. Operating System (Windows, Linux, etc).

Black Box Testing Type

The following are the several categories of black box testing:
1. Functional Testing
2. Regression Testing
3. Nonfunctional Testing (NFT)

Functional Testing: It determines the system’s software functional requirements.

Regression Testing: It ensures that the newly added code is compatible with the existing code. In
other words, a new software update has no impact on the functionality of the software. This is carried
out after a system maintenance operation and upgrades.
Nonfunctional Testing: Nonfunctional testing is also known as NFT. This testing is not functional
testing of software. It focuses on the software’s performance, usability, and scalability.

Tools Used for Black Box Testing:

1. Appium
2. Selenium
3. Microsoft Coded UI
4. Applitools
5. HP QTP.

Advantages of Black Box Testing:

 The tester does not need to have more functional knowledge or programming skills to implement the
Black Box Testing.
 It is efficient for implementing the tests in the larger system.
 Tests are executed from the user’s or client’s point of view.
 Test cases are easily reproducible.
 It is used in finding the ambiguity and contradictions in the functional specifications.
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 Disadvantages of Black Box Testing:
 There is a possibility of repeating the same tests while implementing the testing process.
 Without clear functional specifications, test cases are difficult to implement.
 It is difficult to execute the test cases because of complex inputs at different stages of testing.
 Sometimes, the reason for the test failure cannot be detected.
 Some programs in the application are not tested.
 It does not reveal the errors in the control structure.
 Working with a large sample space of inputs can be exhaustive and consumes a lot of time.

White box testing techniques analyze the internal structures the used data structures, internal design,
code structure, and the working of the software rather than just the functionality as in black box
testing. It is also called glass box testing or clear box testing or structural testing. White Box Testing
is also known as transparent testing, open box testing.
Prerequisite – Software Testing | Basics

Working process of white box testing:

 Input: Requirements, Functional specifications, design documents, source code.
 Processing: Performing risk analysis for guiding through the entire process.
 Proper test planning: Designing test cases so as to cover the entire code. Execute rinse-repeat until
error-free software is reached. Also, the results are communicated.
 Output: Preparing final report of the entire testing process.
Testing techniques:
 Statement coverage: In this technique, the aim is to traverse all statement at least once. Hence, each
line of code is tested. In case of a flowchart, every node must be traversed at least once. Since all lines
of code are covered, helps in pointing out faulty code.

Statement Coverage Example

 Branch Coverage: In this technique, test cases are designed so that each branch from all decision
points are traversed at least once. In a flowchart, all edges must be traversed at least once.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 4 test cases required such that all branches of all decisions are covered, i.e, all edges of flowchart are

covered

 Condition Coverage: In this technique, all individual conditions must be covered as shown in the
following example:

1. READ X, Y
2. IF(X == 0 || Y == 0)
3. PRINT ‘0’
4. #TC1 – X = 0, Y = 55
5. #TC2 – X = 5, Y = 0

 Multiple Condition Coverage: In this technique, all the possible combinations of the possible
outcomes of conditions are tested at least once. Let’s consider the following example:

1. READ X, Y
2. IF(X == 0 || Y == 0)
3. PRINT ‘0’
4. #TC1: X = 0, Y = 0
5. #TC2: X = 0, Y = 5
6. #TC3: X = 55, Y = 0
7. #TC4: X = 55, Y = 5

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Basis Path Testing: In this technique, control flow graphs are made from code or flowchart and then
Cyclomatic complexity is calculated which defines the number of independent paths so that the
minimal number of test cases can be designed for each independent path.

Steps:
1. Make the corresponding control flow graph
2. Calculate the cyclomatic complexity
3. Find the independent paths
4. Design test cases corresponding to each independent path
5. V(G) = P + 1, where P is the number of predicate nodes in the flow graph
6. V(G) = E – N + 2, where E is the number of edges and N is the total number of nodes
7. V(G) = Number of non-overlapping regions in the graph
8. #P1: 1 – 2 – 4 – 7 – 8
9. #P2: 1 – 2 – 3 – 5 – 7 – 8
10. #P3: 1 – 2 – 3 – 6 – 7 – 8
11. #P4: 1 – 2 – 4 – 7 – 1 – . . . – 7 – 8

 Loop Testing: Loops are widely used and these are fundamental to many algorithms hence, their
testing is very important. Errors often occur at the beginnings and ends of loops.

1. Simple loops: For simple loops of size n, test cases are designed that:
 Skip the loop entirely
 Only one pass through the loop
 2 passes
 m passes, where m < n
 n-1 ans n+1 passes
2. Nested loops: For nested loops, all the loops are set to their minimum count and we start from the
innermost loop. Simple loop tests are conducted for the innermost loop and this is worked outwards
till all the loops have been tested.

3. Concatenated loops: Independent loops, one after another. Simple loop tests are applied for each. If
they’re not independent, treat them like nesting.

White Testing is Performed in 2 Steps:

1. Tester should understand the code well
2. Tester should write some code for test Cases and execute them
Tools required for White box Testing:
 PyUnit
 Sqlmap
 Nmap
 Parasoft Jtest
 Nunit
 VeraUnit
 CppUnit

Advantages:
1. White box testing is very thorough as the entire code and structures are tested.
2. It results in the optimization of code removing error and helps in removing extra lines of code.
3. It can start at an earlier stage as it doesn’t require any interface as in case of black box testing.
4. Easy to automate.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

5. White box testing can be easily started in Software Development Life Cycle.
6. Easy Code Optimization.

Disadvantages:
1. Main disadvantage is that it is very expensive.
2. Redesign of code and rewriting code needs test cases to be written again.
3. Testers are required to have in-depth knowledge of the code and programming language as opposed to
black box testing.
4. Missing functionalities cannot be detected as the code that exists is tested.
5. Very complex and at times not realistic.
6. Much more chances of Errors in production.

Unit Testing:

 Unit Testing is a software testing technique by means of which individual units of software i.e. group of
computer program modules, usage procedures, and operating procedures are tested to determine whether
they are suitable for use or not.
 It is a testing method using which every independent module is tested to determine if there is an issue by
the developer himself.
 It is correlated with the functional correctness of the independent modules.
 Unit Testing is defined as a type of software testing where individual components of a software are
tested.
 Unit Testing of the software product is carried out during the development of an application.
 An individual component may be either an individual function or a procedure.
 Unit Testing is typically performed by the developer. In SDLC or V Model, Unit testing is the first level
of testing done before integration testing.
 Unit testing is such a type of testing technique that is usually performed by developers.
 Although due to the reluctance of developers to test, quality assurance engineers also do unit testing.

Objective of Unit Testing:

The objective of Unit Testing is:

1. To isolate a section of code.

2. To verify the correctness of the code.
3. To test every function and procedure.
4. To fix bugs early in the development cycle and to save costs.
5. To help the developers to understand the code base and enable them to make changes quickly.
6. To help with code reuse.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Types of Unit Testing:

There are 2 types of Unit Testing: Manual, and Automated.

Work flow of Unit Testing:

Unit Testing Techniques:

There are 3 types of Unit Testing Techniques. They are

1. Black Box Testing: This testing technique is used in covering the unit tests for input, user interface, and
output parts.
2. White Box Testing: This technique is used in testing the functional behavior of the system by giving
the input and checking the functionality output including the internal design structure and code of the
modules.
3. Gray Box Testing: This technique is used in executing the relevant test cases, test methods, test
functions, and analyzing the code performance for the modules.

Unit Testing Tools:

Here are some commonly used Unit Testing tools:

1. Jtest
2. Junit
3. NUnit
4. EMMA
5. PHPUnit

Advantages of Unit Testing:

1. Unit Testing allows developers to learn what functionality is provided by a unit and how to use it to gain
a basic understanding of the unit API.
2. Unit testing allows the programmer to refine code and make sure the module works properly.
3. Unit testing enables testing parts of the project without waiting for others to be completed.

Disadvantages of Unit Testing:

1. The process is time-consuming for writing the unit test cases.

2. Unit Testing will not cover all the errors in the module because there is a chance of having errors in the
modules while doing integration testing.
3. Unit Testing is not efficient for checking the errors in the UI(User Interface) part of the module.
4. It requires more time for maintenance when the source code is changed frequently.
5. It cannot cover the non-functional testing parameters such as scalability, the performance of the system,
etc.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Integration Testing

 Integration testing is the process of testing the interface between two software units or modules.
 It focuses on determining the correctness of the interface.
 The purpose of integration testing is to expose faults in the interaction between integrated units.
 Once all the modules have been unit tested, integration testing is performed.
 Integration testing can be done by picking module by module.
 This can be done so that there should be proper sequence to be followed.
 And also if you don’t want to miss out on any integration scenarios then you have to follow the proper
sequence.
 Exposing the defects is the major focus of the integration testing and the time of interaction between the
integrated units.

Integration test approaches:

There are four types of integration testing approaches. Those approaches are the following:

1. Big-Bang Integration Testing – It is the simplest integration testing approach, where all the modules
are combined and the functionality is verified after the completion of individual module testing. In
simple words, all the modules of the system are simply put together and tested. This approach is
practicable only for very small systems. If an error is found during the integration testing, it is very
difficult to localize the error as the error may potentially belong to any of the modules being integrated.
So, debugging errors reported during big bang integration testing is very expensive to fix.

Advantages:
 It is convenient for small systems.

Disadvantages:
 There will be quite a lot of delay because you would have to wait for all the modules to be integrated.
 High risk critical modules are not isolated and tested on priority since all modules are tested at once.
 Not Good for long Projects.

2. Bottom-Up Integration Testing – In bottom-up testing, each module at lower levels is tested with
higher modules until all modules are tested. The primary purpose of this integration testing is that each
subsystem tests the interfaces among various modules making up the subsystem. This integration testing
uses test drivers to drive and pass appropriate data to the lower level modules.

Advantages:
 In bottom-up testing, no stubs are required.
 A principle advantage of this integration testing is that several disjoint subsystems can be tested
simultaneously.
 It is easy to create the test conditions.
 Best for the applications that uses bottom up design approach.
 It is Easy to observe the test results.

Disadvantages:
 Driver modules must be produced.
 In this testing, the complexity that occurs when the system is made up of a large number of small
subsystems.
 As Far modules have been created, there is no working model can be represented.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 3. Top-Down Integration Testing – Top-down integration testing technique is used in order to simulate
the behavior of the lower-level modules that are not yet integrated. In this integration testing, testing
takes place from top to bottom. First, high-level modules are tested and then low-level modules and
finally integrating the low-level modules to a high level to ensure the system is working as intended.

Advantages:
 Separately debugged module.
 Few or no drivers needed.
 It is more stable and accurate at the aggregate level.
 Easier isolation of interface errors.
 In this, design defects can be found in the early stages.

Disadvantages:
 Needs many Stubs.
 Modules at lower level are tested inadequately.
 It is difficult to observe the test output.
 It is difficult to stub design.

4. Mixed Integration Testing – A mixed integration testing is also called sandwiched integration
testing. A mixed integration testing follows a combination of top down and bottom-up testing
approaches. In top-down approach, testing can start only after the top-level module have been coded and
unit tested. In bottom-up approach, testing can start only after the bottom level modules are ready. This
sandwich or mixed approach overcomes this shortcoming of the top-down and bottom-up approaches. It
is also called the hybrid integration testing. also, stubs and drivers are used in mixed integration testing.

Advantages:
 Mixed approach is useful for very large projects having several sub projects.
 This Sandwich approach overcomes this shortcoming of the top-down and bottom-up approaches.
 Parallel test can be performed in top and bottom layer tests.

Disadvantages:
 For mixed integration testing, it requires very high cost because one part has Top-down approach while
another part has bottom-up approach.
 This integration testing cannot be used for smaller systems with huge interdependence between different
modules.

Regression Testing

 Regression Testing is the process of testing the modified parts of the code and the parts that might get
affected due to the modifications to ensure that no new errors have been introduced in the software after
the modifications have been made.
 Regression means return of something and in the software field, it refers to the return of a bug.

When to do regression testing?

 When a new functionality is added to the system and the code has been modified to absorb and integrate
that functionality with the existing code.
 When some defect has been identified in the software and the code is debugged to fix it.
 When the code is modified to optimize its working.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Process of Regression testing:

 Firstly, whenever we make some changes to the source code for any reasons like adding new
functionality, optimization, etc.
 Then our program when executed fails in the previously designed test suite for obvious reasons.
 After the failure, the source code is debugged in order to identify the bugs in the program.
 After identification of the bugs in the source code, appropriate modifications are made.
 Then appropriate test cases are selected from the already existing test suite which covers all the modified
and affected parts of the source code.
 We can add new test cases if required. In the end regression testing is performed using the selected test
cases.

Techniques for the selection of Test cases for Regression Testing:

 Select all test cases: In this technique, all the test cases are selected from the already existing test suite.
It is the most simple and safest technique but not much efficient.
 Select test cases randomly: In this technique, test cases are selected randomly from the existing test-
suite but it is only useful if all the test cases are equally good in their fault detection capability which is
very rare. Hence, it is not used in most of the cases.
 Select modification traversing test cases: In this technique, only those test cases are selected which
covers and tests the modified portions of the source code the parts which are affected by these
modifications?
 Select higher priority test cases: In this technique, priority codes are assigned to each test case of the
test suite based upon their bug detection capability, customer requirements, etc. After assigning the
priority codes, test cases with highest priorities are selected for the process of regression [Link]
case with highest priority has highest rank. For example, test case with priority code 2 is less important
than test case with priority code 1.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Tools for regression testing:

In regression testing, we generally select the test cases form the existing test suite itself and hence, we
need not to compute their expected output and it can be easily automated due to this reason. Automating
the process of regression testing will be very much effective and time saving.

Most commonly used tools for regression testing are:

 Selenium
 WATIR (Web Application Testing In Ruby)
 QTP (Quick Test Professional)
 RFT (Rational Functional Tester)
 Winrunner
 Silktest

Advantages of Regression Testing:

 It ensures that no new bugs has been introduced after adding new functionalities to the system.
 As most of the test cases used in Regression Testing are selected from the existing test suite and we
already know their expected outputs. Hence, it can be easily automated by the automated tools.
 It helps to maintain the quality of the source code.

Disadvantages of Regression Testing:

 It can be time and resource consuming if automated tools are not used.
 It is required even after very small changes in the code.

Smoke Testing:

 Smoke Testing is a software testing method that determines whether the employed build is stable or not.
 It acts as a confirmation of whether the quality assurance team can proceed with further testing.
 Smoke tests are a minimum set of tests run on each build.
 Smoke testing is a process where the software build is deployed to a quality assurance environment and
is verified to ensure the stability of the application.
 Smoke Testing is also known as Confidence Testing or Build Verification Testing.
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 In other words, we verify whether the important features are working and there are no showstoppers in
the build that is under testing.
 It is a mini and quick regression test of major functionality. Smoke testing shows that the product is
ready for testing.
 This helps in determining if the build is flawed so as to make any further testing a waste of time and
resources.

Characteristics of Smoke Testing:

Following are the characteristics of the smoke testing:

 Smoke testing is documented.

 Smoke testing may be stable as well as unstable.
 Smoke testing is scripted.
 Smoke testing is a type of regression testing.

Smoke Testing is usually carried out by quality assurance engineers.

Goal of Smoke Testing:

The aim of Smoke Testing is:

1. To detect any early defects in a software product.

2. To demonstrate system stability.
3. To demonstrate conformance to requirements.
4. To assure that the acute functionalities of the program are working fine.
5. To measures the stability of the software product by performing testing.
6. To test all over the function of the software product.

Types of Smoke Testing:

There are three types of Smoke Testing:

1. Manual Testing: In this, the tester has to write, develop, modify or update the test cases for each built
product. Either the tester has to write test scripts for existing features or new features.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

2. Automated Testing: In this, the tool will handle the testing process by itself providing the relevant
tests. It is very helpful when the project should be completed in a limited time.
3. Hybrid Testing: As the name implies, it is the combination of both manual and automated testing.
Here, the tester has to write test cases by himself and he can also automate the tests using the tool. It
increases the performance of the testing as it combines both manual checking and tool.

Applying Smoke Testing at different levels:

It is applicable at 3 levels of testing. They are

 Acceptance Testing Level

 System Testing Level
 Integration testing Level

Tools used for Smoke Testing:

 Selenium
 PhantomJS

These tools are used while implementing the automated test cases.

Advantages of Smoke Testing:

1. Smoke testing is easy to perform.

2. It helps in identifying defects in the early stages.
3. It improves the quality of the system.
4. Smoke testing reduces the risk of failure.
5. Smoke testing makes progress easier to access.
6. It saves test effort and time.
7. It makes it easy to detect critical errors and helps in the correction of errors.
8. It runs quickly.
9. It minimizes integration risks.

Disadvantages of Smoke Testing:

1. Smoke Testing does not cover all the functionality in the application. Only a certain part of the testing is
done.
2. Errors may occur even after implementing all the smoke tests.
3. In the case of manual smoke testing, it takes a lot of time to execute the testing process for larger
projects.
4. It will not be implemented against the negative tests or with the invalid input.
5. It usually consists of a minimum number of test cases and hence we cannot find the other issues that
happened during the testing process.

Sanity Testing

Sanity Testing is a subset of regression testing. Sanity testing is performed to ensure that the code
changes that are made are working as properly. Sanity testing is a stoppage to check whether testing for
the build can proceed or not. The focus of the team during sanity testing process is to validate the
functionality of the application and not detailed testing. Sanity testing is generally performed on build
where the production deployment is required immediately like a critical bug fix.
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 Functionality of Sanity Testing:

The major functionality of sanity testing is to determine that the changes or the proposed functionality
are working as expected. If the sanity test fails, software product is rejected by the testing team to save
time and money. It is performed only after the software product has passed the smoke test and Quality
Assurance team has accepted for further testing.

Example of Sanity Testing:

In an e-commerce project, main modules are login page, home page, user profile page, user registration
etc. There is a defect in the login page when the password field accepts less than four alpha numeric
characters and the requirement mentions that this password field should not be below eight characters.
Hence, the defect is reported by the testing team to the development team to resolve it. Then the
development team fixes the reported defect and sends it to the testing team for clearance. Then the
testing team checks whether the changes done are working fine or not. It is also determined if it does
have an impact on other related functionalities. Now there is a functionality to update the password in
the user profile page. As part of the sanity testing, login page is validated as well as the profile page to
ensure that the checks are working fine at both the places.

Features of Sanity Testing:

1. Subset of Regression Testing:

Sanity testing is a subset of regression testing and focuses on the smaller section of the application.

2. Unscripted:
Most of the times sanity testing is not scripted.
3. Not documented:

Usually sanity testing is undocumented.

4. Narrow and deep:

Sanity testing is narrow and deep approach of testing where limited functionalities are covered deeply.

5. Performed by testers:

Sanity testing is normally performed by testers.

Advantages of Sanity Testing:

 Sanity testing helps in quickly identify defects in the core functionality.

 It can be carried out in lesser time as no documentation is required for sanity testing.
 If the defects are found during sanity testing, project is rejected that is helpful in saving time for
execution of regression tests.
 This testing technique is not so expensive when compared to another types of testing.
 It helps to identify the dependent missing objects.
 It is used to verify a small functionality of the system application whether it is still working or not even
after a small change.
 It helps in the scenario when the time for testing of the product is limited or having less time to complete
the test.
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 Disadvantages of Sanity Testing:

 It focuses only on the functions and commands of the system application.

 It is not possible to cover all the test cases in test scenarios.
 It covers only little functionality in the system application. Issues in the unchecked functionalities can’t
be recovered.
 Sanity testing is usually unscripted. Hence, future references are not available.
 It doesn’t cover the design structure level and hence it will be difficult for development team to identify
and fix the issues.

Retesting in Software Testing

Retesting is a procedure where we need to check that particular test cases which are found with some
bugs during the execution time. Retesting also occurs when the product is already tested and due to
some problems it needs to be tested again. This tests is named as retesting.

Retesting depends on the developer department that they are going to accept the bug testing or reject.
Retesting is done when there is a specific bug when the bug is rejected by the developer and the tester
department needs to tests the issues when the user reports a problem for retesting and fixing of an issue
for better application and better workflow.

Characteristics of retesting:

 It is a copy of testing which are done with the same files and processes in a new build.
 Retesting can only be implemented when a particular test cases are involved which are considered as
failed tests.
 It takes the surety of successful tests and working flow as it has to be.
 During execution time whenever any bug is occurred and that test is declined by the developer, in that
situation the testers department tests that file and they try to find the actual issue and they also do the
retesting of that bug to make sure whether that bug is actual or not.
 Sometimes the whole program needs to be retested to verify the quality of the program.
 The cases which are being retested that cannot be computerized.

Use of Re-testing in Software Testing:

 Retesting is used when there is any specific error or bug which needs to be verified.
 It is used when the bug is rejected by the developers then the testing department tests whether that bug is
actual or not.
 It is also used to check the whole system to verify the final functionality.
 It also checks the quality of a specific part of a system.
 When some user demands for retesting of their system.

Instance of Retesting:

Let’s take an example of Face book page, we need an Facebook account for that first we need to sign up
for creating an account.

The user adds every personal details of himself/herself like Name, DOB, Address, School details etc.

After adding all the details the last step is to click on the “Sign up” button.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 While clicking on sign up button the user finds that the sign up button is not working.

This bug is analyzed by the user and he reports to the developer and developer successfully fixed it.

After fixing of bug again it is assigned to the tester for rechecking and workflow of the application,
during this time the tester will only check the “Sign up” button, this process is known as retesting.

Advantages of Retesting:

 Retesting ensures that the bug is being fixed completely and working successfully according to the user.
 It enhances the quality of the product or application.
 The bug can be fixed in a short period of time as it targets on a particular issue.
 Retesting doesn’t required any specific or another software for testing.
 It can perform with the same data and same process with new build for its execution.

Disadvantages of Retesting:

 Retesting needs new build for qualifying verification process of the bug.
 If once the testing process is started then at that time, the test cases of the retesting we can obtain but not
previously.
 At the time of testing, the retesting cannot be computerized.
 When the retesting results unsuccessful, it requires more time and efforts for fixing all the issues.

Alpha Testing

Alpha Testing is a type of software testing performed to identify bugs before releasing the product to
real users or to the public.

Alpha Testing is one of the user acceptance testing’s.

This is referred to as alpha testing only because it is done early on, near the end of the development of
the software.

Alpha testing is commonly performed by homestead software engineers or quality assurance staff.

It is the last testing stage before the software is released into the real world.

Objective of Alpha Testing:

1. The objective of alpha testing is to refine the software product by finding the bugs that were not
discovered during the previous tests.
2. The objective of alpha testing is to refine the software product by fixing the bugs that were not
discovered during the previous tests.
3. The objective of alpha testing is to involve customers deep into the process of development.
4. The objective of alpha testing is to give better insight into the software’s reliability at the early stages of
development.

Alpha Testing Process:

1. Review the design specification and functional requirements.

2. Develop comprehensive test cases and test plans.
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
3. Execute test plan
4. Log defects
5. Retest once the issues have been fixed

Phases of Alpha Testing:

There are two phases in alpha testing:

1st Phase: The first phase of testing is done by in-house developers or software engineers. They either
use hardware-aided debuggers or debugger software. The aim is to catch bugs quickly. Usually while
alpha testing, a tester comes across to lots of bugs, crashes, missing features, and docs.

2nd Phase: The second phase of alpha testing is done by software quality assurance staff for additional
testing in an environment. It includes a black box as well as white box testing.

Advantages of Alpha Testing:

 Better insight about the software’s reliability at its early stages.

 Free up your team for other projects.
 It reduces delivery time to market.
 Early feedback helps to improve software quality.

Disadvantages of Alpha Testing:

 It will need a longer time for test plan execution if the project is large.
 Sometimes, the defects in the products can be unknown during this alpha testing.
 It is difficult to test the entire product since it is still under development.
 For smaller projects, time spent on alpha testing is not worthy enough.
 It does not carry out reliability and security testing.
 This test will only cover the business requirements mentioned by the client. The project team will not go
through the deep testing of each and every module.
 It requires a separate lab environment for testing.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Beta Testing

Beta Testing is performed by real users of the software application in a real environment.

Beta testing is one of the types of User Acceptance Testing.

A Beta version of the software, whose feedback is needed, is released to a limited number of end-users
of the product to obtain feedback on the product quality.

Beta testing helps in minimization of product failure risks and it provides increased quality of the
product through customer validation.

It is the last test before shipping a product to the customers. One of the major advantages of beta testing
is direct feedback from customers.

Characteristics of Beta Testing:

1. Beta Testing is performed by clients or users who are not employees of the company.
2. Reliability, security, and robustness are checked during beta testing.
3. Beta Testing commonly uses black-box testing.
4. Beta testing is carried out in the user’s location.
5. Beta testing doesn’t require a lab or testing environment.

Types of Beta Testing:

There are different types of beta testing:

1. Traditional Beta testing: Product is distributed to the target market and related data is gathered in all
aspects. This data can be used for Product improvement.
2. Public Beta Testing: Product is released publicly to the world through online channels and data can be
collected from anyone. Based on feedback, product improvements can be done. For example, Microsoft
conducted the largest of all Beta Tests for its operating system Windows 8 before officially releasing it.
3. Technical Beta Testing: Product is released to a group of employees of an organization and collects
feedback/data from the employees of the organization.
4. Focused Beta Testing: Software product is released to the market for collecting feedback on specific
features of the program. For example, important functionality of the software.
5. Post-release Beta Testing: Software product is released to the market and data is collected to make
improvements for the future release of the product.

Criteria for Beta Testing:

 Sign off a document on Alpha testing.

 The Beta version of the software should be ready
 Environment ready to release the software application to the public
 Tool to capture real-time faults

Tools used for Beta Testing:

 TestFairy
 CenterCode
 TryMyUI
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 UserTesting
 TestRail
 Usersnap
 Zephyr
 TestFlight

Advantages of Beta Testing:

 It reduces product failure risk via customer validation.

 Beta Testing allows a company to test post-launch infrastructure.
 It helps in improving product quality via customer feedback.
 Cost-effective compared to similar data gathering methods.
 It creates goodwill with customers and increases customer satisfaction.

Disadvantages of Beta Testing:

 Sometimes, it is complex to follow the errors or bugs because the testing environment varies from user
to user.
 There is a chance of having duplication of errors or bugs.
 The development team and the testing team are not having control over this real-time test environment.
 This testing is a time consuming process since it involves real time users or clients and hence delay in
the overall feedback about the entire product.
 The users who are testing the product should have enough knowledge about the working of the entire
application or product. Otherwise, the testing will not be implemented in an efficient manner.

Stress Testing Difficulty Level : Basic

Stress Testing is a software testing technique that determines the robustness of software by testing
beyond the limits of normal operation.

Stress testing is particularly important for critical software but is used for all types of software.

Stress testing emphasizes robustness, availability, and error handling under a heavy load rather than
what is correct behavior under normal situations.

Stress testing is defined as a type of software testing that verifies the stability and reliability of the
system.

This test particularly determines the system on its robustness and error handling under extremely heavy
load conditions.

It even tests beyond the normal operating point and analyses how the system works under extreme
conditions.

Stress testing is performed to ensure that the system would not crash under crunch situations. Stress
testing is also known as Endurance Testing or Torture Testing.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Characteristics of Stress Testing:

1. Stress testing analyzes the behavior of the system after a failure.

2. Stress testing makes sure that the system recovers after failure.
3. It checks whether the system works under abnormal conditions.
4. It ensures to display of appropriate error messages when the system is under stress.
5. It verifies that unexpected failures do not cause security issues.
6. It verifies whether the system has saved the data before crashing or not.

Stress Testing Process: Types of Stress Testing:

1. Server-client Stress Testing: In this stress testing, testing is carried out across all clients from the
server.
2. Product Stress Testing: Product stress testing concentrates on discovering defects related to data
locking and blocking, network issues, and performance congestion in a software product.
3. Transaction Stress Testing: Transaction stress testing is performed on one or more transactions
between two or more applications. It is carried out for fine-tuning and optimizing the system.
4. Systematic Stress Testing: Systematic stress testing is integrated testing that is used to perform tests
across multiple systems running on the same server. It is used to discover defects where one application
data blocks another application.
5. Analytical Stress Testing: Analytical stress testing is performed to test the system with abnormal
parameters or conditions that are unlikely to happen in a real scenario. It is carried out to find defects in
unusual scenarios like a large number of users logged at the same time or a database going offline when
it is accessed from a website.

Stress Testing Tools:

1. Jmeter
2. Load Runner
3. Stress Tester
4. Neo load

Metrics of Stress Testing:

Metrics are used to evaluate the performance of the stress and it is usually carried out at the end of the
stress scripts or tests. Some of the metrics are given below.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

1. Pages Per Second: Number of pages requested per second and number of pages loaded per second.
2. Pages Retrieved: Average time taken to retrieve all information from the particular page.
3. Byte Retrieved: Average time is taken to retrieve the first byte of information from the page.
4. Transaction Response Time: Average time is taken to load or perform transactions between the
applications.
5. Transactions per Second: It takes count of the number of transactions loaded per second successfully
and it also counts the number of failures that occurred.
6. Failure of Connection: It takes count of the number of times that the client faced the connection failure
in their system.
7. Failure of System attempts: It takes count of the number of failed attempts in the system.
8. Rounds: It takes count of the number of test or script conditions executed by the clients successfully
and it keeps track of the number of rounds failed.

Advantages of Stress Testing:

1. Stress testing determines the behavior of the system after failure and ensures that the system recovers
quickly.
2. Stress testing ensures that system failure doesn’t cause security issues.
3. Stress testing makes the system work in normal as well as abnormal conditions in an appropriate way.

Disadvantages of Stress Testing:

1. The manual process of stress testing takes a longer time to complete and it is a complicated process.
2. Good scripting knowledge for implementing the script test cases for the particular tool is required.
3. There is a need for external resources to implement stress testing. It leads to an extra amount of
resources and time.
4. In the case of a licensed stress testing tool, it charges more than the average amount of cost.
5. In the case of some open-source tools, there is a need for a load testing tool additionally for setting up
the stress testing environment.
6. If proper stress scripts or test cases are not implemented then there will be a chance of failure of some
resources and wastage of time.

Performance Testing

Performance Testing is a type of software testing that ensures software applications to perform
properly under their expected workload.

It is a testing technique carried out to determine system performance in terms of sensitivity, reactivity
and stability under a particular workload.

Performance Testing is the process of analyzing the quality and capability of a product.

It is a testing method performed to determine the system performance in terms of speed, reliability and
stability under varying workload.

Performance testing is also known as Perf Testing.

Performance Testing Attributes:

 Speed:
It determines whether the software product responds rapidly.
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 Scalability:
It determines amount of load the software product can handle at a time.
 Stability:
It determines whether the software product is stable in case of varying workloads.
 Reliability:
It determines whether the software product is secure or not.

Objective of Performance Testing:

1. The objective of performance testing is to eliminate performance congestion.

2. It uncovers what is needed to be improved before the product is launched in market.
3. The objective of performance testing is to make software rapid.
4. The objective of performance testing is to make software stable and reliable.

Types of Performance Testing:

1. Load testing:

It checks the product’s ability to perform under anticipated user loads. The objective is to identify
performance congestion before the software product is launched in market.

2. Stress testing:

It involves testing a product under extreme workloads to see whether it handles high traffic or not. The
objective is to identify the breaking point of a software product.

3. Endurance testing:

It is performed to ensure the software can handle the expected load over a long period of time.

4. Spike testing:

It tests the product’s reaction to sudden large spikes in the load generated by users.

5. Volume testing:

In volume testing large number of data is saved in a database and the overall software system’s behavior
is observed. The objective is to check product’s performance under varying database volumes.

6. Scalability testing:

In scalability testing, software application’s effectiveness is determined in scaling up to support an

increase in user load. It helps in planning capacity addition to your software system.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Performance Testing Process:

Performance Testing Tools:

1. Jmeter
2. Open STA
3. Load Runner
4. Web Load

Advantages of Performance Testing :

 Performance testing ensures the speed, load capability, accuracy and other performances of the system.
 It identifies, monitors and resolves the issues if anything occurs.
 It ensures the great optimization of the software and also allows large number of users to use it on same
time.
 It ensures the client as well as end-customers satisfaction.

Disadvantages of Performance Testing :

 Sometimes, users may find performance issues in the real time environment.
 Team members who are writing test scripts or test cases in the automation tool should have high-level of
knowledge.
 Team members should have high proficiency to debug the test cases or test scripts.
 Low performances in the real environment may lead to lose large number of users.

Object Oriented Testing

Software typically undergoes many levels of testing, from unit testing to system or acceptance testing.

Typically, in-unit testing, small “units”, or modules of the software, are tested separately with focus on
testing the code of that module.

In higher, order testing (e.g, acceptance testing), the entire system (or a subsystem) is tested with the
focus on testing the functionality or external behavior of the system.

As information systems are becoming more complex, the object-oriented paradigm is gaining popularity
because of its benefits in analysis, design, and coding.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Conventional testing methods cannot be applied for testing classes because of problems involved in
testing classes, abstract classes, inheritance, dynamic binding, message, passing, polymorphism,
concurrency, etc.

Testing classes is a fundamentally different problem than testing functions. A function (or a procedure)
has a clearly defined input-output behavior, while a class does not have an input-output behavior
specification.

We can test a method of a class using approaches for testing functions, but we cannot test the class using
these approaches.

According to Davis the dependencies occurring in conventional systems are:

 Data dependencies between variables

 Calling dependencies between modules
 Functional dependencies between a module and the variable it computes
 Definitional dependencies between a variable and its types.

But in Object-Oriented systems there are following additional dependencies:

 Class to class dependencies

 Class to method dependencies
 Class to message dependencies
 Class to variable dependencies
 Method to variable dependencies
 Method to message dependencies
 Method to method dependencies

Issues in Testing Classes:

Additional testing techniques are, therefore, required to test these dependencies. Another issue of
interest is that it is not possible to test the class dynamically, only its instances i.e, objects can be tested.
Similarly, the concept of inheritance opens various issues e.g., if changes are made to a parent class or
superclass, in a larger system of a class it will be difficult to test subclasses individually and isolate the
error to one class.

In object-oriented programs, control flow is characterized by message passing among objects, and the
control flow switches from one object to another by inter-object communication. Consequently, there is
no control flow within a class like functions. This lack of sequential control flow within a class requires
different approaches for testing. Furthermore, in a function, arguments passed to the function with
global data determine the path of execution within the procedure. But, in an object, the state associated
with the object also influences the path of execution, and methods of a class can communicate among
themselves through this state because this state is persistent across invocations of methods. Hence, for
testing objects, the state of an object has to play an important role.

Techniques of object-oriented testing are as follows:

1. Fault Based Testing:

This type of checking permits for coming up with test cases supported the consumer specification or the
code or both. It tries to identify possible faults (areas of design or code that may lead to errors.). For all
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 of these faults, a test case is developed to “flush” the errors out. These tests also force each time of code
to be executed.

This method of testing does not find all types of errors. However, incorrect specification and interface
errors can be missed. These types of errors can be uncovered by function testing in the traditional testing
model. In the object-oriented model, interaction errors can be uncovered by scenario-based testing. This
form of Object oriented-testing can only test against the client’s specifications, so interface errors are
still missed.

2. Class Testing Based on Method Testing:

This approach is the simplest approach to test classes. Each method of the class performs a well defined
cohesive function and can, therefore, be related to unit testing of the traditional testing techniques.
Therefore all the methods of a class can be involved at least once to test the class.

3. Random Testing:

It is supported by developing a random test sequence that tries the minimum variety of operations
typical to the behavior of the categories

4. Partition Testing:

This methodology categorizes the inputs and outputs of a category so as to check them severely. This
minimizes the number of cases that have to be designed.

5. Scenario-based Testing:

It primarily involves capturing the user actions then stimulating them to similar actions throughout the
test.
These tests tend to search out interaction form of error.

Acceptance Testing

Acceptance Testing is a method of software testing where a system is tested for acceptability. The
major aim of this test is to evaluate the compliance of the system with the business requirements and
assess whether it is acceptable for delivery or not. Standard Definition of Acceptance Testing:

It is a formal testing according to user needs, requirements and business processes conducted to
determine whether a system satisfies the acceptance criteria or not and to enable the users, customers or
other authorized entities to determine whether to accept the system or not.

Acceptance Testing is the last phase of software testing performed after System Testing and before
making the system available for actual use.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Types of Acceptance Testing:

1. User Acceptance Testing (UAT): User acceptance testing is used to determine whether the product is
working for the user correctly. Specific requirements which are quite often used by the customers are
primarily picked for the testing purpose. This is also termed as End-User Testing.
2. Business Acceptance Testing (BAT): BAT is used to determine whether the product meets the business
goals and purposes or not. BAT mainly focuses on business profits which are quite challenging due to
the changing market conditions and new technologies so the current implementation may have to being
changed which results in extra budgets.
3. Contract Acceptance Testing (CAT): CAT is a contract that specifies that once the product goes live,
within a predetermined period, the acceptance test must be performed and it should pass all the
acceptance use cases. Here is a contract termed a Service Level Agreement (SLA), which includes the
terms where the payment will be made only if the Product services are in-line with all the requirements,
which means the contract is fulfilled. Sometimes, this contract happens before the product goes live.
There should be a well-defined contract in terms of the period of testing, areas of testing, conditions on
issues encountered at later stages, payments, etc.
4. Regulations Acceptance Testing (RAT): RAT is used to determine whether the product violates the
rules and regulations that are defined by the government of the country where it is being released. This
may be unintentional but will impact negatively on the business. Generally, the product or application
that is to be released in the market, has to go under RAT, as different countries or regions have different
rules and regulations defined by its governing bodies. If any rules and regulations are violated for any
country then that country or the specific region then the product will not be released in that country or
region. If the product is released even though there is a violation then only the vendors of the product
will be directly responsible.
5. Operational Acceptance Testing (OAT): OAT is used to determine the operational readiness of the
product and is non-functional testing. It mainly includes testing of recovery, compatibility,
maintainability, reliability, etc. OAT assures the stability of the product before it is released to
production.
6. Alpha Testing: Alpha testing is used to determine the product in the development testing environment
by a specialized testers team usually called alpha testers.
7. Beta Testing: Beta testing is used to assess the product by exposing it to the real end-users, usually
called beta testers in their environment. Feedback is collected from the users and the defects are fixed.
Also, this helps in enhancing the product to give a rich user experience.

Use of Acceptance Testing:

 To find the defects missed during the functional testing phase.

 How well the product is developed.
 A product is what actually the customers need.
 Feedback help in improving the product performance and user experience.
 Minimize or eliminate the issues arising from the production.

Advantages of Acceptance Testing:

 This testing helps the project team to know the further requirements from the users directly as it involves
the users for testing.
 Automated test execution.
 It brings confidence and satisfaction to the clients as they are directly involved in the testing process.
 It is easier for the user to describe their requirement.
 It covers only the Black-Box testing process and hence the entire functionality of the product will be
tested.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 Disadvantages of Acceptance Testing:

 Users should have basic knowledge about the product or application.

 Sometimes, users don’t want to participate in the testing process.
 The feedback for the testing takes long time as it involves many users and the opinions may differ from
one user to another user.
 Development team is not participated in this testing process.

Validation Testing:

The process of evaluating software during the development process or at the end of the development
process to determine whether it satisfies specified business requirements.

Validation Testing ensures that the product actually meets the client's needs. It can also be defined as to
demonstrate that the product fulfills its intended use when deployed on appropriate environment.

It answers to the question, Are we building the right product?

Validation Testing - Workflow:

Validation testing can be best demonstrated using V-Model. The Software/product under test is
evaluated during this type of testing.

Activities:

 Unit Testing
 Integration Testing
 System Testing
 User Acceptance Testing

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 System Testing:

System Testing is a type of software testing that is performed on a complete integrated system to
evaluate the compliance of the system with the corresponding requirements.
In system testing, integration testing passed components is taken as input.
The goal of integration testing is to detect any irregularity between the units that are integrated
together.
System testing detects defects within both the integrated units and the whole system.
The result of system testing is the observed behavior of a component or a system when it is tested.
System Testing is carried out on the whole system in the context of either system requirement
specifications or functional requirement specifications or in the context of both.
System testing tests the design and behavior of the system and also the expectations of the customer. It
is performed to test the system beyond the bounds mentioned in the software requirements
specification (SRS).
System Testing is basically performed by a testing team that is independent of the development team
that helps to test the quality of the system impartial.
It has both functional and non-functional testing.
System Testing is a black-box testing.
System Testing is performed after the integration testing and before the acceptance testing.

System Testing Process: System Testing is performed in the following steps:

 Test Environment Setup: Create testing environment for the better quality testing.
 Create Test Case: Generate test case for the testing process.
 Create Test Data: Generate the data that is to be tested.
 Execute Test Case: After the generation of the test case and the test data, test cases are executed.
 Defect Reporting: Defects in the system are detected.
 Regression Testing: It is carried out to test the side effects of the testing process.
 Log Defects: Defects are fixed in this step.
 Retest: If the test is not successful then again test is performed.

Types of System Testing:

 Performance Testing: Performance Testing is a type of software testing that is carried out to test the
speed, scalability, stability and reliability of the software product or application.
 Load Testing: Load Testing is a type of software testing which is carried out to determine the
behavior of a system or software product under extreme load.
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 Stress Testing: Stress Testing is a type of software testing performed to check the robustness of the
system under the varying loads.
 Scalability Testing: Scalability Testing is a type of software testing which is carried out to check the
performance of a software application or system in terms of its capability to scale up or scale down the
number of user request load.

Tools used for System Testing:

1. JMeter
2. Gallen Framework
3. Selenium

Advantages of System Testing:

 The testers do not require more knowledge of programming to carry out this testing.
 It will test the entire product or software so that we will easily detect the errors or defects which
cannot be identified during the unit testing and integration testing.
 The testing environment is similar to that of the real time production or business environment.
 It checks the entire functionality of the system with different test scripts and also it covers the
technical and business requirements of clients.
 After this testing, the product will almost cover all the possible bugs or errors and hence the
development team will confidently go ahead with acceptance testing.

Disadvantages of System Testing:

 This testing is time consuming process than other testing techniques since it checks the entire product
or software.
 The cost for the testing will be high since it covers the testing of entire software.
 It needs good debugging tool otherwise the hidden errors will not be found.

Debugging:

In the context of software engineering, debugging is the process of fixing a bug in the software.

In other words, it refers to identifying, analyzing, and removing errors.

This activity begins after the software fails to execute properly and concludes by solving the problem
and successfully testing the software.

It is considered to be an extremely complex and tedious task because errors need to be resolved at all
stages of debugging.

Debugging Process: Steps involved in debugging are:

 Problem identification and report preparation.

 Assigning the report to the software engineer to the defect to verify that it is genuine.
 Defect Analysis using modeling, documentation, finding and testing candidate flaws, etc.
 Defect Resolution by making required changes to the system.
 Validation of corrections.

The debugging process will always have one of two outcomes:

1. The cause will be found and corrected.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 2. The cause will not be found.

Later, the person performing debugging may suspect a cause, design a test case to help validate that
suspicion and work toward error correction in an iterative fashion.

During debugging, we encounter errors that range from mildly annoying to catastrophic.

As the consequences of an error increase, the amount of pressure to find the cause also increases.

Often, pressure sometimes forces a software developer to fix one error and at the same time introduce
two more.

Debugging Approaches/Strategies:

1. Brute Force: Study the system for a larger duration in order to understand the system. It helps the
debugger to construct different representations of systems to be debugging depending on the need. A
study of the system is also done actively to find recent changes made to the software.
2. Backtracking: Backward analysis of the problem which involves tracing the program backward from
the location of the failure message in order to identify the region of faulty code. A detailed study of the
region is conducted to find the cause of defects.
3. Forward analysis of the program involves tracing the program forwards using breakpoints or print
statements at different points in the program and studying the results. The region where the wrong
outputs are obtained is the region that needs to be focused on to find the defect.
4. Using the past experience of the software debug the software with similar problems in nature. The
success of this approach depends on the expertise of the debugger.
5. Cause elimination: it introduces the concept of binary partitioning. Data related to the error occurrence
are organized to isolate potential causes.

Debugging Tools:

Debugging tool is a computer program that is used to test and debug other programs. A lot of public
domain software like gdb and dbx are available for debugging.

They offer console-based command-line interfaces.

Examples of automated debugging tools include code based tracers, profilers, interpreters, etc. Some of
the widely used debuggers are:

 Radare2
 WinDbg
 Valgrind

Difference between Debugging and Testing:

Debugging is different from testing. Testing focuses on finding bugs, errors, etc whereas debugging
starts after a bug has been identified in the software.

Testing is used to ensure that the program is correct and it was supposed to do with a certain minimum
success rate.

Testing can be manual or automated.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 There are several different types of testing like unit testing, integration testing, alpha and beta testing,
etc.

Debugging requires a lot of knowledge, skills, and expertise. It can be supported by some automated
tools available but is more of a manual process as every bug is different and requires a different
technique, unlike a pre-defined testing mechanism.

PRODUCT METRICS

Product Metrics

In software development process, a working product is developed at the end of each successful phase.
Each product can be measured at any stage of its development.

Metrics are developed for these products so that they can indicate whether a product is developed
according to the user requirements.

If a product does not meet user requirements, then the necessary actions are taken in the respective
phase.

Product metrics help software engineer to detect and correct potential problems before they result in
catastrophic defects.

In addition, product metrics assess the internal product attributes in order to know the efficiency of the
following.

 Analysis, design, and code model

 Potency of test cases
 Overall quality of the software under development.

Various metrics formulated for products in the development process are listed below.

 Metrics for analysis model: These address various aspects of the analysis model such as system
functionality, system size, and so on.
 Metrics for design model: These allow software engineers to assess the quality of design and include
architectural design metrics, component-level design metrics, and so on.
 Metrics for source code: These assess source code complexity, maintainability, and other
characteristics.
 Metrics for testing: These help to design efficient and effective test cases and also evaluate the
effectiveness of testing.
 Metrics for maintenance: These assess the stability of the software product.

Software Quality:

Traditionally, a high-quality product is outlined in terms of its fitness of purpose.

That is, a high-quality product wills specifically what the users need it to try to.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 For code merchandise, the fitness of purpose is typically taken in terms of satisfaction of the wants
arranged down within the SRS document.

Though “fitness of purpose” could be a satisfactory definition of quality for several merchandises like an
automobile, a table fan, a grinding machine, etc. – for code merchandise, “fitness of purpose” isn’t a
completely satisfactory definition of quality.

To convey associate degree example, think about software that’s functionally correct.

It performs all functions as laid out in the SRS document. But, it has an associate degree virtually
unusable program.

Despite the fact that it should be functionally correct, we have a tendency to can’t think about it to be a
high-quality product.

Another example is also that of a product that will everything that the users need however has associate
degree virtually incomprehensible and not maintainable code.

Therefore, the normal construct of quality as “fitness of purpose” for code merchandise isn’t totally
satisfactory.

The modern read of high-quality associates with software many quality factors like the following:

 Portability:
A software is claimed to be transportable, if it may be simply created to figure in several package
environments, in several machines, with alternative code merchandise, etc.
 Usability:
A software has smart usability if completely different classes of users (i.e. each knowledgeable and
novice users) will simply invoke the functions of the merchandise.
 Reusability:
A software has smart reusability if completely different modules of the merchandise will simply be
reused to develop new merchandise.
 Correctness:
A software is correct if completely different needs as laid out in the SRS document are properly
enforced.
 Maintainability:
A software is reparable, if errors may be simply corrected as and once they show up, new functions may
be simply added to the merchandise, and therefore the functionalities of the merchandise may be simply
changed, etc

Metrics for the Analysis Model

There are only a few metrics that have been proposed for the analysis model.

However, it is possible to use metrics for project estimation in the context of the analysis model.

These metrics are used to examine the analysis model with the objective of predicting the size of the
resultant system.

Size acts as an indicator of increased coding, integration, and testing effort; sometimes it also acts as an
indicator of complexity involved in the software design.
CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 Function point and lines of code are the commonly used methods for size estimation.

Function Point (FP) Metric

The function point metric, which was proposed by A.J Albrecht, is used to measure the functionality
delivered by the system, estimate the effort, predict the number of errors, and estimate the number of
components in the system.

Function point is derived by using a relationship between the complexity of software and the
information domain value.

Information domain values used in function point include the number of external inputs, external
outputs, external inquire, internal logical files, and the number of external interface files.

Lines of Code (LOC)

Lines of code (LOC) are one of the most widely used methods for size estimation.

LOC can be defined as the number of delivered lines of code, excluding comments and blank lines.

It is highly dependent on the programming language used as code writing varies from one programming
language to another.

For example, lines of code written (for a large program) in assembly language are more than lines of
code written in C++.

From LOC, simple size-oriented metrics can be derived such as errors per KLOC (thousand lines of
code), defects per KLOC, cost per KLOC, and so on. LOC has also been used to predict program
complexity, development effort, programmer performance, and so on.

For example, Hasltead proposed a number of metrics, which are used to calculate program length,
program volume, program difficulty, and development effort.

Metrics for the Design Model of the Product:

Metrics simply measures quantitative assessment that focuses on countable values most commonly used
for comparing and tracking performance of system. Metrics are used in different scenarios like
analyzing model, design model, source code, testing, and maintenance. Metrics for design modeling
allows developers or software engineers to evaluate or estimate quality of design and include various
architecture and component-level designs.

Metrics by Glass and Card:

In designing a product, it is very important to have efficient management of complexity. Complexity

itself means very difficult to understand. We know that systems are generally complex as they have
many interconnected components that make it difficult to understand. Glass and Card are two scientists
who have suggested three design complexity measures. These are given below:

1. Structural Complexity –

Structural complexity depends upon fan-out for modules. It can be defined as :

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 S(k) = f2out(k)

Where fout represents fan-out for module k (fan-out means number of modules that are subordinating
module k).

2. Data Complexity –

Data complexity is complexity within interface of internal module. It is size and intricacy of data. For
some module k, it can be defined as:

D(k) = tot_var(k) / [fout(k)+1]

Where tot_var is total number of input and output variables going to and coming out of module.

3. System Complexity –

System complexity is combination of structural and data complexity. It can be denoted as:

Sy(k) = S(k)+D(k)

When structural, data, and system complexity get increased, overall architectural complexity also gets
increased.

Complexity metrics –

Complexity metrics are used to measure complexity of overall software.

The computation if complexity metrics can be done with help of a flow graph. It is sometimes called
cyclomatic complexity.

The cyclomatic complexity is a useful metric to indicate complexity of software system.

Without use of complexity metrics, it is very difficult and time-consuming to determine complexity in
designing products where risk cost emanates.

Even continuous complexity analysis makes it difficult for project team and management to solve
problem.

Measuring Software complexity leads to improve code quality, increase productivity, meet architectural
standards, reduce overall cost, increases robustness, etc.

To calculate cyclomatic complexity, following equation is used:

Cyclomatic complexity= E - N + 2

Where, E is total number of edges and N is total number of nodes.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

Example –
In diagram given below, you can see number of edges and number of nodes.

So, the Cyclomatic complexity can be calculated as –

Given,

E = 10,

N=8

So,

Cyclomatic complexity

=E-N+2

= 10 – 8 + 2

=4

Metrics for Coding

Halstead proposed the first analytic laws for computer science by using a set of primitive measures,
which can be derived once the design phase is complete and code is generated.

These measures are listed below.

nl = number of distinct operators in a program

n2 = number of distinct operands in a program

N1 = total number of operators

N2= total number of operands.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

By using these measures, Halstead developed an expression for overall program length, program
volume, program difficulty, development effort, and so on.

Program length (N) can be calculated by using the following equation.

N = n1log2nl + n2 log2n2.

Program volume (V) can be calculated by using the following equation.

V = N log2 (n1+n2).

Note that program volume depends on the programming language used and represents the volume of
information (in bits) required to specify a program.

Volume ratio (L)can be calculated by using the following equation.

L = Volume of the most compact form of a program/Volume of the actual program

Where, value of L must be less than 1. Volume ratio can also be calculated by using the following
equation.

L = (2/n1)* (n2/N2).

Program difficulty level (D) and effort (E)can be calculated by using the following equations.

D = (n1/2)*(N2/n2).

E = D * V.

Metrics for Software Testing

Majority of the metrics used for testing focus on testing process rather than the technical characteristics
of test.

Generally, testers use metrics for analysis, design, and coding to guide them in design and execution of
test cases.

Function point can be effectively used to estimate testing effort.

Various characteristics like errors discovered, number of test cases needed, testing effort, and so on can
be determined by estimating the number of function points in the current project and comparing them
with any previous project.

Metrics used for architectural design can be used to indicate how integration testing can be carried out.

In addition, cyclomatic complexity can be used effectively as a metric in the basis-path testing to
determine the number of test cases needed.

Halstead measures can be used to derive metrics for testing effort. By using program volume (V) and
program level (PL), Halstead effort (e) can be calculated by the following equations.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA

 e = V/ PL

Where

PL = 1/ [(n1/2) * (N2/n2)] … (1)

For a particular module (z), the percentage of overall testing effort allocated can be calculated by the
following equation.

Percentage of testing effort (z) = e(z)/∑e(i)

Where, e(z) is calculated for module z with the help of equation (1).

Summation in the denominator is the sum of Halstead effort (e) in all the modules of the system.

For developing metrics for object-oriented (OO) testing, different types of design metrics that have a
direct impact on the testability of object-oriented system are considered.

While developing metrics for OO testing, inheritance and encapsulation are also considered.

A set of metrics proposed for OO testing is listed below.

 Lack of cohesion in methods (LCOM): This indicates the number of states to be tested. LCOM
indicates the number of methods that access one or more same attributes.

The value of LCOM is 0, if no methods access the same attributes.

As the value of LCOM increases, more states need to be tested.

 Percent public and protected (PAP): This shows the number of class attributes, which are public or
protected.

Probability of adverse effects among classes increases with increase in value of PAP as public and
protected attributes lead to potentially higher coupling.

 Public access to data members (PAD): This shows the number of classes that can access attributes of
another class.

Adverse effects among classes increase as the value of PAD increases.

 Number of root classes (NOR): This specifies the number of different class hierarchies, which are
described in the design model.

Testing effort increases with increase in NOR.

 Fan-in (FIN): This indicates multiple inheritances.

If value of FIN is greater than 1, it indicates that the class inherits its attributes and operations from
many root classes.

Note that this situation (where FIN> 1) should be avoided.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA
 Metrics for Software Maintenance

For the maintenance activities, metrics have been designed explicitly. IEEE has proposed Software
Maturity Index (SMI), which provides indications relating to the stability of software product.

For calculating SMI, following parameters are considered.

 Number of modules in current release (MT)

 Number of modules that have been changed in the current release (F e)
 Number of modules that have been added in the current release (Fa)
 Number of modules that have been deleted from the current release (F d)

Once all the parameters are known, SMI can be calculated by using the following equation.

SMI = [MT– (Fa+ Fe + Fd)]/MT.

Note that a product begins to stabilize as 8MI reaches 1.0.

SMI can also be used as a metric for planning software maintenance activities by developing empirical
models in order to know the effort required for maintenance.

CSE III YEAR I SEM SOFTWARE ENGINEERING NALLA CHAITANYA