SOFTWARE TESTING
Exam-Ready Answers | BCA University Examination
Q1. Explain testing objectives.
Definition (Frame 9.1): Software testing is a formal process carried out by a specialized testing team
in which a software unit, several integrated software units or an entire software package are
examined by running the programs on a computer, according to approved test procedures on
approved test cases.
Direct Objectives (Frame 9.2):
• To identify and reveal as many errors as possible in the tested software.
• To bring the tested software, after correction of errors and retesting, to an acceptable level of quality.
• To perform the required tests efficiently and effectively, within budgetary and scheduling limitations.
Indirect Objective:
• To compile a record of software errors for use in error prevention (by corrective and preventive actions).
Note: The goal "to prove that the software is ready" is intentionally NOT included. As Myers (1979) stated: "If
your goal is to show the absence of errors, you won't discover many." The phrase "acceptable level of quality" is
used because bug-free software is still a utopian aspiration.
Q2. Discuss the differences between the various testing strategies, their advantages and
disadvantages.
Two Basic Testing Strategies:
1. Big Bang Testing
The entire software is tested as a whole once the completed package is available. No modular or step-by-step
testing is done.
Disadvantages:
• Identification of errors is very difficult when the entire software is treated as one unit.
• Despite vast resources invested, error detection effectiveness is relatively low.
• Error correction is laborious and may affect multiple modules simultaneously.
• Testing resource estimation and scheduling become fuzzy and unreliable.
2. Incremental Testing
Software is tested module by module (Unit Tests), then in groups (Integration Tests), and finally as a whole
(System Test). Incremental testing uses two strategies: Bottom-Up and Top-Down.
Advantages:
• Higher percentages of errors are identified because testing is done on small modules.
• Error identification and correction is simpler and requires fewer resources.
• Errors are caught early, preventing migration to a later, more complex stage.
Disadvantages:
• Large quantity of programming resources required for preparation of stubs and drivers.
• Numerous testing operations must be carried out for the same program.
Conclusion: It is generally accepted that incremental testing should be preferred.
Bottom-Up vs. Top-Down (Comparison Table):
Aspect Bottom-Up Testing Top-Down Testing
Order Lowest level modules tested first; Main (highest level) module tested
main module tested last. first; lowest level modules tested
last.
Main Advantage Relative ease of performance. Entire program can be
demonstrated early; supports early
identification of design errors.
Main Disadvantage Program as a whole is observed Preparing stubs is relatively
only at the last stage. difficult; analysis of results is
harder.
Requires Drivers (substitute upper-level Stubs (dummy lower-level
modules). modules).
• Stub: Dummy module that replaces an unavailable lower-level module. Used in Top-Down testing.
• Driver: Substitute upper-level module that passes test data to the tested module. Used in Bottom-Up
testing.
Q3. Describe the concepts of black box testing and white box testing, and discuss their
advantages and disadvantages.
Black Box Testing (Functionality Testing):
IEEE Definition: Testing that ignores the internal mechanism of a system or component and focuses
solely on the outputs generated in response to selected inputs and execution conditions.
Identifies bugs only according to software malfunctioning as revealed in its erroneous outputs. The internal path
of calculations is disregarded.
Advantages:
• Allows the tester to carry out almost all test classes, most of which can only be implemented by black box
tests.
• Requires considerably fewer resources than white box testing for the same software package.
Disadvantages:
• Coincidental aggregation of errors may produce a correct output, preventing error detection.
• No control of line coverage; a substantial proportion of code lines may remain untested.
• Cannot test the quality of coding and its adherence to coding standards.
White Box Testing (Structural / Glass Box Testing):
IEEE Definition: Testing that takes into account the internal mechanism of a system or component.
Examines internal calculation paths in order to identify bugs.
Verifies every program statement. Enables data processing and calculations correctness tests, software
qualification tests, maintainability tests, and reusability tests.
Advantages:
• Direct statement-by-statement checking of code verifies that algorithms were correctly defined and coded.
• Provides line coverage follow-up with lists of lines not yet executed, so testers can initiate additional test
cases.
• Ascertains quality of coding work and adherence to coding standards.
Disadvantages:
• Requires vast resources, much above those needed for black box testing.
• Cannot test software performance in terms of availability, reliability, load durability, and similar factors.
Note: Due to cost considerations, most testing currently carried out is black box testing. White box testing is
limited to very high-risk software modules.
Q4. Describe the various types of black box tests.
Black box tests are classified into three groups based on McCall's factor model:
Group 1: Operation Factor Testing Classes
1. Output Correctness Tests: The most resource-consuming test class. Uses equivalence class partitioning to
divide inputs into valid and invalid equivalence classes (EC) to improve efficiency and coverage.
2. Documentation Tests: Tests user manuals, installation manuals, and programmer manuals for
completeness, correctness, and style.
3. Availability (Reaction Time) Tests: Tests the time needed to obtain requested information or for firmware
to react, under regular and maximum load conditions.
4. Reliability Tests: Tests failures to provide service over time: average time between failures, average
recovery time, and average downtime per month.
5. Stress Tests: Load tests check functional performance under maximum operational load. Durability tests
check performance under physically extreme conditions such as high temperature, humidity, and electrical
failures.
6. Software System Security Tests: Tests access control, database backup and recovery, and transaction
logging. Hackers are sometimes invited to join tester teams for high-security systems.
7. Training Usability Tests: Measures resources (hours of training) needed for a new employee to reach a
defined level of system acquaintance.
8. Operational Usability Tests: Focuses on operator productivity; tests aspects of the system that affect
regular user performance. Can be performed manually or by automated follow-up software.
Group 2: Revision Factor Testing Classes
9. Maintainability Tests: Tests system structure, programmer's manual, and internal documentation
compliance with standards for future maintenance support.
10. Flexibility Tests: Tests software characteristics that support flexibility, such as modular structure and
parametric options.
11. Testability Tests: Tests special features added to programs that assist testers, such as intermediate
checkpoints, predefined log files, and diagnostic tools.
Group 3: Transition Factor Testing Classes
12. Portability Tests: Verifies that the software system is operable in different environments such as
different operating systems and hardware platforms.
13. Reusability Tests: Examines whether packaging and documentation of programs listed for reuse
conform to the reusable software library standards.
14. Software Interoperability Tests: Tests software capabilities for interfacing with other software packages
or equipment to operate jointly as one system.
15. Equipment Interoperability Tests: Tests firmware interfacing with other equipment units and software
packages throughout the system.
Q5. List the main types of automated software tests.
The main types of automated tests (Frame 10.5) are:
16. Code Auditing: Performs automated qualification testing. Checks compliance of code to specified
standards and procedures. Reports deviations and provides a statistical summary. Verifies module size,
loop/subroutine nesting levels, naming conventions, and unreachable code lines.
17. Coverage Monitoring: Produces reports about line coverage achieved when implementing a given test
case file. Output includes percentage of lines covered and listings of uncovered lines. A vital tool for white
box tests.
18. Functional Tests: Replaces manual black box correctness tests. Test cases are recorded in a test case
database and run by a computer program. After corrections, regression tests are re-run automatically using
the same test case database with minimal manpower.
19. Load Tests: Based on simulated scenarios of maximum load situations the software will confront. Virtual
users and virtual events emulate real user behavior. Measures expected reaction times across a range of
loads to evaluate performance under daily use conditions.
20. Test Management: Provides comprehensive follow-up and reporting of testing and error correction.
Features include test plan preparation, listing of detected errors, correction schedule tracking, error
tracking through detection and regression tests, and summary reports.
Q6. Discuss the advantages and disadvantages of automated computerized testing compared
to manual testing.
Advantages of Automated Testing:
21. Accuracy and Completeness of Performance: All tests and test cases are carried out completely and
accurately. Manual testing suffers from tester weariness and low concentration.
22. Accuracy of Results Log and Summary Reports: Automated tests are programmed for accurate error
reporting. Manual testers may overlook errors in their logs.
23. Comprehensive Information: Test data stored in a database makes queries and reports far more
available than after manual tests, enhancing input for preventive and corrective actions.
24. Few Manpower Resources for Test Execution: Manual testing is a major consumer of manpower.
Automated testing requires far fewer resources for actual test execution.
25. Shorter Testing Duration: Automated tests run far faster than manual tests and can run uninterrupted
24 hours a day, 7 days a week.
26. Complete Regression Tests: Minimal time and manpower make it possible to rerun the complete test
set, substantially reducing the risk of undetected errors introduced during corrections.
27. Test Classes Beyond Manual Scope: Enables performance of load tests, availability tests, and efficiency
tests for medium and large systems, which are almost impossible to perform manually.
Disadvantages of Automated Testing:
28. High Investment in Package Purchasing and Training: Organizations must invest in software packages
and extensive (and expensive) staff training.
29. High Package Development Cost: When available packages do not suit requirements, custom-made
packages must be developed at high cost.
30. High Manpower for Test Preparation: Resources required for preparing automated test procedures are
usually substantially higher than for manual procedures.
31. Considerable Testing Areas Left Uncovered: Automated packages do not cover the entire variety of
development tools and applications, forcing a mix of manual and automated testing.
Empirical Finding (Dustin et al., 1999):
Automated test preparation consumed 65% more resources than manual preparation on average, but
automated test execution was 18.7 times faster. The break-even point (N) was approximately 2 regression runs,
meaning automated testing is preferred whenever more than one regression test run is required.
Q7. Explain the implementation of alpha and beta site test and discuss their advantages and
disadvantages.
Alpha and beta site tests are used to obtain quality feedback from potential users of COTS software packages.
They replace the customer acceptance test, which is impractical under commercial software development
conditions. However, they should never replace formal software tests performed by the developer.
Alpha Site Tests:
Tests of a new software package performed at the developer's site. The customer applies the new software to
their own organizational requirements, examining the package from angles not expected by the testing team.
Errors identified are reported to the developer.
Beta Site Tests:
More commonly applied than alpha site tests. An advanced version of the software package is offered free of
charge to one or more potential users at their own sites (beta sites). Users report all errors found during trials or
regular usage. Some developers involve hundreds or thousands of participants.
Advantages (apply to both):
• Identification of Unexpected Errors: Users examine software differently and reveal errors professional
testers rarely identify.
• Wider Population in Search of Errors: Wide range of participants contributes experience and potential for
revealing hidden errors.
• Low Costs: Participants are not paid; the only costs are the package and its delivery free of charge.
Disadvantages (apply to both):
• Lack of Systematic Testing: Participants are not obligated to prepare orderly reports and leave many
applications untested.
• Low Quality Error Reports: Participants are not professional testers; reports are often faulty and error
conditions are difficult to reconstruct.
• Difficult to Reproduce Test Environment: Uncontrolled testing environment creates difficulties in
identifying causes of reported errors.
• Much Effort Required to Examine Reports: High investment of time and resources needed due to
repetitions and low-quality reporting.
Implementation Tip: Initiate alpha site testing first. Delay beta site testing until alpha tests are completed and
analyzed, as premature beta testing of immature software can cause highly negative publicity.
Q8. List the infrastructure tools that support maintenance quality assurance.
The major SQA maintenance infrastructure tools are:
32. Maintenance Procedures and Work Instructions: Applied mainly for corrective maintenance and user
support activities. Covers remote and on-site handling of failure requests, user support services, QA control
of correction activities, customer satisfaction surveys, and certification of maintenance team members.
33. Supporting Quality Devices: Specialized devices developed by the maintenance department including:
checklists for locating failure causes, templates for reporting how failures were solved, and checklists for
preparing mini testing procedure documents.
34. Training and Certification of Maintenance Teams: Special training and certification are crucial for
corrective maintenance teams. Training plans must address staffing needs during peak loads and personnel
replacement at short notice. Rigorous programs are required to maintain contracted service levels.
35. Preventive and Corrective Actions: Records of software failures and user support requests lead to
preventive and corrective actions. The Corrective Action Board (CAB) reviews changes in service patterns,
increased repair times, and increased failure rates to drive process improvement.
36. Configuration Management: Maintenance teams are most dependent on configuration management.
Key applications: (a) Failure Repairs: provides current code and version information for the customer's
installed software. (b) Group Replacement: supports planning and rollout of updated software versions to
all customers using the same version.
37. Software Maintenance Documentation and Quality Records: Documentation is prepared to supply data
for preventive and corrective actions, support handling of future failure reports, and provide evidence in
response to customer claims and complaints.
Q9. List the main managerial tools for controlling software maintenance quality and explain
their importance.
Managerial control SQA tools for maintenance are designed to improve control of maintenance activities and
provide early alarms for reduced quality of service. The three main tools are:
38. Performance Controls for Corrective Maintenance Services: Implemented through periodic reporting,
scheduled staff meetings, visits to maintenance support centers, and analysis of maintenance metrics
reports. Yield management alarms such as: increased resource utilization, decreased rate of remote
repairs, increased on-site repairs at distant locations, increased schedule failure rates, increased faulty
repair rates, and lower customer satisfaction scores.
39. Quality Metrics for Software Maintenance: Used to identify trends in maintenance efficiency,
effectiveness, and customer satisfaction. Provide the quantitative basis for decisions on: estimating
resource requirements, comparing operational methods, initiating preventive and corrective actions, and
preparing proposals for new or adjusted maintenance services.
40. Quality Costs of Software Maintenance: Classified into six classes: (1) Prevention costs: training and
instruction of maintenance team. (2) Appraisal costs: error detection through reviews and surveys. (3)
Managerial preparation and control costs: preparation of maintenance plans. (4) Internal failure costs:
corrections initiated before customer complaints. (5) External failure costs: corrections initiated by
customer complaints. (6) Managerial failure costs: damages resulting from staff shortage or inadequate
task organization.
Importance:
These tools assist management in decisions regarding investment in service improvement, development of
improved software versions, comparison of operational methods, and estimation of resource requirements for
maintenance services.
Q10. List the types of external participants, and explain the benefits they provide to the
contractor.
External participants contribute to a project but are not contractors or the contractor's partners. Three main
types exist:
41. Subcontractors (Outsourcing Organizations): Undertake to carry out parts of a project. Benefits: (a)
Staff availability: provides additional manpower. (b) Special expertise: offers specialized skills not available
in-house. (c) Low prices: performs tasks at a lower cost than in-house development.
42. Suppliers of COTS Software and Reused Software Modules: Provide ready-to-use software components.
Benefits: (a) Savings in development resources and budgeted funds. (b) Shorter timetable as ready-made
software eliminates development time. (c) Higher quality software, since components are already tested
and corrected by developers and previous customers.
43. The Customer as Participant: The customer performs parts of the project themselves. Benefits: (a)
Applies the customer's special expertise. (b) Meets commercial or security needs. (c) Keeps internal
development staff occupied. (d) Prevents future maintenance problems. Note: drawbacks to the customer-
supplier relationship are outweighed by the customer's contributions.
Q11. Describe the risks for the contractor associated with turning to external participants.
The main risks associated with introducing external participants are:
44. Delays in Completion of the Project: External participants (especially subcontractors and customer
teams) are often late in supplying their parts. Control over their obligations is often loose, causing late
recognition of delays and leaving insufficient time for reorganization.
45. Low Quality of Project Parts: Quality problems include: (a) Defects: higher than expected number of
defects, often more severe than expected. (b) Non-standard coding and documentation: violations of style
and structure. This causes difficulties in testing and maintenance phases, and extra correction time can
cause project delays even when parts are delivered on time.
46. Future Maintenance Difficulties: Two difficult maintenance situations arise: (a) The contractor is solely
responsible for maintenance but faces incomplete and non-standard code from external participants,
causing lower-quality service and higher costs. (b) If multiple organizations supply maintenance, each takes
limited responsibility, forcing the customer to search for the body responsible for each specific failure.
Damages grow significantly in such multi-maintainer situations.
47. Loss of Control Over Project Parts: Communication with external participant teams may be interrupted
for weeks, preventing accurate assessment of project progress. Alerts about difficulties and staff shortages
reach the contractor belatedly, drastically reducing possibilities for timely solutions.
Q12. List the main contributions of real CASE tools to product quality.
CASE (Computer Aided Software Engineering) tools contribute to software product quality by reducing errors
introduced in each development phase. The contributions of real CASE tools against the nine causes of software
errors (Section 2.3) are as follows:
Cause of Software Error Contribution of Real CASE Tools
1. Faulty Requirements Definition Almost no contribution. Computerized verification of
requirements consistency is rarely possible.
2. Client-Developer Communication Failures Almost no contribution. Communication failures can
only be flagged when a change is inconsistent with
repository information.
3. Deliberate Deviations from Requirements High contribution. Repository-based tools identify
deviations from recorded requirements as
inconsistent errors. Requirements tracing and cross-
Cause of Software Error Contribution of Real CASE Tools
referenced query tools also help.
4. Logical Design Errors High contribution. Re-engineering enables automated
generation of legacy system designs in the repository.
Repository use identifies design omissions and
inconsistencies.
5. Coding Errors Very high contribution. Automated code generation
achieves full consistency with design recorded in the
repository, eliminating coding errors entirely.
6. Non-compliance with Coding Instructions Very high contribution. Automated code generation
ensures full compliance with documentation and
coding standards, including standardization of
structure and style.
7. Shortcomings of the Testing Process High contribution. Automated testing tools perform
full regression and load testing. Repository cross-
referenced queries and consistency checks prevent
most software errors.
8. Procedure Errors High contribution. Software configuration
management controls versions, revisions, and
installations. Repository tools applied to corrections
and changes prevent most procedural errors.
9. Documentation Errors High contribution. Repository automatically generates
full and updated documentation prior to each
correction or change, preventing errors from
incomplete or inaccurate documentation.