■ Software Testing — Part 2
Black Box Testing & McCall's Quality Factors
Simple & Easy Study Guide
Introduction: What is Black Box Testing?
Black Box Testing means we test the software without looking at its internal code. We only care about:
Does it give the correct output for a given input?
Black box testing covers many quality-related test classes based on McCall's Quality Factor Model,
which organizes software quality into three groups:
Group Quality Factors Covered
Product Operation
Correctness, Availability, Reliability, Efficiency, Integrity, Usability
(How it runs day-to-day)
Product Revision
Maintainability, Flexibility, Testability
(How easy to fix/change)
Product Transition
Portability, Reusability, Interoperability
(How easy to move/reuse)
GROUP 1 — Product Operation Tests
1. Correctness — Documentation Tests
We often forget to test the documents that come with the software — but they are just as important as the
code itself! If the manual is wrong, users will be confused even if the software works perfectly.
■ Types of Documentation That Must Be Tested:
Document What It Contains Why Test It?
Functional Description Overview of what the software does Users decide if this software suits their needs based on this
Installation Manual How to install, hardware needs, setup instructions
Wrong instructions = software won't be installed correctly
User Manual How to use features, how to recover from Users
errors follow this daily — errors here cause real problems
Programmer Manual Code structure, algorithms, bug fixes, howDevelopers
to maintainuse this to fix bugs and improve the system
■ A Documentation Test Plan Must Check:
• Completeness: Are all required documents present and finished?
• Correctness: Are the instructions in the manual actually correct?
• Style & Editing: Is the document clear and does it follow standards?
■ Easy Example: Imagine a bank installs new ATM software. The user manual says: 'Press GREEN
button to confirm.' But the actual ATM has a BLUE confirm button. The software works fine — but the
manual is wrong! A documentation correctness test would catch this error before the ATMs go live.
2. Correctness — Availability Tests
Availability means: How fast does the system respond when a user asks for something? Also called
reaction time. For example, how quickly does a hospital system pull up a patient's records when a doctor
requests them?
Key Points:
• Tests must be done under normal load AND maximum/peak load conditions.
• Availability requirements are usually different for regular vs. maximum workloads.
• Availability tests are often combined with Stress/Load Tests.
■ Easy Example: A hospital's patient records system must respond within 2 seconds normally, and
within 5 seconds during peak hours (e.g., emergency situations with 500 doctors logged in). If it takes 15
seconds during peak load — it FAILS the availability test.
3. Reliability — Reliability Tests
Reliability means: How often does the system fail, and how fast does it recover? We measure reliability
using these three values:
• Average time between failures — how long the system runs before crashing
• Average recovery time — how long it takes to get back up after a crash
• Average downtime per month — total hours/minutes the system is unavailable monthly
■■ Important: These tests should only be done after computerized simulations have already been run
and the entire system is complete.
✈■ Easy Example: An airline booking system must have no more than 1 crash per month, and must
recover within 10 minutes if it does crash. If it crashes 5 times in a month and takes 2 hours to recover
— it fails reliability testing.
4. Efficiency — Stress Tests
Stress Tests check how the system behaves under very heavy load — much heavier than normal. There
are two types:
■ Type 1: Load Tests
Load tests check if the system can handle the maximum expected number of users or transactions.
Done AFTER the software is fully completed. Tests are run at loads higher than the requirement to
make sure there's a safety buffer.
■ Real Example from Slides — 'Music in the Air':
A music store chain runs an online service for price quotes and orders.
Normal weekday traffic: 10 orders/min, 20 price quotes/min
Peak Saturday traffic: 30 orders/min, 100 price quotes/min
Requirement spec (with future growth): 60 orders/min, 200 price quotes/min
Actual test load: 75 orders/min & 250 price quotes/min
Why test at 75 and 250? Because we test 25% higher than the spec to make sure the system won't fail
even if growth is faster than expected. If the system survives 75/250, it will definitely handle 60/200 in
real life.
■ Type 2: Durability Tests
Durability tests check if the system can work correctly over long periods of time under difficult real-world
conditions. Especially important for firmware (software embedded in hardware).
• Firmware examples: Weapon systems, vehicles, weather equipment
• Tested against: Extreme temperatures, dust, physical bumps (road vibration)
• Software systems: Tested against sudden power cuts, voltage spikes
■ Easy Example: A GPS system in a truck must work correctly in freezing winters (-20°C), hot
summers (+50°C), and on bumpy roads. Durability testing puts the hardware through all these
conditions to make sure the firmware doesn't crash.
5. Integrity — Security Tests
Security Tests check that unauthorized people cannot access the system or its data. Think of it like
testing the locks on a bank vault.
Security tests focus on:
• Access Control: Only authorized users (with correct passwords/roles) can get in
• Multi-level Access: Different users see different things (e.g., admin vs. regular user)
• Detecting Unauthorized Access: Can the system detect hacking attempts?
• Backup & Recovery: Can data be recovered after a security breach?
■ Easy Example: A banking app is tested by ethical hackers (penetration testers) who try to log in
without a password, access other users' accounts, or bypass the OTP verification. If they succeed —
the software fails the security test and must be fixed.
6. Usability Tests
Usability means: Is the software easy to learn and easy to use? There are two types of usability tests:
■ Type 1: Training Usability Tests
This measures: How many hours does a new employee need to become comfortable with the
system? The goal is to make sure the software doesn't require too much training time, especially when
there are many users.
■■ Type 2: Operational Usability Tests
This measures: How productive are employees when using the system day-to-day? If the software is
clunky and slow to operate, workers will be less productive — that's a usability failure.
✈■ Real Case Study from Slides — Bengal Tours Travel Agency:
Bengal Tours is a travel agency with 25 permanent staff. During spring/summer, they hire 20–25
temporary workers (students and senior citizens). They are considering buying a software called
'Tourplanex' for flight and vacation planning.
Question: Why are Training & Operational Usability tests especially important here?
Answer: Because temporary staff (who join seasonally) need to learn the software QUICKLY. There's
no time for long training. So the agency should:
• Phase 1 — Training Usability Test: Have the software company train some temporary employees
and measure how many hours/minutes they need to handle a typical customer inquiry.
• Phase 2 — Operational Usability Test: Measure how fast trained staff can complete real tasks (e.g.,
booking a flight in under 3 minutes). If it takes 15 minutes per booking, the software is too slow and will
hurt the agency's productivity during peak season.
GROUP 2 — Product Revision Tests
These tests check how easy it is to change, fix, or improve the software after it's built.
7. Maintainability Tests
Maintainability = How easy is it to fix bugs or update the software later? These tests check:
• Does the system structure follow proper coding standards?
• Is the programmer's manual complete and well-written?
• Are there enough comments and documentation inside the code?
■ Easy Example: A developer leaves a company. Another developer must fix a bug in the code. If there
are no comments in the code and no programmer manual, fixing the bug could take weeks. Good
maintainability means a new developer can understand and fix things quickly.
8. Flexibility Tests
Flexibility = How easily can the software be changed to meet new requirements? Flexibility tests check
that the software has a good modular structure — meaning it's built from separate, independent pieces
that can be updated without breaking everything else.
■ Easy Example: An e-commerce website adds a new 'Buy Now with Crypto' payment option. If the
payment module is flexible and modular, the developer just adds the new crypto module without
touching any other part. If not flexible, adding one feature might break the entire checkout system.
9. Testability Tests
Testability = Does the software have built-in features that make it easy to test? Good software includes
special testing aids such as:
• Intermediate result checkpoints — view results at specific points during processing
• Predefined log files — automatically record what the system does for review
• Diagnostic tools — automatically detect and report warning conditions
■ Easy Example: An electricity plant's control software automatically logs every reading every 5
seconds. If something goes wrong, engineers can check the log to see exactly what happened. This
built-in logging makes the system highly testable — problems are easy to diagnose and fix.
■ Key Point: Testability features must be written into the software requirements document from the
beginning — they should NOT be added as an afterthought.
GROUP 3 — Product Transition Tests
These tests check how well the software can be moved to new environments or reused in other
projects.
10. Portability Tests
Portability = Can the software run on different operating systems, hardware, or environments?
Portability tests verify that the software works correctly across:
• Different operating systems (Windows, Linux, macOS)
• Different hardware specifications (old vs. new computers)
• Different communication equipment and network standards
■ Easy Example: A company builds a web app that works perfectly on Chrome on Windows. Portability
tests check: Does it also work on Safari on Mac? On Firefox? On a mobile browser? On a slow internet
connection? Each environment is a portability test.
11. Reusability Tests
Reusability = Can parts of this software (modules, components) be used again in future projects?
This is especially important in object-oriented programming. Reusability tests check:
• Are reusable modules properly packaged and documented?
• Do they follow reusability standards?
• Are they truly independent so they can work in another project?
■■ Easy Example: A company builds a Payment Gateway module for one e-commerce site. If built
with reusability in mind, the same module can be plugged into their next e-commerce project without
rewriting it. Reusability tests verify this module truly works independently.
12. Interoperability Tests
Interoperability = Can the software communicate and work with OTHER software systems?
Modern software rarely works alone — it must send and receive data from other systems. Interoperability
tests check that this data exchange works correctly using international standards.
■ Easy Example: A hospital management system must send patient data to:
• The pharmacy software (for prescriptions)
• The insurance company software (for billing)
• The lab equipment (for test results)
Interoperability tests verify that the hospital system correctly sends/receives data from all these different
systems, following global data-exchange standards (like HL7 for healthcare).
■ Complete Quick-Reference Summary
# Test Type Simple Question It Answers Group
1 Documentation Are all manuals correct and complete? Operation
2 Availability Does the system respond fast enough? Operation
3 Reliability How often does it crash and how fast does it recover? Operation
4 Load (Stress) Can it handle maximum + extra traffic? Operation
5 Durability Does it survive harsh conditions over time? Operation
6 Security Can unauthorized people be kept out? Operation
7 Training Usability How long to train a new user? Operation
8 Operational Usability How productive are users with this software? Operation
9 Maintainability Can developers easily fix and update it? Revision
10 Flexibility Can it be changed without breaking things? Revision
11 Testability Does it have built-in testing aids? Revision
12 Portability Does it run on different systems/OS? Transition
13 Reusability Can its parts be reused in other projects? Transition
14 Interoperability Can it communicate with other software? Transition
■ Remember: Black Box Testing = Testing WHAT the software does, not HOW it does it. McCall's model
covers all quality aspects — from daily operation to long-term maintenance and future reuse!