An API example and a test matrix
# Test Scenario Category Test Action Test Action Description
Category
1 Basic positive tests (happy
paths)
Execute API call with valid Validate 1. All requests should return 2XX HTTP status
required parameters status code: code
2. Returned status code is according to spec:
– 200 OK for GET requests
– 201 for POST or PUT requests creating a new
resource
– 200, 202, or 204 for a DELETE operation and
so on
Validate 1. Response is a well-formed JSON object
payload:
2. Response structure is according to data
model (schema validation: field names and
field types are as expected, including nested
objects; field values are as expected; non-
nullable fields are not null, etc.)
Validate 1. For GET requests, verify there is NO STATE
state: CHANGE in the system (idempotence)
2. For POST, DELETE, PATCH, PUT
operations
– Ensure action has been performed correctly in
the system by:
– Performing appropriate GET request and
inspecting response
– Refreshing the UI in the web application and
verifying new state (only applicable to manual
testing)
An API example and a test matrix
Validate Verify that HTTP headers are as expected,
headers: includingcontent-type,connection,cache-
control,expires,
access-control-allow-origin,keep-
alive, HSTS, and other standard header fields
– according to spec.
Verify that information is NOT leaked via
headers (e.g.X-Powered-Byheader is not sent
to user).
Performance Response is received in a timely manner (within
sanity: reasonable expected time) — as defined in the
test plan.
2 Positive + optional
parameters
Execute API call with valid
required parameters AND
valid optional parameters
Run same tests as in #1, this
time including the endpoint’s
optional parameters (e.g.,
filter, sort, limit, skip, etc.)
Validate As in #1
status code:
Validate Verify response structure and content as in #1.
payload:
In addition, check the following parameters:
– filter: ensure the response is filtered on the
specified value.
– sort: specify field on which to sort, test
ascending and descending options. Ensure the
response is sorted according to selected field
and sort direction.
– skip: ensure the specified number of results
An API example and a test matrix
from the start of the dataset is skipped
– limit: ensure dataset size is bounded by
specified limit.
– limit + skip: Test pagination
Check combinations of all optional fields (fields
+ sort + limit + skip) and verify expected
response.
Validate As in #1
state:
Validate As in #1
headers:
Performance As in #1
sanity:
3 Negative testing – valid
input
Execute API calls with valid
input that attempts illegal
operations. i.e.:
– Attempting to create a
resource with a name that
already exists (e.g., user
configuration with the same
name)
– Attempting to delete a
resource that doesn’t
exist (e.g., user configuration
with no such ID)
– Attempting to update a
resource with illegal valid data
(e.g., rename a configuration
to an existing name)
An API example and a test matrix
– Attempting illegal operation
(e.g., delete a user
configuration without
permission.)
And so forth.
Validate 1. Verify that an erroneous HTTP status code is
status code: sent (NOT 2XX)
2. Verify that the HTTP status code is in
accordance with error case as defined in spec
Validate 1. Verify that error response is received
payload:
2. Verify that error format is according to spec.
e.g., error is a valid JSON object or a plain
string (as defined in spec)
3. Verify that there is a clear, descriptive error
message/description field
4. Verify error description is correct for this error
case and in accordance with spec
Validate As in #1
headers:
Performance Ensure error is received in a timely manner
sanity: (within reasonable expected time)