Verification of Digital Systems

Vinay Reddy
Department of Electronics & Communication Engineering

1
VERIFICATION OF DIGITAL SYSTEMS

Chapter 9
Functional Coverage

Vinay Reddy
Department of Electronics and Communication Engineering
2
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 Functional coverage is a user-defined metric that measures how much of the design
specification, as enumerated by features in the test plan, has been exercised.

 What is a test?

• A test defines a specific set of parameters for data & configuration

• Generating random test = randomizing stimuli, apply them to the design, check if it
behaves as expected

3
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Diff b/w Code & Functional coverage

Code coverage tells you how much lines of your code
module dff (clk,reset,d,q,q_bar); are exercised.
input clk,reset,d; Tells what portion of the code is executed.
output q,q_bar;
reg q,q_bar;
always @ (posedge clk or posedge reset)
begin
q <= d;
q_bar <= !q;
end
endmodule

4
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage  The covergroup construct is a user-defined type.

Covergroup
covergroup cov_grp_name ( [coverage_event] );

coverpoint_name : coverpoint var_name1;

coverpoint_name : coverpoint var_name2;

cross coverpoint_name,coverpoint_name;
endgroup

5
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage  The covergroup construct is a user-defined type.

Covergroup

6
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

7
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

8
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage Edit code - EDA Playground

9
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

16 address options

10
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

16 address options

When bin count value is less than the range ?

11
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

What happens if single bin is allowed for multiple values?

12
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

13
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Note: we have to create user-defined bins to monitor functional coverage 14

VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 The covergroup construct is a user-defined type.

 The type definition is written once, and multiple instances of that type can be created in
different contexts.
 Similar to a class, once defined, a covergroup instance can be created via the new()
operator.
 A covergroup can be defined in a package, module,program,interface, checker, or class.

 Types of Covergroups :

 Standalone
 Embeded covergroup

15
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

16
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Covergroup definition is not a part of class

Create a handle
Create the object and use it

17
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

18
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Covergroup definition is part of class

19
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Covergroup definition is part of class

Custom constructor is compulsory
Taking handle for embedded covergroup is a compile error

20
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

21
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

22
VERIFICATION OF DIGITAL SYSTEMS

Chapter 9
Functional Coverage
Textbook Part

Vinay Reddy
Department of Electronics and Communication Engineering
23
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Coverage
What is Coverage?
Coverage is a generic term for measuring progress to complete design verification.

How is Coverage done?

The coverage tools gather information during a simulation and then postprocess it to produce a
coverage report.

Why do we need Coverage?

Coverage holes can be identified in the report and then existing tests can be modified or new
tests can be created to fill the holes. 24
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Coverage Types

 Code Coverage
 Functional Coverage
 Bug rate
 Assertion Coverage

25
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Code Coverage

Can measure:

 how many lines of code have been executed (line coverage)

 which paths through the code and expressions have been executed (path coverage)

 which single bit variables have had the values 0 or 1 (toggle coverage)

 which states and transitions in a state machine have been visited (FSM Coverage)

26
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 Code coverage measures how thoroughly the tests have exercised the
“implementation” of the design specification, and not the verification plan.

 Cannot catch missing features.

 Reset logic missing in the above D-FF specification. However, code coverage
report would show full coverage. 27
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Functional Coverage

 Functional coverage is a measure of what functionalities/features of the design

have been exercised by the tests.

 This can be useful in constrained random verification (CRV) to know what features
have been covered by a set of tests.

 For example, the verification plan for the D-FF discussed earlier would mention not
only its data storage but also how it resets to a known state. 28
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Figure 9.1 shows the feedback loop to analyze the coverage results and decide on which actions to
take in order to converge on 100% coverage.

 Your first choice is to run existing tests

with more seeds.

 The second is to build new constraints.

 Only resort to creating directed tests if

absolutely necessary.

29
 VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Bug Rate
 An indirect way to measure coverage is to look at the rate at which fresh bugs are
found.

 Every time the rate slows down, it is necessary to find different ways to create
corner cases.

30
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Assertion Coverage
 Assertions check relationships between design signals, either once or over a
period of time.

 Coded using the “assert property” statement.

 “cover property” statement can be used to look for interesting signal values or
design states.

 How often the assertions are triggered during a test can be measured using
assertion coverage.

31
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 If the coverage levels are steadily growing -> you may just need to run existing tests with
new random seeds, or even just run longer tests.

 If the coverage growth has started to slow -> you can add additional constraints to generate
more “interesting” stimuli.

 When you reach a plateau -> some parts of the design are not being exercised, so you need
to create more tests.

 Lastly, when your functional coverage values near 100%, check the bug rate.

 If bugs are still being found, you may not be measuring true coverage for some areas of your
design.

32
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Gathering Coverage Data

 Run a test with multiple seeds
 Check for pass/fail
 Analyze coverage across multiple runs

33
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Functional Coverage Strategies

1. Gather Information, Not Data
-> The corner cases for a FIFO are Full and Empty. If the transition from Empty to Full and back to Empty is made, all the levels
in between are covered.
-> Design signals with a large range should be broken down into smaller ranges, plus corner cases.

2. Only Measure What You are Going to Use

-> Gathering functional coverage data can be expensive, and so only measure what is to be analyzed and used to improve the
tests.

3. Measuring Completeness
-> All coverage measurements and the bug rate need to be checked to see if the goal has been met.

34
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Coverage comparison

 The tests are not exercising the full design, perhaps from
an inadequate verification plan.

 The tests are unable to put the testbench in all its

interesting states.

 The existing strategies have saturated and different

approaches like new combinations of design blocks and
error generators need to be tried.

35
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Simple Functional Coverage Example

36
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage
Samples 9.3 and 9.4 have part of a coverage report from VCS.
To improve your functional coverage, the easiest strategies are to run more simulation cycles, or
to try new random seeds.

37
The at least column specifies how many hits are needed before a bin is considered covered.
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Anatomy of a Cover Group

 A cover group is similar to a class – it is defined once and then instantiated

one or more times using new().

 It can be defined in a package, module, program, interface or class.

 It contains cover points, options, formal arguments and an optional trigger.

38
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Anatomy of a Cover Group

 Recommendations:
1) Use clear names for cover groups: Helps in reading coverage reports
2) Don’t define a cover group in a data class
3) Label the cover points

 Cover groups must be defined at the appropriate level of abstraction.

 The sampling of transactions should happen only after it is received by DUT.

 If the transaction contains an error, it needs a different cover point created just for error
handling.
39
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage
Anatomy of a Cover Group
Defining a Cover Group in a Class
 If the cover group is defined in a class, it is known as
an embedded covergroup.

 In this case, you do not make a separate name when

you construct it; just use the original cover group
name.

 You must construct an embedded covergroup in the

class’s constructor, as opposed to a non-embedded
cover group that can be constructed at any time.

40
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

[Link]

41
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

42
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

43
VERIFICATION OF DIGITAL SYSTEMS

Chapter 9
Functional Coverage
bins – Conceptual Part

Vinay Reddy
Department of Electronics and Communication Engineering
44
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

45
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Total no of bins : 6

46
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Total no of bins : 8

47
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 The default bin is not considered for coverage calculation

 All the uncovered values comes under the default bin
 Eg: if address is 11, debug bin increments

Note: when ever there is a hit on debug bin, it denotes address 11

(unnecessary address is generated)
48
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

49
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

50
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

51
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 Ignore bins will not be considered for coverage calculation

 What is the use?? - for documentation purpose
- I want bins for all the values except 2,3,4,5

52
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 Illegal bins will not be considered for coverage calculation

 The values 6 to 9 should not occur. If occurred it should not be passed.
 If illegal bins have a count of 1 or more, then there are bugs present.
 If illegal bins are Zero, then its bug free.

53
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

54
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

55
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Conditions on bins

Conditions on Coverpoints

Conditions on cross 56
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

57
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

58
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

2 bins

2 bins

4 bins

Total : 8 bins

59
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

60
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

61
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

No of bins 4

No of bins 4

62
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

63
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

64
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

65
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

66
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

67
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

68
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

69
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

4 bins

4 bins
6 bins
These are not covered, hence auto bins are
created for this combination of cross

3 bins
Total – 17 bins 70
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

71
VERIFICATION OF DIGITAL SYSTEMS

Chapter 9
Functional Coverage
bins – Textbook Part

Vinay Reddy
Department of Electronics and Communication Engineering
72
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Data Sampling

How is coverage information gathered?

When you specify a variable or expression in a cover point, SystemVerilog creates a number of “bins”
to record how many times each value has been seen.

These bins are the basic units of measurement for functional coverage.

73
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Individual Bins and Total Coverage

 To calculate the coverage for a point, you first have to determine the total number of possible
values, also known as the domain.

 A cover point that is a 3-bit variable has the domain 0:7 and is normally divided into eight bins.

 If, during simulation, values belonging to seven bins are sampled, the report will show 7/8 or 87.5%
coverage for this point.

 All these points are combined to show the coverage for the entire group, and then all the groups are
combined to give a coverage percentage.
74
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Creating Bins Automatically

 System Verilog automatically creates bins for cover points. It looks at the domain of the
sampled expression to determine the range of possible values.

 For an expression that is N bits wide, there are 2 ^ N possible values. For the 3-bit variable
dst , there are 8 possible values.

75
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Limiting the Number of Automatic Bins Created

 The cover group option auto_bin_max specifi es the maximum number of bins to automatically create,
with a default of 64 bins.

 For example, a 16-bit variable has 65,536 possible values, so each of the 64 bins covers 1024 values.

 The coverage report from VCS shows the two bins.

 This simulation achieved 100% coverage because the

eight dst values were mapped to two bins.

 Since both bins have sampled values, your coverage is

100%

76
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Sample 9.12 used auto_bin_max as an option for the cover point only. You can also use it
as an option for the entire group as shown in Sample 9.14 .

77
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Sampling Expressions

Addition C uses a dummy constant to force SystemVerilog to

use 2-bit precision.

D, the first value is cast to be a 2-bit value with the cast

operator, so 1+1=2.

78
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Sampling Expressions
For example, sampling a 3-bit header length (0:7) plus a 4-bit payload length (0:15) creates only 2 4 or
16 bins, which may not be enough if your transactions can actually be from 0 to 22 bytes long.

Also, the expression has an additional

dummy constant so that the transaction
length is computed with 5-bit precision, for
a maximum of 32 auto-generated bins.
79
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

User-Defined Bins Find a Bug

 After sampling many random transactions, the group has

95.83% coverage.

 A quick look at the report in Sample 9.17 shows the problem

— the length of 23 (17 hex) was never seen. The longest
header is 7, and the longest payload is 15, for a total of 22,
not 23!

 If you change to the bins declaration to use 0:22, the

coverage jumps to 100%. The user-defined bins found a bug
in the test. 80
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Naming the Cover Point Bins

81
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 When you define the bins, you are restricting the values used for coverage to those that
are interesting to you.

 SystemVerilog no longer automatically creates bins, and it ignores values that do not fall
into a predefined bin.

 More importantly, only the bins you create are used to calculate functional coverage.

 You get 100% coverage only as long as you get a hit in every specified bin.

82
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 In Sample 9.20 , the $ in the range for bin neg represents the negative number
furthest from zero: 32¢h8000_0000, or -2,147,483,648.

 The $ in bin pos represents the largest signed positive value, 32’h7FFF_FFFF, or
2,147,483,647.
83
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Conditional Coverage
You can use the iff keyword to add a condition to a cover point.

Alternately, you can use the start and stop functions

to control individual instances of cover groups.

84
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Creating Bins for Enumerated Types

Here is part of the coverage report from VCS, Sample 9.24 showing the bins
for the enumerated types.

When you gather coverage on enumerated types,

auto_bin_max does not apply.

85
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage
Transition Coverage
You can specify state transitions for a cover point.

For example, you can check if dst ever went from 0 to 1, 2, or 3 as shown in Sample 9.25 .

86
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Wildcard States and Transitions

You use the wildcard keyword to create multiple states and transitions.

Any X, Z, or ? in the expression is treated as a wildcard for 0 or 1.

87
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage
Ignoring Values

you can let SystemVerilog automatically create bins, and then use ignore_bins
to tell which values to exclude from functional coverage calculation like in Sample
9.27 .

88
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

 If you define bins either explicitly or by using the auto_bin_max option, and then ignore
them, the ignored bins do not contribute to the calculation of coverage.

 In Sample 9.28 , four bins are initially created using the auto_bin_max option: 0:1, 2:3, 4:5,
and 6:7.

 However, then the uppermost bin is eliminated by ignore_bins , so in the end only three bins
are created.
89
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Illegal Bins
 Some sampled values not only should be ignored, but also should cause an error if they are seen.

 This is best done in the testbench’s monitor code, but can also be done by labeling a bin with illegal_bins.

90
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

State Machine Coverage

 If a cover group is used on a state machine, you can use bins to list the
specific states, and transitions for the arcs.

91
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Cross Coverage

 Cross coverage measures what values were seen for two or more cover points
at the same time.

 Note that when you measure cross coverage of a variable with N values, and of
another with M values, SystemVerilog needs N X M cross bins to store all the
combinations.

92
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Basic Cross Coverage Example

Sample 9.30 creates cover points for [Link] and [Link] .

Then the two points are crossed to show all combinations.

SystemVerilog creates a total of 128 (8 X 16) bins. 93
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

94
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

95
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Labeling Cross Coverage Bins

If you want more readable cross coverage bin names, you can label the individual cover point bins as
demonstrated in Sample 9.32.

96
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

97
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Excluding Cross Coverage Bins

98
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Excluding Cross Coverage Bins

The first ignore_bins in Sample 9.34 just excludes bins where
dst is 7 and any value of kind . Since kind is a 4-bit value, this
statement excludes 12 bins, as misc’s values of 4–7 don’t count
because of the default .

99
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Excluding Cross Coverage Bins

The second ignore_bins is more selective, ignoring bins
where dst is 0 and kind is 9, 10, or 11, for a total of 3
bins.

100
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Excluding Cross Coverage Bins

The ignore_bins lo uses bin names to exclude
[Link] that is 1, 2, or 3.

101
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage
Excluding Cover Points from the Total Coverage Metric
If you are only sampling a variable or expression in a coverpoint to be used in a cross
statement, you should set its weight to 0 so that it does not contribute to the total coverage.

102
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

As stated in the LRM,

 option shall be used with coverage instances.

 type_option shall be used once per coverage type.

103
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Cross Coverage Alternatives

104
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

105
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

106
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Triggering a Cover Group

 The two major parts of functional coverage are
the sampled data values and
the time when they are sampled.

 When new values are ready (such as when a transaction has completed), your
testbench triggers the cover group.

 This can be done directly with the sample function, as shown in Sample 9.5 , or by
using a coverage event in the covergroup definition. 107
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Cover Group with an Event Trigger

In Sample 9.9 , the cover group CovDst9 is sampled when the

testbench triggers the trans_ready event .

108
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Triggering on a System Verilog Assertion

109
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Cover Group with a User Defined Sample Argument List

110
VERIFICATION OF DIGITAL SYSTEMS
Functional Coverage

Cover Group with a User Defined Sample Argument List

User defined argument list

111
 THANK YOU

Vinay Reddy
Department of Electronics & Communication Engineering
vinay@[Link]
+91 9945730244

112