SystemVerilog
for
Verification
Syllabus
• Verification Guidelines: Introduction, Verification Process, Verification
Plan, Verification Methodology Manual, Basic Testbench Functionality,
Directed Testing, Methodology Basics, Constrained-Random Stimulus,
Functional Coverage, Testbench Components, Layered Testbench,
• Data Types: Built-in Data Types, Fixed-Size Arrays, Dynamic Arrays,
Queues, Creating New Types with typedef, Creating User-Defined
Structures, Enumerated Types, Constants, Strings
• Procedural Statements and Routines: Procedural Statements, Tasks,
Functions, and Void Functions
• Basic Oop: Where to Define a Class, OOP Terminology, Understanding
Dynamic Objects
• SystemVerilog Assertions: Types of Assertions and examples
• Threads and Inter-process Communication: Working with Threads,
Inter-process Communication, Events, Semaphores, Mailboxes, Building a
Testbench with Threads and IPC
• Functional Coverage: Coverage Types, Functional Coverage Strategies,
Simple Functional Coverage Example, Coverage Options, Parameterized
Cover Groups, Analyzing Coverage Data, Measuring Coverage Statistics
During Simulation
Syllabus
• Introduction to formal verification: Introduction to formal techniques
and property specification, Reachability analysis, Elements of property
languages, Property language layers, PSL basics, Formal test plan process,
Techniques for proving properties: Abstraction reduction, Compositional
reasoning, Counter abstraction, Gradual Exhaustive formal verification
• Text books:
• SystemVerilog for Verification: A Guide to Learning the Testbench
Language Features, Chris Spear, Springer 2006
• Writing Testbenches Using SystemVerilog, Janick Bergeron, Springer, 2006
• SystemVerilog for Design: A Guide to Using SystemVerilog for Hardware
Design and Modeling, 2nd Edition, Stuart Sutherland, Simon Davidman and
Peter Flake, Springer
• Reference books:
• Writing Testbenches: Functional Verification of HDL Models, Second
edition, Janick Bergeron, Kluwer Academic Publishers, 2003.
• Open Verification Methodology Cookbook, Mark Glasser, Springer, 2009
• Principles of Functional Verification, Andreas S. Meyer, Elsevier Science,
2004
Syllabus
• Course Objectives: The objective of this course is to provide students with
1. insight to apply System Verilog concepts to do synthesis, analysis and
architecture design.
2. understanding of SystemVerilog and SVA for verification, and understand the
improvements in verification efficiency.
3. Understand advanced verification features, such as the practical use of
classes, randomization, checking, and coverage.
4. knowledge to Communicate the purpose and results of a design experiment
in written and oral presentations.
• Course Outcome: Upon completion of this course, students should
demonstrate the ability to
1. Use SystemVerilog to create correct, efficient, and re-usable models for
digital designs
2. Use SystemVerilog to create testbenches for digital designs
3. Understand and effectively exploit new constructs in SystemVerilog for
verification
Verification
Guidelines
Verification Guidelines
• Before start to learn details of the SystemVerilog language, you need to
understand how you plan to verify your particular design and how this
influences the testbench structure.
• All testbenches share some common structure of stimulus generation and
response checking.
• This section introduces a set of guidelines and coding styles for designing
and constructing a testbench that meets your particular needs.
• It assumes you already know the Verilog language and want to learn the
SystemVerilog Hardware Verification Language (HVL).
• Typical features of an HVL that distinguish it from a Hardware Description
Language such as Verilog or VHDL are
– Constrained-random stimulus generation
– Functional coverage
– Higher-level structures, especially Object Oriented Programming
– Multi-threading and interprocess communication
– Support for HDL types such as Verilog’s 4-state values
– Tight integration with event-simulator for control of the design
The Verification Process
• The purpose of a verification engineer is to make sure the device can
accomplish the task successfully — that is, the design is an accurate
representation of the specification.
• As a verification engineer, you must read the hardware specification, create
the verification plan, and then follow it to build tests showing the RTL code
correctly implements the features.
• As a verification engineer you can never prove there are no bugs left, so
you need to constantly come up with new verification tactics.
• The verification plan is closely tied to the hardware specification and
contains a description of what features need to be exercised and the
techniques to be used. These steps may include directed or random testing,
assertions, HW/ SW co-verification, emulation, formal proofs, and use of
verification IP.
• Basic Testbench Functionality: The purpose of a testbench is to determine
the correctness of the design under test (DUT). This is accomplished by the
following steps.
The Verification Process
• Generate stimulus
• Apply stimulus to the DUT
• Capture the response
• Check for correctness
• Measure progress against the overall verification goals
• Some steps are accomplished automatically by the testbench, while others
are manually determined by the designer.
• Directed Testing: Traditionally, when faced with the task of verifying the
correctness of a design, you may have used directed tests. Using this
approach, you look at the hardware specification and write a verification
plan with a list of tests, each of which concentrated on a set of related
features. Armed with this plan, you write stimulus vectors that exercise
these features in the DUT. You then simulate the DUT with these vectors
and manually review the resulting log files and waveforms to make sure the
design does what you expect. Once the test works correctly, you check off
the test in the verification plan and move to the next.
The Verification Process
• Methodology Basics: Uses the following principles.
• Constrained-random stimulus
• Functional coverage
• Layered testbench using transactors
• Common testbench for all tests
• Test-specific code kept separate from testbench.
• All these principles are related. Random stimulus is crucial for exercising
complex designs. A directed test finds the bugs you expect to be in the
design, while a random test can find bugs you never anticipated.
• When using random stimulus, you need functional coverage to measure
verification progress. Furthermore, once you start using automatically
generated stimulus, you need an automated way to predict the results,
generally a scoreboard or reference model.
• A layered testbench helps you control the complexity by breaking the
problem into manageable pieces. Transactors provide a useful pattern for
building these pieces. With appropriate planning, you can build a testbench
infrastructure that can be shared by all tests and does not have to be
continually modified.
The Verification Process
• Every test you create shares this common testbench, as opposed to directed
tests where each is written from scratch.
• The result is that your single constrained-random testbench is now finding
bugs faster than the many directed ones as shown in figure 1.1.
The Verification Process
• Constrained-Random Stimulus: While you want the simulator to
generate the stimulus, you don’t want totally random values. You use the
SystemVerilog language to describe the format of the stimulus (“address is
32-bits, opcode is X, Y, or Z, length < 32 bytes”), and the simulator picks
values that meet the constraints.
• These values are sent into the design, and also into a high-level model that
predicts what the result should be. The design’s actual output is compared
with the predicted output.
• Figure 1.2 shows the coverage for constrained-random tests over the total
design space.
• First, notice that a random test
often covers a wider space than a
directed one.
• you may still have to write a
few directed tests to find cases
not covered by any other
constrained random tests.
The Verification Process
• Figure 1.3 shows the paths to achieve complete coverage.
• What Should You Randomize? randomizing the stimulus to a design, the
first thing that you might think of is the data fields. This stimulus is the
easiest to create – just call $random. The problem is that this gives a very
low payback in terms of bugs found. The primary types of bugs found with
random data are data path errors, perhaps with bit-level mistakes. You need
to find bugs in the control logic.
The Verification Process
• We need to think broadly about all design input, such as the following.
• Device configuration
• Environment configuration
• Input data
• Protocol exceptions
• Delays
• Errors and violations
• Device and environment configuration: What is the most common reason
why bugs are missed during testing of the RTL design? Not enough
different configurations are tried.
• In the real world, your device operates in an environment containing other
components. When you are verifying the DUT, it is connected to a
testbench that mimics this environment. You should randomize the entire
environment configuration, including the length of the simulation, number
of devices, and how they are configured.
• Input data: Actually this approach is fairly straightforward as long as you
carefully prepare your transaction classes.
The Verification Process
• Protocol exceptions, errors, and violations: when a device such as a PC or cell
phone locks up.
• Just as you are trying to provoke the hardware with ill-formed commands, you
should also try to catch these occurrences. You should add checker code to look for
these violations. Your code should at least print a warning message when this
occurs, and preferably generate an error and wind down the test. It is frustrating to
spend hours tracking back through code trying to find the root of a malfunction,
especially when you could have caught it close to the source with a simple
assertion.
• Delays and synchronization: How fast should your testbench send in stimulus?
Always use constrained random delays to help catch protocol bugs. A test that uses
the shortest delays runs the fastest, but it won’t create all possible stimulus. You can
create a testbench that talks to another block at the fastest rate, but subtle bugs are
often revealed when intermittent delays are introduced.
• Parallel random testing: A directed test has a testbench that produces a unique set
of stimulus and response vectors. To change the stimulus, you need to change the
test. A random test consists of the testbench code plus a random seed. If you run the
same test 50 times, each with a unique seed, you will get 50 different sets of
stimuli. Running with multiple seeds broadens the coverage of your test and
leverages your work.
The Verification Process
• Functional Coverage: previous sections shown how to create stimuli that
can randomly walk through the entire space of possible inputs. With this
approach, your testbench visits some areas often, but takes too long to
reach all possible states. Unreachable states will never be visited, even
given unlimited simulation time.
• The process of measuring and using functional coverage consists of several
steps. First, you add code to the testbench to monitor the stimulus going
into the device, and its reaction and response, to determine what
functionality has been exercised. Next, the data from one or more
simulations is combined into a report. Lastly, you need to analyze the
results and determine how to create new stimulus to reach untested
conditions and logic.
• Feedback from functional coverage to stimulus: A random test evolves
using feedback. As the functional coverage asymptotically approaches its
limit, you need to change the test to find new approaches to reach
uncovered areas of the design. This is known as “coverage-driven
verification.”
The Verification Process
• Testbench Components: In simulation, the testbench wraps around the DUT, just
as a hardware tester connects to a physical chip. Both the testbench and tester
provide stimulus and capture responses.
• The difference is that your testbench needs to work over a wide range of levels of
abstraction, creating transactions and sequences, which are eventually transformed
into bit vectors. A tester just works at the bit level.
• What goes into that testbench block in figure 1.4? It is made of many bus functional
models (BFM), that can be think of as testbench components — to the DUT they
look like real components, but are part of the testbench, not RTL.
• If the real device connects to AMBA, USB, PCI, and SPI buses, you have to build
equivalent components in your testbench that can generate stimulus and check the
response. These are not detailed, synthesizable models but instead, high-level
transactors that obey the protocol, but execute more quickly
• Layered Testbench: A key concept for any modern verification methodology is the layered
testbench. This process make the testbench more complex, but it actually helps to make your
task easier by dividing the code into smaller pieces that can be developed separately.
• Flat testbench: General way to write a testbench
module test(PAddr, PWrite, PSel, PRData, Rst, clk);
// Port declarations omitted...
initial begin
// Drive reset
A task to drive the APB pins –
Rst <= 0;
#100 Rst <= 1; Bus Write
// Drive Control bus
@(posedge clk) task write(reg [15:0] addr, reg
PAddr <= 16’h50; [31:0] data);
PWData <= 32’h50; // Drive Control bus
PWrite <= 1'b1; @(posedge clk)
PSel <= 1'b1;
// Toggle PEnable PAddr <= addr;
@(posedge clk) PWData <= data;
PEnable <= 1'b1; PWrite <= 1'b1;
@(posedge clk) PSel <= 1'b1;
PEnable <= 1'b0; // Toggle Penable
// Check the result
if ([Link][16’h50] == 32’h50) @(posedge clk)
$display("Success"); PEnable <= 1'b1;
else @(posedge clk)
$display("Error, wrong value in memory"); PEnable <= 1'b0;
$finish; endtask
end
endmodule
The Verification Process
• Now your testbench became simpler.
module test(PAddr, PWrite, PSel, PRData, Rst, clk);
// Port declarations omitted...
initial begin
reset(); // Reset the device
write(16’h50, 32’h50); // Write data into memory
// Check the result
if ([Link][16’h50] == 32’h50)
$display("Success");
else
$display("Error, wrong value in memory");
$finish;
end
Endmodule
• By taking the common actions, such as reset, bus reads and writes, and
putting them in a routine, you became more efficient and made fewer
mistakes. This creation of the physical and command layers is the first step
to a layered testbench.
The Verification Process
• The signal and command layers: Figure 1.5 shows the lower layers of a
testbench.
• Signal layer contains the design under test and the signals that connect it to
the testbench.
• The DUT’s inputs are driven by the driver that runs single commands such
as bus read or write. The DUT’s output drives the monitor that takes signal
transitions and groups them together into commands. Assertions also cross
the command/signal layer, as they look at individual signals but look for
changes across an entire command.
The Verification Process
• The functional layer: The functional layer shown in figure 1.6 feeds the
command layer. The agent block (called the transactor in the VMM)
receives higher-level transactions such as DMA read or write and breaks
them into individual commands. These commands are also sent to the
scoreboard that predicts the results of the transaction. The checker
compares the commands from the monitor with those in the scoreboard.
The Verification Process
• The scenario layer: The functional layer is driven by the generator in the
scenario layer as shown in figure 1.7.
• Scenario? An example device is an MP3 player that can concurrently play
music from its storage, download new music from a host, and respond to input
from the user, such as volume and track controls. Each of these operations is a
scenario. Downloading a music file takes several steps, such as control register
reads and writes to set up the operation, multiple DMA writes to transfer the
song, and then another group of reads and writes. The scenario layer of your
testbench orchestrate all these steps with constrained-random values for
parameters such as track size and memory location.
The blocks in the
testbench environment
(inside the dashed line)
are written at the start of
development.
The Verification Process
• The test layer and functional coverage: You are now at the top of the
testbench, the test layer, as shown in Figure 1.8. This top-level test is a
conductor , guides the efforts of others. The test contains the constraints to
create the stimulus. Functional coverage measures the progress of all tests
in fullfilling the requirements in the verification plan. The functional
coverage code changes through the project as the various criteria complete.
Because it is constantly being modified, it is not part of the environment.
The Verification Process
• Building a Layered Testbench: how to map the components into
SystemVerilog constructs.
• Creating a simple driver: The driver receives commands from the agent,
may inject errors or add delays, and then breaks the command down into
individual signal changes such as bus requests, handshakes, etc. The
general term for such a testbench block is a “transactor,” which, at its core,
is a loop as shown below:
• task run();
• done = 0;
• while (!done) begin
• // Get the next transaction
• // Make transformations
• // Send out transactions
• end
• Endtask
• OOP concepts presents how to create an object that includes the routines
and data for a transactor. These objects are sent between transactors using
SystemVerilog mailboxes. Threads and Inter-process communication
presents many ways to exchange data between the different layers and to
synchronize the transactors.
The Verification Process
• Simulation Environment Phases: There are three primary phases to
coordinate the testbench, so all the code for a project works together, these
are Build, Run, and Wrap-up. Each is divided into smaller steps.
• The Build phase is divided into the following steps:
• Generate Configuration: randomize the configuration of the DUT and the
surrounding environment.
• Build Environment: Allocate and connect the testbench components based
on the configuration. A testbench component is one that only exists in the
testbench, as opposed to physical components in the design that are built
with RTL code.
• Reset the DUT.
• Configure the DUT based on generated configuration from the first step.
• The Run phase is where the test actually runs. It has the following steps:
• Start environment: run the testbench components such as BFMs and
stimulus generators.
• Run the test: start the test and then wait for it to complete. Starting from the
top, wait for a layer to drain all the inputs from the previous layer (if any),
wait for the current layer to become idle and wait for next lower layer.
The Verification Process
• The Wrap-up phase has two steps:
• Sweep: After the lowest layer completes, you need to wait for the final
transactions to drain out of the DUT.
• Report: Once the DUT is idle, sweep the testbench for lost data.
Sometimes the scoreboard holds transactions that never came out, perhaps
because they were dropped by the DUT. With this information you can
create the final report on whether the test passed or failed. If it failed, be
sure to delete any functional coverage data, as it may not be correct.
The Verification Process
• Flow for System Verilog Verification: Several steps to creating a constrained-
random test. The most significant is building the layered testbench, including the
self-checking portion.
– The benefit of this work is shared by all tests, so it is well worth the effort.
• The next step is creating the stimulus specific to a goal in the verification plan.
– You may be crafting random constraints, or devious ways of injecting errors or
protocol violations. Building one of these may take more time than making
several directed tests, but the payoff will be much higher. A constrained-
random test that tries thousands of different protocol variations is worth more
than the handful of directed tests that could have been created in the same
amount of time.
• The third step in constrained-random testing is functional coverage.
– This task starts with the creation of a strong verification plan with clear goals
that can be easily measured.
• Next you need to create the SystemVerilog code that instruments the environment
and gathers the data.
• Lastly, and most importantly, you need to analyze the results to determine if you
have met the goals, and if not, how you should modify the tests.