PCIe Overview
PCIe Overview
- Shivappa K M
Introduction
PCI and PCIe History
PCIe 4.0
PCI 2.1
• 66 MHz 3.3v
1994
PCI 2.0
• PCI Power Management
1993
PCI 1.0
1992
• PCI Plug and Play Configuration Model
What is wrong with PCI?
What is wrong with PCI?
Bandwidth
Multi-drop bus limits
Frequency
Large pin-count causes board
routing problems
No Quality of Service (QoS)
Limited Power Management
Capabilities
Limited Reliability,
Availability, Serviceability
Scalability for different market
segments and applications
Ease of use
Bus architecture
CPU
Bridge/
Memory
controller
PCI Bus
EISA - 32 8.3 33 -
Link topology Master/target, transaction are Master/target, transaction No master/target, PCIe identifies
identified by who mastered it are identified by who the transactions by identifiers.
mastered it
+3.3 volt
8 +3.3v JTAG5 TMS
power
+3.3 volt
9 JTAG1 +TRST# +3.3v
power
3.3v volt +3.3 volt
10 3.3Vaux +3.3v
power power
Link
Elements of PCIe 11 WAKE#
Mechanical Key
Reactivation
PWRGD Power Good
• End Point - Request or complete PCI transaction itself 16 GND Ground HSIp(0) Receiver Lane
0,
Hotplug Differential
17 PRSNT#2 HSIn(0) pair
• Switch - Traffic Director between multiple PCIe links 18 GND
detect
Sample Topology
Layered Architecture Overview
Software
- PCI Compatible Configuration & Driver model
Software
- PCI Express Enhanced Configuration
Transaction
- Split transaction packet based protocol
Transaction
- PCI-like Load-Store transaction semantics
- Credit based flow control, Virtual Channels
Data Link
Data Link - Logical connection between devices
- Provides reliable data transport
Physical
Physical - Electrical interface and Signaling mechanism
- Interface initialization and maintenance
Mechanical
Mechanical - Market segment specific form factors
Data Flow Between Devices
Software Software
Transaction Transaction
Physical Physical
Mechanical Mechanical
Physical Layer
▪ Logical Functions
Software 8b/10b Encoding & Decoding
Scrambling & Unscrambling
Reset & Initialization
Transaction Multi-lane deskew
Receiver elastic buffer
Link width and Lane mapping
Data Link negotiation
Lane and Polarity reversal
Physical
▪ Electrical Functions
Logical Transmitter & Receiver
Embedded Clock extraction &
Electrical alignment
Link Power Management
Mechanical
PCI Express Signaling
▪ 8b/10b Encoding
Limits run lengths of 1’s and 0’s
Data transitions enable bit-level clock recovery at the receiver
▪ Routing Flexibility
Removes skew relationships between clock and data signals
▪ High Performance
High frequencies
Skew/Jitter between bits
Link Training example
▪ Configuration and
initialization of the link
Detect
Detect
Detect
▪ Detect if receiver is present
▪ Recovering from link errors
▪ Transitioning to and Polling
Polling Disabled
Polling
restarting from low power ▪ Achieve Bit lock, Symbol lock
states ▪ Configure Lane Polarity
▪ Link training & negotiation Configuration
Configuration Hot Reset Configuration
Link width ▪ Negotiate Link width and Lane number
▪ Lane-to-lane deskew
Lane reversal
L0
Polarity inversion
L0 L0 Loopback
Topology components
Root Complex
End Point CPU
Switch
PCI Express – PCI Bridge
Root Complex Memory
Split Transaction
End to
Packet based
R C
End
C
Bridge
Request packet Switch
R
Completion packet
End to end and Local link PCI End
Point
End
Point
End
Point
level
Local
Packet Sources & Types
Transaction Layer
Physical Layer
Packet Format - DLLP
Physical Layer
Flow Control
TLP
Device A Device B
FC Update
4
4. Device A advertises the released
buffer space to Device B 5
Mem Read
Latency control
VC0 VCn VC0 VCn
Blocking – ordering and flow control
All Transaction Layer Packet (TLP) headers contain the First 4 bytes of header
TLP data must be 4-byte naturally aligned and in increments of 4-byte Double Words.
Fmt[1:0] – Format of TLP
Type[4:0] – Type of TLP
TC[2:0] – Traffic Class (virtual channels)
Attr[1:0] – Attributes(snoop and relaxed ordering)
TD – 1b indicates presence of TLP digest in the form of a single DW at the end of TLP
EP – indicates the TLP is poisoned
Length[9:0] – Length of data payload in DW
TLP Header format:Type Encoding
TLP Header Format: Requester ID and Tag
Tag[7:0] is a 8-bit field generated by each Requestor, and it must be unique for all outstanding
Requests that require a Completion for that Requester
For Requests which do not require Completion (Posted Requests), the value in the Tag[7:0] field
is undefined and may contain any value
Requester ID and Tag combined form a global identifier (Transaction ID).
ID routing uses the Bus, Device, and Function Numbers to specify the destination Device for the
TLP
Zero Length Read: A Memory Read Request of 1 DW with no bytes enabled, or “zero length
Read,” may be used by devices as a type of flush Request. For a Requester, the flush semantic
allows a device to ensure that previously issued Posted Writes have been completed at their PCI
Express destination.
TLP Header : Memory Requests
64-bit Memory Request
All Message Requests include the following fields in addition to the common header fields
Requester ID[15:0] and Tag[7:0], forming the Transaction ID.
Message Code[7:0] – Specifies the particular Message Request.
INTx Interrupt Signaling
Power Management
Error Signaling
Locked Transaction Support
Slot Power Limit Support
Vendor-Defined Messages
Hot-Plug Signaling
Message Requests are posted and do not require Completion.
TLP Header : INTx Messaging
INTx Message Routing
Power Management Messages do not include a data payload (TLP Type is Msg).
The Length field is reserved.
Power Management Messages must use the default Traffic Class designator (TC0).
Receivers must check for violations of this rule. If a Receiver determines that a TLP
violates this rule, it must handle the TLP as a Malformed TLP.
TLP Header : Error Messaging
ERROR Message Routing
Error Signaling Messages do not include a data payload (TLP Type is Msg).
The Length field is reserved.
Error Signaling Messages must use the default Traffic Class designator (TC0)
Receivers must check for violations of this rule. If a Receiver determines that a
TLP violates this rule, it must handle the TLP as a Malformed TLP.
TLP Header payload & TLP digest
TLP Data payload
Data payload is appended at the end of TLP header based on type of packet.
Maximum payload size is 1024 double words(4 bytes)
Address and length combination should not cross 4Kbytes boundary.
When the data is included in TLP, first data corresponds to lowest byte address.
TLP digest
Data link layer inserts 32-bit LCRC to outgoing TLP, if the error has occurred to TLP before it
reaches DL, LCRC won’t detect.
To ensure end-to-end data integrity, 32-bit ECRC is generated.
Ack/Nak DLLP:
Ack DLLP: Sequence number acknowledgement on successful receipt of TLP
Nak DLLP:Sequence number negative acknowledgement, triggers retry TLPs.
FC DLLP:Flow control,
initFC1, initFC2 and Update FC
initFC1 and initFC2 are sent during FC initialization for virtual channel.
Update FC is sent during normal link operation indicating how much buffer space available for TLPs.
Framing : Special characters are inserted to mark Start and End of packet.
Physical layer: Scrambling scheme
Scrambling is a technique used in PCI Express (PCIe) to randomize data and remove repetitive patterns. It helps to:
• Balance the signal: Scrambling creates transitions to DC-balance the signal.
• Reduce EMI noise: Scrambling removes electromagnetic interference (EMI) noise, which can be significant for PCI Express
transmission lines.
• Reduce the possibility of electrical resonances: Scrambling reduces the possibility of electrical resonances on the link
L0
Polling Recovery (Full On)
loopback
L3
Electric Idle:
When Transmit pairs TX+ and TX- are held at constant value.
Link Training and Status State Machine(LTSSM)
Detect state :
First state to enter upon cold reset, warm reset.
Determines whether the device connected on the other side on per lane basis.
Both the devices select the data rate 2.5Gb/s.
If x4 device connected to x2 device, x2 link will be formed after 12 ms timeout on
other 2 lanes.
Polling State:
Training ordered sets TS1 and TS2 are used to establish bit and symbol alignment and
to exchange physical layer parameters such as data rate, clock synchronization, lane
numbering, lane polarity, enable data scrambling.
Training ordered sets are groups of 16 8/10-bit encoded symbols and never scrambled
Configuration:
In Configuration, both the Transmitter and Receiver are sending and receiving data at
the negotiated data rate.
The Lanes of a Port configure into a Link through a width and Lane negotiation
sequence.
Lane-to-Lane de-skew must occur, scrambling can be disabled, the N_FTS is set, and
the Disable or Loopback states can be entered.
Physical Layer:Link and Training
Recovery:
In Recovery, both the Transmitter and Receiver are sending and receiving data using the configured
Link and Lane number as well as the previously negotiated data rate.
Recovery allows a configured Link to re-establish bit lock, Symbol lock, and Lane-to-Lane de-skew.
Recovery is also used to set a new N_FTS and enter the Loopback, Disable, Hot Reset, and
Configuration states.
L0
L0 is the normal operational state where data and control packets can be transmitted and received.
All power management states are entered from this state.
L0s
L0s is intended as a power savings state.
L0s allows a Link to quickly enter and recover from a power conservation state without going through
Recovery.
The entry to L0s occurs after receiving an Electrical Idle ordered set.
The exit from L0s to L0 must re-establish bit lock, Symbol lock, and Lane-to-Lane de-skew.
A Transmitter and Receiver Lane pair on a Port are not required to both be in L0s simultaneously.
L1
L1 is intended as a power savings state.
The L1 state allows an additional power savings over L0s at the cost of additional resume latency.
The entry to L1 occurs after being directed by the Data Link Layer and receiving an Electrical Idle
ordered set.
Physical Layer:Link Training Status State Machine
L2
Power can be aggressively conserved in L2. Most of the Transmitter and Receiver may be shut off.
Main power and clocks are not guaranteed.
Disabled
The intent of the Disabled state is to allow a configured Link to be disabled until directed or Electrical
Idle is exited (i.e., due to a hot removal and insertion).
after entering Disabled. Disabled uses bit 1 (Disable Link) in the Training Control field which is sent
within the TS1 and TS2 training ordered set.
A Link can enter Disabled if directed by a higher Layer. A Link can also reach the Disable state by
receiving two consecutive TS1 ordered sets with the Disable bit asserted
Loopback
Loopback is intended for test and fault isolation use.
Loopback can operate on either a per Lane or configured Link basis.
A Loopback Master is the component requesting Loopback.
A Loopback Slave is the component looping back the data.
Loopback uses bit 2 (Loopback) in the Training Control field which is sent within the TS1 and TS2
training ordered set.
Hot Reset
Hot Reset uses bit 0 (Hot Reset) in the Training Control field which is sent within the TS1 and TS2
training ordered set.
A Link can enter Hot Reset if directed by a higher Layer. A Link can also reach the Hot Reset state by
receiving two consecutive TS1 ordered sets with the Hot Reset bit asserted
Thank You