What is PCIe 5.0?
The 5th generation of Peripheral Component Interconnect Express is known as PCI Express 5.0. It is also referred to as 5th PCIe, PCIe 5, PCI v5, or simply PCIe 5.0. First introduced in 2003, PCIe technology has become the standard interface for connecting high-speed components to the motherboard using a point-to-point access bus.
Following a seven-year gap between the completion of PCIe 3.0 and PCIe 4.0, the development and release of the PCI Express 5.0 specifications followed quickly on the heels of 4.0 with yet another 2X increase in bandwidth. The finalized PCIe 5.0 standard has been released by PCI-SIG.
PCIe 5.0 Test Tools
Test standards and practices continue to be challenged with each new PCIe release, and PCIe Gen5 is no exception. Revised Electrical Idle Exit Ordered Set (EIEOS) and clocking features have impacted test practices at both the hardware and system levels. Fully downward compatible protocol analyzers capable of supporting 32GT/sec data link speed operations, such as the VIAVI Xgig Analyzer, are invaluable for performing the latest PCIe 5.0 testing and debugging processes. Superior memory, storage capacity, and segmentation for higher volume upstream and downstream traffic capture allows long sequences to be recorded with specific packets filtered out for robust protocol analysis.
Alternate protocols, as described in the new PCI Express 5.0 specification, also require full test support, as this improved versatility now permits other protocols to leverage the proven PCIe physical layer stack. For example, the Compute Express Link (CXL) alternate protocol, which is supported by many industry leaders, provides an optimized protocol stack with cache coherency that is ideally suited for low latency interfaces. Newly added tech such as equalization bypass options and precoding at PCIe 5.0 speeds, make cutting-edge protocol analyzers with up-to-date features sets an invaluable asset.
Jamming capability remains essential to test coverage, as the simulation of network traffic in real time creates an important litmus test for PCIe hardware. Intelligent and protocol-aware jammers such as the Xgig Jammer platform support PCIe 5.0 test setup inline operation, automated discovery and regression testing, as well as test support over a wide range of protocols.
The Xgig Expert software package, standard with all Xgig Analyzers, supports and enhances these outstanding PCIe Gen 5 test capabilities. Software features include automated trace data capture and analysis over a user-friendly interface that provides unparalleled visibility across all protocol layers and network topologies.
The Medusa Labs Test Tools Suite (MLTT) is an application layer software tool enabling configurable network traffic generation and analysis, with no additional equipment required. Stress testing of network hardware is used to efficiently uncover errors, while MLTT benchmarking and data integrity test tools help to accelerate design validation and system bring-up.
PCIe 5.0 Release Date
The final release of the PCI Express 5.0 standard on May 29, 2019 was the culmination of an accelerated 18-month development cycle deemed necessary to address the escalating performance demands of data-intensive applications.
Like all previous generations, PCIe 5.0 maintains backwards compatibility with past iterations, although the lowest version (speed) between the PCIe slot and connecting card remains the gating factor. In addition to the bandwidth increase, the PCIe 5.0 spec also includes electrical enhancements to improve signal integrity and mechanical updates to improve connector performance.
Although the final release of PCIe 4.0 was completed years ago in June of 2017, the commercialization of requisite Gen 4 components continued well beyond the PCIe 5.0 release date. The timing of the PCIe Gen 5 release will provide a unique “leap-frogging” option for hardware manufacturers, once PCIe 5.0 components and products are commercially available in 2021. A direct transition from PCIe 3.0 to 5.0 produces a 4X “speed bump”, with a PCI Express Gen 5 x4 slot delivering the same bandwidth performance as a PCIe 3.0 x16 full-size slot, thereby freeing up valuable connection real estate.
As has been the case with each successive PCIe release, a coexistence between PCIe Express 5.0 and previous PCIe versions, enabled by the intrinsic backwards compatibility of the PCIe interface, allows the most demanding high-performance applications, such as cloud computing and Artificial Intelligence (AI), to take advantage of the highest available transfer rate, while previous generations of technology remain in use for less demanding applications.
Creating the PCIe 5.0 Ecosystem
The PCIe 5.0 release date in 2019 was just one of many incremental steps on the path to implementation. The PCIe Gen 5 Card Electromechanical (CEM) specification, critical for defining system integration practices, remains in development, with full release expected in late 2020. Preliminary compliance and interoperability testing must also be completed successfully. These additional milestones may extend the introduction of the first certified PCIe 5.0 commercial product offerings into mid-2021.
With PCIe incorporated into almost every type of computing system in use today, the symbiotic customer and supply chain infrastructure for PCI Express 5.0 includes many of the world’s largest corporations in the electronics, computing, data storage, and E-commerce industries. This includes intellectual property (IP) suppliers enabling PCIe Gen 5 functionality, switch and retimer manufacturers, along with PCIe 5.0 motherboard, storage device, and graphics controller suppliers. These essential building blocks feed into the advanced computer systems and hardware for data centers and other fundamental network deployments.
Market Segments for PCIe 5.0 Early Adoption
Not all enterprise and market segments are expected to adopt PCI Express Gen 5 technology simultaneously. IP markets will require early validation capabilities to ensure functionality in either FPJ or silicon formats. CPU, Ethernet, and select Accelerator segments will also be heavily involved in the early roll-out of PCIe 5.0. Data center servers and high-performance computing (HPC) infrastructure, already struggling to meet the ever-expanding bandwidth and latency demands, will quickly absorb the inherent benefits of PCIe Gen 5 technology as it becomes available.
PCIe 5.0 Protocol Analysis Users
Faster validation and debug cycles are essential for accelerated time to market. This creates a greater demand for advanced PCIe Gen 5 protocol analysis tools for system integration teams performing verification and validation, and debugging teams working to qualify components and resolve interoperability issues. Performance tuning teams for devices, drivers, and application software also reap a strong return on investment from advanced PCIe 5.0 protocol analysis functionality.
PCIe Gen 5 Speed
The speed doubling convention of PCIe remains consistent with the release of PCIe 5.0. Using the same 128b/130b encoding method that has been standard since PCIe version 3.0, PCIe 5.0 will deliver 64 GB/sec of throughput in each direction. Since PCIe technology allows data to flow full duplex bidirectionally, the total throughput for both directions combined amounts to 128 GB/sec.
The encoding standard prior to PCIe 3.0 was 8b/10b, meaning eight bits of data were encoded and transmitted as a 10-bit number. This in turn created a 20% performance overhead factor that reduced a raw bit transfer rate of 2.5 GT/sec to a net bandwidth of just 2.0 Gbit/sec. This more efficient 1.5% overhead factor remains in effect with the PCIe 5.0 encoding convention.
The phenomenal speed of PCIe 5.0 enables the equivalent of a typical Blu-ray disc contents to be transferred to non-volatile memory (NVM) on a PCI 5.0 motherboard in under one second. Although this exceptional speed may seem like a luxury, it has been necessitated by enhancements to network architecture in other arenas. For example, 400G Ethernet requires 50 GB/sec of bandwidth in each direction to interface with the CPU at maximum capacity.
400G Ethernet requires 50 GB/sec of bandwidth in each direction to interface with the CPU at maximum capacity. With PCIe 4.0, the 32 GB/sec available on a full size x16 slot proved to be insufficient. Using PCIe 5.0 technology, the available bandwidth exceeds the requirements of this interface with room to spare.
In addition to Ethernet, this continuous improvement cycle is driven by the advent of real-time systems with latency-critical performance requirements, such as autonomous driving, defense applications requiring instantaneous response, and critical financial security applications where hacking attempts must be thwarted immediately. Individual users of multi-GPU systems and premium graphics cards also reap tangible benefits from PCIe 5.0 speed and bandwidth enhancement.
PCI Express 5.0 Specification
The PCI Express 5.0 specification might be classified as a natural evolution of the backwards compatible PCIe standard with no inherent link or transaction layer changes included in this iteration. The 5.0 specification continues to benefit from the scaled flow control and extended tags and credits established with PCIe 4.0.
A new CEM connector designated for add-in cards has also been added. Improvements to signal integrity and connector design features have improved overall performance and reliability. Physical layer enhancements also include an update to the EIEOS, SKP ordered sets, and equalization sequences.
The PCI Express 5.0 specification has been generally lauded by hardware manufacturers and industry insiders. In particular, the enhancements made to improve testability, accelerate link training, and provide alternate protocol support have been recognized as exceptional PCIe 5.0 characteristics. This industry consensus, along with the relatively benign set of implementation prerequisites required to transition from 4.0 to 5.0, have spearheaded the aggressive hardware development and commercialization targets.
PCIe 5.0 Challenges
Advancements and specification changes accompanying the PCIe 5.0 release have further amplified many of the PCIe architectural challenges that complicate test and development activities. This includes a 2x reduction requirement for transmitter (Tx) jitter and a 3x reference clock jitter reduction.
Signal loss remains an ongoing architectural challenge with PCIe 5.0. Retimers, redrivers and alternate printed circuit board (PCB) base materials are some of the cost/benefit options available to hardware designers to mitigate these concerns. A retimer can be used to retransmit the signal or a redriver can be used to amplify it, with either method improving the physical reach of the link.
The PCI Express 5.0 specification defines a channel loss budget of 36db, which is only ~28% greater than the PCIe 4.0 threshold. The insertion loss inherent to PCIe technology continues with the PCIe 5.0 release, so the loss levels experienced with FR4 PCB construction are no longer tenable and alternate materials like MEGTRON, costing up to 2.5 times more than FR4, have become compulsory. New equalization circuit designs for the transmitter (TX) and receiver (RX) have also been necessitated by the channel requirements for PCI Express 5.0, and lane margining at Rx for both voltage and timing has become mandatory.
PCIe 5 vs PCIe 4
In making the jump from PCIe 4.0 to PCI Express 5.0, speed doubling, backwards compatibility and an accelerated release cycle were three essential pillars upon which the implementation strategy was constructed.
New features that enabled or supported the speed increase were prioritized over other recommended or requested changes. For example, the EIEOS and data bit rate definition changes were required to enable the speed increase, but fundamental PCIe elements such as the encoding method and target bit error rate (BER) remained constant. The signaling and scrambling schemes also remained consistent with PCIe Gen 4, and existing transmitter (Tx) and receiver (Rx) test methods were leveraged where possible to minimize the implementation impact.
Despite the emphasis on time-to-market and compatibility, other important design changes between PCIe versions 4.0 and 5.0 necessarily impacted mating hardware and test practices. These included clock data recovery (CDR) with a second-order response and a CEM connector that is only compatible with a surface mount PCBA footprint, although it remains backwards compatible at the add-in card interface. PCI Express 5.0 also supports alternate protocols, through modified TS1/TS2 sequences.
Consistency between PCIe 4.0 and PCIe 5.0 standards was further necessitated by the unusually long release time for the 4.0 standard, as the network landscape and bandwidth demand continued to unfold in the background. This has virtually assured an overlap period between the two standards, making commonality in design and test practices essential for a smooth transition.
PCIe 5.0 Physical Layer Updates
In addition to the more stringent jitter requirements, channel loss budget constraints, and lane margining requirements for voltage and time that accompanied the PCI Express Gen 5 release, additional physical layer changes were necessitated by the speed increase, while other advancements were included to maintain the requisite backwards compatibility with previous PCIe versions.
Ordered set changes were an important modification accompanying the PCI Express 5.0 specification release. The EIEOS ordered set is used to facilitate an exit from an electrical idle state. The familiar pattern of sixteen zeroes and ones used for each PCIe 4.0 ordered pair became thirty-two zeroes and ones repeated for each lane in the PCIe Gen 5 convention. Back-to-back (repeated) EIEOS signals are an additional PCIe 5.0 protocol change. The Start of Data Stream Ordered Set (SDS) has also been updated, so that PCI Express Gen 5 data stream origin points can be clearly distinguished by the receiver.
Training sequences (TS1/TS2) have benefitted from innovative new options intended to facilitate the PCIe Gen 5 speed doubling. Training sequences are a necessary precursor for link bring-up and equalization (EQ), but can also cause delays as the ordered sets progress through each speed support increment, starting from 2.5 GT/sec and moving step-wise to the 32.0 GT/sec PCIe Gen 5 speed. To remedy this dilemma, EQ bypass options have been provided to essentially “skip” the intermediate speed equalization levels, or to omit equalization altogether by using the “No EQ” option for immediate transition to the L0 active data transfer state.
New fields for alternate protocol ID and enhanced precoding support have also been added to the modified TS1 and TS2 of PCIe Gen 5. Once the negotiation between system and device is successful, the link can immediately move to the L0 state at the highest supported speed and begin to transfer data using the negotiated alternate protocol. Should an alternate protocol negotiation fail, the system can quickly revert to the backbone PCIe 5.0 protocol.
PCIe 5.0 Test Scenarios and Solutions
A nearly unlimited variety of link conditions and scenarios creates troubleshooting challenges during both the bring-up (pre-L0) phase and fully-up (L0) phases of PCIe 5.0 link activity. The best PCIe Gen 5 test solutions enable a layered, systematic approach that significantly reduces troubleshooting time and effort while improving ongoing system performance metrics.
Link bring-up issues observed during the Link Training and Status State Machine (LTSSM) state include signal integrity and detection issues, incorrect link speeds, and other potential conditions that can be effectively diagnosed using a protocol analyzer. After the L0 state is attained, performance inefficiencies such as excessive replays, recovery issues, and delays at various layers of the protocol stack can also be efficiently detected and mitigated with the assistance of an advanced PCIe protocol analyzer.
One common PHY layer condition requiring precision diagnostic capability is experienced during the exit state from electrical idle (EI), where mismatches between Tx and Rx logic can introduce latency. The VIAVI Xgig Analyzer identifies these conditions reliably by employing short lock times and advanced post-processing capabilities, thereby minimizing the loss of captured data during low power state transitions.
Real-time metrics for the Physical layer are extremely important for monitoring overall signal health and the recovery data of links. For example, when the link is operating in the L0 state, repeated replays and recoveries, potentially creating a significant downgrade in system performance, often go undetected. Xgig Real-time monitoring, metrics, and post-capture analysis features with per-lane resolution effectively monitor and diagnose non-acknowledgments (NAKs), replays, link errors, and flow control (FC) statistics such as receiver buffer overflows and excessive transaction queue depths.
The Future of PCIe 5.0
The breakneck cadence of PCIe release dates seems destined to continue with the final specification release for PCIe 6.0 expected in 2021. This new iteration will continue the traditional bandwidth doubling and backwards compatibility that have signified the PCIe standard, this time reaching a staggering 256 GB/sec of bidirectional bandwidth. This will effectively put PCIe on par with the VRAM bandwidth of a low-end GPU.
To enable yet another two-fold speed increase and maintain high reliability standards, pulse amplitude modulation (PAM4) and forward error correction (FEC) technology will be incorporated. Artificial Intelligence and Machine Learning are potential beneficiaries of this enhancement, as their performance relies upon a combination of exceptional speed, low latency, and rapid access to multiple peripherals simultaneously.
PCIe Gen 5 is yet another successful leap in I/O bussing technology. It seems that the PCIe 5.0 specification will be keeping pace with Moore’s law while shedding the networking architecture bottleneck mantle for the foreseeable future. With new and improved PCIe 5.0 test tools arriving every day, this progress should continue successfully through the release of PCIe 6.0 and many future generations to come.