PCIe NVMe

Non-Volatile Memory Express (NVMe) is a solid-state memory interface protocol developed to leverage the characteristics of the PCIe buss and optimize NVM performance. Introduced in 2011, NVMe uses the PCIe interface to map commands and responses to the host computer. 

Due to the streamlined command set inherent to NVMe, fewer than half the quantity of instructions used in a conventional SCSI to SAS interface are required. This technology has produced impressive results with read and write speeds up to four times faster than conventional SATA SSDs. NVMe provides up to 64 queues, breaking the paradigm of contiguous circle memory utilized for decades in conventional hard drives. 

PCI Express

Peripheral Component Interconnect Express (PCIe) is the industry standard high-speed bus convention for nearly all internal computer connections. The point-to-point topology, multiple lanes and innovative clocking scheme are among the many features that have made PCIe a revolutionary bus architecture. 

PCI Express cards can perform as many as 16 transfers in parallel. With the widespread acceptance and popularity of PCIe, internal and external expansion cards have continued to adopt this platform. In recent years, solid state device (SSD) technology has also reaped the benefits of PCIe.

NVMe Over Fabrics

NVMe over Fabrics (NVMe-oF) has extended the range of NVMe beyond the computer and into a wide range of networking fabrics. NVMe-oF also establishes the use of alternate transports to PCIe that extend the distances across which an NVMe host device and NVMe storage device can be connected. This specification is targeted to shared SSD storage over a network, such as Fiber Channel, Ethernet and InfiniBand. 

While this new enhancement increases distances over which NVMEe devices can be accessed and connected, the goal is to introduce less than 10µs of additional latency in the process. Among the required characteristics of an NVME-oF network are credit-based flow and low latency fabric. The NVMe-oF specification is about 90 percent equivalent to the NVMe specification with the transport mechanisms defined and characterized for additional guidance. 

PCIe and NVME Working Together

The amalgamation of PCIe and NVMe is a classic example of the whole becoming greater than the sum of the parts. Although the benefits of NVM vs. rotating-disk storage have long been acknowledged, using an SAS or SATA interface for the SSD introduces an additional bottleneck.

While traditional architecture included I/O controllers between the CPU and storage, PCIe has provided a convenient shortcut. With a direct connection from PCIe to CPU, the NVMe device now has a path of least resistance that leads to scalable performance through lane utilization and lower power consumption and latency, among other tangible benefits.

By tailoring to the specific characteristics of NVM and Flash, NVMe was developed specifically for PCIe, rather than adapting an existing protocol such as SATA, thereby creating a perfect union of opportunity and innovation. It is important to note that the NVMe designation does not indicate any specific type of memory or device, but rather a standardized communication protocol that lays the groundwork for NVM through a PCIe interface. 

Viewing the NVMe Traffic

The unparalleled results observed when NVMe is coupled with PCIe speak for themselves. For example, both read and write speeds are more than double that of 12Gbps SAS, and seek times are more than ten times faster than with SATA SSD. With the resultant increase in traffic, an analyzer with PCIe Gen 4.0 capability, superior memory capacity, and the segmentation features required to facilitate NVMe upstream and downstream, traffic decoding has become an essential tool. 

The VIAVI Gen 4 PCIe Analyzer feature set enables complete visibility into these sedulous traffic flows, and PCIe and NVMe traffic can be decoded at all levels of the stack. With NVMe Management Interface (NVMe-MI), the command set and architecture for NVMe storage is observable through SMB capture and trigger. 

Testing the NVMe Protocol

Robust testing of the NVMe storage protocol necessitates an advanced and versatile test solution. The jamming and analysis capabilities of the Gen 4 PCIe Analyzer addresses these requirements with equal aplomb. 

Advanced triggering and search features include training sequences, ordered sets, and queue pairs. Xgig is an integrated platform and also allows multiple protocols to be supported in a multi-function environment by simply connecting multiple chassis. The Xgig platform enables accurate as well as expedient troubleshooting and protocol testing. Error injection or “jamming” provides a means to introduce errors into the system, thereby verifying the efficacy of the error recovery process during protocol testing.

The Future of NVMe

The next revision of the NVMe protocol, version 1.4, will support IO determinism, meaning SSDs of the not-so-distant future will be treated as a conglomeration of smaller sub-SSDs by the host, thereby enabling independent, parallel IO processing by each sub-SSD. This in turn will lead to reduced read latency and improved overall performance. 

NVMe will continue to evolve through NVMe 1.4 and beyond, adding additional features and capability while maintaining the original benefits that have made it unique.

Although the traditional hard drive may endure as a mainstay for years to come, NVM continues to gain traction, with NVMe at the forefront. The speed and versatility of NVMe coupled with PCIe has secured this combination as a viable add-on or alternate memory solution, and enhancements will doubtless continue in the years ahead. The future of NVMe is secured, with only the imagination and ingenuity of designers limiting the uptick in capacity and potential applications.