Maximizing PCIe NVMe SSDs: Exploring NVMe over Fabrics

In the realm of storage technology, the demand for high-performance solutions has led to the development of innovative approaches. One such advancement is NVMe over Fabrics (NVMe-oF), an implementation of the NVMe standard that expands the possibilities of PCIe NVMe SSDs beyond traditional limitations[1]. This blog post explores how NVMe-oF harnesses the power of storage networking fabrics to deliver remarkable performance gains and scalability.

Understanding NVMe and NVMe-oF

NVMe, or Non-Volatile Memory Express, has redefined storage interfaces, offering unparalleled speed compared to legacy protocols like SATA and SAS. NVMe-oF takes this a step further by enabling the NVMe protocol to traverse storage networking fabrics. This solution bridges the gap between local storage and remote access, ushering in new dimensions of performance and flexibility.

Benefits of NVMe-oF

NVMe-oF introduces a host of benefits that make it an attractive choice for modern storage solutions:

  1. Storage Disaggregation: With NVMe-oF, compute and storage are decoupled, allowing independent scaling according to operational needs.
  2. Fabric Agnosticism: NVMe-oF seamlessly integrates with a range of networking fabrics, from Fibre Channel to RDMA protocols like InfiniBand and RoCE.
  3. Limited Performance Impact: Despite its remote nature, NVMe-oF delivers speeds approaching local NVMe storage with minimal latency impact.

NVMe-oF in Different Environments

This innovative technology adapts well to various environments, presenting opportunities for independent resource scaling.[2] Unlike traditional setups where compute and storage are tightly intertwined, NVMe-oF allows for dynamic expansion of storage resources without affecting compute components, and vice versa. This elasticity translates into heightened flexibility for both hardware and resource allocation.[3]

Storage access with NVMe-oF illustrated with a diagram.
Storage access with NVMe-oF
Diagram showing storage access with traditional local NVMe storage.
Local NVMe storage access

Efficient Storage Access with NVMe-oF

In NVMe-oF architecture, compute and storage separation paves the way for unlimited connections. Numerous client devices can access a shared NVMe storage pool, ensuring optimized utilization based on specific requirements. This separation facilitates seamless upgrades or changes to client devices, without necessitating adjustments on the storage side. This architecture empowers organizations to tailor their infrastructure to evolving demands.

Diagram describing the overall architecture of NVMe-oF used in a storage network.
NVMe-oF storage architecture

Enhancing Networked Storage Management

NVMe-oF serves as a vital abstraction layer, facilitating direct communication between compute and storage nodes. This streamlined interaction eliminates unnecessary translation overhead[4] and, consequently, reduces latency. The result is storage access that rivals local NVMe solutions, blurring the lines between remote and local access.

Versatility Across Network Fabrics

NVMe-oF’s adaptability across diverse network fabrics amplifies its practicality. Whether leveraging Fibre Channel, TCP/IP, or RDMA (including InfiniBand, RoCE, and more), NVMe-oF seamlessly integrates into existing infrastructure. This fabric-agnostic nature empowers network operators to unlock the full potential of NVMe in their preferred networking environments.

Diagram showing a selection of network fabrics that can be used with NVMe-oF.
a selection of network fabrics that can be used with NVMe-oF

Low Latency Across Distances

A hallmark of NVMe-oF is its ability to provide near-local NVMe performance across distances. Remarkably, the NVMe-oF standard introduces a mere 10 µs latency overhead compared to local NVMe. This nominal overhead highlights the efficiency of the architecture, with the bulk of latency stemming from the underlying NAND flash on the NVMe SSD.

Bar chart displaying the difference in latency between different types of storage.
Source: NVM Express®, Intel Open Source Technology Center

Optimizing NVMe-oF: Enterprise-class SSDs and Customized Solutions

Embracing the NVMe-oF standard, Exascend presents enterprise-class NVMe SSDs with firmware optimized for this technology. This compatibility facilitates seamless integration into next-generation storage infrastructure. Exascend’s commitment to NVMe-oF underscores its dedication to realizing the full potential of networked NVMe storage.

Exascend’s involvement in the NVMe-oF landscape extends beyond compatibility. Our exclusive technologies and customization services further enhance NVMe-oF-capable SSDs. This flexibility empowers network infrastructure operators to deploy NVMe-oF even in challenging environments, previously unattainable with enterprise drives.

Conclusion

NVMe over Fabrics emerges as a groundbreaking advancement in the world of storage technology. By untethering the potential of PCIe NVMe SSDs from local constraints, NVMe-oF redefines performance and scalability. Its ability to maintain low latency across distances and seamless integration into various network fabrics showcases its adaptability and transformative impact. As enterprises increasingly seek performance-driven storage solutions, NVMe-oF shines as a beacon of innovation, revolutionizing storage access and management.

Explore our M.2, U.2/U.3, and EDSFF NVMe SSDs

Explore our storage solutions for servers and data center


[1] NVM Express® Moves Into The Future, NVM Express.

[2] Benefits of Disaggregating NVMe Storage with NVMe-oF Technology, NVM Express.

[3] NVMe-oF: Looking Beyond Performance Hero Numbers, Storage Networking Industry Association.

[4] When NVMe over Fabrics Meets Arm: Performance and Implications, IEEE.

Scroll to Top

Exascend Newsletter

Subscribe to our newsletter to receive updates about products, services, events and resources you may find of interest.

Download whitepaper

There's no time like the present.

Send us an inquiry