Monday, April 10, 2023

NVMe & sysfs: a helper script

The Linux NVMe driver exports a really useful set of information to sysfs. While there's absolutely no comparison to nvme-cli's capabilities, there are a few nice things about using the information reported by sysfs to do basic inspection of the NVMe drives connected to a system. First, the information is available in user space, so super user privileges are not required. Second, there are no dependencies to download. Lastly, you can get some information that's not available in nvme-cli, or would require a set of nvme-cli commands to execute and parse.

I'm often adding drives to different systems and interested in the drives' NUMA affinity and whether they negotiated to the proper PCIe generation and link width. It's very easy to retrieve this information via sysfs and print it along with other useful information about the drives such as model numbers, firmware revisions, and namespace configurations.

I have a small Bash function that I use in my own custom .rc file. The output looks like this:

$ lsnvme

   Controller (PCI Address)    : nvme0 (0000:e6:00.0)
   Model                       : Dell DC NVMe PE8010 RI U.2 960GB
   Controller Type & Transport : reserved over pcie
   NUMA Node                   : 1
   Link Information            : 16.0 GT/s PCIe x4
   Serial Number               : SSA9N4572I1309E1N
   Firmware Revision           : 1.1.0
   Attached Namespace Count    : 1

      Namespace nvme0n1 (nsid = 1)
         Formatted / Physical Sector Size : 512 / 512
         Size in GB (512-Byte Sectors)    : 960 (1875385008)
         Globally Unique ID - NGUID       : NA
         Universally Unique ID - UUID     : NA
         Worldwide ID - WWID              : eui.ace42e00162bbe86

   Controller (PCI Address)    : nvme1 (0000:17:00.0)
   Model                       : CSD-3310
   Controller Type & Transport : io over pcie
   NUMA Node                   : 0
   Link Information            : 16.0 GT/s PCIe x4
   Serial Number               : UE2237C0787M
   Firmware Revision           : U3219141
   Attached Namespace Count    : 1

      Namespace nvme1n1 (nsid = 1)
         Formatted / Physical Sector Size : 512 / 4096
         Size in GB (512-Byte Sectors)    : 3840 (7501476528)
         Globally Unique ID - NGUID       : 55453232-3337-4330-4f55-49013738374d
         Universally Unique ID - UUID     : 55453232-3337-4330-4f55-49013738374d
         Worldwide ID - WWID              : eui.55453232333743304f5549013738374d


The dirt simple Bash function I use can be found here. I have an NVMe-centric way of looking at drives (controllers being very distinct entities from their attached namespaces) and often deal with multiple namespaces. The output format I prefer may not be optimal for someone prefers a namespace-centric presentation of the same information. The main point here is that if you find yourself needing to work with NVMe drives frequently, it might be worth adding a sysfs-based NVMe inspection function to your shell that's suited to your needs.

No comments:

Post a Comment

Three Things I Wish Every Storage Software Vendor Provided

In my work on SSDs, I have the opportunity to test a wide variety of storage software from parallel filesystems to high performance database...