Virtual Block

The physical addressing interface nvm_addr provides full control over the construction of vectorized IO commands. It is known that with great power comes great responsibility. Responsibility which increase the cognitive load on the developer when integrating vectorized IO at the application level.

liblightnvm, therefore, introduces a pure software abstraction, a virtual block, to reduce the cognitive load for application developers.

A virtual block behaves as a physical, that is, the constraints of working with NAND media also apply to a virtual block. However, the abstraction encapsulates the command and address construction of parallel vectorized IO and exposes a flat address space which is read/written in a manner equivalent to the read/write primitives offered by libc.

The minimum construction encapsulates managing address-construction across planes and setting plane hints. See section Plane Span. Increased utilization of parallel units in a device is achieved by constructing a virtual block as either a Block Set or a Block Line.

Note

C API documentation of the virtual block is available in nvm_vblk - Virtual Block

Plane Span

A virtual block will at a minimum consist of the given block address across all planes. So it by default encapsulates concerns regarding plane-mode constraints.

We erase, write, and read the virtual block:

nvm_vblk set_erase /dev/nvme0n1 0x0000000000000002
nvm_vblk set_write /dev/nvme0n1 0x0000000000000002
nvm_vblk set_read /dev/nvme0n1 0x0000000000000002

Yielding:

vblk:
  dev: {pmode: 'DUAL'}
  nbytes: 16777216
  nmbytes: 16
naddrs: 1
addrs:
  - 0x0000000000000002: {ch: 00, lun: 00, pl: 0, blk: 0002, pg: 000, sec: 0}
nvm_vblk_erase: {elapsed: 0.006637}

vblk:
  dev: {pmode: 'DUAL'}
  nbytes: 16777216
  nmbytes: 16
naddrs: 1
addrs:
  - 0x0000000000000002: {ch: 00, lun: 00, pl: 0, blk: 0002, pg: 000, sec: 0}
nvm_buf_alloc: {elapsed: 0.000015}
nvm_buf_fill: {elapsed: 0.022563}
nvm_vblk_write: {elapsed: 0.886366}

vblk:
  dev: {pmode: 'DUAL'}
  nbytes: 16777216
  nmbytes: 16
naddrs: 1
addrs:
  - 0x0000000000000002: {ch: 00, lun: 00, pl: 0, blk: 0002, pg: 000, sec: 0}
nvm_buf_alloc: {elapsed: 0.000012}
nvm_vblk_read: {elapsed: 0.138818}

Block Set

To obtain parallel IO, in addition to plane-mode, we construct a virtual block consisting of multiple physical blocks on distinct parallel units (LUNs):

(0x0000000000000002){ ch(00), lun(00), pl(0), blk(0002), pg(000), sec(0) }
(0x0a0700000000014d){ ch(10), lun(07), pl(0), blk(0333), pg(000), sec(0) }
(0x0301000000000014){ ch(03), lun(01), pl(0), blk(0020), pg(000), sec(0) }
(0x0500000000000190){ ch(05), lun(00), pl(0), blk(0400), pg(000), sec(0) }

We erase, write and read the virtual block:

nvm_vblk set_erase /dev/nvme0n1 0x0000000000000002 0x0a0700000000014d 0x0301000000000014 0x0500000000000190
nvm_vblk set_write /dev/nvme0n1 0x0000000000000002 0x0a0700000000014d 0x0301000000000014 0x0500000000000190
nvm_vblk set_read /dev/nvme0n1 0x0000000000000002 0x0a0700000000014d 0x0301000000000014 0x0500000000000190

Yielding:

vblk:
  dev: {pmode: 'DUAL'}
  nbytes: 67108864
  nmbytes: 64
naddrs: 4
addrs:
  - 0x0000000000000002: {ch: 00, lun: 00, pl: 0, blk: 0002, pg: 000, sec: 0}
  - 0x0a0700000000014d: {ch: 10, lun: 07, pl: 0, blk: 0333, pg: 000, sec: 0}
  - 0x0301000000000014: {ch: 03, lun: 01, pl: 0, blk: 0020, pg: 000, sec: 0}
  - 0x0500000000000190: {ch: 05, lun: 00, pl: 0, blk: 0400, pg: 000, sec: 0}
nvm_vblk_erase: {elapsed: 0.024083}

vblk:
  dev: {pmode: 'DUAL'}
  nbytes: 67108864
  nmbytes: 64
naddrs: 4
addrs:
  - 0x0000000000000002: {ch: 00, lun: 00, pl: 0, blk: 0002, pg: 000, sec: 0}
  - 0x0a0700000000014d: {ch: 10, lun: 07, pl: 0, blk: 0333, pg: 000, sec: 0}
  - 0x0301000000000014: {ch: 03, lun: 01, pl: 0, blk: 0020, pg: 000, sec: 0}
  - 0x0500000000000190: {ch: 05, lun: 00, pl: 0, blk: 0400, pg: 000, sec: 0}
nvm_buf_alloc: {elapsed: 0.000012}
nvm_buf_fill: {elapsed: 0.067980}
nvm_vblk_write: {elapsed: 0.954087}

vblk:
  dev: {pmode: 'DUAL'}
  nbytes: 67108864
  nmbytes: 64
naddrs: 4
addrs:
  - 0x0000000000000002: {ch: 00, lun: 00, pl: 0, blk: 0002, pg: 000, sec: 0}
  - 0x0a0700000000014d: {ch: 10, lun: 07, pl: 0, blk: 0333, pg: 000, sec: 0}
  - 0x0301000000000014: {ch: 03, lun: 01, pl: 0, blk: 0020, pg: 000, sec: 0}
  - 0x0500000000000190: {ch: 05, lun: 00, pl: 0, blk: 0400, pg: 000, sec: 0}
nvm_buf_alloc: {elapsed: 0.000014}
nvm_vblk_read: {elapsed: 0.146335}

This demonstrates a weak-scaling experiment, increasing the workload proportionally with the parallel units consumes approximately the same amount of wall-clock time, thus achieving near linear speedup by utilizing parallel units on the device.

Note

The difference between the plane span and the block set is merely that the virtual block creation uses multiple block-addresses instead of one

Block Line

The block line provides convenient construction of a virtual block spanning lines of physical blocks. With a block line, the block index is fixed and one can specify true subsets of the parallel units. For example, block 10 on all parallel units on the given device, which in this case has 16 channels with each eight LUNs:

nvm_vblk line_erase /dev/nvme0n1 0 15 0 7 10
nvm_vblk line_write /dev/nvme0n1 0 15 0 7 10
nvm_vblk line_read /dev/nvme0n1 0 15 0 7 10

Yielding:

vblk:
  dev: {pmode: 'DUAL'}
  nbytes: 2147483648
  nmbytes: 2048
naddrs: 128
addrs:

... block address output omitted for brevity ...

  - 0x0e0700000000000a: {ch: 14, lun: 07, pl: 0, blk: 0010, pg: 000, sec: 0}
  - 0x0f0700000000000a: {ch: 15, lun: 07, pl: 0, blk: 0010, pg: 000, sec: 0}
nvm_vblk_erase: {elapsed: 0.085505}

vblk:
  dev: {pmode: 'DUAL'}
  nbytes: 2147483648
  nmbytes: 2048
naddrs: 128
addrs:

... block address output omitted for brevity ...

  - 0x0e0700000000000a: {ch: 14, lun: 07, pl: 0, blk: 0010, pg: 000, sec: 0}
  - 0x0f0700000000000a: {ch: 15, lun: 07, pl: 0, blk: 0010, pg: 000, sec: 0}
nvm_buf_alloc: {elapsed: 0.000015}
nvm_buf_fill: {elapsed: 0.907555}

vblk:
  dev: {pmode: 'DUAL'}
  nbytes: 2147483648
  nmbytes: 2048
naddrs: 128

... block address output omitted for brevity ...

  - 0x0d0700000000000a: {ch: 13, lun: 07, pl: 0, blk: 0010, pg: 000, sec: 0}
  - 0x0e0700000000000a: {ch: 14, lun: 07, pl: 0, blk: 0010, pg: 000, sec: 0}
  - 0x0f0700000000000a: {ch: 15, lun: 07, pl: 0, blk: 0010, pg: 000, sec: 0}
nvm_buf_alloc: {elapsed: 0.000014}