Virtual Block¶
The command interface nvm_cmd provides full control over the construction
of vectorized IO commands. It is known that with great power comes great
responsibility. Responsibility which increase the cognitive load on the
developer when integrating vectorized IO at the application level.
liblightnvm, therefore, introduces a pure software abstraction, a virtual block, to reduce the cognitive load for application developers.
A virtual block behaves as a physical, that is, the constraints of working with NAND media also apply to a virtual block. However, the abstraction encapsulates the command and address construction of parallel vectorized IO and exposes a flat address space which is read/written in a manner equivalent to the read/write primitives offered by libc.
See section Single Chunk for the minimal construction of virtual blocks. Increased utilization of parallel units in a device is achieved by constructing a virtual block as either a Chunk Set or a Chunk Line.
Note
C API documentation of the virtual block is available in nvm_vblk - Virtual Block
Single Chunk¶
A virtual block will at a minimum consist of a single chunk.
We erase, write, and read the virtual block:
nvm_vblk erase /dev/nvme0n1 0x0402008e00000000
nvm_vblk write /dev/nvme0n1 0x0402008e00000000
nvm_vblk read /dev/nvme0n1 0x0402008e00000000
Yielding:
vblk:
dev: {pmode: 'SNGL'}
nblks: 1
nmbytes: 24
pos_write: 0
pos_read: 0
flags: 0x08c8
naddrs: 1
addrs:
- {val: 0x0402008e00000000, pugrp: 04, punit: 02, chunk: 0142, sectr: 0000}
nvm_vblk_erase: {elapsed: 0.0050, mb: 24.00, mbsec: 4809.93}
vblk:
dev: {pmode: 'SNGL'}
nblks: 1
nmbytes: 24
pos_write: 0
pos_read: 0
flags: 0x08c8
naddrs: 1
addrs:
- {val: 0x0402008e00000000, pugrp: 04, punit: 02, chunk: 0142, sectr: 0000}
nvm_buf_alloc: {elapsed: 0.000017}
nvm_buf_fill: {elapsed: 0.016964}
nvm_vblk_write: {elapsed: 0.8327, mb: 24.00, mbsec: 28.82}
vblk:
dev: {pmode: 'SNGL'}
nblks: 1
nmbytes: 24
pos_write: 0
pos_read: 0
flags: 0x08c8
naddrs: 1
addrs:
- {val: 0x0402008e00000000, pugrp: 04, punit: 02, chunk: 0142, sectr: 0000}
nvm_buf_alloc: {elapsed: 0.000017}
nvm_buf_fill: {elapsed: 0.015488}
nvm_vblk_read: {elapsed: 0.1971, mb: 24.00, mbsec: 121.76}
Chunk Set¶
To obtain parallel IO, we construct a virtual block consisting of multiple chunks on distinct parallel units:
{val: 0x0000008e00000000, pugrp: 00, punit: 00, chunk: 0142, sectr: 0000}
{val: 0x0100008e00000000, pugrp: 01, punit: 00, chunk: 0142, sectr: 0000}
{val: 0x0200008e00000000, pugrp: 02, punit: 00, chunk: 0142, sectr: 0000}
{val: 0x0300008e00000000, pugrp: 03, punit: 00, chunk: 0142, sectr: 0000}
We erase, write and read the virtual block:
nvm_vblk set_erase /dev/nvme0n1 0x0000008e00000000 0x0100008e00000000 0x0200008e00000000 0x0300008e00000000
nvm_vblk set_write /dev/nvme0n1 0x0000008e00000000 0x0100008e00000000 0x0200008e00000000 0x0300008e00000000
nvm_vblk set_read /dev/nvme0n1 0x0000008e00000000 0x0100008e00000000 0x0200008e00000000 0x0300008e00000000
Yielding:
vblk:
dev: {pmode: 'SNGL'}
nblks: 4
nmbytes: 96
pos_write: 0
pos_read: 0
flags: 0x08c8
naddrs: 4
addrs:
- {val: 0x0000008e00000000, pugrp: 00, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0100008e00000000, pugrp: 01, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0200008e00000000, pugrp: 02, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0300008e00000000, pugrp: 03, punit: 00, chunk: 0142, sectr: 0000}
nvm_vblk_erase: {elapsed: 0.0051, mb: 96.00, mbsec: 18777.97}
vblk:
dev: {pmode: 'SNGL'}
nblks: 4
nmbytes: 96
pos_write: 0
pos_read: 0
flags: 0x08c8
naddrs: 4
addrs:
- {val: 0x0000008e00000000, pugrp: 00, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0100008e00000000, pugrp: 01, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0200008e00000000, pugrp: 02, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0300008e00000000, pugrp: 03, punit: 00, chunk: 0142, sectr: 0000}
nvm_buf_alloc: {elapsed: 0.000018}
nvm_buf_fill: {elapsed: 0.084907}
nvm_vblk_write: {elapsed: 0.8687, mb: 96.00, mbsec: 110.51}
vblk:
dev: {pmode: 'SNGL'}
nblks: 4
nmbytes: 96
pos_write: 0
pos_read: 0
flags: 0x08c8
naddrs: 4
addrs:
- {val: 0x0000008e00000000, pugrp: 00, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0100008e00000000, pugrp: 01, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0200008e00000000, pugrp: 02, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0300008e00000000, pugrp: 03, punit: 00, chunk: 0142, sectr: 0000}
nvm_buf_alloc: {elapsed: 0.000004}
nvm_buf_fill: {elapsed: 0.042715}
nvm_vblk_read: {elapsed: 0.1791, mb: 96.00, mbsec: 535.95}
This demonstrates a weak-scaling experiment, increasing the workload proportionally with the parallel units consumes approximately the same amount of wall-clock time, thus achieving near linear speedup by utilizing parallel units on the device.
Note
The difference between the plane span and the block set is merely that the virtual block creation uses multiple block-addresses instead of one
Chunk Line¶
The chunk line provides convenient construction of a virtual block spanning lines of chunks. With a chunk line, the block index is fixed and one can specify true subsets of the parallel units.
For example, chunk 142 on all parallel units on the given device, which in this case has eight parallel unit groups, each with four parallel units:
nvm_vblk line_erase /dev/nvme0n1 0 7 0 3 142
nvm_vblk line_write /dev/nvme0n1 0 7 0 3 142
nvm_vblk line_read /dev/nvme0n1 0 7 0 3 142
Yielding:
vblk:
dev: {pmode: 'SNGL'}
nblks: 32
nmbytes: 768
pos_write: 0
pos_read: 0
flags: 0x08c8
naddrs: 32
addrs:
- {val: 0x0000008e00000000, pugrp: 00, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0100008e00000000, pugrp: 01, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0200008e00000000, pugrp: 02, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0300008e00000000, pugrp: 03, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0400008e00000000, pugrp: 04, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0500008e00000000, pugrp: 05, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0600008e00000000, pugrp: 06, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0700008e00000000, pugrp: 07, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0001008e00000000, pugrp: 00, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0101008e00000000, pugrp: 01, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0201008e00000000, pugrp: 02, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0301008e00000000, pugrp: 03, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0401008e00000000, pugrp: 04, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0501008e00000000, pugrp: 05, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0601008e00000000, pugrp: 06, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0701008e00000000, pugrp: 07, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0002008e00000000, pugrp: 00, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0102008e00000000, pugrp: 01, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0202008e00000000, pugrp: 02, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0302008e00000000, pugrp: 03, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0402008e00000000, pugrp: 04, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0502008e00000000, pugrp: 05, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0602008e00000000, pugrp: 06, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0702008e00000000, pugrp: 07, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0003008e00000000, pugrp: 00, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0103008e00000000, pugrp: 01, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0203008e00000000, pugrp: 02, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0303008e00000000, pugrp: 03, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0403008e00000000, pugrp: 04, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0503008e00000000, pugrp: 05, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0603008e00000000, pugrp: 06, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0703008e00000000, pugrp: 07, punit: 03, chunk: 0142, sectr: 0000}
nvm_vblk_erase: {elapsed: 0.0053, mb: 768.00, mbsec: 144101.25}
vblk:
dev: {pmode: 'SNGL'}
nblks: 32
nmbytes: 768
pos_write: 0
pos_read: 0
flags: 0x08c8
naddrs: 32
addrs:
- {val: 0x0000008e00000000, pugrp: 00, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0100008e00000000, pugrp: 01, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0200008e00000000, pugrp: 02, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0300008e00000000, pugrp: 03, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0400008e00000000, pugrp: 04, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0500008e00000000, pugrp: 05, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0600008e00000000, pugrp: 06, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0700008e00000000, pugrp: 07, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0001008e00000000, pugrp: 00, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0101008e00000000, pugrp: 01, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0201008e00000000, pugrp: 02, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0301008e00000000, pugrp: 03, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0401008e00000000, pugrp: 04, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0501008e00000000, pugrp: 05, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0601008e00000000, pugrp: 06, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0701008e00000000, pugrp: 07, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0002008e00000000, pugrp: 00, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0102008e00000000, pugrp: 01, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0202008e00000000, pugrp: 02, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0302008e00000000, pugrp: 03, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0402008e00000000, pugrp: 04, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0502008e00000000, pugrp: 05, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0602008e00000000, pugrp: 06, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0702008e00000000, pugrp: 07, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0003008e00000000, pugrp: 00, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0103008e00000000, pugrp: 01, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0203008e00000000, pugrp: 02, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0303008e00000000, pugrp: 03, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0403008e00000000, pugrp: 04, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0503008e00000000, pugrp: 05, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0603008e00000000, pugrp: 06, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0703008e00000000, pugrp: 07, punit: 03, chunk: 0142, sectr: 0000}
nvm_buf_alloc: {elapsed: 0.000019}
nvm_buf_fill: {elapsed: 0.436813}
nvm_vblk_write: {elapsed: 0.8846, mb: 768.00, mbsec: 868.15}
vblk:
dev: {pmode: 'SNGL'}
nblks: 32
nmbytes: 768
pos_write: 0
pos_read: 0
flags: 0x08c8
naddrs: 32
addrs:
- {val: 0x0000008e00000000, pugrp: 00, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0100008e00000000, pugrp: 01, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0200008e00000000, pugrp: 02, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0300008e00000000, pugrp: 03, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0400008e00000000, pugrp: 04, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0500008e00000000, pugrp: 05, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0600008e00000000, pugrp: 06, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0700008e00000000, pugrp: 07, punit: 00, chunk: 0142, sectr: 0000}
- {val: 0x0001008e00000000, pugrp: 00, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0101008e00000000, pugrp: 01, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0201008e00000000, pugrp: 02, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0301008e00000000, pugrp: 03, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0401008e00000000, pugrp: 04, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0501008e00000000, pugrp: 05, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0601008e00000000, pugrp: 06, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0701008e00000000, pugrp: 07, punit: 01, chunk: 0142, sectr: 0000}
- {val: 0x0002008e00000000, pugrp: 00, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0102008e00000000, pugrp: 01, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0202008e00000000, pugrp: 02, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0302008e00000000, pugrp: 03, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0402008e00000000, pugrp: 04, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0502008e00000000, pugrp: 05, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0602008e00000000, pugrp: 06, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0702008e00000000, pugrp: 07, punit: 02, chunk: 0142, sectr: 0000}
- {val: 0x0003008e00000000, pugrp: 00, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0103008e00000000, pugrp: 01, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0203008e00000000, pugrp: 02, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0303008e00000000, pugrp: 03, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0403008e00000000, pugrp: 04, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0503008e00000000, pugrp: 05, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0603008e00000000, pugrp: 06, punit: 03, chunk: 0142, sectr: 0000}
- {val: 0x0703008e00000000, pugrp: 07, punit: 03, chunk: 0142, sectr: 0000}
nvm_buf_alloc: {elapsed: 0.000005}
nvm_buf_fill: {elapsed: 0.347876}
nvm_vblk_read: {elapsed: 0.3606, mb: 768.00, mbsec: 2129.79}