Bring Up / Teardown¶
Create a pblk instance named
pblkdev using parallel units 0-127:
sudo nvme lnvm create -d /dev/nvme0n1 -n pblkdev -t pblk -b 0 -e 127
pblkdev is removed by invoking:
sudo nvme lnvm remove –n pblkdev
Imbalanced instance An instances has an unbalanced configuration of parallel units.
Overlapping instance(s) One of more instances use the same parallel units.
State and Debugging (sysfs)¶
Once the pblk instance is created and the block device available, a sysfs interface is exposed that allows further communication with the pblk instance. This interface can be found at /dev/block/$INSTANCE_NAME/pblk/. Some of the available files are always exposed to the user, while others that require collecting statistics on the FTL itself, require the debug mode activate.
The following subsections describe each file and groups them by Read Files, Debug Read Files Control Knobs, and Debug Knobs.
Current state of the rate limiter in terms of the number of in-flight user and garbage-collection (GC) I/O.
There are 53 entries on the write buffer which contain user I/O data out of 16384 total entries on the buffer. There are 0 entries being used for GC I/O out of 0 available. This means that GC is not running.
This is because there are 130921 free blocks out of 136320 total blocks on the device, and GC does not run until the 32768 free block threshold is reached. At <4096 free blocks, user I/O stops.
All these parameters are configurable
Current state of the write buffer.
16384 53 16 16 64 0 4294967295 - 53/16383/0 – 0
Total entries: 16384, mem:53, subm:16, sync:16, l2p_update:64, sync_point:0, number of entries being used:53, number of free entries:16330, entries to be flushed:0, entries queued on completion: 0.
Lines lists as well as the metadata for the current data line.
lines GC: full:12, high:1, mid:0, low:5, empty:2 data (21) cur:16, left:523248/523248, vsc:523248, s:523248, map:0/524288 (0)
There are 12 lines with all sectors invalidated, 1 line with a high invalid sector count, 5 lines with a low sector count and 2 lines where all sectors are valid. The current data line is line 21. The write pointer is at sector 16 and there are 523248 sectors left out of 523248 (line is new).
The valid sector count (vsc) and the number of sectors in line is also 523248. Since the line is empty, no sectors have been map (0/523248). Finally, the line is using the metadata line 0 (out of 2 available).
General information about all data lines.
smeta - len:65536, secs:16 emeta - len:4194304, sec:1024, bb_start:64 bitmap lengths: sec:65536, blk:16, lun:16 blk_line:128, sec_line:524288, sec_blk:4096
The metadata placed at the beginning of the line occupies 16 sectors (length 65536 bytes). The metadata placed at the end of the line occupies 1024 sectors (length 4194304 bytes), and it spreads across 64 blocks. The bitmaps used for sectors are 65536 bytes long and for blocks and LUNs 16 bytes long. Each line is formed by 128 blocks. Each block has 4096 sectors, which translates into 524288 sectors per line.
pblk ppa format and the ppa format for the device in terms of offset and length.
- block (13 bits) – page (8 bits) – LUN (3 bits) – channel (4 bits) – plane (2 bits) – sector (2 bits)
- block (13 bits) – page (8 bits) – LUN (3 bits) – plane (2 bits) – sector (2 bits) – channel (4 bits)
Accumulated counts of global errors that occurred since the pblk target was instantiated.
read_failed=0, read_high_ecc=0, read_empty=0, read_failed_gc=0, write_failed=0, erase_failed=0
LUNs forming the pblk instance, its order and their state.
pblk: pos:106, ch:10, lun:6 – 0
The LUN in the position 106 corresponds to channel 10, LUN 6. The LUN is not busy.
Shows the state of the GC thread.
The GC thread is enabled but it is not active (no lines are being garbage collected).
Debug Read Files¶
The files described in this sections are only available when the kernel is
Detailed stats about the pblk instance.
0 0 1659328 0 0 1659328 1659328 1659328 0 0 0 0 0 693
Inflight writes:0, inflight reads:0, write request:1659328, flush requests:0, padded sectors:0, submitted writes:1659328, synced writes:1659328, completed writes:1659328, inflight metadata I/Os:0, completed meta I/Os:0, recovery writes:0, GC writes:0, requeued writes:0, synced reads: 693
Show state of each entry in the write buffer.
entry:8161 - flags:512
Entry 8161 is a writable entry. The following flags can be present in the write buffer and encoded as:
# I/O Type PBLK_IOTYPE_USER = 1 PBLK_IOTYPE_GC = 2 # Buffer State PBLK_FLUSH_ENTRY = 1 << 6 (64) PBLK_WRITTEN_DATA = 1 << 7 (128) PBLK_SUBMITTED_ENTRY = 1 << 8 (256) PBLK_WRITABLE_ENTRY = 1 << 9 (512)
Force the GC to run, independently of the number of free lines.
echo “1” > /sys/block/pblkdev/pblk/gc_force
Forces garbage collection routine to run.
The files described in this section are only available when the kernel is
Query the L2P table for the specified LBA range.
echo “0-1023” > /sys/block/pblkdev/pblk/l2p_map
lba:0(oob:0) - ppa: 300000000020002: ch:3,lun:0,blk:2,pg:2,pl:0,sec:0 LBA 0 is mapped to ppa 0x300000000020002 (ch:3,lun:0,blk:2,pg:2,pl:0,sec:0). Besides querying the L2P map, a read command is sent to the device and the result of the out-of-bound area is showed too. Note that the out-of-bound area contains the LBA to which the physical sector is mapped to.
Check sanity of the L2P table for a given PPA address on a LBA range.
At a given point in time, a PPA address should only be mapped to a single LBA address. If a PPA is mapped to several LBAs then a corruption on the L2P table has occurred.
Usage example (
echo "0-0-10" > /sys/block/pblkdev/pblk/l2p_sanity
Provides no output on success.
In case a corruption is found, the output will be similar to:
lba:0(oob:0) - ppa: 300000000020002: ch:3,lun:0,blk:2,pg:2,pl:0,sec:0 lba:0(oob:0) - ppa: 300000000020003: ch:3,lun:0,blk:2,pg3,pl:0,sec:0
Which translates into: LBA 0 is mapped to two different PPAS: 0x300000000020002 and 0x300000000020003.
Query the metadata stored on the media for the given line.
Usage example (
echo "0-0" > /sys/block/test/pblk/line_metadata
$CMD can be one of the following:
- 0: Print only the metadata
- 1: Print the metadata and the LBA list (L2P portion for line)
- 3: Print metadata and force to print the LBA list even if CRC checks fail
Query bad block metadata for the given line
echo "0" > /sys/block/test/pblk/line_bb
Provides an output similar to:
line 0, nr_bb:0, smeta:8, emeta:1024/0
Line 0 contains 0 bad blocks; smeta takes 8 sectors; emeta takes 1024 sectors and there are 0 bad blocks that affect emeta.
emeta is typically spread along the last page of all the blocks on the line. If a block is bad, then a block will contain 2 pages of metadata