Notice

This document is for a development version of Ceph.

BlueStore Internals

Small write strategies

  • U: Uncompressed write of a complete, new blob.

    • write to new blob

    • kv commit

  • P: Uncompressed partial write to unused region of an existing blob.

    • write to unused chunk(s) of existing blob

    • kv commit

  • W: WAL overwrite: commit intent to overwrite, then overwrite async. Must be chunk_size = MAX(block_size, csum_block_size) aligned.

    • kv commit

    • wal overwrite (chunk-aligned) of existing blob

  • N: Uncompressed partial write to a new blob. Initially sparsely utilized. Future writes will either be P or W.

    • write into a new (sparse) blob

    • kv commit

  • R+W: Read partial chunk, then to WAL overwrite.

    • read (out to chunk boundaries)

    • kv commit

    • wal overwrite (chunk-aligned) of existing blob

  • C: Compress data, write to new blob.

    • compress and write to new blob

    • kv commit

Possible future modes

  • F: Fragment lextent space by writing small piece of data into a piecemeal blob (that collects random, noncontiguous bits of data we need to write).

    • write to a piecemeal blob (min_alloc_size or larger, but we use just one block of it)

    • kv commit

  • X: WAL read/modify/write on a single block (like legacy bluestore). No checksum.

    • kv commit

    • wal read/modify/write

Mapping

This very roughly maps the type of write onto what we do when we encounter a given blob. In practice it’s a bit more complicated since there might be several blobs to consider (e.g., we might be able to W into one or P into another), but it should communicate a rough idea of strategy.

raw

raw (cached)

csum (4 KB)

csum (16 KB)

comp (128 KB)

128+ KB (over)write

U

U

U

U

C

64 KB (over)write

U

U

U

U

U or C

4 KB overwrite

W

P | W

P | W

P | R+W

P | N (F?)

100 byte overwrite

R+W

P | W

P | R+W

P | R+W

P | N (F?)

100 byte append

R+W

P | W

P | R+W

P | R+W

P | N (F?)

4 KB clone overwrite

P | N

P | N

P | N

P | N

N (F?)

100 byte clone overwrite

P | N

P | N

P | N

P | N

N (F?)