Filestore Config Reference

Note

Since the Luminous release of Ceph, Filestore has not been Ceph’s default storage back end. Since the Luminous release of Ceph, BlueStore has been Ceph’s default storage back end. However, Filestore OSDs are still supported. See OSD Back Ends. See BlueStore Migration for instructions explaining how to replace an existing Filestore back end with a BlueStore back end.

filestore debug omap check

Description

Debugging check on synchronization. Expensive. For debugging only.

Type

Boolean

Required

No

Default

false

Extended Attributes

Extended Attributes (XATTRs) are important for Filestore OSDs. However, Certain disadvantages can occur when the underlying file system is used for the storage of XATTRs: some file systems have limits on the number of bytes that can be stored in XATTRs, and your file system might in some cases therefore run slower than would an alternative method of storing XATTRs. For this reason, a method of storing XATTRs extrinsic to the underlying file system might improve performance. To implement such an extrinsic method, refer to the following settings.

If the underlying file system has no size limit, then Ceph XATTRs are stored as inline xattr, using the XATTRs provided by the file system. But if there is a size limit (for example, ext4 imposes a limit of 4 KB total), then some Ceph XATTRs will be stored in a key/value database when the limit is reached. More precisely, this begins to occur when either the filestore_max_inline_xattr_size or filestore_max_inline_xattrs threshold is reached.

filestore_max_inline_xattr_size

Description

Defines the maximum size per object of an XATTR that can be stored in the file system (for example, XFS, Btrfs, ext4). The specified size should not be larger than the file system can handle. Using the default value of 0 instructs Filestore to use the value specific to the file system.

Type

Unsigned 32-bit Integer

Required

No

Default

0

filestore_max_inline_xattr_size_xfs

Description

Defines the maximum size of an XATTR that can be stored in the XFS file system. This setting is used only if filestore_max_inline_xattr_size == 0.

Type

Unsigned 32-bit Integer

Required

No

Default

65536

filestore_max_inline_xattr_size_btrfs

Description

Defines the maximum size of an XATTR that can be stored in the Btrfs file system. This setting is used only if filestore_max_inline_xattr_size == 0.

Type

Unsigned 32-bit Integer

Required

No

Default

2048

filestore_max_inline_xattr_size_other

Description

Defines the maximum size of an XATTR that can be stored in other file systems. This setting is used only if filestore_max_inline_xattr_size == 0.

Type

Unsigned 32-bit Integer

Required

No

Default

512

filestore_max_inline_xattrs

Description

Defines the maximum number of XATTRs per object that can be stored in the file system. Using the default value of 0 instructs Filestore to use the value specific to the file system.

Type

32-bit Integer

Required

No

Default

0

filestore_max_inline_xattrs_xfs

Description

Defines the maximum number of XATTRs per object that can be stored in the XFS file system. This setting is used only if filestore_max_inline_xattrs == 0.

Type

32-bit Integer

Required

No

Default

10

filestore_max_inline_xattrs_btrfs

Description

Defines the maximum number of XATTRs per object that can be stored in the Btrfs file system. This setting is used only if filestore_max_inline_xattrs == 0.

Type

32-bit Integer

Required

No

Default

10

filestore_max_inline_xattrs_other

Description

Defines the maximum number of XATTRs per object that can be stored in other file systems. This setting is used only if filestore_max_inline_xattrs == 0.

Type

32-bit Integer

Required

No

Default

2

Synchronization Intervals

Filestore must periodically quiesce writes and synchronize the file system. Each synchronization creates a consistent commit point. When the commit point is created, Filestore is able to free all journal entries up to that point. More-frequent synchronization tends to reduce both synchronization time and the amount of data that needs to remain in the journal. Less-frequent synchronization allows the backing file system to coalesce small writes and metadata updates, potentially increasing synchronization efficiency but also potentially increasing tail latency.

filestore_max_sync_interval

Description

Defines the maximum interval (in seconds) for synchronizing Filestore.

Type

Double

Required

No

Default

5

filestore_min_sync_interval

Description

Defines the minimum interval (in seconds) for synchronizing Filestore.

Type

Double

Required

No

Default

.01

Flusher

The Filestore flusher forces data from large writes to be written out using sync_file_range prior to the synchronization. Ideally, this action reduces the cost of the eventual synchronization. In practice, however, disabling ‘filestore_flusher’ seems in some cases to improve performance.

filestore_flusher

Description

Enables the Filestore flusher.

Type

Boolean

Required

No

Default

false

Deprecated since version v.65.

filestore_flusher_max_fds

Description

Defines the maximum number of file descriptors for the flusher.

Type

Integer

Required

No

Default

512

Deprecated since version v.65.

filestore_sync_flush

Description

Enables the synchronization flusher.

Type

Boolean

Required

No

Default

false

Deprecated since version v.65.

filestore_fsync_flushes_journal_data

Description

Flushes journal data during file-system synchronization.

Type

Boolean

Required

No

Default

false

Queue

The following settings define limits on the size of the Filestore queue:

filestore_queue_max_ops

Description

Defines the maximum number of in-progress operations that Filestore accepts before it blocks the queueing of any new operations.

Type

Integer

Required

No. Minimal impact on performance.

Default

50

filestore_queue_max_bytes

Description

Defines the maximum number of bytes permitted per operation.

Type

Integer

Required

No

Default

100 << 20

Timeouts

filestore_op_threads

Description

Defines the number of file-system operation threads that execute in parallel.

Type

Integer

Required

No

Default

2

filestore_op_thread_timeout

Description

Defines the timeout (in seconds) for a file-system operation thread.

Type

Integer

Required

No

Default

60

filestore_op_thread_suicide_timeout

Description

Defines the timeout (in seconds) for a commit operation before the commit is cancelled.

Type

Integer

Required

No

Default

180

B-Tree Filesystem

filestore_btrfs_snap

Description

Enables snapshots for a btrfs Filestore.

Type

Boolean

Required

No. Used only for btrfs.

Default

true

filestore_btrfs_clone_range

Description

Enables cloning ranges for a btrfs Filestore.

Type

Boolean

Required

No. Used only for btrfs.

Default

true

Journal

filestore_journal_parallel

Description

Enables parallel journaling, default for btrfs.

Type

Boolean

Required

No

Default

false

filestore_journal_writeahead

Description

Enables write-ahead journaling, default for XFS.

Type

Boolean

Required

No

Default

false

filestore_journal_trailing

Description

Deprecated. Never use.

Type

Boolean

Required

No

Default

false

Misc

filestore_merge_threshold

Description

Defines the minimum number of files permitted in a subdirectory before the subdirectory is merged into its parent directory. NOTE: A negative value means that subdirectory merging is disabled.

Type

Integer

Required

No

Default

-10

filestore_split_multiple

Description

(filestore_split_multiple * abs(filestore_merge_threshold) + (rand() % filestore_split_rand_factor)) * 16 is the maximum number of files permitted in a subdirectory before the subdirectory is split into child directories.

Type

Integer

Required

No

Default

2

filestore_split_rand_factor

Description

A random factor added to the split threshold to avoid too many (expensive) Filestore splits occurring at the same time. For details, see filestore_split_multiple. To change this setting for an existing OSD, it is necessary to take the OSD offline before running the ceph-objectstore-tool apply-layout-settings command.

Type

Unsigned 32-bit Integer

Required

No

Default

20

filestore_update_to

Description

Limits automatic upgrades to a specified version of Filestore. Useful in cases in which you want to avoid upgrading to a specific version.

Type

Integer

Required

No

Default

1000

filestore_blackhole

Description

Drops any new transactions on the floor, similar to redirecting to NULL.

Type

Boolean

Required

No

Default

false

filestore_dump_file

Description

Defines the file that transaction dumps are stored on.

Type

Boolean

Required

No

Default

false

filestore_kill_at

Description

Injects a failure at the nth opportunity.

Type

String

Required

No

Default

false

filestore_fail_eio

Description

Fail/Crash on EIO.

Type

Boolean

Required

No

Default

true