CephFS with Ceph-CSI

The CephFS driver (cephfs.csi.ceph.com) provisions shared POSIX file systems backed by CephFS. Volumes support ReadWriteMany access, so multiple pods on multiple nodes can mount the same volume simultaneously. Each PersistentVolume (PV) is a CephFS subvolume (see FS Volumes and Subvolumes).

Prepare a File System

Create a CephFS volume, or reuse an existing one:

ceph fs volume create mycephfs

The driver stores provisioned volumes as subvolumes in a subvolume group. Create the group that your StorageClass will use. The Ceph-CSI default group name is csi:

ceph fs subvolumegroup create mycephfs csi

Create a CephX User

Create a user for the driver, restricted to the file system and the subvolume group created above:

ceph auth get-or-create client.csi-cephfs \
    mgr 'allow rw' \
    mon 'allow r fsname=mycephfs' \
    mds 'allow r fsname=mycephfs path=/volumes, allow rws fsname=mycephfs path=/volumes/csi' \
    osd 'allow rwx tag cephfs metadata=mycephfs, allow rw tag cephfs data=mycephfs'

Record the generated key. It is stored in a Kubernetes Secret that the StorageClass references. See CephFS Client Capabilities for background on CephFS authentication and the up-to-date capability requirements in the Ceph-CSI capabilities documentation.

Define a StorageClass

A minimal StorageClass for the CephFS driver looks like this:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-cephfs
provisioner: cephfs.csi.ceph.com
parameters:
  clusterID: <cluster id>
  fsName: mycephfs
reclaimPolicy: Delete
allowVolumeExpansion: true

Complete examples, including the secret references that the provisioner and node plugin require, are maintained in the Ceph-CSI CephFS examples. See Deploying Ceph-CSI for how the clusterID value is determined.

Features

The CephFS driver supports volume expansion, snapshots and clones through the standard Kubernetes APIs, and both kernel and FUSE mounters on the nodes. See the Ceph-CSI documentation for the full feature matrix and configuration details.

Brought to you by the Ceph Foundation

The Ceph Documentation is a community resource funded and hosted by the non-profit Ceph Foundation. If you would like to support this and our other efforts, please consider joining now.