The Parts of a Ceph Cluster

A Ceph cluster is a set of daemons (background programs) that run on one or more hosts. Three kinds are always present: Monitors, Managers, and OSDs. Two more are added for specific storage interfaces: Metadata Servers for CephFS, and Object Gateways for object storage.

Daemon	Process	What it does	How many
Monitor	`ceph-mon`	Holds the maps of the cluster state (monitor map, Manager map, OSD map, MDS map, CRUSH map) that daemons use to coordinate, and handles authentication between daemons and clients.	Three for redundancy; a majority must be up or the cluster stops. One is enough for a test cluster.
Manager	`ceph-mgr`	Tracks runtime metrics and cluster state (storage utilization, performance, load) and hosts Python modules for orchestration (deploying daemons, through `ceph orch`), the Dashboard, data balancing, and access for non-native clients such as NFS.	At least two for high availability, ideally one per Monitor; with one, a Manager restart pauses the Dashboard and modules. One is enough for a test cluster.
OSD	`ceph-osd`	Manages one unit of underlying storage, usually an entire HDD or SSD. Stores data as objects, replicates, recovers, and rebalances it, and reports OSDs that stop answering heartbeat checks.	At least as many as the number of object replicas or shards; below that, pools cannot become healthy. Three for redundancy in production.
Metadata Server	`ceph-mds`	Stores the metadata of the Ceph File System (directory listings, file attributes). Needed only if you use CephFS. See Deploy CephFS and Ceph File System.	At least one active per file system, plus a standby.
Object Gateway	`ceph-radosgw`	An HTTP gateway that provides S3-compatible and Swift-compatible APIs. Needed only if you use object storage.	One, or more behind a load balancer.

How Data Is Placed

Ceph stores data as objects in logical pools. Ceph hashes each object name to a placement group (PG), and the CRUSH algorithm calculates which OSDs store that PG. Because every daemon and client can run the same calculation, nothing keeps a central lookup table, and the cluster can scale, rebalance, and recover on its own.

Additional Resources

Brought to you by the Ceph Foundation

The Ceph Documentation is a community resource funded and hosted by the non-profit Ceph Foundation. If you would like to support this and our other efforts, please consider joining now.