This document is for a development version of Ceph.
Deploying additional monitors¶
A typical Ceph cluster has three or five monitor daemons that are spread across different hosts. We recommend deploying five monitors if there are five or more nodes in your cluster.
Ceph deploys monitor daemons automatically as the cluster grows and Ceph scales back monitor daemons automatically as the cluster shrinks. The smooth execution of this automatic growing and shrinking depends upon proper subnet configuration.
The cephadm bootstrap procedure assigns the first monitor daemon in the
cluster to a particular subnet.
cephadm designates that subnet as the
default subnet of the cluster. New monitor daemons will be assigned by
default to that subnet unless cephadm is instructed to do otherwise.
If all of the ceph monitor daemons in your cluster are in the same subnet,
manual administration of the ceph monitor daemons is not necessary.
cephadm will automatically add up to five monitors to the subnet, as
needed, as new hosts are added to the cluster.
Designating a Particular Subnet for Monitors¶
To designate a particular IP subnet for use by ceph monitor daemons, use a
command of the following form, including the subnet’s address in CIDR
ceph config set mon public_network *<mon-cidr-network>*
For example:ceph config set mon public_network 10.1.2.0/24
Cephadm deploys new monitor daemons only on hosts that have IP addresses in the designated subnet.
Changing the number of monitors from the default¶
If you want to adjust the default of 5 monitors, run this command:
ceph orch apply mon *<number-of-monitors>*
Deploying monitors only to specific hosts¶
To deploy monitors on a specific set of hosts, run this command:
ceph orch apply mon *<host1,host2,host3,...>*
Be sure to include the first (bootstrap) host in this list.
Using Host Labels¶
You can control which hosts the monitors run on by making use of host labels.
To set the
mon label to the appropriate hosts, run this command:
ceph orch host label add *<hostname>* mon
To view the current hosts and labels, run this command:ceph orch host ls
For example:ceph orch host label add host1 mon ceph orch host label add host2 mon ceph orch host label add host3 mon ceph orch host lsHOST ADDR LABELS STATUS host1 mon host2 mon host3 mon host4 host5
Tell cephadm to deploy monitors based on the label by running this command:ceph orch apply mon label:mon
See also host labels.
Deploying Monitors on a Particular Network¶
You can explicitly specify the IP address or CIDR network for each monitor and control where each monitor is placed. To disable automated monitor deployment, run this command:
ceph orch apply mon --unmanaged
To deploy each additional monitor:ceph orch daemon add mon *<host1:ip-or-network1> [<host1:ip-or-network-2>...]*
For example, to deploy a second monitor on
newhost1using an IP address
10.1.2.123and a third monitor on
10.1.2.0/24, run the following commands:ceph orch apply mon --unmanaged ceph orch daemon add mon newhost1:10.1.2.123 ceph orch daemon add mon newhost2:10.1.2.0/24
The apply command can be confusing. For this reason, we recommend using YAML specifications.
ceph orch apply moncommand supersedes the one before it. This means that you must use the proper comma-separated list-based syntax when you want to apply monitors to more than one host. If you do not use the proper syntax, you will clobber your work as you go.
For example:ceph orch apply mon host1 ceph orch apply mon host2 ceph orch apply mon host3
This results in only one host having a monitor applied to it: host 3.
(The first command creates a monitor on host1. Then the second command clobbers the monitor on host1 and creates a monitor on host2. Then the third command clobbers the monitor on host2 and creates a monitor on host3. In this scenario, at this point, there is a monitor ONLY on host3.)
To make certain that a monitor is applied to each of these three hosts, run a command like this:ceph orch apply mon "host1,host2,host3"
There is another way to apply monitors to multiple hosts: a
yamlfile can be used. Instead of using the “ceph orch apply mon” commands, run a command of this form:ceph orch apply -i file.yaml
Here is a sample file.yaml file:service_type: mon placement: hosts: - host1 - host2 - host3