.. _mgr-ceph-secrets: =================== Ceph Secrets Module =================== The ``ceph_secrets`` manager module provides centralised secret storage for Ceph operators and Ceph manager modules. Instead of embedding plaintext credentials in service specifications or configuration objects, secrets are stored once and referenced by URI. Ceph manager modules such as cephadm and rook can store secret payloads centrally, reference them by URI, and resolve them to plaintext only when needed at deploy time. For example, a cephadm-managed service may need an API token. Instead of embedding that token directly in the service specification, an operator can store it once: .. prompt:: bash $ ceph secret set cephadm/service/my-service/api_token -i /tmp/api-token and then use the URI ``secret:/cephadm/service/my-service/api_token`` in the service spec instead of the plaintext token. A cephadm integration can then resolve the URI at deploy time and write the token only into the daemon files that need it. Secrets are stored in the Mon KV store under the ``secret_store/v1/`` prefix and are organised by namespace, scope, and name. Each secret is versioned and carries ``created``/``updated`` timestamps. A per-namespace epoch counter is incremented on every ``set`` and on any ``rm`` that actually removes a secret, allowing consumers to detect changes without fetching the full secret list. .. note:: The ``mon`` backend stores secrets in the Mon KV store, which is not an external KMS or vault. Users and MGR modules with sufficient Ceph permissions can still reveal or resolve stored secret values. Namespaces provide logical and storage isolation, not an authorisation boundary. If the module is not already enabled, run:: ceph mgr module enable ceph_secrets .. contents:: :local: :depth: 2 Concepts ======== Namespace --------- A namespace is a logical storage boundary, typically matching the name of the consuming MGR module (e.g. ``cephadm``, ``rook``). Secrets in different namespaces are stored independently and have independent epoch counters. Namespaces are not an authorisation boundary; any caller with access to the ``ceph_secrets`` module can read secrets from any namespace. Scope ----- Within a namespace, every secret has a scope that encodes what the secret belongs to: .. list-table:: :header-rows: 1 :widths: 15 40 45 * - Scope - Meaning - CLI path form * - ``global`` - Cluster-wide secret, not tied to any specific target - ``/global/`` * - ``service`` - Secret for a specific named service - ``/service//`` * - ``host`` - Secret for a specific host - ``/host//`` * - ``custom`` - Arbitrary slash-delimited path under the namespace - ``/custom/`` Path grammar ------------ All path segments (namespace, scope target, name, and each segment of a custom path) must match ``[A-Za-z0-9._-]+`` and must not end with ``'.'``. Empty segments, percent-encoding, and leading/trailing whitespace are rejected. Custom scope paths may contain multiple slash-separated segments (e.g. ``cephadm/custom/app/db/password``). Two custom paths that share a common prefix are independent secrets; ``cephadm/custom/a/b`` and ``cephadm/custom/a/b/c`` do not conflict. Secret URIs ----------- Internally, secrets are identified by URIs of the form:: secret://global/ secret://service// secret://host// secret://custom/ These URIs appear in the output of :ref:`scan_refs ` and may be embedded in configuration objects as opaque references to be resolved at deploy time. CLI Reference ============= All CLI commands use the path form described above. Responses are JSON by default; pass ``--format yaml`` for YAML output. .. _mgr-ceph-secrets-set: secret set ---------- Create or update a secret. The input file content is stored as an opaque string:: ceph secret set -i Secret data must not be empty. Other contents, including leading or trailing whitespace and final newlines, are preserved exactly. Callers that need to store structured data should encode it themselves, for example as JSON, and decode it after retrieval or resolution. If the secret already exists its data is replaced and its version incremented, unless the existing secret policy marks it non-editable (set via the Python API with ``editable=False``), in which case the update is rejected. The ``created`` timestamp is set on the first write and is never changed thereafter; ``updated`` is refreshed on every write. Example: .. prompt:: bash $ ceph secret set cephadm/service/my-service/api_token -i /tmp/api-token {"metadata": {"version": 1, "created": "2025-06-01T12:00:00Z", "updated": "2025-06-01T12:00:00Z"}} .. _mgr-ceph-secrets-get: secret get ---------- Retrieve the metadata (and, optionally, the data) for a secret:: ceph secret get [--reveal] [--format {json|yaml}] Without ``--reveal``, the ``data`` field is omitted from the response to avoid accidental exposure in terminal output or logs. With ``--reveal``, ``data`` contains the stored opaque string. Example: .. prompt:: bash $ ceph secret get cephadm/service/my-service/api_token { "metadata": { "version": 1, "created": "2025-06-01T12:00:00Z", "updated": "2025-06-01T12:00:00Z" } } ceph secret get cephadm/service/my-service/api_token --reveal { "metadata": { "version": 1, "created": "2025-06-01T12:00:00Z", "updated": "2025-06-01T12:00:00Z" }, "data": "api-token-value" } .. _mgr-ceph-secrets-get-value: secret get-value ---------------- Return the raw secret data string directly, with no JSON envelope:: ceph secret get-value Unlike ``secret get --reveal``, the output is the stored string itself, making it suitable for use in shell scripts and pipelines. Returns an error if the secret does not exist. Example: .. prompt:: bash $ ceph secret get-value cephadm/service/my-service/api_token api-token-value .. _mgr-ceph-secrets-ls: secret ls --------- List secrets, optionally filtered by namespace, scope, and/or target:: ceph secret ls [--namespace ] [--scope ] [--sec_target ] [--reveal] [--show_internals] [--format {json|yaml}] Without filters, all secrets across all namespaces are listed. ``--reveal`` includes the stored opaque data string in each output record. ``--show_internals`` additionally includes the ``policy`` object (``user_made`` and ``editable`` flags) in each record. Example: .. prompt:: bash $ ceph secret ls --namespace cephadm --scope host --sec_target node1 { "cephadm/host/node1/ssh_key": { "metadata": { "version": 3, "created": "2025-05-10T08:00:00Z", "updated": "2025-06-01T09:00:00Z" }, "ref": { "namespace": "cephadm", "scope": "host", "target": "node1", "name": "ssh_key" } } } .. _mgr-ceph-secrets-rm: secret rm --------- Remove a secret:: ceph secret rm The operation is idempotent: removing a secret that does not exist succeeds and reports ``"status": "not_found"`` rather than an error. Example: .. prompt:: bash $ ceph secret rm cephadm/service/my-service/api_token {"status": "removed"} ceph secret rm cephadm/service/my-service/api_token {"status": "not_found"} .. _mgr-ceph-secrets-get-epoch: Epoch ----- The epoch for a namespace is accessible via the Python API only (see :ref:`Epoch-based change detection `). There is no CLI command for it. Module API ========== Other MGR modules should consume ``ceph_secrets`` via ``CephSecretsClient`` (``src/pybind/mgr/ceph_secrets_client.py``) rather than calling ``mgr.remote()`` directly. The client file, along with ``ceph_secrets_types.py``, lives at the top level of ``src/pybind/mgr/`` so any MGR module can import it without depending on the ``ceph_secrets`` package internals. .. code-block:: python from ceph_secrets_client import CephSecretsClient from ceph_secrets_types import SecretScope client = CephSecretsClient(self) # self is a MgrModule instance # Store a secret as an opaque string client.secret_set( namespace="cephadm", scope=SecretScope.HOST, target="node1", name="ssh_key", data="AQB...==", ) # Retrieve metadata (no data unless reveal=True) rec = client.secret_get("cephadm", SecretScope.HOST, "node1", "ssh_key") if rec: version = rec["metadata"]["version"] # Remove client.secret_rm("cephadm", SecretScope.HOST, "node1", "ssh_key") CephSecretsClient methods -------------------------- .. list-table:: :header-rows: 1 :widths: 35 65 * - Method - Description * - ``secret_set(namespace, scope, target, name, data, ...)`` - Create or update a secret. ``data`` must be a non-empty opaque string. Increments the version and refreshes ``updated`` on each call; ``created`` is set only on the first write. Optional ``user_made`` and ``editable`` flags control the stored policy; ``editable=False`` blocks future updates but does not prevent removal. * - ``secret_get(namespace, scope, target, name, reveal=False)`` - Return the secret's metadata dict, plus the opaque ``data`` string if *reveal* is True. Returns ``None`` if the secret does not exist. * - ``secret_get_value(namespace, scope, target, name)`` - Return the stored opaque string directly, or ``None`` if the secret does not exist. Use this when only the value is needed and the metadata envelope is not required. * - ``secret_get_version(namespace, scope, target, name)`` - Return the current version integer, or ``None`` if not found. A cheaper alternative to ``secret_get`` when only change-detection is needed. * - ``secret_get_versions(uris)`` - Batch-fetch version numbers for a list of canonical ``secret:/...`` URIs. Returns a dict keyed by URI; missing secrets map to ``None``. Malformed URIs are skipped entirely, so a missing key indicates malformed input rather than an absent secret. * - ``secret_rm(namespace, scope, target, name)`` - Remove a secret. Idempotent: returns ``False`` if not found. * - ``secret_get_epoch(namespace)`` - Return the current epoch for the namespace. Use as a cheap change-detector before a full refresh. * - ``scan_refs(obj, namespace)`` - Walk a JSON-like object and return secret-like reference strings found within it. This includes valid whole-value secret URIs and malformed or embedded secret-like references that should be reported to the caller. Only canonical ``secret:/...`` URIs should be passed to ``secret_get_versions``. * - ``scan_unresolved_refs(obj, namespace)`` - Like ``scan_refs`` but returns only references that are missing, malformed, embedded inside a larger string, or cannot currently be read successfully. Useful for pre-flight validation. * - ``resolve_object(obj)`` - Walk a JSON-like object and replace every whole-value ``secret:/...`` URI string with the stored opaque string for the referenced secret. .. _mgr-ceph-secrets-scan: Secret URI embedding and resolution ----------------------------------- Configuration objects that need to reference secrets without embedding plaintext can store ``secret:/...`` URIs as string values. The module resolves them on demand: .. code-block:: python spec = { "service_type": "my-service", "spec": { "api_token": " secret:/cephadm/service/my-service/api_token ", }, } # Check for missing, malformed, embedded, or unresolvable secret references unresolved = client.scan_unresolved_refs(spec, namespace="cephadm") if unresolved: raise RuntimeError(f"Unresolved secret references: {unresolved}") # Replace URI references with plaintext values resolved = client.resolve_object(spec) # resolved["spec"]["api_token"] now holds the stored API token .. note:: Resolution replaces the entire string value of a field. Surrounding whitespace around a URI reference is ignored, so values such as ``" secret:/cephadm/global/foo "`` are accepted. Embedding a secret URI inside a larger string (for example ``"Bearer secret:/..."``) is not supported; after trimming surrounding whitespace, the field value must be the URI. The resolved value is the stored opaque string. The module does not parse stored data as JSON and does not support field-level URI selection. If a caller stores structured data, it must encode and decode that structure itself. .. _mgr-ceph-secrets-epoch: Epoch-based change detection ---------------------------- Consumers that maintain a local cache of secrets can use the epoch to avoid unnecessary full refreshes: .. code-block:: python def maybe_refresh(client, namespace, last_epoch): current_epoch = client.secret_get_epoch(namespace) if current_epoch == last_epoch: return last_epoch # nothing changed # ... do a full refresh ... return current_epoch last_epoch = 0 last_epoch = maybe_refresh(client, "cephadm", last_epoch) The epoch is incremented by every successful ``set`` operation and by ``rm`` only when an existing secret is actually removed (an idempotent ``rm`` returning ``"not_found"`` does not bump the epoch). It starts at 0 for a namespace that has never been written to, and mutations in one namespace never affect another namespace's epoch. Configuration ============= .. mgr_module:: ceph_secrets .. confval:: secrets_backend :type: str :default: ``mon`` The storage backend used for secrets. Currently only the Mon KV store (``mon``) is supported. This option is reserved for future backends (e.g. HashiCorp Vault).