Notice

This document is for a development version of Ceph.

Librados (Python)

The rados module is a thin Python wrapper for librados.

Installation

To install Python libraries for Ceph, see Getting librados for Python.

Getting Started

You can create your own Ceph client using Python. The following tutorial will show you how to import the Ceph Python module, connect to a Ceph cluster, and perform object operations as a client.admin user.

Note

To use the Ceph Python bindings, you must have access to a running Ceph cluster. To set one up quickly, see Getting Started.

First, create a Python source file for your Ceph client.

vim client.py

Import the Module

To use the rados module, import it into your source file.

     import rados

Configure a Cluster Handle

Before connecting to the Ceph Storage Cluster, create a cluster handle. By default, the cluster handle assumes a cluster named ceph (i.e., the default for deployment tools, and our Getting Started guides too), and a client.admin user name. You may change these defaults to suit your needs.

To connect to the Ceph Storage Cluster, your application needs to know where to find the Ceph Monitor. Provide this information to your application by specifying the path to your Ceph configuration file, which contains the location of the initial Ceph monitors.

     import rados, sys

     #Create Handle Examples.
     cluster = rados.Rados(conffile='ceph.conf')
     cluster = rados.Rados(conffile=sys.argv[1])
     cluster = rados.Rados(conffile = 'ceph.conf', conf = dict (keyring = '/path/to/keyring'))

Ensure that the conffile argument provides the path and file name of your Ceph configuration file. You may use the sys module to avoid hard-coding the Ceph configuration path and file name.

Your Python client also requires a client keyring. For this example, we use the client.admin key by default. If you would like to specify the keyring when creating the cluster handle, you may use the conf argument. Alternatively, you may specify the keyring path in your Ceph configuration file. For example, you may add something like the following line to your Ceph configuration file:

keyring = /path/to/ceph.client.admin.keyring

For additional details on modifying your configuration via Python, see Cluster Handle API.

Connect to the Cluster

Once you have a cluster handle configured, you may connect to the cluster. With a connection to the cluster, you may execute methods that return information about the cluster.

     import rados, sys

     cluster = rados.Rados(conffile='ceph.conf')
     print("\nlibrados version: {}".format(str(cluster.version())))
     print("Will attempt to connect to: {}".format(str(cluster.conf_get('mon host'))))

     cluster.connect()
     print("\nCluster ID: {}".format(cluster.get_fsid()))

     print("\n\nCluster Statistics")
     print("==================")
     cluster_stats = cluster.get_cluster_stats()

     for key, value in cluster_stats.items():
             print(key, value)

By default, Ceph authentication is on. Your application will need to know the location of the keyring. The python-ceph module doesn’t have the default location, so you need to specify the keyring path. The easiest way to specify the keyring is to add it to the Ceph configuration file. The following Ceph configuration file example uses the client.admin keyring.

     [global]
     # ... elided configuration
     keyring = /path/to/keyring/ceph.client.admin.keyring

Manage Pools

When connected to the cluster, the Rados API allows you to manage pools. You can list pools, check for the existence of a pool, create a pool and delete a pool.

     print("\n\nPool Operations")
     print("===============")

     print("\nAvailable Pools")
     print("----------------")
     pools = cluster.list_pools()

     for pool in pools:
             print(pool)

     print("\nCreate 'test' Pool")
     print("------------------")
     cluster.create_pool('test')

     print("\nPool named 'test' exists: {}".format(str(cluster.pool_exists('test'))))
     print("\nVerify 'test' Pool Exists")
     print("-------------------------")
     pools = cluster.list_pools()

     for pool in pools:
             print(pool)

     print("\nDelete 'test' Pool")
     print("------------------")
     cluster.delete_pool('test')
     print("\nPool named 'test' exists: {}".format(str(cluster.pool_exists('test'))))

Input/Output Context

Reading from and writing to the Ceph Storage Cluster requires an input/output context (ioctx). You can create an ioctx with the open_ioctx() or open_ioctx2() method of the Rados class. The ioctx_name parameter is the name of the pool and pool_id is the ID of the pool you wish to use.

     ioctx = cluster.open_ioctx('data')

or

     ioctx = cluster.open_ioctx2(pool_id)

Once you have an I/O context, you can read/write objects, extended attributes, and perform a number of other operations. After you complete operations, ensure that you close the connection. For example:

     print("\nClosing the connection.")
     ioctx.close()

Writing, Reading and Removing Objects

Once you create an I/O context, you can write objects to the cluster. If you write to an object that doesn’t exist, Ceph creates it. If you write to an object that exists, Ceph overwrites it (except when you specify a range, and then it only overwrites the range). You may read objects (and object ranges) from the cluster. You may also remove objects from the cluster. For example:

print("\nWriting object 'hw' with contents 'Hello World!' to pool 'data'.")
ioctx.write_full("hw", "Hello World!")

print("\n\nContents of object 'hw'\n------------------------\n")
print(ioctx.read("hw"))

print("\nRemoving object 'hw'")
ioctx.remove_object("hw")

Writing and Reading XATTRs

Once you create an object, you can write extended attributes (XATTRs) to the object and read XATTRs from the object. For example:

print("\n\nWriting XATTR 'lang' with value 'en_US' to object 'hw'")
ioctx.set_xattr("hw", "lang", "en_US")

print("\n\nGetting XATTR 'lang' from object 'hw'\n")
print(ioctx.get_xattr("hw", "lang"))

Listing Objects

If you want to examine the list of objects in a pool, you may retrieve the list of objects and iterate over them with the object iterator. For example:

object_iterator = ioctx.list_objects()

while True :

        try :
                rados_object = object_iterator.__next__()
                print("Object contents = {}".format(rados_object.read()))

        except StopIteration :
                break

# Or alternatively
[print("Object contents = {}".format(obj.read())) for obj in ioctx.list_objects()]

The Object class provides a file-like interface to an object, allowing you to read and write content and extended attributes. Object operations using the I/O context provide additional functionality and asynchronous capabilities.

Cluster Handle API

The Rados class provides an interface into the Ceph Storage Cluster.

You do not need to be connected to the Ceph Storage Cluster to get and set configuration values, read the Ceph configuration file, or parse arguments. See Storage Cluster Configuration for details on settings. Pool operations, cluster statistics, and the mon_command(), osd_command(), mgr_command(), and pg_command() methods require a connection. The Ceph CLI sends its commands through these command methods.

class rados.Rados(*args, **kwargs)

This class wraps librados functions

require_state(*args)

Checks if the Rados object is in a special state

Parameters:: args -- Any number of states to check as separate arguments
Raises:: RadosStateError

blocklist_add(client_address, expire_seconds=0)

Blocklist a client from the OSDs

Parameters:

client_address (str) -- client address
expire_seconds (int) -- number of seconds to blocklist

Raises:

Error

conf_defaults: conf_defaults: object

conf_get(option)

Get the value of a configuration option

Parameters:: option (str) -- which option to read
Return type:: Optional[str]
Returns:: value of the option or None
Raises:: TypeError

conf_parse_argv(args): Parse known arguments from args, and remove; returned args contain only those unknown to ceph

conf_parse_env(var='CEPH_ARGS'): Parse known arguments from an environment variable, normally CEPH_ARGS.

conf_read_file(path=None)

Configure the cluster handle using a Ceph config file.

Parameters:: path (Optional[str]) -- path to the config file

conf_set(option, val)

Set the value of a configuration option

Parameters:

option (str) -- which option to set
option -- value of the option

Raises:

TypeError, ObjectNotFound

conffile: conffile: object

connect(timeout=0)

Connect to the cluster. Use shutdown() to release resources.

Parameters:: timeout (int) -- Any supplied timeout value is currently ignored.

create_pool(pool_name, crush_rule=None, auid=None)

Create a pool: - with default settings: if crush_rule=None and auid=None - with a specific CRUSH rule: crush_rule given - with a specific auid: auid given - with a specific CRUSH rule and auid: crush_rule and auid given

Parameters:

pool_name (str) -- name of the pool to create
crush_rule (Optional[int]) -- rule to use for placement in the new pool
auid (Optional[int]) -- id of the owner of the new pool

Raises:

TypeError, Error

delete_pool(pool_name)

Delete a pool and all data inside it.

The pool is removed from the cluster immediately, but the actual data is deleted in the background.

Parameters:: pool_name (str) -- name of the pool to delete
Raises:: TypeError, Error

get_addrs(self): Get associated client addresses with this RADOS session.

get_cluster_stats()

Read usage info about the cluster

This tells you total space, space used, space available, and number of objects. These are not updated immediately when data is written, they are eventually consistent. :rtype: Dict[str, int] :returns: contains the following keys:

kb (int) - total space

kb_used (int) - space used

kb_avail (int) - free space available

num_objects (int) - number of objects

get_fsid()

Get the fsid of the cluster as a hexadecimal string.

Raises:: Error
Return type:: str
Returns:: cluster fsid

get_inconsistent_pgs(pool_id)

List inconsistent placement groups in the given pool

Parameters:: pool_id (int) -- ID of the pool in which PGs are listed
Return type:: List[str]
Returns:: inconsistent placement groups

get_instance_id()

Get a global id for current instance

Return type:: int

get_pool_base_tier(pool_id)

Get base pool

Return type:: int
Returns:: base pool, or pool_id if tiering is not configured for the pool

list_pools()

Gets a list of pool names.

Return type:: List[str]
Returns:: list of pool names.

mgr_command(cmd, inbuf, timeout=0, target=None)

Return type:: Tuple[int, str, bytes]
Returns:: (int ret, string outbuf, string outs)

mon_command(cmd, inbuf, timeout=0, target=None)

Send a command to the mon.

mon_command[_target](cmd, inbuf, outbuf, outbuflen, outs, outslen)

Parameters:

cmd (str) -- JSON formatted string.
inbuf (bytes) -- optional string.
timeout (int) -- This parameter is ignored.
target (Union[int, str, None]) -- name or rank of a specific mon. Optional

Return type:

Tuple[int, bytes, str]

Returns:

(int ret, string outbuf, string outs)

Example:

>>> import json
>>> c = Rados(conffile='/etc/ceph/ceph.conf')
>>> c.connect()
>>> cmd = json.dumps({"prefix": "osd safe-to-destroy", "ids": ["2"], "format": "json"})
>>> c.mon_command(cmd, b'')

monitor_callback: monitor_callback: object

monitor_callback2: monitor_callback2: object

monitor_log(level, callback=None, arg=None)

monitor_log2(level, callback=None, arg=None)

open_ioctx(ioctx_name)

Create an io context

The io context allows you to perform operations within a particular pool.

Parameters:: ioctx_name (str) -- name of the pool
Raises:: TypeError, Error
Return type:: Ioctx
Returns:: Rados Ioctx object

open_ioctx2(pool_id)

Create an io context

The io context allows you to perform operations within a particular pool.

Parameters:: pool_id (int) -- ID of the pool
Raises:: TypeError, Error
Return type:: Ioctx
Returns:: Rados Ioctx object

osd_command(osdid, cmd, inbuf, timeout=0)

osd_command(osdid, cmd, inbuf, outbuf, outbuflen, outs, outslen)

Return type:: Tuple[int, bytes, str]
Returns:: (int ret, string outbuf, string outs)

parsed_args: parsed_args: object

pg_command(pgid, cmd, inbuf, timeout=0)

pg_command(pgid, cmd, inbuf, outbuf, outbuflen, outs, outslen)

Return type:: Tuple[int, bytes, str]
Returns:: (int ret, string outbuf, string outs)

ping_monitor(mon_id)

Ping a monitor to assess liveness

May be used as a simply way to assess liveness, or to obtain information about the monitor in a simple way even in the absence of quorum.

Parameters:: mon_id (str) -- the ID portion of the monitor’s name (i.e., mon.<ID>)
Returns:: the string reply from the monitor

pool_exists(pool_name)

Checks if a given pool exists.

Parameters:: pool_name (str) -- name of the pool to check
Raises:: TypeError, Error
Return type:: bool
Returns:: whether the pool exists, false otherwise.

pool_lookup(pool_name)

Returns a pool’s ID based on its name.

Parameters:: pool_name (str) -- name of the pool to look up
Raises:: TypeError, Error
Return type:: int
Returns:: pool ID, or None if it doesn’t exist

pool_reverse_lookup(pool_id)

Returns a pool’s name based on its ID.

Parameters:: pool_id (int) -- ID of the pool to look up
Raises:: TypeError, Error
Return type:: Optional[str]
Returns:: pool name, or None if it doesn’t exist

rados_id: rados_id: object

service_daemon_register(service, daemon, metadata)

Parameters:

service (str) -- service name (e.g. “rgw”)
daemon (str) -- daemon name (e.g. “gwfoo”)
metadata (dict) -- static metadata about the register daemon (e.g., the version of Ceph, the kernel version.)

service_daemon_update(status)

shutdown(self): Disconnects from the cluster. Call this explicitly when a Rados.connect()ed object is no longer used.

state: state: object

version()

Get the version number of the librados C library.

Return type:: Version
Returns:: a tuple of (major, minor, extra) components of the librados version

wait_for_latest_osdmap()

Return type:: int

Input/Output Context API

To write data to and read data from the Ceph Object Store, you must create an Input/Output context (ioctx). The Rados class provides open_ioctx() and open_ioctx2() methods. The remaining ioctx operations involve invoking methods of the Ioctx and other classes.

The Ceph Storage Cluster stores data as objects. You can read and write objects synchronously or asynchronously, and you can read and write from offsets. You may set extended attributes (XATTRs) on an object, and you can retrieve a list of objects or XATTRs and iterate over them.

class rados.Ioctx(rados, name)

rados.Ioctx object

aio_append(object_name, to_append, oncomplete=None, onsafe=None)

Asynchronously append data to an object

Queues the write and returns.

Parameters:

object_name (str) -- name of the object
to_append (bytes) -- data to append
offset -- byte offset in the object to begin writing at
oncomplete (Optional[Callable]) -- what to do when the write is safe and complete in memory on all replicas
onsafe (Optional[Callable]) -- what to do when the write is safe and complete on storage on all replicas

Raises:

Error

Return type:

Completion

Returns:

completion object

aio_cmpext(object_name, cmp_buf, offset=0, oncomplete=None)

Asynchronously compare an on-disk object range with a buffer

Parameters:

object_name (str) -- the name of the object
cmp_buf (bytes) -- buffer containing bytes to be compared with object contents
offset (int) -- object byte offset at which to start the comparison
oncomplete (Optional[Callable]) -- what to do when the write is safe and complete in memory on all replicas

Raises:

TypeError

Return type:

Completion

Returns:

0 - on success, negative error code on failure, (-MAX_ERRNO - mismatch_off) on mismatch

aio_execute(object_name, cls, method, data, length=8192, oncomplete=None, onsafe=None)

Asynchronously execute an OSD class method on an object.

oncomplete and onsafe will be called with the data returned from the plugin as well as the completion:

oncomplete(completion, data) onsafe(completion, data)

Parameters:

object_name (str) -- name of the object
cls (str) -- name of the object class
method (str) -- name of the method
data (bytes) -- input data
length (int) -- size of output buffer in bytes (default=8192)
oncomplete (Optional[Callable[[Completion, bytes], None]]) -- what to do when the execution is complete
onsafe (Optional[Callable[[Completion, bytes], None]]) -- what to do when the execution is safe and complete

Raises:

Error

Return type:

Completion

Returns:

completion object

aio_flush(self)

Block until all pending writes in an io context are safe

Raises:: Error

aio_notify(obj, oncomplete, msg='', timeout_ms=5000)

Asynchronously send a rados notification to an object

Return type:: Completion

aio_read(object_name, length, offset, oncomplete=None)

Asynchronously read data from an object

oncomplete will be called with the returned read value as well as the completion:

oncomplete(completion, data_read)

Parameters:

object_name (str) -- name of the object to read from
length (int) -- the number of bytes to read
offset (int) -- byte offset in the object to begin reading from
oncomplete (Optional[Callable]) -- what to do when the read is complete

Raises:

Error

Return type:

Completion

Returns:

completion object

aio_remove(object_name, oncomplete=None, onsafe=None)

Asynchronously remove an object

Parameters:

object_name (str) -- name of the object to remove
oncomplete (Optional[Callable]) -- what to do when the remove is safe and complete in memory on all replicas
onsafe (Optional[Callable]) -- what to do when the remove is safe and complete on storage on all replicas

Raises:

Error

Return type:

Completion

Returns:

completion object

aio_rmxattr(object_name, xattr_name, oncomplete=None)

Asynchronously delete an extended attribute from an object

Parameters:

object_name (str) -- the name of the object to remove xattr from
xattr_name (str) -- which extended attribute to remove
oncomplete (Optional[Callable]) -- what to do when the rmxattr completes

Raises:

Error

Return type:

Completion

Returns:

completion object

aio_setxattr(object_name, xattr_name, xattr_value, oncomplete=None)

Asynchronously set an extended attribute on an object

Parameters:

object_name (str) -- the name of the object to set xattr to
xattr_name (str) -- which extended attribute to set
xattr_value (bytes) -- the value of the extended attribute
oncomplete (Optional[Callable]) -- what to do when the setxttr completes

Raises:

Error

Return type:

Completion

Returns:

completion object

aio_stat(object_name, oncomplete)

Asynchronously get object stats (size/mtime)

oncomplete will be called with the returned size and mtime as well as the completion:

oncomplete(completion, size, mtime)

Parameters:

object_name (str) -- the name of the object to get stats from
oncomplete (Callable[[Completion, Optional[int], Optional[struct_time]], None]) -- what to do when the stat is complete

Raises:

Error

Return type:

Completion

Returns:

completion object

aio_write(object_name, to_write, offset=0, oncomplete=None, onsafe=None)

Write data to an object asynchronously

Queues the write and returns.

Parameters:

object_name (str) -- name of the object
to_write (bytes) -- data to write
offset (int) -- byte offset in the object to begin writing at
oncomplete (Optional[Callable[[Completion], None]]) -- what to do when the write is safe and complete in memory on all replicas
onsafe (Optional[Callable[[Completion], None]]) -- what to do when the write is safe and complete on storage on all replicas

Raises:

Error

Return type:

Completion

Returns:

completion object

aio_write_full(object_name, to_write, oncomplete=None, onsafe=None)

Asynchronously write an entire object

The object is filled with the provided data. If the object exists, it is atomically truncated and then written. Queues the write and returns.

Parameters:

object_name (str) -- name of the object
to_write (bytes) -- data to write
oncomplete (Optional[Callable]) -- what to do when the write is safe and complete in memory on all replicas
onsafe (Optional[Callable]) -- what to do when the write is safe and complete on storage on all replicas

Raises:

Error

Return type:

Completion

Returns:

completion object

aio_writesame(object_name, to_write, write_len, offset=0, oncomplete=None)

Asynchronously write the same buffer multiple times

Parameters:

object_name (str) -- name of the object
to_write (bytes) -- data to write
write_len (int) -- total number of bytes to write
offset (int) -- byte offset in the object to begin writing at
oncomplete (Optional[Callable]) -- what to do when the writesame is safe and complete in memory on all replicas

Raises:

Error

Return type:

Completion

Returns:

completion object

alignment()

Returns pool alignment

Return type:: int
Returns:: Number of alignment bytes required by the current pool, or None if alignment is not required.

append(key, data)

Append data to an object synchronously

Parameters:

key (str) -- name of the object
data (bytes) -- data to write

Raises:

TypeError

Raises:

LogicError

Returns:

int - 0 on success

application_enable(app_name, force=False)

Enable an application on an OSD pool

Parameters:

app_name (str) -- application name
force (bool) -- False if only a single app should exist per pool

Raises:

Error

application_list()

Returns a list of enabled applications

Return type:: List[str]
Returns:: list of app name string

application_metadata_get(app_name, key)

Gets application metadata on an OSD pool for the given key

Parameters:

app_name (str) -- application name
key (str) -- metadata key

Return type:

str

Returns:

str - metadata value

Raises:

Error

application_metadata_list(app_name)

Returns a list of enabled applications

Parameters:: app_name (str) -- application name
Return type:: List[Tuple[str, str]]
Returns:: list of key/value tuples

application_metadata_remove(app_name, key)

Remove application metadata from an OSD pool

Parameters:

app_name (str) -- application name
key (str) -- metadata key

Raises:

Error

application_metadata_set(app_name, key, value)

Sets application metadata on an OSD pool

Parameters:

app_name (str) -- application name
key (str) -- metadata key
value (str) -- metadata value

Raises:

Error

break_lock(key, name, client, cookie)

Release a shared or exclusive lock on an object, which was taken by the specified client.

Parameters:

key (str) -- name of the object
name (str) -- name of the lock
client (str) -- the client currently holding the lock
cookie (str) -- cookie of the lock

Raises:

TypeError

Raises:

Error

clear_omap(write_op)

Remove all key/value pairs from an object

Parameters:: write_op (WriteOp) -- write operation object

close(self)

Close a rados.Ioctx object.

This just tells librados that you no longer need to use the io context. It may not be freed immediately if there are pending asynchronous requests on it, but you should not use an io context again after calling this function on it.

cmpext(key, cmp_buf, offset=0)

Compare an on-disk object range with a buffer :type key: str :param key: the name of the object :type cmp_buf: bytes :param cmp_buf: buffer containing bytes to be compared with object contents :type offset: int :param offset: object byte offset at which to start the comparison

Raises:: TypeError
Raises:: Error
Return type:: int
Returns:: 0 - on success, negative error code on failure, (-MAX_ERRNO - mismatch_off) on mismatch

complete_completions: complete_completions: object

create_read_op()

create read operation object. need call release_read_op after use

Return type:: ReadOp

create_self_managed_snap(self)

Creates a self-managed snapshot

Returns:: snap id on success
Raises:: Error

create_snap(snap_name)

Create a pool-wide snapshot

Parameters:: snap_name (str) -- the name of the snapshot
Raises:: TypeError
Raises:: Error

create_write_op()

create write operation object. need call release_write_op after use

Return type:: WriteOp

dup(self): Duplicate IoCtx

execute(key, cls, method, data, length=8192)

Execute an OSD class method on an object.

Parameters:

key (str) -- name of the object
cls (str) -- name of the object class
method (str) -- name of the method
data (bytes) -- input data
length (int) -- size of output buffer in bytes (default=8192)

Raises:

TypeError

Raises:

Error

Return type:

Tuple[int, object]

Returns:

(ret, method output)

get_last_version()

Return the version of the last object read or written to.

This exposes the internal version number of the last object read or written via this io context

Return type:: int
Returns:: version of the last object used

get_locator_key()

Get the locator_key of context

Return type:: str
Returns:: locator_key

get_namespace()

Get the namespace of context

Return type:: str
Returns:: namespace

get_omap_keys(read_op, start_after, max_return, omap_key_type=<method 'decode' of 'bytes' objects>)

get the omap keys

Parameters:

read_op (ReadOp) -- read operation object
start_after (Union[str, bytes]) -- list keys starting after start_after
max_return (int) -- list no more than max_return key/value pairs

Return type:

Tuple[OmapIterator, int]

Returns:

an iterator over the requested omap values, return value from this action

get_omap_vals(read_op, start_after, filter_prefix, max_return, omap_key_type=<method 'decode' of 'bytes' objects>)

get the omap values

Parameters:

read_op (ReadOp) -- read operation object
start_after (Union[str, bytes]) -- list keys starting after start_after
filter_prefix (Union[str, bytes]) -- list only keys beginning with filter_prefix
max_return (int) -- list no more than max_return key/value pairs

Return type:

Tuple[OmapIterator, int]

Returns:

an iterator over the requested omap values, return value from this action

get_omap_vals_by_keys(read_op, keys, omap_key_type=<method 'decode' of 'bytes' objects>)

get the omap values by keys

Parameters:

read_op (ReadOp) -- read operation object
keys (Sequence[Union[str, bytes]]) -- input key tuple

Return type:

Tuple[OmapIterator, int]

Returns:

an iterator over the requested omap values, return value from this action

get_pool_id()

Get pool id

Return type:: int
Returns:: int - pool id

get_pool_name()

Get pool name

Return type:: str
Returns:: pool name

get_stats()

Get pool usage statistics

Return type:

Dict[str, int]

Returns:

dict contains the following keys:

num_bytes (int) - size of pool in bytes
num_kb (int) - size of pool in kbytes
num_objects (int) - number of objects in the pool
num_object_clones (int) - number of object clones
num_object_copies (int) - number of object copies
num_objects_missing_on_primary (int) - number of objects
missing on primary
num_objects_unfound (int) - number of unfound objects
num_objects_degraded (int) - number of degraded objects
num_rd (int) - bytes read
num_rd_kb (int) - kbytes read
num_wr (int) - bytes written
num_wr_kb (int) - kbytes written

get_xattr(key, xattr_name)

Get the value of an extended attribute on an object.

Parameters:

key (str) -- the name of the object to get xattr from
xattr_name (str) -- which extended attribute to read

Raises:

TypeError

Raises:

Error

Return type:

bytes

Returns:

value of the xattr

get_xattrs(oid)

Start iterating over xattrs on an object.

Parameters:: oid (str) -- the name of the object to get xattrs from
Raises:: TypeError
Raises:: Error
Return type:: XattrIterator
Returns:: XattrIterator

list_lockers(key, name)

List clients that have locked the named object lock and information about the lock.

Parameters:

key (str) -- name of the object
name (str) -- name of the lock

Returns:

dict - contains the following keys:

tag - the tag associated with the lock (every additional locker must use the same tag)
exclusive - boolean indicating whether the lock is exclusive or shared
lockers - a list of (client, cookie, address) tuples

Raises:

TypeError

Raises:

Error

list_objects()

Get ObjectIterator on rados.Ioctx object.

Return type:: ObjectIterator
Returns:: ObjectIterator

list_snaps()

Get SnapIterator on rados.Ioctx object.

Return type:: SnapIterator
Returns:: SnapIterator

locator_key: locator_key: object

lock: lock: object

lock_exclusive(key, name, cookie, desc='', duration=None, flags=0)

Take an exclusive lock on an object

Parameters:

key (str) -- name of the object
name (str) -- name of the lock
cookie (str) -- cookie of the lock
desc (str) -- description of the lock
duration (Optional[int]) -- duration of the lock in seconds
flags (int) -- flags

Raises:

TypeError

Raises:

Error

lock_shared(key, name, cookie, tag, desc='', duration=None, flags=0)

Take a shared lock on an object

Parameters:

key (str) -- name of the object
name (str) -- name of the lock
cookie (str) -- cookie of the lock
tag (str) -- tag of the lock
desc (str) -- description of the lock
duration (Optional[int]) -- duration of the lock in seconds
flags (int) -- flags

Raises:

TypeError

Raises:

Error

lookup_snap(snap_name)

Get the id of a pool snapshot

Parameters:: snap_name (str) -- the name of the snapshot to lookop
Raises:: TypeError
Raises:: Error
Return type:: Snap
Returns:: Snap - on success

name: name: object

notify(obj, msg='', timeout_ms=5000)

Send a rados notification to an object.

Parameters:

obj (str) -- the name of the object to notify
msg (str) -- optional message to send in the notification
timeout_ms (int) -- notify timeout (in ms)

Raises:

TypeError

Raises:

Error

Return type:

bool

Returns:

True on success, otherwise raise an error

nspace: nspace: object

operate_aio_read_op(read_op, oid, oncomplete=None, onsafe=None, flag=0)

execute the real read operation

Parameters:

read_op (ReadOp) -- read operation object
oid (str) -- object name
oncomplete (Optional[Callable[[Completion], None]]) -- what to do when the remove is safe and complete in memory on all replicas
onsafe (Optional[Callable[[Completion], None]]) -- what to do when the remove is safe and complete on storage on all replicas
flag (int) -- flags to apply to the entire operation

Return type:

Completion

operate_aio_write_op(write_op, oid, oncomplete=None, onsafe=None, mtime=0, flags=0)

execute the real write operation asynchronously

Parameters:

write_op (WriteOp) -- write operation object
oid (str) -- object name
oncomplete (Optional[Callable[[Completion], None]]) -- what to do when the remove is safe and complete in memory on all replicas
onsafe (Optional[Callable[[Completion], None]]) -- what to do when the remove is safe and complete on storage on all replicas
mtime (int) -- the time to set the mtime to, 0 for the current time
flags (int) -- flags to apply to the entire operation

Raises:

Error

Return type:

Completion

Returns:

completion object

operate_read_op(read_op, oid, flag=0)

execute the real read operation

Parameters:

read_op (ReadOp) -- read operation object
oid (str) -- object name
flag (int) -- flags to apply to the entire operation

operate_write_op(write_op, oid, mtime=0, flags=0)

execute the real write operation

Parameters:

write_op (WriteOp) -- write operation object
oid (str) -- object name
mtime (int) -- the time to set the mtime to, 0 for the current time
flags (int) -- flags to apply to the entire operation

read(key, length=8192, offset=0)

Read data from an object synchronously

Parameters:

key (str) -- name of the object
length (int) -- the number of bytes to read (default=8192)
offset (int) -- byte offset in the object to begin reading at

Raises:

TypeError

Raises:

Error

Return type:

bytes

Returns:

data read from object

release_read_op(read_op)

release memory alloc by create_read_op

Parameters:: read_op (ReadOp) -- read_op object

release_write_op(self, write_op): release memory alloc by create_write_op

remove_object(key)

Delete an object

This does not delete any snapshots of the object.

Parameters:: key (str) -- the name of the object to delete
Raises:: TypeError
Raises:: Error
Return type:: bool
Returns:: True on success

remove_omap_keys(write_op, keys)

remove omap keys specifiled

Parameters:

write_op (WriteOp) -- write operation object
keys (Sequence[Union[str, bytes]]) -- input key tuple

remove_omap_range2(write_op, key_begin, key_end): Remove key/value pairs from an object whose keys are in the range [key_begin, key_end) :type write_op: WriteOp :param write_op: write operation object :type key_begin: Union[str, bytes] :param key_begin: the lower bound of the key range to remove :type key_end: Union[str, bytes] :param key_end: the upper bound of the key range to remove

remove_self_managed_snap(snap_id)

Removes a self-managed snapshot

Parameters:: snap_id (int) -- the name of the snapshot
Raises:: TypeError
Raises:: Error

remove_snap(snap_name)

Removes a pool-wide snapshot

Parameters:: snap_name (str) -- the name of the snapshot
Raises:: TypeError
Raises:: Error

require_ioctx_open(self)

Checks if the rados.Ioctx object state is ‘open’

Raises:: IoctxStateError

rm_xattr(key, xattr_name)

Removes an extended attribute on from an object.

Parameters:

key (str) -- the name of the object to remove xattr from
xattr_name (str) -- which extended attribute to remove

Raises:

TypeError

Raises:

Error

Return type:

bool

Returns:

True on success, otherwise raise an error

rollback_self_managed_snap(oid, snap_id)

Rolls an specific object back to a self-managed snapshot revision

Parameters:

oid (str) -- the name of the object
snap_id (int) -- the name of the snapshot

Raises:

TypeError

Raises:

Error

safe_completions: safe_completions: object

set_locator_key(loc_key)

Set the key for mapping objects to pgs within an io context.

The key is used instead of the object name to determine which placement groups an object is put in. This affects all subsequent operations of the io context - until a different locator key is set, all objects in this io context will be placed in the same pg.

Parameters:: loc_key (str) -- the key to use as the object locator, or NULL to discard any previously set key
Raises:: TypeError

set_namespace(nspace)

Set the namespace for objects within an io context.

The namespace in addition to the object name fully identifies an object. This affects all subsequent operations of the io context - until a different namespace is set, all objects in this io context will be placed in the same namespace.

Parameters:: nspace (str) -- the namespace to use, or None/”” for the default namespace
Raises:: TypeError

set_omap(write_op, keys, values)

set keys values to write_op

Parameters:

write_op (WriteOp) -- write_operation object
keys (Sequence[Union[str, bytes]]) -- a tuple of keys
values (Sequence[bytes]) -- a tuple of values

set_osdmap_full_try(self): Set global osdmap_full_try label to true

set_read(snap_id)

Set the snapshot for reading objects.

To stop to read from snapshot, use set_read(LIBRADOS_SNAP_HEAD)

Parameters:: snap_id (int) -- the snapshot Id
Raises:: TypeError

set_self_managed_snap_write(snaps)

Updates the write context to the specified self-managed snapshot ids.

Parameters:: snaps (Sequence[Union[int, str]]) -- all associated self-managed snapshot ids
Raises:: TypeError
Raises:: Error

set_xattr(key, xattr_name, xattr_value)

Set an extended attribute on an object.

Parameters:

key (str) -- the name of the object to set xattr to
xattr_name (str) -- which extended attribute to set
xattr_value (bytes) -- the value of the extended attribute

Raises:

TypeError

Raises:

Error

Return type:

bool

Returns:

True on success, otherwise raise an error

snap_rollback(oid, snap_name)

Rollback an object to a snapshot

Parameters:

oid (str) -- the name of the object
snap_name (str) -- the name of the snapshot

Raises:

TypeError

Raises:

Error

stat(key)

Get object stats (size/mtime)

Parameters:: key (str) -- the name of the object to get stats from
Raises:: TypeError
Raises:: Error
Return type:: Tuple[int, struct_time]
Returns:: (size,timestamp)

state: state: object

trunc(key, size)

Resize an object

If this enlarges the object, the new area is logically filled with zeroes. If this shrinks the object, the excess data is removed.

Parameters:

key (str) -- the name of the object to resize
size (int) -- the new size of the object in bytes

Raises:

TypeError

Raises:

Error

Return type:

int

Returns:

0 on success, otherwise raises error

unlock(key, name, cookie)

Release a shared or exclusive lock on an object

Parameters:

key (str) -- name of the object
name (str) -- name of the lock
cookie (str) -- cookie of the lock

Raises:

TypeError

Raises:

Error

unset_osdmap_full_try(self): Unset

watch(obj, callback, error_callback=None, timeout=None)

Register an interest in an object.

Parameters:

obj (str) -- the name of the object to notify
callback (Callable[[int, str, int, bytes], None]) -- what to do when a notify is received on this object
error_callback (Optional[Callable[[int], None]]) -- what to do when the watch session encounters an error
timeout (Optional[int]) -- how many seconds the connection will keep after disconnection

Raises:

TypeError

Raises:

Error

Return type:

Watch

Returns:

watch_id - internal id assigned to this watch

write(key, data, offset=0)

Write data to an object synchronously

Parameters:

key (str) -- name of the object
data (bytes) -- data to write
offset (int) -- byte offset in the object to begin writing at

Raises:

TypeError

Raises:

LogicError

Returns:

int - 0 on success

write_full(key, data)

Write an entire object synchronously.

The object is filled with the provided data. If the object exists, it is atomically truncated and then written.

Parameters:

key (str) -- name of the object
data (bytes) -- data to write

Raises:

TypeError

Raises:

Error

Returns:

int - 0 on success

writesame(key, data, write_len, offset=0)

Write the same buffer multiple times

Parameters:

key (str) -- name of the object
data (bytes) -- data to write
write_len (int) -- total number of bytes to write
offset (int) -- byte offset in the object to begin writing at

Raises:

TypeError

Raises:

LogicError

Object Interface

From an I/O context, you can retrieve a list of objects from a pool and iterate over them. The object interface makes each object look like a file, and you may perform synchronous operations on the objects. For asynchronous operations, use the I/O context methods.

class rados.Object(ioctx, key, locator_key=None, nspace=None)

Rados object wrapper, makes the object look like a file

require_object_exists(self)

seek(position)

Other Classes

Ioctx methods return these classes for asynchronous completions, batched read and write operations, watches, snapshots, and iteration.

class rados.Completion

completion object

get_return_value()

Get the return value of an asychronous operation

The return value is set when the operation is complete or safe, whichever comes first.

Return type:: int
Returns:: return value of the operation

ioctx: ioctx: rados.Ioctx

is_complete()

Has an asynchronous operation completed?

This does not imply that the safe callback has finished.

Return type:: bool
Returns:: True if the operation is completed

is_safe()

Is an asynchronous operation safe?

This does not imply that the safe callback has finished.

Return type:: bool
Returns:: True if the operation is safe

oncomplete: oncomplete: object

onsafe: onsafe: object

wait_for_complete(self)

Wait for an asynchronous operation to complete

This does not imply that the complete callback has finished.

wait_for_complete_and_cb(self)

Wait for an asynchronous operation to complete and for the complete callback to have returned

Returns:: whether the operation is completed

wait_for_safe(self)

Wait for an asynchronous operation to be marked safe

wait_for_safe() is an alias of wait_for_complete() since Luminous

wait_for_safe_and_cb(self): Wait for an asynchronous operation to be marked safe and for the safe callback to have returned

class rados.WriteOp

append(to_write)

Append data to an object synchronously

Parameters:: to_write (bytes) -- data to write

assert_version(version)

Check if object’s version is the expected one.

Parameters:

version (int) -- expected version of the object
type -- int

cmpext(cmp_buf, offset=0)

Ensure that given object range (extent) satisfies comparison

Parameters:

cmp_buf (bytes) -- buffer containing bytes to be compared with object contents
offset (int) -- object byte offset at which to start the comparison

create(self)

execute(cls, method, data)

Execute an OSD class method on an object

Parameters:

cls (str) -- name of the object class
method (str) -- name of the method
data (bytes) -- input data

new(exclusive=None): Create the object.

omap_cmp(key, val, cmp_op=0)

Ensure that an omap key value satisfies comparison

Parameters:

key (Union[str, bytes]) -- omap key whose associated value is evaluated for comparison
val (Union[str, bytes]) -- value to compare with
cmp_op (int) -- comparison operator, one of LIBRADOS_CMPXATTR_OP_EQ (1), LIBRADOS_CMPXATTR_OP_GT (3), or LIBRADOS_CMPXATTR_OP_LT (5).

release(self)

remove(self): Remove object.

rm_xattr(xattr_name): Removes an extended attribute on from an object. :type xattr_name: str :param xattr_name: name of the xattr to remove

set_flags(flags=0)

Set flags for the last operation added to this write_op.

Parameters:: flags (int) -- flags to apply to the last operation

set_xattr(xattr_name, xattr_value)

Set an extended attribute on an object.

Parameters:

xattr_name (str) -- name of the xattr
xattr_value (bytes) -- buffer to set xattr to

truncate(offset)

Truncate an object.

Parameters:: offset (int) -- byte offset in the object to begin truncating at

write(to_write, offset=0)

Write to offset.

Parameters:

to_write (bytes) -- data to write
offset (int) -- byte offset in the object to begin writing at

write_full(to_write)

Write whole object, atomically replacing it.

Parameters:: to_write (bytes) -- data to write

writesame(to_write, write_len, offset=0)

Write the same buffer multiple times

Parameters:

to_write (bytes) -- data to write
write_len (int) -- total number of bytes to write
offset (int) -- byte offset in the object to begin writing at

zero(offset, length)

Zero part of an object.

Parameters:

offset (int) -- byte offset in the object to begin writing at
offset -- number of zero to write

class rados.ReadOp

cmpext(cmp_buf, offset=0)

Ensure that given object range (extent) satisfies comparison

Parameters:

cmp_buf (bytes) -- buffer containing bytes to be compared with object contents
offset (int) -- object byte offset at which to start the comparison

create(self)

release(self)

set_flags(flags=0)

Set flags for the last operation added to this read_op.

Parameters:: flags (int) -- flags to apply to the last operation

class rados.Watch

watch object

check(self)

Check on watch validity.

Raises:: Error
Returns:: timedelta since last confirmed valid

close(self)

Unregister an interest in an object.

Raises:: Error

get_id()

Return type:: int

class rados.Snap

Snapshot object

get_timestamp()

Find when a snapshot in the current pool occurred

Raises:: Error
Return type:: float
Returns:: the data and time the snapshot was created

ioctx: ioctx: rados.Ioctx

name: name: object

snap_id: snap_id: ‘rados_snap_t’

class rados.ObjectIterator

rados.Ioctx Object iterator

__next__(self)

Get the next object name and locator in the pool

Raises:: StopIteration
Returns:: next rados.Ioctx Object

ioctx: ioctx: object

class rados.XattrIterator

Extended attribute iterator

__next__(self)

Get the next xattr on the object

Raises:: StopIteration
Returns:: pair - of name and value of the next Xattr

ioctx: ioctx: rados.Ioctx

oid: oid: object

class rados.OmapIterator

Omap iterator

__next__(self): Get the next key-value pair in the object :returns: next rados.OmapItem

ioctx: ioctx: rados.Ioctx

omap_key_type: omap_key_type: object

class rados.SnapIterator

Snapshot iterator

__next__(self) → 'Snap'

Get the next Snapshot

Raises:: Error, StopIteration
Returns:: next snapshot

ioctx: ioctx: rados.Ioctx

Brought to you by the Ceph Foundation

The Ceph Documentation is a community resource funded and hosted by the non-profit Ceph Foundation. If you would like to support this and our other efforts, please consider joining now.