teuthology.orchestra package


teuthology.orchestra.cluster module

Cluster definition part of context, Cluster is used to save connection information.

class teuthology.orchestra.cluster.Cluster(remotes=None)

Bases: object

Manage SSH connections to a cluster of machines.

add(remote, roles)

Add roles to the list of remotes.


Return a cluster without remotes that have all of given roles.

This is the opposite of only.


Return a cluster with only the remotes that have all of given roles.

For roles given as strings, they are matched against the roles on a remote, and the remote passes the check only if all the roles listed are present.

Argument can be callable, and will act as a match on roles of the remote. The matcher will be evaluated one role at a time, but a match on any role is good enough. Note that this is subtly diffent from the behavior of string roles, but is logical if you consider a callable to be similar to passing a non-string object with an __eq__ method.

For example:

web = mycluster.only(lambda role: role.startswith('web-'))

Run a command on all the nodes in this cluster.

Goes through nodes in alphabetical order.

If you don’t specify wait=False, this will be sequentially.

Returns a list of RemoteProcess.

sh(script, **kwargs)

Run a command on all the nodes in this cluster.

Goes through nodes in alphabetical order.

Returns a list of the command outputs correspondingly.

write_file(file_name, content, sudo=False, perms=None, owner=None)

Write text to a file on each node.

  • file_name – file name

  • content – file content

  • sudo – use sudo

  • perms – file permissions (passed to chmod) ONLY if sudo is True

teuthology.orchestra.connection module

Connection utilities

teuthology.orchestra.connection.connect(user_at_host, host_key=None, keep_alive=False, timeout=60, _SSHClient=None, _create_key=None, retry=True, key_filename=None)

ssh connection routine.

  • user_at_hostuser@host

  • host_key – ssh key

  • keep_alive – keep_alive indicator

  • timeout – timeout in seconds

  • _SSHClient – client, default is paramiko ssh client

  • _create_key – routine to create a key (defaults to local reate_key)

  • retry – Whether or not to retry failed connection attempts (eventually giving up if none succeed). Default is True

  • key_filename – Optionally override which private key to use.


ssh connection.

teuthology.orchestra.connection.create_key(keytype, key)

Create an ssh-rsa, ssh-dss or ssh-ed25519 key.


break apart user@host fields into user and host.

teuthology.orchestra.console module

class teuthology.orchestra.console.PhysicalConsole(name, ipmiuser=None, ipmipass=None, ipmidomain=None, logfile=None, timeout=40)

Bases: teuthology.orchestra.console.RemoteConsole

Physical Console (set from getRemoteConsole)

check_power(state, timeout=None)

Check power. Retry if EOF encountered on power check read.


Check status. Returns True if console is at login prompt


Perform physical hard reset. Retry if EOF returned from read and wait for login when complete.


Power cycle and wait for login.


timeout – How long to wait for login


Physical power off. Loop checking cmd return.


Physical power off for an interval. Wait for login when complete.


interval – Length of power-off period.


Physical power on. Loop checking cmd return.


Using the subprocess module, spawn an ipmitool process using ‘sol activate’ and redirect its output to a file.


a psutil.Popen object

class teuthology.orchestra.console.RemoteConsole

Bases: object


Extract the name portion from remote name strings.

class teuthology.orchestra.console.VirtualConsole(name)

Bases: teuthology.orchestra.console.RemoteConsole

Virtual Console (set from getRemoteConsole)

check_power(state, timeout=None)

Return true if vm domain state indicates power is on.


Return true if running.


Simiulate hard reset


Simiulate virtual machine power cycle


Simiulate power off


Simiulate power off for an interval.


Simiulate power on

teuthology.orchestra.monkey module

Monkey patches (paramiko support)


Silence an an unhelpful Deprecation Warning triggered by Paramiko.

Not strictly a monkeypatch.


Imitate Python 2.7 feature Logger.getChild.


Silence some noise paramiko likes to log.

Not strictly a monkeypatch.


Run all the patch_* functions in this module.

teuthology.orchestra.opsys module

class teuthology.orchestra.opsys.OS(name=None, version=None, codename=None)

Bases: object

Class that parses either /etc/os-release or the output of ‘lsb_release -a’ and provides OS name and version information.

Must be initialized with OS.from_lsb_release or OS.from_os_release

classmethod from_lsb_release(lsb_release_str)

Parse output from lsb_release -a and populate attributes

Given output like:

Distributor ID: Ubuntu Description: Ubuntu 12.04.4 LTS Release: 12.04 Codename: precise

Attributes will be:

name = ‘ubuntu’ version = ‘12.04’ codename = ‘precise’

Additionally, we set the package type:

package_type = ‘deb’

classmethod from_os_release(os_release_str)

Parse /etc/os-release and populate attributes

Given output like:

NAME=”Ubuntu” VERSION=”12.04.4 LTS, Precise Pangolin” ID=ubuntu ID_LIKE=debian PRETTY_NAME=”Ubuntu precise (12.04.4 LTS)” VERSION_ID=”12.04”

Attributes will be:

name = ‘ubuntu’ version = ‘12.04’ codename = None

Additionally, we set the package type:

package_type = ‘deb’

classmethod version_codename(name, version_or_codename)

Return (version, codename) based on one input, trying to infer which we’re given

teuthology.orchestra.remote module

Support for paramiko remote objects.

class teuthology.orchestra.remote.Remote(name, ssh=None, shortname=None, console=None, host_key=None, keep_alive=True)

Bases: object

A connection to a remote host.

This is a higher-level wrapper around Paramiko’s SSHClient.

property arch
chcon(file_path, context)

Set the SELinux context of a given file.

VMs and non-RPM-based hosts will skip this operation because ours currently have SELinux disabled.

  • file_path – The path to the file

  • context – The SELinux context to be used

chmod(file_path, permissions)

As super-user, set permissions on the remote file specified.

property cidr

The network (in CIDR notation) used by the remote’s SSH connection

connect(timeout=None, create_key=None, context='connect')
property console
get_file(path, sudo=False, dest_dir='/tmp')

Fetch a remote file, and return its local filename.

  • sudo – Use sudo on the remote end to read a file that requires it. Defaults to False.

  • dest_dir – Store the file in this directory. If it is /tmp, generate a unique filename; if not, use the original filename.


The path to the local file

get_tar(path, to_path, sudo=False)

Tar a remote directory and copy it locally

get_tar_stream(path, sudo=False)

Tar-compress a remote directory and return the RemoteProcess for streaming

property host_key
property hostname
property init_system

Which init system does the remote use?


‘systemd’ or None

property interface

The interface used by the current SSH connection

property inventory_info
property ip_address
property is_online
property is_vm
property machine_type
mkdtemp(suffix=None, parentdir=None)

Create a temporary directory on remote machine and return it’s path.

mktemp(suffix=None, parentdir=None)

Make a remote temporary file

Returns: the path of the temp file created.

property os
put_file(path, dest_path, sudo=False)

Copy a local filename to a remote file

reconnect(timeout=None, socket_timeout=None, sleep_time=30)

Attempts to re-establish connection. Returns True for success; False for failure.


This calls orchestra.run.run with our SSH client.

TODO refactor to move run.run here?

sh(script, **kwargs)

Shortcut for run method.


my_name = remote.sh(‘whoami’) remote_date = remote.sh(‘date’)

sh_file(script, label='script', sudo=False, **kwargs)

Run shell script after copying its contents to a remote file

  • script – string with script text, or file object

  • sudo – run command with sudo if True, run as user name if string value (defaults to False)

  • label – string value which will be part of file name

Returns: stdout

property shortname
property system_type

System type decorator

teuthology.orchestra.remote.getRemoteConsole(name, ipmiuser=None, ipmipass=None, ipmidomain=None, logfile=None, timeout=60)

Return either VirtualConsole or PhysicalConsole depending on name.

teuthology.orchestra.run module

Paramiko run support

class teuthology.orchestra.run.KludgeFile(wrapped)

Bases: object

Wrap Paramiko’s ChannelFile in a way that lets f.close() actually cause an EOF for the remote command.


Close and shutdown.

class teuthology.orchestra.run.Raw(value)

Bases: object

Raw objects are passed to remote objects and are not processed locally.

class teuthology.orchestra.run.RemoteProcess(client, args, check_status=True, hostname=None, label=None, timeout=None, wait=True, logger=None, cwd=None)

Bases: object

An object to begin and monitor execution of a process on a remote host

deadlock_warning = 'Using PIPE for %s without wait=False would deadlock'

Execute remote command

property finished

self.returncode if the process is finished; else None

setup_output_stream(stream_obj, stream_name)

Block until remote process finishes.



class teuthology.orchestra.run.Sentinel(name)

Bases: object

Sentinel – used to define PIPE file-like object.

teuthology.orchestra.run.copy_and_close(src, fdst)

copyfileobj call wrapper.

teuthology.orchestra.run.copy_file_to(src, logger, stream=None)

Copy file :param src: file to be copied. :param logger: the logger object :param stream: an optional file-like object which will receive a copy of


teuthology.orchestra.run.copy_to_log(f, logger, loglevel=20, capture=None)

Copy line by line from file in f to the log from logger

  • f – source stream object

  • logger – the destination logger object

  • loglevel – the level of logging data

  • capture – an optional stream object for data copy


Internal quote wrapper.

teuthology.orchestra.run.run(client, args, stdin=None, stdout=None, stderr=None, logger=None, check_status=True, wait=True, name=None, label=None, timeout=None, cwd=None, omit_sudo=False)

Run a command remotely. If any of ‘args’ contains shell metacharacters that you want to pass unquoted, pass it as an instance of Raw(); otherwise it will be quoted with pipes.quote() (single quote, and single quotes enclosed in double quotes).

  • client – SSHConnection to run the command with

  • args (list of string) – command to run

  • stdin – Standard input to send; either a string, a file-like object, None, or PIPE. PIPE means caller is responsible for closing stdin, or command may never exit.

  • stdout – What to do with standard output. Either a file-like object, a logging.Logger, PIPE, or None for copying to default log. PIPE means caller is responsible for reading, or command may never exit.

  • stderr – What to do with standard error. See stdout.

  • logger – If logging, write stdout/stderr to “out” and “err” children of this logger. Defaults to logger named after this module.

  • check_status – Whether to raise CommandFailedError on non-zero exit status, and . Defaults to True. All signals and connection loss are made to look like SIGHUP.

  • wait – Whether to wait for process to exit. If False, returned r.exitstatus s a gevent.event.AsyncResult, and the actual status is available via .get().

  • name – Human readable name (probably hostname) of the destination host

  • label – Can be used to label or describe what the command is doing.

  • timeout – timeout value for args to complete on remote channel of paramiko

  • cwd – Directory in which the command should be executed.

teuthology.orchestra.run.spawn_asyncresult(fn, *args, **kwargs)

Spawn a Greenlet and pass it’s results to an AsyncResult.

This function is useful to shuffle data from a Greenlet to AsyncResult, which then again is useful because any Greenlets that raise exceptions will cause tracebacks to be shown on stderr by gevent, even when .link_exception has been called. Using an AsyncResult avoids this.

teuthology.orchestra.run.wait(processes, timeout=None)

Wait for all given processes to exit.

Raise if any one of them fails.

Optionally, timeout after ‘timeout’ seconds.

Module contents