teuthology.orchestra package

Submodules

teuthology.orchestra.cluster module

Cluster definition part of context, Cluster is used to save connection information.

class teuthology.orchestra.cluster.Cluster(remotes=None)

Bases: object

Manage SSH connections to a cluster of machines.

add(remote, roles)

Add roles to the list of remotes.

exclude(*roles)

Return a cluster without remotes that have all of given roles.

This is the opposite of only.

only(*roles)

Return a cluster with only the remotes that have all of given roles.

For roles given as strings, they are matched against the roles on a remote, and the remote passes the check only if all the roles listed are present.

Argument can be callable, and will act as a match on roles of the remote. The matcher will be evaluated one role at a time, but a match on any role is good enough. Note that this is subtly diffent from the behavior of string roles, but is logical if you consider a callable to be similar to passing a non-string object with an __eq__ method.

For example:

web = mycluster.only(lambda role: role.startswith('web-'))
run(**kwargs)

Run a command on all the nodes in this cluster.

Goes through nodes in alphabetical order.

If you don’t specify wait=False, this will be sequentially.

Returns a list of RemoteProcess.

write_file(file_name, content, sudo=False, perms=None, owner=None)

Write text to a file on each node.

Parameters:
  • file_name – file name
  • content – file content
  • sudo – use sudo
  • perms – file permissions (passed to chmod) ONLY if sudo is True

teuthology.orchestra.connection module

Connection utilities

teuthology.orchestra.connection.connect(user_at_host, host_key=None, keep_alive=False, timeout=60, _SSHClient=None, _create_key=None, retry=True, key_filename=None)

ssh connection routine.

Parameters:
  • user_at_hostuser@host
  • host_key – ssh key
  • keep_alive – keep_alive indicator
  • timeout – timeout in seconds
  • _SSHClient – client, default is paramiko ssh client
  • _create_key – routine to create a key (defaults to local reate_key)
  • retry – Whether or not to retry failed connection attempts (eventually giving up if none succeed). Default is True
  • key_filename – Optionally override which private key to use.
Returns:

ssh connection.

teuthology.orchestra.connection.create_key(keytype, key)

Create an ssh-rsa or ssh-dss key.

teuthology.orchestra.connection.split_user(user_at_host)

break apart user@host fields into user and host.

teuthology.orchestra.console module

class teuthology.orchestra.console.PhysicalConsole(name, ipmiuser=None, ipmipass=None, ipmidomain=None, logfile=None, timeout=20)

Bases: teuthology.orchestra.console.RemoteConsole

Physical Console (set from getRemoteConsole)

check_power(state, timeout=None)

Check power. Retry if EOF encountered on power check read.

check_status(timeout=None)

Check status. Returns True if console is at login prompt

hard_reset(wait_for_login=True)

Perform physical hard reset. Retry if EOF returned from read and wait for login when complete.

power_cycle(timeout=300)

Power cycle and wait for login.

Parameters:timeout – How long to wait for login
power_off()

Physical power off. Loop checking cmd return.

power_off_for_interval(interval=30)

Physical power off for an interval. Wait for login when complete.

Parameters:interval – Length of power-off period.
power_on()

Physical power on. Loop checking cmd return.

spawn_sol_log(dest_path)

Using the subprocess module, spawn an ipmitool process using ‘sol activate’ and redirect its output to a file.

Returns:a psutil.Popen object
class teuthology.orchestra.console.RemoteConsole
getShortName(name=None)

Extract the name portion from remote name strings.

class teuthology.orchestra.console.VirtualConsole(name)

Bases: teuthology.orchestra.console.RemoteConsole

Virtual Console (set from getRemoteConsole)

check_power(state, timeout=None)

Return true if vm domain state indicates power is on.

check_status(timeout=None)

Return true if running.

hard_reset()

Simiulate hard reset

power_cycle()

Simiulate virtual machine power cycle

power_off()

Simiulate power off

power_off_for_interval(interval=30)

Simiulate power off for an interval.

power_on()

Simiulate power on

teuthology.orchestra.monkey module

Monkey patches (paramiko support)

teuthology.orchestra.monkey.patch_001_paramiko_deprecation()

Silence an an unhelpful Deprecation Warning triggered by Paramiko.

Not strictly a monkeypatch.

teuthology.orchestra.monkey.patch_100_logger_getChild()

Imitate Python 2.7 feature Logger.getChild.

teuthology.orchestra.monkey.patch_100_paramiko_log()

Silence some noise paramiko likes to log.

Not strictly a monkeypatch.

teuthology.orchestra.monkey.patch_100_trigger_rekey()
teuthology.orchestra.monkey.patch_all()

Run all the patch_* functions in this module.

teuthology.orchestra.opsys module

class teuthology.orchestra.opsys.OS(name=None, version=None, codename=None)

Bases: object

Class that parses either /etc/os-release or the output of ‘lsb_release -a’ and provides OS name and version information.

Must be initialized with OS.from_lsb_release or OS.from_os_release

codename
classmethod from_lsb_release(lsb_release_str)

Parse output from lsb_release -a and populate attributes

Given output like:
Distributor ID: Ubuntu Description: Ubuntu 12.04.4 LTS Release: 12.04 Codename: precise
Attributes will be:
name = ‘ubuntu’ version = ‘12.04’ codename = ‘precise’
Additionally, we set the package type:
package_type = ‘deb’
classmethod from_os_release(os_release_str)

Parse /etc/os-release and populate attributes

Given output like:
NAME=”Ubuntu” VERSION=”12.04.4 LTS, Precise Pangolin” ID=ubuntu ID_LIKE=debian PRETTY_NAME=”Ubuntu precise (12.04.4 LTS)” VERSION_ID=”12.04”
Attributes will be:
name = ‘ubuntu’ version = ‘12.04’ codename = None
Additionally, we set the package type:
package_type = ‘deb’
name
package_type
to_dict()
version
classmethod version_codename(name, version_or_codename)

Return (version, codename) based on one input, trying to infer which we’re given

teuthology.orchestra.remote module

Support for paramiko remote objects.

class teuthology.orchestra.remote.Remote(name, ssh=None, shortname=None, console=None, host_key=None, keep_alive=True)

Bases: object

A connection to a remote host.

This is a higher-level wrapper around Paramiko’s SSHClient.

arch
chcon(file_path, context)

Set the SELinux context of a given file.

VMs and non-RPM-based hosts will skip this operation because ours currently have SELinux disabled.

Parameters:
  • file_path – The path to the file
  • context – The SELinux context to be used
chmod(file_path, permissions)

As super-user, set permissions on the remote file specified.

cidr

The network (in CIDR notation) used by the remote’s SSH connection

connect(timeout=None, create_key=None, context='connect')
console
ensure_online()
get_file(path, sudo=False, dest_dir='/tmp')

Fetch a remote file, and return its local filename.

Parameters:
  • sudo – Use sudo on the remote end to read a file that requires it. Defaults to False.
  • dest_dir – Store the file in this directory. If it is /tmp, generate a unique filename; if not, use the original filename.
Returns:

The path to the local file

get_tar(path, to_path, sudo=False)

Tar a remote directory and copy it locally

get_tar_stream(path, sudo=False)

Tar-compress a remote directory and return the RemoteProcess for streaming

host_key
hostname
init_system

Which init system does the remote use?

Returns:‘systemd’ or None
interface

The interface used by the current SSH connection

inventory_info
ip_address
is_online
is_vm
machine_type
mkdtemp(suffix=None, parentdir=None)

Create a temporary directory on remote machine and return it’s path.

mktemp(suffix=None, parentdir=None)

Make a remote temporary file

Returns: the path of the temp file created.

os
put_file(path, dest_path, sudo=False)

Copy a local filename to a remote file

reconnect(timeout=None, socket_timeout=None, sleep_time=30)

Attempts to re-establish connection. Returns True for success; False for failure.

remove(path)
run(**kwargs)

This calls orchestra.run.run with our SSH client.

TODO refactor to move run.run here?

sh(script, **kwargs)

Shortcut for run method.

Usage:
my_name = remote.sh(‘whoami’) remote_date = remote.sh(‘date’)
sh_file(script, label='script', sudo=False, **kwargs)

Run shell script after copying its contents to a remote file

Parameters:
  • script – string with script text, or file object
  • sudo – run command with sudo if True, run as user name if string value (defaults to False)
  • label – string value which will be part of file name

Returns: stdout

shortname
system_type

System type decorator

teuthology.orchestra.remote.getRemoteConsole(name, ipmiuser=None, ipmipass=None, ipmidomain=None, logfile=None, timeout=20)

Return either VirtualConsole or PhysicalConsole depending on name.

teuthology.orchestra.run module

Paramiko run support

class teuthology.orchestra.run.KludgeFile(wrapped)

Bases: object

Wrap Paramiko’s ChannelFile in a way that lets f.close() actually cause an EOF for the remote command.

close()

Close and shutdown.

class teuthology.orchestra.run.Raw(value)

Bases: object

Raw objects are passed to remote objects and are not processed locally.

class teuthology.orchestra.run.RemoteProcess(client, args, check_status=True, hostname=None, label=None, timeout=None, wait=True, logger=None, cwd=None)

Bases: object

An object to begin and monitor execution of a process on a remote host

add_greenlet(greenlet)
args
check_status
client
command
deadlock_warning = 'Using PIPE for %s without wait=False would deadlock'
execute()

Execute remote command

exitstatus
finished
greenlets
hostname
label
logger
poll()
Returns:self.returncode if the process is finished; else None
remote
returncode
setup_output_stream(stream_obj, stream_name)
setup_stdin(stream_obj)
stderr
stdin
stdout
timeout
wait()

Block until remote process finishes.

Returns:self.returncode
class teuthology.orchestra.run.Sentinel(name)

Bases: object

Sentinel – used to define PIPE file-like object.

teuthology.orchestra.run.copy_and_close(src, fdst)

copyfileobj call wrapper.

teuthology.orchestra.run.copy_file_to(src, logger, stream=None)

Copy file :param src: file to be copied. :param logger: the logger object :param stream: an optional file-like object which will receive a copy of

src.
teuthology.orchestra.run.copy_to_log(f, logger, loglevel=20, capture=None)

Copy line by line from file in f to the log from logger

Parameters:
  • f – source stream object
  • logger – the destination logger object
  • loglevel – the level of logging data
  • capture – an optional stream object for data copy
teuthology.orchestra.run.quote(args)

Internal quote wrapper.

teuthology.orchestra.run.run(client, args, stdin=None, stdout=None, stderr=None, logger=None, check_status=True, wait=True, name=None, label=None, timeout=None, cwd=None, omit_sudo=False)

Run a command remotely. If any of ‘args’ contains shell metacharacters that you want to pass unquoted, pass it as an instance of Raw(); otherwise it will be quoted with pipes.quote() (single quote, and single quotes enclosed in double quotes).

Parameters:
  • client – SSHConnection to run the command with
  • args (list of string) – command to run
  • stdin – Standard input to send; either a string, a file-like object, None, or PIPE. PIPE means caller is responsible for closing stdin, or command may never exit.
  • stdout – What to do with standard output. Either a file-like object, a logging.Logger, PIPE, or None for copying to default log. PIPE means caller is responsible for reading, or command may never exit.
  • stderr – What to do with standard error. See stdout.
  • logger – If logging, write stdout/stderr to “out” and “err” children of this logger. Defaults to logger named after this module.
  • check_status – Whether to raise CommandFailedError on non-zero exit status, and . Defaults to True. All signals and connection loss are made to look like SIGHUP.
  • wait – Whether to wait for process to exit. If False, returned r.exitstatus s a gevent.event.AsyncResult, and the actual status is available via .get().
  • name – Human readable name (probably hostname) of the destination host
  • label – Can be used to label or describe what the command is doing.
  • timeout – timeout value for args to complete on remote channel of paramiko
  • cwd – Directory in which the command should be executed.
teuthology.orchestra.run.spawn_asyncresult(fn, *args, **kwargs)

Spawn a Greenlet and pass it’s results to an AsyncResult.

This function is useful to shuffle data from a Greenlet to AsyncResult, which then again is useful because any Greenlets that raise exceptions will cause tracebacks to be shown on stderr by gevent, even when .link_exception has been called. Using an AsyncResult avoids this.

teuthology.orchestra.run.wait(processes, timeout=None)

Wait for all given processes to exit.

Raise if any one of them fails.

Optionally, timeout after ‘timeout’ seconds.

Module contents