wiki:DeterLabCommands

< Back to Core Reference

DeterLab Shell Commands

The following commands are available from the commandline on users.isi.deterlab.net. Note that commands should be pre-pended with the path: /usr/testbed/bin.

For example, to start an experiment, you would use:

/usr/testbed/bin/startexp [options]

Transport Layers

The DeterLab XMLRPC server can be accessed via two different transport layers: SSH and SSL.

How to use SSH keys Follow these directions if you are unfamiliar with using SSH.

How to use SSL You need to request a certificate from the DeterLab website in order to use the SSL based server.

  • Click the My DeterLab menu item in the navbar, clik the Profile tab on the page and then click on the Generate SSL Certificate link.
  • Enter a passphrase to use to encrypt the private key.
  • Once the key has been created, you will be given a link to download a text version (in PEM format). Simply provide this certificate as an input to your SSL client.

Operational Commands

startexp: Start an DeterLab experiment

startexp [-q] [-i [-w]] [-f] [-N] [-E description] [-g gid]
         [-S reason] [-L reason] [-a <time>] [-l <time>]
         -p <pid> -e <eid> <nsfile>

Options:

-i
Swapin immediately; by default, the experiment is batched
-w
Wait for non-batchmode experiment to preload or swapin
-f
Preload experiment (do not swapin or queue yet)
-q
Be less verbose
-S
Experiment cannot be swapped; must provide reason
-L
Experiment cannot be IDLE swapped; must provide reason
-a
Auto swapout NN minutes after experiment is swapped in
-l
Auto swapout NN minutes after experiment goes idle
-E
A concise sentence describing your experiment
-g
The subgroup in which to create the experiment
-p
The project in which to create the experiment
-e
The experiment name (unique, alphanumeric, no blanks)
-N
Suppress most email to the user and testbed-ops
nsfile
NS file to parse for experiment


batchexp: Synonym for startexp

This is a legacy command. See command description for startexp.


endexp: Terminate an experiment

endexp [-w] [-N] -e pid,eid
endexp [-w] [-N] pid eid

Options:

-w
Wait for experiment to finish terminating
-e
Project and Experiment ID
-N
Suppress most email to the user and testbed-ops

Notes:

  • Use with caution''' This will tear down your experiment and you will not be able to swap it back in.
  • By default, endexp runs in the background, sending you email when the transition has completed. Use the -w option to wait

in the foreground, returning exit status. Email is still sent.

  • The experiment may be terminated when it is currently swapped in or swapped out.

delay_config: Change the link shaping characteristics for a link or LAN

delay_config [options] -e pid,eid link PARAM=value ...
delay_config [options] pid eid link PARAM=value ...

Options:

-m
Modify virtual experiment as well as current state
-s
Select the source of the link to change
-e
Project and Experiment ID to operate on
link
Name of link from your NS file (ie: 'link1')

Parameters:

BANDWIDTH=NNN
N=bandwidth (10-100000 Kbits per second)
PLR=NNN
N=lossrate (0 <= plr < 1)
DELAY=NNN
N=delay (one-way delay in milliseconds > 0)
LIMIT=NNN
The queue size in bytes or packets
QUEUE-IN-BYTES=N
0 means in packets, 1 means in bytes

RED/GRED Options: (only if link was specified as RED/GRED)

MAXTHRESH=NNN
Maximum threshold for the average Q size
THRESH=NNN
Minimum threshold for the average Q size
LINTERM=NNN
Packet dropping probability
Q_WEIGHT=NNN
For calculating the average queue size


modexp: Modify experiment

modexp [-r] [-s] [-w] [-N] -e pid,eid nsfile
modexp [-r] [-s] [-w] [-N] pid eid nsfile

Options:

-w
Wait for experiment to finish swapping
-e
Project and Experiment ID
-r
Reboot nodes (when experiment is active)
-s
Restart event scheduler (when experiment is active)
-N
Suppress most email to the user and testbed-ops

Notes:

  • By default, modexp runs in the background, sending you email when the transition has completed. Use the -w option to wait

in the foreground, returning exit status. Email is still sent.

  • The experiment can be either swapped in or swapped out.
    • If the experiment is swapped out, the new NS file replaces the existing NS file (the virtual topology is updated).
    • If the experiment is swapped in (active), the physical topology is also updated, subject to the -r and -s options above.

swapexp: Swap experiment in or out

swapexp -e pid,eid in|out
swapexp pid eid in|out

Options:

-w
Wait for experiment to finish swapping
-e
Project and Experiment ID
-N
Suppress most email to the user and testbed-ops
in
Swap experiment in (must currently be swapped out)
out
Swap experiment out (must currently be swapped in)

Note:

  • By default, swapexp runs in the background, sending you email when the transition has completed. Use the -w option to wait in the foreground, returning exit status. Email is still sent.

create_image: Create a disk image from a node

create_image [options] imageid node

Options:

-w
Wait for image to be created
-p
Project ID of imageid
imageid
Name of the image
node
Node to create image from (pcXXX)

Example:
The following command creates or recreates an image for a particular project:

create_image -p <proj> <imageid> <node>

eventsys_control: Start/Stop/Restart? the event system

eventsys_control -e pid,eid start|stop|replay
eventsys_control pid eid start|stop|replay

Options:

-e
Project and Experiment ID
stop
Stop the event scheduler
start
Start the event stream from time index 0
replay
Replay the event stream from time index 0


loghole: Downloads and manages an experiment's log files

This utility downloads log files from certain directories on the experimental nodes (e.g. /local/logs) to the DeterLab users machine. After downloading, it can also be used to produce and manage archives of the log files.

Using this utility to manage an experiment's log files is encouraged because it will transfer the logs in a network-friendly manner and is already integrated with the rest of DeterLab. For example, any programs executed using the DeterLab event-system will have their standard output/error automatically placed in the /local/logs directory. The tool can also be used to preserve multiple trials of an experiment by producing and managing zip archives of the logs.

loghole [-hVdqva] [-e pid/eid] [-s server] [-P port] action [...]

loghole sync [-nPs] [-r remotedir] [-l localdir] [node1 node2 ...]

loghole validate

loghole  archive [-k (i-delete|space-is-needed)] [-a days] [-c comment]
    [-d] [archive-name]

loghole change [-k (i-delete|space-is-needed)] [-a days]  [-c  comment]
    archive-name1 [archive-name2 ...]

loghole list [-O1!Xo] [-m atmost] [-s megabytes]

loghole show [archive-name]

loghole clean [-fne] [node1 node2 ...]

loghole gc [-n] [-m atmost] [-s megabytes]

Options:

-h, --help
Print the usage message for the loghole utility as a whole or, if an action is given, the usage message for that action.
-V, --version
Print out version information and exit.
-d, --debug
Output debugging messages.
-q, --quiet
Decrease the level of verbosity, this is subtractive, so multiple uses of this option will make the utility quieter and quieter. The default level of verbosity is human-readable, below that is machine-readable, and below that is silent. For example, the default output from the "list" action looks like:
         [ ] foobar.1.zip   10/15
         [!] foobar.0.zip   10/13
Using a single -q option changes the output to look like:
        foobar.1.zip
        foobar.0.zip
-e, --experiment=PID/EID
Specify the experiment(s) to operate on using the Project ID (or project name) and Experiment ID (or experiment name). If multiple -e options are given, the action will apply to all of them. This option overrides the default behavior of inferring the experiment from (Note: this sentence was cut off in the Emulab documentation).

Examples:
To synchronize the log directory for experiment neptune/myexp with the log holes on the experimental nodes: [vmars@users ~] loghole -e neptune/myexp sync

To archive the newly recovered logs and print out just the name of the new log file: [vmars@users ~] loghole -e neptune/myexp -q archive

More information:
To see the detailed documentation of loghole, view the man page (loghole man) on users.


os_load: Reload disks on selected nodes or all nodes in an experiment

os_load [options] node [node ...]
os_load [options] -e pid,eid

Options:

-i
Specify image name; otherwise load default image
-p
Specify project for finding image name (-i)
-s
Do *not* wait for nodes to finish reloading
-m
Specify internal image id (instead of -i and -p)
-r
Do *not* reboot nodes; do that yourself
-e
Reboot all nodes in an experiment
node
Node to reboot (pcXXX)


portstats: Get portstats from the switches

portstats <-p | pid eid> [vname ...] [vname:port ...]

Options:

-e
Show only error counters
-a
Show all stats
-z
Zero out counts for selected counters after printing
-q
Quiet: don't actually print counts - useful with -z
-c
Print absolute, rather than relative, counts
-p
The machines given are physical, not virtual, node IDs. No pid and eid should be given with this option

Warning:
If only pid and eid are given, prints out information about all ports in the experiment. Otherwise, output is limited to the nodes and/or ports given.

Notes:

  • Statistics are reported from the switch's perspective. This means that In packets are those sent FROM the node, and Out packets are those sent TO the node.
  • In the output, packets described as 'NUnicast' or 'NUcast' are non-unicast (broadcast or multicast) packets.

node_reboot: Reboot selected nodes or all nodes in an experiment

Use this if you need to powercycle a node.

node_reboot [options] node [node ...]
node_reboot [options] -e pid,eid

Options:

-w
Wait for nodes is come back up
-c
Reconfigure nodes instead of rebooting
-f
Power cycle nodes (skip reboot!)
-e
Reboot all nodes in an experiment
node
Node to reboot (pcXXX)

Notes:

  • You may provide more than one node on the command line.
  • Be aware that you may power cycle only nodes in projects that you are member of.
  • node_reboot does its very best to perform a clean reboot before resorting to cycling the power to the node. This is to prevent the damage that can occur from constant power cycling over a long period of time. For this reason, node_reboot may delay a minute or two if it detects that the machine is still responsive to network transmission. In any event, please try to reboot your nodes first (see above).
  • You may reboot all the nodes in an experiment by using the -e option to specify the project and experiment names. This option is provided as a shorthand method for rebooting large groups of nodes.

Example:

The following command will reboot all of the nodes reserved in the "multicast" experiment in the "testbed" project.

    node_reboot -e testbed,multicast

expwait: Wait for experiment to reach a state

expwait [-t timeout] -e pid,eid state
expwait [-t timeout] pid eid state

Options:

-e
Project and Experiment ID in format <projectID>/<experimentID>
-t
Maximum time to wait (in seconds).

Informational Commands

node_list: Print physical mapping of nodes in an experiment

node_list [options] -e pid,eid

Options:

-e
Project and Experiment ID to list
-p
Print physical (DETER database) names (default)
-P
Like -p, but include node type
-v
Print virtual (experiment assigned) names
-h
Print physical name of host for virtual nodes
-c
Print container VMs and physical nodes

Notes:

  • This command now queries the XMLRPC interface as it used to do. Users who had been using script_wrapper.py node_list to access this function should use /usr/testbed/bin/node_list instead.
  • The -c flag that outputs containerized node names has been modified in two ways.
    • Names are produced without the DNS qualifiers as node names provided by other options of this command are. A node in a VM container named a will be reported as a not a.exp.proj as earlier versions of this feature did.
    • This option now reports embedded_pnode containers as well (physical machines). If no containers VMs are present in an experiment, node_list -c and node_list -v produce identical output.
  • The node_list command is now available as node_summary.

expinfo: Get information about an experiment

expinfo [-n] [-m] [-l] [-d] [-a] -e pid,eid
expinfo [-n] [-m] [-l] [-d] [-a] pid eid

Options:

-e
Project and Experiment ID
-n
Show node info
-m
Show node mapping
-l
Show link info
-a
Show all of the above


node_avail: Print free node counts

node_avail [-p project] [-c class] [-t type]

Options:

-p project
Specify project credentials for node types that are restricted
-c class
The node class (Default: pc)
-t type
The node type

Example:
The following command will print free nodes on pc850 nodes:

    $ node_avail -t pc850

node_avail_list: Print physical node_ids of available nodes

node_avail_list [-p project] [-c class] [-t type] [-n nodes]

Options:

-p project
Specify project credentials for node types that are restricted
-c class
The node class (Default: pc)
-t type
The node type
-n pcX,pcY,...,pcZ
A list of physical node_ids

Example:
The following command will print the physical node_ids for available pc850 nodes:

    $ node_avail_list -t pc850

nscheck: Check and NS file for parser errors

nscheck nsfile

Option:

nsfile
Path to NS file you to wish check for parse errors

< Back to Core Reference

Last modified 3 years ago Last modified on Nov 8, 2015, 11:32:06 PM