110 lines
3.7 KiB
Plaintext
110 lines
3.7 KiB
Plaintext
Here I discuss Urchin's general execution flow and how it is handled
|
|
specifically when tests are run on remote environments.
|
|
|
|
|
|
Steps of an Urchin run
|
|
----------------------
|
|
When Urchin runs a directory of files, it goes through the following
|
|
steps.
|
|
|
|
1. Head
|
|
2. Test
|
|
3. Foot
|
|
4. Reporting
|
|
|
|
Urchin stores files in a temporary directory, creating a new directory
|
|
on each invocation. The directory contains these things.
|
|
|
|
* head (file)
|
|
* test (file)
|
|
* foot (file)
|
|
* stdout (directory)
|
|
|
|
When run on remotes, the temporary directory corresponding to the local
|
|
master process additionally has these files.
|
|
|
|
* remote-test
|
|
|
|
Messages from the head, test, and foot steps go in the corresponding
|
|
files. In the head and foot phases, messages are just simple prints.
|
|
Messages from the test phase always correspond to a particular test
|
|
file, and they are written to the test file in a delimiter-separated
|
|
format.
|
|
|
|
Stdout and stderr from test runs are written to files in the stdout
|
|
directory, one file per test file per shell that the file is run in.
|
|
|
|
The reporting phase
|
|
----------------------
|
|
In most cases Urchin begins printing to the screen only during the
|
|
reporting phase. The only case where anything is printed beforehand is
|
|
when Urchin is run with -vvvv; that sets "+x", so the commands are
|
|
printed as they run, though all other output is still suppressed.
|
|
|
|
Test results are reported in the reporting phase. Four output formats
|
|
are available.
|
|
|
|
1. Urchin's human-readable format (default)
|
|
2. Test Anything Protocol
|
|
3. Delimiter-separated values (used internally)
|
|
4. Remote Urchin worker output
|
|
|
|
Most of the output is generated based on the delimiter-separated values
|
|
in the test log file. The first two formats also include stdout and
|
|
stderr from the tests, depending on verbosity level flags; when it needs
|
|
these, Urchin reads them from appropriate files in the temporary
|
|
directory.
|
|
|
|
I could discuss the further details of each format elsewhere.
|
|
|
|
Remotes
|
|
----------------------
|
|
When Urchin runs tests on a remote, it copies tests to the remote and
|
|
then calls Urchin on the remote with "--format=remote". This specifies
|
|
the following.
|
|
|
|
* The temporary directory should be kept, rather than deleted, after
|
|
Urchin runs.
|
|
* The path of the temporary directory should be printed as output.
|
|
* No other output should be printed to stdout.
|
|
|
|
After the remote Urchin finishes running, the local urchin downloads
|
|
the remote Urchin's test log file from the temporary directory.
|
|
It modifies the file to include the remote's name and then concatenates
|
|
the result to the "remote-test" file in the local temporary directory.
|
|
For example, the file from the remote might look like this,
|
|
|
|
:sh:Counting tests/.test/faila:0:not_ok
|
|
|
|
and the result might look like this.
|
|
|
|
nsa:sh:Counting tests/.test/faila:0:not_ok
|
|
|
|
This gets processed in the reporting step like usual, according to
|
|
whatever format is specified. Instead of printing just "sh" as the
|
|
environment in which the particular test was run, the report will print
|
|
"sh on nsa".
|
|
|
|
When it needs the stdout files, it prints them over ssh.
|
|
|
|
New flags
|
|
----------
|
|
In making this remotes feature, I wound up adding some others.
|
|
|
|
-r, --remote SSH host to use as a remote
|
|
-F, --format Output format, one of "urchin", "tap", "dsv", "remote"
|
|
|
|
Urchin runs only locally by default. If you pass at least one --remote
|
|
flag, Urchin runs tests only on the specified remotes; it can't run both
|
|
locally and remotely in the same run. If you want to do that, you could
|
|
wait until I add that feature, or you can add "localhost" as a remote.
|
|
|
|
Settings that I'm thinking about
|
|
|
|
* Port for rsync/ssh
|
|
* SSH protocol version
|
|
* --rsync-path
|
|
|
|
Can those all be set in ssh_config? Probably not --rsync-path, but
|
|
I guess I could just fix it on the remote.
|