tlevine/urchin
1
0
Fork 0
Code Issues 5 Pull Requests 1 Packages Projects Releases Wiki Activity

document remote test-running

Browse Source
This commit is contained in:
Thomas Levine 2016-04-08 20:56:15 +00:00
parent 0227fc73f4
commit c99f6f2919
1 changed files with 28 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
execution-flow
View File

@ -1,3 +1,9 @@
Here I discuss Urchin's general execution flow and how it is handled
specifically when tests are run on remote environments.
Steps of an Urchin run
----------------------
When Urchin runs a directory of files, it goes through the following When Urchin runs a directory of files, it goes through the following
steps. steps.
@ -28,6 +34,8 @@ format.
Stdout and stderr from test runs are written to files in the stdout Stdout and stderr from test runs are written to files in the stdout
directory, one file per test file per shell that the file is run in. directory, one file per test file per shell that the file is run in.
The reporting phase
----------------------
In most cases Urchin begins printing to the screen only during the In most cases Urchin begins printing to the screen only during the
reporting phase. The only case where anything is printed beforehand is reporting phase. The only case where anything is printed beforehand is
when Urchin is run with -vvvv; that sets "+x", so the commands are when Urchin is run with -vvvv; that sets "+x", so the commands are
@ -36,13 +44,24 @@ printed as they run, though all other output is still suppressed.
Test results are reported in the reporting phase. Four output formats Test results are reported in the reporting phase. Four output formats
are available. are available.
* Urchin's human-readable format (default) 1. Urchin's human-readable format (default)
* Test Anything Protocol 2. Test Anything Protocol
* Delimiter-separated values (used internally) 3. Delimiter-separated values (used internally)
* Remote Urchin worker output 4. Remote Urchin worker output
Most of the output is generated based on the delimiter-separated values
in the test log file. The first two formats also include stdout and
stderr from the tests, depending on verbosity level flags; when it needs
these, Urchin reads them from appropriate files in the temporary
directory.
I could discuss the further details of each format elsewhere.
Remotes
----------------------
When Urchin runs tests on a remote, it copies tests to the remote and When Urchin runs tests on a remote, it copies tests to the remote and
then calls Urchin with "--format=remote". This specifies the following. then calls Urchin on the remote with "--format=remote". This specifies
the following.
* The temporary directory should be kept, rather than deleted, after * The temporary directory should be kept, rather than deleted, after
Urchin runs. Urchin runs.
@ -55,13 +74,15 @@ It modifies the file to include the remote's name and then concatenates
the result to the "remote-test" file in the local temporary directory. the result to the "remote-test" file in the local temporary directory.
For example, the file from the remote might look like this, For example, the file from the remote might look like this,
sh:Counting tests/.test/faila:0:not_ok :sh:Counting tests/.test/faila:0:not_ok
and the result might look like this. and the result might look like this.
sh on nsa:Counting tests/.test/faila:0:not_ok nsa:sh:Counting tests/.test/faila:0:not_ok
This gets processed in the reporting step like usual, according to This gets processed in the reporting step like usual, according to
whatever format is specified. Instead of printing just "sh" as the whatever format is specified. Instead of printing just "sh" as the
environment in which the particular test was run, the report will print environment in which the particular test was run, the report will print
"sh on nsa". "sh on nsa".
When it needs the stdout files, it prints them over ssh.