Compare commits
1 Commits
remote-tes
...
max-forks
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9d1b16ed57 |
31
HISTORY
31
HISTORY
@@ -1,18 +1,10 @@
|
|||||||
HISTORY
|
HISTORY
|
||||||
=======
|
=======
|
||||||
|
|
||||||
Version 0.1.0
|
Version 0.1.0-rc1
|
||||||
---------------------
|
---------------------
|
||||||
This release includes breaking changes.
|
This release includes breaking changes.
|
||||||
|
|
||||||
### Cross-OS testing
|
|
||||||
I have started testing Urchin across multiple operating systems.
|
|
||||||
This gives access to more shells, as some shels are easier to install on
|
|
||||||
certain operating systems.
|
|
||||||
|
|
||||||
With this cross-OS test suite, I have extended support to more shells.
|
|
||||||
A later version of Urchin could include a remote testing feature.
|
|
||||||
|
|
||||||
### Test root directory
|
### Test root directory
|
||||||
We introduce a concept of the root directory of a test suite.
|
We introduce a concept of the root directory of a test suite.
|
||||||
Such a concept is important in case you want to run subsets of your
|
Such a concept is important in case you want to run subsets of your
|
||||||
@@ -20,18 +12,18 @@ test suite, as we need to know how far up to apply the setup
|
|||||||
and teardown files.
|
and teardown files.
|
||||||
|
|
||||||
The Urchin root directory is determined by moving higher in the directory
|
The Urchin root directory is determined by moving higher in the directory
|
||||||
tree in search of a file named `.urchin_root`.
|
tree in search of a file named `.urchin`.
|
||||||
The closest directory that contains such a file is considered the root.
|
The closest directory that contains such a file is considered the root.
|
||||||
In the following filesystem, for example, `/a/b/c` would be the root.
|
In the following filesystem, for example, `/a/b/c` would be the root.
|
||||||
|
|
||||||
mkdir -p /a/b/c/d
|
mkdir -p /a/b/c/d
|
||||||
touch /a/b/c/d/e
|
touch /a/b/c/d/e
|
||||||
chmod +x /a/b/c/d/e
|
chmod +x /a/b/c/d/e
|
||||||
touch /a/b/c/.urchin_root
|
touch /a/b/c/.urchin
|
||||||
urchin /a/b/c/d
|
urchin /a/b/c/d
|
||||||
|
|
||||||
There are two situations in which we would stop looking without having
|
There are two situations in which we would stop looking without having
|
||||||
found a `.urchin_root` file.
|
found a `.urchin` file.
|
||||||
|
|
||||||
1. The system root, `/`, because we can't go any higher
|
1. The system root, `/`, because we can't go any higher
|
||||||
2. A directory that starts with a dot, because an urchin call on a higher
|
2. A directory that starts with a dot, because an urchin call on a higher
|
||||||
@@ -80,7 +72,7 @@ might look for the dependency and then skip if it does not see the dependency.
|
|||||||
It might look like this.
|
It might look like this.
|
||||||
|
|
||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
if ! which inkscape; then
|
if which inkscape; then
|
||||||
exit 3 # status code 3 for skip
|
exit 3 # status code 3 for skip
|
||||||
fi
|
fi
|
||||||
inkscape blah blah ...
|
inkscape blah blah ...
|
||||||
@@ -105,13 +97,6 @@ turned on.
|
|||||||
|
|
||||||
Parallel processing and shell cycling are both enabled by default.
|
Parallel processing and shell cycling are both enabled by default.
|
||||||
|
|
||||||
You may want make only some directories run in series, you can create
|
|
||||||
".urchin_dir" files in those directories.
|
|
||||||
If .urchin_dir contains the phrase "series", run that directory in series
|
|
||||||
rather than in parallel.
|
|
||||||
This is helpful when directories actually need to run in series
|
|
||||||
and also when running all your tests in parallel crashes your computer.
|
|
||||||
|
|
||||||
### Options
|
### Options
|
||||||
Long options are now available for all command line flags.
|
Long options are now available for all command line flags.
|
||||||
For example, the `-s` flag is now available as `--shell` as well.
|
For example, the `-s` flag is now available as `--shell` as well.
|
||||||
@@ -179,6 +164,10 @@ set as an environment variable, and the latter was set with the -s flag..
|
|||||||
Urchin now uses the -s flag for both of these settings, and it mostly ignores
|
Urchin now uses the -s flag for both of these settings, and it mostly ignores
|
||||||
the exported TEST_SHELL variable.
|
the exported TEST_SHELL variable.
|
||||||
|
|
||||||
|
Urchin also inspects the shebang line differently. Previously, Urchin would
|
||||||
|
vary the shells with which a test is run if the shebang line either was absent
|
||||||
|
or was #!/bin/sh. Now it varies the shell only if the shebang line is absent.
|
||||||
|
|
||||||
If you pass -n/--disable-cycling, Urchin will invoke tests ordinarily and will
|
If you pass -n/--disable-cycling, Urchin will invoke tests ordinarily and will
|
||||||
only set the TEST_SHELL variable if it does not exist. If the TEST_SHELL
|
only set the TEST_SHELL variable if it does not exist. If the TEST_SHELL
|
||||||
variable is absent, it will be set to /bin/sh.
|
variable is absent, it will be set to /bin/sh.
|
||||||
@@ -240,7 +229,7 @@ other Urchin call), and the test suite is recursively descended. Setup and
|
|||||||
teardown files are sourced, and everything but the specified test file is
|
teardown files are sourced, and everything but the specified test file is
|
||||||
otherwise ignored.
|
otherwise ignored.
|
||||||
|
|
||||||
If you don't explicitly specify the Urchin root with a .urchin_root file, we
|
If you don't explicitly specify the Urchin root with a .urchin file, we
|
||||||
consider the test suite root directory to be the parent of the file that
|
consider the test suite root directory to be the parent of the file that
|
||||||
you ran Urchin on.
|
you ran Urchin on.
|
||||||
|
|
||||||
|
|||||||
8
Makefile
8
Makefile
@@ -1,8 +0,0 @@
|
|||||||
.PHONY: test install
|
|
||||||
|
|
||||||
test:
|
|
||||||
./urchin tests
|
|
||||||
./urchin -s sh -v ./cross-os-tests
|
|
||||||
|
|
||||||
install:
|
|
||||||
cp ./urchin /usr/bin
|
|
||||||
82
SORTING
82
SORTING
@@ -1,67 +1,5 @@
|
|||||||
On the criteria for ordering
|
On the criteria for ordering
|
||||||
==============================
|
==============================
|
||||||
|
|
||||||
I was confused by the documentation for sort's "-d" flag. This confusion
|
|
||||||
relates to GNU coreutil's locale-specific sort. [^]
|
|
||||||
|
|
||||||
Below I discuss sort order differences between different implementations
|
|
||||||
of sort and of sh "*" for my particular environments.
|
|
||||||
|
|
||||||
Sorting with sort
|
|
||||||
------------
|
|
||||||
Consider the following two sort commands.
|
|
||||||
|
|
||||||
printf '@ b\n- d\n? a\n~ c\n! e\n' | sort
|
|
||||||
printf '@ b\n- d\n? a\n~ c\n! e\n' | sort -d
|
|
||||||
|
|
||||||
With BusyBox v1.23.2 on NixOS 15.09, the first of these commands returns
|
|
||||||
ASCIIbetical order,
|
|
||||||
|
|
||||||
! e
|
|
||||||
- d
|
|
||||||
? a
|
|
||||||
@ b
|
|
||||||
~ c
|
|
||||||
|
|
||||||
and the second returns dictionary order.
|
|
||||||
|
|
||||||
? a
|
|
||||||
@ b
|
|
||||||
~ c
|
|
||||||
- d
|
|
||||||
! e
|
|
||||||
|
|
||||||
With GNU coreutils version 8.24 on NixOS, both commands return
|
|
||||||
dictionary order. The same is true for GNU coreutils version 8.23 on
|
|
||||||
Debian Wheezy.
|
|
||||||
|
|
||||||
? a
|
|
||||||
@ b
|
|
||||||
~ c
|
|
||||||
- d
|
|
||||||
! e
|
|
||||||
|
|
||||||
IEEE Std 1003.1, 2013 Edition [^^] specifies that the "-d" flag should
|
|
||||||
enable dictionary order. All of these versions of sort have clear
|
|
||||||
documentation about the order that should be returned when the "-d" flag
|
|
||||||
is set, (See --help, man, or info.) and the implementations match the
|
|
||||||
documentation as far as I can tell.
|
|
||||||
|
|
||||||
I have found no explicit documentation from any relevant source as to
|
|
||||||
what the default sort order should be. On the other hand, they all
|
|
||||||
suggest that "-d" produces an order different from the default order.
|
|
||||||
|
|
||||||
In GNU coreutils 8.24, for example, "-d" is a direction to "consider
|
|
||||||
only blanks and alphanumeric characters". It lacks any mention that the
|
|
||||||
"-d" flag has no effect or that it is the default. Furthermore, on my
|
|
||||||
first reading, I took it to mean that the default is to consider all
|
|
||||||
characters and that "-d" limits the considered characters to blanks and
|
|
||||||
alphanumeric characters.
|
|
||||||
|
|
||||||
|
|
||||||
Sorting in *
|
|
||||||
-------------
|
|
||||||
I think this is related to the order returned by "*" in sh.
|
|
||||||
The following sh code creates several files in a directory and then
|
The following sh code creates several files in a directory and then
|
||||||
calls "*", listing them in order.
|
calls "*", listing them in order.
|
||||||
|
|
||||||
@@ -70,8 +8,7 @@ calls "*", listing them in order.
|
|||||||
done
|
done
|
||||||
for file in *; do echo "$file"; done
|
for file in *; do echo "$file"; done
|
||||||
|
|
||||||
On one computer, running FreeBSD, the order is apparently
|
On one computer, running FreeBSD, the order is apparently ASCIIbetical.
|
||||||
ASCIIbetical.
|
|
||||||
|
|
||||||
! e
|
! e
|
||||||
- d
|
- d
|
||||||
@@ -79,7 +16,7 @@ ASCIIbetical.
|
|||||||
@ b
|
@ b
|
||||||
~ c
|
~ c
|
||||||
|
|
||||||
On two GNU systems, running NixOS and Debian, respectively, output is
|
On another computer, running NixOS, the following commands print results
|
||||||
in dictionary order. I'm not exactly sure what dictionary order is, but
|
in dictionary order. I'm not exactly sure what dictionary order is, but
|
||||||
it is something like sorting on the alphabetical characters before
|
it is something like sorting on the alphabetical characters before
|
||||||
sorting on the rest of the line.
|
sorting on the rest of the line.
|
||||||
@@ -90,9 +27,16 @@ sorting on the rest of the line.
|
|||||||
- d
|
- d
|
||||||
! e
|
! e
|
||||||
|
|
||||||
(I don't really know what dictionary order is, I was able to determine
|
While I don't really know what dictionary order is, I was able to determine
|
||||||
that the above results are in dictionary order because of my investigation of
|
that the above results are in dictionary order because of my investigation of
|
||||||
incompatible implementations of sort.)
|
incompatible implementations of sort. Consider the following two sort
|
||||||
|
commands.
|
||||||
|
|
||||||
[^] https://www.gnu.org/software/coreutils/faq/coreutils-faq.html#Sort-does-not-sort-in-normal-order_0021
|
printf '@ b\n- d\n? a\n~ c\n! e\n' | sort
|
||||||
[^^] http://pubs.opengroup.org/onlinepubs/9699919799/
|
printf '@ b\n- d\n? a\n~ c\n! e\n' | sort -d
|
||||||
|
|
||||||
|
With BSD sort, the first of these commands print ASCIIbetical order and
|
||||||
|
the second prints dictionary order. With GNU sort, both print dictionary
|
||||||
|
order.
|
||||||
|
|
||||||
|
How annoying.
|
||||||
|
|||||||
238
TODO
238
TODO
@@ -26,14 +26,11 @@ Windows
|
|||||||
Try running Urchin in Windows somehow. Interpreters include
|
Try running Urchin in Windows somehow. Interpreters include
|
||||||
|
|
||||||
* CygWin (https://www.cygwin.com/)
|
* CygWin (https://www.cygwin.com/)
|
||||||
* https://cygwin.com/setup-x86.exe
|
|
||||||
* MSYS (http://mingw.org/wiki/msys)
|
* MSYS (http://mingw.org/wiki/msys)
|
||||||
* GNU on Windows (https://github.com/bmatzelle/gow/wiki)
|
* GNU on Windows (https://github.com/bmatzelle/gow/wiki)
|
||||||
* Git for Windows (https://git-scm.com/download/win)
|
* Git for Windows (https://git-scm.com/download/win)
|
||||||
* https://github.com/git-for-windows/git/releases/download/v2.7.2.windows.1/Git-2.7.2-32-bit.exe
|
|
||||||
* win-bash (http://win-bash.sourceforge.net/)
|
* win-bash (http://win-bash.sourceforge.net/)
|
||||||
|
|
||||||
|
|
||||||
shall
|
shall
|
||||||
----------
|
----------
|
||||||
Add shall to my NYC*BUG talk.
|
Add shall to my NYC*BUG talk.
|
||||||
@@ -49,232 +46,15 @@ List some shell linters somewhere.
|
|||||||
* checkbashisms
|
* checkbashisms
|
||||||
|
|
||||||
|
|
||||||
|
Set parallel and series
|
||||||
|
----------
|
||||||
|
I want to be able to make only some directories run in series.
|
||||||
|
|
||||||
|
* Rename .urchin to .urchin_root.
|
||||||
|
* Look for a .urchin_dir file.
|
||||||
|
* If .urchin_dir contains "series", run that directory in series rather
|
||||||
|
than in parallel.
|
||||||
|
|
||||||
Rename to something other than "test"?
|
Rename to something other than "test"?
|
||||||
----------
|
----------
|
||||||
Maybe wait until I have a use for this.
|
Maybe wait until I have a use for this.
|
||||||
|
|
||||||
More sort alternatives
|
|
||||||
-----------
|
|
||||||
awk
|
|
||||||
https://stackoverflow.com/questions/20250937/sorting-lines-in-a-file-alphabetically-using-awk-and-or-sed
|
|
||||||
bash
|
|
||||||
https://stackoverflow.com/questions/7442417/how-to-sort-an-array-in-bash
|
|
||||||
|
|
||||||
Alternatives
|
|
||||||
--------------
|
|
||||||
JSON.sh test suite
|
|
||||||
|
|
||||||
Running in multiple environments
|
|
||||||
-----------------------------------
|
|
||||||
Setup for other environments includes the following.
|
|
||||||
|
|
||||||
* Installing packages
|
|
||||||
* `touch .zshrc`
|
|
||||||
* Copy urchin and tests
|
|
||||||
|
|
||||||
|
|
||||||
Fixtures
|
|
||||||
------------
|
|
||||||
I want to change the way that fixtures are done.
|
|
||||||
|
|
||||||
Instead of using setup, teardown, &c., use ordinary programs from within
|
|
||||||
your tests. For example.
|
|
||||||
|
|
||||||
# tests/.fixtures/tmp-dir
|
|
||||||
tmp=$(mktemp -d)
|
|
||||||
cd $tmp
|
|
||||||
@$
|
|
||||||
code=$?
|
|
||||||
cd /
|
|
||||||
rm -Rf $tmp
|
|
||||||
exit $code
|
|
||||||
|
|
||||||
# tests/blah
|
|
||||||
../.fixtures/tmp-dir 'blah blah blah'
|
|
||||||
|
|
||||||
It's best if I can wrap a bunch of commands in braces or paratheses
|
|
||||||
rather than just one command. Is there a nice way to do that?
|
|
||||||
|
|
||||||
Once I have this new way, I guess I might as well keep the old way.
|
|
||||||
I think the setup, teardown thing can be easier if you only have simple
|
|
||||||
fixtures. And since I'm going to keep it, I'm going to add another one.
|
|
||||||
|
|
||||||
* setup_dir runs once for the present directory.
|
|
||||||
* setup_children runs once for each child.
|
|
||||||
* setup_file runs once for each file descendent.
|
|
||||||
|
|
||||||
The present `setup` is renamed to `setup_children`, and the new
|
|
||||||
`setup_file` runs on each file (not directory) that is a child,
|
|
||||||
grandchild, great-grandchild, and so on.
|
|
||||||
|
|
||||||
Dependency checking
|
|
||||||
----------------------
|
|
||||||
You might want to skip tests based on dependencies. Currently you can
|
|
||||||
conditionally skip tests one at a time by exiting with code 3. I want to
|
|
||||||
be able to skip an entire directory.
|
|
||||||
|
|
||||||
So we add a new magic file called `dep`. If it exists, it is run before
|
|
||||||
everything else in the directory.
|
|
||||||
|
|
||||||
* If it exits with code 0, tests continue as if dep did not exist.
|
|
||||||
* If it exits with code 3, all tests in the directory are marked as
|
|
||||||
skipped.
|
|
||||||
* If it exits with code 1, all tests in the directory are marked as
|
|
||||||
failed. To make the implementation easier, I'll probably treat the
|
|
||||||
directory as a single test in this case.
|
|
||||||
|
|
||||||
A note on magic files
|
|
||||||
-------------------------
|
|
||||||
It is nice to have access to things like setup and dep (magic files)
|
|
||||||
once in a while, but you need to be doing rather substantial testing
|
|
||||||
before they make your test suite simpler; the documentation should
|
|
||||||
strongly recommend writing your tests without magic files and then
|
|
||||||
refactoring and only then considering moving things to magic files.
|
|
||||||
|
|
||||||
Remote testing
|
|
||||||
----------------
|
|
||||||
In order to test Urchin across multiple operating systems, I have
|
|
||||||
already added tests in Urchin's test suite that run Urchin tests in
|
|
||||||
remote servers. I would like to move this to Urchin itself so that
|
|
||||||
Urchin can test other things on remote servers.
|
|
||||||
|
|
||||||
Urchin's output presently looks like this.
|
|
||||||
|
|
||||||
Cycling with the following shells: sh bash dash mksh zsh
|
|
||||||
Running tests at 2016-04-07T12:33:49
|
|
||||||
|
|
||||||
Flags/
|
|
||||||
> --timeout output
|
|
||||||
. bash (0 seconds)
|
|
||||||
. dash (0 seconds)
|
|
||||||
. mksh (0 seconds)
|
|
||||||
. sh (0 seconds)
|
|
||||||
. zsh (0 seconds)
|
|
||||||
|
|
||||||
Done, took 1 second.
|
|
||||||
5 tests passed.
|
|
||||||
0 tests skipped.
|
|
||||||
0 tests failed.
|
|
||||||
|
|
||||||
After the change, the output should look like this.
|
|
||||||
|
|
||||||
Cycling with the following shells: sh dash mksh
|
|
||||||
Running tests at 2016-04-07T12:33:49
|
|
||||||
|
|
||||||
Flags/
|
|
||||||
> --timeout output
|
|
||||||
. dash on localhost (0 seconds)
|
|
||||||
. dash on localhost:8080 (0 seconds)
|
|
||||||
. dash on tlevine@hpux.polarhome.com (0 seconds)
|
|
||||||
. mksh on localhost (0 seconds)
|
|
||||||
. mksh on tlevine@hpux.polarhome.com (0 seconds)
|
|
||||||
. sh on localhost (0 seconds)
|
|
||||||
. sh on localhost:8080 (0 seconds)
|
|
||||||
. sh on tlevine@hpux.polarhome.com (0 seconds)
|
|
||||||
|
|
||||||
Done, took 1 second.
|
|
||||||
8 tests passed.
|
|
||||||
0 tests skipped.
|
|
||||||
0 tests failed.
|
|
||||||
|
|
||||||
This is just how the output should look; the tests run in whatever order
|
|
||||||
makes sense.
|
|
||||||
|
|
||||||
Bugs
|
|
||||||
-------
|
|
||||||
|
|
||||||
Both md5sum and md5 should be supported.
|
|
||||||
|
|
||||||
Trouble logging in to hpux, irix, miros, netbsd, tru64, qnx, ....
|
|
||||||
|
|
||||||
$ rsync -e 'ssh -p 785' urchin tlevine@hpux.polarhome.com:.blah
|
|
||||||
HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
|
|
||||||
Welcome to HPUX/PA... member of polarhome.com realm
|
|
||||||
|
|
||||||
bash: rsync: command not found
|
|
||||||
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
|
|
||||||
rsync error: remote command not found (code 127) at io.c(226)
|
|
||||||
[sender=3.1.1]
|
|
||||||
|
|
||||||
|
|
||||||
OpenIndiana grep does not support -q
|
|
||||||
|
|
||||||
I get `/urchin: syntax error at line 84: \`}' unexpected` on
|
|
||||||
unixware and solaris.
|
|
||||||
|
|
||||||
mktemp
|
|
||||||
|
|
||||||
> tlevine@hpux.polarhome.com -p 785
|
|
||||||
F sh (8 seconds)
|
|
||||||
|
|
|
||||||
| HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
|
|
||||||
| Welcome to HPUX/PA... member of polarhome.com realm
|
|
||||||
|
|
|
||||||
|
|
|
||||||
| HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
|
|
||||||
| Welcome to HPUX/PA... member of polarhome.com realm
|
|
||||||
|
|
|
||||||
| mktemp: option requires an argument -- d
|
|
||||||
| ./urchin[96]: /tmp/tlevinea21441/log: Cannot create the specified file.
|
|
||||||
|
|
||||||
date
|
|
||||||
|
|
||||||
tlevine@hpux64$ ./urchin tests/ -n -vv
|
|
||||||
date: bad format character - s
|
|
||||||
|
|
||||||
So I need a portable seconds-from epoch
|
|
||||||
|
|
||||||
I also need to handle when no arguments are passed to urchin.
|
|
||||||
|
|
||||||
Exit code is wrong for which on HP-UX
|
|
||||||
|
|
||||||
## `$(...)`
|
|
||||||
Solaris doesn't support `$(...)`; you need `\`...\`` instead.
|
|
||||||
|
|
||||||
tlevine@solaris$ ./urchin --run-in-series tests/Errors/
|
|
||||||
./urchin: syntax error at line 84: `tmp=$' unexpected
|
|
||||||
|
|
||||||
I use this a lot.
|
|
||||||
|
|
||||||
$ grep -c '\$(' urchin
|
|
||||||
52
|
|
||||||
|
|
||||||
Darn
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Update tests to support
|
|
||||||
|
|
||||||
* md5
|
|
||||||
* rsync
|
|
||||||
* mktemp
|
|
||||||
* epoch
|
|
||||||
* Report cycling by default
|
|
||||||
* New format for reporting cycling
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Support systems without rsync
|
|
||||||
|
|
||||||
|
|
||||||
BSD mktemp
|
|
||||||
|
|
||||||
| NetBSD 6.1.3
|
|
||||||
| Welcome to NetBSD ...member of polarhome.com realm
|
|
||||||
|
|
|
||||||
| Usage: mktemp [-dqu] [-p <tmpdir>] {-t prefix | template ...}
|
|
||||||
| mkdir: : No such file or directory
|
|
||||||
| ./urchin: cannot create /log: permission denied
|
|
||||||
|
|
||||||
|
|
||||||
NetBSD
|
|
||||||
|
|
||||||
md5: unknown option -- q
|
|
||||||
usage: cksum [-n] [-a algorithm [-ptx] [-s string]] [-o 1|2]
|
|
||||||
[file ... | -c [-w] [sumfile]]
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,14 +0,0 @@
|
|||||||
params="$(basename "${0}")"
|
|
||||||
|
|
||||||
hostname="$(echo "${params}" | cut -d\ -f1)"
|
|
||||||
if echo "${params}" | grep -q \ ; then
|
|
||||||
flags="$(echo "${params}" | cut -d\ -f2-)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
urchin_dir=.urchin-cross-shell-test
|
|
||||||
|
|
||||||
rsync --archive -e "ssh ${flags}" $RSYNC_FLAGS \
|
|
||||||
../urchin ../tests "${hostname}":"${urchin_dir}" ||
|
|
||||||
scp -r ${flags} ../urchin ../tests "${hostname}":"${urchin_dir}"
|
|
||||||
ssh "${hostname}" ${flags} \
|
|
||||||
"cd ${urchin_dir} && ./urchin --run-in-series tests"
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
# apt-get install bash dash ksh posh pdksh mksh yash zsh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
RSYNC_FLAGS='--rsync-path=/usr/local/bin/rsync'
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
# SSH public key needs to be in ~/.etc/ssh/authorized_keys
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
. ./.run
|
|
||||||
1
packages/.gitignore
vendored
1
packages/.gitignore
vendored
@@ -1 +0,0 @@
|
|||||||
*.tar.gz
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
name=urchin-$(../urchin --version)
|
|
||||||
|
|
||||||
tmp=$(mktemp -d)
|
|
||||||
mkdir $tmp/$name
|
|
||||||
cp ../urchin ../readme.md ../AUTHORS ../COPYING $tmp/$name
|
|
||||||
cd $tmp
|
|
||||||
tar czf $name.tar.gz $name
|
|
||||||
cd - > /dev/null
|
|
||||||
mv $tmp/$name.tar.gz .
|
|
||||||
rm -R $tmp
|
|
||||||
55
readme.md
55
readme.md
@@ -43,15 +43,25 @@ Urchin depends on the following programs.
|
|||||||
* timeout
|
* timeout
|
||||||
* sort
|
* sort
|
||||||
|
|
||||||
Vanilla installations of modern BSD and GNU systems usually include all
|
All of the above programs are usually included on base BSD installations.
|
||||||
of these programs.
|
On GNU systems it should be sufficient to install the busybox package.
|
||||||
|
|
||||||
|
Urchin uses sort to format its output. GNU sort (as of GNU coreutils version
|
||||||
|
8.24) lacks the ability to sort in lexicographic order, and this feature is
|
||||||
|
necessary for the output to look right. If your version of sort lacks this
|
||||||
|
feature, Urchin will try to use one of the following tools for sorting.
|
||||||
|
|
||||||
|
If no acceptable sorting program is available, Urchin will print a warning
|
||||||
|
and use the incomplete sort that is installed on your system. This is not a
|
||||||
|
big deal; if your test files all start with alphanumeric letters, the output
|
||||||
|
should look fine.
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
Urchin is contained in a single file, so you can install it by copying it to a
|
Urchin is contained in a single file, so you can install it by copying it to a
|
||||||
directory in your `PATH`. For example, you can run the following as root.
|
directory in your `PATH`. For example, you can run the following as root.
|
||||||
|
|
||||||
cd /usr/local/bin
|
cd /usr/local/bin
|
||||||
wget https://raw.githubusercontent.com/tlevine/urchin/v0.1.0-rc3/urchin
|
wget https://raw.githubusercontent.com/tlevine/urchin/v0.0.6/urchin
|
||||||
chmod +x urchin
|
chmod +x urchin
|
||||||
|
|
||||||
Urchin can be installed with npm too.
|
Urchin can be installed with npm too.
|
||||||
@@ -108,36 +118,14 @@ Files are only run if they are executable, and files beginning with `.` are
|
|||||||
ignored. Thus, fixtures and libraries can be included sloppily within the test
|
ignored. Thus, fixtures and libraries can be included sloppily within the test
|
||||||
directory tree. The test passes if the file exits 0; otherwise, it fails.
|
directory tree. The test passes if the file exits 0; otherwise, it fails.
|
||||||
|
|
||||||
urchin looks for files within a directory in the following manner,
|
Tests files and subdirectories are run in ASCIIbetical order within each
|
||||||
|
directory; that is,
|
||||||
|
urchin looks for files within a directory in the following manner.
|
||||||
|
|
||||||
for file in *; do
|
for file in *; do
|
||||||
do_something_with_test_file $file
|
do_something_with_test_file $file
|
||||||
done
|
done
|
||||||
|
|
||||||
so files are run in whatever order `*` produces. The order is
|
|
||||||
configured in your environment, at least in
|
|
||||||
[GNU systems](https://www.gnu.org/software/coreutils/faq/coreutils-faq.html#Sort-does-not-sort-in-normal-order_0021).
|
|
||||||
Other systems may ignore the locales configured in the environment and
|
|
||||||
always produce ASCIIbetical order.
|
|
||||||
|
|
||||||
Results are always printed in ASCIIbetical order, regardless of what
|
|
||||||
order the tests ran in.
|
|
||||||
|
|
||||||
Below you can see how the locale can affect the order.
|
|
||||||
|
|
||||||
$ printf '!c\n@a\n~b\n' | LC_COLLATE=C sort
|
|
||||||
!c
|
|
||||||
@a
|
|
||||||
~b
|
|
||||||
$ printf '!c\n@a\n~b\n' | LC_COLLATE=en_US.UTF-8 sort
|
|
||||||
@a
|
|
||||||
~b
|
|
||||||
!c
|
|
||||||
$ printf '!c\n@a\n~b\n' | sort -d
|
|
||||||
@a
|
|
||||||
~b
|
|
||||||
!c
|
|
||||||
|
|
||||||
### Writing cross-shell compatibility tests for testing shell code
|
### Writing cross-shell compatibility tests for testing shell code
|
||||||
|
|
||||||
While you could write your test scripts to explicitly invoke the functionality
|
While you could write your test scripts to explicitly invoke the functionality
|
||||||
@@ -191,9 +179,14 @@ shell.)
|
|||||||
|
|
||||||
Note that only test scripts that either have no shebang line at all or
|
Note that only test scripts that either have no shebang line at all or
|
||||||
have shebang line `#!/bin/sh` are invoked with the specified shell.
|
have shebang line `#!/bin/sh` are invoked with the specified shell.
|
||||||
This allows non-shell test scripts or test scripts for other languages
|
This allows non-shell test scripts or test scripts for specific
|
||||||
or for specific shells to coexist with those whose invocation should be
|
shells to coexist with those whose invocation should be controlled by `-s`.
|
||||||
controlled by `-s`.
|
|
||||||
|
To test with multiple shells in sequence, use something like:
|
||||||
|
|
||||||
|
for shell in sh bash ksh zsh; do
|
||||||
|
urchin -s $shell ./tests
|
||||||
|
done
|
||||||
|
|
||||||
## References
|
## References
|
||||||
|
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
series
|
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
# This script should run with /bin/sh
|
||||||
|
# regardless of whether -s or -n is passed.
|
||||||
|
|
||||||
|
ps -o pid,comm,args | grep $$ | grep /bin/sh
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
|
|
||||||
# Assuming that urchin was invoked with `-s bash`,
|
|
||||||
# this script should be being run with bash.
|
|
||||||
|
|
||||||
ps -o pid,comm,args | grep $$ | grep .special-shell
|
|
||||||
@@ -1 +1 @@
|
|||||||
! $TEST_SHELL ../../urchin ../Flags/Urchin\ format|grep -- --pretty
|
! $TEST_SHELL ../../urchin ../Flags/Urchin\ format|grep -- --color
|
||||||
|
|||||||
@@ -1 +1,3 @@
|
|||||||
|
set -e
|
||||||
|
$TEST_SHELL ../../urchin --shell sh .slow-tests
|
||||||
! $TEST_SHELL ../../urchin --shell sh --timeout 0.3 .slow-tests
|
! $TEST_SHELL ../../urchin --shell sh --timeout 0.3 .slow-tests
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
../../urchin -T aoeu .testsuite 2>&1 | grep Bad
|
|
||||||
../../urchin -T .testsuite 2>&1 | grep Bad
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
$TEST_SHELL ../../urchin --shell sh --timeout 0.3 .slow-tests 2>&1 |
|
|
||||||
grep -v -- --timeout |
|
|
||||||
grep timeout
|
|
||||||
test $? = 1
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
$TEST_SHELL ../../urchin --shell sh .slow-tests --timeout 1000
|
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -e
|
set -e
|
||||||
|
|
||||||
! $TEST_SHELL ../../urchin -vv --run-in-series --exit-on-fail \
|
! $TEST_SHELL ../../urchin --run-in-series --exit-on-fail \
|
||||||
./.test_-e,--exit-on-fail > $tmp
|
./.test_-e,--exit-on-fail > $tmp
|
||||||
|
|
||||||
grep '1 should run.' $tmp
|
grep '1 should run.' $tmp
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
|
|
||||||
./
|
./
|
||||||
> a
|
> a
|
||||||
F sh (1 second)
|
✗ sh (1 second)
|
||||||
| This is stdout from a.
|
# This is stdout from a.
|
||||||
./
|
./
|
||||||
> b
|
> b
|
||||||
. sh (1 second)
|
✓ sh (1 second)
|
||||||
./
|
./
|
||||||
> c
|
> c
|
||||||
(File is not executable.)
|
(File is not executable.)
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
./
|
./
|
||||||
> a
|
> a
|
||||||
[31m✗ [0msh (1 second)
|
[31m✗ [0msh (1 second)
|
||||||
| This is stdout from a.
|
# This is stdout from a.
|
||||||
./
|
./
|
||||||
> b
|
> b
|
||||||
[32m✓ [0msh (1 second)
|
[32m✓ [0msh (1 second)
|
||||||
|
|||||||
@@ -1,3 +1,3 @@
|
|||||||
$TEST_SHELL ../../urchin -v -s sh -t .testsuite/ |
|
$TEST_SHELL ../../urchin -s sh -t .testsuite/ |
|
||||||
sed -e 1,2\ d -e /second/d > $tmp
|
sed -e 1d -e /second/d > $tmp
|
||||||
diff $tmp .tap-output-expectation
|
diff $tmp .tap-output-expectation
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
lines=$(
|
|
||||||
$TEST_SHELL ../../urchin -v -s sh -t .testsuite/ |
|
|
||||||
tee $tmp | grep -v '^#' | wc -l)
|
|
||||||
cat $tmp
|
|
||||||
test $lines -eq 4
|
|
||||||
@@ -1,3 +1,3 @@
|
|||||||
$TEST_SHELL ../../urchin -vv -s sh .testsuite/ |
|
$TEST_SHELL ../../urchin -vv -s sh .testsuite/ |
|
||||||
sed -e 1,2\ d -e 's/. seconds\?/1 second/' > $tmp
|
sed -e 1d -e 's/. seconds\?/1 second/' > $tmp
|
||||||
diff $tmp .urchin-output-expectation
|
diff $tmp .urchin-output-expectation
|
||||||
|
|||||||
@@ -1,3 +1,3 @@
|
|||||||
$TEST_SHELL ../../urchin --pretty -vv --shell sh .testsuite/ |
|
$TEST_SHELL ../../urchin --color -vv --shell sh .testsuite/ |
|
||||||
sed -e 1,2\ d -e 's/. seconds\?/1 second/' > $tmp
|
sed -e 1d -e 's/. seconds\?/1 second/' > $tmp
|
||||||
diff $tmp .urchin-output-expectation-color
|
diff $tmp .urchin-output-expectation-color
|
||||||
|
|||||||
@@ -1,3 +0,0 @@
|
|||||||
echo "$1" > $tmp
|
|
||||||
TESTING_URCHIN_INTERNALS=true . ../../../urchin
|
|
||||||
has_shebang_line $tmp
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
./.run '#!/bin/bash'
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
! ./.run ''
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
! ./.run '
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
'
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
./.run '#!/usr/bin/env true'
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
export tmp=$(mktemp)
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
! ./.run '#!/bin/sh'
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
rm -R "$tmp"
|
|
||||||
4
tests/Internals/sort_python
Executable file
4
tests/Internals/sort_python
Executable file
@@ -0,0 +1,4 @@
|
|||||||
|
set -e
|
||||||
|
TESTING_URCHIN_INTERNALS=true . ../../urchin
|
||||||
|
unsorted='@ b\n- d\n? a\n~ c\n! e\n'
|
||||||
|
test $(printf "${unsorted}" | sort_python | cut -d\ -f2|tr -d '\n') = edabc
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
|
|
||||||
[ $(grep -c 'setup has run' $log) -eq '1' ]
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
|
|
||||||
[ $(grep -c 'setup has run' $log) -eq '2' ]
|
|
||||||
3
tests/Setup and Teardown/.test/setup has run twice a
Executable file
3
tests/Setup and Teardown/.test/setup has run twice a
Executable file
@@ -0,0 +1,3 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
[ $(grep -c 'setup has run' $log) -gt '2' ]
|
||||||
3
tests/Setup and Teardown/.test/setup has run twice b
Executable file
3
tests/Setup and Teardown/.test/setup has run twice b
Executable file
@@ -0,0 +1,3 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
[ $(grep -c 'setup has run' $log) -gt '2' ]
|
||||||
343
urchin
343
urchin
@@ -51,52 +51,13 @@ set -e
|
|||||||
# Kill subprocesses on interrupt.
|
# Kill subprocesses on interrupt.
|
||||||
trap "kill -$$; exit" HUP INT TERM
|
trap "kill -$$; exit" HUP INT TERM
|
||||||
|
|
||||||
DEFAULT_SHELLS='
|
DEFAULT_SHELLS='sh bash dash mksh zsh'
|
||||||
sh
|
|
||||||
bash
|
|
||||||
dash
|
|
||||||
ksh
|
|
||||||
posh
|
|
||||||
pdksh
|
|
||||||
mksh
|
|
||||||
yash
|
|
||||||
zsh
|
|
||||||
'
|
|
||||||
if [ -n "${ZSH_VERSION}" ]; then
|
if [ -n "${ZSH_VERSION}" ]; then
|
||||||
# avoid "no matches found: *" error when directories are empty
|
# avoid "no matches found: *" error when directories are empty
|
||||||
setopt NULL_GLOB
|
setopt NULL_GLOB
|
||||||
emulate sh
|
emulate sh
|
||||||
fi
|
fi
|
||||||
|
|
||||||
epoch_date() {
|
|
||||||
date +%s
|
|
||||||
}
|
|
||||||
epoch_pax() {
|
|
||||||
# Based on http://stackoverflow.com/a/7262588/407226
|
|
||||||
tmp="$(mktemp_file)"
|
|
||||||
echo "ibase=8;$({ pax -wx cpio "${tmp}"; echo; } | cut -c 48-59)" | bc
|
|
||||||
rm "${tmp}"
|
|
||||||
}
|
|
||||||
|
|
||||||
mktemp_dir() {
|
|
||||||
# Support HP-UX mktemp that has wrong exit codes and
|
|
||||||
# can't make directories.
|
|
||||||
tmp=$(mktemp /tmp/urchin.XXXXXXXX)
|
|
||||||
if test -f "${tmp}"; then
|
|
||||||
rm "${tmp}"
|
|
||||||
fi
|
|
||||||
mkdir "${tmp}"
|
|
||||||
echo "${tmp}"
|
|
||||||
}
|
|
||||||
mktemp_file() {
|
|
||||||
tmp=$(mktemp /tmp/urchin.XXXXXXXX)
|
|
||||||
if ! test -f "${tmp}"; then
|
|
||||||
> "${tmp}"
|
|
||||||
fi
|
|
||||||
echo "${tmp}"
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
validate_test_arg() {
|
validate_test_arg() {
|
||||||
# Must be a file or directory
|
# Must be a file or directory
|
||||||
if [ ! -e "${1}" ]; then
|
if [ ! -e "${1}" ]; then
|
||||||
@@ -109,7 +70,7 @@ validate_test_arg() {
|
|||||||
root="$(urchin_root "${1}")"
|
root="$(urchin_root "${1}")"
|
||||||
if ! {
|
if ! {
|
||||||
basename "$(fullpath "${root}")" |
|
basename "$(fullpath "${root}")" |
|
||||||
grep -qi 'test' || "${force}"
|
grep -i 'test' > /dev/null || "${force}"
|
||||||
}; then
|
}; then
|
||||||
echo 'The root directory of the tests that you are running urchin on
|
echo 'The root directory of the tests that you are running urchin on
|
||||||
does not contain the word "test", so I am not running,
|
does not contain the word "test", so I am not running,
|
||||||
@@ -119,54 +80,28 @@ validate_test_arg() {
|
|||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
# All temporary files go here
|
sort_python() {
|
||||||
urchin_tmp=$(mktemp_dir)
|
python -c 'import sys
|
||||||
> "${urchin_tmp}/log"
|
for line in sorted(sys.stdin.readlines()):
|
||||||
|
sys.stdout.write(line)
|
||||||
|
'
|
||||||
|
}
|
||||||
|
|
||||||
|
# All temporary files go here
|
||||||
|
urchin_tmp=$(mktemp -d)
|
||||||
|
> "${urchin_tmp}/log"
|
||||||
urchin_exit() {
|
urchin_exit() {
|
||||||
rm -Rf "${urchin_tmp}"
|
rm -Rf "${urchin_tmp}"
|
||||||
exit "$@"
|
exit "$@"
|
||||||
}
|
}
|
||||||
|
|
||||||
if which md5 1> /dev/null 2> /dev/null; then
|
|
||||||
urchin_md5=md5
|
|
||||||
elif which md5sum 1> /dev/null 2> /dev/null; then
|
|
||||||
urchin_md5=md5sum
|
|
||||||
else
|
|
||||||
echo Could not find MD5 hash command >&2
|
|
||||||
urchin_exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
if epoch_date 2>&1 > /dev/null; then
|
|
||||||
epoch=epoch_date
|
|
||||||
elif epoch_pax 2>&1 > /dev/null; then
|
|
||||||
epoch=epoch_pax
|
|
||||||
else
|
|
||||||
echo I could not find a seconds counter. >&2
|
|
||||||
urchin_exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
stdout_file() {
|
stdout_file() {
|
||||||
the_test="${1}"
|
the_test="${1}"
|
||||||
the_shell="${2}"
|
the_shell="${2}"
|
||||||
host="${3}"
|
|
||||||
|
|
||||||
if test -n "${host}"; then
|
x="${urchin_tmp}/stdout$(fullpath "$the_test")"
|
||||||
# This assumes the tests ran on a remote and have been copied.
|
mkdir -p "${x}"
|
||||||
x="${urchin_tmp}/remote/${host}/stdout${the_test}"
|
echo "${x}/$(echo "${the_shell}" | md5sum | cut -d\ -f1)"
|
||||||
else
|
|
||||||
# This can be run during the tests.
|
|
||||||
x="${urchin_tmp}/stdout$(fullpath "$the_test")"
|
|
||||||
mkdir -p "${x}"
|
|
||||||
fi
|
|
||||||
|
|
||||||
case "${urchin_md5}" in
|
|
||||||
md5sum) y=$(echo "${the_shell}" | md5sum | cut -d\ -f1) ;;
|
|
||||||
md5) y=$(echo "${the_shell}" | md5 | sed 's/.* //') ;;
|
|
||||||
*) echo md5 command is not configured >&2; urchin_exit 1;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
echo "${x}/${y}"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# Expand relative paths
|
# Expand relative paths
|
||||||
@@ -189,7 +124,7 @@ urchin_root() {
|
|||||||
|
|
||||||
abscurrent="$(fullpath "${1}")"
|
abscurrent="$(fullpath "${1}")"
|
||||||
if test "${abscurrent}" = / ||
|
if test "${abscurrent}" = / ||
|
||||||
basename "${abscurrent}" | grep -q '^\.' ; then
|
basename "${abscurrent}" | grep '^\.' > /dev/null; then
|
||||||
# Stop traversing upwards at / and at hidden directories.
|
# Stop traversing upwards at / and at hidden directories.
|
||||||
if test -d "${orig}"; then
|
if test -d "${orig}"; then
|
||||||
echo "${orig}"
|
echo "${orig}"
|
||||||
@@ -201,7 +136,7 @@ urchin_root() {
|
|||||||
return 1
|
return 1
|
||||||
elif test -f "${current}"; then
|
elif test -f "${current}"; then
|
||||||
urchin_root "$(dirname -- "${current}")" "${orig}"
|
urchin_root "$(dirname -- "${current}")" "${orig}"
|
||||||
elif test -f "${current}"/.urchin_root; then
|
elif test -f "${current}"/.urchin; then
|
||||||
remove_trailing_slash "${current}"
|
remove_trailing_slash "${current}"
|
||||||
else
|
else
|
||||||
urchin_root "${current}"/.. "${orig}"
|
urchin_root "${current}"/.. "${orig}"
|
||||||
@@ -209,7 +144,7 @@ urchin_root() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
# Urchin version number
|
# Urchin version number
|
||||||
VERSION=0.1.1-unstable
|
VERSION=0.1.0-rc1
|
||||||
|
|
||||||
indent() {
|
indent() {
|
||||||
level="${1}"
|
level="${1}"
|
||||||
@@ -230,8 +165,8 @@ recurse() {
|
|||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
if echo "${requested_path}" | grep -q "^${potential_test}" ||
|
if echo "${requested_path}" | grep "^${potential_test}" > /dev/null ||
|
||||||
echo "${potential_test}" | grep -q "^${requested_path}" ; then
|
echo "${potential_test}" | grep "^${requested_path}" > /dev/null; then
|
||||||
if test "$(dirname "${potential_test}")" = \
|
if test "$(dirname "${potential_test}")" = \
|
||||||
"$(dirname "${requested_path}")" &&
|
"$(dirname "${requested_path}")" &&
|
||||||
test "${potential_test}" != "${requested_path}"; then
|
test "${potential_test}" != "${requested_path}"; then
|
||||||
@@ -247,15 +182,7 @@ recurse() {
|
|||||||
if [ -d "${potential_test}" ]; then
|
if [ -d "${potential_test}" ]; then
|
||||||
(
|
(
|
||||||
cd -- "${potential_test}"
|
cd -- "${potential_test}"
|
||||||
if test -f .urchin_dir && grep -q series .urchin_dir ; then
|
if test -f setup_dir; then . ./setup_dir; fi
|
||||||
run_in_series_dir=true
|
|
||||||
else
|
|
||||||
run_in_series_dir=false
|
|
||||||
fi
|
|
||||||
|
|
||||||
if test -f setup_dir; then
|
|
||||||
. ./setup_dir
|
|
||||||
fi
|
|
||||||
|
|
||||||
for test in *; do
|
for test in *; do
|
||||||
if test "${test}" = '*' && ! test -e "${test}"; then
|
if test "${test}" = '*' && ! test -e "${test}"; then
|
||||||
@@ -263,23 +190,28 @@ recurse() {
|
|||||||
break
|
break
|
||||||
fi
|
fi
|
||||||
|
|
||||||
recurse "${requested_path}" "${test}" "${cycle_shell}" \
|
(
|
||||||
"${TEST_SHELL}" &
|
if test -f setup; then . ./setup; fi
|
||||||
|
if recurse "${requested_path}" "${test}" "${cycle_shell}" \
|
||||||
|
"${TEST_SHELL}"; then
|
||||||
|
exit_code=0
|
||||||
|
else
|
||||||
|
exit_code="${?}"
|
||||||
|
fi
|
||||||
|
if test -f teardown; then . ./teardown; fi
|
||||||
|
exit "${exit_code}"
|
||||||
|
) &
|
||||||
|
|
||||||
if "${run_in_series}" || "${run_in_series_dir}"; then
|
if "${run_in_series}"; then
|
||||||
if wait "${!}"; then exit_code=0; else exit_code="${?}"; fi
|
if wait "${!}"; then exit_code=0; else exit_code="${?}"; fi
|
||||||
if "${exit_on_not_ok}" && test "${exit_code}" -ne 0; then
|
if "${exit_on_not_ok}" && test "${exit_code}" -ne 0; then
|
||||||
if test -f teardown_dir; then
|
if test -f teardown_dir; then . ./teardown_dir; fi
|
||||||
. ./teardown_dir
|
|
||||||
fi
|
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
wait
|
wait
|
||||||
if test -f teardown_dir; then
|
if test -f teardown_dir; then . ./teardown_dir; fi
|
||||||
. ./teardown_dir
|
|
||||||
fi
|
|
||||||
)
|
)
|
||||||
elif [ -f "${potential_test}" ]; then
|
elif [ -f "${potential_test}" ]; then
|
||||||
cd -- "$(dirname -- "${potential_test}")"
|
cd -- "$(dirname -- "${potential_test}")"
|
||||||
@@ -288,12 +220,10 @@ recurse() {
|
|||||||
# that reflects the specified or implied shell to use for shell-code tests.
|
# that reflects the specified or implied shell to use for shell-code tests.
|
||||||
while read the_test_shell; do
|
while read the_test_shell; do
|
||||||
(
|
(
|
||||||
if test -f setup; then
|
if test -f setup; then . ./setup; fi
|
||||||
. ./setup
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Run the test
|
# Run the test
|
||||||
start=$("${epoch}")
|
start=$(date +%s)
|
||||||
set +e
|
set +e
|
||||||
{
|
{
|
||||||
if "${cycle_shell}"; then
|
if "${cycle_shell}"; then
|
||||||
@@ -304,7 +234,7 @@ recurse() {
|
|||||||
"${the_test_shell}" "${potential_test}"
|
"${the_test_shell}" "${potential_test}"
|
||||||
fi
|
fi
|
||||||
else
|
else
|
||||||
# Shell cycling is disabled with -n; use the present value of
|
# Shell cycling is disabled with -d; use the present value of
|
||||||
# TEST_SHELL or default to /bin/sh
|
# TEST_SHELL or default to /bin/sh
|
||||||
if [ -n "${TEST_SHELL}" ]; then
|
if [ -n "${TEST_SHELL}" ]; then
|
||||||
$TIMEOUT "${potential_test}"
|
$TIMEOUT "${potential_test}"
|
||||||
@@ -315,11 +245,9 @@ recurse() {
|
|||||||
} > "$(stdout_file "${potential_test}" "${the_test_shell}")" 2>&1
|
} > "$(stdout_file "${potential_test}" "${the_test_shell}")" 2>&1
|
||||||
exit_code="${?}"
|
exit_code="${?}"
|
||||||
set -e
|
set -e
|
||||||
finish=$("${epoch}")
|
finish=$(date +%s)
|
||||||
|
|
||||||
if test -f teardown; then
|
if test -f teardown; then . ./setup; fi
|
||||||
. ./teardown
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ "${exit_code}" -eq 0 ]; then
|
if [ "${exit_code}" -eq 0 ]; then
|
||||||
result=ok
|
result=ok
|
||||||
@@ -338,16 +266,14 @@ recurse() {
|
|||||||
if "${run_in_series}"; then
|
if "${run_in_series}"; then
|
||||||
if wait "${!}"; then exit_code=0; else exit_code="${?}"; fi
|
if wait "${!}"; then exit_code=0; else exit_code="${?}"; fi
|
||||||
if "${exit_on_not_ok}" && test "${exit_code}" -ne 0; then
|
if "${exit_on_not_ok}" && test "${exit_code}" -ne 0; then
|
||||||
if test -f teardown_dir; then
|
if test -f teardown_dir; then . ./teardown_dir; fi
|
||||||
. ./teardown_dir
|
|
||||||
fi
|
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
done < "${shell_list}"
|
done < "${shell_list}"
|
||||||
wait
|
wait
|
||||||
else
|
else
|
||||||
echo "${potential_test}: Neither file nor directory!?" >&2
|
echo "${potential_test}: Neither file nor directory!?" > /dev/stderr
|
||||||
fi
|
fi
|
||||||
else
|
else
|
||||||
# Shell is ''
|
# Shell is ''
|
||||||
@@ -362,22 +288,15 @@ report_outcome() {
|
|||||||
start="${4}"
|
start="${4}"
|
||||||
finish="${5}"
|
finish="${5}"
|
||||||
|
|
||||||
host="${6}"
|
|
||||||
if test -n "${host}"; then
|
|
||||||
onhost=" on ${host}"
|
|
||||||
fi
|
|
||||||
|
|
||||||
escaped_root="$(fullpath "${root}" | sed 's/\//\\\//g')"
|
escaped_root="$(fullpath "${root}" | sed 's/\//\\\//g')"
|
||||||
elapsed=$(($finish - $start))
|
elapsed=$(($finish - $start))
|
||||||
|
|
||||||
if "${print_margins}" || "${tap_format}"; then
|
if "${tap_format}"; then
|
||||||
if $tap_format; then printf \#\ ; fi
|
printf \#\
|
||||||
echo Ran tests at $(date +%Y-%m-%dT%H:%M:%S) with the following shells:
|
fi
|
||||||
if $tap_format; then printf \#\ ; fi
|
if "${print_margins}" || "${tap_format}"; then
|
||||||
cat "${shell_list}" | tr '\n' \
|
echo Running tests at $(date +%Y-%m-%dT%H:%M:%S)
|
||||||
echo
|
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
|
||||||
for number in n oks skips not_oks; do
|
for number in n oks skips not_oks; do
|
||||||
eval "${number}=0"
|
eval "${number}=0"
|
||||||
@@ -385,8 +304,8 @@ report_outcome() {
|
|||||||
|
|
||||||
# Use a temporary file rather than a pipe because a pipe starts a sub-shell
|
# Use a temporary file rather than a pipe because a pipe starts a sub-shell
|
||||||
# and thus makes the above variables local.
|
# and thus makes the above variables local.
|
||||||
sorted_log_file=$(mktemp_file)
|
sorted_log_file=$(mktemp)
|
||||||
cat "${log_file}" | LC_COLLATE=C sort > "${sorted_log_file}"
|
cat "${log_file}" | "${sort}" > "${sorted_log_file}"
|
||||||
|
|
||||||
while read line; do
|
while read line; do
|
||||||
abspath=$(echo "${line}" | cut -f1)
|
abspath=$(echo "${line}" | cut -f1)
|
||||||
@@ -418,13 +337,13 @@ report_outcome() {
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
if test -z "${the_shell}"; then
|
if test -z "${the_shell}"; then
|
||||||
the_shell='File is not executable'
|
the_shell='File is not executable.'
|
||||||
fi
|
fi
|
||||||
echo "${not}ok $n - ${path} (${the_shell}${onhost}) ${skip}"
|
echo "${not}ok $n - ${path} (${the_shell}) ${skip}"
|
||||||
if { test "${result}" = not_ok && "${print_not_ok_stdout}"; } ||
|
if { test "${result}" = not_ok && "${print_not_ok_stdout}"; } ||
|
||||||
{ test "${result}" = ok && "${print_ok_stdout}"; }; then
|
{ test "${result}" = ok && "${print_ok_stdout}"; }; then
|
||||||
echo '# ------------ Begin output ------------'
|
echo '# ------------ Begin output ------------'
|
||||||
sed 's/^/# /' "$(stdout_file "${abspath}" "${the_shell}" "${host}")"
|
sed 's/^/# /' "$(stdout_file "${abspath}" "${the_shell}")"
|
||||||
echo '# ------------ End output ------------'
|
echo '# ------------ End output ------------'
|
||||||
fi
|
fi
|
||||||
echo "# Previous test took ${file_elapsed} seconds."
|
echo "# Previous test took ${file_elapsed} seconds."
|
||||||
@@ -437,44 +356,45 @@ report_outcome() {
|
|||||||
printf "$(dirname -- "${path}")/\n> $(basename -- "${path}")\n"
|
printf "$(dirname -- "${path}")/\n> $(basename -- "${path}")\n"
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
case "${result}" in
|
case "${result}" in
|
||||||
ok)
|
ok)
|
||||||
if "${print_ok}"; then
|
if "${print_ok}"; then
|
||||||
header
|
header
|
||||||
|
# On success, print a green '✓'
|
||||||
if "${print_in_color}"; then
|
if "${print_in_color}"; then
|
||||||
printf "\033[32m${success_mark} \033[0m"
|
printf '\033[32m✓ \033[0m'
|
||||||
else
|
else
|
||||||
printf "${success_mark} "
|
printf '✓ '
|
||||||
fi
|
fi
|
||||||
echo "${the_shell}${onhost} (${file_elapsed} $(plural second $file_elapsed))"
|
echo "${the_shell} ("${file_elapsed}" $(plural second $file_elapsed))"
|
||||||
fi
|
fi
|
||||||
;;
|
;;
|
||||||
not_ok)
|
not_ok)
|
||||||
if "${print_not_ok}"; then
|
if "${print_not_ok}"; then
|
||||||
header
|
header
|
||||||
|
# On not_ok, print a red '✗'
|
||||||
if "${print_in_color}"; then
|
if "${print_in_color}"; then
|
||||||
printf "\033[31m${fail_mark} \033[0m"
|
printf '\033[31m✗ \033[0m'
|
||||||
else
|
else
|
||||||
printf "${fail_mark} "
|
printf '✗ '
|
||||||
fi
|
fi
|
||||||
echo "${the_shell}${onhost} (${file_elapsed} $(plural second $file_elapsed))"
|
echo "${the_shell} ("${file_elapsed}" $(plural second $file_elapsed))"
|
||||||
fi
|
fi
|
||||||
;;
|
;;
|
||||||
skip)
|
skip)
|
||||||
if "${print_ok}"; then
|
if "${print_ok}"; then
|
||||||
header
|
header
|
||||||
if test -z "${the_shell}"; then
|
if test -z "${the_shell}"; then
|
||||||
echo " (File is not executable${onhost}.)"
|
echo ' (File is not executable.)'
|
||||||
else
|
else
|
||||||
echo " ${the_shell}${onhost} (${file_elapsed} $(plural second $file_elapsed))"
|
echo " ${the_shell} ("${file_elapsed}" $(plural second $file_elapsed))"
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
if { test "${result}" = not_ok && "${print_not_ok_stdout}"; } ||
|
if { test "${result}" = not_ok && "${print_not_ok_stdout}"; } ||
|
||||||
{ test "${result}" = ok && "${print_ok_stdout}"; }; then
|
{ test "${result}" = ok && "${print_ok_stdout}"; }; then
|
||||||
sed 's/^/ | /' "$(stdout_file "${abspath}" "${the_shell}" "${host}")"
|
sed 's/^/ # /' "$(stdout_file "${abspath}" "${the_shell}")"
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -504,10 +424,10 @@ report_outcome() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
has_shebang_line() {
|
has_shebang_line() {
|
||||||
head -n 1 "${1}" | grep -v '^#!/bin/sh$' | grep -q '^#!'
|
head -n 1 "${1}" | grep -qE '^#!'
|
||||||
}
|
}
|
||||||
|
|
||||||
USAGE="usage: $0 [options]... [test file or directory]..."
|
USAGE="usage: $0 [<options>] <test directory>"
|
||||||
|
|
||||||
urchin_help() {
|
urchin_help() {
|
||||||
cat <<EOF
|
cat <<EOF
|
||||||
@@ -522,15 +442,14 @@ particular test file once per shell.
|
|||||||
On each run,
|
On each run,
|
||||||
|
|
||||||
1. The TEST_SHELL environment variable is set to the particular shell.
|
1. The TEST_SHELL environment variable is set to the particular shell.
|
||||||
2. If the test file lacks a shebang line or has a shebang line of
|
2. If the test file lacks a shebang line, the test script is also
|
||||||
"#!/bin/sh", the test script is also executed in that shell.
|
executed in that shell.
|
||||||
|
|
||||||
The following flags affect how this multiple-shell testing is handled.
|
The following flags affect how this multiple-shell testing is handled.
|
||||||
|
|
||||||
-s, --shell <shell> Tell Urchin to use a different list of shells.
|
-s, --shell <shell> Tell Urchin to use a different list of shells.
|
||||||
(You can pass this flag multiple times.)
|
(You can pass this flag multiple times.)
|
||||||
|
-d, --disable-cycling Disable the cycling of shells; Urchin will
|
||||||
-n, --disable-cycling Disable the cycling of shells; Urchin will
|
|
||||||
execute test files ordinarily, implicitly using
|
execute test files ordinarily, implicitly using
|
||||||
sh for files that lack shebang lines. It will
|
sh for files that lack shebang lines. It will
|
||||||
set the TEST_SHELL variable to "/bin/sh" if and
|
set the TEST_SHELL variable to "/bin/sh" if and
|
||||||
@@ -540,9 +459,10 @@ The following flags affect how Urchin processes tests.
|
|||||||
|
|
||||||
-b, --run-in-series Run tests in series. The default is to run tests
|
-b, --run-in-series Run tests in series. The default is to run tests
|
||||||
in parallel where possible.
|
in parallel where possible.
|
||||||
|
-n, --max-forks Maximum number of parallel tests (Default is 50.)
|
||||||
-e, --exit-on-fail Stop running if any single test fails.
|
-e, --exit-on-fail Stop running if any single test fails.
|
||||||
This can be useful if you are running something
|
This is useful if you are running something
|
||||||
other than test files with Urchin.
|
configuration files with Urchin.
|
||||||
-T, --timeout <seconds> Kill a test if it runs for longer than the
|
-T, --timeout <seconds> Kill a test if it runs for longer than the
|
||||||
specified duration. The default is no timeout.
|
specified duration. The default is no timeout.
|
||||||
-f, --force Force running even if the test directory's name
|
-f, --force Force running even if the test directory's name
|
||||||
@@ -552,11 +472,9 @@ These options affect how results are formatted. Options -q, and -v
|
|||||||
have no effect when combined with --tap. -vv, -vvv, and -vvvv do have
|
have no effect when combined with --tap. -vv, -vvv, and -vvvv do have
|
||||||
effect when combined with --tap.
|
effect when combined with --tap.
|
||||||
|
|
||||||
-p, --pretty Print results in color and with fancy symbols.
|
-c, --color Print results in color.
|
||||||
-t, --tap Format output in Test Anything Protocol (TAP)
|
-t, --tap Format output in Test Anything Protocol (TAP)
|
||||||
|
|
||||||
And these options affect how much is printed.
|
|
||||||
|
|
||||||
-q, --quiet Print nothing to stdout;
|
-q, --quiet Print nothing to stdout;
|
||||||
the only output is the exit code.
|
the only output is the exit code.
|
||||||
(default verbosity) Print names of failed tests and counts
|
(default verbosity) Print names of failed tests and counts
|
||||||
@@ -564,21 +482,14 @@ And these options affect how much is printed.
|
|||||||
-v Print stdout from failing tests.
|
-v Print stdout from failing tests.
|
||||||
-vv Print names of passed tests.
|
-vv Print names of passed tests.
|
||||||
-vvv, --verbose Print stdout from all tests.
|
-vvv, --verbose Print stdout from all tests.
|
||||||
-vvvv, --debug Run with set -x.
|
-vvvv, --debug Print debugging messages (XXX not implemented)
|
||||||
|
|
||||||
The remaining flags provide information about urchin.
|
The remaining flags provide information about urchin.
|
||||||
|
|
||||||
-h, --help Display this help.
|
-h, --help Display this help.
|
||||||
--version Display the version number.
|
--version Display the version number.
|
||||||
|
|
||||||
Urchin recognizes certain environment variables.
|
Go to https://github.com/tlevine/urchin for documentation on writing tests.
|
||||||
|
|
||||||
TEST_SHELL This is sometimes over-ridden; see -s.
|
|
||||||
RUN_IN_SERIES Set this to true to have the same effect as
|
|
||||||
-b/--run-in-series. This is helpful if you are
|
|
||||||
calling urchin inside an urchin test suite.
|
|
||||||
|
|
||||||
Go to https://thomaslevine.com/!/urchin/ for documentation on writing tests.
|
|
||||||
|
|
||||||
EOF
|
EOF
|
||||||
}
|
}
|
||||||
@@ -604,13 +515,10 @@ validate_strings() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
argv="$@"
|
max_forks=50
|
||||||
|
|
||||||
cycle_shell=true
|
cycle_shell=true
|
||||||
shell_list="${urchin_tmp}"/shell_list
|
shell_list="${urchin_tmp}"/shell_list
|
||||||
test_arg_list="${urchin_tmp}"/test_list
|
test_arg_list="${urchin_tmp}"/test_list
|
||||||
> "${test_arg_list}"
|
|
||||||
remotes_list="${urchin_tmp}"/remotes
|
|
||||||
run_in_series=false
|
run_in_series=false
|
||||||
force=false
|
force=false
|
||||||
exit_on_not_ok=false
|
exit_on_not_ok=false
|
||||||
@@ -626,6 +534,8 @@ main() {
|
|||||||
do
|
do
|
||||||
case "${1}" in
|
case "${1}" in
|
||||||
-b|--run-in-series) run_in_series=true;;
|
-b|--run-in-series) run_in_series=true;;
|
||||||
|
-n|--max-forks) shift
|
||||||
|
max_forks="${1}";;
|
||||||
-e|--exit-on-fail) exit_on_not_ok=true;;
|
-e|--exit-on-fail) exit_on_not_ok=true;;
|
||||||
-f|--force) force=true;;
|
-f|--force) force=true;;
|
||||||
-s|--shell)
|
-s|--shell)
|
||||||
@@ -639,26 +549,17 @@ main() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
validate_strings "${shell_for_sh_tests}" 'Shell paths'
|
validate_strings "${shell_for_sh_tests}" 'Shell paths'
|
||||||
if echo "${shell_for_sh_tests}" | grep -q \ ; then
|
if echo "${shell_for_sh_tests}" | grep \ > /dev/null; then
|
||||||
echo "Warning: It is best if shell paths contain no spaces so that
|
echo "Warning: It is best if shell paths contain no spaces so that
|
||||||
you don't need to quote the TEST_SHELL variable." >&2
|
you don't need to quote the TEST_SHELL variable." > /dev/stderr
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "${shell_for_sh_tests}" >> "${shell_list}"
|
echo "${shell_for_sh_tests}" >> "${shell_list}"
|
||||||
;;
|
;;
|
||||||
-n|--disable-cycling) cycle_shell=false;;
|
-d|--disable-cycling) cycle_shell=false;;
|
||||||
-t|--tap) tap_format=true;;
|
-t|--tap) tap_format=true;;
|
||||||
-T|--timeout)
|
-T|--timeout) shift; urchin_timeout="${1}" ;;
|
||||||
shift
|
-c|--color) print_in_color=true;;
|
||||||
urchin_timeout="${1}"
|
|
||||||
if ! {
|
|
||||||
echo "${urchin_timeout}" |
|
|
||||||
grep '[0-9][0-9.]*\(s\|m\|h\|d\|\)'
|
|
||||||
}; then
|
|
||||||
echo Bad timeout argument: "${urchin_timeout}" >&2
|
|
||||||
urchin_exit 1
|
|
||||||
fi ;;
|
|
||||||
-p|--pretty) print_in_color=true;;
|
|
||||||
|
|
||||||
-q|--quiet) print_not_ok=false
|
-q|--quiet) print_not_ok=false
|
||||||
print_margins=false;;
|
print_margins=false;;
|
||||||
@@ -668,10 +569,7 @@ main() {
|
|||||||
-vvv|--verbose)print_not_ok_stdout=true
|
-vvv|--verbose)print_not_ok_stdout=true
|
||||||
print_ok=true;
|
print_ok=true;
|
||||||
print_ok_stdout=true;;
|
print_ok_stdout=true;;
|
||||||
-vvvv|--debug) print_not_ok_stdout=true
|
-vvvv|--debug) echo 'Not implemented' > /dev/stderr && exit 1;;
|
||||||
print_ok=true;
|
|
||||||
print_ok_stdout=true
|
|
||||||
set -x;;
|
|
||||||
|
|
||||||
-h|--help) urchin_help
|
-h|--help) urchin_help
|
||||||
urchin_exit 0;;
|
urchin_exit 0;;
|
||||||
@@ -685,17 +583,6 @@ main() {
|
|||||||
esac
|
esac
|
||||||
shift
|
shift
|
||||||
done
|
done
|
||||||
if "${RUN_IN_SERIES}" 2> /dev/null; then
|
|
||||||
run_in_series=true
|
|
||||||
fi
|
|
||||||
|
|
||||||
if $print_in_color; then
|
|
||||||
success_mark=✓
|
|
||||||
fail_mark=✗
|
|
||||||
else
|
|
||||||
success_mark=.
|
|
||||||
fail_mark=F
|
|
||||||
fi
|
|
||||||
|
|
||||||
# -------------------- VALIDATE INPUT -------------------- #
|
# -------------------- VALIDATE INPUT -------------------- #
|
||||||
if ! "${cycle_shell}" && test -f "${shell_list}"; then
|
if ! "${cycle_shell}" && test -f "${shell_list}"; then
|
||||||
@@ -707,7 +594,7 @@ main() {
|
|||||||
if ! test -f "${shell_list}"; then
|
if ! test -f "${shell_list}"; then
|
||||||
if $cycle_shell; then
|
if $cycle_shell; then
|
||||||
for shell in $DEFAULT_SHELLS; do
|
for shell in $DEFAULT_SHELLS; do
|
||||||
if which $shell 1> /dev/null 2> /dev/null; then
|
if which $shell > /dev/null; then
|
||||||
echo $shell >> "$shell_list"
|
echo $shell >> "$shell_list"
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
@@ -716,14 +603,30 @@ main() {
|
|||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Warn about strange sort commands
|
||||||
|
weird_string='@ b\n- d\n? a\n~ c\n! e\n'
|
||||||
|
sort_result="$(printf "${weird_string}" | sort | cut -d\ -f2 | tr -d '\n')"
|
||||||
|
if test "${sort_result}" = edacb; then
|
||||||
|
sort=sort
|
||||||
|
else
|
||||||
|
if which python > /dev/null; then
|
||||||
|
sort=sort_python
|
||||||
|
else
|
||||||
|
echo 'Your version of sort sorts in dictionary order (-d) by default.
|
||||||
|
Depending on how you name your tests, your Urchin output may look strange.
|
||||||
|
If this is a problem, install BusyBox or BSD coreutils.' >&2
|
||||||
|
sort=sort
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
if test -n "${urchin_timeout}"; then
|
if test -n "${urchin_timeout}"; then
|
||||||
# Choose the timeout command
|
# Choose the timeout command
|
||||||
if timeout -t 0 true 2> /dev/null; then
|
if timeout -t 0 true; then
|
||||||
TIMEOUT="timeout -t ${urchin_timeout}"
|
TIMEOUT="timeout -t ${urchin_timeout}"
|
||||||
elif timeout 0 true 2> /dev/null; then
|
elif timeout 0 true; then
|
||||||
TIMEOUT="timeout ${urchin_timeout}"
|
TIMEOUT="timeout ${urchin_timeout}"
|
||||||
else
|
else
|
||||||
echo I couldn\'t figure out how to use your version of timeout >&2
|
echo I couldn\'t figure out to use your version of timeout >&2
|
||||||
urchin_exit 1
|
urchin_exit 1
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
@@ -734,7 +637,7 @@ main() {
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
# -------------------- REALLY RUN -------------------- #
|
# -------------------- REALLY RUN -------------------- #
|
||||||
start=$("${epoch}")
|
start=$(date +%s)
|
||||||
|
|
||||||
# 1 test file or folder to run
|
# 1 test file or folder to run
|
||||||
# 2 urchin root
|
# 2 urchin root
|
||||||
@@ -744,42 +647,16 @@ main() {
|
|||||||
recurse "$(fullpath "${seed}")" "${root}" "${cycle_shell}" \
|
recurse "$(fullpath "${seed}")" "${root}" "${cycle_shell}" \
|
||||||
"${TEST_SHELL}" || break
|
"${TEST_SHELL}" || break
|
||||||
done < "${test_arg_list}"
|
done < "${test_arg_list}"
|
||||||
finish=$("${epoch}")
|
finish=$(date +%s)
|
||||||
|
|
||||||
if test $(cat "${urchin_tmp}"/log | wc -l) -eq 0; then
|
test $(cat "${urchin_tmp}"/log | wc -l) -gt 0 || {
|
||||||
echo 'No tests found' >&2
|
echo 'No tests found' > /dev/stderr
|
||||||
urchin_exit 1
|
urchin_exit 1
|
||||||
fi
|
}
|
||||||
|
|
||||||
if test -n "${RUNNING_ON_REMOTE}"; then
|
report_outcome "${root}" "${tap_format}" "${urchin_tmp}"/log "${start}" \
|
||||||
echo "${urchin_tmp}"
|
"${finish}"
|
||||||
elif test -f "${remotes_list}"; then
|
urchin_exit "${?}"
|
||||||
while read remote; do
|
|
||||||
remote_main $remote "${argv}"
|
|
||||||
done < "${remotes_list}"
|
|
||||||
else
|
|
||||||
report_outcome "${root}" "${tap_format}" "${urchin_tmp}"/log \
|
|
||||||
"${start}" "${finish}"
|
|
||||||
urchin_exit "${?}"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
remote_main() {
|
|
||||||
hostname="${1}"; shift
|
|
||||||
flags="${1}"; shift
|
|
||||||
urchin_dir=.urchin-cross-shell-test
|
|
||||||
|
|
||||||
rsync --archive -e "ssh ${flags}"
|
|
||||||
../urchin ../tests "${hostname}":"${urchin_dir}" ||
|
|
||||||
scp -r ${flags} ../urchin ../tests "${hostname}":"${urchin_dir}"
|
|
||||||
|
|
||||||
remote_tmp="$(ssh "${hostname}" ${flags} "cd ${urchin_dir} && ./urchin $@")"
|
|
||||||
|
|
||||||
remotedir="${hostname}":"${remote_tmp}"
|
|
||||||
localdir="${urchin_tmp}/remote/${host}"
|
|
||||||
rsync --archive -e "ssh ${flags}" "${remotedir}" "${localdir}" ||
|
|
||||||
scp -r ${flags} "${remotedir}"
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
test -n "${TESTING_URCHIN_INTERNALS}" || main "$@"
|
test -n "${TESTING_URCHIN_INTERNALS}" || main "$@"
|
||||||
|
|||||||
BIN
urchin_test_on_msys64_1.png
Normal file
BIN
urchin_test_on_msys64_1.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 99 KiB |
BIN
urchin_test_on_msys64_2.png
Normal file
BIN
urchin_test_on_msys64_2.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 115 KiB |
Reference in New Issue
Block a user