21 Commits

Author SHA1 Message Date
Thomas Levine
5e3c918675 add + indent symbol so directories line up 2016-01-28 15:12:34 +00:00
Thomas Levine
f5f98c8c2f fix fixture to not expect stderr 2016-01-28 15:11:27 +00:00
Thomas Levine
ce175cc3df test stdout stderr for TAP 2016-01-28 14:55:11 +00:00
Thomas Levine
cabaaa7ba4 TAP test 2016-01-28 14:51:43 +00:00
Thomas Levine
a663085069 remove awk dependency in test suite
I'm on NixOS, so awk isn't in /usr/bin. I can search for it with
/usr/bin/env, but then I can't use -f in the shebang line.
2016-01-27 10:51:04 +00:00
Thomas Levine
6bb606a3cc test plan 2016-01-27 10:39:07 +00:00
Thomas Levine
1c93e9a5c2 logfile 2016-01-27 10:31:51 +00:00
Thomas Levine
508e695dc3 error on fail 2016-01-27 10:25:21 +00:00
Thomas Levine
a51d96631f print skip count at end 2016-01-27 10:24:28 +00:00
Thomas Levine
1cd9991587 print stdout for tap and not-tap 2016-01-27 10:17:33 +00:00
Thomas Levine
0f1c2848b4 tap indentation comments for directories 2016-01-27 10:14:22 +00:00
Thomas Levine
9d10e12633 assorted tap stuff 2016-01-27 09:48:30 +00:00
Thomas Levine
40979f6e18 more tap 2016-01-27 01:13:55 +00:00
Thomas Levine
33e158e8f7 more tap 2016-01-27 01:08:04 +00:00
Thomas Levine
7ecacad132 oops 2016-01-27 01:05:43 +00:00
Thomas Levine
c8df46014d test test successes better 2016-01-27 01:05:08 +00:00
Thomas Levine
de2da89169 convert indents to comments 2016-01-27 00:50:55 +00:00
Thomas Levine
a6d6730e74 start writing non-tap cases 2016-01-27 00:32:51 +00:00
Thomas Levine
d01e993041 call urchin -s in cross-shell tests 2016-01-25 14:00:47 +00:00
Thomas Levine
b5c6464eab remove "urchin -x" test 2016-01-25 13:57:14 +00:00
Thomas Levine
329fc27929 remove "urchin -x"
unnecessary now that shall exists
2016-01-25 13:35:54 +00:00
11 changed files with 44 additions and 104 deletions

View File

@@ -1,9 +0,0 @@
Authors
-------
David Jones
Michael Klement
Thomas Levine
Maintainer
-------
Thomas Levine <_@thomaslevine.com>

20
HISTORY
View File

@@ -1,23 +1,5 @@
HISTORY HISTORY
======= -------
Version 0.0.6
---------------------
* Produce TAP output with the -t flag.
* Add a + sign in front of directories in the normal output so that they
line up with non-directories.
* Display skipped tests in the normal output and in the TAP output.
* Correct some things in the documentation.
* Rearrange things in the documentation to be more clear.
* Pass the -e flag to exit urchin if any single test fails.
* Remove the undocumented, experimental -x flag now that shall exists.
* Display version number with the -v flag.
* Document why Urchin is called "Urchin"
These changes are made somewhat separately in the branches "exit-on-fail",
"remove-urchin-x", "tap", and "update-readme". They are rebased into one
branch, "tlevine-2016-02", for merging into "master".
Version 0.0.5 Version 0.0.5
--------------------- ---------------------

View File

@@ -2,11 +2,11 @@
# Run urchin in a bunch of different shells, # Run urchin in a bunch of different shells,
# including a shell that isn't quite POSIX-compatible (zsh) # including a shell that isn't quite POSIX-compatible (zsh)
for shell in dash bash mksh ksh zsh; do for shell in dash bash ksh zsh; do
if which $shell > /dev/null 2> /dev/null; then if which $shell > /dev/null 2> /dev/null; then
echo echo
echo Running urchin tests in $shell echo Running urchin tests in $shell
$shell urchin -s $shell tests | tail -n 4 $shell urchin -s $shell tests | tail -n 3
else else
echo echo
echo Skipping $shell because it is not in the PATH echo Skipping $shell because it is not in the PATH

View File

@@ -4,13 +4,9 @@
/ /_/ / / / /__/ / / / / / / / / /_/ / / / /__/ / / / / / / /
\__,_/_/ \___/_/ /_/_/_/ /_/ \__,_/_/ \___/_/ /_/_/_/ /_/
Urchin is a file-based test harness, normally used for testing shell programs. Urchin is a test framework for shell. It is implemented in
It is written in portable shell and should thus work on GNU/Linux, BSD portable /bin/sh and should work on GNU/Linux, Mac OS X, and
(including Mac OS X), and other Unix-like platforms. other Unix platforms.
Urchin is called "Urchin" because
[sea urchins](https://en.wikipedia.org/wiki/Sea_urchin)
have shells called "tests".
## Try it out ## Try it out
Urchin's tests are written in Urchin, so you can run them to see what Urchin Urchin's tests are written in Urchin, so you can run them to see what Urchin
@@ -31,15 +27,14 @@ run this:
cd urchin cd urchin
./cross-shell-tests ./cross-shell-tests
## Install ## Globally
Urchin is contained in a single file, so you can install it by copying it to a Download Urchin like so (as root) (or use npm, below):
directory in your `PATH`. For example, you can run the following as root.
cd /usr/local/bin cd /usr/local/bin
wget https://raw.github.com/tlevine/urchin/master/urchin wget https://raw.github.com/tlevine/urchin/master/urchin
chmod +x urchin chmod +x urchin
Urchin can be installed with npm too. Can be installed with npm too:
npm install -g urchin npm install -g urchin
@@ -91,14 +86,15 @@ Files are only run if they are executable, and files beginning with `.` are
ignored. Thus, fixtures and libraries can be included sloppily within the test ignored. Thus, fixtures and libraries can be included sloppily within the test
directory tree. The test passes if the file exits 0; otherwise, it fails. directory tree. The test passes if the file exits 0; otherwise, it fails.
Tests files and subdirectories are run in ASCIIbetical order within each In case you care about the order in which your tests execute, consider that
directory; that is,
urchin looks for files within a directory in the following manner. urchin looks for files within a directory in the following manner.
for file in *; do for file in *; do
do_something_with_test_file $file do_something_with_test_file $file
done done
Tests within a directory are executed in whatever order `*` returns.
### Writing cross-shell compatibility tests for testing shell code ### Writing cross-shell compatibility tests for testing shell code
While you could write your test scripts to explicitly invoke the functionality While you could write your test scripts to explicitly invoke the functionality
@@ -110,20 +106,24 @@ The specific approach depends on your test scenario:
* (b) Your scripts _source_ scripts containing portable shell code. * (b) Your scripts _source_ scripts containing portable shell code.
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts #### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
Urchin sets the `TEST_SHELL` environment variable so that you may change the
shell with which your tests call other shell programs. To run your test
scripts in multiple shells you must call `$TEST_SHELL` in your tests and then
run urchin with the appropriate option.
First, consider using [shall](https://github.com/mklement0/shall).
#!/usr/bin/env shall
echo This is a test file.
Alternatively, you can use urchin's built-in recognition of the
`TEST_SHELL` environment variable.
In your test scripts, invoke the shell scripts to test via the shell In your test scripts, invoke the shell scripts to test via the shell
specified in environment variable `TEST_SHELL` rather than directly; specified in environment variable `TEST_SHELL` rather than directly;
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`). e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
Note that if you alsow want your test scripts to work when run directly,
outside of Urchin, be sure to target scripts that happen to be in the
current directory with prefix `./`; e.g., `$TEST_SHELL ./baz`
(rather than `$TEST_SHELL baz`).
On invocation of Urchin, prepend a definition of environment variable Then, on invocation of Urchin, prepend a definition of environment variable
`TEST_SHELL` specifying the shell to test with, e.g., `TEST_SHELL` specifying the shell to test with, e.g.: `TEST_SHELL=zsh urchin ./tests`.
TEST_SHELL=zsh urchin ./tests
To test with multiple shells in sequence, use something like: To test with multiple shells in sequence, use something like:
for shell in sh bash ksh zsh; do for shell in sh bash ksh zsh; do
@@ -131,20 +131,14 @@ To test with multiple shells in sequence, use something like:
done done
If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
scripts can rely on `$TEST_SHELL` always containing a value when Urchin runs scripts can rely on `$TEST_SHELL` always containing a value.
them.
That said, we still recommand that you account for the possibility that
`$TEST_SHELL` does not contain a value so that you may run your test scripts
without Urchin. Supporting this case is very simple; when you invoke scripts
that happen to be in the current directory, be sure to use the prefix `./`,
e.g., `$TEST_SHELL ./baz` rather than `$TEST_SHELL baz`.
#### (b) Cross-shell tests with test scripts that _source_ shell scripts #### (b) Cross-shell tests with test scripts that _source_ shell scripts
If you _source_ shell code in your test scripts, it is the test scripts If you _source_ shell code in your test scripts, it is the test scripts
themselves that must be run with the shell specified. themselves that must be run with the shell specified.
Urchin supports the `-s <shell>` option, which instructs To that end, Urchin supports the `-s <shell>` option, which instructs
Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`. Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`.
(In addition, Urchin sets environment variable `TEST_SHELL` to the specified (In addition, Urchin sets environment variable `TEST_SHELL` to the specified
shell.) shell.)
@@ -160,12 +154,21 @@ To test with multiple shells in sequence, use something like:
urchin -s $shell ./tests urchin -s $shell ./tests
done done
Also consider using [shall](https://github.com/mklement0/shall). <!--
It does something similar, but the interface may be more intuitive. #### (c) Cross shell tests with `urchin -x` (experimental)
If you run urchin with the `-x` flag, it will be as if you ran
`$TEST_SHELL`. Unless `$TEST_SHELL` isn't set, in which case it'll
be as if you ran `/bin/sh`. Putting this in she shebang line might
eventually work out to be a cleaner way of doing cross-shell testing.
#!/usr/bin/env shall #!/usr/bin/env urchin -x
echo This is a test file. test a = a
It might make sense if you do this.
export TEST_SHELL=zsh && urchin -x
export TEST_SHELL=bash && urchin -x
-->
## Alternatives to Urchin ## Alternatives to Urchin
Alternatives to Urchin are discussed in Alternatives to Urchin are discussed in
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/). [this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).

View File

@@ -1 +0,0 @@
false

View File

@@ -1 +0,0 @@
false

View File

@@ -1 +0,0 @@
false

View File

@@ -1,11 +0,0 @@
tmp=$(mktemp)
../urchin -e -f ./.die-on-fail > $tmp
result=$?
grep '1 should run.' $tmp
grep '2 should run.' $tmp
grep -v '3 should not run.' $tmp
grep -v '4 should not run.' $tmp
rm $tmp
exit $result

View File

@@ -1,2 +0,0 @@
#!/bin/sh
../urchin -v | grep '[0-9.]\{3,\}'

26
urchin
View File

@@ -4,9 +4,6 @@
# which breaks fullpath(). # which breaks fullpath().
unset CDPATH unset CDPATH
# Urchin version number
VERSION=0.0.6
fullpath() { fullpath() {
( (
cd -- "$1" cd -- "$1"
@@ -57,13 +54,6 @@ recurse() {
# $2 instead of $indent_level so it doesn't clash # $2 instead of $indent_level so it doesn't clash
recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests" recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
exit_code=$?
if $exit_on_fail && test $exit_code -ne 0; then
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
[ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
return 1
fi
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file" [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
done done
@@ -78,6 +68,7 @@ recurse() {
else else
if [ -x "$potential_test" ] if [ -x "$potential_test" ]
then then
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file" [ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
# Run the test # Run the test
@@ -89,6 +80,7 @@ recurse() {
fi fi
exit_code="$?" exit_code="$?"
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file" [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
if [ $exit_code -eq 0 ]; then if [ $exit_code -eq 0 ]; then
result=success result=success
@@ -140,10 +132,6 @@ recurse() {
;; ;;
esac esac
fi fi
if $exit_on_fail && test 0 -ne $exit_code; then
return 1
fi
fi fi
[ $indent_level -eq 0 ] && rm "$stdout_file" [ $indent_level -eq 0 ] && rm "$stdout_file"
} }
@@ -163,14 +151,10 @@ $USAGE
-s <shell> Invoke test scripts that either have no shebang line at all or -s <shell> Invoke test scripts that either have no shebang line at all or
have shebang line "#!/bin/sh" with the specified shell. have shebang line "#!/bin/sh" with the specified shell.
-e Stop running if any single test fails. This is helpful if you want
to use Urchin to run things other than tests, such as a set of
configuration scripts.
-f Force running even if the test directory's name does not -f Force running even if the test directory's name does not
contain the word "test". contain the word "test".
-t Format output in Test Anything Protocol (TAP) -t Format output in Test Anything Protocol (TAP)
-h, --help This help. -h This help.
-v Display the version number.
Go to https://github.com/tlevine/urchin for documentation on writing tests. Go to https://github.com/tlevine/urchin for documentation on writing tests.
@@ -247,12 +231,10 @@ urchin_molly_guard() {
shell_for_sh_tests= shell_for_sh_tests=
force=false force=false
exit_on_fail=false
tap_format=false tap_format=false
while [ $# -gt 0 ] while [ $# -gt 0 ]
do do
case "$1" in case "$1" in
-e) exit_on_fail=true;;
-f) force=true;; -f) force=true;;
-s) -s)
shift shift
@@ -262,8 +244,6 @@ do
-t) tap_format=true;; -t) tap_format=true;;
-h|--help) urchin_help -h|--help) urchin_help
exit 0;; exit 0;;
-v) echo "$VERSION"
exit;;
-*) urchin_help >&2 -*) urchin_help >&2
exit 1;; exit 1;;
*) break;; *) break;;