59 Commits

Author SHA1 Message Date
Thomas Levine
1941bf716a not working 2016-01-25 14:41:21 +00:00
Thomas Levine
9f175f19b3 first pass implementation of exit on fail 2016-01-25 14:35:54 +00:00
Thomas Levine
9043430750 document unimplemented -e flag 2016-01-25 14:20:40 +00:00
Thomas Levine
f2d63b2953 Test die on fail. 2016-01-25 14:14:13 +00:00
Thomas Levine
d01e993041 call urchin -s in cross-shell tests 2016-01-25 14:00:47 +00:00
Thomas Levine
b5c6464eab remove "urchin -x" test 2016-01-25 13:57:14 +00:00
Thomas Levine
329fc27929 remove "urchin -x"
unnecessary now that shall exists
2016-01-25 13:35:54 +00:00
Thomas Levine
48e5090091 start writing alternatives 2015-12-09 07:46:16 -05:00
Thomas Levine
5abf088eb8 add new ideas 2015-10-19 12:52:24 -04:00
Thomas Levine
c84851cb5c document the ordering of tests 2015-08-21 09:03:15 -04:00
Thomas Levine
510476f924 mention shall in the documentation 2015-07-05 17:39:38 +00:00
Thomas Levine
f01869fb97 packaging 2014-12-08 02:08:00 +00:00
Michael Klement
0385dcd86e Fixed blog link in read-me and commented out experimental -x option for now. 2014-12-05 18:50:19 -05:00
Michael Klement
2847b020b4 Make sure that CDPATH isn't set, as it causes cd to behave unpredictably - notably, it can produce output, which breaks fullpath().
Also: Improved CLI help, updated URLs in read-me, cleaned up package.json:

I've tried to clarify the intent of `-x` in the CLI help, but I haven't touched the read-me in that respect.

I don't see any benefit to `-x`:

* Just using `#/bin/sh` as the shebang line in combination with `-s <shell>` gives you the same functionality,
* When it comes to invoking scripts from _within_ test scripts, nothing can do the work for you: you consciously have to mark the invocation with _something_ to indicate that it should be controlled from the outside; it won't get any easier than `$TEST_SHELL ...`
* Finally, using a shebang line such as `#!/usr/bin/env urchin -x` is problematic for two reasons:
  * Some platforms can handle only *1* argument in a shebang line.
  * In a _package-local_ installation, `#!/usr/bin/env` may not find the Urchin executable.

I'm also not sure how the following (from `readme.md`) fits in the picture:

> It might make sense if you do this.

    export TEST_SHELL=zsh && urchin -x
    export TEST_SHELL=bash && urchin -x

(As an aside: To achieve the same thing, you don't need `export`; `TEST_SHELL=zsh urchin -x`  and `TEST_SHELL=bash urchin -x`  is the better choice.)
How does this relate to use in a  _shebang line_?

`urchin_help()` now uses a here-doc: easier to maintain, and should work in all Bourne-like shells.

`readmeFilename` removed from `package.json`:

> "The readmeFilename does not need to ever be in your actual package.json file" - npm/npm#3573
2014-12-03 09:54:15 -05:00
Michael Klement
fae24e926a Improved CLI help, updated URLs in read-me 2014-11-06 22:21:05 -05:00
Thomas Levine
2ab070c353 fix the -x documentation 2014-11-06 00:52:34 +00:00
Thomas Levine
292663dd73 contributors 2014-11-06 00:51:14 +00:00
Thomas Levine
90abc2640d update the -h help text 2014-11-05 17:49:53 +00:00
Thomas Levine
19ed3b02e8 HISTORY 2014-11-05 17:49:08 +00:00
Thomas Levine
b0429315ec urchin -sh to urchin -x 2014-11-05 17:47:36 +00:00
Thomas Levine
5e06ffb1c1 version 2014-11-05 17:47:01 +00:00
Thomas Levine
bf8c404784 fix -sh 2014-11-05 17:46:19 +00:00
Thomas Levine
bb41d5a156 zsh support 2014-11-05 17:38:22 +00:00
Thomas Levine
ed09351df6 fix a test 2014-11-05 17:28:54 +00:00
Thomas Levine
1e2d4cc8a3 remove gitHead 2014-11-05 17:27:26 +00:00
Thomas Levine
f8d921b5e5 document urchin -sh 2014-11-05 17:26:01 +00:00
Thomas Levine
1f115df222 urchin -sh 2014-11-05 17:18:31 +00:00
Thomas Levine
eafed6ac59 put history back 2014-11-05 16:54:36 +00:00
Thomas Levine
8273661440 tlevine maintainer 2014-11-05 16:53:17 +00:00
Thomas Levine
f57e99be6f we're not maintaining the history, and i don't care 2014-11-05 16:52:13 +00:00
Michael Klement
afc02582fb Fixed typo in read-me. 2014-10-18 10:41:22 -04:00
Michael Klement
2f97421164 Hint added to read-me re standalone invocation of tests that use TEST_SHELL; revision of wording in read-me; argument checking no longer treats a missing dir. argument as an explicitly specified argument. 2014-10-18 10:34:30 -04:00
Michael Klement
1e9611e732 Support for cross-shell testing added, via option -s <shell> and env. variable TEST_SHELL.
* For **tests that _source_ shell scripts**: **option `-s <shell>`** now tells urchin to invoke test scripts with the specified shell (only shebang-less and `#!/bin/sh` tests scripts).
* For **tests that _invoke_ schell scripts**: instruct users to write their tests to always **invoke via environment variable `TEST_SHELL` (e.g., `$TEST_SHELL ../foo`)**, and invoke urchin with that variable defined as needed, e.g., `TEST_SHELL=ksh urchin ./tests`; urchin defaults `TEST_SHELL` to `/bin/sh`.

See updated `readme.md` for details.
2014-10-17 17:16:12 -04:00
Michael Klement
deb77cb5de Various bug fixes and cleanup.
* bug fixes:
  * fixed directory-existence argument check
  * fixed how `.urchin_stdout` files are cleaned up
  * fixed instances of error/warning messages outputting to stdout instead of stderr
* documentation, help and error-message improvements:
  * added specific error message if the (non-option) argument is not a directory
  * improved wording of CLI help
  * readme.md: replaced obsolete URL http://www.urchin.sh with https://github.com/scraperwiki/urchin
  * readme.md: made the fact clearer that `./cross-shell-tests` only tests urchin's _own_ cross-shell compatibility
  * HISTORY: fixed typo
* formatting and logging improvements:
  * added timestamp to the beginning of log files (`.urchin.log`)
  * captured output from failed tests is now printed in *red* to draw attention
  * test summary now prints the number of failed tests in the appropriate color for instant feedback (green, if none failed; red, otherwise)
* cleanup
  * removed unused test files, simplified some tests
2014-10-16 16:37:51 -04:00
Tom Mortimer-Jones
f12636e029 Merge pull request #6 from scraperwiki/exit-code
Set the exit status to reflect pass/fail.
2014-03-17 14:04:22 +00:00
David Jones
0587a615f3 Document the non zero exit code case. 2014-03-17 14:00:30 +00:00
David Jones
b95b010c0d Set the exit status to reflect pass/fail.
For Travis compatibility, we set the exit code to non-zero when
any tests fail.

Fixes #5.
2014-03-17 13:43:13 +00:00
David Jones
98c23a2ab8 Merge pull request #4 from tlevine/link-blog-post
Link to the blog post
2013-10-14 01:20:38 -07:00
Thomas Levine
fc711672d6 alternatives 2013-10-13 16:48:42 +00:00
David Jones
15099082db 0.0.2 2013-07-26 12:53:25 +01:00
David Jones
8d53676743 Ensure we can cd into directories starting with a hyphen. Fixes #3. 2013-07-26 12:51:01 +01:00
David Jones
8913cbc195 Removing space at the end of a line. 2013-07-02 09:46:21 +01:00
David Jones
17b4629ba5 when testing, just cd into urchin 2013-07-02 09:43:59 +01:00
David Jones
85bfafffc8 /bin/sh not login shell 2013-07-02 09:43:16 +01:00
David Jones
153b04b462 Merge branch 'master' of github.com:scraperwiki/urchin 2013-07-02 09:25:49 +01:00
David Jones
1cf65a723d Merge pull request #2 from tlevine/cross-shell-test-runner
Cross shell test runner
2013-07-01 08:17:52 -07:00
David Jones
c2f4d92704 Hide fixtures directory. 2013-06-28 10:58:25 +01:00
David Jones
31da4fae92 Print date and duration. 2013-06-28 10:48:43 +01:00
David Jones
e892c565cb Removed spurious output from grep. 2013-06-28 10:47:51 +01:00
Thomas Levine
5aace72fc6 remove conflicting ksh calls 2013-06-27 11:45:45 -07:00
Thomas Levine
a749c8dae3 fix urchin call in XSD test 2013-06-27 11:45:05 -07:00
Thomas Levine
da9a0c9948 documentation 2013-06-27 11:44:19 -07:00
Thomas Levine
91a4467f3e cross-shell test runner in runtests 2013-06-27 11:33:16 -07:00
David Jones
0110a72965 remove pointless use of echo 2013-06-27 18:27:40 +01:00
David Jones
d9902c0b11 ASCII art from figlet 2013-06-26 08:22:54 +01:00
David Jones
a900722fb7 Fix bug when test is called "-n" 2013-06-26 08:15:07 +01:00
David Jones
4d3a9eddb5 npm instructions. 2013-06-21 17:14:44 +01:00
David Jones
3062e58a75 Use singular / plural form correctly. 2013-06-20 18:56:29 +01:00
David Jones
222ba59d69 Test for singular. 2013-06-20 18:43:11 +01:00
41 changed files with 433 additions and 98 deletions

33
HISTORY
View File

@@ -1,18 +1,31 @@
History
======
HISTORY
-------
Version 0.0.5
---------------------
* urchin now unsets `CDPATH`.
* The documentation for `urchin -x` was removed because it was confusing.
Not done
------
We made the colors brighter.
Version 0.0.4
---------------------
* Switch urchin -x to urchin -sh and fix some problems with it
* Documentation
We allowed for multiple tests to be written in a file
Version 0.0.3
---------------------
General tidying
We added flags for verbosity and for Molly guards
Run with different shells in three ways
Done
-----
* urchin -s
* $TEST_SHELL variable with $TEST_SHELL
* $TEST_SHELL variable with urchin -sh
We adjusted the input parameters so it is harder to accidentally run all executible files in your home directory.
Set NULL_GLOB so zsh doesn't print a warning.
Before version 0.0.3
----------------------
We adjusted the input parameters so it is harder to accidentally run all executable files in you
r home directory.
We added directory-based indents.

2
alternatives Normal file
View File

@@ -0,0 +1,2 @@
Totally different syntax and similar features, plus TAP output
https://github.com/sstephenson/bats

15
cross-shell-tests Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/sh
# Run urchin in a bunch of different shells,
# including a shell that isn't quite POSIX-compatible (zsh)
for shell in dash bash ksh zsh; do
if which $shell > /dev/null 2> /dev/null; then
echo
echo Running urchin tests in $shell
$shell urchin -s $shell tests | tail -n 3
else
echo
echo Skipping $shell because it is not in the PATH
fi
done
echo

View File

@@ -1,26 +1,32 @@
{
"name": "urchin",
"version": "0.0.1",
"version": "0.0.5",
"description": "Test framework for shell",
"main": "urchin",
"directories": {
"test": "tests"
},
"scripts": {
"test": "./urchin tests"
"test": "export PATH=\"$PWD:$PATH\" && urchin tests"
},
"bin": "./urchin",
"repository": {
"type": "git",
"url": "git://github.com/scraperwiki/urchin.git"
"url": "git://github.com/tlevine/urchin.git"
},
"keywords": [
"shell",
"test",
"testing"
],
"author": "ScraperWiki Limited <feedback@scraperwiki.com>",
"license": "BSD",
"readmeFilename": "readme.md",
"gitHead": "015052dcfd7152a0df037bef1bc5452e0c956695"
"author": "Thomas Levine <_@thomaslevine.com>",
"contributors": [
{"name": "Thomas Levine", "email": "_@thomaslevine.com"},
{"name": "David Jones", "email": "drj@pobox.com"},
{"name": "Francis Irving", "email": "francis@flourish.org"},
{"name": "Zarino Zappia", "email": "mail@zarino.co.uk"},
{"name": "Tom Mortimer-Jones", "email": "tom@morty.co.uk"},
{"name": "Michael Klement", "email": "mklement0@gmail.com"}
],
"license": "BSD"
}

129
readme.md
View File

@@ -1,7 +1,8 @@
, , ,_ _, , , ___, , ,
| | |_) / |_|, ' | |\ |
'\__| '| \ '\_ '| | _|_, |'\|
` ' ` ` ' ` ' ' `
__ _
__ ____________/ /_ (_)___
/ / / / ___/ ___/ __ \/ / __ \
/ /_/ / / / /__/ / / / / / / /
\__,_/_/ \___/_/ /_/_/_/ /_/
Urchin is a test framework for shell. It is implemented in
portable /bin/sh and should work on GNU/Linux, Mac OS X, and
@@ -11,23 +12,38 @@ other Unix platforms.
Urchin's tests are written in Urchin, so you can run them to see what Urchin
is like. Clone the repository
git clone git://github.com/scraperwiki/urchin.git
git clone git://github.com/tlevine/urchin.git
Run the tests
cd urchin/tests
../urchin .
cd urchin
./urchin tests
The above command will run the tests in your system's default
shell, /bin/sh (on recent Ubuntu this is dash, but it could be
ksh or bash on other systems); to test urchin's cross-shell compatibility,
run this:
cd urchin
./cross-shell-tests
## Globally
Download Urchin like so (as root)
Download Urchin like so (as root) (or use npm, below):
wget -O /usr/local/bin https://raw.github.com/scraperwiki/urchin/master/urchin
chmod +x /usr/local/bin/urchin
cd /usr/local/bin
wget https://raw.github.com/tlevine/urchin/master/urchin
chmod +x urchin
Can be installed with npm too:
npm install -g urchin
Now you can run it.
urchin <test directory>
Run `urchin -h` to get command-line help.
## Writing tests
Make a root directory for your tests. Inside it, put executable files that
exit `0` on success and something else on fail. Non-executable files and hidden
@@ -69,3 +85,96 @@ within the particular directory, and the `teardown` file is run right after.
Files are only run if they are executable, and files beginning with `.` are
ignored. Thus, fixtures and libraries can be included sloppily within the test
directory tree. The test passes if the file exits 0; otherwise, it fails.
In case you care about the order in which your tests execute, consider that
urchin looks for files within a directory in the following manner.
for file in *; do
do_something_with_test_file $file
done
Tests within a directory are executed in whatever order `*` returns.
### Writing cross-shell compatibility tests for testing shell code
While you could write your test scripts to explicitly invoke the functionality
to test with various shells, Urchin facilitates a more flexible approach.
The specific approach depends on your test scenario:
* (a) Your test scripts _invoke_ scripts containing portable shell code.
* (b) Your scripts _source_ scripts containing portable shell code.
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
First, consider using [shall](https://github.com/mklement0/shall).
#!/usr/bin/env shall
echo This is a test file.
Alternatively, you can use urchin's built-in recognition of the
`TEST_SHELL` environment variable.
In your test scripts, invoke the shell scripts to test via the shell
specified in environment variable `TEST_SHELL` rather than directly;
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
Note that if you alsow want your test scripts to work when run directly,
outside of Urchin, be sure to target scripts that happen to be in the
current directory with prefix `./`; e.g., `$TEST_SHELL ./baz`
(rather than `$TEST_SHELL baz`).
Then, on invocation of Urchin, prepend a definition of environment variable
`TEST_SHELL` specifying the shell to test with, e.g.: `TEST_SHELL=zsh urchin ./tests`.
To test with multiple shells in sequence, use something like:
for shell in sh bash ksh zsh; do
TEST_SHELL=$shell urchin ./tests
done
If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
scripts can rely on `$TEST_SHELL` always containing a value.
#### (b) Cross-shell tests with test scripts that _source_ shell scripts
If you _source_ shell code in your test scripts, it is the test scripts
themselves that must be run with the shell specified.
To that end, Urchin supports the `-s <shell>` option, which instructs
Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`.
(In addition, Urchin sets environment variable `TEST_SHELL` to the specified
shell.)
Note that only test scripts that either have no shebang line at all or
have shebang line `#!/bin/sh` are invoked with the specified shell.
This allows non-shell test scripts or test scripts for specific
shells to coexist with those whose invocation should be controlled by `-s`.
To test with multiple shells in sequence, use something like:
for shell in sh bash ksh zsh; do
urchin -s $shell ./tests
done
<!--
#### (c) Cross shell tests with `urchin -x` (experimental)
If you run urchin with the `-x` flag, it will be as if you ran
`$TEST_SHELL`. Unless `$TEST_SHELL` isn't set, in which case it'll
be as if you ran `/bin/sh`. Putting this in she shebang line might
eventually work out to be a cleaner way of doing cross-shell testing.
#!/usr/bin/env urchin -x
test a = a
It might make sense if you do this.
export TEST_SHELL=zsh && urchin -x
export TEST_SHELL=bash && urchin -x
-->
## Alternatives to Urchin
Alternatives to Urchin are discussed in
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
## Ideas for new features
* Support [Nagios plugins](https://nagios-plugins.org/doc/guidelines.html)
* Stop running if a test fails so one can use Urchin as a
[setup framework](https://github.com/tlevine/urchin/issues/16).

View File

@@ -0,0 +1 @@
false

View File

@@ -0,0 +1 @@
false

View File

@@ -0,0 +1 @@
false

1
tests/.gitignore vendored
View File

@@ -1 +1,2 @@
.urchin.log
.urchin_stdout

1
tests/.print-arg-3 Normal file
View File

@@ -0,0 +1 @@
echo $3

View File

@@ -0,0 +1,6 @@
#!/bin/sh
cd ..
export CDPATH=$PWD
./urchin -f 'tests/urchin exit code' >/dev/null

View File

@@ -0,0 +1,3 @@
#!/bin/sh
../../urchin -h | grep -- -s

View File

@@ -1,3 +0,0 @@
#!/bin/sh
../../urchin -h|grep -i xsd

View File

@@ -0,0 +1 @@
false

View File

@@ -0,0 +1,2 @@
#!/bin/sh
../../urchin .test-one | grep '1 test failed'

View File

@@ -0,0 +1,2 @@
#!/bin/sh
../../urchin .test-one | grep '1 test passed'

View File

@@ -0,0 +1,7 @@
#!/bin/sh
# Assuming that urchin was invoked with `TEST_SHELL=bash urchin ...`, $TEST_SHELL should contain 'bash'.
echo "\$TEST_SHELL: $TEST_SHELL"
[ "$TEST_SHELL" = 'bash' ]

View File

@@ -0,0 +1,4 @@
#!/bin/sh
# Invoke a simple test command with $TEST_SHELL as the executable.
[ "$($TEST_SHELL -c 'echo $0')" = "$TEST_SHELL" ]

View File

@@ -0,0 +1,5 @@
#!/usr/bin/awk -f
# This script will only succeed if it is indeed processed by awk.
BEGIN { print "ok" }

View File

@@ -0,0 +1,10 @@
#!/bin/sh
# Assuming that urchin was invoked with `-s bash`, this script should be being run with bash.
this_shell=$(ps -o comm= -p $$ && :)
echo "Running shell: $this_shell"
[ "$this_shell" = 'bash' ]

View File

@@ -0,0 +1,10 @@
# By design, this file has no shebang line.
# Assuming that urchin was invoked with `-s bash`, this script should be being run with bash.
this_shell=$(ps -o comm= -p $$ && :)
echo "Running shell: $this_shell"
[ "$this_shell" = 'bash' ]

View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Assuming that urchin was invoked with `-s bash`, $TEST_SHELL should contain 'bash'.
echo "Running shell: $(ps -o comm= -p $$ && :)"
echo "\$TEST_SHELL: $TEST_SHELL"
[ "$TEST_SHELL" = 'bash' ]

View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
# (for test scripts that want to invoke shell scripts with a specified shell).
which bash 2>/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 1; }
# Test if $TEST_SHELL, when placed in urchin's environment, is passed through to the test scripts.
TEST_SHELL=bash ../../urchin ./.test-TEST_SHELL-passed-through

View File

@@ -0,0 +1,8 @@
#!/bin/sh
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
# (for test scripts that want to invoke shell scripts with a specified shell).
# Test if $TEST_SHELL - if *defined, but empty* - is exported with value '/bin/sh' by urchin
# and thus has that value inside the scripts.
TEST_SHELL= ../../urchin ./.test-TEST_SHELL-undefined_or_empty

View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
# (for test scripts that want to invoke shell scripts with a specified shell).
# Test if $TEST_SHELL - if *undefined* - is exported with value '/bin/sh' by urchin
# and thus has that value inside test scripts.
unset -v TEST_SHELL
../../urchin ./.test-TEST_SHELL-undefined_or_empty

View File

@@ -0,0 +1,8 @@
#!/bin/sh
# Tests the `-s <shell> option, which invokes shebang-less and sh-shebang-line test scripts with the specified shell (for testing *sourced* shell code).
which bash >/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 2; }
which /usr/bin/awk >/dev/null || { echo "Cannot test -s option: /usr/bin/awk not found." >&2; exit 2; }
../../urchin -s bash ./.test-run-by-specified-shell

11
tests/Die on fail. Executable file
View File

@@ -0,0 +1,11 @@
tmp=$(mktemp)
../urchin -e -f ./.die-on-fail > $tmp
result=$?
grep '1 should run.' $tmp
grep '2 should run.' $tmp
grep -v '3 should not run.' $tmp
grep -v '4 should not run.' $tmp
rm $tmp
exit $result

View File

@@ -1,3 +1,3 @@
#!/bin/sh
grep blah fixtures/bladyblah > /dev/null
grep blah .fixtures/bladyblah > /dev/null

View File

@@ -1,3 +0,0 @@
#!/bin/sh
exit 1

View File

@@ -1,3 +0,0 @@
#!/bin/sh
../../../urchin .fixture | grep 'http://www.w3.org/2001/XMLSchema'

View File

@@ -1,3 +0,0 @@
#!/bin/sh
export variable_set_in_setup='value'

View File

View File

@@ -0,0 +1,13 @@
#!/bin/sh
# Borrow this test which has a failure in it (3 in fact), and
# run it using urchin:
../../urchin "../Counts should be kept of successes and failures./.test" > /dev/null
# Now invert the result, we want _this_ test to fail if the exit
# code was 0.
case $? in
0) exit 8;;
*) exit 0;;
esac

View File

@@ -0,0 +1 @@
../../urchin .test > /dev/null

175
urchin
View File

@@ -1,8 +1,12 @@
#!/bin/sh
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably - notably, it can produce output,
# which breaks fullpath().
unset CDPATH
fullpath() {
(
cd "$1"
cd -- "$1"
pwd
)
}
@@ -15,27 +19,41 @@ indent() {
recurse() {
potential_test="$1"
indent_level="$2"
shell_for_sh_tests="$3"
[ "$potential_test" = 'setup_dir' ] && return
[ "$potential_test" = 'teardown_dir' ] && return
[ "$potential_test" = 'setup' ] && return
[ "$potential_test" = 'teardown' ] && return
echo > "$stdout_file"
[ $indent_level -eq 0 ] && : > "$stdout_file"
if [ -d "$potential_test" ]
then
(
indent $indent_level
echo " ${potential_test}"
cd "$potential_test"
cd -- "$potential_test"
[ -f setup_dir ] && [ -x setup_dir ] && ./setup_dir >> "$stdout_file"
if [ -n "$ZSH_VERSION" ]; then
# avoid "no matches found: *" error when directories are empty
setopt NULL_GLOB
fi
for test in *
do
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
# $2 instead of $indent_level so it doesn't clash
recurse "${test}" $(( $2 + 1 ))
recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
exit_code=$?
if $exit_on_fail && test $exit_code -ne 0; then
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
[ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
return 1
fi
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
done
@@ -48,96 +66,159 @@ recurse() {
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
# Run the test
./"$potential_test" > "$stdout_file" 2>&1
if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
then
TEST_SHELL="$TEST_SHELL" "$shell_for_sh_tests" ./"$potential_test" > "$stdout_file" 2>&1
else
TEST_SHELL="$TEST_SHELL" ./"$potential_test" > "$stdout_file" 2>&1
fi
exit_code="$?"
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
indent $indent_level
if [ "$exit_code" = '0' ]
if [ $exit_code -eq 0 ]
then
# On success, print a '✓'
# On success, print a green '✓'
printf '\033[32m✓ \033[0m'
echo "${potential_test}"
echo "${potential_test} passed" >> "$logfile"
printf '%s\n' "${potential_test}"
printf '%s\n' "${potential_test} passed" >> "$logfile"
else
# On fail, print a red '✗'
printf '\033[31m✗ \033[0m'
echo "${potential_test}"
echo "${potential_test} failed" >> "$logfile"
printf '%s\n' "${potential_test}"
printf '%s\n' "${potential_test} failed" >> "$logfile"
printf '\033[31m' # Print output captured from failed test in red.
cat "$stdout_file"
printf '\033[0m'
fi
rm "$stdout_file"
if $exit_on_fail && test 0 -ne $exit_code; then
return 1
fi
fi
[ $indent_level -eq 0 ] && rm "$stdout_file"
}
USAGE="usage: $0 <test directory>"
has_sh_or_no_shebang_line() {
head -n 1 "$1" | grep -vqE '^#!' && return 0 # no shebang line at all
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' && return 0 # shebang line is '#!/bin/sh' or legal variations thereof
return 1
}
USAGE="usage: $0 [<options>] <test directory>"
urchin_help() {
echo
echo "$USAGE"
echo
echo '-f Force urchin to run on directories not named "test".'
echo '-h This help'
echo
echo '--xsd Output xUnit XML schema for an integration server.'
echo
echo 'Go to http://www.urchin.sh for documentation on writing tests.'
echo
cat <<EOF
$USAGE
-s <shell> Invoke test scripts that either have no shebang line at all or
have shebang line "#!/bin/sh" with the specified shell.
-e Stop running if any single test fails. This is helpful if you want
to use Urchin to run things other than tests, such as a set of
configuration scripts.
-f Force running even if the test directory's name does not
contain the word "test".
-h This help.
Go to https://github.com/tlevine/urchin for documentation on writing tests.
EOF
}
plural () {
# Make $1 a plural according to the number $2.
# If $3 is supplied, use that instead of "${1}s".
# Result is written to stdout.
if [ "$2" = 1 ]
then
printf '%s\n' "$1"
else
printf '%s\n' "${3-${1}s}"
fi
}
urchin_go() {
echo Running tests
echo Running tests at $(date +%Y-%m-%dT%H:%M:%S) | tee "$logfile"
start=$(date +%s)
echo > "$logfile"
recurse "$1" 0
# Determine the environment variable to define for test scripts
# that reflects the specified or implied shell to use for shell-code tests.
# - Set it to the shell specified via -s, if any.
# - Otherwise, use its present value, if non-empty.
# - Otherwise, default to '/bin/sh'.
if [ -n "$2" ]
then
TEST_SHELL="$2"
elif [ -z "$TEST_SHELL" ]
then
TEST_SHELL='/bin/sh'
fi
echo Done
echo $(grep -e 'passed$' "$logfile"|wc -l) tests passed.
echo $(grep -e 'failed$' "$logfile"|wc -l) tests failed.
recurse "$1" 0 "$2" # test folder -- indentation level -- [shell to invoke test scripts with]
finish=$(date +%s)
elapsed=$(($finish - $start))
echo "Done, took $elapsed $(plural second $elapsed)."
set -- $(grep -e 'passed$' "$logfile"|wc -l) $(grep -e 'failed$' "$logfile"|wc -l)
printf '%s\n' "$1 $(plural test "$1") passed."
[ $2 -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
printf '%s\n' "$2 $(plural test "$2") failed."
printf '\033[m'
return "$2"
}
urchin_molly_guard() {
{
echo
echo 'The directory on which you are running urchin is not'
echo 'called "test", so I am not running in case that'
echo 'was an accident. Use the -f flag if you really want'
echo 'to run urchin on that directory.'
echo 'The name of the directory on which you are running urchin'
echo 'does not contain the word "test", so I am not running,'
echo 'in case that was an accident. Use the -f flag if you really'
echo 'want to run urchin on that directory.'
echo
} >&2
exit 1
}
FORCE=false
shell_for_sh_tests=
force=false
exit_on_fail=false
while [ $# -gt 0 ]
do
case "$1" in
-f) FORCE=true;;
-e) exit_on_fail=true;;
-f) force=true;;
-s)
shift
shell_for_sh_tests=$1
which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
;;
-h|--help) urchin_help
exit 0;;
# --xsd) action=testsuite;;
# --) shift; break;;
-*) urchin_help 1>&2
-*) urchin_help >&2
exit 1;;
*) break;;
esac
shift
done
# Verify argument for main stuff
if [ "$#" != '1' ] || [ ! -d "$1" ]
then
[ -n "$1" ] && [ ! -d "$1" ] && echo "Not a directory: '$1'" >&2
echo "$USAGE" >&2
exit 2
fi
# Constants
logfile=$(fullpath "$1")/.urchin.log
stdout_file=$(fullpath "$1")/.urchin_stdout
# Verify argument for main stuff
if [ "$#" != '1' ] && [ ! -d "$1" ]
then
echo "$USAGE"
exit 1
fi
# Run or present the Molly guard.
if echo "$(basename "$(fullpath "$1")")" | grep test || $FORCE
if basename "$(fullpath "$1")" | grep -Fi 'test' > /dev/null || $force
then
urchin_go "$1"
urchin_go "$1" "$shell_for_sh_tests"
else
urchin_molly_guard
fi