55 Commits

Author SHA1 Message Date
Thomas Levine
56a4e5d8af introduction paragraph 2016-01-25 14:06:33 +00:00
Thomas Levine
d81b7efdc3 asciibetical 2016-01-25 13:56:33 +00:00
Thomas Levine
7f5455739d update readme
* move shall reference to the right place
* further explain cross-shell testing methods
2016-01-25 13:49:36 +00:00
Thomas Levine
48e5090091 start writing alternatives 2015-12-09 07:46:16 -05:00
Thomas Levine
5abf088eb8 add new ideas 2015-10-19 12:52:24 -04:00
Thomas Levine
c84851cb5c document the ordering of tests 2015-08-21 09:03:15 -04:00
Thomas Levine
510476f924 mention shall in the documentation 2015-07-05 17:39:38 +00:00
Thomas Levine
f01869fb97 packaging 2014-12-08 02:08:00 +00:00
Michael Klement
0385dcd86e Fixed blog link in read-me and commented out experimental -x option for now. 2014-12-05 18:50:19 -05:00
Michael Klement
2847b020b4 Make sure that CDPATH isn't set, as it causes cd to behave unpredictably - notably, it can produce output, which breaks fullpath().
Also: Improved CLI help, updated URLs in read-me, cleaned up package.json:

I've tried to clarify the intent of `-x` in the CLI help, but I haven't touched the read-me in that respect.

I don't see any benefit to `-x`:

* Just using `#/bin/sh` as the shebang line in combination with `-s <shell>` gives you the same functionality,
* When it comes to invoking scripts from _within_ test scripts, nothing can do the work for you: you consciously have to mark the invocation with _something_ to indicate that it should be controlled from the outside; it won't get any easier than `$TEST_SHELL ...`
* Finally, using a shebang line such as `#!/usr/bin/env urchin -x` is problematic for two reasons:
  * Some platforms can handle only *1* argument in a shebang line.
  * In a _package-local_ installation, `#!/usr/bin/env` may not find the Urchin executable.

I'm also not sure how the following (from `readme.md`) fits in the picture:

> It might make sense if you do this.

    export TEST_SHELL=zsh && urchin -x
    export TEST_SHELL=bash && urchin -x

(As an aside: To achieve the same thing, you don't need `export`; `TEST_SHELL=zsh urchin -x`  and `TEST_SHELL=bash urchin -x`  is the better choice.)
How does this relate to use in a  _shebang line_?

`urchin_help()` now uses a here-doc: easier to maintain, and should work in all Bourne-like shells.

`readmeFilename` removed from `package.json`:

> "The readmeFilename does not need to ever be in your actual package.json file" - npm/npm#3573
2014-12-03 09:54:15 -05:00
Michael Klement
fae24e926a Improved CLI help, updated URLs in read-me 2014-11-06 22:21:05 -05:00
Thomas Levine
2ab070c353 fix the -x documentation 2014-11-06 00:52:34 +00:00
Thomas Levine
292663dd73 contributors 2014-11-06 00:51:14 +00:00
Thomas Levine
90abc2640d update the -h help text 2014-11-05 17:49:53 +00:00
Thomas Levine
19ed3b02e8 HISTORY 2014-11-05 17:49:08 +00:00
Thomas Levine
b0429315ec urchin -sh to urchin -x 2014-11-05 17:47:36 +00:00
Thomas Levine
5e06ffb1c1 version 2014-11-05 17:47:01 +00:00
Thomas Levine
bf8c404784 fix -sh 2014-11-05 17:46:19 +00:00
Thomas Levine
bb41d5a156 zsh support 2014-11-05 17:38:22 +00:00
Thomas Levine
ed09351df6 fix a test 2014-11-05 17:28:54 +00:00
Thomas Levine
1e2d4cc8a3 remove gitHead 2014-11-05 17:27:26 +00:00
Thomas Levine
f8d921b5e5 document urchin -sh 2014-11-05 17:26:01 +00:00
Thomas Levine
1f115df222 urchin -sh 2014-11-05 17:18:31 +00:00
Thomas Levine
eafed6ac59 put history back 2014-11-05 16:54:36 +00:00
Thomas Levine
8273661440 tlevine maintainer 2014-11-05 16:53:17 +00:00
Thomas Levine
f57e99be6f we're not maintaining the history, and i don't care 2014-11-05 16:52:13 +00:00
Michael Klement
afc02582fb Fixed typo in read-me. 2014-10-18 10:41:22 -04:00
Michael Klement
2f97421164 Hint added to read-me re standalone invocation of tests that use TEST_SHELL; revision of wording in read-me; argument checking no longer treats a missing dir. argument as an explicitly specified argument. 2014-10-18 10:34:30 -04:00
Michael Klement
1e9611e732 Support for cross-shell testing added, via option -s <shell> and env. variable TEST_SHELL.
* For **tests that _source_ shell scripts**: **option `-s <shell>`** now tells urchin to invoke test scripts with the specified shell (only shebang-less and `#!/bin/sh` tests scripts).
* For **tests that _invoke_ schell scripts**: instruct users to write their tests to always **invoke via environment variable `TEST_SHELL` (e.g., `$TEST_SHELL ../foo`)**, and invoke urchin with that variable defined as needed, e.g., `TEST_SHELL=ksh urchin ./tests`; urchin defaults `TEST_SHELL` to `/bin/sh`.

See updated `readme.md` for details.
2014-10-17 17:16:12 -04:00
Michael Klement
deb77cb5de Various bug fixes and cleanup.
* bug fixes:
  * fixed directory-existence argument check
  * fixed how `.urchin_stdout` files are cleaned up
  * fixed instances of error/warning messages outputting to stdout instead of stderr
* documentation, help and error-message improvements:
  * added specific error message if the (non-option) argument is not a directory
  * improved wording of CLI help
  * readme.md: replaced obsolete URL http://www.urchin.sh with https://github.com/scraperwiki/urchin
  * readme.md: made the fact clearer that `./cross-shell-tests` only tests urchin's _own_ cross-shell compatibility
  * HISTORY: fixed typo
* formatting and logging improvements:
  * added timestamp to the beginning of log files (`.urchin.log`)
  * captured output from failed tests is now printed in *red* to draw attention
  * test summary now prints the number of failed tests in the appropriate color for instant feedback (green, if none failed; red, otherwise)
* cleanup
  * removed unused test files, simplified some tests
2014-10-16 16:37:51 -04:00
Tom Mortimer-Jones
f12636e029 Merge pull request #6 from scraperwiki/exit-code
Set the exit status to reflect pass/fail.
2014-03-17 14:04:22 +00:00
David Jones
0587a615f3 Document the non zero exit code case. 2014-03-17 14:00:30 +00:00
David Jones
b95b010c0d Set the exit status to reflect pass/fail.
For Travis compatibility, we set the exit code to non-zero when
any tests fail.

Fixes #5.
2014-03-17 13:43:13 +00:00
David Jones
98c23a2ab8 Merge pull request #4 from tlevine/link-blog-post
Link to the blog post
2013-10-14 01:20:38 -07:00
Thomas Levine
fc711672d6 alternatives 2013-10-13 16:48:42 +00:00
David Jones
15099082db 0.0.2 2013-07-26 12:53:25 +01:00
David Jones
8d53676743 Ensure we can cd into directories starting with a hyphen. Fixes #3. 2013-07-26 12:51:01 +01:00
David Jones
8913cbc195 Removing space at the end of a line. 2013-07-02 09:46:21 +01:00
David Jones
17b4629ba5 when testing, just cd into urchin 2013-07-02 09:43:59 +01:00
David Jones
85bfafffc8 /bin/sh not login shell 2013-07-02 09:43:16 +01:00
David Jones
153b04b462 Merge branch 'master' of github.com:scraperwiki/urchin 2013-07-02 09:25:49 +01:00
David Jones
1cf65a723d Merge pull request #2 from tlevine/cross-shell-test-runner
Cross shell test runner
2013-07-01 08:17:52 -07:00
David Jones
c2f4d92704 Hide fixtures directory. 2013-06-28 10:58:25 +01:00
David Jones
31da4fae92 Print date and duration. 2013-06-28 10:48:43 +01:00
David Jones
e892c565cb Removed spurious output from grep. 2013-06-28 10:47:51 +01:00
Thomas Levine
5aace72fc6 remove conflicting ksh calls 2013-06-27 11:45:45 -07:00
Thomas Levine
a749c8dae3 fix urchin call in XSD test 2013-06-27 11:45:05 -07:00
Thomas Levine
da9a0c9948 documentation 2013-06-27 11:44:19 -07:00
Thomas Levine
91a4467f3e cross-shell test runner in runtests 2013-06-27 11:33:16 -07:00
David Jones
0110a72965 remove pointless use of echo 2013-06-27 18:27:40 +01:00
David Jones
d9902c0b11 ASCII art from figlet 2013-06-26 08:22:54 +01:00
David Jones
a900722fb7 Fix bug when test is called "-n" 2013-06-26 08:15:07 +01:00
David Jones
4d3a9eddb5 npm instructions. 2013-06-21 17:14:44 +01:00
David Jones
3062e58a75 Use singular / plural form correctly. 2013-06-20 18:56:29 +01:00
David Jones
222ba59d69 Test for singular. 2013-06-20 18:43:11 +01:00
37 changed files with 413 additions and 102 deletions

33
HISTORY
View File

@@ -1,18 +1,31 @@
History HISTORY
====== -------
Version 0.0.5
---------------------
* urchin now unsets `CDPATH`.
* The documentation for `urchin -x` was removed because it was confusing.
Not done Version 0.0.4
------ ---------------------
We made the colors brighter. * Switch urchin -x to urchin -sh and fix some problems with it
* Documentation
We allowed for multiple tests to be written in a file Version 0.0.3
---------------------
General tidying
We added flags for verbosity and for Molly guards Run with different shells in three ways
Done * urchin -s
----- * $TEST_SHELL variable with $TEST_SHELL
* $TEST_SHELL variable with urchin -sh
We adjusted the input parameters so it is harder to accidentally run all executible files in your home directory. Set NULL_GLOB so zsh doesn't print a warning.
Before version 0.0.3
----------------------
We adjusted the input parameters so it is harder to accidentally run all executable files in you
r home directory.
We added directory-based indents. We added directory-based indents.

2
alternatives Normal file
View File

@@ -0,0 +1,2 @@
Totally different syntax and similar features, plus TAP output
https://github.com/sstephenson/bats

15
cross-shell-tests Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/sh
# Run urchin in a bunch of different shells,
# including a shell that isn't quite POSIX-compatible (zsh)
for shell in dash bash ksh zsh; do
if which $shell > /dev/null 2> /dev/null; then
echo
echo Running urchin tests in $shell
$shell urchin tests | tail -n 3
else
echo
echo Skipping $shell because it is not in the PATH
fi
done
echo

View File

@@ -1,26 +1,32 @@
{ {
"name": "urchin", "name": "urchin",
"version": "0.0.1", "version": "0.0.5",
"description": "Test framework for shell", "description": "Test framework for shell",
"main": "urchin", "main": "urchin",
"directories": { "directories": {
"test": "tests" "test": "tests"
}, },
"scripts": { "scripts": {
"test": "./urchin tests" "test": "export PATH=\"$PWD:$PATH\" && urchin tests"
}, },
"bin": "./urchin", "bin": "./urchin",
"repository": { "repository": {
"type": "git", "type": "git",
"url": "git://github.com/scraperwiki/urchin.git" "url": "git://github.com/tlevine/urchin.git"
}, },
"keywords": [ "keywords": [
"shell", "shell",
"test", "test",
"testing" "testing"
], ],
"author": "ScraperWiki Limited <feedback@scraperwiki.com>", "author": "Thomas Levine <_@thomaslevine.com>",
"license": "BSD", "contributors": [
"readmeFilename": "readme.md", {"name": "Thomas Levine", "email": "_@thomaslevine.com"},
"gitHead": "015052dcfd7152a0df037bef1bc5452e0c956695" {"name": "David Jones", "email": "drj@pobox.com"},
{"name": "Francis Irving", "email": "francis@flourish.org"},
{"name": "Zarino Zappia", "email": "mail@zarino.co.uk"},
{"name": "Tom Mortimer-Jones", "email": "tom@morty.co.uk"},
{"name": "Michael Klement", "email": "mklement0@gmail.com"}
],
"license": "BSD"
} }

132
readme.md
View File

@@ -1,33 +1,50 @@
, , ,_ _, , , ___, , , __ _
| | |_) / |_|, ' | |\ | __ ____________/ /_ (_)___
'\__| '| \ '\_ '| | _|_, |'\| / / / / ___/ ___/ __ \/ / __ \
` ' ` ` ' ` ' ' ` / /_/ / / / /__/ / / / / / / /
\__,_/_/ \___/_/ /_/_/_/ /_/
Urchin is a test framework for shell. It is implemented in Urchin is a file-based test harness, normally used for testing shell programs.
portable /bin/sh and should work on GNU/Linux, Mac OS X, and It is written in portable shell and should thus work on GNU/Linux, BSD
other Unix platforms. (including Mac OS X), and other Unix-like platforms.
## Try it out ## Try it out
Urchin's tests are written in Urchin, so you can run them to see what Urchin Urchin's tests are written in Urchin, so you can run them to see what Urchin
is like. Clone the repository is like. Clone the repository
git clone git://github.com/scraperwiki/urchin.git git clone git://github.com/tlevine/urchin.git
Run the tests Run the tests
cd urchin/tests cd urchin
../urchin . ./urchin tests
## Globally The above command will run the tests in your system's default
Download Urchin like so (as root) shell, /bin/sh (on recent Ubuntu this is dash, but it could be
ksh or bash on other systems); to test urchin's cross-shell compatibility,
run this:
wget -O /usr/local/bin https://raw.github.com/scraperwiki/urchin/master/urchin cd urchin
chmod +x /usr/local/bin/urchin ./cross-shell-tests
## Install
Urchin is contained in a single file, so you can install it by copying it to a
directory in your `PATH`. For example, you can run the following as root.
cd /usr/local/bin
wget https://raw.github.com/tlevine/urchin/master/urchin
chmod +x urchin
Urchin can be installed with npm too.
npm install -g urchin
Now you can run it. Now you can run it.
urchin <test directory> urchin <test directory>
Run `urchin -h` to get command-line help.
## Writing tests ## Writing tests
Make a root directory for your tests. Inside it, put executable files that Make a root directory for your tests. Inside it, put executable files that
exit `0` on success and something else on fail. Non-executable files and hidden exit `0` on success and something else on fail. Non-executable files and hidden
@@ -61,7 +78,7 @@ and directories have special meanings.
Directories are processed in a depth-first order. When a particular directory Directories are processed in a depth-first order. When a particular directory
is processed, `setup_dir` is run before everything else in the directory, including is processed, `setup_dir` is run before everything else in the directory, including
subdirectories. `teardown_dir` is run after everything else in the directory. subdirectories. `teardown_dir` is run after everything else in the directory.
A directory's `setup` file, if it exists, is run right before each test file A directory's `setup` file, if it exists, is run right before each test file
within the particular directory, and the `teardown` file is run right after. within the particular directory, and the `teardown` file is run right after.
@@ -69,3 +86,88 @@ within the particular directory, and the `teardown` file is run right after.
Files are only run if they are executable, and files beginning with `.` are Files are only run if they are executable, and files beginning with `.` are
ignored. Thus, fixtures and libraries can be included sloppily within the test ignored. Thus, fixtures and libraries can be included sloppily within the test
directory tree. The test passes if the file exits 0; otherwise, it fails. directory tree. The test passes if the file exits 0; otherwise, it fails.
Tests files and subdirectories are run in ASCIIbetical order within each
directory; that is,
urchin looks for files within a directory in the following manner.
for file in *; do
do_something_with_test_file $file
done
### Writing cross-shell compatibility tests for testing shell code
While you could write your test scripts to explicitly invoke the functionality
to test with various shells, Urchin facilitates a more flexible approach.
The specific approach depends on your test scenario:
* (a) Your test scripts _invoke_ scripts containing portable shell code.
* (b) Your scripts _source_ scripts containing portable shell code.
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
Urchin sets the `TEST_SHELL` environment variable so that you may change the
shell with which your tests call other shell programs. To run your test
scripts in multiple shells you must call `$TEST_SHELL` in your tests and then
run urchin with the appropriate option.
In your test scripts, invoke the shell scripts to test via the shell
specified in environment variable `TEST_SHELL` rather than directly;
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
On invocation of Urchin, prepend a definition of environment variable
`TEST_SHELL` specifying the shell to test with, e.g.,
TEST_SHELL=zsh urchin ./tests
To test with multiple shells in sequence, use something like:
for shell in sh bash ksh zsh; do
TEST_SHELL=$shell urchin ./tests
done
If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
scripts can rely on `$TEST_SHELL` always containing a value when Urchin runs
them.
That said, we still recommand that you account for the possibility that
`$TEST_SHELL` does not contain a value so that you may run your test scripts
without Urchin. Supporting this case is very simple; when you invoke scripts
that happen to be in the current directory, be sure to use the prefix `./`,
e.g., `$TEST_SHELL ./baz` rather than `$TEST_SHELL baz`.
#### (b) Cross-shell tests with test scripts that _source_ shell scripts
If you _source_ shell code in your test scripts, it is the test scripts
themselves that must be run with the shell specified.
Urchin supports the `-s <shell>` option, which instructs
Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`.
(In addition, Urchin sets environment variable `TEST_SHELL` to the specified
shell.)
Note that only test scripts that either have no shebang line at all or
have shebang line `#!/bin/sh` are invoked with the specified shell.
This allows non-shell test scripts or test scripts for specific
shells to coexist with those whose invocation should be controlled by `-s`.
To test with multiple shells in sequence, use something like:
for shell in sh bash ksh zsh; do
urchin -s $shell ./tests
done
Also consider using [shall](https://github.com/mklement0/shall).
It does something similar, but the interface may be more intuitive.
#!/usr/bin/env shall
echo This is a test file.
## Alternatives to Urchin
Alternatives to Urchin are discussed in
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
## Ideas for new features
* Support [Nagios plugins](https://nagios-plugins.org/doc/guidelines.html)
* Stop running if a test fails so one can use Urchin as a
[setup framework](https://github.com/tlevine/urchin/issues/16).

1
tests/.gitignore vendored
View File

@@ -1 +1,2 @@
.urchin.log .urchin.log
.urchin_stdout

1
tests/.print-arg-3 Normal file
View File

@@ -0,0 +1 @@
echo $3

View File

@@ -0,0 +1,6 @@
#!/bin/sh
cd ..
export CDPATH=$PWD
./urchin -f 'tests/urchin exit code' >/dev/null

View File

@@ -0,0 +1,3 @@
#!/bin/sh
../../urchin -h | grep -- -s

View File

@@ -1,3 +0,0 @@
#!/bin/sh
../../urchin -h|grep -i xsd

View File

@@ -0,0 +1 @@
false

View File

@@ -0,0 +1,2 @@
#!/bin/sh
../../urchin .test-one | grep '1 test failed'

View File

@@ -0,0 +1,2 @@
#!/bin/sh
../../urchin .test-one | grep '1 test passed'

View File

@@ -0,0 +1,7 @@
#!/bin/sh
# Assuming that urchin was invoked with `TEST_SHELL=bash urchin ...`, $TEST_SHELL should contain 'bash'.
echo "\$TEST_SHELL: $TEST_SHELL"
[ "$TEST_SHELL" = 'bash' ]

View File

@@ -0,0 +1,4 @@
#!/bin/sh
# Invoke a simple test command with $TEST_SHELL as the executable.
[ "$($TEST_SHELL -c 'echo $0')" = "$TEST_SHELL" ]

View File

@@ -0,0 +1,5 @@
#!/usr/bin/awk -f
# This script will only succeed if it is indeed processed by awk.
BEGIN { print "ok" }

View File

@@ -0,0 +1,10 @@
#!/bin/sh
# Assuming that urchin was invoked with `-s bash`, this script should be being run with bash.
this_shell=$(ps -o comm= -p $$ && :)
echo "Running shell: $this_shell"
[ "$this_shell" = 'bash' ]

View File

@@ -0,0 +1,10 @@
# By design, this file has no shebang line.
# Assuming that urchin was invoked with `-s bash`, this script should be being run with bash.
this_shell=$(ps -o comm= -p $$ && :)
echo "Running shell: $this_shell"
[ "$this_shell" = 'bash' ]

View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Assuming that urchin was invoked with `-s bash`, $TEST_SHELL should contain 'bash'.
echo "Running shell: $(ps -o comm= -p $$ && :)"
echo "\$TEST_SHELL: $TEST_SHELL"
[ "$TEST_SHELL" = 'bash' ]

View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
# (for test scripts that want to invoke shell scripts with a specified shell).
which bash 2>/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 1; }
# Test if $TEST_SHELL, when placed in urchin's environment, is passed through to the test scripts.
TEST_SHELL=bash ../../urchin ./.test-TEST_SHELL-passed-through

View File

@@ -0,0 +1,8 @@
#!/bin/sh
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
# (for test scripts that want to invoke shell scripts with a specified shell).
# Test if $TEST_SHELL - if *defined, but empty* - is exported with value '/bin/sh' by urchin
# and thus has that value inside the scripts.
TEST_SHELL= ../../urchin ./.test-TEST_SHELL-undefined_or_empty

View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
# (for test scripts that want to invoke shell scripts with a specified shell).
# Test if $TEST_SHELL - if *undefined* - is exported with value '/bin/sh' by urchin
# and thus has that value inside test scripts.
unset -v TEST_SHELL
../../urchin ./.test-TEST_SHELL-undefined_or_empty

View File

@@ -0,0 +1,8 @@
#!/bin/sh
# Tests the `-s <shell> option, which invokes shebang-less and sh-shebang-line test scripts with the specified shell (for testing *sourced* shell code).
which bash >/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 2; }
which /usr/bin/awk >/dev/null || { echo "Cannot test -s option: /usr/bin/awk not found." >&2; exit 2; }
../../urchin -s bash ./.test-run-by-specified-shell

View File

@@ -1,4 +1,3 @@
#!/bin/sh #!/bin/sh
mkdir /tmp/urchintmp ! ../../urchin ./.chainsaw
! ../../urchin /tmp/urchintmp

View File

@@ -1,3 +1,3 @@
#!/bin/sh #!/bin/sh
grep blah fixtures/bladyblah > /dev/null grep blah .fixtures/bladyblah > /dev/null

View File

@@ -1,3 +0,0 @@
#!/bin/sh
exit 1

View File

@@ -1,3 +0,0 @@
#!/bin/sh
../../../urchin .fixture | grep 'http://www.w3.org/2001/XMLSchema'

View File

@@ -1,3 +0,0 @@
#!/bin/sh
export variable_set_in_setup='value'

View File

@@ -0,0 +1,3 @@
#!/bin/sh
test c = $(../urchin -x .print-arg-3 a 'b b b b' c d e)

View File

View File

@@ -0,0 +1,13 @@
#!/bin/sh
# Borrow this test which has a failure in it (3 in fact), and
# run it using urchin:
../../urchin "../Counts should be kept of successes and failures./.test" > /dev/null
# Now invert the result, we want _this_ test to fail if the exit
# code was 0.
case $? in
0) exit 8;;
*) exit 0;;
esac

View File

@@ -0,0 +1 @@
../../urchin .test > /dev/null

181
urchin
View File

@@ -1,8 +1,12 @@
#!/bin/sh #!/bin/sh
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably - notably, it can produce output,
# which breaks fullpath().
unset CDPATH
fullpath() { fullpath() {
( (
cd "$1" cd -- "$1"
pwd pwd
) )
} }
@@ -15,27 +19,34 @@ indent() {
recurse() { recurse() {
potential_test="$1" potential_test="$1"
indent_level="$2" indent_level="$2"
shell_for_sh_tests="$3"
[ "$potential_test" = 'setup_dir' ] && return [ "$potential_test" = 'setup_dir' ] && return
[ "$potential_test" = 'teardown_dir' ] && return [ "$potential_test" = 'teardown_dir' ] && return
[ "$potential_test" = 'setup' ] && return [ "$potential_test" = 'setup' ] && return
[ "$potential_test" = 'teardown' ] && return [ "$potential_test" = 'teardown' ] && return
echo > "$stdout_file" [ $indent_level -eq 0 ] && : > "$stdout_file"
if [ -d "$potential_test" ] if [ -d "$potential_test" ]
then then
( (
indent $indent_level indent $indent_level
echo " ${potential_test}" echo " ${potential_test}"
cd "$potential_test" cd -- "$potential_test"
[ -f setup_dir ] && [ -x setup_dir ] && ./setup_dir >> "$stdout_file" [ -f setup_dir ] && [ -x setup_dir ] && ./setup_dir >> "$stdout_file"
if [ -n "$ZSH_VERSION" ]; then
# avoid "no matches found: *" error when directories are empty
setopt NULL_GLOB
fi
for test in * for test in *
do do
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file" [ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
# $2 instead of $indent_level so it doesn't clash # $2 instead of $indent_level so it doesn't clash
recurse "${test}" $(( $2 + 1 )) recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file" [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
done done
@@ -48,96 +59,160 @@ recurse() {
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file" [ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
# Run the test # Run the test
./"$potential_test" > "$stdout_file" 2>&1 if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
then
TEST_SHELL="$TEST_SHELL" "$shell_for_sh_tests" ./"$potential_test" > "$stdout_file" 2>&1
else
TEST_SHELL="$TEST_SHELL" ./"$potential_test" > "$stdout_file" 2>&1
fi
exit_code="$?" exit_code="$?"
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file" [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
indent $indent_level indent $indent_level
if [ "$exit_code" = '0' ] if [ $exit_code -eq 0 ]
then then
# On success, print a '✓' # On success, print a green '✓'
printf '\033[32m✓ \033[0m' printf '\033[32m✓ \033[0m'
echo "${potential_test}" printf '%s\n' "${potential_test}"
echo "${potential_test} passed" >> "$logfile" printf '%s\n' "${potential_test} passed" >> "$logfile"
else else
# On fail, print a red '✗' # On fail, print a red '✗'
printf '\033[31m✗ \033[0m' printf '\033[31m✗ \033[0m'
echo "${potential_test}" printf '%s\n' "${potential_test}"
echo "${potential_test} failed" >> "$logfile" printf '%s\n' "${potential_test} failed" >> "$logfile"
printf '\033[31m' # Print output captured from failed test in red.
cat "$stdout_file" cat "$stdout_file"
printf '\033[0m'
fi fi
rm "$stdout_file" fi
[ $indent_level -eq 0 ] && rm "$stdout_file"
}
has_sh_or_no_shebang_line() {
head -n 1 "$1" | grep -vqE '^#!' && return 0 # no shebang line at all
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' && return 0 # shebang line is '#!/bin/sh' or legal variations thereof
return 1
}
USAGE="usage: $0 [<options>] <test directory>"
urchin_help() {
cat <<EOF
$USAGE
-s <shell> Invoke test scripts that either have no shebang line at all or
have shebang line "#!/bin/sh" with the specified shell.
-f Force running even if the test directory's name does not
contain the word "test".
-h This help.
Go to https://github.com/tlevine/urchin for documentation on writing tests.
EOF
# [Experimental -x option left undocumented for now.]
# -x [Experimental; not meant for direct invocation, but for use in
# the shebang line of test scripts]
# Run with "\$TEST_SHELL", falling back on /bin/sh.
}
plural () {
# Make $1 a plural according to the number $2.
# If $3 is supplied, use that instead of "${1}s".
# Result is written to stdout.
if [ "$2" = 1 ]
then
printf '%s\n' "$1"
else
printf '%s\n' "${3-${1}s}"
fi fi
} }
USAGE="usage: $0 <test directory>"
urchin_help() {
echo
echo "$USAGE"
echo
echo '-f Force urchin to run on directories not named "test".'
echo '-h This help'
echo
echo '--xsd Output xUnit XML schema for an integration server.'
echo
echo 'Go to http://www.urchin.sh for documentation on writing tests.'
echo
}
urchin_go() { urchin_go() {
echo Running tests echo Running tests at $(date +%Y-%m-%dT%H:%M:%S) | tee "$logfile"
start=$(date +%s)
echo > "$logfile" # Determine the environment variable to define for test scripts
recurse "$1" 0 # that reflects the specified or implied shell to use for shell-code tests.
# - Set it to the shell specified via -s, if any.
# - Otherwise, use its present value, if non-empty.
# - Otherwise, default to '/bin/sh'.
if [ -n "$2" ]
then
TEST_SHELL="$2"
elif [ -z "$TEST_SHELL" ]
then
TEST_SHELL='/bin/sh'
fi
echo Done recurse "$1" 0 "$2" # test folder -- indentation level -- [shell to invoke test scripts with]
echo $(grep -e 'passed$' "$logfile"|wc -l) tests passed.
echo $(grep -e 'failed$' "$logfile"|wc -l) tests failed. finish=$(date +%s)
elapsed=$(($finish - $start))
echo "Done, took $elapsed $(plural second $elapsed)."
set -- $(grep -e 'passed$' "$logfile"|wc -l) $(grep -e 'failed$' "$logfile"|wc -l)
printf '%s\n' "$1 $(plural test "$1") passed."
[ $2 -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
printf '%s\n' "$2 $(plural test "$2") failed."
printf '\033[m'
return "$2"
} }
urchin_molly_guard() { urchin_molly_guard() {
echo {
echo 'The directory on which you are running urchin is not' echo
echo 'called "test", so I am not running in case that' echo 'The name of the directory on which you are running urchin'
echo 'was an accident. Use the -f flag if you really want' echo 'does not contain the word "test", so I am not running,'
echo 'to run urchin on that directory.' echo 'in case that was an accident. Use the -f flag if you really'
echo echo 'want to run urchin on that directory.'
echo
} >&2
exit 1 exit 1
} }
FORCE=false shell_for_sh_tests=
force=false
while [ $# -gt 0 ] while [ $# -gt 0 ]
do do
case "$1" in case "$1" in
-f) FORCE=true;; -f) force=true;;
-s)
shift
shell_for_sh_tests=$1
which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
;;
-x) # [EXPERIMENTAL; UNDOCUMENTED FOR NOW] `urchin -x <test-script>` in a test script's shebang line is equivalent to invoking that script with `"$TEST_SHELL" <test-script>`
shift
urchinsh=${TEST_SHELL:-/bin/sh}
"$urchinsh" "$@"
exit $?;;
-h|--help) urchin_help -h|--help) urchin_help
exit 0;; exit 0;;
# --xsd) action=testsuite;; -*) urchin_help >&2
# --) shift; break;;
-*) urchin_help 1>&2
exit 1;; exit 1;;
*) break;; *) break;;
esac esac
shift shift
done done
# Verify argument for main stuff
if [ "$#" != '1' ] || [ ! -d "$1" ]
then
[ -n "$1" ] && [ ! -d "$1" ] && echo "Not a directory: '$1'" >&2
echo "$USAGE" >&2
exit 2
fi
# Constants # Constants
logfile=$(fullpath "$1")/.urchin.log logfile=$(fullpath "$1")/.urchin.log
stdout_file=$(fullpath "$1")/.urchin_stdout stdout_file=$(fullpath "$1")/.urchin_stdout
# Verify argument for main stuff
if [ "$#" != '1' ] && [ ! -d "$1" ]
then
echo "$USAGE"
exit 1
fi
# Run or present the Molly guard. # Run or present the Molly guard.
if echo "$(basename "$(fullpath "$1")")" | grep test || $FORCE if basename "$(fullpath "$1")" | grep -Fi 'test' > /dev/null || $force
then then
urchin_go "$1" urchin_go "$1" "$shell_for_sh_tests"
else else
urchin_molly_guard urchin_molly_guard
fi fi