55 Commits

Author SHA1 Message Date
Thomas Levine
56a4e5d8af introduction paragraph 2016-01-25 14:06:33 +00:00
Thomas Levine
d81b7efdc3 asciibetical 2016-01-25 13:56:33 +00:00
Thomas Levine
7f5455739d update readme
* move shall reference to the right place
* further explain cross-shell testing methods
2016-01-25 13:49:36 +00:00
Thomas Levine
48e5090091 start writing alternatives 2015-12-09 07:46:16 -05:00
Thomas Levine
5abf088eb8 add new ideas 2015-10-19 12:52:24 -04:00
Thomas Levine
c84851cb5c document the ordering of tests 2015-08-21 09:03:15 -04:00
Thomas Levine
510476f924 mention shall in the documentation 2015-07-05 17:39:38 +00:00
Thomas Levine
f01869fb97 packaging 2014-12-08 02:08:00 +00:00
Michael Klement
0385dcd86e Fixed blog link in read-me and commented out experimental -x option for now. 2014-12-05 18:50:19 -05:00
Michael Klement
2847b020b4 Make sure that CDPATH isn't set, as it causes cd to behave unpredictably - notably, it can produce output, which breaks fullpath().
Also: Improved CLI help, updated URLs in read-me, cleaned up package.json:

I've tried to clarify the intent of `-x` in the CLI help, but I haven't touched the read-me in that respect.

I don't see any benefit to `-x`:

* Just using `#/bin/sh` as the shebang line in combination with `-s <shell>` gives you the same functionality,
* When it comes to invoking scripts from _within_ test scripts, nothing can do the work for you: you consciously have to mark the invocation with _something_ to indicate that it should be controlled from the outside; it won't get any easier than `$TEST_SHELL ...`
* Finally, using a shebang line such as `#!/usr/bin/env urchin -x` is problematic for two reasons:
  * Some platforms can handle only *1* argument in a shebang line.
  * In a _package-local_ installation, `#!/usr/bin/env` may not find the Urchin executable.

I'm also not sure how the following (from `readme.md`) fits in the picture:

> It might make sense if you do this.

    export TEST_SHELL=zsh && urchin -x
    export TEST_SHELL=bash && urchin -x

(As an aside: To achieve the same thing, you don't need `export`; `TEST_SHELL=zsh urchin -x`  and `TEST_SHELL=bash urchin -x`  is the better choice.)
How does this relate to use in a  _shebang line_?

`urchin_help()` now uses a here-doc: easier to maintain, and should work in all Bourne-like shells.

`readmeFilename` removed from `package.json`:

> "The readmeFilename does not need to ever be in your actual package.json file" - npm/npm#3573
2014-12-03 09:54:15 -05:00
Michael Klement
fae24e926a Improved CLI help, updated URLs in read-me 2014-11-06 22:21:05 -05:00
Thomas Levine
2ab070c353 fix the -x documentation 2014-11-06 00:52:34 +00:00
Thomas Levine
292663dd73 contributors 2014-11-06 00:51:14 +00:00
Thomas Levine
90abc2640d update the -h help text 2014-11-05 17:49:53 +00:00
Thomas Levine
19ed3b02e8 HISTORY 2014-11-05 17:49:08 +00:00
Thomas Levine
b0429315ec urchin -sh to urchin -x 2014-11-05 17:47:36 +00:00
Thomas Levine
5e06ffb1c1 version 2014-11-05 17:47:01 +00:00
Thomas Levine
bf8c404784 fix -sh 2014-11-05 17:46:19 +00:00
Thomas Levine
bb41d5a156 zsh support 2014-11-05 17:38:22 +00:00
Thomas Levine
ed09351df6 fix a test 2014-11-05 17:28:54 +00:00
Thomas Levine
1e2d4cc8a3 remove gitHead 2014-11-05 17:27:26 +00:00
Thomas Levine
f8d921b5e5 document urchin -sh 2014-11-05 17:26:01 +00:00
Thomas Levine
1f115df222 urchin -sh 2014-11-05 17:18:31 +00:00
Thomas Levine
eafed6ac59 put history back 2014-11-05 16:54:36 +00:00
Thomas Levine
8273661440 tlevine maintainer 2014-11-05 16:53:17 +00:00
Thomas Levine
f57e99be6f we're not maintaining the history, and i don't care 2014-11-05 16:52:13 +00:00
Michael Klement
afc02582fb Fixed typo in read-me. 2014-10-18 10:41:22 -04:00
Michael Klement
2f97421164 Hint added to read-me re standalone invocation of tests that use TEST_SHELL; revision of wording in read-me; argument checking no longer treats a missing dir. argument as an explicitly specified argument. 2014-10-18 10:34:30 -04:00
Michael Klement
1e9611e732 Support for cross-shell testing added, via option -s <shell> and env. variable TEST_SHELL.
* For **tests that _source_ shell scripts**: **option `-s <shell>`** now tells urchin to invoke test scripts with the specified shell (only shebang-less and `#!/bin/sh` tests scripts).
* For **tests that _invoke_ schell scripts**: instruct users to write their tests to always **invoke via environment variable `TEST_SHELL` (e.g., `$TEST_SHELL ../foo`)**, and invoke urchin with that variable defined as needed, e.g., `TEST_SHELL=ksh urchin ./tests`; urchin defaults `TEST_SHELL` to `/bin/sh`.

See updated `readme.md` for details.
2014-10-17 17:16:12 -04:00
Michael Klement
deb77cb5de Various bug fixes and cleanup.
* bug fixes:
  * fixed directory-existence argument check
  * fixed how `.urchin_stdout` files are cleaned up
  * fixed instances of error/warning messages outputting to stdout instead of stderr
* documentation, help and error-message improvements:
  * added specific error message if the (non-option) argument is not a directory
  * improved wording of CLI help
  * readme.md: replaced obsolete URL http://www.urchin.sh with https://github.com/scraperwiki/urchin
  * readme.md: made the fact clearer that `./cross-shell-tests` only tests urchin's _own_ cross-shell compatibility
  * HISTORY: fixed typo
* formatting and logging improvements:
  * added timestamp to the beginning of log files (`.urchin.log`)
  * captured output from failed tests is now printed in *red* to draw attention
  * test summary now prints the number of failed tests in the appropriate color for instant feedback (green, if none failed; red, otherwise)
* cleanup
  * removed unused test files, simplified some tests
2014-10-16 16:37:51 -04:00
Tom Mortimer-Jones
f12636e029 Merge pull request #6 from scraperwiki/exit-code
Set the exit status to reflect pass/fail.
2014-03-17 14:04:22 +00:00
David Jones
0587a615f3 Document the non zero exit code case. 2014-03-17 14:00:30 +00:00
David Jones
b95b010c0d Set the exit status to reflect pass/fail.
For Travis compatibility, we set the exit code to non-zero when
any tests fail.

Fixes #5.
2014-03-17 13:43:13 +00:00
David Jones
98c23a2ab8 Merge pull request #4 from tlevine/link-blog-post
Link to the blog post
2013-10-14 01:20:38 -07:00
Thomas Levine
fc711672d6 alternatives 2013-10-13 16:48:42 +00:00
David Jones
15099082db 0.0.2 2013-07-26 12:53:25 +01:00
David Jones
8d53676743 Ensure we can cd into directories starting with a hyphen. Fixes #3. 2013-07-26 12:51:01 +01:00
David Jones
8913cbc195 Removing space at the end of a line. 2013-07-02 09:46:21 +01:00
David Jones
17b4629ba5 when testing, just cd into urchin 2013-07-02 09:43:59 +01:00
David Jones
85bfafffc8 /bin/sh not login shell 2013-07-02 09:43:16 +01:00
David Jones
153b04b462 Merge branch 'master' of github.com:scraperwiki/urchin 2013-07-02 09:25:49 +01:00
David Jones
1cf65a723d Merge pull request #2 from tlevine/cross-shell-test-runner
Cross shell test runner
2013-07-01 08:17:52 -07:00
David Jones
c2f4d92704 Hide fixtures directory. 2013-06-28 10:58:25 +01:00
David Jones
31da4fae92 Print date and duration. 2013-06-28 10:48:43 +01:00
David Jones
e892c565cb Removed spurious output from grep. 2013-06-28 10:47:51 +01:00
Thomas Levine
5aace72fc6 remove conflicting ksh calls 2013-06-27 11:45:45 -07:00
Thomas Levine
a749c8dae3 fix urchin call in XSD test 2013-06-27 11:45:05 -07:00
Thomas Levine
da9a0c9948 documentation 2013-06-27 11:44:19 -07:00
Thomas Levine
91a4467f3e cross-shell test runner in runtests 2013-06-27 11:33:16 -07:00
David Jones
0110a72965 remove pointless use of echo 2013-06-27 18:27:40 +01:00
David Jones
d9902c0b11 ASCII art from figlet 2013-06-26 08:22:54 +01:00
David Jones
a900722fb7 Fix bug when test is called "-n" 2013-06-26 08:15:07 +01:00
David Jones
4d3a9eddb5 npm instructions. 2013-06-21 17:14:44 +01:00
David Jones
3062e58a75 Use singular / plural form correctly. 2013-06-20 18:56:29 +01:00
David Jones
222ba59d69 Test for singular. 2013-06-20 18:43:11 +01:00
37 changed files with 413 additions and 102 deletions

33
HISTORY
View File

@@ -1,18 +1,31 @@
History
======
HISTORY
-------
Version 0.0.5
---------------------
* urchin now unsets `CDPATH`.
* The documentation for `urchin -x` was removed because it was confusing.
Not done
------
We made the colors brighter.
Version 0.0.4
---------------------
* Switch urchin -x to urchin -sh and fix some problems with it
* Documentation
We allowed for multiple tests to be written in a file
Version 0.0.3
---------------------
General tidying
We added flags for verbosity and for Molly guards
Run with different shells in three ways
Done
-----
* urchin -s
* $TEST_SHELL variable with $TEST_SHELL
* $TEST_SHELL variable with urchin -sh
We adjusted the input parameters so it is harder to accidentally run all executible files in your home directory.
Set NULL_GLOB so zsh doesn't print a warning.
Before version 0.0.3
----------------------
We adjusted the input parameters so it is harder to accidentally run all executable files in you
r home directory.
We added directory-based indents.

2
alternatives Normal file
View File

@@ -0,0 +1,2 @@
Totally different syntax and similar features, plus TAP output
https://github.com/sstephenson/bats

15
cross-shell-tests Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/sh
# Run urchin in a bunch of different shells,
# including a shell that isn't quite POSIX-compatible (zsh)
for shell in dash bash ksh zsh; do
if which $shell > /dev/null 2> /dev/null; then
echo
echo Running urchin tests in $shell
$shell urchin tests | tail -n 3
else
echo
echo Skipping $shell because it is not in the PATH
fi
done
echo

View File

@@ -1,26 +1,32 @@
{
"name": "urchin",
"version": "0.0.1",
"version": "0.0.5",
"description": "Test framework for shell",
"main": "urchin",
"directories": {
"test": "tests"
},
"scripts": {
"test": "./urchin tests"
"test": "export PATH=\"$PWD:$PATH\" && urchin tests"
},
"bin": "./urchin",
"repository": {
"type": "git",
"url": "git://github.com/scraperwiki/urchin.git"
"url": "git://github.com/tlevine/urchin.git"
},
"keywords": [
"shell",
"test",
"testing"
],
"author": "ScraperWiki Limited <feedback@scraperwiki.com>",
"license": "BSD",
"readmeFilename": "readme.md",
"gitHead": "015052dcfd7152a0df037bef1bc5452e0c956695"
"author": "Thomas Levine <_@thomaslevine.com>",
"contributors": [
{"name": "Thomas Levine", "email": "_@thomaslevine.com"},
{"name": "David Jones", "email": "drj@pobox.com"},
{"name": "Francis Irving", "email": "francis@flourish.org"},
{"name": "Zarino Zappia", "email": "mail@zarino.co.uk"},
{"name": "Tom Mortimer-Jones", "email": "tom@morty.co.uk"},
{"name": "Michael Klement", "email": "mklement0@gmail.com"}
],
"license": "BSD"
}

130
readme.md
View File

@@ -1,33 +1,50 @@
, , ,_ _, , , ___, , ,
| | |_) / |_|, ' | |\ |
'\__| '| \ '\_ '| | _|_, |'\|
` ' ` ` ' ` ' ' `
__ _
__ ____________/ /_ (_)___
/ / / / ___/ ___/ __ \/ / __ \
/ /_/ / / / /__/ / / / / / / /
\__,_/_/ \___/_/ /_/_/_/ /_/
Urchin is a test framework for shell. It is implemented in
portable /bin/sh and should work on GNU/Linux, Mac OS X, and
other Unix platforms.
Urchin is a file-based test harness, normally used for testing shell programs.
It is written in portable shell and should thus work on GNU/Linux, BSD
(including Mac OS X), and other Unix-like platforms.
## Try it out
Urchin's tests are written in Urchin, so you can run them to see what Urchin
is like. Clone the repository
git clone git://github.com/scraperwiki/urchin.git
git clone git://github.com/tlevine/urchin.git
Run the tests
cd urchin/tests
../urchin .
cd urchin
./urchin tests
## Globally
Download Urchin like so (as root)
The above command will run the tests in your system's default
shell, /bin/sh (on recent Ubuntu this is dash, but it could be
ksh or bash on other systems); to test urchin's cross-shell compatibility,
run this:
wget -O /usr/local/bin https://raw.github.com/scraperwiki/urchin/master/urchin
chmod +x /usr/local/bin/urchin
cd urchin
./cross-shell-tests
## Install
Urchin is contained in a single file, so you can install it by copying it to a
directory in your `PATH`. For example, you can run the following as root.
cd /usr/local/bin
wget https://raw.github.com/tlevine/urchin/master/urchin
chmod +x urchin
Urchin can be installed with npm too.
npm install -g urchin
Now you can run it.
urchin <test directory>
Run `urchin -h` to get command-line help.
## Writing tests
Make a root directory for your tests. Inside it, put executable files that
exit `0` on success and something else on fail. Non-executable files and hidden
@@ -69,3 +86,88 @@ within the particular directory, and the `teardown` file is run right after.
Files are only run if they are executable, and files beginning with `.` are
ignored. Thus, fixtures and libraries can be included sloppily within the test
directory tree. The test passes if the file exits 0; otherwise, it fails.
Tests files and subdirectories are run in ASCIIbetical order within each
directory; that is,
urchin looks for files within a directory in the following manner.
for file in *; do
do_something_with_test_file $file
done
### Writing cross-shell compatibility tests for testing shell code
While you could write your test scripts to explicitly invoke the functionality
to test with various shells, Urchin facilitates a more flexible approach.
The specific approach depends on your test scenario:
* (a) Your test scripts _invoke_ scripts containing portable shell code.
* (b) Your scripts _source_ scripts containing portable shell code.
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
Urchin sets the `TEST_SHELL` environment variable so that you may change the
shell with which your tests call other shell programs. To run your test
scripts in multiple shells you must call `$TEST_SHELL` in your tests and then
run urchin with the appropriate option.
In your test scripts, invoke the shell scripts to test via the shell
specified in environment variable `TEST_SHELL` rather than directly;
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
On invocation of Urchin, prepend a definition of environment variable
`TEST_SHELL` specifying the shell to test with, e.g.,
TEST_SHELL=zsh urchin ./tests
To test with multiple shells in sequence, use something like:
for shell in sh bash ksh zsh; do
TEST_SHELL=$shell urchin ./tests
done
If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
scripts can rely on `$TEST_SHELL` always containing a value when Urchin runs
them.
That said, we still recommand that you account for the possibility that
`$TEST_SHELL` does not contain a value so that you may run your test scripts
without Urchin. Supporting this case is very simple; when you invoke scripts
that happen to be in the current directory, be sure to use the prefix `./`,
e.g., `$TEST_SHELL ./baz` rather than `$TEST_SHELL baz`.
#### (b) Cross-shell tests with test scripts that _source_ shell scripts
If you _source_ shell code in your test scripts, it is the test scripts
themselves that must be run with the shell specified.
Urchin supports the `-s <shell>` option, which instructs
Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`.
(In addition, Urchin sets environment variable `TEST_SHELL` to the specified
shell.)
Note that only test scripts that either have no shebang line at all or
have shebang line `#!/bin/sh` are invoked with the specified shell.
This allows non-shell test scripts or test scripts for specific
shells to coexist with those whose invocation should be controlled by `-s`.
To test with multiple shells in sequence, use something like:
for shell in sh bash ksh zsh; do
urchin -s $shell ./tests
done
Also consider using [shall](https://github.com/mklement0/shall).
It does something similar, but the interface may be more intuitive.
#!/usr/bin/env shall
echo This is a test file.
## Alternatives to Urchin
Alternatives to Urchin are discussed in
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
## Ideas for new features
* Support [Nagios plugins](https://nagios-plugins.org/doc/guidelines.html)
* Stop running if a test fails so one can use Urchin as a
[setup framework](https://github.com/tlevine/urchin/issues/16).

1
tests/.gitignore vendored
View File

@@ -1 +1,2 @@
.urchin.log
.urchin_stdout

1
tests/.print-arg-3 Normal file
View File

@@ -0,0 +1 @@
echo $3

View File

@@ -0,0 +1,6 @@
#!/bin/sh
cd ..
export CDPATH=$PWD
./urchin -f 'tests/urchin exit code' >/dev/null

View File

@@ -0,0 +1,3 @@
#!/bin/sh
../../urchin -h | grep -- -s

View File

@@ -1,3 +0,0 @@
#!/bin/sh
../../urchin -h|grep -i xsd

View File

@@ -0,0 +1 @@
false

View File

@@ -0,0 +1,2 @@
#!/bin/sh
../../urchin .test-one | grep '1 test failed'

View File

@@ -0,0 +1,2 @@
#!/bin/sh
../../urchin .test-one | grep '1 test passed'

View File

@@ -0,0 +1,7 @@
#!/bin/sh
# Assuming that urchin was invoked with `TEST_SHELL=bash urchin ...`, $TEST_SHELL should contain 'bash'.
echo "\$TEST_SHELL: $TEST_SHELL"
[ "$TEST_SHELL" = 'bash' ]

View File

@@ -0,0 +1,4 @@
#!/bin/sh
# Invoke a simple test command with $TEST_SHELL as the executable.
[ "$($TEST_SHELL -c 'echo $0')" = "$TEST_SHELL" ]

View File

@@ -0,0 +1,5 @@
#!/usr/bin/awk -f
# This script will only succeed if it is indeed processed by awk.
BEGIN { print "ok" }

View File

@@ -0,0 +1,10 @@
#!/bin/sh
# Assuming that urchin was invoked with `-s bash`, this script should be being run with bash.
this_shell=$(ps -o comm= -p $$ && :)
echo "Running shell: $this_shell"
[ "$this_shell" = 'bash' ]

View File

@@ -0,0 +1,10 @@
# By design, this file has no shebang line.
# Assuming that urchin was invoked with `-s bash`, this script should be being run with bash.
this_shell=$(ps -o comm= -p $$ && :)
echo "Running shell: $this_shell"
[ "$this_shell" = 'bash' ]

View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Assuming that urchin was invoked with `-s bash`, $TEST_SHELL should contain 'bash'.
echo "Running shell: $(ps -o comm= -p $$ && :)"
echo "\$TEST_SHELL: $TEST_SHELL"
[ "$TEST_SHELL" = 'bash' ]

View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
# (for test scripts that want to invoke shell scripts with a specified shell).
which bash 2>/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 1; }
# Test if $TEST_SHELL, when placed in urchin's environment, is passed through to the test scripts.
TEST_SHELL=bash ../../urchin ./.test-TEST_SHELL-passed-through

View File

@@ -0,0 +1,8 @@
#!/bin/sh
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
# (for test scripts that want to invoke shell scripts with a specified shell).
# Test if $TEST_SHELL - if *defined, but empty* - is exported with value '/bin/sh' by urchin
# and thus has that value inside the scripts.
TEST_SHELL= ../../urchin ./.test-TEST_SHELL-undefined_or_empty

View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
# (for test scripts that want to invoke shell scripts with a specified shell).
# Test if $TEST_SHELL - if *undefined* - is exported with value '/bin/sh' by urchin
# and thus has that value inside test scripts.
unset -v TEST_SHELL
../../urchin ./.test-TEST_SHELL-undefined_or_empty

View File

@@ -0,0 +1,8 @@
#!/bin/sh
# Tests the `-s <shell> option, which invokes shebang-less and sh-shebang-line test scripts with the specified shell (for testing *sourced* shell code).
which bash >/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 2; }
which /usr/bin/awk >/dev/null || { echo "Cannot test -s option: /usr/bin/awk not found." >&2; exit 2; }
../../urchin -s bash ./.test-run-by-specified-shell

View File

@@ -1,3 +1,3 @@
#!/bin/sh
grep blah fixtures/bladyblah > /dev/null
grep blah .fixtures/bladyblah > /dev/null

View File

@@ -1,3 +0,0 @@
#!/bin/sh
exit 1

View File

@@ -1,3 +0,0 @@
#!/bin/sh
../../../urchin .fixture | grep 'http://www.w3.org/2001/XMLSchema'

View File

@@ -1,3 +0,0 @@
#!/bin/sh
export variable_set_in_setup='value'

View File

@@ -0,0 +1,3 @@
#!/bin/sh
test c = $(../urchin -x .print-arg-3 a 'b b b b' c d e)

View File

View File

@@ -0,0 +1,13 @@
#!/bin/sh
# Borrow this test which has a failure in it (3 in fact), and
# run it using urchin:
../../urchin "../Counts should be kept of successes and failures./.test" > /dev/null
# Now invert the result, we want _this_ test to fail if the exit
# code was 0.
case $? in
0) exit 8;;
*) exit 0;;
esac

View File

@@ -0,0 +1 @@
../../urchin .test > /dev/null

181
urchin
View File

@@ -1,8 +1,12 @@
#!/bin/sh
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably - notably, it can produce output,
# which breaks fullpath().
unset CDPATH
fullpath() {
(
cd "$1"
cd -- "$1"
pwd
)
}
@@ -15,27 +19,34 @@ indent() {
recurse() {
potential_test="$1"
indent_level="$2"
shell_for_sh_tests="$3"
[ "$potential_test" = 'setup_dir' ] && return
[ "$potential_test" = 'teardown_dir' ] && return
[ "$potential_test" = 'setup' ] && return
[ "$potential_test" = 'teardown' ] && return
echo > "$stdout_file"
[ $indent_level -eq 0 ] && : > "$stdout_file"
if [ -d "$potential_test" ]
then
(
indent $indent_level
echo " ${potential_test}"
cd "$potential_test"
cd -- "$potential_test"
[ -f setup_dir ] && [ -x setup_dir ] && ./setup_dir >> "$stdout_file"
if [ -n "$ZSH_VERSION" ]; then
# avoid "no matches found: *" error when directories are empty
setopt NULL_GLOB
fi
for test in *
do
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
# $2 instead of $indent_level so it doesn't clash
recurse "${test}" $(( $2 + 1 ))
recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
done
@@ -48,96 +59,160 @@ recurse() {
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
# Run the test
./"$potential_test" > "$stdout_file" 2>&1
if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
then
TEST_SHELL="$TEST_SHELL" "$shell_for_sh_tests" ./"$potential_test" > "$stdout_file" 2>&1
else
TEST_SHELL="$TEST_SHELL" ./"$potential_test" > "$stdout_file" 2>&1
fi
exit_code="$?"
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
indent $indent_level
if [ "$exit_code" = '0' ]
if [ $exit_code -eq 0 ]
then
# On success, print a '✓'
# On success, print a green '✓'
printf '\033[32m✓ \033[0m'
echo "${potential_test}"
echo "${potential_test} passed" >> "$logfile"
printf '%s\n' "${potential_test}"
printf '%s\n' "${potential_test} passed" >> "$logfile"
else
# On fail, print a red '✗'
printf '\033[31m✗ \033[0m'
echo "${potential_test}"
echo "${potential_test} failed" >> "$logfile"
printf '%s\n' "${potential_test}"
printf '%s\n' "${potential_test} failed" >> "$logfile"
printf '\033[31m' # Print output captured from failed test in red.
cat "$stdout_file"
printf '\033[0m'
fi
rm "$stdout_file"
fi
[ $indent_level -eq 0 ] && rm "$stdout_file"
}
has_sh_or_no_shebang_line() {
head -n 1 "$1" | grep -vqE '^#!' && return 0 # no shebang line at all
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' && return 0 # shebang line is '#!/bin/sh' or legal variations thereof
return 1
}
USAGE="usage: $0 [<options>] <test directory>"
urchin_help() {
cat <<EOF
$USAGE
-s <shell> Invoke test scripts that either have no shebang line at all or
have shebang line "#!/bin/sh" with the specified shell.
-f Force running even if the test directory's name does not
contain the word "test".
-h This help.
Go to https://github.com/tlevine/urchin for documentation on writing tests.
EOF
# [Experimental -x option left undocumented for now.]
# -x [Experimental; not meant for direct invocation, but for use in
# the shebang line of test scripts]
# Run with "\$TEST_SHELL", falling back on /bin/sh.
}
plural () {
# Make $1 a plural according to the number $2.
# If $3 is supplied, use that instead of "${1}s".
# Result is written to stdout.
if [ "$2" = 1 ]
then
printf '%s\n' "$1"
else
printf '%s\n' "${3-${1}s}"
fi
}
USAGE="usage: $0 <test directory>"
urchin_help() {
echo
echo "$USAGE"
echo
echo '-f Force urchin to run on directories not named "test".'
echo '-h This help'
echo
echo '--xsd Output xUnit XML schema for an integration server.'
echo
echo 'Go to http://www.urchin.sh for documentation on writing tests.'
echo
}
urchin_go() {
echo Running tests
echo Running tests at $(date +%Y-%m-%dT%H:%M:%S) | tee "$logfile"
start=$(date +%s)
echo > "$logfile"
recurse "$1" 0
# Determine the environment variable to define for test scripts
# that reflects the specified or implied shell to use for shell-code tests.
# - Set it to the shell specified via -s, if any.
# - Otherwise, use its present value, if non-empty.
# - Otherwise, default to '/bin/sh'.
if [ -n "$2" ]
then
TEST_SHELL="$2"
elif [ -z "$TEST_SHELL" ]
then
TEST_SHELL='/bin/sh'
fi
echo Done
echo $(grep -e 'passed$' "$logfile"|wc -l) tests passed.
echo $(grep -e 'failed$' "$logfile"|wc -l) tests failed.
recurse "$1" 0 "$2" # test folder -- indentation level -- [shell to invoke test scripts with]
finish=$(date +%s)
elapsed=$(($finish - $start))
echo "Done, took $elapsed $(plural second $elapsed)."
set -- $(grep -e 'passed$' "$logfile"|wc -l) $(grep -e 'failed$' "$logfile"|wc -l)
printf '%s\n' "$1 $(plural test "$1") passed."
[ $2 -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
printf '%s\n' "$2 $(plural test "$2") failed."
printf '\033[m'
return "$2"
}
urchin_molly_guard() {
echo
echo 'The directory on which you are running urchin is not'
echo 'called "test", so I am not running in case that'
echo 'was an accident. Use the -f flag if you really want'
echo 'to run urchin on that directory.'
echo
{
echo
echo 'The name of the directory on which you are running urchin'
echo 'does not contain the word "test", so I am not running,'
echo 'in case that was an accident. Use the -f flag if you really'
echo 'want to run urchin on that directory.'
echo
} >&2
exit 1
}
FORCE=false
shell_for_sh_tests=
force=false
while [ $# -gt 0 ]
do
case "$1" in
-f) FORCE=true;;
-f) force=true;;
-s)
shift
shell_for_sh_tests=$1
which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
;;
-x) # [EXPERIMENTAL; UNDOCUMENTED FOR NOW] `urchin -x <test-script>` in a test script's shebang line is equivalent to invoking that script with `"$TEST_SHELL" <test-script>`
shift
urchinsh=${TEST_SHELL:-/bin/sh}
"$urchinsh" "$@"
exit $?;;
-h|--help) urchin_help
exit 0;;
# --xsd) action=testsuite;;
# --) shift; break;;
-*) urchin_help 1>&2
-*) urchin_help >&2
exit 1;;
*) break;;
esac
shift
done
# Verify argument for main stuff
if [ "$#" != '1' ] || [ ! -d "$1" ]
then
[ -n "$1" ] && [ ! -d "$1" ] && echo "Not a directory: '$1'" >&2
echo "$USAGE" >&2
exit 2
fi
# Constants
logfile=$(fullpath "$1")/.urchin.log
stdout_file=$(fullpath "$1")/.urchin_stdout
# Verify argument for main stuff
if [ "$#" != '1' ] && [ ! -d "$1" ]
then
echo "$USAGE"
exit 1
fi
# Run or present the Molly guard.
if echo "$(basename "$(fullpath "$1")")" | grep test || $FORCE
if basename "$(fullpath "$1")" | grep -Fi 'test' > /dev/null || $force
then
urchin_go "$1"
urchin_go "$1" "$shell_for_sh_tests"
else
urchin_molly_guard
fi