11 Commits

Author SHA1 Message Date
Thomas Levine
56a4e5d8af introduction paragraph 2016-01-25 14:06:33 +00:00
Thomas Levine
d81b7efdc3 asciibetical 2016-01-25 13:56:33 +00:00
Thomas Levine
7f5455739d update readme
* move shall reference to the right place
* further explain cross-shell testing methods
2016-01-25 13:49:36 +00:00
Thomas Levine
48e5090091 start writing alternatives 2015-12-09 07:46:16 -05:00
Thomas Levine
5abf088eb8 add new ideas 2015-10-19 12:52:24 -04:00
Thomas Levine
c84851cb5c document the ordering of tests 2015-08-21 09:03:15 -04:00
Thomas Levine
510476f924 mention shall in the documentation 2015-07-05 17:39:38 +00:00
Thomas Levine
f01869fb97 packaging 2014-12-08 02:08:00 +00:00
Michael Klement
0385dcd86e Fixed blog link in read-me and commented out experimental -x option for now. 2014-12-05 18:50:19 -05:00
Michael Klement
2847b020b4 Make sure that CDPATH isn't set, as it causes cd to behave unpredictably - notably, it can produce output, which breaks fullpath().
Also: Improved CLI help, updated URLs in read-me, cleaned up package.json:

I've tried to clarify the intent of `-x` in the CLI help, but I haven't touched the read-me in that respect.

I don't see any benefit to `-x`:

* Just using `#/bin/sh` as the shebang line in combination with `-s <shell>` gives you the same functionality,
* When it comes to invoking scripts from _within_ test scripts, nothing can do the work for you: you consciously have to mark the invocation with _something_ to indicate that it should be controlled from the outside; it won't get any easier than `$TEST_SHELL ...`
* Finally, using a shebang line such as `#!/usr/bin/env urchin -x` is problematic for two reasons:
  * Some platforms can handle only *1* argument in a shebang line.
  * In a _package-local_ installation, `#!/usr/bin/env` may not find the Urchin executable.

I'm also not sure how the following (from `readme.md`) fits in the picture:

> It might make sense if you do this.

    export TEST_SHELL=zsh && urchin -x
    export TEST_SHELL=bash && urchin -x

(As an aside: To achieve the same thing, you don't need `export`; `TEST_SHELL=zsh urchin -x`  and `TEST_SHELL=bash urchin -x`  is the better choice.)
How does this relate to use in a  _shebang line_?

`urchin_help()` now uses a here-doc: easier to maintain, and should work in all Bourne-like shells.

`readmeFilename` removed from `package.json`:

> "The readmeFilename does not need to ever be in your actual package.json file" - npm/npm#3573
2014-12-03 09:54:15 -05:00
Michael Klement
fae24e926a Improved CLI help, updated URLs in read-me 2014-11-06 22:21:05 -05:00
6 changed files with 85 additions and 49 deletions

View File

@@ -1,6 +1,11 @@
HISTORY HISTORY
------- -------
Version 0.0.5
---------------------
* urchin now unsets `CDPATH`.
* The documentation for `urchin -x` was removed because it was confusing.
Version 0.0.4 Version 0.0.4
--------------------- ---------------------
* Switch urchin -x to urchin -sh and fix some problems with it * Switch urchin -x to urchin -sh and fix some problems with it

2
alternatives Normal file
View File

@@ -0,0 +1,2 @@
Totally different syntax and similar features, plus TAP output
https://github.com/sstephenson/bats

View File

@@ -1,6 +1,6 @@
{ {
"name": "urchin", "name": "urchin",
"version": "0.0.4", "version": "0.0.5",
"description": "Test framework for shell", "description": "Test framework for shell",
"main": "urchin", "main": "urchin",
"directories": { "directories": {
@@ -24,10 +24,9 @@
{"name": "Thomas Levine", "email": "_@thomaslevine.com"}, {"name": "Thomas Levine", "email": "_@thomaslevine.com"},
{"name": "David Jones", "email": "drj@pobox.com"}, {"name": "David Jones", "email": "drj@pobox.com"},
{"name": "Francis Irving", "email": "francis@flourish.org"}, {"name": "Francis Irving", "email": "francis@flourish.org"},
{"name": "Zarino Zappia", "email": "mail@zarino.co.uk"} {"name": "Zarino Zappia", "email": "mail@zarino.co.uk"},
{"name": "Tom Mortimer-Jones", "email": "tom@morty.co.uk"} {"name": "Tom Mortimer-Jones", "email": "tom@morty.co.uk"},
{"name": "Michael Klement", "email": "mklement0@gmail.com"} {"name": "Michael Klement", "email": "mklement0@gmail.com"}
], ],
"license": "BSD", "license": "BSD"
"readmeFilename": "readme.md"
} }

View File

@@ -4,22 +4,22 @@
/ /_/ / / / /__/ / / / / / / / / /_/ / / / /__/ / / / / / / /
\__,_/_/ \___/_/ /_/_/_/ /_/ \__,_/_/ \___/_/ /_/_/_/ /_/
Urchin is a test framework for shell. It is implemented in Urchin is a file-based test harness, normally used for testing shell programs.
portable /bin/sh and should work on GNU/Linux, Mac OS X, and It is written in portable shell and should thus work on GNU/Linux, BSD
other Unix platforms. (including Mac OS X), and other Unix-like platforms.
## Try it out ## Try it out
Urchin's tests are written in Urchin, so you can run them to see what Urchin Urchin's tests are written in Urchin, so you can run them to see what Urchin
is like. Clone the repository is like. Clone the repository
git clone git://github.com/scraperwiki/urchin.git git clone git://github.com/tlevine/urchin.git
Run the tests Run the tests
cd urchin cd urchin
./urchin tests ./urchin tests
The above command will run the tests in your systems default The above command will run the tests in your system's default
shell, /bin/sh (on recent Ubuntu this is dash, but it could be shell, /bin/sh (on recent Ubuntu this is dash, but it could be
ksh or bash on other systems); to test urchin's cross-shell compatibility, ksh or bash on other systems); to test urchin's cross-shell compatibility,
run this: run this:
@@ -27,14 +27,15 @@ run this:
cd urchin cd urchin
./cross-shell-tests ./cross-shell-tests
## Globally ## Install
Download Urchin like so (as root) (or use npm, below): Urchin is contained in a single file, so you can install it by copying it to a
directory in your `PATH`. For example, you can run the following as root.
cd /usr/local/bin cd /usr/local/bin
wget https://raw.github.com/scraperwiki/urchin/master/urchin wget https://raw.github.com/tlevine/urchin/master/urchin
chmod +x urchin chmod +x urchin
Can be installed with npm too: Urchin can be installed with npm too.
npm install -g urchin npm install -g urchin
@@ -86,6 +87,14 @@ Files are only run if they are executable, and files beginning with `.` are
ignored. Thus, fixtures and libraries can be included sloppily within the test ignored. Thus, fixtures and libraries can be included sloppily within the test
directory tree. The test passes if the file exits 0; otherwise, it fails. directory tree. The test passes if the file exits 0; otherwise, it fails.
Tests files and subdirectories are run in ASCIIbetical order within each
directory; that is,
urchin looks for files within a directory in the following manner.
for file in *; do
do_something_with_test_file $file
done
### Writing cross-shell compatibility tests for testing shell code ### Writing cross-shell compatibility tests for testing shell code
While you could write your test scripts to explicitly invoke the functionality While you could write your test scripts to explicitly invoke the functionality
@@ -97,17 +106,20 @@ The specific approach depends on your test scenario:
* (b) Your scripts _source_ scripts containing portable shell code. * (b) Your scripts _source_ scripts containing portable shell code.
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts #### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
Urchin sets the `TEST_SHELL` environment variable so that you may change the
shell with which your tests call other shell programs. To run your test
scripts in multiple shells you must call `$TEST_SHELL` in your tests and then
run urchin with the appropriate option.
In your test scripts, invoke the shell scripts to test via the shell In your test scripts, invoke the shell scripts to test via the shell
specified in environment variable `TEST_SHELL` rather than directly; specified in environment variable `TEST_SHELL` rather than directly;
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`). e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
Note that if you alsow want your test scripts to work when run directly,
outside of Urchin, be sure to target scripts that happen to be in the
current directory with prefix `./`; e.g., `$TEST_SHELL ./baz`
(rather than `$TEST_SHELL baz`).
Then, on invocation of Urchin, prepend a definition of environment variable On invocation of Urchin, prepend a definition of environment variable
`TEST_SHELL` specifying the shell to test with, e.g.: `TEST_SHELL=zsh urchin ./tests`. `TEST_SHELL` specifying the shell to test with, e.g.,
TEST_SHELL=zsh urchin ./tests
To test with multiple shells in sequence, use something like: To test with multiple shells in sequence, use something like:
for shell in sh bash ksh zsh; do for shell in sh bash ksh zsh; do
@@ -115,14 +127,20 @@ To test with multiple shells in sequence, use something like:
done done
If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
scripts can rely on `$TEST_SHELL` always containing a value. scripts can rely on `$TEST_SHELL` always containing a value when Urchin runs
them.
That said, we still recommand that you account for the possibility that
`$TEST_SHELL` does not contain a value so that you may run your test scripts
without Urchin. Supporting this case is very simple; when you invoke scripts
that happen to be in the current directory, be sure to use the prefix `./`,
e.g., `$TEST_SHELL ./baz` rather than `$TEST_SHELL baz`.
#### (b) Cross-shell tests with test scripts that _source_ shell scripts #### (b) Cross-shell tests with test scripts that _source_ shell scripts
If you _source_ shell code in your test scripts, it is the test scripts If you _source_ shell code in your test scripts, it is the test scripts
themselves that must be run with the shell specified. themselves that must be run with the shell specified.
To that end, Urchin supports the `-s <shell>` option, which instructs Urchin supports the `-s <shell>` option, which instructs
Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`. Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`.
(In addition, Urchin sets environment variable `TEST_SHELL` to the specified (In addition, Urchin sets environment variable `TEST_SHELL` to the specified
shell.) shell.)
@@ -138,20 +156,18 @@ To test with multiple shells in sequence, use something like:
urchin -s $shell ./tests urchin -s $shell ./tests
done done
#### (c) Cross shell tests with `urchin -x` (experimental) Also consider using [shall](https://github.com/mklement0/shall).
If you run urchin with the `-x` flag, it will be as if you ran It does something similar, but the interface may be more intuitive.
`$TEST_SHELL`. Unless `$TEST_SHELL` isn't set, in which case it'll
be as if you ran `/bin/sh`. Putting this in she shebang line might
eventually work out to be a cleaner way of doing cross-shell testing.
#!/usr/bin/env urchin -x #!/usr/bin/env shall
test a = a echo This is a test file.
It might make sense if you do this.
export TEST_SHELL=zsh && urchin -x
export TEST_SHELL=bash && urchin -x
## Alternatives to Urchin ## Alternatives to Urchin
Alternatives to Urchin are discussed in Alternatives to Urchin are discussed in
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/). [this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
## Ideas for new features
* Support [Nagios plugins](https://nagios-plugins.org/doc/guidelines.html)
* Stop running if a test fails so one can use Urchin as a
[setup framework](https://github.com/tlevine/urchin/issues/16).

View File

@@ -0,0 +1,6 @@
#!/bin/sh
cd ..
export CDPATH=$PWD
./urchin -f 'tests/urchin exit code' >/dev/null

36
urchin
View File

@@ -1,5 +1,9 @@
#!/bin/sh #!/bin/sh
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably - notably, it can produce output,
# which breaks fullpath().
unset CDPATH
fullpath() { fullpath() {
( (
cd -- "$1" cd -- "$1"
@@ -95,19 +99,23 @@ has_sh_or_no_shebang_line() {
USAGE="usage: $0 [<options>] <test directory>" USAGE="usage: $0 [<options>] <test directory>"
urchin_help() { urchin_help() {
echo cat <<EOF
echo "$USAGE"
echo $USAGE
echo '-s <shell> Invoke test scripts that either have no shebang line or'
echo ' shebang line "#!/bin/sh" with the specified shell.' -s <shell> Invoke test scripts that either have no shebang line at all or
echo '-f Force running even if the test directory'\''s name does not' have shebang line "#!/bin/sh" with the specified shell.
echo ' contain the word "test".' -f Force running even if the test directory's name does not
echo '-x Run "$TEST_SHELL", falling back on /bin/sh. This might be' contain the word "test".
echo ' useful in the shebang line (experimental).' -h This help.
echo '-h This help.'
echo Go to https://github.com/tlevine/urchin for documentation on writing tests.
echo 'Go to https://github.com/tlevine/urchin for documentation on writing tests.'
echo EOF
# [Experimental -x option left undocumented for now.]
# -x [Experimental; not meant for direct invocation, but for use in
# the shebang line of test scripts]
# Run with "\$TEST_SHELL", falling back on /bin/sh.
} }
plural () { plural () {
@@ -175,7 +183,7 @@ do
shell_for_sh_tests=$1 shell_for_sh_tests=$1
which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; } which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
;; ;;
-x) # `urchin -sh` is equivalent to "$TEST_SHELL" -x) # [EXPERIMENTAL; UNDOCUMENTED FOR NOW] `urchin -x <test-script>` in a test script's shebang line is equivalent to invoking that script with `"$TEST_SHELL" <test-script>`
shift shift
urchinsh=${TEST_SHELL:-/bin/sh} urchinsh=${TEST_SHELL:-/bin/sh}
"$urchinsh" "$@" "$urchinsh" "$@"