8 Commits

Author SHA1 Message Date
Thomas Levine
48e5090091 start writing alternatives 2015-12-09 07:46:16 -05:00
Thomas Levine
5abf088eb8 add new ideas 2015-10-19 12:52:24 -04:00
Thomas Levine
c84851cb5c document the ordering of tests 2015-08-21 09:03:15 -04:00
Thomas Levine
510476f924 mention shall in the documentation 2015-07-05 17:39:38 +00:00
Thomas Levine
f01869fb97 packaging 2014-12-08 02:08:00 +00:00
Michael Klement
0385dcd86e Fixed blog link in read-me and commented out experimental -x option for now. 2014-12-05 18:50:19 -05:00
Michael Klement
2847b020b4 Make sure that CDPATH isn't set, as it causes cd to behave unpredictably - notably, it can produce output, which breaks fullpath().
Also: Improved CLI help, updated URLs in read-me, cleaned up package.json:

I've tried to clarify the intent of `-x` in the CLI help, but I haven't touched the read-me in that respect.

I don't see any benefit to `-x`:

* Just using `#/bin/sh` as the shebang line in combination with `-s <shell>` gives you the same functionality,
* When it comes to invoking scripts from _within_ test scripts, nothing can do the work for you: you consciously have to mark the invocation with _something_ to indicate that it should be controlled from the outside; it won't get any easier than `$TEST_SHELL ...`
* Finally, using a shebang line such as `#!/usr/bin/env urchin -x` is problematic for two reasons:
  * Some platforms can handle only *1* argument in a shebang line.
  * In a _package-local_ installation, `#!/usr/bin/env` may not find the Urchin executable.

I'm also not sure how the following (from `readme.md`) fits in the picture:

> It might make sense if you do this.

    export TEST_SHELL=zsh && urchin -x
    export TEST_SHELL=bash && urchin -x

(As an aside: To achieve the same thing, you don't need `export`; `TEST_SHELL=zsh urchin -x`  and `TEST_SHELL=bash urchin -x`  is the better choice.)
How does this relate to use in a  _shebang line_?

`urchin_help()` now uses a here-doc: easier to maintain, and should work in all Bourne-like shells.

`readmeFilename` removed from `package.json`:

> "The readmeFilename does not need to ever be in your actual package.json file" - npm/npm#3573
2014-12-03 09:54:15 -05:00
Michael Klement
fae24e926a Improved CLI help, updated URLs in read-me 2014-11-06 22:21:05 -05:00
13 changed files with 71 additions and 43 deletions

View File

@@ -1,6 +1,11 @@
HISTORY
-------
Version 0.0.5
---------------------
* urchin now unsets `CDPATH`.
* The documentation for `urchin -x` was removed because it was confusing.
Version 0.0.4
---------------------
* Switch urchin -x to urchin -sh and fix some problems with it

2
alternatives Normal file
View File

@@ -0,0 +1,2 @@
Totally different syntax and similar features, plus TAP output
https://github.com/sstephenson/bats

View File

@@ -1,6 +1,6 @@
{
"name": "urchin",
"version": "0.0.4",
"version": "0.0.5",
"description": "Test framework for shell",
"main": "urchin",
"directories": {
@@ -24,10 +24,9 @@
{"name": "Thomas Levine", "email": "_@thomaslevine.com"},
{"name": "David Jones", "email": "drj@pobox.com"},
{"name": "Francis Irving", "email": "francis@flourish.org"},
{"name": "Zarino Zappia", "email": "mail@zarino.co.uk"}
{"name": "Tom Mortimer-Jones", "email": "tom@morty.co.uk"}
{"name": "Zarino Zappia", "email": "mail@zarino.co.uk"},
{"name": "Tom Mortimer-Jones", "email": "tom@morty.co.uk"},
{"name": "Michael Klement", "email": "mklement0@gmail.com"}
],
"license": "BSD",
"readmeFilename": "readme.md"
"license": "BSD"
}

View File

@@ -12,14 +12,14 @@ other Unix platforms.
Urchin's tests are written in Urchin, so you can run them to see what Urchin
is like. Clone the repository
git clone git://github.com/scraperwiki/urchin.git
git clone git://github.com/tlevine/urchin.git
Run the tests
cd urchin
./urchin tests
The above command will run the tests in your systems default
The above command will run the tests in your system's default
shell, /bin/sh (on recent Ubuntu this is dash, but it could be
ksh or bash on other systems); to test urchin's cross-shell compatibility,
run this:
@@ -31,7 +31,7 @@ run this:
Download Urchin like so (as root) (or use npm, below):
cd /usr/local/bin
wget https://raw.github.com/scraperwiki/urchin/master/urchin
wget https://raw.github.com/tlevine/urchin/master/urchin
chmod +x urchin
Can be installed with npm too:
@@ -86,6 +86,15 @@ Files are only run if they are executable, and files beginning with `.` are
ignored. Thus, fixtures and libraries can be included sloppily within the test
directory tree. The test passes if the file exits 0; otherwise, it fails.
In case you care about the order in which your tests execute, consider that
urchin looks for files within a directory in the following manner.
for file in *; do
do_something_with_test_file $file
done
Tests within a directory are executed in whatever order `*` returns.
### Writing cross-shell compatibility tests for testing shell code
While you could write your test scripts to explicitly invoke the functionality
@@ -98,6 +107,13 @@ The specific approach depends on your test scenario:
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
First, consider using [shall](https://github.com/mklement0/shall).
#!/usr/bin/env shall
echo This is a test file.
Alternatively, you can use urchin's built-in recognition of the
`TEST_SHELL` environment variable.
In your test scripts, invoke the shell scripts to test via the shell
specified in environment variable `TEST_SHELL` rather than directly;
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
@@ -138,6 +154,7 @@ To test with multiple shells in sequence, use something like:
urchin -s $shell ./tests
done
<!--
#### (c) Cross shell tests with `urchin -x` (experimental)
If you run urchin with the `-x` flag, it will be as if you ran
`$TEST_SHELL`. Unless `$TEST_SHELL` isn't set, in which case it'll
@@ -151,7 +168,13 @@ It might make sense if you do this.
export TEST_SHELL=zsh && urchin -x
export TEST_SHELL=bash && urchin -x
-->
## Alternatives to Urchin
Alternatives to Urchin are discussed in
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
## Ideas for new features
* Support [Nagios plugins](https://nagios-plugins.org/doc/guidelines.html)
* Stop running if a test fails so one can use Urchin as a
[setup framework](https://github.com/tlevine/urchin/issues/16).

View File

@@ -0,0 +1,6 @@
#!/bin/sh
cd ..
export CDPATH=$PWD
./urchin -f 'tests/urchin exit code' >/dev/null

View File

@@ -1,4 +0,0 @@
#! ../../../urchin -x
running=$(ps -o comm= -p $$ && :)
echo "Running shell: $running"
[ "$running" = bash ]

View File

@@ -1,3 +0,0 @@
#!/bin/sh
export TEST_SHELL=/bin/zsh
echo '[ "$TEST_SHELL" = /bin/zsh ] ; exit $?' | ../../urchin -x

View File

@@ -1,3 +0,0 @@
#!/bin/sh
test c = $(../../urchin -x .print-arg-3 a 'b b b b' c d e)

View File

@@ -1,3 +0,0 @@
#!/bin/sh
export TEST_SHELL=/bin/zsh
echo 'test -n "$ZSH_VERSION"; exit $?' | ../../urchin -x

View File

@@ -1,3 +0,0 @@
#!/bin/sh
export TEST_SHELL=/bin/bash
../../urchin .test-urchin-x

View File

@@ -0,0 +1,3 @@
#!/bin/sh
test c = $(../urchin -x .print-arg-3 a 'b b b b' c d e)

42
urchin
View File

@@ -1,5 +1,9 @@
#!/bin/sh
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably - notably, it can produce output,
# which breaks fullpath().
unset CDPATH
fullpath() {
(
cd -- "$1"
@@ -95,19 +99,23 @@ has_sh_or_no_shebang_line() {
USAGE="usage: $0 [<options>] <test directory>"
urchin_help() {
echo
echo "$USAGE"
echo
echo '-s <shell> Invoke test scripts that either have no shebang line or'
echo ' shebang line "#!/bin/sh" with the specified shell.'
echo '-f Force running even if the test directory'\''s name does not'
echo ' contain the word "test".'
echo '-x Run "$TEST_SHELL", falling back on /bin/sh. This might be'
echo ' useful in the shebang line (experimental).'
echo '-h This help.'
echo
echo 'Go to https://github.com/tlevine/urchin for documentation on writing tests.'
echo
cat <<EOF
$USAGE
-s <shell> Invoke test scripts that either have no shebang line at all or
have shebang line "#!/bin/sh" with the specified shell.
-f Force running even if the test directory's name does not
contain the word "test".
-h This help.
Go to https://github.com/tlevine/urchin for documentation on writing tests.
EOF
# [Experimental -x option left undocumented for now.]
# -x [Experimental; not meant for direct invocation, but for use in
# the shebang line of test scripts]
# Run with "\$TEST_SHELL", falling back on /bin/sh.
}
plural () {
@@ -175,12 +183,10 @@ do
shell_for_sh_tests=$1
which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
;;
-x) # `urchin -sh` is equivalent to "$TEST_SHELL"
-x) # [EXPERIMENTAL; UNDOCUMENTED FOR NOW] `urchin -x <test-script>` in a test script's shebang line is equivalent to invoking that script with `"$TEST_SHELL" <test-script>`
shift
#current_shell=$(ps -o comm= -p $$ && :)
#urchinsh=${TEST_SHELL:-$current_shell}
export TEST_SHELL=${TEST_SHELL:-/bin/sh}
"$TEST_SHELL" "$@"
urchinsh=${TEST_SHELL:-/bin/sh}
"$urchinsh" "$@"
exit $?;;
-h|--help) urchin_help
exit 0;;