Support for cross-shell testing added, via option `-s <shell>` and env. variable `TEST_SHELL`.
* For **tests that _source_ shell scripts**: **option `-s <shell>`** now tells urchin to invoke test scripts with the specified shell (only shebang-less and `#!/bin/sh` tests scripts). * For **tests that _invoke_ schell scripts**: instruct users to write their tests to always **invoke via environment variable `TEST_SHELL` (e.g., `$TEST_SHELL ../foo`)**, and invoke urchin with that variable defined as needed, e.g., `TEST_SHELL=ksh urchin ./tests`; urchin defaults `TEST_SHELL` to `/bin/sh`. See updated `readme.md` for details.
This commit is contained in:
Michael Klement
parent
deb77cb5de
commit
1e9611e732
51
readme.md
51
readme.md
|
|
@ -42,6 +42,8 @@ Now you can run it.
|
|||
|
||||
urchin <test directory>
|
||||
|
||||
Run `urchin -h` to get command-line help.
|
||||
|
||||
## Writing tests
|
||||
Make a root directory for your tests. Inside it, put executable files that
|
||||
exit `0` on success and something else on fail. Non-executable files and hidden
|
||||
|
|
@ -84,6 +86,55 @@ Files are only run if they are executable, and files beginning with `.` are
|
|||
ignored. Thus, fixtures and libraries can be included sloppily within the test
|
||||
directory tree. The test passes if the file exits 0; otherwise, it fails.
|
||||
|
||||
### Writing cross-shell compatibility tests for testing shell code
|
||||
|
||||
While you could write your test scripts to explicitly invoke the functionality
|
||||
to test with various shells, urchin facilitates a more flexible approach.
|
||||
|
||||
The specific approach depends on your test scenario:
|
||||
|
||||
* (a) Your test scripts _invoke_ scripts containing portable shell code.
|
||||
* (b) Your scripts _source_ scripts containing portable shell code.
|
||||
|
||||
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
|
||||
|
||||
Write your test scripts to invoke the shell scripts to test via the shell
|
||||
specified in environment variable `TEST_SHELL` rather than directly;
|
||||
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`)
|
||||
|
||||
Then, on invocation of urchin, prepend a definition of environment variable `TEST_SHELL`
|
||||
specifying the shell to test with, e.g.: `TEST_SHELL=zsh urchin ./tests`.
|
||||
To test with multiple shells in sequence, use something like:
|
||||
|
||||
for shell in sh bash ksh zsh; do
|
||||
TEST_SHELL=$shell urchin ./tests
|
||||
done
|
||||
|
||||
If `TEST_SHELL` has no value, urchin defines it as `/bin/sh`, so the test
|
||||
scripts can rely on `$TEST_SHELL` always containing a value.
|
||||
|
||||
#### (b) Cross-shell tests with test scripts that _source_ shell scripts
|
||||
|
||||
If you _source_ shell code in your test scripts, it is the test scripts
|
||||
themselves that must be run with the shell specified.
|
||||
|
||||
To that end, urchin supports the `-s <shell>` option, which instructs
|
||||
urchin to invoke the test scripts with the specified shell; e.g., `-s bash`
|
||||
|
||||
Note that only test scripts that either have no shebang line at all or
|
||||
have shebang line '#!/bin/sh' are invoked with the specified shell.
|
||||
This allows non-shell test scripts or test scripts for _specific, hard-coded_
|
||||
shells to coexist with those whose invocation should be controlled by `-s`.
|
||||
|
||||
To test with multiple shells in sequence, use something like:
|
||||
|
||||
for shell in sh bash ksh zsh; do
|
||||
urchin -s $shell ./tests
|
||||
done
|
||||
|
||||
Urchin will also define environment variable `TEST_SHELL` to contain the
|
||||
the shell specified via `-s`.
|
||||
|
||||
## Alternatives to Urchin
|
||||
Alternatives to Urchin are discussed in
|
||||
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
#!/bin/sh
|
||||
|
||||
../../urchin -h | grep -- -s
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Assuming that urchin was invoked with `TEST_SHELL=bash urchin ...`, $TEST_SHELL should contain 'bash'.
|
||||
|
||||
echo "\$TEST_SHELL: $TEST_SHELL"
|
||||
|
||||
[ "$TEST_SHELL" = 'bash' ]
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Invoke a simple test command with $TEST_SHELL as the executable.
|
||||
[ "$($TEST_SHELL -c 'echo $0')" = "$TEST_SHELL" ]
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
#!/usr/bin/awk -f
|
||||
|
||||
# This script will only succeed if it is indeed processed by awk.
|
||||
|
||||
BEGIN { print "ok" }
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Assuming that urchin was invoked with `-s bash`, this script should be being run with bash.
|
||||
|
||||
this_shell=$(ps -o comm= -p $$ && :)
|
||||
|
||||
echo "Running shell: $this_shell"
|
||||
|
||||
[ "$this_shell" = 'bash' ]
|
||||
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
# By design, this file has no shebang line.
|
||||
|
||||
# Assuming that urchin was invoked with `-s bash`, this script should be being run with bash.
|
||||
|
||||
this_shell=$(ps -o comm= -p $$ && :)
|
||||
|
||||
echo "Running shell: $this_shell"
|
||||
|
||||
[ "$this_shell" = 'bash' ]
|
||||
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Assuming that urchin was invoked with `-s bash`, $TEST_SHELL should contain 'bash'.
|
||||
|
||||
echo "Running shell: $(ps -o comm= -p $$ && :)"
|
||||
|
||||
echo "\$TEST_SHELL: $TEST_SHELL"
|
||||
|
||||
[ "$TEST_SHELL" = 'bash' ]
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
|
||||
# (for test scripts that want to invoke shell scripts with a specified shell).
|
||||
|
||||
which bash 2>/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 1; }
|
||||
|
||||
# Test if $TEST_SHELL, when placed in urchin's environment, is passed through to the test scripts.
|
||||
TEST_SHELL=bash ../../urchin ./.test-TEST_SHELL-passed-through
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
|
||||
# (for test scripts that want to invoke shell scripts with a specified shell).
|
||||
|
||||
# Test if $TEST_SHELL - if *defined, but empty* - is exported with value '/bin/sh' by urchin
|
||||
# and thus has that value inside the scripts.
|
||||
TEST_SHELL= ../../urchin ./.test-TEST_SHELL-undefined_or_empty
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Tests support for either passing through or defining a default value for environment variable TEST_SHELL.
|
||||
# (for test scripts that want to invoke shell scripts with a specified shell).
|
||||
|
||||
# Test if $TEST_SHELL - if *undefined* - is exported with value '/bin/sh' by urchin
|
||||
# and thus has that value inside test scripts.
|
||||
unset -v TEST_SHELL
|
||||
../../urchin ./.test-TEST_SHELL-undefined_or_empty
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Tests the `-s <shell> option, which invokes shebang-less and sh-shebang-line test scripts with the specified shell (for testing *sourced* shell code).
|
||||
|
||||
which bash >/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 2; }
|
||||
which /usr/bin/awk >/dev/null || { echo "Cannot test -s option: /usr/bin/awk not found." >&2; exit 2; }
|
||||
|
||||
../../urchin -s bash ./.test-run-by-specified-shell
|
||||
56
urchin
56
urchin
|
|
@ -15,6 +15,7 @@ indent() {
|
|||
recurse() {
|
||||
potential_test="$1"
|
||||
indent_level="$2"
|
||||
shell_for_sh_tests="$3"
|
||||
|
||||
[ "$potential_test" = 'setup_dir' ] && return
|
||||
[ "$potential_test" = 'teardown_dir' ] && return
|
||||
|
|
@ -35,7 +36,7 @@ recurse() {
|
|||
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
|
||||
|
||||
# $2 instead of $indent_level so it doesn't clash
|
||||
recurse "${test}" $(( $2 + 1 ))
|
||||
recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
|
||||
|
||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||
done
|
||||
|
|
@ -48,9 +49,15 @@ recurse() {
|
|||
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
|
||||
|
||||
# Run the test
|
||||
./"$potential_test" > "$stdout_file" 2>&1
|
||||
if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
|
||||
then
|
||||
TEST_SHELL="$TEST_SHELL" "$shell_for_sh_tests" ./"$potential_test" > "$stdout_file" 2>&1
|
||||
else
|
||||
TEST_SHELL="$TEST_SHELL" ./"$potential_test" > "$stdout_file" 2>&1
|
||||
fi
|
||||
exit_code="$?"
|
||||
|
||||
|
||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||
|
||||
indent $indent_level
|
||||
|
|
@ -73,15 +80,23 @@ recurse() {
|
|||
[ $indent_level -eq 0 ] && rm "$stdout_file"
|
||||
}
|
||||
|
||||
USAGE="usage: $0 <test directory>"
|
||||
has_sh_or_no_shebang_line() {
|
||||
head -n 1 "$1" | grep -vqE '^#!' && return 0 # no shebang line at all
|
||||
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' && return 0 # shebang line is '#!/bin/sh' or legal variations thereof
|
||||
return 1
|
||||
}
|
||||
|
||||
USAGE="usage: $0 [<options>] <test directory>"
|
||||
|
||||
urchin_help() {
|
||||
echo
|
||||
echo "$USAGE"
|
||||
echo
|
||||
echo '-f Force urchin to run on directories whose name does not contain'
|
||||
echo ' the word "test".'
|
||||
echo '-h This help'
|
||||
echo '-s <shell> Invoke test scripts that either have no shebang line or'
|
||||
echo ' shebang line "#!/bin/sh" with the specified shell.'
|
||||
echo '-f Force running even if the test directory'\''s name does not'
|
||||
echo ' contain the word "test".'
|
||||
echo '-h This help.'
|
||||
# echo
|
||||
# echo '--xsd Output xUnit XML schema for an integration server.'
|
||||
echo
|
||||
|
|
@ -105,7 +120,20 @@ urchin_go() {
|
|||
echo Running tests at $(date +%Y-%m-%dT%H:%M:%S) | tee "$logfile"
|
||||
start=$(date +%s)
|
||||
|
||||
recurse "$1" 0 # test folder, indentation level
|
||||
# Determine the environment variable to define for test scripts
|
||||
# that reflects the specified or implied shell to use for shell-code tests.
|
||||
# - Set it to the shell specified via -s, if any.
|
||||
# - Otherwise, use its present value, if non-empty.
|
||||
# - Otherwise, default to '/bin/sh'.
|
||||
if [ -n "$2" ]
|
||||
then
|
||||
TEST_SHELL="$2"
|
||||
elif [ -z "$TEST_SHELL" ]
|
||||
then
|
||||
TEST_SHELL='/bin/sh'
|
||||
fi
|
||||
|
||||
recurse "$1" 0 "$2" # test folder -- indentation level -- [shell to invoke test scripts with]
|
||||
|
||||
finish=$(date +%s)
|
||||
elapsed=$(($finish - $start))
|
||||
|
|
@ -130,16 +158,20 @@ urchin_molly_guard() {
|
|||
exit 1
|
||||
}
|
||||
|
||||
shell_for_sh_tests=
|
||||
force=false
|
||||
while [ $# -gt 0 ]
|
||||
do
|
||||
case "$1" in
|
||||
-f) force=true;;
|
||||
-s)
|
||||
shift
|
||||
shell_for_sh_tests=$1
|
||||
which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
|
||||
;;
|
||||
-h|--help) urchin_help
|
||||
exit 0;;
|
||||
# --xsd) action=testsuite;;
|
||||
# --) shift; break;;
|
||||
-*) urchin_help 1>&2
|
||||
-*) urchin_help >&2
|
||||
exit 1;;
|
||||
*) break;;
|
||||
esac
|
||||
|
|
@ -151,7 +183,7 @@ if [ "$#" != '1' ] || [ ! -d "$1" ]
|
|||
then
|
||||
[ -d "$1" ] || echo "Not a directory: '$1'" >&2
|
||||
echo "$USAGE" >&2
|
||||
exit 1
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Constants
|
||||
|
|
@ -161,7 +193,7 @@ stdout_file=$(fullpath "$1")/.urchin_stdout
|
|||
# Run or present the Molly guard.
|
||||
if basename "$(fullpath "$1")" | grep -Fi 'test' > /dev/null || $force
|
||||
then
|
||||
urchin_go "$1"
|
||||
urchin_go "$1" "$shell_for_sh_tests"
|
||||
else
|
||||
urchin_molly_guard
|
||||
fi
|
||||
|
|
|
|||
Loading…
Reference in New Issue