9 Commits

Author SHA1 Message Date
Thomas Levine
a4d09016ab blarg 2016-02-26 18:27:40 +00:00
Thomas Levine
499b6ee131 update history 2016-02-26 18:20:39 +00:00
Thomas Levine
fb2a8e72e6 implement run on a single file 2016-02-26 18:19:26 +00:00
Thomas Levine
e7d5a35409 Automatic commit with j 2016-02-26 18:16:24 +00:00
Thomas Levine
5aba84246f test single file 2016-02-26 17:54:53 +00:00
Thomas Levine
15acd676f9 nagios explanation 2016-02-26 17:43:05 +00:00
Thomas Levine
0c73e5eb44 move skip tests to history 2016-02-26 17:39:18 +00:00
Thomas Levine
bb103757e4 implement exit code 3 thing 2016-02-26 17:37:39 +00:00
Thomas Levine
dce0adc768 test skip on exit code 3 2016-02-26 17:37:12 +00:00
16 changed files with 63 additions and 50 deletions

31
HISTORY
View File

@@ -4,9 +4,40 @@ HISTORY
Version 0.0.7 Version 0.0.7
--------------------- ---------------------
### Molly-guard
The Molly-guard is now more accepting. For example, you no longer need to The Molly-guard is now more accepting. For example, you no longer need to
pass -f in this case: https://github.com/creationix/nvm/issues/357 pass -f in this case: https://github.com/creationix/nvm/issues/357
### Skipping tests
Previously, tests were run if they were executable and were otherwise marked
as skipped. Now, an executable script can indicate that it is skipped by
exiting with code 3. For example, if a test requires some dependancy, it
might look for the dependency and then skip if it does not see the dependency.
It might look like this.
#!/bin/sh
if which inkscape; then
exit 3 # status code 3 for skip
fi
inkscape blah blah ...
I chose status code 3 sort of arbitrarily at first, but it turns out that it
would the appropriate status code if these tests were Nagios plugins, as the
concept of skipping a test is similar to the Nagios concept of unknown service
status (https://nagios-plugins.org/doc/guidelines.html#AEN78).
### Run on a file
It is now possible to run urchin on a single file.
This occurred to me when I wanted to run
urchin test/fast/Unit\ tests/nvm_ls_current
on the nvm tests. I wound up running this instead.
urchin test/fast/Unit\ tests/ | grep nvm_ls_current
The Molly guard is assessed, and the corresponding setup, setup_dir,
teardown, and teardown_dir files are run in the appropriate order.
Version 0.0.6 Version 0.0.6
--------------------- ---------------------

28
TODO
View File

@@ -1,18 +1,6 @@
Things I want Things I want
============= =============
Skip tests
-------------
I want tests to be able to decide to skip themselves. For example, if a test
requires some dependancy, it might look for the dependency and then skip if it
does not see the dependency. It might look like this.
#!/bin/sh
if which parallel > /dev/null; then
exit 3 # status code 3 for skip
fi
parallel blah blah ...
Test speed Test speed
------------- -------------
Make tests run faster. Make tests run faster.
@@ -135,22 +123,6 @@ cleanly create and teardown temporary files.
On the other hand, this could just be sourced explicitly in the test file, On the other hand, this could just be sourced explicitly in the test file,
without the special setup and teardown feature. without the special setup and teardown feature.
Run on a file
----------------
Presently you can run urchin only on a directory.
It would be neat if you could run it on a file as well.
This occurred to me when I wanted to run
urchin test/fast/Unit\ tests/nvm_ls_current
on the nvm tests. I wound up running this instead.
urchin test/fast/Unit\ tests/ | grep nvm_ls_current
The Molly guard would be assessed, and the corresponding setup, setup_dir,
teardown, and teardown_dir files would be run in the appropriate order.
Running automated tasks Running automated tasks
------------------------- -------------------------
Urchin might be appropriate for if you have lots of tasks that you want to run Urchin might be appropriate for if you have lots of tasks that you want to run

0
tests/Single file/.test Executable file
View File

View File

@@ -0,0 +1 @@
echo setup

View File

@@ -0,0 +1 @@
echo setup_dir

View File

@@ -0,0 +1 @@
echo teardown

View File

@@ -0,0 +1 @@
echo teardown_dir

View File

@@ -0,0 +1,2 @@
echo thetest
exit 1

View File

@@ -0,0 +1,3 @@
#!/bin/sh
../../urchin .test
# This will exit 0 if it worked.

View File

@@ -0,0 +1,2 @@
#!/bin/sh
! ../../urchin not-a-file

View File

@@ -0,0 +1,4 @@
tmp=$(mktemp)
echo "Using temp file $tmp"
../../urchin .wrappers > $tmp
diff $tmp .wrapper-expectation

1
tests/skip/.test/fail Executable file
View File

@@ -0,0 +1 @@
exit 1

1
tests/skip/.test/skip Executable file
View File

@@ -0,0 +1 @@
exit 3

1
tests/skip/.test/succeed Executable file
View File

@@ -0,0 +1 @@
exit 0

View File

@@ -0,0 +1 @@
test $(../../urchin -t .test/ | grep -c SKIP) -eq 1

35
urchin
View File

@@ -16,13 +16,6 @@ unset CDPATH
# Urchin version number # Urchin version number
VERSION=0.0.6 VERSION=0.0.6
fullpath() {
(
cd -- "$1"
pwd
)
}
indent() { indent() {
level="$1" level="$1"
if test "$level" -gt 0; then if test "$level" -gt 0; then
@@ -103,6 +96,8 @@ recurse() {
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file" [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
if [ $exit_code -eq 0 ]; then if [ $exit_code -eq 0 ]; then
result=success result=success
elif [ $exit_code -eq 3 ]; then
result=skip
else else
result=fail result=fail
fi fi
@@ -282,7 +277,7 @@ do
which "$shell_for_sh_tests" > /dev/null || { which "$shell_for_sh_tests" > /dev/null || {
echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2 echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2
urchin_help >&2 urchin_help >&2
exit 2 exit 11
} ;; } ;;
-t) tap_format=true;; -t) tap_format=true;;
-h|--help) urchin_help -h|--help) urchin_help
@@ -296,21 +291,17 @@ do
shift shift
done done
# Verify argument for main stuff
if [ "$#" != '1' ] || [ ! -d "$1" ]
then
[ -n "$1" ] && [ ! -d "$1" ] && echo "Not a directory: '$1'" >&2
echo "$USAGE" >&2
exit 2
fi
# Constants
logfile=$(fullpath "$1")/.urchin.log
stdout_file=$(fullpath "$1")/.urchin_stdout
# Run or present the Molly guard. # Run or present the Molly guard.
if fullpath "$1" | grep -Fi 'test' > /dev/null || $force thetest=$(readlink -f "$1")
then if echo "$thetest" | grep -Fi 'test' > /dev/null || $force; then
if test -d "$1"; then
d="$thetest"
else
d=$(echo "$thetest" | sed 's/\/[^\/]\{1,\}$//')
cd "$d"
fi
logfile="$d/.urchin.log"
stdout_file="$d/.urchin_stdout"
urchin_go "$1" "$shell_for_sh_tests" urchin_go "$1" "$shell_for_sh_tests"
else else
urchin_molly_guard urchin_molly_guard