Compare commits
33 Commits
parallel
...
urchin-roo
Author | SHA1 | Date | |
---|---|---|---|
|
40950a0b52 | ||
|
dc53523deb | ||
|
a797fc2c8b | ||
|
e07b340906 | ||
|
a4a0aabbef | ||
|
f4d3ffa0e4 | ||
|
28832f22a1 | ||
|
f5ef61633e | ||
|
2836d4b9a9 | ||
|
972fe6bc35 | ||
|
633d6d32c9 | ||
|
b9d72aef72 | ||
|
92cec52c97 | ||
|
b9a067c68e | ||
|
e9d6b73dbd | ||
|
b8bd097f5c | ||
|
f83df14868 | ||
|
fdc7129921 | ||
|
e5ba45ae96 | ||
|
31e0b9fcb7 | ||
|
3efcf0aa33 | ||
|
97faea610c | ||
|
f9ddefcf54 | ||
|
54b9e5887d | ||
|
c9af70b947 | ||
|
83af249dcd | ||
|
160222c0eb | ||
|
4cd9d41a1e | ||
|
e14d8edc05 | ||
|
15acd676f9 | ||
|
0c73e5eb44 | ||
|
bb103757e4 | ||
|
dce0adc768 |
16
HISTORY
16
HISTORY
@@ -7,6 +7,22 @@ Version 0.0.7
|
|||||||
The Molly-guard is now more accepting. For example, you no longer need to
|
The Molly-guard is now more accepting. For example, you no longer need to
|
||||||
pass -f in this case: https://github.com/creationix/nvm/issues/357
|
pass -f in this case: https://github.com/creationix/nvm/issues/357
|
||||||
|
|
||||||
|
Previously, tests were run if they were executable and were otherwise marked
|
||||||
|
as skipped. Now, an executable script can indicate that it is skipped by
|
||||||
|
exiting with code 3. For example, if a test requires some dependancy, it
|
||||||
|
might look for the dependency and then skip if it does not see the dependency.
|
||||||
|
It might look like this.
|
||||||
|
|
||||||
|
#!/bin/sh
|
||||||
|
if which inkscape; then
|
||||||
|
exit 3 # status code 3 for skip
|
||||||
|
fi
|
||||||
|
inkscape blah blah ...
|
||||||
|
|
||||||
|
I chose status code 3 sort of arbitrarily at first, but it turns out that it
|
||||||
|
would the appropriate status code if these tests were Nagios plugins, as the
|
||||||
|
concept of skipping a test is similar to the Nagios concept of unknown service
|
||||||
|
status (https://nagios-plugins.org/doc/guidelines.html#AEN78).
|
||||||
|
|
||||||
Version 0.0.6
|
Version 0.0.6
|
||||||
---------------------
|
---------------------
|
||||||
|
24
TODO
24
TODO
@@ -1,18 +1,6 @@
|
|||||||
Things I want
|
Things I want
|
||||||
=============
|
=============
|
||||||
|
|
||||||
Skip tests
|
|
||||||
-------------
|
|
||||||
I want tests to be able to decide to skip themselves. For example, if a test
|
|
||||||
requires some dependancy, it might look for the dependency and then skip if it
|
|
||||||
does not see the dependency. It might look like this.
|
|
||||||
|
|
||||||
#!/bin/sh
|
|
||||||
if which parallel > /dev/null; then
|
|
||||||
exit 3 # status code 3 for skip
|
|
||||||
fi
|
|
||||||
parallel blah blah ...
|
|
||||||
|
|
||||||
Test speed
|
Test speed
|
||||||
-------------
|
-------------
|
||||||
Make tests run faster.
|
Make tests run faster.
|
||||||
@@ -31,6 +19,11 @@ Hmm or maybe there's a compromise: Tell people to mount /tmp as a tmpfs so
|
|||||||
that temp files are fast. Maybe allow people to set some other directory as
|
that temp files are fast. Maybe allow people to set some other directory as
|
||||||
the temporary file place, in case they want a different tmpfs location.
|
the temporary file place, in case they want a different tmpfs location.
|
||||||
|
|
||||||
|
In order to run things in parallel, we have to change how we do the
|
||||||
|
stdout_file. I think it's easiest to create separate files for each test and
|
||||||
|
to save them in testroot/.urchin/stdout/$filename. The test root would be
|
||||||
|
defined as the closest ancestor containing a .urchin directory.
|
||||||
|
|
||||||
Options
|
Options
|
||||||
-------------
|
-------------
|
||||||
I want long options. For example, there's presently -f and -e.
|
I want long options. For example, there's presently -f and -e.
|
||||||
@@ -151,6 +144,13 @@ on the nvm tests. I wound up running this instead.
|
|||||||
The Molly guard would be assessed, and the corresponding setup, setup_dir,
|
The Molly guard would be assessed, and the corresponding setup, setup_dir,
|
||||||
teardown, and teardown_dir files would be run in the appropriate order.
|
teardown, and teardown_dir files would be run in the appropriate order.
|
||||||
|
|
||||||
|
In order to know how far up the tree to evaluate the setup, &c. files,
|
||||||
|
I think it would make sense to require that a ".urchin" file be placed in the
|
||||||
|
root of the tests. Urchin would keep going up until it sees this file, and it
|
||||||
|
would evaluate the appropriate setup, &c. files from there down to the
|
||||||
|
particular test file of interest. We would also use this for testing
|
||||||
|
directtories more correctly.
|
||||||
|
|
||||||
Running automated tasks
|
Running automated tasks
|
||||||
-------------------------
|
-------------------------
|
||||||
Urchin might be appropriate for if you have lots of tasks that you want to run
|
Urchin might be appropriate for if you have lots of tasks that you want to run
|
||||||
|
2
tests/.urchin/.gitignore
vendored
Normal file
2
tests/.urchin/.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
*
|
||||||
|
!.gitignore
|
6
tests/Test root contains .urchin./.meta-assess
Executable file
6
tests/Test root contains .urchin./.meta-assess
Executable file
@@ -0,0 +1,6 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
observed=$(../../urchin "$1" .testsuite/a/b)
|
||||||
|
expected=.testsuite/a/b/../..
|
||||||
|
|
||||||
|
test "$observed" = "$expected"
|
2
tests/Test root contains .urchin./.testsuite/.urchin/.gitignore
vendored
Normal file
2
tests/Test root contains .urchin./.testsuite/.urchin/.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
*
|
||||||
|
!.gitignore
|
1
tests/Test root contains .urchin./.testsuite/a/b/c/testcase
Executable file
1
tests/Test root contains .urchin./.testsuite/a/b/c/testcase
Executable file
@@ -0,0 +1 @@
|
|||||||
|
#!/usr/bin/env true
|
2
tests/Test root contains .urchin./Assess root on a file.
Executable file
2
tests/Test root contains .urchin./Assess root on a file.
Executable file
@@ -0,0 +1,2 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
../../urchin --root .testsuite/a/b/c/testcase
|
2
tests/Test root contains .urchin./Assess root with --root.
Executable file
2
tests/Test root contains .urchin./Assess root with --root.
Executable file
@@ -0,0 +1,2 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
./.meta-assess --root
|
2
tests/Test root contains .urchin./Assess root with -r.
Executable file
2
tests/Test root contains .urchin./Assess root with -r.
Executable file
@@ -0,0 +1,2 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
./.meta-assess -r
|
2
tests/Test root contains .urchin./No root set
Executable file
2
tests/Test root contains .urchin./No root set
Executable file
@@ -0,0 +1,2 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
../../urchin --root /bin 2>&1 | grep "'/bin/.urchin'"
|
2
tests/Test root contains .urchin./Wrong filename
Executable file
2
tests/Test root contains .urchin./Wrong filename
Executable file
@@ -0,0 +1,2 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
! ../../urchin --root .testsuite/a/not-a-file
|
4
tests/get_stdout_file/t
Executable file
4
tests/get_stdout_file/t
Executable file
@@ -0,0 +1,4 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
I_AM_SOURCING_URCHIN_IN_A_TEST=true . ../../urchin
|
||||||
|
test $(get_stdout_file t) = aoeu
|
1
tests/skip/.test/fail
Executable file
1
tests/skip/.test/fail
Executable file
@@ -0,0 +1 @@
|
|||||||
|
exit 1
|
1
tests/skip/.test/skip
Executable file
1
tests/skip/.test/skip
Executable file
@@ -0,0 +1 @@
|
|||||||
|
exit 3
|
1
tests/skip/.test/succeed
Executable file
1
tests/skip/.test/succeed
Executable file
@@ -0,0 +1 @@
|
|||||||
|
exit 0
|
1
tests/skip/Skip on exit code 3.
Executable file
1
tests/skip/Skip on exit code 3.
Executable file
@@ -0,0 +1 @@
|
|||||||
|
test $(../../urchin -t .test/ | grep -c SKIP) -eq 1
|
181
urchin
181
urchin
@@ -7,7 +7,7 @@
|
|||||||
# No part of urchin, including this file, may be copied, modified, propagated,
|
# No part of urchin, including this file, may be copied, modified, propagated,
|
||||||
# or distributed except according to the terms contained in the COPYING file.
|
# or distributed except according to the terms contained in the COPYING file.
|
||||||
|
|
||||||
|
set +e
|
||||||
|
|
||||||
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably -
|
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably -
|
||||||
# notably, it can produce output, which breaks fullpath().
|
# notably, it can produce output, which breaks fullpath().
|
||||||
@@ -16,11 +16,42 @@ unset CDPATH
|
|||||||
# Urchin version number
|
# Urchin version number
|
||||||
VERSION=0.0.6
|
VERSION=0.0.6
|
||||||
|
|
||||||
fullpath() {
|
urchin_root() {
|
||||||
(
|
# Call recursively but remember the original argument.
|
||||||
cd -- "$1"
|
current="$(remove_trailing_slash "$1")"
|
||||||
pwd
|
if test -n "$2"; then
|
||||||
)
|
orig="$2"
|
||||||
|
else
|
||||||
|
orig="$1"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if test "$(readlink -f -- "$1")" = /; then
|
||||||
|
# Stop traversing upwards at /
|
||||||
|
if test -d "$orig"; then
|
||||||
|
origdir="$orig"
|
||||||
|
else
|
||||||
|
origdir=$(dirname "$orig")
|
||||||
|
fi
|
||||||
|
echo "You need to create the .urchin directory in the root of your tests,
|
||||||
|
maybe like this:
|
||||||
|
|
||||||
|
mkdir '$(readlink -f -- "$(remove_trailing_slash "$origdir")")/.urchin'
|
||||||
|
" >&2
|
||||||
|
return 1
|
||||||
|
elif ! test -e "$current"; then
|
||||||
|
echo "$current: No such file or directory">&2
|
||||||
|
return 1
|
||||||
|
elif test -f "$current"; then
|
||||||
|
urchin_root "$(dirname "$current")" "$orig"
|
||||||
|
elif test -d "$current"/.urchin; then
|
||||||
|
remove_trailing_slash "$current"
|
||||||
|
elif test "$current" != . && test "$current" != .. &&
|
||||||
|
echo "$current" | grep '^\.' && > /dev/null; then
|
||||||
|
# Stop traversing upwards at hidden directories.
|
||||||
|
urchin_root / "$orig"
|
||||||
|
else
|
||||||
|
urchin_root "$current"/.. "$orig"
|
||||||
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
indent() {
|
indent() {
|
||||||
@@ -30,16 +61,40 @@ indent() {
|
|||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
remove_trailing_slash() {
|
||||||
|
echo "$1" | sed s/\\/$//
|
||||||
|
}
|
||||||
|
|
||||||
|
escape_slashes() {
|
||||||
|
echo "$1" | sed s+/+\\\\/+g
|
||||||
|
}
|
||||||
|
|
||||||
|
get_stdout_file() {
|
||||||
|
root="$(readlink -f -- "$(urchin_root "$1")")"
|
||||||
|
test_file="$(readlink -f -- "$1")"
|
||||||
|
eroot="$(escape_slashes "$root")"
|
||||||
|
base="$(echo "$test_file" | sed "s/^$eroot/$eroot\/.urchin/")"
|
||||||
|
if test -d "$1"; then
|
||||||
|
echo "$base"/.index
|
||||||
|
else
|
||||||
|
echo "$base"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
recurse() {
|
recurse() {
|
||||||
potential_test="$1"
|
potential_test="$1"
|
||||||
indent_level="$2"
|
indent_level="$2"
|
||||||
shell_for_sh_tests="$3"
|
shell_for_sh_tests="$3"
|
||||||
|
|
||||||
|
[ "$potential_test" = '.urchin' ] && return
|
||||||
[ "$potential_test" = 'setup_dir' ] && return
|
[ "$potential_test" = 'setup_dir' ] && return
|
||||||
[ "$potential_test" = 'teardown_dir' ] && return
|
[ "$potential_test" = 'teardown_dir' ] && return
|
||||||
[ "$potential_test" = 'setup' ] && return
|
[ "$potential_test" = 'setup' ] && return
|
||||||
[ "$potential_test" = 'teardown' ] && return
|
[ "$potential_test" = 'teardown' ] && return
|
||||||
|
|
||||||
|
stdout_file="$(get_stdout_file "$potential_test")"
|
||||||
|
mkdir -p "$(dirname "$stdout_file")"
|
||||||
|
|
||||||
[ $indent_level -eq 0 ] && : > "$stdout_file"
|
[ $indent_level -eq 0 ] && : > "$stdout_file"
|
||||||
|
|
||||||
if [ -d "$potential_test" ]
|
if [ -d "$potential_test" ]
|
||||||
@@ -67,8 +122,10 @@ recurse() {
|
|||||||
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
|
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
|
||||||
|
|
||||||
# $2 instead of $indent_level so it doesn't clash
|
# $2 instead of $indent_level so it doesn't clash
|
||||||
recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
|
set +e
|
||||||
|
recurse "${test}" "$(( $2 + 1 ))" "$shell_for_sh_tests"
|
||||||
exit_code=$?
|
exit_code=$?
|
||||||
|
set -e
|
||||||
|
|
||||||
if $exit_on_fail && test $exit_code -ne 0; then
|
if $exit_on_fail && test $exit_code -ne 0; then
|
||||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||||
@@ -103,6 +160,8 @@ recurse() {
|
|||||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||||
if [ $exit_code -eq 0 ]; then
|
if [ $exit_code -eq 0 ]; then
|
||||||
result=success
|
result=success
|
||||||
|
elif [ $exit_code -eq 3 ]; then
|
||||||
|
result=skip
|
||||||
else
|
else
|
||||||
result=fail
|
result=fail
|
||||||
fi
|
fi
|
||||||
@@ -159,7 +218,6 @@ recurse() {
|
|||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
[ $indent_level -eq 0 ] && rm "$stdout_file"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
has_sh_or_no_shebang_line() {
|
has_sh_or_no_shebang_line() {
|
||||||
@@ -189,6 +247,7 @@ $USAGE
|
|||||||
contain the word "test".
|
contain the word "test".
|
||||||
-t Format output in Test Anything Protocol (TAP)
|
-t Format output in Test Anything Protocol (TAP)
|
||||||
-h, --help This help.
|
-h, --help This help.
|
||||||
|
-r, --root Print the Urchin root for a particular file or directory.
|
||||||
-v Display the version number.
|
-v Display the version number.
|
||||||
|
|
||||||
Go to https://github.com/tlevine/urchin for documentation on writing tests.
|
Go to https://github.com/tlevine/urchin for documentation on writing tests.
|
||||||
@@ -209,12 +268,11 @@ plural () {
|
|||||||
}
|
}
|
||||||
|
|
||||||
urchin_go() {
|
urchin_go() {
|
||||||
rm -f "$logfile"
|
|
||||||
if "$tap_format"; then
|
if "$tap_format"; then
|
||||||
printf \#\
|
printf \#\
|
||||||
fi
|
fi
|
||||||
echo Running tests at $(date +%Y-%m-%dT%H:%M:%S)
|
echo Running tests at $(date +%Y-%m-%dT%H:%M:%S)
|
||||||
start=$(date +%s)
|
start="$(date +%s)"
|
||||||
|
|
||||||
# Determine the environment variable to define for test scripts
|
# Determine the environment variable to define for test scripts
|
||||||
# that reflects the specified or implied shell to use for shell-code tests.
|
# that reflects the specified or implied shell to use for shell-code tests.
|
||||||
@@ -229,7 +287,10 @@ urchin_go() {
|
|||||||
TEST_SHELL='/bin/sh'
|
TEST_SHELL='/bin/sh'
|
||||||
fi
|
fi
|
||||||
|
|
||||||
recurse "$1" 0 "$2" # test folder -- indentation level -- [shell to invoke test scripts with]
|
# 1 test folder
|
||||||
|
# 2 indentation level
|
||||||
|
# 3 shell to invoke test scripts with
|
||||||
|
recurse "$1" 0 "$2"
|
||||||
|
|
||||||
finish=$(date +%s)
|
finish=$(date +%s)
|
||||||
elapsed=$(($finish - $start))
|
elapsed=$(($finish - $start))
|
||||||
@@ -251,7 +312,7 @@ urchin_go() {
|
|||||||
printf '%s\n' "$failed $(plural test "$failed") failed."
|
printf '%s\n' "$failed $(plural test "$failed") failed."
|
||||||
printf '\033[m'
|
printf '\033[m'
|
||||||
fi
|
fi
|
||||||
rm -f "$logfile"
|
# rm -f "$logfile"
|
||||||
test -z "$failed" || test "$failed" -eq '0'
|
test -z "$failed" || test "$failed" -eq '0'
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -267,51 +328,57 @@ urchin_molly_guard() {
|
|||||||
exit 1
|
exit 1
|
||||||
}
|
}
|
||||||
|
|
||||||
shell_for_sh_tests=
|
if test -z "$I_AM_SOURCING_URCHIN_IN_A_TEST"; then
|
||||||
force=false
|
shell_for_sh_tests=
|
||||||
exit_on_fail=false
|
force=false
|
||||||
tap_format=false
|
exit_on_fail=false
|
||||||
while [ $# -gt 0 ]
|
tap_format=false
|
||||||
do
|
while [ $# -gt 0 ]
|
||||||
case "$1" in
|
do
|
||||||
-e) exit_on_fail=true;;
|
case "$1" in
|
||||||
-f) force=true;;
|
-e) exit_on_fail=true;;
|
||||||
-s)
|
-f) force=true;;
|
||||||
shift
|
-r|--root)
|
||||||
shell_for_sh_tests=$1
|
shift
|
||||||
which "$shell_for_sh_tests" > /dev/null || {
|
urchin_root "$1"
|
||||||
echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2
|
|
||||||
urchin_help >&2
|
|
||||||
exit 2
|
|
||||||
} ;;
|
|
||||||
-t) tap_format=true;;
|
|
||||||
-h|--help) urchin_help
|
|
||||||
exit 0;;
|
|
||||||
-v) echo "$VERSION"
|
|
||||||
exit;;
|
exit;;
|
||||||
-*) urchin_help >&2
|
-s)
|
||||||
exit 1;;
|
shift
|
||||||
*) break;;
|
shell_for_sh_tests=$1
|
||||||
esac
|
which "$shell_for_sh_tests" > /dev/null || {
|
||||||
shift
|
echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2
|
||||||
done
|
urchin_help >&2
|
||||||
|
exit 11
|
||||||
|
} ;;
|
||||||
|
-t) tap_format=true;;
|
||||||
|
-h|--help) urchin_help
|
||||||
|
exit 0;;
|
||||||
|
-v) echo "$VERSION"
|
||||||
|
exit;;
|
||||||
|
-*) urchin_help >&2
|
||||||
|
exit 1;;
|
||||||
|
*) break;;
|
||||||
|
esac
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
|
||||||
# Verify argument for main stuff
|
# Verify argument for main stuff
|
||||||
if [ "$#" != '1' ] || [ ! -d "$1" ]
|
if [ "$#" != '1' ] || [ ! -d "$1" ]
|
||||||
then
|
then
|
||||||
[ -n "$1" ] && [ ! -d "$1" ] && echo "Not a directory: '$1'" >&2
|
[ -n "$1" ] && [ ! -d "$1" ] && echo "Not a directory: '$1'" >&2
|
||||||
echo "$USAGE" >&2
|
echo "$USAGE" >&2
|
||||||
exit 2
|
exit 11
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Constants
|
# Run or present the Molly guard.
|
||||||
logfile=$(fullpath "$1")/.urchin.log
|
if ! urchin_root "$1" > /dev/null; then
|
||||||
stdout_file=$(fullpath "$1")/.urchin_stdout
|
exit 1
|
||||||
|
elif basename "$(readlink -f -- "$(urchin_root "$1")")" |
|
||||||
# Run or present the Molly guard.
|
grep -Fi 'test' > /dev/null || $force; then
|
||||||
if fullpath "$1" | grep -Fi 'test' > /dev/null || $force
|
logfile="$(readlink -f -- "$(urchin_root "$1")/.urchin/.log")"
|
||||||
then
|
printf '' > "$logfile"
|
||||||
urchin_go "$1" "$shell_for_sh_tests"
|
urchin_go "$1" "$shell_for_sh_tests"
|
||||||
else
|
else
|
||||||
urchin_molly_guard
|
urchin_molly_guard
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
|
Reference in New Issue
Block a user