19 Commits

Author SHA1 Message Date
Thomas Levine
1737b4cbbb source setup and teardown 2016-02-26 16:32:52 +00:00
Thomas Levine
3f0e925975 move molly-guard change from todo to history 2016-02-26 16:26:22 +00:00
Thomas Levine
43674f12e2 more accepting molly-guard 2016-02-26 16:23:41 +00:00
Thomas Levine
9bb1366549 simpler truth test 2016-02-26 16:19:32 +00:00
Thomas Levine
643a1bf576 test a more accepting molly-guard 2016-02-26 16:18:49 +00:00
Thomas Levine
08ea7bb88f specify version in readme 2016-02-26 16:14:54 +00:00
Thomas Levine
e5ada4899a task runner ideas 2016-02-26 15:57:21 +00:00
Thomas Levine
c818033faa list alternatives 2016-02-14 16:38:02 +00:00
Thomas Levine
88572b6c90 to support single file runs 2016-02-14 07:44:07 +00:00
Thomas Levine
3e5e7334e8 sourcing idea 2016-02-11 20:18:35 +00:00
Thomas Levine
87bb9f4660 link to COPYING file on web 2016-02-08 16:09:00 +00:00
Thomas Levine
caabcd25ff update HISTORY 2016-02-08 16:06:22 +00:00
Thomas Levine
8f82cf5aeb make long lines shorter 2016-02-08 16:05:56 +00:00
Thomas Levine
385ec3fe81 add a copying notice to the urchin excutable 2016-02-08 16:02:03 +00:00
Thomas Levine
d4eb99ffb0 rename LICENSE -> COPYING 2016-02-08 15:59:56 +00:00
Thomas Levine
ed939ff39a update history 2016-02-08 15:59:47 +00:00
Thomas Levine
d24a71d0d7 copyright notice
add Thomas Levine and Michael Klement
2016-02-08 15:59:16 +00:00
Thomas Levine
ceae1811e2 nagios todo 2016-02-08 15:51:53 +00:00
Thomas Levine
2ac3a52707 fix printf to work in mksh 2016-02-08 15:50:12 +00:00
12 changed files with 120 additions and 40 deletions

View File

@@ -1,3 +1,5 @@
Copyright (c) 2013, 2014, 2015, 2016 Thomas Levine
Copyright (c) 2014, Michael Klement
Copyright (c) 2012, ScraperWiki Limited Copyright (c) 2012, ScraperWiki Limited
All rights reserved. All rights reserved.

10
HISTORY
View File

@@ -1,6 +1,13 @@
HISTORY HISTORY
======= =======
Version 0.0.7
---------------------
The Molly-guard is now more accepting. For example, you no longer need to
pass -f in this case: https://github.com/creationix/nvm/issues/357
Version 0.0.6 Version 0.0.6
--------------------- ---------------------
@@ -14,6 +21,9 @@ Version 0.0.6
* Remove the undocumented, experimental -x flag now that shall exists. * Remove the undocumented, experimental -x flag now that shall exists.
* Display version number with the -v flag. * Display version number with the -v flag.
* Document why Urchin is called "Urchin" * Document why Urchin is called "Urchin"
* Update TODO
* Support mksh (Change a printf command.)
* Make long lines shorter.
These changes are made somewhat separately in the branches "exit-on-fail", These changes are made somewhat separately in the branches "exit-on-fail",
"remove-urchin-x", "tap", and "update-readme". They are rebased into one "remove-urchin-x", "tap", and "update-readme". They are rebased into one

55
TODO
View File

@@ -1,13 +1,6 @@
Things I want Things I want
============= =============
Molly guard
-------------
The Molly-guard should be more accepting so that people don't have to use it
all the time and thus get used to using it. For example, you shouldn't need to
pass -f in this case.
https://github.com/creationix/nvm/issues/357
Test speed Test speed
------------- -------------
Make tests run faster. Make tests run faster.
@@ -109,3 +102,51 @@ in the case of Urchin.
This is as far as I have gotten with contemplating license changes. For now This is as far as I have gotten with contemplating license changes. For now
we're sticking with the original MIT-style license, but it's easy to change we're sticking with the original MIT-style license, but it's easy to change
licenses later. licenses later.
Nagios plugins
-----------------
It would be cool to run Nagios plugins with Urchin. This is already possible,
actually, but it might be worth giving some special thought to it.
https://nagios-plugins.org/doc/guidelines.html
Source setup and teardown
--------------------
If setup and teardown are sourced instead of executed, maybe we can more
cleanly create and teardown temporary files.
(
. ./setup
./$thetestfile
. ./teardown
)
On the other hand, this could just be sourced explicitly in the test file,
without the special setup and teardown feature.
Run on a file
----------------
Presently you can run urchin only on a directory.
It would be neat if you could run it on a file as well.
This occurred to me when I wanted to run
urchin test/fast/Unit\ tests/nvm_ls_current
on the nvm tests. I wound up running this instead.
urchin test/fast/Unit\ tests/ | grep nvm_ls_current
The Molly guard would be assessed, and the corresponding setup, setup_dir,
teardown, and teardown_dir files would be run in the appropriate order.
Running automated tasks
-------------------------
Urchin might be appropriate for if you have lots of tasks that you want to run
periodically; add an urchin call to your crontab, and call all of your other
tasks with urchin. Here are some features that might make urchin better for
this sort of thing.
* Time how long each test/job takes
* Optionally kill tests/jobs after a specific timeout threshold
* Send output of different tests/jobs to different files for each file
descriptor (STDOUT, STDERR)

View File

@@ -1,2 +1,9 @@
Totally different syntax and similar features, plus TAP output Totally different syntax and similar features, plus TAP output
https://github.com/sstephenson/bats https://github.com/sstephenson/bats
Relatively similar interface
https://github.com/mlafeldt/sharness
Lists of alternatives
https://thomaslevine.com/!/shell-testing/
https://github.com/mlafeldt/sharness#alternatives

View File

@@ -2,7 +2,7 @@
# Run urchin in a bunch of different shells, # Run urchin in a bunch of different shells,
# including a shell that isn't quite POSIX-compatible (zsh) # including a shell that isn't quite POSIX-compatible (zsh)
for shell in dash bash ksh zsh; do for shell in dash bash mksh ksh zsh; do
if which $shell > /dev/null 2> /dev/null; then if which $shell > /dev/null 2> /dev/null; then
echo echo
echo Running urchin tests in $shell echo Running urchin tests in $shell

View File

@@ -36,7 +36,7 @@ Urchin is contained in a single file, so you can install it by copying it to a
directory in your `PATH`. For example, you can run the following as root. directory in your `PATH`. For example, you can run the following as root.
cd /usr/local/bin cd /usr/local/bin
wget https://raw.github.com/tlevine/urchin/master/urchin wget https://raw.githubusercontent.com/tlevine/urchin/v0.0.6/urchin
chmod +x urchin chmod +x urchin
Urchin can be installed with npm too. Urchin can be installed with npm too.
@@ -169,9 +169,3 @@ It does something similar, but the interface may be more intuitive.
## Alternatives to Urchin ## Alternatives to Urchin
Alternatives to Urchin are discussed in Alternatives to Urchin are discussed in
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/). [this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
## Ideas for new features
* Support [Nagios plugins](https://nagios-plugins.org/doc/guidelines.html)
* Stop running if a test fails so one can use Urchin as a
[setup framework](https://github.com/tlevine/urchin/issues/16).

View File

@@ -1,3 +1,5 @@
#!/bin/sh #!/bin/sh
! ../../urchin ./.chainsaw tmp=$(mktemp -d)/blah
echo '#!/usr/bin/env true' > $tmp
! ../../urchin $tmp

61
urchin
View File

@@ -1,7 +1,16 @@
#!/bin/sh #!/bin/sh
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably - notably, it can produce output, # This file is part of urchin. It is subject to the license terms in the
# which breaks fullpath(). # COPYING file found in the top-level directory of this distribution or at
# https://raw.githubusercontent.com/tlevine/urchin/master/COPYING
# No part of urchin, including this file, may be copied, modified, propagated,
# or distributed except according to the terms contained in the COPYING file.
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably -
# notably, it can produce output, which breaks fullpath().
unset CDPATH unset CDPATH
# Urchin version number # Urchin version number
@@ -16,7 +25,9 @@ fullpath() {
indent() { indent() {
level="$1" level="$1"
printf "%$((2 * ${level}))s" if test "$level" -gt 0; then
printf "%$((2 * ${level}))s"
fi
} }
recurse() { recurse() {
@@ -44,7 +55,7 @@ recurse() {
( (
cd -- "$potential_test" cd -- "$potential_test"
[ -f setup_dir ] && [ -x setup_dir ] && ./setup_dir >> "$stdout_file" [ -f setup_dir ] && [ -x setup_dir ] && . ./setup_dir >> "$stdout_file"
if [ -n "$ZSH_VERSION" ]; then if [ -n "$ZSH_VERSION" ]; then
# avoid "no matches found: *" error when directories are empty # avoid "no matches found: *" error when directories are empty
@@ -53,21 +64,21 @@ recurse() {
for test in * for test in *
do do
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file" [ -f setup ] && [ -x setup ] && . ./setup >> "$stdout_file"
# $2 instead of $indent_level so it doesn't clash # $2 instead of $indent_level so it doesn't clash
recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests" recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
exit_code=$? exit_code=$?
if $exit_on_fail && test $exit_code -ne 0; then if $exit_on_fail && test $exit_code -ne 0; then
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file" [ -f teardown ] && [ -x teardown ] && . ./teardown >> "$stdout_file"
[ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file" [ -f teardown_dir ] && [ -x teardown_dir ] && . ./teardown_dir >> "$stdout_file"
return 1 return 1
fi fi
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file" [ -f teardown ] && [ -x teardown ] && . ./teardown >> "$stdout_file"
done done
[ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file" [ -f teardown_dir ] && [ -x teardown_dir ] && . ./teardown_dir >> "$stdout_file"
) )
if $tap_format; then if $tap_format; then
indent $indent_level | sed 's/ /#/g' indent $indent_level | sed 's/ /#/g'
@@ -78,7 +89,7 @@ recurse() {
else else
if [ -x "$potential_test" ] if [ -x "$potential_test" ]
then then
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file" [ -f setup ] && [ -x setup ] && . ./setup >> "$stdout_file"
# Run the test # Run the test
if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test" if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
@@ -89,7 +100,7 @@ recurse() {
fi fi
exit_code="$?" exit_code="$?"
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file" [ -f teardown ] && [ -x teardown ] && . ./teardown >> "$stdout_file"
if [ $exit_code -eq 0 ]; then if [ $exit_code -eq 0 ]; then
result=success result=success
else else
@@ -131,7 +142,10 @@ recurse() {
# On fail, print a red '✗' # On fail, print a red '✗'
printf '\033[31m✗ \033[0m' printf '\033[31m✗ \033[0m'
printf '%s\n' "${potential_test}" printf '%s\n' "${potential_test}"
printf '\033[31m' # Print output captured from failed test in red.
# Print output captured from failed test in red.
printf '\033[31m'
cat "$stdout_file" cat "$stdout_file"
printf '\033[0m' printf '\033[0m'
;; ;;
@@ -149,8 +163,13 @@ recurse() {
} }
has_sh_or_no_shebang_line() { has_sh_or_no_shebang_line() {
head -n 1 "$1" | grep -vqE '^#!' && return 0 # no shebang line at all # no shebang line at all
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' && return 0 # shebang line is '#!/bin/sh' or legal variations thereof head -n 1 "$1" | grep -vqE '^#!' && return 0
# shebang line is '#!/bin/sh' or legal variations thereof
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' &&
return 0
return 1 return 1
} }
@@ -225,7 +244,10 @@ urchin_go() {
echo "Done, took $elapsed $(plural second $elapsed)." echo "Done, took $elapsed $(plural second $elapsed)."
printf '%s\n' "$passed $(plural test "$passed") passed." printf '%s\n' "$passed $(plural test "$passed") passed."
printf '%s\n' "$skipped $(plural test "$skipped") skipped." printf '%s\n' "$skipped $(plural test "$skipped") skipped."
[ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
# If tests failed, print the message in red, otherwise in green.
[ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m'
printf '%s\n' "$failed $(plural test "$failed") failed." printf '%s\n' "$failed $(plural test "$failed") failed."
printf '\033[m' printf '\033[m'
fi fi
@@ -257,8 +279,11 @@ do
-s) -s)
shift shift
shell_for_sh_tests=$1 shell_for_sh_tests=$1
which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; } which "$shell_for_sh_tests" > /dev/null || {
;; echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2
urchin_help >&2
exit 2
} ;;
-t) tap_format=true;; -t) tap_format=true;;
-h|--help) urchin_help -h|--help) urchin_help
exit 0;; exit 0;;
@@ -284,7 +309,7 @@ logfile=$(fullpath "$1")/.urchin.log
stdout_file=$(fullpath "$1")/.urchin_stdout stdout_file=$(fullpath "$1")/.urchin_stdout
# Run or present the Molly guard. # Run or present the Molly guard.
if basename "$(fullpath "$1")" | grep -Fi 'test' > /dev/null || $force if fullpath "$1" | grep -Fi 'test' > /dev/null || $force
then then
urchin_go "$1" "$shell_for_sh_tests" urchin_go "$1" "$shell_for_sh_tests"
else else