Compare commits
1 Commits
single-fil
...
more-shell
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7126872a8b |
41
HISTORY
41
HISTORY
@@ -1,44 +1,6 @@
|
|||||||
HISTORY
|
HISTORY
|
||||||
=======
|
=======
|
||||||
|
|
||||||
Version 0.0.7
|
|
||||||
---------------------
|
|
||||||
|
|
||||||
### Molly-guard
|
|
||||||
The Molly-guard is now more accepting. For example, you no longer need to
|
|
||||||
pass -f in this case: https://github.com/creationix/nvm/issues/357
|
|
||||||
|
|
||||||
### Skipping tests
|
|
||||||
Previously, tests were run if they were executable and were otherwise marked
|
|
||||||
as skipped. Now, an executable script can indicate that it is skipped by
|
|
||||||
exiting with code 3. For example, if a test requires some dependancy, it
|
|
||||||
might look for the dependency and then skip if it does not see the dependency.
|
|
||||||
It might look like this.
|
|
||||||
|
|
||||||
#!/bin/sh
|
|
||||||
if which inkscape; then
|
|
||||||
exit 3 # status code 3 for skip
|
|
||||||
fi
|
|
||||||
inkscape blah blah ...
|
|
||||||
|
|
||||||
I chose status code 3 sort of arbitrarily at first, but it turns out that it
|
|
||||||
would the appropriate status code if these tests were Nagios plugins, as the
|
|
||||||
concept of skipping a test is similar to the Nagios concept of unknown service
|
|
||||||
status (https://nagios-plugins.org/doc/guidelines.html#AEN78).
|
|
||||||
|
|
||||||
### Run on a file
|
|
||||||
It is now possible to run urchin on a single file.
|
|
||||||
This occurred to me when I wanted to run
|
|
||||||
|
|
||||||
urchin test/fast/Unit\ tests/nvm_ls_current
|
|
||||||
|
|
||||||
on the nvm tests. I wound up running this instead.
|
|
||||||
|
|
||||||
urchin test/fast/Unit\ tests/ | grep nvm_ls_current
|
|
||||||
|
|
||||||
The Molly guard is assessed, and the corresponding setup, setup_dir,
|
|
||||||
teardown, and teardown_dir files are run in the appropriate order.
|
|
||||||
|
|
||||||
Version 0.0.6
|
Version 0.0.6
|
||||||
---------------------
|
---------------------
|
||||||
|
|
||||||
@@ -52,9 +14,6 @@ Version 0.0.6
|
|||||||
* Remove the undocumented, experimental -x flag now that shall exists.
|
* Remove the undocumented, experimental -x flag now that shall exists.
|
||||||
* Display version number with the -v flag.
|
* Display version number with the -v flag.
|
||||||
* Document why Urchin is called "Urchin"
|
* Document why Urchin is called "Urchin"
|
||||||
* Update TODO
|
|
||||||
* Support mksh (Change a printf command.)
|
|
||||||
* Make long lines shorter.
|
|
||||||
|
|
||||||
These changes are made somewhat separately in the branches "exit-on-fail",
|
These changes are made somewhat separately in the branches "exit-on-fail",
|
||||||
"remove-urchin-x", "tap", and "update-readme". They are rebased into one
|
"remove-urchin-x", "tap", and "update-readme". They are rebased into one
|
||||||
|
|||||||
@@ -1,5 +1,3 @@
|
|||||||
Copyright (c) 2013, 2014, 2015, 2016 Thomas Levine
|
|
||||||
Copyright (c) 2014, Michael Klement
|
|
||||||
Copyright (c) 2012, ScraperWiki Limited
|
Copyright (c) 2012, ScraperWiki Limited
|
||||||
All rights reserved.
|
All rights reserved.
|
||||||
|
|
||||||
136
TODO
136
TODO
@@ -1,136 +0,0 @@
|
|||||||
Things I want
|
|
||||||
=============
|
|
||||||
|
|
||||||
Test speed
|
|
||||||
-------------
|
|
||||||
Make tests run faster.
|
|
||||||
https://github.com/bike-barn/hermit/issues/62
|
|
||||||
|
|
||||||
First, easier thing is probably to run tests in parallel.
|
|
||||||
|
|
||||||
Second, also easier thing is to tell people to save things to RAM rather than
|
|
||||||
disk whenever they can.
|
|
||||||
|
|
||||||
Third, harder thing is to put the test suite in RAM automatically. Maybe the
|
|
||||||
whole test directory, which includes fixtures, gets copied to a tmpfs if one
|
|
||||||
exists.
|
|
||||||
|
|
||||||
Hmm or maybe there's a compromise: Tell people to mount /tmp as a tmpfs so
|
|
||||||
that temp files are fast. Maybe allow people to set some other directory as
|
|
||||||
the temporary file place, in case they want a different tmpfs location.
|
|
||||||
|
|
||||||
Options
|
|
||||||
-------------
|
|
||||||
I want long options. For example, there's presently -f and -e.
|
|
||||||
I want to make them -f|--force and -e|--exit.
|
|
||||||
|
|
||||||
Environment variables
|
|
||||||
-------------
|
|
||||||
Do something to make it easier to debug environment variables, because that is
|
|
||||||
often confusing.
|
|
||||||
https://github.com/creationix/nvm/issues/719
|
|
||||||
https://github.com/creationix/nvm/issues/589
|
|
||||||
|
|
||||||
Documenting that people should run "env" when their tests fail might be good
|
|
||||||
enough.
|
|
||||||
|
|
||||||
Licensing and copyright
|
|
||||||
------------------------
|
|
||||||
* Reference all owners and years in the Copyright file
|
|
||||||
* Consider copyleft licenses
|
|
||||||
* Add license notices to other files if necessary
|
|
||||||
|
|
||||||
Packaging
|
|
||||||
------------
|
|
||||||
Package for package managers.
|
|
||||||
|
|
||||||
* I want NixOS, of course.
|
|
||||||
* Debian is probably the big one.
|
|
||||||
|
|
||||||
Other interesting package managers
|
|
||||||
|
|
||||||
* Update the npm package
|
|
||||||
* Homebrew (for Mac)
|
|
||||||
|
|
||||||
Windows
|
|
||||||
----------
|
|
||||||
Try running Urchin in Windows somehow. Interpreters include
|
|
||||||
|
|
||||||
* CygWin (https://www.cygwin.com/)
|
|
||||||
* MSYS (http://mingw.org/wiki/msys)
|
|
||||||
* GNU on Windows (https://github.com/bmatzelle/gow/wiki)
|
|
||||||
* Git for Windows (https://git-scm.com/download/win)
|
|
||||||
* win-bash (http://win-bash.sourceforge.net/)
|
|
||||||
|
|
||||||
Consider copyleft licenses
|
|
||||||
----------
|
|
||||||
ScraperWiki owns the original version of Urchin (Thomas Levine did the early
|
|
||||||
work as part of his work for ScraperWiki.) and originally licensed it under an
|
|
||||||
MIT-style license. Other people made changes after this original ScraperWiki
|
|
||||||
version. As of January 2016, they are just Thomas Levine (when he wasn't
|
|
||||||
working for ScraperWiki) and Michael Klement.
|
|
||||||
|
|
||||||
The original license was MIT just because that's what ScraperWiki put on
|
|
||||||
everything. Should we change the license?
|
|
||||||
|
|
||||||
The MIT-style license grants pretty much all rights. It says that you need
|
|
||||||
to attribute when you redistribute source code, but you don't
|
|
||||||
necessarily have to redistribute source code.
|
|
||||||
|
|
||||||
A copyleft license adds the restriction that modified versions of the
|
|
||||||
code need to be licensed under the same license. GNU licenses in
|
|
||||||
particular require that source code be released if non-source versions are
|
|
||||||
released, and the different GNU licenses differ in what how the
|
|
||||||
non-source version is defined. (The original, GPL, discusses compiled
|
|
||||||
binaries.) Copyleft doesn't mean anything specific for commercial use.
|
|
||||||
|
|
||||||
MIT-licensed code can be modified and then licensed as GPL, because MIT
|
|
||||||
license allows that, but GPL code can't be modified as MIT, because MIT
|
|
||||||
doesn't allow that. And if we get all of the authors to agree on it, we
|
|
||||||
can always add whatever crazy license we want, regardless of what we
|
|
||||||
have already.
|
|
||||||
|
|
||||||
The distinction between MIT-style and GNU-something might matter quite little
|
|
||||||
in the case of Urchin.
|
|
||||||
|
|
||||||
1. Urchin is written in an interpreted language (shell), so it might be
|
|
||||||
hard to distribute usefully without providing the source code.
|
|
||||||
2. Urchin just runs tests; it doesn't get compiled with the rest of the
|
|
||||||
code (also because it's in shell). Thus, I think a GPL license on
|
|
||||||
Urchin wouldn't infect the code being tested.
|
|
||||||
|
|
||||||
This is as far as I have gotten with contemplating license changes. For now
|
|
||||||
we're sticking with the original MIT-style license, but it's easy to change
|
|
||||||
licenses later.
|
|
||||||
|
|
||||||
Nagios plugins
|
|
||||||
-----------------
|
|
||||||
It would be cool to run Nagios plugins with Urchin. This is already possible,
|
|
||||||
actually, but it might be worth giving some special thought to it.
|
|
||||||
https://nagios-plugins.org/doc/guidelines.html
|
|
||||||
|
|
||||||
Source setup and teardown
|
|
||||||
--------------------
|
|
||||||
If setup and teardown are sourced instead of executed, maybe we can more
|
|
||||||
cleanly create and teardown temporary files.
|
|
||||||
|
|
||||||
(
|
|
||||||
. ./setup
|
|
||||||
./$thetestfile
|
|
||||||
. ./teardown
|
|
||||||
)
|
|
||||||
|
|
||||||
On the other hand, this could just be sourced explicitly in the test file,
|
|
||||||
without the special setup and teardown feature.
|
|
||||||
|
|
||||||
Running automated tasks
|
|
||||||
-------------------------
|
|
||||||
Urchin might be appropriate for if you have lots of tasks that you want to run
|
|
||||||
periodically; add an urchin call to your crontab, and call all of your other
|
|
||||||
tasks with urchin. Here are some features that might make urchin better for
|
|
||||||
this sort of thing.
|
|
||||||
|
|
||||||
* Time how long each test/job takes
|
|
||||||
* Optionally kill tests/jobs after a specific timeout threshold
|
|
||||||
* Send output of different tests/jobs to different files for each file
|
|
||||||
descriptor (STDOUT, STDERR)
|
|
||||||
@@ -1,9 +1,2 @@
|
|||||||
Totally different syntax and similar features, plus TAP output
|
Totally different syntax and similar features, plus TAP output
|
||||||
https://github.com/sstephenson/bats
|
https://github.com/sstephenson/bats
|
||||||
|
|
||||||
Relatively similar interface
|
|
||||||
https://github.com/mlafeldt/sharness
|
|
||||||
|
|
||||||
Lists of alternatives
|
|
||||||
https://thomaslevine.com/!/shell-testing/
|
|
||||||
https://github.com/mlafeldt/sharness#alternatives
|
|
||||||
|
|||||||
@@ -36,7 +36,7 @@ Urchin is contained in a single file, so you can install it by copying it to a
|
|||||||
directory in your `PATH`. For example, you can run the following as root.
|
directory in your `PATH`. For example, you can run the following as root.
|
||||||
|
|
||||||
cd /usr/local/bin
|
cd /usr/local/bin
|
||||||
wget https://raw.githubusercontent.com/tlevine/urchin/v0.0.6/urchin
|
wget https://raw.github.com/tlevine/urchin/master/urchin
|
||||||
chmod +x urchin
|
chmod +x urchin
|
||||||
|
|
||||||
Urchin can be installed with npm too.
|
Urchin can be installed with npm too.
|
||||||
@@ -169,3 +169,9 @@ It does something similar, but the interface may be more intuitive.
|
|||||||
## Alternatives to Urchin
|
## Alternatives to Urchin
|
||||||
Alternatives to Urchin are discussed in
|
Alternatives to Urchin are discussed in
|
||||||
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
|
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
|
||||||
|
|
||||||
|
## Ideas for new features
|
||||||
|
|
||||||
|
* Support [Nagios plugins](https://nagios-plugins.org/doc/guidelines.html)
|
||||||
|
* Stop running if a test fails so one can use Urchin as a
|
||||||
|
[setup framework](https://github.com/tlevine/urchin/issues/16).
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
echo setup
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
echo setup_dir
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
echo teardown
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
echo teardown_dir
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
echo thetest
|
|
||||||
exit 1
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
../../urchin .test
|
|
||||||
# This will exit 0 if it worked.
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
! ../../urchin not-a-file
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
tmp=$(mktemp)
|
|
||||||
echo "Using temp file $tmp"
|
|
||||||
../../urchin .wrappers > $tmp
|
|
||||||
diff $tmp .wrapper-expectation
|
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
exit 0
|
||||||
@@ -1 +0,0 @@
|
|||||||
#!/usr/bin/env true
|
|
||||||
@@ -1 +1,3 @@
|
|||||||
#!/usr/bin/env true
|
#!/bin/sh
|
||||||
|
|
||||||
|
exit 0
|
||||||
|
|||||||
@@ -1,3 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
|
|
||||||
../../urchin .foo/test/bar
|
|
||||||
@@ -1,5 +1,3 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
|
|
||||||
tmp=$(mktemp -d)/blah
|
! ../../urchin ./.chainsaw
|
||||||
echo '#!/usr/bin/env true' > $tmp
|
|
||||||
! ../../urchin $tmp
|
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
exit 1
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
exit 3
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
exit 0
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
test $(../../urchin -t .test/ | grep -c SKIP) -eq 1
|
|
||||||
76
urchin
76
urchin
@@ -1,26 +1,22 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
|
|
||||||
# This file is part of urchin. It is subject to the license terms in the
|
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably - notably, it can produce output,
|
||||||
# COPYING file found in the top-level directory of this distribution or at
|
# which breaks fullpath().
|
||||||
# https://raw.githubusercontent.com/tlevine/urchin/master/COPYING
|
|
||||||
|
|
||||||
# No part of urchin, including this file, may be copied, modified, propagated,
|
|
||||||
# or distributed except according to the terms contained in the COPYING file.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably -
|
|
||||||
# notably, it can produce output, which breaks fullpath().
|
|
||||||
unset CDPATH
|
unset CDPATH
|
||||||
|
|
||||||
# Urchin version number
|
# Urchin version number
|
||||||
VERSION=0.0.6
|
VERSION=0.0.6
|
||||||
|
|
||||||
|
fullpath() {
|
||||||
|
(
|
||||||
|
cd -- "$1"
|
||||||
|
pwd
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
indent() {
|
indent() {
|
||||||
level="$1"
|
level="$1"
|
||||||
if test "$level" -gt 0; then
|
printf "%$((2 * ${level}))s"
|
||||||
printf "%$((2 * ${level}))s"
|
|
||||||
fi
|
|
||||||
}
|
}
|
||||||
|
|
||||||
recurse() {
|
recurse() {
|
||||||
@@ -96,8 +92,6 @@ recurse() {
|
|||||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||||
if [ $exit_code -eq 0 ]; then
|
if [ $exit_code -eq 0 ]; then
|
||||||
result=success
|
result=success
|
||||||
elif [ $exit_code -eq 3 ]; then
|
|
||||||
result=skip
|
|
||||||
else
|
else
|
||||||
result=fail
|
result=fail
|
||||||
fi
|
fi
|
||||||
@@ -137,10 +131,7 @@ recurse() {
|
|||||||
# On fail, print a red '✗'
|
# On fail, print a red '✗'
|
||||||
printf '\033[31m✗ \033[0m'
|
printf '\033[31m✗ \033[0m'
|
||||||
printf '%s\n' "${potential_test}"
|
printf '%s\n' "${potential_test}"
|
||||||
|
printf '\033[31m' # Print output captured from failed test in red.
|
||||||
# Print output captured from failed test in red.
|
|
||||||
printf '\033[31m'
|
|
||||||
|
|
||||||
cat "$stdout_file"
|
cat "$stdout_file"
|
||||||
printf '\033[0m'
|
printf '\033[0m'
|
||||||
;;
|
;;
|
||||||
@@ -158,13 +149,8 @@ recurse() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
has_sh_or_no_shebang_line() {
|
has_sh_or_no_shebang_line() {
|
||||||
# no shebang line at all
|
head -n 1 "$1" | grep -vqE '^#!' && return 0 # no shebang line at all
|
||||||
head -n 1 "$1" | grep -vqE '^#!' && return 0
|
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' && return 0 # shebang line is '#!/bin/sh' or legal variations thereof
|
||||||
|
|
||||||
# shebang line is '#!/bin/sh' or legal variations thereof
|
|
||||||
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' &&
|
|
||||||
return 0
|
|
||||||
|
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -239,10 +225,7 @@ urchin_go() {
|
|||||||
echo "Done, took $elapsed $(plural second $elapsed)."
|
echo "Done, took $elapsed $(plural second $elapsed)."
|
||||||
printf '%s\n' "$passed $(plural test "$passed") passed."
|
printf '%s\n' "$passed $(plural test "$passed") passed."
|
||||||
printf '%s\n' "$skipped $(plural test "$skipped") skipped."
|
printf '%s\n' "$skipped $(plural test "$skipped") skipped."
|
||||||
|
[ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
|
||||||
# If tests failed, print the message in red, otherwise in green.
|
|
||||||
[ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m'
|
|
||||||
|
|
||||||
printf '%s\n' "$failed $(plural test "$failed") failed."
|
printf '%s\n' "$failed $(plural test "$failed") failed."
|
||||||
printf '\033[m'
|
printf '\033[m'
|
||||||
fi
|
fi
|
||||||
@@ -274,11 +257,8 @@ do
|
|||||||
-s)
|
-s)
|
||||||
shift
|
shift
|
||||||
shell_for_sh_tests=$1
|
shell_for_sh_tests=$1
|
||||||
which "$shell_for_sh_tests" > /dev/null || {
|
which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
|
||||||
echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2
|
;;
|
||||||
urchin_help >&2
|
|
||||||
exit 11
|
|
||||||
} ;;
|
|
||||||
-t) tap_format=true;;
|
-t) tap_format=true;;
|
||||||
-h|--help) urchin_help
|
-h|--help) urchin_help
|
||||||
exit 0;;
|
exit 0;;
|
||||||
@@ -291,17 +271,21 @@ do
|
|||||||
shift
|
shift
|
||||||
done
|
done
|
||||||
|
|
||||||
|
# Verify argument for main stuff
|
||||||
|
if [ "$#" != '1' ] || [ ! -d "$1" ]
|
||||||
|
then
|
||||||
|
[ -n "$1" ] && [ ! -d "$1" ] && echo "Not a directory: '$1'" >&2
|
||||||
|
echo "$USAGE" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Constants
|
||||||
|
logfile=$(fullpath "$1")/.urchin.log
|
||||||
|
stdout_file=$(fullpath "$1")/.urchin_stdout
|
||||||
|
|
||||||
# Run or present the Molly guard.
|
# Run or present the Molly guard.
|
||||||
thetest=$(readlink -f "$1")
|
if basename "$(fullpath "$1")" | grep -Fi 'test' > /dev/null || $force
|
||||||
if echo "$thetest" | grep -Fi 'test' > /dev/null || $force; then
|
then
|
||||||
if test -d "$1"; then
|
|
||||||
d="$thetest"
|
|
||||||
else
|
|
||||||
d=$(echo "$thetest" | sed 's/\/[^\/]\{1,\}$//')
|
|
||||||
cd "$d"
|
|
||||||
fi
|
|
||||||
logfile="$d/.urchin.log"
|
|
||||||
stdout_file="$d/.urchin_stdout"
|
|
||||||
urchin_go "$1" "$shell_for_sh_tests"
|
urchin_go "$1" "$shell_for_sh_tests"
|
||||||
else
|
else
|
||||||
urchin_molly_guard
|
urchin_molly_guard
|
||||||
|
|||||||
Binary file not shown.
|
Before Width: | Height: | Size: 99 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 115 KiB |
Reference in New Issue
Block a user