Compare commits
21 Commits
Author | SHA1 | Date | |
---|---|---|---|
|
5e3c918675 | ||
|
f5f98c8c2f | ||
|
ce175cc3df | ||
|
cabaaa7ba4 | ||
|
a663085069 | ||
|
6bb606a3cc | ||
|
1c93e9a5c2 | ||
|
508e695dc3 | ||
|
a51d96631f | ||
|
1cd9991587 | ||
|
0f1c2848b4 | ||
|
9d10e12633 | ||
|
40979f6e18 | ||
|
33e158e8f7 | ||
|
7ecacad132 | ||
|
c8df46014d | ||
|
de2da89169 | ||
|
a6d6730e74 | ||
|
d01e993041 | ||
|
b5c6464eab | ||
|
329fc27929 |
9
AUTHORS
9
AUTHORS
@@ -1,9 +0,0 @@
|
||||
Authors
|
||||
-------
|
||||
David Jones
|
||||
Michael Klement
|
||||
Thomas Levine
|
||||
|
||||
Maintainer
|
||||
-------
|
||||
Thomas Levine <_@thomaslevine.com>
|
20
HISTORY
20
HISTORY
@@ -1,23 +1,5 @@
|
||||
HISTORY
|
||||
=======
|
||||
|
||||
Version 0.0.6
|
||||
---------------------
|
||||
|
||||
* Produce TAP output with the -t flag.
|
||||
* Add a + sign in front of directories in the normal output so that they
|
||||
line up with non-directories.
|
||||
* Display skipped tests in the normal output and in the TAP output.
|
||||
* Correct some things in the documentation.
|
||||
* Rearrange things in the documentation to be more clear.
|
||||
* Pass the -e flag to exit urchin if any single test fails.
|
||||
* Remove the undocumented, experimental -x flag now that shall exists.
|
||||
* Display version number with the -v flag.
|
||||
* Document why Urchin is called "Urchin"
|
||||
|
||||
These changes are made somewhat separately in the branches "exit-on-fail",
|
||||
"remove-urchin-x", "tap", and "update-readme". They are rebased into one
|
||||
branch, "tlevine-2016-02", for merging into "master".
|
||||
-------
|
||||
|
||||
Version 0.0.5
|
||||
---------------------
|
||||
|
111
TODO
111
TODO
@@ -1,111 +0,0 @@
|
||||
Things I want
|
||||
=============
|
||||
|
||||
Molly guard
|
||||
-------------
|
||||
The Molly-guard should be more accepting so that people don't have to use it
|
||||
all the time and thus get used to using it. For example, you shouldn't need to
|
||||
pass -f in this case.
|
||||
https://github.com/creationix/nvm/issues/357
|
||||
|
||||
Test speed
|
||||
-------------
|
||||
Make tests run faster.
|
||||
https://github.com/bike-barn/hermit/issues/62
|
||||
|
||||
First, easier thing is probably to run tests in parallel.
|
||||
|
||||
Second, also easier thing is to tell people to save things to RAM rather than
|
||||
disk whenever they can.
|
||||
|
||||
Third, harder thing is to put the test suite in RAM automatically. Maybe the
|
||||
whole test directory, which includes fixtures, gets copied to a tmpfs if one
|
||||
exists.
|
||||
|
||||
Hmm or maybe there's a compromise: Tell people to mount /tmp as a tmpfs so
|
||||
that temp files are fast. Maybe allow people to set some other directory as
|
||||
the temporary file place, in case they want a different tmpfs location.
|
||||
|
||||
Options
|
||||
-------------
|
||||
I want long options. For example, there's presently -f and -e.
|
||||
I want to make them -f|--force and -e|--exit.
|
||||
|
||||
Environment variables
|
||||
-------------
|
||||
Do something to make it easier to debug environment variables, because that is
|
||||
often confusing.
|
||||
https://github.com/creationix/nvm/issues/719
|
||||
https://github.com/creationix/nvm/issues/589
|
||||
|
||||
Documenting that people should run "env" when their tests fail might be good
|
||||
enough.
|
||||
|
||||
Licensing and copyright
|
||||
------------------------
|
||||
* Reference all owners and years in the Copyright file
|
||||
* Consider copyleft licenses
|
||||
* Add license notices to other files if necessary
|
||||
|
||||
Packaging
|
||||
------------
|
||||
Package for package managers.
|
||||
|
||||
* I want NixOS, of course.
|
||||
* Debian is probably the big one.
|
||||
|
||||
Other interesting package managers
|
||||
|
||||
* Update the npm package
|
||||
* Homebrew (for Mac)
|
||||
|
||||
Windows
|
||||
----------
|
||||
Try running Urchin in Windows somehow. Interpreters include
|
||||
|
||||
* CygWin (https://www.cygwin.com/)
|
||||
* MSYS (http://mingw.org/wiki/msys)
|
||||
* GNU on Windows (https://github.com/bmatzelle/gow/wiki)
|
||||
* Git for Windows (https://git-scm.com/download/win)
|
||||
* win-bash (http://win-bash.sourceforge.net/)
|
||||
|
||||
Consider copyleft licenses
|
||||
----------
|
||||
ScraperWiki owns the original version of Urchin (Thomas Levine did the early
|
||||
work as part of his work for ScraperWiki.) and originally licensed it under an
|
||||
MIT-style license. Other people made changes after this original ScraperWiki
|
||||
version. As of January 2016, they are just Thomas Levine (when he wasn't
|
||||
working for ScraperWiki) and Michael Klement.
|
||||
|
||||
The original license was MIT just because that's what ScraperWiki put on
|
||||
everything. Should we change the license?
|
||||
|
||||
The MIT-style license grants pretty much all rights. It says that you need
|
||||
to attribute when you redistribute source code, but you don't
|
||||
necessarily have to redistribute source code.
|
||||
|
||||
A copyleft license adds the restriction that modified versions of the
|
||||
code need to be licensed under the same license. GNU licenses in
|
||||
particular require that source code be released if non-source versions are
|
||||
released, and the different GNU licenses differ in what how the
|
||||
non-source version is defined. (The original, GPL, discusses compiled
|
||||
binaries.) Copyleft doesn't mean anything specific for commercial use.
|
||||
|
||||
MIT-licensed code can be modified and then licensed as GPL, because MIT
|
||||
license allows that, but GPL code can't be modified as MIT, because MIT
|
||||
doesn't allow that. And if we get all of the authors to agree on it, we
|
||||
can always add whatever crazy license we want, regardless of what we
|
||||
have already.
|
||||
|
||||
The distinction between MIT-style and GNU-something might matter quite little
|
||||
in the case of Urchin.
|
||||
|
||||
1. Urchin is written in an interpreted language (shell), so it might be
|
||||
hard to distribute usefully without providing the source code.
|
||||
2. Urchin just runs tests; it doesn't get compiled with the rest of the
|
||||
code (also because it's in shell). Thus, I think a GPL license on
|
||||
Urchin wouldn't infect the code being tested.
|
||||
|
||||
This is as far as I have gotten with contemplating license changes. For now
|
||||
we're sticking with the original MIT-style license, but it's easy to change
|
||||
licenses later.
|
@@ -6,7 +6,7 @@ for shell in dash bash ksh zsh; do
|
||||
if which $shell > /dev/null 2> /dev/null; then
|
||||
echo
|
||||
echo Running urchin tests in $shell
|
||||
$shell urchin -s $shell tests | tail -n 4
|
||||
$shell urchin -s $shell tests | tail -n 3
|
||||
else
|
||||
echo
|
||||
echo Skipping $shell because it is not in the PATH
|
||||
|
73
readme.md
73
readme.md
@@ -4,13 +4,9 @@
|
||||
/ /_/ / / / /__/ / / / / / / /
|
||||
\__,_/_/ \___/_/ /_/_/_/ /_/
|
||||
|
||||
Urchin is a file-based test harness, normally used for testing shell programs.
|
||||
It is written in portable shell and should thus work on GNU/Linux, BSD
|
||||
(including Mac OS X), and other Unix-like platforms.
|
||||
|
||||
Urchin is called "Urchin" because
|
||||
[sea urchins](https://en.wikipedia.org/wiki/Sea_urchin)
|
||||
have shells called "tests".
|
||||
Urchin is a test framework for shell. It is implemented in
|
||||
portable /bin/sh and should work on GNU/Linux, Mac OS X, and
|
||||
other Unix platforms.
|
||||
|
||||
## Try it out
|
||||
Urchin's tests are written in Urchin, so you can run them to see what Urchin
|
||||
@@ -31,15 +27,14 @@ run this:
|
||||
cd urchin
|
||||
./cross-shell-tests
|
||||
|
||||
## Install
|
||||
Urchin is contained in a single file, so you can install it by copying it to a
|
||||
directory in your `PATH`. For example, you can run the following as root.
|
||||
## Globally
|
||||
Download Urchin like so (as root) (or use npm, below):
|
||||
|
||||
cd /usr/local/bin
|
||||
wget https://raw.github.com/tlevine/urchin/master/urchin
|
||||
chmod +x urchin
|
||||
|
||||
Urchin can be installed with npm too.
|
||||
Can be installed with npm too:
|
||||
|
||||
npm install -g urchin
|
||||
|
||||
@@ -91,14 +86,15 @@ Files are only run if they are executable, and files beginning with `.` are
|
||||
ignored. Thus, fixtures and libraries can be included sloppily within the test
|
||||
directory tree. The test passes if the file exits 0; otherwise, it fails.
|
||||
|
||||
Tests files and subdirectories are run in ASCIIbetical order within each
|
||||
directory; that is,
|
||||
In case you care about the order in which your tests execute, consider that
|
||||
urchin looks for files within a directory in the following manner.
|
||||
|
||||
for file in *; do
|
||||
do_something_with_test_file $file
|
||||
done
|
||||
|
||||
Tests within a directory are executed in whatever order `*` returns.
|
||||
|
||||
### Writing cross-shell compatibility tests for testing shell code
|
||||
|
||||
While you could write your test scripts to explicitly invoke the functionality
|
||||
@@ -110,20 +106,24 @@ The specific approach depends on your test scenario:
|
||||
* (b) Your scripts _source_ scripts containing portable shell code.
|
||||
|
||||
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
|
||||
Urchin sets the `TEST_SHELL` environment variable so that you may change the
|
||||
shell with which your tests call other shell programs. To run your test
|
||||
scripts in multiple shells you must call `$TEST_SHELL` in your tests and then
|
||||
run urchin with the appropriate option.
|
||||
|
||||
First, consider using [shall](https://github.com/mklement0/shall).
|
||||
|
||||
#!/usr/bin/env shall
|
||||
echo This is a test file.
|
||||
|
||||
Alternatively, you can use urchin's built-in recognition of the
|
||||
`TEST_SHELL` environment variable.
|
||||
In your test scripts, invoke the shell scripts to test via the shell
|
||||
specified in environment variable `TEST_SHELL` rather than directly;
|
||||
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
|
||||
Note that if you alsow want your test scripts to work when run directly,
|
||||
outside of Urchin, be sure to target scripts that happen to be in the
|
||||
current directory with prefix `./`; e.g., `$TEST_SHELL ./baz`
|
||||
(rather than `$TEST_SHELL baz`).
|
||||
|
||||
On invocation of Urchin, prepend a definition of environment variable
|
||||
`TEST_SHELL` specifying the shell to test with, e.g.,
|
||||
|
||||
TEST_SHELL=zsh urchin ./tests
|
||||
|
||||
Then, on invocation of Urchin, prepend a definition of environment variable
|
||||
`TEST_SHELL` specifying the shell to test with, e.g.: `TEST_SHELL=zsh urchin ./tests`.
|
||||
To test with multiple shells in sequence, use something like:
|
||||
|
||||
for shell in sh bash ksh zsh; do
|
||||
@@ -131,20 +131,14 @@ To test with multiple shells in sequence, use something like:
|
||||
done
|
||||
|
||||
If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
|
||||
scripts can rely on `$TEST_SHELL` always containing a value when Urchin runs
|
||||
them.
|
||||
|
||||
That said, we still recommand that you account for the possibility that
|
||||
`$TEST_SHELL` does not contain a value so that you may run your test scripts
|
||||
without Urchin. Supporting this case is very simple; when you invoke scripts
|
||||
that happen to be in the current directory, be sure to use the prefix `./`,
|
||||
e.g., `$TEST_SHELL ./baz` rather than `$TEST_SHELL baz`.
|
||||
scripts can rely on `$TEST_SHELL` always containing a value.
|
||||
|
||||
#### (b) Cross-shell tests with test scripts that _source_ shell scripts
|
||||
|
||||
If you _source_ shell code in your test scripts, it is the test scripts
|
||||
themselves that must be run with the shell specified.
|
||||
|
||||
Urchin supports the `-s <shell>` option, which instructs
|
||||
To that end, Urchin supports the `-s <shell>` option, which instructs
|
||||
Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`.
|
||||
(In addition, Urchin sets environment variable `TEST_SHELL` to the specified
|
||||
shell.)
|
||||
@@ -160,12 +154,21 @@ To test with multiple shells in sequence, use something like:
|
||||
urchin -s $shell ./tests
|
||||
done
|
||||
|
||||
Also consider using [shall](https://github.com/mklement0/shall).
|
||||
It does something similar, but the interface may be more intuitive.
|
||||
<!--
|
||||
#### (c) Cross shell tests with `urchin -x` (experimental)
|
||||
If you run urchin with the `-x` flag, it will be as if you ran
|
||||
`$TEST_SHELL`. Unless `$TEST_SHELL` isn't set, in which case it'll
|
||||
be as if you ran `/bin/sh`. Putting this in she shebang line might
|
||||
eventually work out to be a cleaner way of doing cross-shell testing.
|
||||
|
||||
#!/usr/bin/env shall
|
||||
echo This is a test file.
|
||||
#!/usr/bin/env urchin -x
|
||||
test a = a
|
||||
|
||||
It might make sense if you do this.
|
||||
|
||||
export TEST_SHELL=zsh && urchin -x
|
||||
export TEST_SHELL=bash && urchin -x
|
||||
-->
|
||||
## Alternatives to Urchin
|
||||
Alternatives to Urchin are discussed in
|
||||
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
|
||||
|
@@ -1 +0,0 @@
|
||||
false
|
@@ -1 +0,0 @@
|
||||
false
|
@@ -1 +0,0 @@
|
||||
false
|
@@ -1,11 +0,0 @@
|
||||
tmp=$(mktemp)
|
||||
../urchin -e -f ./.die-on-fail > $tmp
|
||||
result=$?
|
||||
|
||||
grep '1 should run.' $tmp
|
||||
grep '2 should run.' $tmp
|
||||
grep -v '3 should not run.' $tmp
|
||||
grep -v '4 should not run.' $tmp
|
||||
|
||||
rm $tmp
|
||||
exit $result
|
@@ -1,2 +0,0 @@
|
||||
#!/bin/sh
|
||||
../urchin -v | grep '[0-9.]\{3,\}'
|
26
urchin
26
urchin
@@ -4,9 +4,6 @@
|
||||
# which breaks fullpath().
|
||||
unset CDPATH
|
||||
|
||||
# Urchin version number
|
||||
VERSION=0.0.6
|
||||
|
||||
fullpath() {
|
||||
(
|
||||
cd -- "$1"
|
||||
@@ -57,13 +54,6 @@ recurse() {
|
||||
|
||||
# $2 instead of $indent_level so it doesn't clash
|
||||
recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
|
||||
exit_code=$?
|
||||
|
||||
if $exit_on_fail && test $exit_code -ne 0; then
|
||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||
[ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
|
||||
return 1
|
||||
fi
|
||||
|
||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||
done
|
||||
@@ -78,6 +68,7 @@ recurse() {
|
||||
else
|
||||
if [ -x "$potential_test" ]
|
||||
then
|
||||
|
||||
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
|
||||
|
||||
# Run the test
|
||||
@@ -89,6 +80,7 @@ recurse() {
|
||||
fi
|
||||
exit_code="$?"
|
||||
|
||||
|
||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||
if [ $exit_code -eq 0 ]; then
|
||||
result=success
|
||||
@@ -140,10 +132,6 @@ recurse() {
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
if $exit_on_fail && test 0 -ne $exit_code; then
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
[ $indent_level -eq 0 ] && rm "$stdout_file"
|
||||
}
|
||||
@@ -163,14 +151,10 @@ $USAGE
|
||||
|
||||
-s <shell> Invoke test scripts that either have no shebang line at all or
|
||||
have shebang line "#!/bin/sh" with the specified shell.
|
||||
-e Stop running if any single test fails. This is helpful if you want
|
||||
to use Urchin to run things other than tests, such as a set of
|
||||
configuration scripts.
|
||||
-f Force running even if the test directory's name does not
|
||||
contain the word "test".
|
||||
-t Format output in Test Anything Protocol (TAP)
|
||||
-h, --help This help.
|
||||
-v Display the version number.
|
||||
-h This help.
|
||||
|
||||
Go to https://github.com/tlevine/urchin for documentation on writing tests.
|
||||
|
||||
@@ -247,12 +231,10 @@ urchin_molly_guard() {
|
||||
|
||||
shell_for_sh_tests=
|
||||
force=false
|
||||
exit_on_fail=false
|
||||
tap_format=false
|
||||
while [ $# -gt 0 ]
|
||||
do
|
||||
case "$1" in
|
||||
-e) exit_on_fail=true;;
|
||||
-f) force=true;;
|
||||
-s)
|
||||
shift
|
||||
@@ -262,8 +244,6 @@ do
|
||||
-t) tap_format=true;;
|
||||
-h|--help) urchin_help
|
||||
exit 0;;
|
||||
-v) echo "$VERSION"
|
||||
exit;;
|
||||
-*) urchin_help >&2
|
||||
exit 1;;
|
||||
*) break;;
|
||||
|
Binary file not shown.
Before Width: | Height: | Size: 99 KiB |
Binary file not shown.
Before Width: | Height: | Size: 115 KiB |
Reference in New Issue
Block a user