Compare commits
4 Commits
Author | SHA1 | Date | |
---|---|---|---|
|
560b55d1fb | ||
|
6f51dd7abb | ||
|
284077d1a1 | ||
|
3470e4b979 |
9
AUTHORS
9
AUTHORS
@@ -1,9 +0,0 @@
|
|||||||
Authors
|
|
||||||
-------
|
|
||||||
David Jones
|
|
||||||
Michael Klement
|
|
||||||
Thomas Levine
|
|
||||||
|
|
||||||
Maintainer
|
|
||||||
-------
|
|
||||||
Thomas Levine <_@thomaslevine.com>
|
|
28
HISTORY
28
HISTORY
@@ -1,31 +1,5 @@
|
|||||||
HISTORY
|
HISTORY
|
||||||
=======
|
-------
|
||||||
|
|
||||||
Version 0.0.6
|
|
||||||
---------------------
|
|
||||||
|
|
||||||
* Produce TAP output with the -t flag.
|
|
||||||
* Add a + sign in front of directories in the normal output so that they
|
|
||||||
line up with non-directories.
|
|
||||||
* Display skipped tests in the normal output and in the TAP output.
|
|
||||||
* Correct some things in the documentation.
|
|
||||||
* Rearrange things in the documentation to be more clear.
|
|
||||||
* Pass the -e flag to exit urchin if any single test fails.
|
|
||||||
* Remove the undocumented, experimental -x flag now that shall exists.
|
|
||||||
* Display version number with the -v flag.
|
|
||||||
* Document why Urchin is called "Urchin"
|
|
||||||
* Update TODO
|
|
||||||
* Support mksh (Change a printf command.)
|
|
||||||
* Make long lines shorter.
|
|
||||||
|
|
||||||
These changes are made somewhat separately in the branches "exit-on-fail",
|
|
||||||
"remove-urchin-x", "tap", and "update-readme". They are rebased into one
|
|
||||||
branch, "tlevine-2016-02", for merging into "master".
|
|
||||||
|
|
||||||
Version 0.0.5
|
|
||||||
---------------------
|
|
||||||
* urchin now unsets `CDPATH`.
|
|
||||||
* The documentation for `urchin -x` was removed because it was confusing.
|
|
||||||
|
|
||||||
Version 0.0.4
|
Version 0.0.4
|
||||||
---------------------
|
---------------------
|
||||||
|
@@ -1,5 +1,3 @@
|
|||||||
Copyright (c) 2013, 2014, 2015, 2016 Thomas Levine
|
|
||||||
Copyright (c) 2014, Michael Klement
|
|
||||||
Copyright (c) 2012, ScraperWiki Limited
|
Copyright (c) 2012, ScraperWiki Limited
|
||||||
All rights reserved.
|
All rights reserved.
|
||||||
|
|
117
TODO
117
TODO
@@ -1,117 +0,0 @@
|
|||||||
Things I want
|
|
||||||
=============
|
|
||||||
|
|
||||||
Molly guard
|
|
||||||
-------------
|
|
||||||
The Molly-guard should be more accepting so that people don't have to use it
|
|
||||||
all the time and thus get used to using it. For example, you shouldn't need to
|
|
||||||
pass -f in this case.
|
|
||||||
https://github.com/creationix/nvm/issues/357
|
|
||||||
|
|
||||||
Test speed
|
|
||||||
-------------
|
|
||||||
Make tests run faster.
|
|
||||||
https://github.com/bike-barn/hermit/issues/62
|
|
||||||
|
|
||||||
First, easier thing is probably to run tests in parallel.
|
|
||||||
|
|
||||||
Second, also easier thing is to tell people to save things to RAM rather than
|
|
||||||
disk whenever they can.
|
|
||||||
|
|
||||||
Third, harder thing is to put the test suite in RAM automatically. Maybe the
|
|
||||||
whole test directory, which includes fixtures, gets copied to a tmpfs if one
|
|
||||||
exists.
|
|
||||||
|
|
||||||
Hmm or maybe there's a compromise: Tell people to mount /tmp as a tmpfs so
|
|
||||||
that temp files are fast. Maybe allow people to set some other directory as
|
|
||||||
the temporary file place, in case they want a different tmpfs location.
|
|
||||||
|
|
||||||
Options
|
|
||||||
-------------
|
|
||||||
I want long options. For example, there's presently -f and -e.
|
|
||||||
I want to make them -f|--force and -e|--exit.
|
|
||||||
|
|
||||||
Environment variables
|
|
||||||
-------------
|
|
||||||
Do something to make it easier to debug environment variables, because that is
|
|
||||||
often confusing.
|
|
||||||
https://github.com/creationix/nvm/issues/719
|
|
||||||
https://github.com/creationix/nvm/issues/589
|
|
||||||
|
|
||||||
Documenting that people should run "env" when their tests fail might be good
|
|
||||||
enough.
|
|
||||||
|
|
||||||
Licensing and copyright
|
|
||||||
------------------------
|
|
||||||
* Reference all owners and years in the Copyright file
|
|
||||||
* Consider copyleft licenses
|
|
||||||
* Add license notices to other files if necessary
|
|
||||||
|
|
||||||
Packaging
|
|
||||||
------------
|
|
||||||
Package for package managers.
|
|
||||||
|
|
||||||
* I want NixOS, of course.
|
|
||||||
* Debian is probably the big one.
|
|
||||||
|
|
||||||
Other interesting package managers
|
|
||||||
|
|
||||||
* Update the npm package
|
|
||||||
* Homebrew (for Mac)
|
|
||||||
|
|
||||||
Windows
|
|
||||||
----------
|
|
||||||
Try running Urchin in Windows somehow. Interpreters include
|
|
||||||
|
|
||||||
* CygWin (https://www.cygwin.com/)
|
|
||||||
* MSYS (http://mingw.org/wiki/msys)
|
|
||||||
* GNU on Windows (https://github.com/bmatzelle/gow/wiki)
|
|
||||||
* Git for Windows (https://git-scm.com/download/win)
|
|
||||||
* win-bash (http://win-bash.sourceforge.net/)
|
|
||||||
|
|
||||||
Consider copyleft licenses
|
|
||||||
----------
|
|
||||||
ScraperWiki owns the original version of Urchin (Thomas Levine did the early
|
|
||||||
work as part of his work for ScraperWiki.) and originally licensed it under an
|
|
||||||
MIT-style license. Other people made changes after this original ScraperWiki
|
|
||||||
version. As of January 2016, they are just Thomas Levine (when he wasn't
|
|
||||||
working for ScraperWiki) and Michael Klement.
|
|
||||||
|
|
||||||
The original license was MIT just because that's what ScraperWiki put on
|
|
||||||
everything. Should we change the license?
|
|
||||||
|
|
||||||
The MIT-style license grants pretty much all rights. It says that you need
|
|
||||||
to attribute when you redistribute source code, but you don't
|
|
||||||
necessarily have to redistribute source code.
|
|
||||||
|
|
||||||
A copyleft license adds the restriction that modified versions of the
|
|
||||||
code need to be licensed under the same license. GNU licenses in
|
|
||||||
particular require that source code be released if non-source versions are
|
|
||||||
released, and the different GNU licenses differ in what how the
|
|
||||||
non-source version is defined. (The original, GPL, discusses compiled
|
|
||||||
binaries.) Copyleft doesn't mean anything specific for commercial use.
|
|
||||||
|
|
||||||
MIT-licensed code can be modified and then licensed as GPL, because MIT
|
|
||||||
license allows that, but GPL code can't be modified as MIT, because MIT
|
|
||||||
doesn't allow that. And if we get all of the authors to agree on it, we
|
|
||||||
can always add whatever crazy license we want, regardless of what we
|
|
||||||
have already.
|
|
||||||
|
|
||||||
The distinction between MIT-style and GNU-something might matter quite little
|
|
||||||
in the case of Urchin.
|
|
||||||
|
|
||||||
1. Urchin is written in an interpreted language (shell), so it might be
|
|
||||||
hard to distribute usefully without providing the source code.
|
|
||||||
2. Urchin just runs tests; it doesn't get compiled with the rest of the
|
|
||||||
code (also because it's in shell). Thus, I think a GPL license on
|
|
||||||
Urchin wouldn't infect the code being tested.
|
|
||||||
|
|
||||||
This is as far as I have gotten with contemplating license changes. For now
|
|
||||||
we're sticking with the original MIT-style license, but it's easy to change
|
|
||||||
licenses later.
|
|
||||||
|
|
||||||
Nagios plugins
|
|
||||||
-----------------
|
|
||||||
It would be cool to run Nagios plugins with Urchin. This is already possible,
|
|
||||||
actually, but it might be worth giving some special thought to it.
|
|
||||||
https://nagios-plugins.org/doc/guidelines.html
|
|
@@ -1,2 +0,0 @@
|
|||||||
Totally different syntax and similar features, plus TAP output
|
|
||||||
https://github.com/sstephenson/bats
|
|
@@ -2,11 +2,11 @@
|
|||||||
# Run urchin in a bunch of different shells,
|
# Run urchin in a bunch of different shells,
|
||||||
# including a shell that isn't quite POSIX-compatible (zsh)
|
# including a shell that isn't quite POSIX-compatible (zsh)
|
||||||
|
|
||||||
for shell in dash bash mksh ksh zsh; do
|
for shell in dash bash ksh zsh; do
|
||||||
if which $shell > /dev/null 2> /dev/null; then
|
if which $shell > /dev/null 2> /dev/null; then
|
||||||
echo
|
echo
|
||||||
echo Running urchin tests in $shell
|
echo Running urchin tests in $shell
|
||||||
$shell urchin -s $shell tests | tail -n 4
|
$shell urchin tests | tail -n 3
|
||||||
else
|
else
|
||||||
echo
|
echo
|
||||||
echo Skipping $shell because it is not in the PATH
|
echo Skipping $shell because it is not in the PATH
|
||||||
|
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "urchin",
|
"name": "urchin",
|
||||||
"version": "0.0.5",
|
"version": "0.0.4",
|
||||||
"description": "Test framework for shell",
|
"description": "Test framework for shell",
|
||||||
"main": "urchin",
|
"main": "urchin",
|
||||||
"directories": {
|
"directories": {
|
||||||
@@ -24,9 +24,10 @@
|
|||||||
{"name": "Thomas Levine", "email": "_@thomaslevine.com"},
|
{"name": "Thomas Levine", "email": "_@thomaslevine.com"},
|
||||||
{"name": "David Jones", "email": "drj@pobox.com"},
|
{"name": "David Jones", "email": "drj@pobox.com"},
|
||||||
{"name": "Francis Irving", "email": "francis@flourish.org"},
|
{"name": "Francis Irving", "email": "francis@flourish.org"},
|
||||||
{"name": "Zarino Zappia", "email": "mail@zarino.co.uk"},
|
{"name": "Zarino Zappia", "email": "mail@zarino.co.uk"}
|
||||||
{"name": "Tom Mortimer-Jones", "email": "tom@morty.co.uk"},
|
{"name": "Tom Mortimer-Jones", "email": "tom@morty.co.uk"}
|
||||||
{"name": "Michael Klement", "email": "mklement0@gmail.com"}
|
{"name": "Michael Klement", "email": "mklement0@gmail.com"}
|
||||||
],
|
],
|
||||||
"license": "BSD"
|
"license": "BSD",
|
||||||
|
"readmeFilename": "readme.md"
|
||||||
}
|
}
|
||||||
|
74
readme.md
74
readme.md
@@ -4,26 +4,22 @@
|
|||||||
/ /_/ / / / /__/ / / / / / / /
|
/ /_/ / / / /__/ / / / / / / /
|
||||||
\__,_/_/ \___/_/ /_/_/_/ /_/
|
\__,_/_/ \___/_/ /_/_/_/ /_/
|
||||||
|
|
||||||
Urchin is a file-based test harness, normally used for testing shell programs.
|
Urchin is a test framework for shell. It is implemented in
|
||||||
It is written in portable shell and should thus work on GNU/Linux, BSD
|
portable /bin/sh and should work on GNU/Linux, Mac OS X, and
|
||||||
(including Mac OS X), and other Unix-like platforms.
|
other Unix platforms.
|
||||||
|
|
||||||
Urchin is called "Urchin" because
|
|
||||||
[sea urchins](https://en.wikipedia.org/wiki/Sea_urchin)
|
|
||||||
have shells called "tests".
|
|
||||||
|
|
||||||
## Try it out
|
## Try it out
|
||||||
Urchin's tests are written in Urchin, so you can run them to see what Urchin
|
Urchin's tests are written in Urchin, so you can run them to see what Urchin
|
||||||
is like. Clone the repository
|
is like. Clone the repository
|
||||||
|
|
||||||
git clone git://github.com/tlevine/urchin.git
|
git clone git://github.com/scraperwiki/urchin.git
|
||||||
|
|
||||||
Run the tests
|
Run the tests
|
||||||
|
|
||||||
cd urchin
|
cd urchin
|
||||||
./urchin tests
|
./urchin tests
|
||||||
|
|
||||||
The above command will run the tests in your system's default
|
The above command will run the tests in your systems default
|
||||||
shell, /bin/sh (on recent Ubuntu this is dash, but it could be
|
shell, /bin/sh (on recent Ubuntu this is dash, but it could be
|
||||||
ksh or bash on other systems); to test urchin's cross-shell compatibility,
|
ksh or bash on other systems); to test urchin's cross-shell compatibility,
|
||||||
run this:
|
run this:
|
||||||
@@ -31,15 +27,14 @@ run this:
|
|||||||
cd urchin
|
cd urchin
|
||||||
./cross-shell-tests
|
./cross-shell-tests
|
||||||
|
|
||||||
## Install
|
## Globally
|
||||||
Urchin is contained in a single file, so you can install it by copying it to a
|
Download Urchin like so (as root) (or use npm, below):
|
||||||
directory in your `PATH`. For example, you can run the following as root.
|
|
||||||
|
|
||||||
cd /usr/local/bin
|
cd /usr/local/bin
|
||||||
wget https://raw.github.com/tlevine/urchin/master/urchin
|
wget https://raw.github.com/scraperwiki/urchin/master/urchin
|
||||||
chmod +x urchin
|
chmod +x urchin
|
||||||
|
|
||||||
Urchin can be installed with npm too.
|
Can be installed with npm too:
|
||||||
|
|
||||||
npm install -g urchin
|
npm install -g urchin
|
||||||
|
|
||||||
@@ -91,14 +86,6 @@ Files are only run if they are executable, and files beginning with `.` are
|
|||||||
ignored. Thus, fixtures and libraries can be included sloppily within the test
|
ignored. Thus, fixtures and libraries can be included sloppily within the test
|
||||||
directory tree. The test passes if the file exits 0; otherwise, it fails.
|
directory tree. The test passes if the file exits 0; otherwise, it fails.
|
||||||
|
|
||||||
Tests files and subdirectories are run in ASCIIbetical order within each
|
|
||||||
directory; that is,
|
|
||||||
urchin looks for files within a directory in the following manner.
|
|
||||||
|
|
||||||
for file in *; do
|
|
||||||
do_something_with_test_file $file
|
|
||||||
done
|
|
||||||
|
|
||||||
### Writing cross-shell compatibility tests for testing shell code
|
### Writing cross-shell compatibility tests for testing shell code
|
||||||
|
|
||||||
While you could write your test scripts to explicitly invoke the functionality
|
While you could write your test scripts to explicitly invoke the functionality
|
||||||
@@ -110,20 +97,17 @@ The specific approach depends on your test scenario:
|
|||||||
* (b) Your scripts _source_ scripts containing portable shell code.
|
* (b) Your scripts _source_ scripts containing portable shell code.
|
||||||
|
|
||||||
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
|
#### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
|
||||||
Urchin sets the `TEST_SHELL` environment variable so that you may change the
|
|
||||||
shell with which your tests call other shell programs. To run your test
|
|
||||||
scripts in multiple shells you must call `$TEST_SHELL` in your tests and then
|
|
||||||
run urchin with the appropriate option.
|
|
||||||
|
|
||||||
In your test scripts, invoke the shell scripts to test via the shell
|
In your test scripts, invoke the shell scripts to test via the shell
|
||||||
specified in environment variable `TEST_SHELL` rather than directly;
|
specified in environment variable `TEST_SHELL` rather than directly;
|
||||||
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
|
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
|
||||||
|
Note that if you alsow want your test scripts to work when run directly,
|
||||||
|
outside of Urchin, be sure to target scripts that happen to be in the
|
||||||
|
current directory with prefix `./`; e.g., `$TEST_SHELL ./baz`
|
||||||
|
(rather than `$TEST_SHELL baz`).
|
||||||
|
|
||||||
On invocation of Urchin, prepend a definition of environment variable
|
Then, on invocation of Urchin, prepend a definition of environment variable
|
||||||
`TEST_SHELL` specifying the shell to test with, e.g.,
|
`TEST_SHELL` specifying the shell to test with, e.g.: `TEST_SHELL=zsh urchin ./tests`.
|
||||||
|
|
||||||
TEST_SHELL=zsh urchin ./tests
|
|
||||||
|
|
||||||
To test with multiple shells in sequence, use something like:
|
To test with multiple shells in sequence, use something like:
|
||||||
|
|
||||||
for shell in sh bash ksh zsh; do
|
for shell in sh bash ksh zsh; do
|
||||||
@@ -131,20 +115,14 @@ To test with multiple shells in sequence, use something like:
|
|||||||
done
|
done
|
||||||
|
|
||||||
If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
|
If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
|
||||||
scripts can rely on `$TEST_SHELL` always containing a value when Urchin runs
|
scripts can rely on `$TEST_SHELL` always containing a value.
|
||||||
them.
|
|
||||||
|
|
||||||
That said, we still recommand that you account for the possibility that
|
|
||||||
`$TEST_SHELL` does not contain a value so that you may run your test scripts
|
|
||||||
without Urchin. Supporting this case is very simple; when you invoke scripts
|
|
||||||
that happen to be in the current directory, be sure to use the prefix `./`,
|
|
||||||
e.g., `$TEST_SHELL ./baz` rather than `$TEST_SHELL baz`.
|
|
||||||
|
|
||||||
#### (b) Cross-shell tests with test scripts that _source_ shell scripts
|
#### (b) Cross-shell tests with test scripts that _source_ shell scripts
|
||||||
|
|
||||||
If you _source_ shell code in your test scripts, it is the test scripts
|
If you _source_ shell code in your test scripts, it is the test scripts
|
||||||
themselves that must be run with the shell specified.
|
themselves that must be run with the shell specified.
|
||||||
|
|
||||||
Urchin supports the `-s <shell>` option, which instructs
|
To that end, Urchin supports the `-s <shell>` option, which instructs
|
||||||
Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`.
|
Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`.
|
||||||
(In addition, Urchin sets environment variable `TEST_SHELL` to the specified
|
(In addition, Urchin sets environment variable `TEST_SHELL` to the specified
|
||||||
shell.)
|
shell.)
|
||||||
@@ -160,11 +138,19 @@ To test with multiple shells in sequence, use something like:
|
|||||||
urchin -s $shell ./tests
|
urchin -s $shell ./tests
|
||||||
done
|
done
|
||||||
|
|
||||||
Also consider using [shall](https://github.com/mklement0/shall).
|
#### (c) Cross shell tests with `urchin -x` (experimental)
|
||||||
It does something similar, but the interface may be more intuitive.
|
If you run urchin with the `-x` flag, it will be as if you ran
|
||||||
|
`$TEST_SHELL`. Unless `$TEST_SHELL` isn't set, in which case it'll
|
||||||
|
be as if you ran `/bin/sh`. Putting this in she shebang line might
|
||||||
|
eventually work out to be a cleaner way of doing cross-shell testing.
|
||||||
|
|
||||||
#!/usr/bin/env shall
|
#!/usr/bin/env urchin -x
|
||||||
echo This is a test file.
|
test a = a
|
||||||
|
|
||||||
|
It might make sense if you do this.
|
||||||
|
|
||||||
|
export TEST_SHELL=zsh && urchin -x
|
||||||
|
export TEST_SHELL=bash && urchin -x
|
||||||
|
|
||||||
## Alternatives to Urchin
|
## Alternatives to Urchin
|
||||||
Alternatives to Urchin are discussed in
|
Alternatives to Urchin are discussed in
|
||||||
|
@@ -1 +0,0 @@
|
|||||||
false
|
|
@@ -1 +0,0 @@
|
|||||||
false
|
|
@@ -1 +0,0 @@
|
|||||||
false
|
|
@@ -1,6 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
|
|
||||||
cd ..
|
|
||||||
export CDPATH=$PWD
|
|
||||||
./urchin -f 'tests/urchin exit code' >/dev/null
|
|
||||||
|
|
@@ -1,3 +1,5 @@
|
|||||||
#!/usr/bin/env true
|
#!/usr/bin/awk -f
|
||||||
true will processed the contents of this script, but that
|
|
||||||
means that nothing will happen and the script will exit 0
|
# This script will only succeed if it is indeed processed by awk.
|
||||||
|
|
||||||
|
BEGIN { print "ok" }
|
||||||
|
@@ -0,0 +1,4 @@
|
|||||||
|
#! ../../../urchin -x
|
||||||
|
running=$(ps -o comm= -p $$ && :)
|
||||||
|
echo "Running shell: $running"
|
||||||
|
[ "$running" = bash ]
|
@@ -3,5 +3,6 @@
|
|||||||
# Tests the `-s <shell> option, which invokes shebang-less and sh-shebang-line test scripts with the specified shell (for testing *sourced* shell code).
|
# Tests the `-s <shell> option, which invokes shebang-less and sh-shebang-line test scripts with the specified shell (for testing *sourced* shell code).
|
||||||
|
|
||||||
which bash >/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 2; }
|
which bash >/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 2; }
|
||||||
|
which /usr/bin/awk >/dev/null || { echo "Cannot test -s option: /usr/bin/awk not found." >&2; exit 2; }
|
||||||
|
|
||||||
../../urchin -s bash ./.test-run-by-specified-shell
|
../../urchin -s bash ./.test-run-by-specified-shell
|
||||||
|
3
tests/Cross-shell test support/urchin -x should pass the $TEST_SHELL
Executable file
3
tests/Cross-shell test support/urchin -x should pass the $TEST_SHELL
Executable file
@@ -0,0 +1,3 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
export TEST_SHELL=/bin/zsh
|
||||||
|
echo '[ "$TEST_SHELL" = /bin/zsh ] ; exit $?' | ../../urchin -x
|
3
tests/Cross-shell test support/urchin -x should start a shell.
Executable file
3
tests/Cross-shell test support/urchin -x should start a shell.
Executable file
@@ -0,0 +1,3 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
test c = $(../../urchin -x .print-arg-3 a 'b b b b' c d e)
|
3
tests/Cross-shell test support/urchin -x should start the $TEST_SHELL
Executable file
3
tests/Cross-shell test support/urchin -x should start the $TEST_SHELL
Executable file
@@ -0,0 +1,3 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
export TEST_SHELL=/bin/zsh
|
||||||
|
echo 'test -n "$ZSH_VERSION"; exit $?' | ../../urchin -x
|
@@ -0,0 +1,3 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
export TEST_SHELL=/bin/bash
|
||||||
|
../../urchin .test-urchin-x
|
@@ -1,11 +0,0 @@
|
|||||||
tmp=$(mktemp)
|
|
||||||
../urchin -e -f ./.die-on-fail > $tmp
|
|
||||||
result=$?
|
|
||||||
|
|
||||||
grep '1 should run.' $tmp
|
|
||||||
grep '2 should run.' $tmp
|
|
||||||
grep -v '3 should not run.' $tmp
|
|
||||||
grep -v '4 should not run.' $tmp
|
|
||||||
|
|
||||||
rm $tmp
|
|
||||||
exit $result
|
|
@@ -1,2 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
../urchin -v | grep '[0-9.]\{3,\}'
|
|
@@ -1,10 +0,0 @@
|
|||||||
# Begin - .testsuite/
|
|
||||||
not ok 1 - a
|
|
||||||
# ------------ Begin output ------------
|
|
||||||
# This is stdout from a.
|
|
||||||
# ------------ End output ------------
|
|
||||||
ok 2 - b
|
|
||||||
ok 3 - # SKIP c
|
|
||||||
# End - .testsuite/
|
|
||||||
# Took 0 seconds.
|
|
||||||
1..3
|
|
@@ -1,4 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
echo This is stderr from a. > /dev/stderr
|
|
||||||
echo This is stdout from a. > /dev/stdout
|
|
||||||
false
|
|
@@ -1,4 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
echo This is stderr from b. > /dev/stderr
|
|
||||||
echo This is stdout from b. > /dev/stdout
|
|
||||||
true
|
|
@@ -1 +0,0 @@
|
|||||||
This should not be run.
|
|
@@ -1,4 +0,0 @@
|
|||||||
tmp=$(mktemp)
|
|
||||||
|
|
||||||
../../urchin -t .testsuite/ | sed 1d > $tmp
|
|
||||||
diff $tmp .expected-output
|
|
235
urchin
235
urchin
@@ -1,21 +1,5 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
|
|
||||||
# This file is part of urchin. It is subject to the license terms in the
|
|
||||||
# COPYING file found in the top-level directory of this distribution or at
|
|
||||||
# https://raw.githubusercontent.com/tlevine/urchin/master/COPYING
|
|
||||||
|
|
||||||
# No part of urchin, including this file, may be copied, modified, propagated,
|
|
||||||
# or distributed except according to the terms contained in the COPYING file.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably -
|
|
||||||
# notably, it can produce output, which breaks fullpath().
|
|
||||||
unset CDPATH
|
|
||||||
|
|
||||||
# Urchin version number
|
|
||||||
VERSION=0.0.6
|
|
||||||
|
|
||||||
fullpath() {
|
fullpath() {
|
||||||
(
|
(
|
||||||
cd -- "$1"
|
cd -- "$1"
|
||||||
@@ -25,9 +9,7 @@ fullpath() {
|
|||||||
|
|
||||||
indent() {
|
indent() {
|
||||||
level="$1"
|
level="$1"
|
||||||
if test "$level" -gt 0; then
|
printf "%$((2 * ${level}))s"
|
||||||
printf "%$((2 * ${level}))s"
|
|
||||||
fi
|
|
||||||
}
|
}
|
||||||
|
|
||||||
recurse() {
|
recurse() {
|
||||||
@@ -44,16 +26,9 @@ recurse() {
|
|||||||
|
|
||||||
if [ -d "$potential_test" ]
|
if [ -d "$potential_test" ]
|
||||||
then
|
then
|
||||||
|
|
||||||
if $tap_format; then
|
|
||||||
indent $indent_level | sed 's/ /#/g'
|
|
||||||
echo "# Begin - ${potential_test}"
|
|
||||||
else
|
|
||||||
indent $indent_level
|
|
||||||
echo "+ ${potential_test}"
|
|
||||||
fi
|
|
||||||
|
|
||||||
(
|
(
|
||||||
|
indent $indent_level
|
||||||
|
echo " ${potential_test}"
|
||||||
cd -- "$potential_test"
|
cd -- "$potential_test"
|
||||||
[ -f setup_dir ] && [ -x setup_dir ] && ./setup_dir >> "$stdout_file"
|
[ -f setup_dir ] && [ -x setup_dir ] && ./setup_dir >> "$stdout_file"
|
||||||
|
|
||||||
@@ -68,132 +43,71 @@ recurse() {
|
|||||||
|
|
||||||
# $2 instead of $indent_level so it doesn't clash
|
# $2 instead of $indent_level so it doesn't clash
|
||||||
recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
|
recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
|
||||||
exit_code=$?
|
|
||||||
|
|
||||||
if $exit_on_fail && test $exit_code -ne 0; then
|
|
||||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
|
||||||
[ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||||
done
|
done
|
||||||
[ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
|
[ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
|
||||||
)
|
|
||||||
if $tap_format; then
|
|
||||||
indent $indent_level | sed 's/ /#/g'
|
|
||||||
echo "# End - ${potential_test}"
|
|
||||||
else
|
|
||||||
echo
|
echo
|
||||||
|
)
|
||||||
|
elif [ -x "$potential_test" ]
|
||||||
|
then
|
||||||
|
|
||||||
|
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
|
||||||
|
|
||||||
|
# Run the test
|
||||||
|
if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
|
||||||
|
then
|
||||||
|
TEST_SHELL="$TEST_SHELL" "$shell_for_sh_tests" ./"$potential_test" > "$stdout_file" 2>&1
|
||||||
|
else
|
||||||
|
TEST_SHELL="$TEST_SHELL" ./"$potential_test" > "$stdout_file" 2>&1
|
||||||
fi
|
fi
|
||||||
else
|
exit_code="$?"
|
||||||
if [ -x "$potential_test" ]
|
|
||||||
|
|
||||||
|
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
||||||
|
|
||||||
|
indent $indent_level
|
||||||
|
if [ $exit_code -eq 0 ]
|
||||||
then
|
then
|
||||||
[ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
|
# On success, print a green '✓'
|
||||||
|
printf '\033[32m✓ \033[0m'
|
||||||
# Run the test
|
printf '%s\n' "${potential_test}"
|
||||||
if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
|
printf '%s\n' "${potential_test} passed" >> "$logfile"
|
||||||
then
|
|
||||||
TEST_SHELL="$TEST_SHELL" "$shell_for_sh_tests" ./"$potential_test" > "$stdout_file" 2>&1
|
|
||||||
else
|
|
||||||
TEST_SHELL="$TEST_SHELL" ./"$potential_test" > "$stdout_file" 2>&1
|
|
||||||
fi
|
|
||||||
exit_code="$?"
|
|
||||||
|
|
||||||
[ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
|
|
||||||
if [ $exit_code -eq 0 ]; then
|
|
||||||
result=success
|
|
||||||
else
|
|
||||||
result=fail
|
|
||||||
fi
|
|
||||||
else
|
else
|
||||||
result=skip
|
# On fail, print a red '✗'
|
||||||
fi
|
printf '\033[31m✗ \033[0m'
|
||||||
|
printf '%s\n' "${potential_test}"
|
||||||
echo "${result}" >> "$logfile"
|
printf '%s\n' "${potential_test} failed" >> "$logfile"
|
||||||
if $tap_format; then
|
printf '\033[31m' # Print output captured from failed test in red.
|
||||||
n=$(grep -ce '^\(success\|fail\|skip\)' "$logfile")
|
cat "$stdout_file"
|
||||||
|
printf '\033[0m'
|
||||||
if [ "$result" == fail ]; then
|
|
||||||
not='not '
|
|
||||||
else
|
|
||||||
not=''
|
|
||||||
fi
|
|
||||||
if [ "$result" == skip ]; then
|
|
||||||
skip='# SKIP '
|
|
||||||
else
|
|
||||||
skip=''
|
|
||||||
fi
|
|
||||||
echo "${not}ok $n - ${skip}${potential_test}"
|
|
||||||
if [ "$result" == fail ]; then
|
|
||||||
echo '# ------------ Begin output ------------'
|
|
||||||
sed 's/^/# /' "$stdout_file"
|
|
||||||
echo '# ------------ End output ------------'
|
|
||||||
fi
|
|
||||||
else
|
|
||||||
indent $indent_level
|
|
||||||
case "$result" in
|
|
||||||
success)
|
|
||||||
# On success, print a green '✓'
|
|
||||||
printf '\033[32m✓ \033[0m'
|
|
||||||
printf '%s\n' "${potential_test}"
|
|
||||||
;;
|
|
||||||
fail)
|
|
||||||
# On fail, print a red '✗'
|
|
||||||
printf '\033[31m✗ \033[0m'
|
|
||||||
printf '%s\n' "${potential_test}"
|
|
||||||
|
|
||||||
# Print output captured from failed test in red.
|
|
||||||
printf '\033[31m'
|
|
||||||
|
|
||||||
cat "$stdout_file"
|
|
||||||
printf '\033[0m'
|
|
||||||
;;
|
|
||||||
skip)
|
|
||||||
printf ' %s\n' "${potential_test}"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
fi
|
|
||||||
|
|
||||||
if $exit_on_fail && test 0 -ne $exit_code; then
|
|
||||||
return 1
|
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
[ $indent_level -eq 0 ] && rm "$stdout_file"
|
[ $indent_level -eq 0 ] && rm "$stdout_file"
|
||||||
}
|
}
|
||||||
|
|
||||||
has_sh_or_no_shebang_line() {
|
has_sh_or_no_shebang_line() {
|
||||||
# no shebang line at all
|
head -n 1 "$1" | grep -vqE '^#!' && return 0 # no shebang line at all
|
||||||
head -n 1 "$1" | grep -vqE '^#!' && return 0
|
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' && return 0 # shebang line is '#!/bin/sh' or legal variations thereof
|
||||||
|
|
||||||
# shebang line is '#!/bin/sh' or legal variations thereof
|
|
||||||
head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' &&
|
|
||||||
return 0
|
|
||||||
|
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
USAGE="usage: $0 [<options>] <test directory>"
|
USAGE="usage: $0 [<options>] <test directory>"
|
||||||
|
|
||||||
urchin_help() {
|
urchin_help() {
|
||||||
cat <<EOF
|
echo
|
||||||
|
echo "$USAGE"
|
||||||
$USAGE
|
echo
|
||||||
|
echo '-s <shell> Invoke test scripts that either have no shebang line or'
|
||||||
-s <shell> Invoke test scripts that either have no shebang line at all or
|
echo ' shebang line "#!/bin/sh" with the specified shell.'
|
||||||
have shebang line "#!/bin/sh" with the specified shell.
|
echo '-f Force running even if the test directory'\''s name does not'
|
||||||
-e Stop running if any single test fails. This is helpful if you want
|
echo ' contain the word "test".'
|
||||||
to use Urchin to run things other than tests, such as a set of
|
echo '-x Run "$TEST_SHELL", falling back on /bin/sh. This might be'
|
||||||
configuration scripts.
|
echo ' useful in the shebang line (experimental).'
|
||||||
-f Force running even if the test directory's name does not
|
echo '-h This help.'
|
||||||
contain the word "test".
|
echo
|
||||||
-t Format output in Test Anything Protocol (TAP)
|
echo 'Go to https://github.com/tlevine/urchin for documentation on writing tests.'
|
||||||
-h, --help This help.
|
echo
|
||||||
-v Display the version number.
|
|
||||||
|
|
||||||
Go to https://github.com/tlevine/urchin for documentation on writing tests.
|
|
||||||
|
|
||||||
EOF
|
|
||||||
}
|
}
|
||||||
|
|
||||||
plural () {
|
plural () {
|
||||||
@@ -209,11 +123,7 @@ plural () {
|
|||||||
}
|
}
|
||||||
|
|
||||||
urchin_go() {
|
urchin_go() {
|
||||||
rm -f "$logfile"
|
echo Running tests at $(date +%Y-%m-%dT%H:%M:%S) | tee "$logfile"
|
||||||
if "$tap_format"; then
|
|
||||||
printf \#\
|
|
||||||
fi
|
|
||||||
echo Running tests at $(date +%Y-%m-%dT%H:%M:%S)
|
|
||||||
start=$(date +%s)
|
start=$(date +%s)
|
||||||
|
|
||||||
# Determine the environment variable to define for test scripts
|
# Determine the environment variable to define for test scripts
|
||||||
@@ -233,26 +143,13 @@ urchin_go() {
|
|||||||
|
|
||||||
finish=$(date +%s)
|
finish=$(date +%s)
|
||||||
elapsed=$(($finish - $start))
|
elapsed=$(($finish - $start))
|
||||||
|
echo "Done, took $elapsed $(plural second $elapsed)."
|
||||||
passed=$(grep -c '^success' "$logfile")
|
set -- $(grep -e 'passed$' "$logfile"|wc -l) $(grep -e 'failed$' "$logfile"|wc -l)
|
||||||
failed=$(grep -c '^fail' "$logfile")
|
printf '%s\n' "$1 $(plural test "$1") passed."
|
||||||
skipped=$(grep -c '^skip' "$logfile")
|
[ $2 -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
|
||||||
if $tap_format; then
|
printf '%s\n' "$2 $(plural test "$2") failed."
|
||||||
echo "# Took $elapsed $(plural second $elapsed)."
|
printf '\033[m'
|
||||||
echo 1..$(($passed + $failed + $skipped))
|
return "$2"
|
||||||
else
|
|
||||||
echo "Done, took $elapsed $(plural second $elapsed)."
|
|
||||||
printf '%s\n' "$passed $(plural test "$passed") passed."
|
|
||||||
printf '%s\n' "$skipped $(plural test "$skipped") skipped."
|
|
||||||
|
|
||||||
# If tests failed, print the message in red, otherwise in green.
|
|
||||||
[ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m'
|
|
||||||
|
|
||||||
printf '%s\n' "$failed $(plural test "$failed") failed."
|
|
||||||
printf '\033[m'
|
|
||||||
fi
|
|
||||||
rm -f "$logfile"
|
|
||||||
test -z "$failed" || test "$failed" -eq '0'
|
|
||||||
}
|
}
|
||||||
|
|
||||||
urchin_molly_guard() {
|
urchin_molly_guard() {
|
||||||
@@ -269,26 +166,24 @@ urchin_molly_guard() {
|
|||||||
|
|
||||||
shell_for_sh_tests=
|
shell_for_sh_tests=
|
||||||
force=false
|
force=false
|
||||||
exit_on_fail=false
|
|
||||||
tap_format=false
|
|
||||||
while [ $# -gt 0 ]
|
while [ $# -gt 0 ]
|
||||||
do
|
do
|
||||||
case "$1" in
|
case "$1" in
|
||||||
-e) exit_on_fail=true;;
|
|
||||||
-f) force=true;;
|
-f) force=true;;
|
||||||
-s)
|
-s)
|
||||||
shift
|
shift
|
||||||
shell_for_sh_tests=$1
|
shell_for_sh_tests=$1
|
||||||
which "$shell_for_sh_tests" > /dev/null || {
|
which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
|
||||||
echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2
|
;;
|
||||||
urchin_help >&2
|
-x) # `urchin -sh` is equivalent to "$TEST_SHELL"
|
||||||
exit 2
|
shift
|
||||||
} ;;
|
#current_shell=$(ps -o comm= -p $$ && :)
|
||||||
-t) tap_format=true;;
|
#urchinsh=${TEST_SHELL:-$current_shell}
|
||||||
|
export TEST_SHELL=${TEST_SHELL:-/bin/sh}
|
||||||
|
"$TEST_SHELL" "$@"
|
||||||
|
exit $?;;
|
||||||
-h|--help) urchin_help
|
-h|--help) urchin_help
|
||||||
exit 0;;
|
exit 0;;
|
||||||
-v) echo "$VERSION"
|
|
||||||
exit;;
|
|
||||||
-*) urchin_help >&2
|
-*) urchin_help >&2
|
||||||
exit 1;;
|
exit 1;;
|
||||||
*) break;;
|
*) break;;
|
||||||
|
Binary file not shown.
Before Width: | Height: | Size: 99 KiB |
Binary file not shown.
Before Width: | Height: | Size: 115 KiB |
Reference in New Issue
Block a user