explain the new cycling

This commit is contained in:
Thomas Levine 2016-02-29 05:07:15 +00:00
parent ff212bab20
commit 32a2247292
3 changed files with 50 additions and 39 deletions

30
HISTORY
View File

@ -156,6 +156,36 @@ to matter quite little in the case of Urchin.
the code (also because it's in shell). Thus, I think a GPL license on Urchin
wouldn't infect the code being tested.
### Specification of the shell to run tests in
Urchin previously had separate methods for setting the TEST_SHELL environment
variable and for setting the shell that would run the tests; the former was
set as an environment variable, and the latter was set with the -s flag..
Urchin now uses the -s flag for both of these settings, and it mostly ignores
the exported TEST_SHELL variable.
Urchin also inspects the shebang line differently. Previously, Urchin would
vary the shells with which a test is run if the shebang line either was absent
or was #!/bin/sh. Now it varies the shell only if the shebang line is absent.
If you pass -n/--disable-cycling, Urchin will invoke tests ordinarily and will
only set the TEST_SHELL variable if it does not exist. If the TEST_SHELL
variable is absent, it will be set to /bin/sh.
Here is how you should write your tests for cross-shell testing, depending on
their structure.
* If you want a test file to run in the same shell every time and to have
access to the TEST_SHELL variable, usually for invoking the program that
you are testing, then set the file's shebang line.
* If you want a test file to be run in a different shell every time, do not
set the shebang line. TEST_SHELL variable will be set to correspond with the
shell that is presently invoking the test file, though you probably won't
need this variable.
* If you want a test file to have access to a TEST_SHELL variable that you
set yourself, pass -n/--disable-cycling to urchin. Urchin will ignore the
shebang lines in this case.
### Source setup and teardown
setup, teardown, setup_dir, and teardown_dir are now sourced instead of
executed; they are referenced a bit like this.

16
TODO
View File

@ -1,16 +1,6 @@
Things I want
=============
Environment variables
-------------
Do something to make it easier to debug environment variables, because that is
often confusing.
https://github.com/creationix/nvm/issues/719
https://github.com/creationix/nvm/issues/589
Documenting that people should run "env" when their tests fail might be good
enough.
Packaging
------------
Package for package managers.
@ -33,3 +23,9 @@ Try running Urchin in Windows somehow. Interpreters include
* Git for Windows (https://git-scm.com/download/win)
* win-bash (http://win-bash.sourceforge.net/)
shall
----------
Add shall to my NYC*BUG talk.
#!/usr/bin/env shall
echo This runs in several shells.

View File

@ -23,14 +23,6 @@ Run the tests
cd urchin
./urchin tests
The above command will run the tests in your system's default
shell, /bin/sh (on recent Ubuntu this is dash, but it could be
ksh or bash on other systems); to test urchin's cross-shell compatibility,
run this:
cd urchin
./cross-shell-tests
## Install
Urchin is contained in a single file, so you can install it by copying it to a
directory in your `PATH`. For example, you can run the following as root.
@ -81,11 +73,13 @@ and directories have special meanings.
teardown
Directories are processed in a depth-first order. When a particular directory
is processed, `setup_dir` is run before everything else in the directory, including
subdirectories. `teardown_dir` is run after everything else in the directory.
is processed, `setup_dir` is sourced before everything else in the directory,
including subdirectories. `teardown_dir` is sourced after everything else in
the directory.
A directory's `setup` file, if it exists, is run right before each test file
within the particular directory, and the `teardown` file is run right after.
A directory's `setup` file, if it exists, is sourced right before each test
file within the particular directory is run, and the `teardown` file is
sourced right after.
Files are only run if they are executable, and files beginning with `.` are
ignored. Thus, fixtures and libraries can be included sloppily within the test
@ -119,16 +113,17 @@ In your test scripts, invoke the shell scripts to test via the shell
specified in environment variable `TEST_SHELL` rather than directly;
e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).
On invocation of Urchin, prepend a definition of environment variable
`TEST_SHELL` specifying the shell to test with, e.g.,
Urchin runs tests in multiple different shells by default; Urchin has a
list of default shells, and the following command will run your tests in
all of those shells that Urchin detects.
TEST_SHELL=zsh urchin ./tests
./urchin ./tests
To test with multiple shells in sequence, use something like:
You can override the default list of shells with the `-s` flag.
for shell in sh bash ksh zsh; do
TEST_SHELL=$shell urchin ./tests
done
urchin -s sh -s ksh ./tests
You can also
If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
scripts can rely on `$TEST_SHELL` always containing a value when Urchin runs
@ -159,13 +154,3 @@ To test with multiple shells in sequence, use something like:
for shell in sh bash ksh zsh; do
urchin -s $shell ./tests
done
Also consider using [shall](https://github.com/mklement0/shall).
It does something similar, but the interface may be more intuitive.
#!/usr/bin/env shall
echo This is a test file.
## Alternatives to Urchin
Alternatives to Urchin are discussed in
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).