call urchin -s in cross-shell tests

unnecessary now that shall exists

AUTHORS

@@ -1,9 +0,0 @@
-Authors
--------
-David Jones
-Michael Klement
-Thomas Levine
-
-Maintainer
--------
-Thomas Levine <_@thomaslevine.com>

HISTORY

@@ -1,23 +1,5 @@
 HISTORY
-=======
+-------
-
-Version 0.0.6
----------------------
-
-* Produce TAP output with the -t flag.
-* Add a + sign in front of directories in the normal output so that they
-  line up with non-directories.
-* Display skipped tests in the normal output and in the TAP output.
-* Correct some things in the documentation.
-* Rearrange things in the documentation to be more clear.
-* Pass the -e flag to exit urchin if any single test fails.
-* Remove the undocumented, experimental -x flag now that shall exists.
-* Display version number with the -v flag.
-* Document why Urchin is called "Urchin"
-
-These changes are made somewhat separately in the branches "exit-on-fail",
-"remove-urchin-x", "tap", and "update-readme". They are rebased into one
-branch, "tlevine-2016-02", for merging into "master".
 
 Version 0.0.5
 ---------------------

cross-shell-tests

@@ -2,11 +2,11 @@
 # Run urchin in a bunch of different shells,
 # including a shell that isn't quite POSIX-compatible (zsh)
 
-for shell in dash bash mksh ksh zsh; do
+for shell in dash bash ksh zsh; do
   if which $shell > /dev/null 2> /dev/null; then
     echo
     echo Running urchin tests in $shell
-    $shell urchin -s $shell tests | tail -n 4
+    $shell urchin -s $shell tests | tail -n 3
   else
     echo
     echo Skipping $shell because it is not in the PATH

readme.md

@@ -4,13 +4,9 @@
     / /_/ / /  / /__/ / / / / / / /
     \__,_/_/   \___/_/ /_/_/_/ /_/ 
 
-Urchin is a file-based test harness, normally used for testing shell programs.
+Urchin is a test framework for shell. It is implemented in
-It is written in portable shell and should thus work on GNU/Linux, BSD
+portable /bin/sh and should work on GNU/Linux, Mac OS X, and
-(including Mac OS X), and other Unix-like platforms.
+other Unix platforms.
-
-Urchin is called "Urchin" because
-[sea urchins](https://en.wikipedia.org/wiki/Sea_urchin)
-have shells called "tests".
 
 ## Try it out
 Urchin's tests are written in Urchin, so you can run them to see what Urchin
@@ -31,15 +27,14 @@ run this:
     cd urchin
     ./cross-shell-tests
 
-## Install
+## Globally
-Urchin is contained in a single file, so you can install it by copying it to a
+Download Urchin like so (as root) (or use npm, below):
-directory in your `PATH`. For example, you can run the following as root.
 
     cd /usr/local/bin
     wget https://raw.github.com/tlevine/urchin/master/urchin
     chmod +x urchin
 
-Urchin can be installed with npm too.
+Can be installed with npm too:
 
     npm install -g urchin
 
@@ -91,14 +86,15 @@ Files are only run if they are executable, and files beginning with `.` are
 ignored. Thus, fixtures and libraries can be included sloppily within the test
 directory tree. The test passes if the file exits 0; otherwise, it fails.
 
-Tests files and subdirectories are run in ASCIIbetical order within each
+In case you care about the order in which your tests execute, consider that
-directory; that is,
 urchin looks for files within a directory in the following manner.
 
     for file in *; do
       do_something_with_test_file $file
     done
 
+Tests within a directory are executed in whatever order `*` returns.
+
 ### Writing cross-shell compatibility tests for testing shell code
 
 While you could write your test scripts to explicitly invoke the functionality
@@ -110,20 +106,24 @@ The specific approach depends on your test scenario:
 * (b) Your scripts _source_ scripts containing portable shell code.
 
 #### (a) Cross-shell tests with test scripts that _invoke_ shell scripts
-Urchin sets the `TEST_SHELL` environment variable so that you may change the
-shell with which your tests call other shell programs. To run your test
-scripts in multiple shells you must call `$TEST_SHELL` in your tests and then
-run urchin with the appropriate option.
 
+First, consider using [shall](https://github.com/mklement0/shall).
+
+    #!/usr/bin/env shall
+    echo This is a test file.
+
+Alternatively, you can use urchin's built-in recognition of the
+`TEST_SHELL` environment variable.
 In your test scripts, invoke the shell scripts to test via the shell
 specified in environment variable `TEST_SHELL` rather than directly;
 e.g.: `$TEST_SHELL ../foo bar` (rather than just `../foo bar`).  
+Note that if you alsow want your test scripts to work when run directly,
+outside of Urchin, be sure to target scripts that happen to be in the 
+current directory with prefix `./`; e.g., `$TEST_SHELL ./baz`
+(rather than `$TEST_SHELL baz`).
 
-On invocation of Urchin, prepend a definition of environment variable
+Then, on invocation of Urchin, prepend a definition of environment variable
-`TEST_SHELL` specifying the shell to test with, e.g.,
+`TEST_SHELL` specifying the shell to test with, e.g.: `TEST_SHELL=zsh urchin ./tests`.  
-
-    TEST_SHELL=zsh urchin ./tests
-
 To test with multiple shells in sequence, use something like:
 
     for shell in sh bash ksh zsh; do
@@ -131,20 +131,14 @@ To test with multiple shells in sequence, use something like:
     done
 
 If `TEST_SHELL` has no value, Urchin defines it as `/bin/sh`, so the test
-scripts can rely on `$TEST_SHELL` always containing a value when Urchin runs
+scripts can rely on `$TEST_SHELL` always containing a value.
-them.
-
-That said, we still recommand that you account for the possibility that
-`$TEST_SHELL` does not contain a value so that you may run your test scripts
-without Urchin. Supporting this case is very simple; when you invoke scripts
-that happen to be in the current directory, be sure to use the prefix `./`,
-e.g., `$TEST_SHELL ./baz` rather than `$TEST_SHELL baz`.
 
 #### (b) Cross-shell tests with test scripts that _source_ shell scripts
+
 If you _source_ shell code in your test scripts, it is the test scripts
 themselves that must be run with the shell specified.
 
-Urchin supports the `-s <shell>` option, which instructs
+To that end, Urchin supports the `-s <shell>` option, which instructs
 Urchin to invoke the test scripts with the specified shell; e.g., `-s bash`.  
 (In addition, Urchin sets environment variable `TEST_SHELL` to the specified
 shell.)
@@ -160,12 +154,21 @@ To test with multiple shells in sequence, use something like:
       urchin -s $shell ./tests
     done
 
-Also consider using [shall](https://github.com/mklement0/shall).
+<!--
-It does something similar, but the interface may be more intuitive.
+#### (c) Cross shell tests with `urchin -x` (experimental)
+If you run urchin with the `-x` flag, it will be as if you ran
+`$TEST_SHELL`. Unless `$TEST_SHELL` isn't set, in which case it'll
+be as if you ran `/bin/sh`. Putting this in she shebang line might
+eventually work out to be a cleaner way of doing cross-shell testing.
 
-    #!/usr/bin/env shall
+    #!/usr/bin/env urchin -x
-    echo This is a test file.
+    test a = a
 
+It might make sense if you do this.
+
+    export TEST_SHELL=zsh && urchin -x
+    export TEST_SHELL=bash && urchin -x
+-->
 ## Alternatives to Urchin
 Alternatives to Urchin are discussed in
 [this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).

tests/.die-on-fail/2 should run.

@@ -1 +0,0 @@
-false

tests/.die-on-fail/3 should not run.

@@ -1 +0,0 @@
-false

tests/.die-on-fail/4 should not run.

@@ -1 +0,0 @@
-false

tests/Cross-shell test support/.test-run-by-specified-shell/With -s, a non-sh test script should be invoked directly.

@@ -1,3 +1,5 @@
-#!/usr/bin/env true
+#!/usr/bin/awk -f
-true will processed the contents of this script, but that
+
-means that nothing will happen and the script will exit 0 
+# This script will only succeed if it is indeed processed by awk.
+
+BEGIN { print "ok" }

tests/Cross-shell test support/The -s option should invoke tests with the specified shell.

@@ -3,5 +3,6 @@
 # Tests the `-s <shell> option, which invokes shebang-less and sh-shebang-line test scripts with the specified shell (for testing *sourced* shell code).
 
 which bash >/dev/null || { echo "Cannot test -s option: bash cannot be located." >&2; exit 2; }
+which /usr/bin/awk >/dev/null || { echo "Cannot test -s option: /usr/bin/awk not found." >&2; exit 2; }
 
 ../../urchin -s bash ./.test-run-by-specified-shell

tests/Exit on fail if -e is passed.

@@ -1,11 +0,0 @@
-tmp=$(mktemp)
-../urchin -e -f ./.die-on-fail > $tmp
-result=$?
-
-grep '1 should run.' $tmp
-grep '2 should run.' $tmp
-grep -v '3 should not run.' $tmp
-grep -v '4 should not run.' $tmp
-
-rm $tmp
-exit $result

tests/Print version on -v.

@@ -1,2 +0,0 @@
-#!/bin/sh
-../urchin -v | grep '[0-9.]\{3,\}'

tests/TAP/.expected-output

@@ -1,10 +0,0 @@
-# Begin - .testsuite/
-not ok 1 - a
-# ------------ Begin output ------------
-# This is stdout from a.
-# ------------ End output ------------
-ok 2 - b
-ok 3 - # SKIP c
-# End - .testsuite/
-# Took 0 seconds.
-1..3

tests/TAP/.testsuite/a

@@ -1,4 +0,0 @@
-#!/bin/sh
-echo This is stderr from a. > /dev/stderr
-echo This is stdout from a. > /dev/stdout
-false

tests/TAP/.testsuite/b

@@ -1,4 +0,0 @@
-#!/bin/sh
-echo This is stderr from b. > /dev/stderr
-echo This is stdout from b. > /dev/stdout
-true

tests/TAP/.testsuite/c

@@ -1 +0,0 @@
-This should not be run.

tests/TAP/Running with -t should produce TAP output.

@@ -1,4 +0,0 @@
-tmp=$(mktemp)
-
-../../urchin -t .testsuite/ | sed 1d > $tmp
-diff $tmp .expected-output

urchin

@@ -4,9 +4,6 @@
 # which breaks fullpath().
 unset CDPATH
 
-# Urchin version number
-VERSION=0.0.6
-
 fullpath() {
   (
     cd -- "$1"
@@ -33,16 +30,9 @@ recurse() {
 
   if [ -d "$potential_test" ]
     then
-
-    if $tap_format; then
-      indent $indent_level | sed 's/  /#/g'
-      echo "# Begin - ${potential_test}"
-    else
-      indent $indent_level
-      echo "+ ${potential_test}"
-    fi
-
     (
+      indent $indent_level
+      echo "  ${potential_test}"
       cd -- "$potential_test"
       [ -f setup_dir ] && [ -x setup_dir ] && ./setup_dir >> "$stdout_file"
 
@@ -57,92 +47,44 @@ recurse() {
 
         # $2 instead of $indent_level so it doesn't clash
         recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
-        exit_code=$?
-
-        if $exit_on_fail && test $exit_code -ne 0; then
-          [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
-          [ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
-          return 1
-        fi
 
         [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
       done
       [ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
-    )
-    if $tap_format; then
-      indent $indent_level | sed 's/  /#/g'
-      echo "# End - ${potential_test}"
-    else
       echo
-    fi
+    )
-  else
+  elif [ -x "$potential_test" ]
-    if [ -x "$potential_test" ]
+    then
+
+    [ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
+
+    # Run the test
+    if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
       then    
-      [ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
+      TEST_SHELL="$TEST_SHELL" "$shell_for_sh_tests" ./"$potential_test" > "$stdout_file" 2>&1
-
-      # Run the test
-      if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
-        then    
-        TEST_SHELL="$TEST_SHELL" "$shell_for_sh_tests" ./"$potential_test" > "$stdout_file" 2>&1
-      else
-        TEST_SHELL="$TEST_SHELL" ./"$potential_test" > "$stdout_file" 2>&1
-      fi
-      exit_code="$?"
-
-      [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
-      if [ $exit_code -eq 0 ]; then
-        result=success
-      else
-        result=fail
-      fi
     else
-      result=skip
+      TEST_SHELL="$TEST_SHELL" ./"$potential_test" > "$stdout_file" 2>&1
     fi
+    exit_code="$?"
 
-    echo "${result}" >> "$logfile"
-    if $tap_format; then
-      n=$(grep -ce '^\(success\|fail\|skip\)' "$logfile")
 
-      if [ "$result" == fail ]; then
+    [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
-        not='not '
+
-      else
+    indent $indent_level
-        not=''
+    if [ $exit_code -eq 0 ]
-      fi
+      then
-      if [ "$result" == skip ]; then
+      # On success, print a green '✓'
-        skip='# SKIP '
+      printf '\033[32m✓ \033[0m'
-      else
+      printf '%s\n' "${potential_test}"
-        skip=''
+      printf '%s\n' "${potential_test} passed" >> "$logfile"
-      fi
-      echo "${not}ok $n - ${skip}${potential_test}"
-      if [ "$result" == fail ]; then
-        echo '# ------------ Begin output ------------'
-        sed 's/^/# /' "$stdout_file"
-        echo '# ------------ End output ------------'
-      fi
     else
-      indent $indent_level
+      # On fail, print a red '✗'
-      case "$result" in
+      printf '\033[31m✗ \033[0m'
-        success)
+      printf '%s\n' "${potential_test}"
-          # On success, print a green '✓'
+      printf '%s\n' "${potential_test} failed" >> "$logfile"
-          printf '\033[32m✓ \033[0m'
+      printf '\033[31m' # Print output captured from failed test in red.
-          printf '%s\n' "${potential_test}"
+      cat "$stdout_file"
-        ;;
+      printf '\033[0m'
-        fail)
-          # On fail, print a red '✗'
-          printf '\033[31m✗ \033[0m'
-          printf '%s\n' "${potential_test}"
-          printf '\033[31m' # Print output captured from failed test in red.
-          cat "$stdout_file"
-          printf '\033[0m'
-        ;;
-        skip)
-          printf '  %s\n' "${potential_test}"
-        ;;
-      esac
-    fi
-
-    if $exit_on_fail && test 0 -ne $exit_code; then
-      return 1
     fi
   fi
   [ $indent_level -eq 0 ] && rm "$stdout_file"
@@ -163,14 +105,9 @@ $USAGE
 
 -s <shell>  Invoke test scripts that either have no shebang line at all or
             have shebang line "#!/bin/sh" with the specified shell.
--e          Stop running if any single test fails. This is helpful if you want
-            to use Urchin to run things other than tests, such as a set of
-            configuration scripts.
 -f          Force running even if the test directory's name does not
             contain the word "test".
--t          Format output in Test Anything Protocol (TAP)
+-h          This help.
--h, --help  This help.
--v          Display the version number.
 
 Go to https://github.com/tlevine/urchin for documentation on writing tests.
 
@@ -190,11 +127,7 @@ plural () {
 }
 
 urchin_go() {
-  rm -f "$logfile"
+  echo Running tests at $(date +%Y-%m-%dT%H:%M:%S) | tee "$logfile"
-  if "$tap_format"; then
-    printf \#\ 
-  fi
-  echo Running tests at $(date +%Y-%m-%dT%H:%M:%S)
   start=$(date +%s)
 
   # Determine the environment variable to define for test scripts
@@ -214,23 +147,13 @@ urchin_go() {
 
   finish=$(date +%s)
   elapsed=$(($finish - $start))
-
+  echo "Done, took $elapsed $(plural second $elapsed)."
-  passed=$(grep -c '^success' "$logfile")
+  set -- $(grep -e 'passed$' "$logfile"|wc -l) $(grep -e 'failed$' "$logfile"|wc -l)
-  failed=$(grep -c '^fail' "$logfile")
+  printf '%s\n' "$1 $(plural test "$1") passed."
-  skipped=$(grep -c '^skip' "$logfile")
+  [ $2 -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
-  if $tap_format; then
+  printf '%s\n' "$2 $(plural test "$2") failed."
-    echo "# Took $elapsed $(plural second $elapsed)."
+  printf '\033[m'
-    echo 1..$(($passed + $failed + $skipped))
+  return "$2"
-  else
-    echo "Done, took $elapsed $(plural second $elapsed)."
-    printf '%s\n' "$passed $(plural test "$passed") passed."
-    printf '%s\n' "$skipped $(plural test "$skipped") skipped."
-    [ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
-    printf '%s\n' "$failed $(plural test "$failed") failed."
-    printf '\033[m'
-  fi
-  rm -f "$logfile"
-  test -z "$failed" || test "$failed" -eq '0'
 }
 
 urchin_molly_guard() {
@@ -247,23 +170,17 @@ urchin_molly_guard() {
 
 shell_for_sh_tests=
 force=false
-exit_on_fail=false
-tap_format=false
 while [ $# -gt 0 ]
 do
     case "$1" in
-        -e) exit_on_fail=true;;
         -f) force=true;;
         -s)
           shift
           shell_for_sh_tests=$1
           which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
           ;;
-        -t) tap_format=true;;
         -h|--help) urchin_help
           exit 0;;
-        -v) echo "$VERSION"
-            exit;;
         -*) urchin_help >&2
             exit 1;;
         *)  break;;