add -- after readlink

add Thomas Levine and Michael Klement

COPYING

@@ -1,3 +1,5 @@
+Copyright (c) 2013, 2014, 2015, 2016 Thomas Levine
+Copyright (c) 2014, Michael Klement
 Copyright (c) 2012, ScraperWiki Limited
 All rights reserved.
 

HISTORY

@@ -1,6 +1,29 @@
 HISTORY
 =======
 
+Version 0.0.7
+---------------------
+
+The Molly-guard is now more accepting. For example, you no longer need to
+pass -f in this case: https://github.com/creationix/nvm/issues/357
+
+Previously, tests were run if they were executable and were otherwise marked
+as skipped. Now, an executable script can indicate that it is skipped by
+exiting with code 3.  For example, if a test requires some dependancy, it
+might look for the dependency and then skip if it does not see the dependency.
+It might look like this.
+
+    #!/bin/sh
+    if which inkscape; then
+      exit 3 # status code 3 for skip
+    fi
+    inkscape blah blah ...
+
+I chose status code 3 sort of arbitrarily at first, but it turns out that it
+would the appropriate status code if these tests were Nagios plugins, as the
+concept of skipping a test is similar to the Nagios concept of unknown service
+status (https://nagios-plugins.org/doc/guidelines.html#AEN78).
+
 Version 0.0.6
 ---------------------
 
@@ -14,6 +37,9 @@ Version 0.0.6
 * Remove the undocumented, experimental -x flag now that shall exists.
 * Display version number with the -v flag.
 * Document why Urchin is called "Urchin"
+* Update TODO
+* Support mksh (Change a printf command.)
+* Make long lines shorter.
 
 These changes are made somewhat separately in the branches "exit-on-fail",
 "remove-urchin-x", "tap", and "update-readme". They are rebased into one

TODO

@@ -1,13 +1,6 @@
 Things I want
 =============
 
-Molly guard
--------------
-The Molly-guard should be more accepting so that people don't have to use it
-all the time and thus get used to using it. For example, you shouldn't need to
-pass -f in this case.
-https://github.com/creationix/nvm/issues/357
-
 Test speed
 -------------
 Make tests run faster.
@@ -26,6 +19,11 @@ Hmm or maybe there's a compromise: Tell people to mount /tmp as a tmpfs so
 that temp files are fast. Maybe allow people to set some other directory as
 the temporary file place, in case they want a different tmpfs location.
 
+In order to run things in parallel, we have to change how we do the
+stdout_file. I think it's easiest to create separate files for each test and
+to save them in testroot/.urchin/stdout/$filename. The test root would be
+defined as the closest ancestor containing a .urchin directory.
+
 Options
 -------------
 I want long options. For example, there's presently -f and -e.
@@ -109,3 +107,58 @@ in the case of Urchin.
 This is as far as I have gotten with contemplating license changes. For now
 we're sticking with the original MIT-style license, but it's easy to change
 licenses later.
+
+Nagios plugins
+-----------------
+It would be cool to run Nagios plugins with Urchin. This is already possible,
+actually, but it might be worth giving some special thought to it.
+https://nagios-plugins.org/doc/guidelines.html
+
+Source setup and teardown
+--------------------
+If setup and teardown are sourced instead of executed, maybe we can more
+cleanly create and teardown temporary files.
+
+    (
+      . ./setup
+      ./$thetestfile
+      . ./teardown
+    )
+
+On the other hand, this could just be sourced explicitly in the test file,
+without the special setup and teardown feature.
+
+Run on a file
+----------------
+Presently you can run urchin only on a directory.
+It would be neat if you could run it on a file as well.
+
+This occurred to me when I wanted to run
+
+    urchin test/fast/Unit\ tests/nvm_ls_current
+
+on the nvm tests. I wound up running this instead.
+
+    urchin test/fast/Unit\ tests/ | grep nvm_ls_current
+
+The Molly guard would be assessed, and the corresponding setup, setup_dir,
+teardown, and teardown_dir files would be run in the appropriate order.
+
+In order to know how far up the tree to evaluate the setup, &c. files,
+I think it would make sense to require that a ".urchin" file be placed in the
+root of the tests. Urchin would keep going up until it sees this file, and it
+would evaluate the appropriate setup, &c. files from there down to the
+particular test file of interest. We would also use this for testing
+directtories more correctly.
+
+Running automated tasks
+-------------------------
+Urchin might be appropriate for if you have lots of tasks that you want to run
+periodically; add an urchin call to your crontab, and call all of your other
+tasks with urchin. Here are some features that might make urchin better for
+this sort of thing.
+                                                      
+* Time how long each test/job takes
+* Optionally kill tests/jobs after a specific timeout threshold
+* Send output of different tests/jobs to different files for each file
+  descriptor (STDOUT, STDERR)

alternatives

@@ -1,2 +1,9 @@
 Totally different syntax and similar features, plus TAP output
 https://github.com/sstephenson/bats
+
+Relatively similar interface
+https://github.com/mlafeldt/sharness
+
+Lists of alternatives
+https://thomaslevine.com/!/shell-testing/
+https://github.com/mlafeldt/sharness#alternatives

cross-shell-tests

@@ -2,7 +2,7 @@
 # Run urchin in a bunch of different shells,
 # including a shell that isn't quite POSIX-compatible (zsh)
 
-for shell in dash bash ksh zsh; do
+for shell in dash bash mksh ksh zsh; do
   if which $shell > /dev/null 2> /dev/null; then
     echo
     echo Running urchin tests in $shell

readme.md

@@ -36,7 +36,7 @@ Urchin is contained in a single file, so you can install it by copying it to a
 directory in your `PATH`. For example, you can run the following as root.
 
     cd /usr/local/bin
-    wget https://raw.github.com/tlevine/urchin/master/urchin
+    wget https://raw.githubusercontent.com/tlevine/urchin/v0.0.6/urchin
     chmod +x urchin
 
 Urchin can be installed with npm too.
@@ -169,9 +169,3 @@ It does something similar, but the interface may be more intuitive.
 ## Alternatives to Urchin
 Alternatives to Urchin are discussed in
 [this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
-
-## Ideas for new features
-
-* Support [Nagios plugins](https://nagios-plugins.org/doc/guidelines.html)
-* Stop running if a test fails so one can use Urchin as a
-    [setup framework](https://github.com/tlevine/urchin/issues/16).

tests/.urchin/.gitignore

@@ -0,0 +1,2 @@
+*
+!.gitignore

tests/Test root contains .urchin./.meta-assess

@@ -0,0 +1,6 @@
+#!/bin/sh
+
+observed=$(../../urchin "$1" .testsuite/a/b)
+expected=.testsuite/a/b/../..
+
+test "$observed" = "$expected"

tests/Test root contains .urchin./.testsuite/.urchin/.gitignore

@@ -0,0 +1,2 @@
+*
+!.gitignore

tests/Test root contains .urchin./.testsuite/a/b/c/testcase

@@ -0,0 +1 @@
+#!/usr/bin/env true

tests/Test root contains .urchin./Assess root on a file.

@@ -0,0 +1,2 @@
+#!/bin/sh
+../../urchin --root .testsuite/a/b/c/testcase

tests/Test root contains .urchin./Assess root with --root.

@@ -0,0 +1,2 @@
+#!/bin/sh
+./.meta-assess --root

tests/Test root contains .urchin./Assess root with -r.

@@ -0,0 +1,2 @@
+#!/bin/sh
+./.meta-assess -r

tests/Test root contains .urchin./No root set

@@ -0,0 +1,2 @@
+#!/bin/sh
+../../urchin --root /bin 2>&1 | grep "'/bin/.urchin'"

tests/Test root contains .urchin./Wrong filename

@@ -0,0 +1,2 @@
+#!/bin/sh
+! ../../urchin --root .testsuite/a/not-a-file

tests/Urchin should do a sanity check so that directories like the user home are not specified as tests./.chainsaw/successful test

@@ -1,3 +0,0 @@
-#!/bin/sh
-
-exit 0

tests/Urchin should do a sanity check so that directories like the user home are not specified as tests./.foo/test/bar/baz

@@ -0,0 +1 @@
+#!/usr/bin/env true

tests/Urchin should do a sanity check so that directories like the user home are not specified as tests./.test/successful test

@@ -1,3 +1 @@
-#!/bin/sh
-
-exit 0
+#!/usr/bin/env true

tests/Urchin should do a sanity check so that directories like the user home are not specified as tests./The Molly-guard should accept arguments that contain test in the middle.

@@ -0,0 +1,3 @@
+#!/bin/sh
+
+../../urchin .foo/test/bar

tests/Urchin should do a sanity check so that directories like the user home are not specified as tests./Urchin should warn if it is run on something that does not look like a test.

@@ -1,3 +1,5 @@
 #!/bin/sh
 
-! ../../urchin ./.chainsaw
+tmp=$(mktemp -d)/blah
+echo '#!/usr/bin/env true' > $tmp
+! ../../urchin $tmp

tests/get_stdout_file/t

@@ -0,0 +1,4 @@
+#!/bin/sh
+
+I_AM_SOURCING_URCHIN_IN_A_TEST=true . ../../urchin
+test $(get_stdout_file t) = aoeu

tests/skip/.test/fail

@@ -0,0 +1 @@
+exit 1

tests/skip/.test/skip

@@ -0,0 +1 @@
+exit 3

tests/skip/.test/succeed

@@ -0,0 +1 @@
+exit 0

tests/skip/Skip on exit code 3.

@@ -0,0 +1 @@
+test $(../../urchin -t .test/ | grep -c SKIP) -eq 1

urchin

@@ -1,22 +1,84 @@
 #!/bin/sh
 
-# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably - notably, it can produce output,
-# which breaks fullpath().
+# This file is part of urchin. It is subject to the license terms in the
+# COPYING file found in the top-level directory of this distribution or at
+# https://raw.githubusercontent.com/tlevine/urchin/master/COPYING
+
+# No part of urchin, including this file, may be copied, modified, propagated,
+# or distributed except according to the terms contained in the COPYING file.
+
+set +e
+
+# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably -
+# notably, it can produce output, which breaks fullpath().
 unset CDPATH
 
 # Urchin version number
 VERSION=0.0.6
 
-fullpath() {
-  (
-    cd -- "$1"
-    pwd
-  )
+urchin_root() {
+  # Call recursively but remember the original argument.
+  current="$(remove_trailing_slash "$1")"
+  if test -n "$2"; then
+    orig="$2"
+  else
+    orig="$1"
+  fi
+
+  if test "$(readlink -f -- "$1")" = /; then
+    # Stop traversing upwards at /
+    if test -d "$orig"; then
+      origdir="$orig"
+    else
+      origdir=$(dirname "$orig")
+    fi
+    echo "You need to create the .urchin directory in the root of your tests,
+maybe like this:
+
+  mkdir '$(readlink -f -- "$(remove_trailing_slash "$origdir")")/.urchin'
+" >&2
+    return 1
+  elif ! test -e "$current"; then
+    echo "$current: No such file or directory">&2
+    return 1
+  elif test -f "$current"; then
+    urchin_root "$(dirname "$current")" "$orig"
+  elif test -d "$current"/.urchin; then
+    remove_trailing_slash "$current"
+  elif test "$current" != . && test "$current" != .. &&
+    echo "$current" | grep '^\.' && > /dev/null; then
+    # Stop traversing upwards at hidden directories.
+    urchin_root / "$orig"
+  else
+    urchin_root "$current"/.. "$orig"
+  fi
 }
 
 indent() {
   level="$1"
-  printf "%$((2 * ${level}))s"
+  if test "$level" -gt 0; then
+    printf "%$((2 * ${level}))s"
+  fi
+}
+
+remove_trailing_slash() {
+  echo "$1" | sed s/\\/$//
+}
+
+escape_slashes() {
+  echo "$1" | sed s+/+\\\\/+g
+}
+
+get_stdout_file() {
+  root="$(readlink -f -- "$(urchin_root "$1")")"
+  test_file="$(readlink -f -- "$1")"
+  eroot="$(escape_slashes "$root")"
+  base="$(echo "$test_file" | sed "s/^$eroot/$eroot\/.urchin/")"
+  if test -d "$1"; then
+    echo "$base"/.index
+  else
+    echo "$base"
+  fi
 }
 
 recurse() {
@@ -24,11 +86,15 @@ recurse() {
   indent_level="$2"
   shell_for_sh_tests="$3"
 
+  [ "$potential_test" = '.urchin' ] && return
   [ "$potential_test" = 'setup_dir' ] && return
   [ "$potential_test" = 'teardown_dir' ] && return
   [ "$potential_test" = 'setup' ] && return
   [ "$potential_test" = 'teardown' ] && return
 
+  stdout_file="$(get_stdout_file "$potential_test")"
+  mkdir -p "$(dirname "$stdout_file")"
+
   [ $indent_level -eq 0 ] && : > "$stdout_file"
 
   if [ -d "$potential_test" ]
@@ -56,8 +122,10 @@ recurse() {
         [ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
 
         # $2 instead of $indent_level so it doesn't clash
-        recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
+        set +e
+        recurse "${test}" "$(( $2 + 1 ))" "$shell_for_sh_tests"
         exit_code=$?
+        set -e
 
         if $exit_on_fail && test $exit_code -ne 0; then
           [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
@@ -92,6 +160,8 @@ recurse() {
       [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
       if [ $exit_code -eq 0 ]; then
         result=success
+      elif [ $exit_code -eq 3 ]; then
+        result=skip
       else
         result=fail
       fi
@@ -131,7 +201,10 @@ recurse() {
           # On fail, print a red '✗'
           printf '\033[31m✗ \033[0m'
           printf '%s\n' "${potential_test}"
-          printf '\033[31m' # Print output captured from failed test in red.
+
+          # Print output captured from failed test in red.
+          printf '\033[31m'
+
           cat "$stdout_file"
           printf '\033[0m'
         ;;
@@ -145,12 +218,16 @@ recurse() {
       return 1
     fi
   fi
-  [ $indent_level -eq 0 ] && rm "$stdout_file"
 }
 
 has_sh_or_no_shebang_line() {
-  head -n 1 "$1" | grep -vqE '^#!' && return 0 # no shebang line at all
-  head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' && return 0  # shebang line is '#!/bin/sh' or legal variations thereof
+  # no shebang line at all
+  head -n 1 "$1" | grep -vqE '^#!' && return 0
+
+  # shebang line is '#!/bin/sh' or legal variations thereof
+  head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' &&
+    return 0  
+
   return 1
 }
 
@@ -170,6 +247,7 @@ $USAGE
             contain the word "test".
 -t          Format output in Test Anything Protocol (TAP)
 -h, --help  This help.
+-r, --root  Print the Urchin root for a particular file or directory.
 -v          Display the version number.
 
 Go to https://github.com/tlevine/urchin for documentation on writing tests.
@@ -190,12 +268,11 @@ plural () {
 }
 
 urchin_go() {
-  rm -f "$logfile"
   if "$tap_format"; then
     printf \#\ 
   fi
   echo Running tests at $(date +%Y-%m-%dT%H:%M:%S)
-  start=$(date +%s)
+  start="$(date +%s)"
 
   # Determine the environment variable to define for test scripts
   # that reflects the specified or implied shell to use for shell-code tests.
@@ -210,7 +287,10 @@ urchin_go() {
     TEST_SHELL='/bin/sh'
   fi
 
-  recurse "$1" 0 "$2"  # test folder -- indentation level -- [shell to invoke test scripts with]
+  # 1 test folder
+  # 2 indentation level
+  # 3 shell to invoke test scripts with
+  recurse "$1" 0 "$2"
 
   finish=$(date +%s)
   elapsed=$(($finish - $start))
@@ -225,11 +305,14 @@ urchin_go() {
     echo "Done, took $elapsed $(plural second $elapsed)."
     printf '%s\n' "$passed $(plural test "$passed") passed."
     printf '%s\n' "$skipped $(plural test "$skipped") skipped."
-    [ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
+
+    # If tests failed, print the message in red, otherwise in green.
+    [ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m'
+
     printf '%s\n' "$failed $(plural test "$failed") failed."
     printf '\033[m'
   fi
-  rm -f "$logfile"
+# rm -f "$logfile"
   test -z "$failed" || test "$failed" -eq '0'
 }
 
@@ -245,48 +328,57 @@ urchin_molly_guard() {
   exit 1
 }
 
-shell_for_sh_tests=
-force=false
-exit_on_fail=false
-tap_format=false
-while [ $# -gt 0 ]
-do
-    case "$1" in
-        -e) exit_on_fail=true;;
-        -f) force=true;;
-        -s)
-          shift
-          shell_for_sh_tests=$1
-          which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
-          ;;
-        -t) tap_format=true;;
-        -h|--help) urchin_help
-          exit 0;;
-        -v) echo "$VERSION"
+if test -z "$I_AM_SOURCING_URCHIN_IN_A_TEST"; then
+  shell_for_sh_tests=
+  force=false
+  exit_on_fail=false
+  tap_format=false
+  while [ $# -gt 0 ]
+  do
+      case "$1" in
+          -e) exit_on_fail=true;;
+          -f) force=true;;
+          -r|--root)
+            shift
+            urchin_root "$1"
             exit;;
-        -*) urchin_help >&2
-            exit 1;;
-        *)  break;;
-    esac
-    shift
-done
+          -s)
+            shift
+            shell_for_sh_tests=$1
+            which "$shell_for_sh_tests" > /dev/null || {
+              echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2
+              urchin_help >&2
+              exit 11
+            } ;;
+          -t) tap_format=true;;
+          -h|--help) urchin_help
+            exit 0;;
+          -v) echo "$VERSION"
+              exit;;
+          -*) urchin_help >&2
+              exit 1;;
+          *)  break;;
+      esac
+      shift
+  done
 
-# Verify argument for main stuff
-if [ "$#" != '1' ] || [ ! -d "$1" ]
-  then
-  [ -n "$1" ] && [ ! -d "$1" ] && echo "Not a directory: '$1'" >&2
-  echo "$USAGE" >&2
-  exit 2
-fi
-
-# Constants
-logfile=$(fullpath "$1")/.urchin.log
-stdout_file=$(fullpath "$1")/.urchin_stdout
-
-# Run or present the Molly guard.
-if basename "$(fullpath "$1")" | grep -Fi 'test' > /dev/null || $force
-  then
-  urchin_go "$1" "$shell_for_sh_tests"
-else
-  urchin_molly_guard
+  # Verify argument for main stuff
+  if [ "$#" != '1' ] || [ ! -d "$1" ]
+    then
+    [ -n "$1" ] && [ ! -d "$1" ] && echo "Not a directory: '$1'" >&2
+    echo "$USAGE" >&2
+    exit 11
+  fi
+
+  # Run or present the Molly guard.
+  if ! urchin_root "$1" > /dev/null; then
+    exit 1
+  elif basename "$(readlink -f -- "$(urchin_root "$1")")" |
+    grep -Fi 'test' > /dev/null || $force; then
+    logfile="$(readlink -f -- "$(urchin_root "$1")/.urchin/.log")"
+    printf '' > "$logfile"
+    urchin_go "$1" "$shell_for_sh_tests"
+  else
+    urchin_molly_guard
+  fi
 fi