Compare commits
	
		
			1 Commits
		
	
	
		
			source-set
			...
			more-shell
		
	
	| Author | SHA1 | Date | |
|---|---|---|---|
| 
						 | 
					7126872a8b | 
							
								
								
									
										10
									
								
								HISTORY
									
									
									
									
									
								
							
							
						
						
									
										10
									
								
								HISTORY
									
									
									
									
									
								
							@@ -1,13 +1,6 @@
 | 
			
		||||
HISTORY
 | 
			
		||||
=======
 | 
			
		||||
 | 
			
		||||
Version 0.0.7
 | 
			
		||||
---------------------
 | 
			
		||||
 | 
			
		||||
The Molly-guard is now more accepting. For example, you no longer need to
 | 
			
		||||
pass -f in this case: https://github.com/creationix/nvm/issues/357
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
Version 0.0.6
 | 
			
		||||
---------------------
 | 
			
		||||
 | 
			
		||||
@@ -21,9 +14,6 @@ Version 0.0.6
 | 
			
		||||
* Remove the undocumented, experimental -x flag now that shall exists.
 | 
			
		||||
* Display version number with the -v flag.
 | 
			
		||||
* Document why Urchin is called "Urchin"
 | 
			
		||||
* Update TODO
 | 
			
		||||
* Support mksh (Change a printf command.)
 | 
			
		||||
* Make long lines shorter.
 | 
			
		||||
 | 
			
		||||
These changes are made somewhat separately in the branches "exit-on-fail",
 | 
			
		||||
"remove-urchin-x", "tap", and "update-readme". They are rebased into one
 | 
			
		||||
 
 | 
			
		||||
@@ -1,5 +1,3 @@
 | 
			
		||||
Copyright (c) 2013, 2014, 2015, 2016 Thomas Levine
 | 
			
		||||
Copyright (c) 2014, Michael Klement
 | 
			
		||||
Copyright (c) 2012, ScraperWiki Limited
 | 
			
		||||
All rights reserved.
 | 
			
		||||
 | 
			
		||||
							
								
								
									
										152
									
								
								TODO
									
									
									
									
									
								
							
							
						
						
									
										152
									
								
								TODO
									
									
									
									
									
								
							@@ -1,152 +0,0 @@
 | 
			
		||||
Things I want
 | 
			
		||||
=============
 | 
			
		||||
 | 
			
		||||
Test speed
 | 
			
		||||
-------------
 | 
			
		||||
Make tests run faster.
 | 
			
		||||
https://github.com/bike-barn/hermit/issues/62
 | 
			
		||||
 | 
			
		||||
First, easier thing is probably to run tests in parallel.
 | 
			
		||||
 | 
			
		||||
Second, also easier thing is to tell people to save things to RAM rather than
 | 
			
		||||
disk whenever they can.
 | 
			
		||||
 | 
			
		||||
Third, harder thing is to put the test suite in RAM automatically. Maybe the
 | 
			
		||||
whole test directory, which includes fixtures, gets copied to a tmpfs if one
 | 
			
		||||
exists.
 | 
			
		||||
 | 
			
		||||
Hmm or maybe there's a compromise: Tell people to mount /tmp as a tmpfs so
 | 
			
		||||
that temp files are fast. Maybe allow people to set some other directory as
 | 
			
		||||
the temporary file place, in case they want a different tmpfs location.
 | 
			
		||||
 | 
			
		||||
Options
 | 
			
		||||
-------------
 | 
			
		||||
I want long options. For example, there's presently -f and -e.
 | 
			
		||||
I want to make them -f|--force and -e|--exit.
 | 
			
		||||
 | 
			
		||||
Environment variables
 | 
			
		||||
-------------
 | 
			
		||||
Do something to make it easier to debug environment variables, because that is
 | 
			
		||||
often confusing.
 | 
			
		||||
https://github.com/creationix/nvm/issues/719
 | 
			
		||||
https://github.com/creationix/nvm/issues/589
 | 
			
		||||
 | 
			
		||||
Documenting that people should run "env" when their tests fail might be good
 | 
			
		||||
enough.
 | 
			
		||||
 | 
			
		||||
Licensing and copyright
 | 
			
		||||
------------------------
 | 
			
		||||
* Reference all owners and years in the Copyright file
 | 
			
		||||
* Consider copyleft licenses
 | 
			
		||||
* Add license notices to other files if necessary
 | 
			
		||||
 | 
			
		||||
Packaging
 | 
			
		||||
------------
 | 
			
		||||
Package for package managers.
 | 
			
		||||
 | 
			
		||||
* I want NixOS, of course.
 | 
			
		||||
* Debian is probably the big one.
 | 
			
		||||
 | 
			
		||||
Other interesting package managers
 | 
			
		||||
 | 
			
		||||
* Update the npm package
 | 
			
		||||
* Homebrew (for Mac)
 | 
			
		||||
 | 
			
		||||
Windows
 | 
			
		||||
----------
 | 
			
		||||
Try running Urchin in Windows somehow. Interpreters include
 | 
			
		||||
 | 
			
		||||
* CygWin (https://www.cygwin.com/)
 | 
			
		||||
* MSYS (http://mingw.org/wiki/msys)
 | 
			
		||||
* GNU on Windows (https://github.com/bmatzelle/gow/wiki)
 | 
			
		||||
* Git for Windows (https://git-scm.com/download/win)
 | 
			
		||||
* win-bash (http://win-bash.sourceforge.net/)
 | 
			
		||||
 | 
			
		||||
Consider copyleft licenses
 | 
			
		||||
----------
 | 
			
		||||
ScraperWiki owns the original version of Urchin (Thomas Levine did the early
 | 
			
		||||
work as part of his work for ScraperWiki.) and originally licensed it under an
 | 
			
		||||
MIT-style license. Other people made changes after this original ScraperWiki
 | 
			
		||||
version. As of January 2016, they are just Thomas Levine (when he wasn't
 | 
			
		||||
working for ScraperWiki) and Michael Klement.
 | 
			
		||||
 | 
			
		||||
The original license was MIT just because that's what ScraperWiki put on
 | 
			
		||||
everything. Should we change the license?
 | 
			
		||||
 | 
			
		||||
The MIT-style license grants pretty much all rights. It says that you need
 | 
			
		||||
to attribute when you redistribute source code, but you don't
 | 
			
		||||
necessarily have to redistribute source code.
 | 
			
		||||
 | 
			
		||||
A copyleft license adds the restriction that modified versions of the
 | 
			
		||||
code need to be licensed under the same license. GNU licenses in
 | 
			
		||||
particular require that source code be released if non-source versions are
 | 
			
		||||
released, and the different GNU licenses differ in what how the
 | 
			
		||||
non-source version is defined. (The original, GPL, discusses compiled
 | 
			
		||||
binaries.) Copyleft doesn't mean anything specific for commercial use.
 | 
			
		||||
 | 
			
		||||
MIT-licensed code can be modified and then licensed as GPL, because MIT
 | 
			
		||||
license allows that, but GPL code can't be modified as MIT, because MIT
 | 
			
		||||
doesn't allow that. And if we get all of the authors to agree on it, we
 | 
			
		||||
can always add whatever crazy license we want, regardless of what we
 | 
			
		||||
have already.
 | 
			
		||||
 | 
			
		||||
The distinction between MIT-style and GNU-something might matter quite little
 | 
			
		||||
in the case of Urchin.
 | 
			
		||||
 | 
			
		||||
1. Urchin is written in an interpreted language (shell), so it might be
 | 
			
		||||
   hard to distribute usefully without providing the source code.
 | 
			
		||||
2. Urchin just runs tests; it doesn't get compiled with the rest of the
 | 
			
		||||
   code (also because it's in shell). Thus, I think a GPL license on
 | 
			
		||||
   Urchin wouldn't infect the code being tested.
 | 
			
		||||
 | 
			
		||||
This is as far as I have gotten with contemplating license changes. For now
 | 
			
		||||
we're sticking with the original MIT-style license, but it's easy to change
 | 
			
		||||
licenses later.
 | 
			
		||||
 | 
			
		||||
Nagios plugins
 | 
			
		||||
-----------------
 | 
			
		||||
It would be cool to run Nagios plugins with Urchin. This is already possible,
 | 
			
		||||
actually, but it might be worth giving some special thought to it.
 | 
			
		||||
https://nagios-plugins.org/doc/guidelines.html
 | 
			
		||||
 | 
			
		||||
Source setup and teardown
 | 
			
		||||
--------------------
 | 
			
		||||
If setup and teardown are sourced instead of executed, maybe we can more
 | 
			
		||||
cleanly create and teardown temporary files.
 | 
			
		||||
 | 
			
		||||
    (
 | 
			
		||||
      . ./setup
 | 
			
		||||
      ./$thetestfile
 | 
			
		||||
      . ./teardown
 | 
			
		||||
    )
 | 
			
		||||
 | 
			
		||||
On the other hand, this could just be sourced explicitly in the test file,
 | 
			
		||||
without the special setup and teardown feature.
 | 
			
		||||
 | 
			
		||||
Run on a file
 | 
			
		||||
----------------
 | 
			
		||||
Presently you can run urchin only on a directory.
 | 
			
		||||
It would be neat if you could run it on a file as well.
 | 
			
		||||
 | 
			
		||||
This occurred to me when I wanted to run
 | 
			
		||||
 | 
			
		||||
    urchin test/fast/Unit\ tests/nvm_ls_current
 | 
			
		||||
 | 
			
		||||
on the nvm tests. I wound up running this instead.
 | 
			
		||||
 | 
			
		||||
    urchin test/fast/Unit\ tests/ | grep nvm_ls_current
 | 
			
		||||
 | 
			
		||||
The Molly guard would be assessed, and the corresponding setup, setup_dir,
 | 
			
		||||
teardown, and teardown_dir files would be run in the appropriate order.
 | 
			
		||||
 | 
			
		||||
Running automated tasks
 | 
			
		||||
-------------------------
 | 
			
		||||
Urchin might be appropriate for if you have lots of tasks that you want to run
 | 
			
		||||
periodically; add an urchin call to your crontab, and call all of your other
 | 
			
		||||
tasks with urchin. Here are some features that might make urchin better for
 | 
			
		||||
this sort of thing.
 | 
			
		||||
                                                      
 | 
			
		||||
* Time how long each test/job takes
 | 
			
		||||
* Optionally kill tests/jobs after a specific timeout threshold
 | 
			
		||||
* Send output of different tests/jobs to different files for each file
 | 
			
		||||
  descriptor (STDOUT, STDERR)
 | 
			
		||||
@@ -1,9 +1,2 @@
 | 
			
		||||
Totally different syntax and similar features, plus TAP output
 | 
			
		||||
https://github.com/sstephenson/bats
 | 
			
		||||
 | 
			
		||||
Relatively similar interface
 | 
			
		||||
https://github.com/mlafeldt/sharness
 | 
			
		||||
 | 
			
		||||
Lists of alternatives
 | 
			
		||||
https://thomaslevine.com/!/shell-testing/
 | 
			
		||||
https://github.com/mlafeldt/sharness#alternatives
 | 
			
		||||
 
 | 
			
		||||
@@ -36,7 +36,7 @@ Urchin is contained in a single file, so you can install it by copying it to a
 | 
			
		||||
directory in your `PATH`. For example, you can run the following as root.
 | 
			
		||||
 | 
			
		||||
    cd /usr/local/bin
 | 
			
		||||
    wget https://raw.githubusercontent.com/tlevine/urchin/v0.0.6/urchin
 | 
			
		||||
    wget https://raw.github.com/tlevine/urchin/master/urchin
 | 
			
		||||
    chmod +x urchin
 | 
			
		||||
 | 
			
		||||
Urchin can be installed with npm too.
 | 
			
		||||
@@ -169,3 +169,9 @@ It does something similar, but the interface may be more intuitive.
 | 
			
		||||
## Alternatives to Urchin
 | 
			
		||||
Alternatives to Urchin are discussed in
 | 
			
		||||
[this blog post](https://blog.scraperwiki.com/2012/12/how-to-test-shell-scripts/).
 | 
			
		||||
 | 
			
		||||
## Ideas for new features
 | 
			
		||||
 | 
			
		||||
* Support [Nagios plugins](https://nagios-plugins.org/doc/guidelines.html)
 | 
			
		||||
* Stop running if a test fails so one can use Urchin as a
 | 
			
		||||
    [setup framework](https://github.com/tlevine/urchin/issues/16).
 | 
			
		||||
 
 | 
			
		||||
@@ -0,0 +1,3 @@
 | 
			
		||||
#!/bin/sh
 | 
			
		||||
 | 
			
		||||
exit 0
 | 
			
		||||
@@ -1 +0,0 @@
 | 
			
		||||
#!/usr/bin/env true
 | 
			
		||||
@@ -1 +1,3 @@
 | 
			
		||||
#!/usr/bin/env true
 | 
			
		||||
#!/bin/sh
 | 
			
		||||
 | 
			
		||||
exit 0
 | 
			
		||||
 
 | 
			
		||||
@@ -1,3 +0,0 @@
 | 
			
		||||
#!/bin/sh
 | 
			
		||||
 | 
			
		||||
../../urchin .foo/test/bar
 | 
			
		||||
@@ -1,5 +1,3 @@
 | 
			
		||||
#!/bin/sh
 | 
			
		||||
 | 
			
		||||
tmp=$(mktemp -d)/blah
 | 
			
		||||
echo '#!/usr/bin/env true' > $tmp
 | 
			
		||||
! ../../urchin $tmp
 | 
			
		||||
! ../../urchin ./.chainsaw
 | 
			
		||||
 
 | 
			
		||||
							
								
								
									
										61
									
								
								urchin
									
									
									
									
									
								
							
							
						
						
									
										61
									
								
								urchin
									
									
									
									
									
								
							@@ -1,16 +1,7 @@
 | 
			
		||||
#!/bin/sh
 | 
			
		||||
 | 
			
		||||
# This file is part of urchin. It is subject to the license terms in the
 | 
			
		||||
# COPYING file found in the top-level directory of this distribution or at
 | 
			
		||||
# https://raw.githubusercontent.com/tlevine/urchin/master/COPYING
 | 
			
		||||
 | 
			
		||||
# No part of urchin, including this file, may be copied, modified, propagated,
 | 
			
		||||
# or distributed except according to the terms contained in the COPYING file.
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably -
 | 
			
		||||
# notably, it can produce output, which breaks fullpath().
 | 
			
		||||
# Make sure that CDPATH isn't set, as it causes `cd` to behave unpredictably - notably, it can produce output,
 | 
			
		||||
# which breaks fullpath().
 | 
			
		||||
unset CDPATH
 | 
			
		||||
 | 
			
		||||
# Urchin version number
 | 
			
		||||
@@ -25,9 +16,7 @@ fullpath() {
 | 
			
		||||
 | 
			
		||||
indent() {
 | 
			
		||||
  level="$1"
 | 
			
		||||
  if test "$level" -gt 0; then
 | 
			
		||||
    printf "%$((2 * ${level}))s"
 | 
			
		||||
  fi
 | 
			
		||||
  printf "%$((2 * ${level}))s"
 | 
			
		||||
}
 | 
			
		||||
 | 
			
		||||
recurse() {
 | 
			
		||||
@@ -55,7 +44,7 @@ recurse() {
 | 
			
		||||
 | 
			
		||||
    (
 | 
			
		||||
      cd -- "$potential_test"
 | 
			
		||||
      [ -f setup_dir ] && [ -x setup_dir ] && . ./setup_dir >> "$stdout_file"
 | 
			
		||||
      [ -f setup_dir ] && [ -x setup_dir ] && ./setup_dir >> "$stdout_file"
 | 
			
		||||
 | 
			
		||||
      if [ -n "$ZSH_VERSION" ]; then
 | 
			
		||||
        # avoid "no matches found: *" error when directories are empty
 | 
			
		||||
@@ -64,21 +53,21 @@ recurse() {
 | 
			
		||||
 | 
			
		||||
      for test in *
 | 
			
		||||
        do
 | 
			
		||||
        [ -f setup ] && [ -x setup ] && . ./setup >> "$stdout_file"
 | 
			
		||||
        [ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
 | 
			
		||||
 | 
			
		||||
        # $2 instead of $indent_level so it doesn't clash
 | 
			
		||||
        recurse "${test}" $(( $2 + 1 )) "$shell_for_sh_tests"
 | 
			
		||||
        exit_code=$?
 | 
			
		||||
 | 
			
		||||
        if $exit_on_fail && test $exit_code -ne 0; then
 | 
			
		||||
          [ -f teardown ] && [ -x teardown ] && . ./teardown >> "$stdout_file"
 | 
			
		||||
          [ -f teardown_dir ] && [ -x teardown_dir ] && . ./teardown_dir >> "$stdout_file"
 | 
			
		||||
          [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
 | 
			
		||||
          [ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
 | 
			
		||||
          return 1
 | 
			
		||||
        fi
 | 
			
		||||
 | 
			
		||||
        [ -f teardown ] && [ -x teardown ] && . ./teardown >> "$stdout_file"
 | 
			
		||||
        [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
 | 
			
		||||
      done
 | 
			
		||||
      [ -f teardown_dir ] && [ -x teardown_dir ] && . ./teardown_dir >> "$stdout_file"
 | 
			
		||||
      [ -f teardown_dir ] && [ -x teardown_dir ] && ./teardown_dir >> "$stdout_file"
 | 
			
		||||
    )
 | 
			
		||||
    if $tap_format; then
 | 
			
		||||
      indent $indent_level | sed 's/  /#/g'
 | 
			
		||||
@@ -89,7 +78,7 @@ recurse() {
 | 
			
		||||
  else
 | 
			
		||||
    if [ -x "$potential_test" ]
 | 
			
		||||
      then
 | 
			
		||||
      [ -f setup ] && [ -x setup ] && . ./setup >> "$stdout_file"
 | 
			
		||||
      [ -f setup ] && [ -x setup ] && ./setup >> "$stdout_file"
 | 
			
		||||
 | 
			
		||||
      # Run the test
 | 
			
		||||
      if [ -n "$shell_for_sh_tests" ] && has_sh_or_no_shebang_line ./"$potential_test"
 | 
			
		||||
@@ -100,7 +89,7 @@ recurse() {
 | 
			
		||||
      fi
 | 
			
		||||
      exit_code="$?"
 | 
			
		||||
 | 
			
		||||
      [ -f teardown ] && [ -x teardown ] && . ./teardown >> "$stdout_file"
 | 
			
		||||
      [ -f teardown ] && [ -x teardown ] && ./teardown >> "$stdout_file"
 | 
			
		||||
      if [ $exit_code -eq 0 ]; then
 | 
			
		||||
        result=success
 | 
			
		||||
      else
 | 
			
		||||
@@ -142,10 +131,7 @@ recurse() {
 | 
			
		||||
          # On fail, print a red '✗'
 | 
			
		||||
          printf '\033[31m✗ \033[0m'
 | 
			
		||||
          printf '%s\n' "${potential_test}"
 | 
			
		||||
 | 
			
		||||
          # Print output captured from failed test in red.
 | 
			
		||||
          printf '\033[31m'
 | 
			
		||||
 | 
			
		||||
          printf '\033[31m' # Print output captured from failed test in red.
 | 
			
		||||
          cat "$stdout_file"
 | 
			
		||||
          printf '\033[0m'
 | 
			
		||||
        ;;
 | 
			
		||||
@@ -163,13 +149,8 @@ recurse() {
 | 
			
		||||
}
 | 
			
		||||
 | 
			
		||||
has_sh_or_no_shebang_line() {
 | 
			
		||||
  # no shebang line at all
 | 
			
		||||
  head -n 1 "$1" | grep -vqE '^#!' && return 0
 | 
			
		||||
 | 
			
		||||
  # shebang line is '#!/bin/sh' or legal variations thereof
 | 
			
		||||
  head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' &&
 | 
			
		||||
    return 0  
 | 
			
		||||
 | 
			
		||||
  head -n 1 "$1" | grep -vqE '^#!' && return 0 # no shebang line at all
 | 
			
		||||
  head -n 1 "$1" | grep -qE '^#![[:blank:]]*/bin/sh($|[[:blank:]])' && return 0  # shebang line is '#!/bin/sh' or legal variations thereof
 | 
			
		||||
  return 1
 | 
			
		||||
}
 | 
			
		||||
 | 
			
		||||
@@ -244,10 +225,7 @@ urchin_go() {
 | 
			
		||||
    echo "Done, took $elapsed $(plural second $elapsed)."
 | 
			
		||||
    printf '%s\n' "$passed $(plural test "$passed") passed."
 | 
			
		||||
    printf '%s\n' "$skipped $(plural test "$skipped") skipped."
 | 
			
		||||
 | 
			
		||||
    # If tests failed, print the message in red, otherwise in green.
 | 
			
		||||
    [ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m'
 | 
			
		||||
 | 
			
		||||
    [ $failed -gt 0 ] && printf '\033[31m' || printf '\033[32m' # If tests failed, print the message in red, otherwise in green.
 | 
			
		||||
    printf '%s\n' "$failed $(plural test "$failed") failed."
 | 
			
		||||
    printf '\033[m'
 | 
			
		||||
  fi
 | 
			
		||||
@@ -279,11 +257,8 @@ do
 | 
			
		||||
        -s)
 | 
			
		||||
          shift
 | 
			
		||||
          shell_for_sh_tests=$1
 | 
			
		||||
          which "$shell_for_sh_tests" > /dev/null || {
 | 
			
		||||
            echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2
 | 
			
		||||
            urchin_help >&2
 | 
			
		||||
            exit 2
 | 
			
		||||
          } ;;
 | 
			
		||||
          which "$shell_for_sh_tests" >/dev/null || { echo "Cannot find specified shell: '$shell_for_sh_tests'" >&2; urchin_help >&2; exit 2; }
 | 
			
		||||
          ;;
 | 
			
		||||
        -t) tap_format=true;;
 | 
			
		||||
        -h|--help) urchin_help
 | 
			
		||||
          exit 0;;
 | 
			
		||||
@@ -309,7 +284,7 @@ logfile=$(fullpath "$1")/.urchin.log
 | 
			
		||||
stdout_file=$(fullpath "$1")/.urchin_stdout
 | 
			
		||||
 | 
			
		||||
# Run or present the Molly guard.
 | 
			
		||||
if fullpath "$1" | grep -Fi 'test' > /dev/null || $force
 | 
			
		||||
if basename "$(fullpath "$1")" | grep -Fi 'test' > /dev/null || $force
 | 
			
		||||
  then
 | 
			
		||||
  urchin_go "$1" "$shell_for_sh_tests"
 | 
			
		||||
else
 | 
			
		||||
 
 | 
			
		||||
										
											Binary file not shown.
										
									
								
							| 
		 Before Width: | Height: | Size: 99 KiB  | 
										
											Binary file not shown.
										
									
								
							| 
		 Before Width: | Height: | Size: 115 KiB  | 
		Reference in New Issue
	
	Block a user