urchin/TODO

Things I want
=============

Wider testing
--------------
Test in other environments

* Specify a few different ones with Nix.
* Some sort of BSD
* Windows

Packaging
------------
Package for package managers.

* I want NixOS, of course.
* Debian is probably the big one.

Other interesting package managers

* Update the npm package
* Homebrew (for Mac)

Windows
----------
Try running Urchin in Windows somehow. Interpreters include

* CygWin (https://www.cygwin.com/)
  * https://cygwin.com/setup-x86.exe
* MSYS (http://mingw.org/wiki/msys)
* GNU on Windows (https://github.com/bmatzelle/gow/wiki)
* Git for Windows (https://git-scm.com/download/win)
  * https://github.com/git-for-windows/git/releases/download/v2.7.2.windows.1/Git-2.7.2-32-bit.exe
* win-bash (http://win-bash.sourceforge.net/)


shall
----------
Add shall to my NYC*BUG talk.

    #!/usr/bin/env shall
    echo This runs in several shells.

Linters
-----------
List some shell linters somewhere.

* ShellCheck
* checkbashisms


Rename to something other than "test"?
----------
Maybe wait until I have a use for this.

More sort alternatives
-----------
awk
  https://stackoverflow.com/questions/20250937/sorting-lines-in-a-file-alphabetically-using-awk-and-or-sed
bash
  https://stackoverflow.com/questions/7442417/how-to-sort-an-array-in-bash

Alternatives
--------------
JSON.sh test suite

Running in multiple environments
-----------------------------------
Setup for other environments includes the following.

* Installing packages
* `touch .zshrc`
* Copy urchin and tests


Fixtures
------------
I want to change the way that fixtures are done.

Instead of using setup, teardown, &c., use ordinary programs from within
your tests. For example.

    # tests/.fixtures/tmp-dir
    tmp=$(mktemp -d)
    cd $tmp
    @$
    code=$?
    cd /
    rm -Rf $tmp
    exit $code

    # tests/blah
    ../.fixtures/tmp-dir 'blah blah blah'

It's best if I can wrap a bunch of commands in braces or paratheses
rather than just one command. Is there a nice way to do that?

Once I have this new way, I guess I might as well keep the old way.
I think the setup, teardown thing can be easier if you only have simple
fixtures. And since I'm going to keep it, I'm going to add another one.

* setup_dir runs once for the present directory.
* setup_children runs once for each child.
* setup_file runs once for each file descendent.

The present `setup` is renamed to `setup_children`, and the new
`setup_file` runs on each file (not directory) that is a child,
grandchild, great-grandchild, and so on.

Dependency checking
----------------------
You might want to skip tests based on dependencies. Currently you can
conditionally skip tests one at a time by exiting with code 3. I want to
be able to skip an entire directory.

So we add a new magic file called `dep`. If it exists, it is run before
everything else in the directory.

* If it exits with code 0, tests continue as if dep did not exist.
* If it exits with code 3, all tests in the directory are marked as
  skipped.
* If it exits with code 1, all tests in the directory are marked as
  failed. To make the implementation easier, I'll probably treat the
  directory as a single test in this case.

A note on magic files
-------------------------
It is nice to have access to things like setup and dep (magic files)
once in a while, but you need to be doing rather substantial testing
before they make your test suite simpler; the documentation should
strongly recommend writing your tests without magic files and then
refactoring and only then considering moving things to magic files.

Remote testing
----------------
In order to test Urchin across multiple operating systems, I have
already added tests in Urchin's test suite that run Urchin tests in
remote servers. I would like to move this to Urchin itself so that
Urchin can test other things on remote servers.

Urchin's output presently looks like this.

    Cycling with the following shells: sh bash dash mksh zsh 
    Running tests at 2016-04-07T12:33:49

    Flags/
    > --timeout output
    . bash (0 seconds)
    . dash (0 seconds)
    . mksh (0 seconds)
    . sh (0 seconds)
    . zsh (0 seconds)

    Done, took 1 second.
    5 tests passed.
    0 tests skipped.
    0 tests failed.

After the change, the output should look like this.

    Cycling with the following shells: sh dash mksh
    Running tests at 2016-04-07T12:33:49

    Flags/
    > --timeout output
    . dash on localhost (0 seconds)
    . dash on localhost:8080 (0 seconds)
    . dash on tlevine@hpux.polarhome.com (0 seconds)
    . mksh on localhost (0 seconds)
    . mksh on tlevine@hpux.polarhome.com (0 seconds)
    . sh on localhost (0 seconds)
    . sh on localhost:8080 (0 seconds)
    . sh on tlevine@hpux.polarhome.com (0 seconds)

    Done, took 1 second.
    8 tests passed.
    0 tests skipped.
    0 tests failed.

This is just how the output should look; the tests run in whatever order
makes sense.

Bugs
-------

Both md5sum and md5 should be supported.

Trouble logging in to hpux, irix, miros, netbsd, tru64, qnx, ....

    $ rsync -e 'ssh -p 785' urchin tlevine@hpux.polarhome.com:.blah
    HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
    Welcome to HPUX/PA... member of polarhome.com realm

    bash: rsync: command not found
    rsync: connection unexpectedly closed (0 bytes received so far) [sender]
    rsync error: remote command not found (code 127) at io.c(226)
    [sender=3.1.1]


OpenIndiana grep does not support -q

I get `/urchin: syntax error at line 84: \`}' unexpected` on
unixware and solaris.

mktemp

    > tlevine@hpux.polarhome.com -p 785
    F sh (8 seconds)
      | 
      | HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
      | Welcome to HPUX/PA... member of polarhome.com realm
      | 
      | 
      | HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
      | Welcome to HPUX/PA... member of polarhome.com realm
      | 
      | mktemp: option requires an argument -- d
      | ./urchin[96]: /tmp/tlevinea21441/log: Cannot create the specified file.

date

    tlevine@hpux64$ ./urchin tests/ -n -vv
    date: bad format character - s

So I need a portable seconds-from epoch

I also need to handle when no arguments are passed to urchin.

Exit code is wrong for which on HP-UX

## `$(...)`
Solaris doesn't support `$(...)`; you need `\`...\`` instead.

    tlevine@solaris$ ./urchin --run-in-series tests/Errors/
    ./urchin: syntax error at line 84: `tmp=$' unexpected

I use this a lot.

    $ grep -c '\$(' urchin 
    52

Darn




Update tests to support

* md5
* rsync
* mktemp
* epoch
* Report cycling by default
* New format for reporting cycling




Support systems without rsync


BSD mktemp

    | NetBSD 6.1.3 
    | Welcome to NetBSD ...member of polarhome.com realm
    | 
    | Usage: mktemp [-dqu] [-p <tmpdir>] {-t prefix | template ...}
    | mkdir: : No such file or directory
    | ./urchin: cannot create /log: permission denied


NetBSD

    md5: unknown option -- q
    usage: cksum [-n] [-a algorithm [-ptx] [-s string]] [-o 1|2]
                 [file ... | -c [-w] [sumfile]]




Things I can use to make things better
------------------------
${x##*blah}
$IFS and set --
Redirection, especiall <<-
Maybe fifo
for x in "$@"
until
readonly
getopts

Variable assignments specified with special built-in utilities remain in
effect after the built-in completes; this shall not be the case with a
regular built-in or other utility.