Things I want ============= Wider testing -------------- Test in other environments * Specify a few different ones with Nix. * Some sort of BSD * Windows Packaging ------------ Package for package managers. * I want NixOS, of course. * Debian is probably the big one. Other interesting package managers * Update the npm package * Homebrew (for Mac) Windows ---------- Try running Urchin in Windows somehow. Interpreters include * CygWin (https://www.cygwin.com/) * https://cygwin.com/setup-x86.exe * MSYS (http://mingw.org/wiki/msys) * GNU on Windows (https://github.com/bmatzelle/gow/wiki) * Git for Windows (https://git-scm.com/download/win) * https://github.com/git-for-windows/git/releases/download/v2.7.2.windows.1/Git-2.7.2-32-bit.exe * win-bash (http://win-bash.sourceforge.net/) shall ---------- Add shall to my NYC*BUG talk. #!/usr/bin/env shall echo This runs in several shells. Linters ----------- List some shell linters somewhere. * ShellCheck * checkbashisms Rename to something other than "test"? ---------- Maybe wait until I have a use for this. More sort alternatives ----------- awk https://stackoverflow.com/questions/20250937/sorting-lines-in-a-file-alphabetically-using-awk-and-or-sed bash https://stackoverflow.com/questions/7442417/how-to-sort-an-array-in-bash Alternatives -------------- JSON.sh test suite Running in multiple environments ----------------------------------- Setup for other environments includes the following. * Installing packages * `touch .zshrc` * Copy urchin and tests Fixtures ------------ I want to change the way that fixtures are done. Instead of using setup, teardown, &c., use ordinary programs from within your tests. For example. # tests/.fixtures/tmp-dir tmp=$(mktemp -d) cd $tmp @$ code=$? cd / rm -Rf $tmp exit $code # tests/blah ../.fixtures/tmp-dir 'blah blah blah' It's best if I can wrap a bunch of commands in braces or paratheses rather than just one command. Is there a nice way to do that? Once I have this new way, I guess I might as well keep the old way. I think the setup, teardown thing can be easier if you only have simple fixtures. And since I'm going to keep it, I'm going to add another one. * setup_dir runs once for the present directory. * setup_children runs once for each child. * setup_file runs once for each file descendent. The present `setup` is renamed to `setup_children`, and the new `setup_file` runs on each file (not directory) that is a child, grandchild, great-grandchild, and so on. Dependency checking ---------------------- You might want to skip tests based on dependencies. Currently you can conditionally skip tests one at a time by exiting with code 3. I want to be able to skip an entire directory. So we add a new magic file called `dep`. If it exists, it is run before everything else in the directory. * If it exits with code 0, tests continue as if dep did not exist. * If it exits with code 3, all tests in the directory are marked as skipped. * If it exits with code 1, all tests in the directory are marked as failed. To make the implementation easier, I'll probably treat the directory as a single test in this case. A note on magic files ------------------------- It is nice to have access to things like setup and dep (magic files) once in a while, but you need to be doing rather substantial testing before they make your test suite simpler; the documentation should strongly recommend writing your tests without magic files and then refactoring and only then considering moving things to magic files. Remote testing ---------------- In order to test Urchin across multiple operating systems, I have already added tests in Urchin's test suite that run Urchin tests in remote servers. I would like to move this to Urchin itself so that Urchin can test other things on remote servers. Urchin's output presently looks like this. Cycling with the following shells: sh bash dash mksh zsh Running tests at 2016-04-07T12:33:49 Flags/ > --timeout output . bash (0 seconds) . dash (0 seconds) . mksh (0 seconds) . sh (0 seconds) . zsh (0 seconds) Done, took 1 second. 5 tests passed. 0 tests skipped. 0 tests failed. After the change, the output should look like this. Cycling with the following shells: sh dash mksh Running tests at 2016-04-07T12:33:49 Flags/ > --timeout output . dash on localhost (0 seconds) . dash on localhost:8080 (0 seconds) . dash on tlevine@hpux.polarhome.com (0 seconds) . mksh on localhost (0 seconds) . mksh on tlevine@hpux.polarhome.com (0 seconds) . sh on localhost (0 seconds) . sh on localhost:8080 (0 seconds) . sh on tlevine@hpux.polarhome.com (0 seconds) Done, took 1 second. 8 tests passed. 0 tests skipped. 0 tests failed. This is just how the output should look; the tests run in whatever order makes sense. Bugs ------- Both md5sum and md5 should be supported. Trouble logging in to hpux, irix, miros, netbsd, tru64, qnx, .... $ rsync -e 'ssh -p 785' urchin tlevine@hpux.polarhome.com:.blah HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta) Welcome to HPUX/PA... member of polarhome.com realm bash: rsync: command not found rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: remote command not found (code 127) at io.c(226) [sender=3.1.1] OpenIndiana grep does not support -q I get `/urchin: syntax error at line 84: \`}' unexpected` on unixware and solaris. mktemp > tlevine@hpux.polarhome.com -p 785 F sh (8 seconds) | | HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta) | Welcome to HPUX/PA... member of polarhome.com realm | | | HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta) | Welcome to HPUX/PA... member of polarhome.com realm | | mktemp: option requires an argument -- d | ./urchin[96]: /tmp/tlevinea21441/log: Cannot create the specified file. date tlevine@hpux64$ ./urchin tests/ -n -vv date: bad format character - s So I need a portable seconds-from epoch I also need to handle when no arguments are passed to urchin. Exit code is wrong for which on HP-UX ## `$(...)` Solaris doesn't support `$(...)`; you need `\`...\`` instead. tlevine@solaris$ ./urchin --run-in-series tests/Errors/ ./urchin: syntax error at line 84: `tmp=$' unexpected I use this a lot. $ grep -c '\$(' urchin 52 Darn Update tests to support * md5 * rsync * mktemp * epoch * Report cycling by default * New format for reporting cycling Support systems without rsync BSD mktemp | NetBSD 6.1.3 | Welcome to NetBSD ...member of polarhome.com realm | | Usage: mktemp [-dqu] [-p ] {-t prefix | template ...} | mkdir: : No such file or directory | ./urchin: cannot create /log: permission denied NetBSD md5: unknown option -- q usage: cksum [-n] [-a algorithm [-ptx] [-s string]] [-o 1|2] [file ... | -c [-w] [sumfile]] Things I can use to make things better ------------------------ ${x##*blah} $IFS and set -- Redirection, especiall <<- Maybe fifo for x in "$@" until readonly getopts Variable assignments specified with special built-in utilities remain in effect after the built-in completes; this shall not be the case with a regular built-in or other utility.