295 lines
7.3 KiB
Plaintext
295 lines
7.3 KiB
Plaintext
Things I want
|
|
=============
|
|
|
|
Wider testing
|
|
--------------
|
|
Test in other environments
|
|
|
|
* Specify a few different ones with Nix.
|
|
* Some sort of BSD
|
|
* Windows
|
|
|
|
Packaging
|
|
------------
|
|
Package for package managers.
|
|
|
|
* I want NixOS, of course.
|
|
* Debian is probably the big one.
|
|
|
|
Other interesting package managers
|
|
|
|
* Update the npm package
|
|
* Homebrew (for Mac)
|
|
|
|
Windows
|
|
----------
|
|
Try running Urchin in Windows somehow. Interpreters include
|
|
|
|
* CygWin (https://www.cygwin.com/)
|
|
* https://cygwin.com/setup-x86.exe
|
|
* MSYS (http://mingw.org/wiki/msys)
|
|
* GNU on Windows (https://github.com/bmatzelle/gow/wiki)
|
|
* Git for Windows (https://git-scm.com/download/win)
|
|
* https://github.com/git-for-windows/git/releases/download/v2.7.2.windows.1/Git-2.7.2-32-bit.exe
|
|
* win-bash (http://win-bash.sourceforge.net/)
|
|
|
|
|
|
shall
|
|
----------
|
|
Add shall to my NYC*BUG talk.
|
|
|
|
#!/usr/bin/env shall
|
|
echo This runs in several shells.
|
|
|
|
Linters
|
|
-----------
|
|
List some shell linters somewhere.
|
|
|
|
* ShellCheck
|
|
* checkbashisms
|
|
|
|
|
|
Rename to something other than "test"?
|
|
----------
|
|
Maybe wait until I have a use for this.
|
|
|
|
More sort alternatives
|
|
-----------
|
|
awk
|
|
https://stackoverflow.com/questions/20250937/sorting-lines-in-a-file-alphabetically-using-awk-and-or-sed
|
|
bash
|
|
https://stackoverflow.com/questions/7442417/how-to-sort-an-array-in-bash
|
|
|
|
Alternatives
|
|
--------------
|
|
JSON.sh test suite
|
|
|
|
Running in multiple environments
|
|
-----------------------------------
|
|
Setup for other environments includes the following.
|
|
|
|
* Installing packages
|
|
* `touch .zshrc`
|
|
* Copy urchin and tests
|
|
|
|
|
|
Fixtures
|
|
------------
|
|
I want to change the way that fixtures are done.
|
|
|
|
Instead of using setup, teardown, &c., use ordinary programs from within
|
|
your tests. For example.
|
|
|
|
# tests/.fixtures/tmp-dir
|
|
tmp=$(mktemp -d)
|
|
cd $tmp
|
|
@$
|
|
code=$?
|
|
cd /
|
|
rm -Rf $tmp
|
|
exit $code
|
|
|
|
# tests/blah
|
|
../.fixtures/tmp-dir 'blah blah blah'
|
|
|
|
It's best if I can wrap a bunch of commands in braces or paratheses
|
|
rather than just one command. Is there a nice way to do that?
|
|
|
|
Once I have this new way, I guess I might as well keep the old way.
|
|
I think the setup, teardown thing can be easier if you only have simple
|
|
fixtures. And since I'm going to keep it, I'm going to add another one.
|
|
|
|
* setup_dir runs once for the present directory.
|
|
* setup_children runs once for each child.
|
|
* setup_file runs once for each file descendent.
|
|
|
|
The present `setup` is renamed to `setup_children`, and the new
|
|
`setup_file` runs on each file (not directory) that is a child,
|
|
grandchild, great-grandchild, and so on.
|
|
|
|
Dependency checking
|
|
----------------------
|
|
You might want to skip tests based on dependencies. Currently you can
|
|
conditionally skip tests one at a time by exiting with code 3. I want to
|
|
be able to skip an entire directory.
|
|
|
|
So we add a new magic file called `dep`. If it exists, it is run before
|
|
everything else in the directory.
|
|
|
|
* If it exits with code 0, tests continue as if dep did not exist.
|
|
* If it exits with code 3, all tests in the directory are marked as
|
|
skipped.
|
|
* If it exits with code 1, all tests in the directory are marked as
|
|
failed. To make the implementation easier, I'll probably treat the
|
|
directory as a single test in this case.
|
|
|
|
A note on magic files
|
|
-------------------------
|
|
It is nice to have access to things like setup and dep (magic files)
|
|
once in a while, but you need to be doing rather substantial testing
|
|
before they make your test suite simpler; the documentation should
|
|
strongly recommend writing your tests without magic files and then
|
|
refactoring and only then considering moving things to magic files.
|
|
|
|
Remote testing
|
|
----------------
|
|
In order to test Urchin across multiple operating systems, I have
|
|
already added tests in Urchin's test suite that run Urchin tests in
|
|
remote servers. I would like to move this to Urchin itself so that
|
|
Urchin can test other things on remote servers.
|
|
|
|
Urchin's output presently looks like this.
|
|
|
|
Cycling with the following shells: sh bash dash mksh zsh
|
|
Running tests at 2016-04-07T12:33:49
|
|
|
|
Flags/
|
|
> --timeout output
|
|
. bash (0 seconds)
|
|
. dash (0 seconds)
|
|
. mksh (0 seconds)
|
|
. sh (0 seconds)
|
|
. zsh (0 seconds)
|
|
|
|
Done, took 1 second.
|
|
5 tests passed.
|
|
0 tests skipped.
|
|
0 tests failed.
|
|
|
|
After the change, the output should look like this.
|
|
|
|
Cycling with the following shells: sh dash mksh
|
|
Running tests at 2016-04-07T12:33:49
|
|
|
|
Flags/
|
|
> --timeout output
|
|
. dash on localhost (0 seconds)
|
|
. dash on localhost:8080 (0 seconds)
|
|
. dash on tlevine@hpux.polarhome.com (0 seconds)
|
|
. mksh on localhost (0 seconds)
|
|
. mksh on tlevine@hpux.polarhome.com (0 seconds)
|
|
. sh on localhost (0 seconds)
|
|
. sh on localhost:8080 (0 seconds)
|
|
. sh on tlevine@hpux.polarhome.com (0 seconds)
|
|
|
|
Done, took 1 second.
|
|
8 tests passed.
|
|
0 tests skipped.
|
|
0 tests failed.
|
|
|
|
This is just how the output should look; the tests run in whatever order
|
|
makes sense.
|
|
|
|
Bugs
|
|
-------
|
|
|
|
Both md5sum and md5 should be supported.
|
|
|
|
Trouble logging in to hpux, irix, miros, netbsd, tru64, qnx, ....
|
|
|
|
$ rsync -e 'ssh -p 785' urchin tlevine@hpux.polarhome.com:.blah
|
|
HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
|
|
Welcome to HPUX/PA... member of polarhome.com realm
|
|
|
|
bash: rsync: command not found
|
|
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
|
|
rsync error: remote command not found (code 127) at io.c(226)
|
|
[sender=3.1.1]
|
|
|
|
|
|
OpenIndiana grep does not support -q
|
|
|
|
I get `/urchin: syntax error at line 84: \`}' unexpected` on
|
|
unixware and solaris.
|
|
|
|
mktemp
|
|
|
|
> tlevine@hpux.polarhome.com -p 785
|
|
F sh (8 seconds)
|
|
|
|
|
| HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
|
|
| Welcome to HPUX/PA... member of polarhome.com realm
|
|
|
|
|
|
|
|
| HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
|
|
| Welcome to HPUX/PA... member of polarhome.com realm
|
|
|
|
|
| mktemp: option requires an argument -- d
|
|
| ./urchin[96]: /tmp/tlevinea21441/log: Cannot create the specified file.
|
|
|
|
date
|
|
|
|
tlevine@hpux64$ ./urchin tests/ -n -vv
|
|
date: bad format character - s
|
|
|
|
So I need a portable seconds-from epoch
|
|
|
|
I also need to handle when no arguments are passed to urchin.
|
|
|
|
Exit code is wrong for which on HP-UX
|
|
|
|
## `$(...)`
|
|
Solaris doesn't support `$(...)`; you need `\`...\`` instead.
|
|
|
|
tlevine@solaris$ ./urchin --run-in-series tests/Errors/
|
|
./urchin: syntax error at line 84: `tmp=$' unexpected
|
|
|
|
I use this a lot.
|
|
|
|
$ grep -c '\$(' urchin
|
|
52
|
|
|
|
Darn
|
|
|
|
|
|
|
|
|
|
Update tests to support
|
|
|
|
* md5
|
|
* rsync
|
|
* mktemp
|
|
* epoch
|
|
* Report cycling by default
|
|
* New format for reporting cycling
|
|
|
|
|
|
|
|
|
|
Support systems without rsync
|
|
|
|
|
|
BSD mktemp
|
|
|
|
| NetBSD 6.1.3
|
|
| Welcome to NetBSD ...member of polarhome.com realm
|
|
|
|
|
| Usage: mktemp [-dqu] [-p <tmpdir>] {-t prefix | template ...}
|
|
| mkdir: : No such file or directory
|
|
| ./urchin: cannot create /log: permission denied
|
|
|
|
|
|
NetBSD
|
|
|
|
md5: unknown option -- q
|
|
usage: cksum [-n] [-a algorithm [-ptx] [-s string]] [-o 1|2]
|
|
[file ... | -c [-w] [sumfile]]
|
|
|
|
|
|
|
|
|
|
Things I can use to make things better
|
|
------------------------
|
|
${x##*blah}
|
|
$IFS and set --
|
|
Redirection, especiall <<-
|
|
Maybe fifo
|
|
for x in "$@"
|
|
until
|
|
readonly
|
|
getopts
|
|
|
|
Variable assignments specified with special built-in utilities remain in
|
|
effect after the built-in completes; this shall not be the case with a
|
|
regular built-in or other utility.
|