urchin/TODO
2016-04-07 12:38:23 +00:00

281 lines
7.0 KiB
Plaintext

Things I want
=============
Wider testing
--------------
Test in other environments
* Specify a few different ones with Nix.
* Some sort of BSD
* Windows
Packaging
------------
Package for package managers.
* I want NixOS, of course.
* Debian is probably the big one.
Other interesting package managers
* Update the npm package
* Homebrew (for Mac)
Windows
----------
Try running Urchin in Windows somehow. Interpreters include
* CygWin (https://www.cygwin.com/)
* https://cygwin.com/setup-x86.exe
* MSYS (http://mingw.org/wiki/msys)
* GNU on Windows (https://github.com/bmatzelle/gow/wiki)
* Git for Windows (https://git-scm.com/download/win)
* https://github.com/git-for-windows/git/releases/download/v2.7.2.windows.1/Git-2.7.2-32-bit.exe
* win-bash (http://win-bash.sourceforge.net/)
shall
----------
Add shall to my NYC*BUG talk.
#!/usr/bin/env shall
echo This runs in several shells.
Linters
-----------
List some shell linters somewhere.
* ShellCheck
* checkbashisms
Rename to something other than "test"?
----------
Maybe wait until I have a use for this.
More sort alternatives
-----------
awk
https://stackoverflow.com/questions/20250937/sorting-lines-in-a-file-alphabetically-using-awk-and-or-sed
bash
https://stackoverflow.com/questions/7442417/how-to-sort-an-array-in-bash
Alternatives
--------------
JSON.sh test suite
Running in multiple environments
-----------------------------------
Setup for other environments includes the following.
* Installing packages
* `touch .zshrc`
* Copy urchin and tests
Fixtures
------------
I want to change the way that fixtures are done.
Instead of using setup, teardown, &c., use ordinary programs from within
your tests. For example.
# tests/.fixtures/tmp-dir
tmp=$(mktemp -d)
cd $tmp
@$
code=$?
cd /
rm -Rf $tmp
exit $code
# tests/blah
../.fixtures/tmp-dir 'blah blah blah'
It's best if I can wrap a bunch of commands in braces or paratheses
rather than just one command. Is there a nice way to do that?
Once I have this new way, I guess I might as well keep the old way.
I think the setup, teardown thing can be easier if you only have simple
fixtures. And since I'm going to keep it, I'm going to add another one.
* setup_dir runs once for the present directory.
* setup_children runs once for each child.
* setup_file runs once for each file descendent.
The present `setup` is renamed to `setup_children`, and the new
`setup_file` runs on each file (not directory) that is a child,
grandchild, great-grandchild, and so on.
Dependency checking
----------------------
You might want to skip tests based on dependencies. Currently you can
conditionally skip tests one at a time by exiting with code 3. I want to
be able to skip an entire directory.
So we add a new magic file called `dep`. If it exists, it is run before
everything else in the directory.
* If it exits with code 0, tests continue as if dep did not exist.
* If it exits with code 3, all tests in the directory are marked as
skipped.
* If it exits with code 1, all tests in the directory are marked as
failed. To make the implementation easier, I'll probably treat the
directory as a single test in this case.
A note on magic files
-------------------------
It is nice to have access to things like setup and dep (magic files)
once in a while, but you need to be doing rather substantial testing
before they make your test suite simpler; the documentation should
strongly recommend writing your tests without magic files and then
refactoring and only then considering moving things to magic files.
Remote testing
----------------
In order to test Urchin across multiple operating systems, I have
already added tests in Urchin's test suite that run Urchin tests in
remote servers. I would like to move this to Urchin itself so that
Urchin can test other things on remote servers.
Urchin's output presently looks like this.
Cycling with the following shells: sh bash dash mksh zsh
Running tests at 2016-04-07T12:33:49
Flags/
> --timeout output
. bash (0 seconds)
. dash (0 seconds)
. mksh (0 seconds)
. sh (0 seconds)
. zsh (0 seconds)
Done, took 1 second.
5 tests passed.
0 tests skipped.
0 tests failed.
After the change, the output should look like this.
Cycling with the following shells: sh dash mksh
Running tests at 2016-04-07T12:33:49
Flags/
> --timeout output
. dash on localhost (0 seconds)
. dash on localhost:8080 (0 seconds)
. dash on tlevine@hpux.polarhome.com (0 seconds)
. mksh on localhost (0 seconds)
. mksh on tlevine@hpux.polarhome.com (0 seconds)
. sh on localhost (0 seconds)
. sh on localhost:8080 (0 seconds)
. sh on tlevine@hpux.polarhome.com (0 seconds)
Done, took 1 second.
8 tests passed.
0 tests skipped.
0 tests failed.
This is just how the output should look; the tests run in whatever order
makes sense.
Bugs
-------
Both md5sum and md5 should be supported.
Trouble logging in to hpux, irix, miros, netbsd, tru64, qnx, ....
$ rsync -e 'ssh -p 785' urchin tlevine@hpux.polarhome.com:.blah
HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
Welcome to HPUX/PA... member of polarhome.com realm
bash: rsync: command not found
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: remote command not found (code 127) at io.c(226)
[sender=3.1.1]
OpenIndiana grep does not support -q
I get `/urchin: syntax error at line 84: \`}' unexpected` on
unixware and solaris.
mktemp
> tlevine@hpux.polarhome.com -p 785
F sh (8 seconds)
|
| HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
| Welcome to HPUX/PA... member of polarhome.com realm
|
|
| HP-UX hpux.polarhome.com B.11.11 U 9000/785 (ta)
| Welcome to HPUX/PA... member of polarhome.com realm
|
| mktemp: option requires an argument -- d
| ./urchin[96]: /tmp/tlevinea21441/log: Cannot create the specified file.
date
tlevine@hpux64$ ./urchin tests/ -n -vv
date: bad format character - s
So I need a portable seconds-from epoch
I also need to handle when no arguments are passed to urchin.
Exit code is wrong for which on HP-UX
## `$(...)`
Solaris doesn't support `$(...)`; you need `\`...\`` instead.
tlevine@solaris$ ./urchin --run-in-series tests/Errors/
./urchin: syntax error at line 84: `tmp=$' unexpected
I use this a lot.
$ grep -c '\$(' urchin
52
Darn
Update tests to support
* md5
* rsync
* mktemp
* epoch
* Report cycling by default
* New format for reporting cycling
Support systems without rsync
BSD mktemp
| NetBSD 6.1.3
| Welcome to NetBSD ...member of polarhome.com realm
|
| Usage: mktemp [-dqu] [-p <tmpdir>] {-t prefix | template ...}
| mkdir: : No such file or directory
| ./urchin: cannot create /log: permission denied
NetBSD
md5: unknown option -- q
usage: cksum [-n] [-a algorithm [-ptx] [-s string]] [-o 1|2]
[file ... | -c [-w] [sumfile]]