Things I want ============= Test speed ------------- Make tests run faster. https://github.com/bike-barn/hermit/issues/62 First, easier thing is probably to run tests in parallel. Second, also easier thing is to tell people to save things to RAM rather than disk whenever they can. Third, harder thing is to put the test suite in RAM automatically. Maybe the whole test directory, which includes fixtures, gets copied to a tmpfs if one exists. Hmm or maybe there's a compromise: Tell people to mount /tmp as a tmpfs so that temp files are fast. Maybe allow people to set some other directory as the temporary file place, in case they want a different tmpfs location. In order to run things in parallel, we have to change how we do the stdout_file. I think it's easiest to create separate files for each test and to save them in testroot/.urchin/stdout/$filename. The test root would be defined as the closest ancestor containing a .urchin directory. Options ------------- I want long options. For example, there's presently -f and -e. I want to make them -f|--force and -e|--exit. Environment variables ------------- Do something to make it easier to debug environment variables, because that is often confusing. https://github.com/creationix/nvm/issues/719 https://github.com/creationix/nvm/issues/589 Documenting that people should run "env" when their tests fail might be good enough. Licensing and copyright ------------------------ * Reference all owners and years in the Copyright file * Consider copyleft licenses * Add license notices to other files if necessary Packaging ------------ Package for package managers. * I want NixOS, of course. * Debian is probably the big one. Other interesting package managers * Update the npm package * Homebrew (for Mac) Windows ---------- Try running Urchin in Windows somehow. Interpreters include * CygWin (https://www.cygwin.com/) * MSYS (http://mingw.org/wiki/msys) * GNU on Windows (https://github.com/bmatzelle/gow/wiki) * Git for Windows (https://git-scm.com/download/win) * win-bash (http://win-bash.sourceforge.net/) Consider copyleft licenses ---------- ScraperWiki owns the original version of Urchin (Thomas Levine did the early work as part of his work for ScraperWiki.) and originally licensed it under an MIT-style license. Other people made changes after this original ScraperWiki version. As of January 2016, they are just Thomas Levine (when he wasn't working for ScraperWiki) and Michael Klement. The original license was MIT just because that's what ScraperWiki put on everything. Should we change the license? The MIT-style license grants pretty much all rights. It says that you need to attribute when you redistribute source code, but you don't necessarily have to redistribute source code. A copyleft license adds the restriction that modified versions of the code need to be licensed under the same license. GNU licenses in particular require that source code be released if non-source versions are released, and the different GNU licenses differ in what how the non-source version is defined. (The original, GPL, discusses compiled binaries.) Copyleft doesn't mean anything specific for commercial use. MIT-licensed code can be modified and then licensed as GPL, because MIT license allows that, but GPL code can't be modified as MIT, because MIT doesn't allow that. And if we get all of the authors to agree on it, we can always add whatever crazy license we want, regardless of what we have already. The distinction between MIT-style and GNU-something might matter quite little in the case of Urchin. 1. Urchin is written in an interpreted language (shell), so it might be hard to distribute usefully without providing the source code. 2. Urchin just runs tests; it doesn't get compiled with the rest of the code (also because it's in shell). Thus, I think a GPL license on Urchin wouldn't infect the code being tested. This is as far as I have gotten with contemplating license changes. For now we're sticking with the original MIT-style license, but it's easy to change licenses later. Nagios plugins ----------------- It would be cool to run Nagios plugins with Urchin. This is already possible, actually, but it might be worth giving some special thought to it. https://nagios-plugins.org/doc/guidelines.html Source setup and teardown -------------------- If setup and teardown are sourced instead of executed, maybe we can more cleanly create and teardown temporary files. ( . ./setup ./$thetestfile . ./teardown ) On the other hand, this could just be sourced explicitly in the test file, without the special setup and teardown feature. Run on a file ---------------- Presently you can run urchin only on a directory. It would be neat if you could run it on a file as well. This occurred to me when I wanted to run urchin test/fast/Unit\ tests/nvm_ls_current on the nvm tests. I wound up running this instead. urchin test/fast/Unit\ tests/ | grep nvm_ls_current The Molly guard would be assessed, and the corresponding setup, setup_dir, teardown, and teardown_dir files would be run in the appropriate order. In order to know how far up the tree to evaluate the setup, &c. files, I think it would make sense to require that a ".urchin" file be placed in the root of the tests. Urchin would keep going up until it sees this file, and it would evaluate the appropriate setup, &c. files from there down to the particular test file of interest. We would also use this for testing directtories more correctly. Running automated tasks ------------------------- Urchin might be appropriate for if you have lots of tasks that you want to run periodically; add an urchin call to your crontab, and call all of your other tasks with urchin. Here are some features that might make urchin better for this sort of thing. * Time how long each test/job takes * Optionally kill tests/jobs after a specific timeout threshold * Send output of different tests/jobs to different files for each file descriptor (STDOUT, STDERR)