more history

This commit is contained in:
Thomas Levine 2016-02-29 03:25:53 +00:00
parent 7386fb7229
commit c47cdf6bc8
2 changed files with 56 additions and 50 deletions

58
HISTORY
View File

@ -32,8 +32,7 @@ found a `.urchin` file.
In either of these cases, Urchin uses the user-specified directory as In either of these cases, Urchin uses the user-specified directory as
the root; this is how Urchin `0.0.*` worked. the root; this is how Urchin `0.0.*` worked.
Molly guard ### Molly guard
------------
The Molly-guard works differently because it now considers the test suite The Molly-guard works differently because it now considers the test suite
root directory. The point of the Molly-guard originally was to protect root directory. The point of the Molly-guard originally was to protect
you from things like this. you from things like this.
@ -56,6 +55,15 @@ As before, you can override the Molly guard with `-f`.
urchin -f build-scripts urchin -f build-scripts
### Consolidation of temporary files in /tmp
All of Urchin's temporary files are now stored in /tmp. Urchin previously
created `.urchin.log` files alongside the tests, which led to such
inconveniences as accidentally commiting them to version control repositories.
This also means that Urchin will keep all of its temporary files in RAM
if you mount a tmpfs on /tmp. On large test suites you may find the tmpfs
to be slightly faster than slower storage media like solid-state drives.
### Skipping of tests ### Skipping of tests
Previously, tests were run if they were executable and were otherwise marked Previously, tests were run if they were executable and were otherwise marked
as skipped. Now, an executable script can indicate that it is skipped by as skipped. Now, an executable script can indicate that it is skipped by
@ -74,6 +82,52 @@ would the appropriate status code if these tests were Nagios plugins, as the
concept of skipping a test is similar to the Nagios concept of unknown service concept of skipping a test is similar to the Nagios concept of unknown service
status (https://nagios-plugins.org/doc/guidelines.html#AEN78). status (https://nagios-plugins.org/doc/guidelines.html#AEN78).
### Parallel test execution
Tests now run in parallel when possible.
Parallel processes come about in two situations when parallel execution is
turned on.
1. All files and immediate subdirectories of one particular directory
are run in parallel. This happens recursively; during the execution
of each particular subdirectory, that subdirectory's children are
also run in parallel.
2. When cycling of shells is enabled, execution of a particular file in
different shells are run parellel.
Parallel processing and shell cycling are both enabled by default.
### Options
Long options are now available for all command line flags.
For example, the `-s` flag is now available as `--shell` as well.
See the help for the full list.
urchin -h
### Source setup and teardown
setup, teardown, setup_dir, and teardown_dir are now sourced instead of
executed; they are referenced a bit like this.
(
. ./setup
./$thetestfile
. ./teardown
)
My intent is that you should be able to export variables in the setup files.
I think it would be fine to invoke the teardown files instead of sourcing them,
but I chose to source them anyway for consistency.
The disadvantage of this, and the reason I have been reluctant to do it,
is that these files now become much harder to debug, so I recommend keeping
your setup and teardown files very simple. I recommend either of the following
strategies if your setup file gets complicated.
1. Rename it to something starting with a dot, and explicitly source it
in your test file.
2. Export a path in your setup file, rewrite your setup file as a shell
program, and put the rewritten file in your path.
Version 0.0.6 Version 0.0.6
--------------------- ---------------------

48
TODO
View File

@ -1,34 +1,6 @@
Things I want Things I want
============= =============
Test speed
-------------
Make tests run faster.
https://github.com/bike-barn/hermit/issues/62
First, easier thing is probably to run tests in parallel.
Second, also easier thing is to tell people to save things to RAM rather than
disk whenever they can.
Third, harder thing is to put the test suite in RAM automatically. Maybe the
whole test directory, which includes fixtures, gets copied to a tmpfs if one
exists.
Hmm or maybe there's a compromise: Tell people to mount /tmp as a tmpfs so
that temp files are fast. Maybe allow people to set some other directory as
the temporary file place, in case they want a different tmpfs location.
In order to run things in parallel, we have to change how we do the
stdout_file. I think it's easiest to create separate files for each test and
to save them in testroot/.urchin/stdout/$filename. The test root would be
defined as the closest ancestor containing a .urchin directory.
Options
-------------
I want long options. For example, there's presently -f and -e.
I want to make them -f|--force and -e|--exit.
Environment variables Environment variables
------------- -------------
Do something to make it easier to debug environment variables, because that is Do something to make it easier to debug environment variables, because that is
@ -108,26 +80,6 @@ This is as far as I have gotten with contemplating license changes. For now
we're sticking with the original MIT-style license, but it's easy to change we're sticking with the original MIT-style license, but it's easy to change
licenses later. licenses later.
Nagios plugins
-----------------
It would be cool to run Nagios plugins with Urchin. This is already possible,
actually, but it might be worth giving some special thought to it.
https://nagios-plugins.org/doc/guidelines.html
Source setup and teardown
--------------------
If setup and teardown are sourced instead of executed, maybe we can more
cleanly create and teardown temporary files.
(
. ./setup
./$thetestfile
. ./teardown
)
On the other hand, this could just be sourced explicitly in the test file,
without the special setup and teardown feature.
Run on a file Run on a file
---------------- ----------------
Presently you can run urchin only on a directory. Presently you can run urchin only on a directory.