more history
This commit is contained in:
Thomas Levine
parent
7386fb7229
commit
c47cdf6bc8
58
HISTORY
58
HISTORY
|
|
@ -32,8 +32,7 @@ found a `.urchin` file.
|
|||
In either of these cases, Urchin uses the user-specified directory as
|
||||
the root; this is how Urchin `0.0.*` worked.
|
||||
|
||||
Molly guard
|
||||
------------
|
||||
### Molly guard
|
||||
The Molly-guard works differently because it now considers the test suite
|
||||
root directory. The point of the Molly-guard originally was to protect
|
||||
you from things like this.
|
||||
|
|
@ -56,6 +55,15 @@ As before, you can override the Molly guard with `-f`.
|
|||
|
||||
urchin -f build-scripts
|
||||
|
||||
### Consolidation of temporary files in /tmp
|
||||
All of Urchin's temporary files are now stored in /tmp. Urchin previously
|
||||
created `.urchin.log` files alongside the tests, which led to such
|
||||
inconveniences as accidentally commiting them to version control repositories.
|
||||
|
||||
This also means that Urchin will keep all of its temporary files in RAM
|
||||
if you mount a tmpfs on /tmp. On large test suites you may find the tmpfs
|
||||
to be slightly faster than slower storage media like solid-state drives.
|
||||
|
||||
### Skipping of tests
|
||||
Previously, tests were run if they were executable and were otherwise marked
|
||||
as skipped. Now, an executable script can indicate that it is skipped by
|
||||
|
|
@ -74,6 +82,52 @@ would the appropriate status code if these tests were Nagios plugins, as the
|
|||
concept of skipping a test is similar to the Nagios concept of unknown service
|
||||
status (https://nagios-plugins.org/doc/guidelines.html#AEN78).
|
||||
|
||||
### Parallel test execution
|
||||
Tests now run in parallel when possible.
|
||||
|
||||
Parallel processes come about in two situations when parallel execution is
|
||||
turned on.
|
||||
|
||||
1. All files and immediate subdirectories of one particular directory
|
||||
are run in parallel. This happens recursively; during the execution
|
||||
of each particular subdirectory, that subdirectory's children are
|
||||
also run in parallel.
|
||||
2. When cycling of shells is enabled, execution of a particular file in
|
||||
different shells are run parellel.
|
||||
|
||||
Parallel processing and shell cycling are both enabled by default.
|
||||
|
||||
### Options
|
||||
Long options are now available for all command line flags.
|
||||
For example, the `-s` flag is now available as `--shell` as well.
|
||||
See the help for the full list.
|
||||
|
||||
urchin -h
|
||||
|
||||
### Source setup and teardown
|
||||
setup, teardown, setup_dir, and teardown_dir are now sourced instead of
|
||||
executed; they are referenced a bit like this.
|
||||
|
||||
(
|
||||
. ./setup
|
||||
./$thetestfile
|
||||
. ./teardown
|
||||
)
|
||||
|
||||
My intent is that you should be able to export variables in the setup files.
|
||||
I think it would be fine to invoke the teardown files instead of sourcing them,
|
||||
but I chose to source them anyway for consistency.
|
||||
|
||||
The disadvantage of this, and the reason I have been reluctant to do it,
|
||||
is that these files now become much harder to debug, so I recommend keeping
|
||||
your setup and teardown files very simple. I recommend either of the following
|
||||
strategies if your setup file gets complicated.
|
||||
|
||||
1. Rename it to something starting with a dot, and explicitly source it
|
||||
in your test file.
|
||||
2. Export a path in your setup file, rewrite your setup file as a shell
|
||||
program, and put the rewritten file in your path.
|
||||
|
||||
Version 0.0.6
|
||||
---------------------
|
||||
|
||||
|
|
|
|||
48
TODO
48
TODO
|
|
@ -1,34 +1,6 @@
|
|||
Things I want
|
||||
=============
|
||||
|
||||
Test speed
|
||||
-------------
|
||||
Make tests run faster.
|
||||
https://github.com/bike-barn/hermit/issues/62
|
||||
|
||||
First, easier thing is probably to run tests in parallel.
|
||||
|
||||
Second, also easier thing is to tell people to save things to RAM rather than
|
||||
disk whenever they can.
|
||||
|
||||
Third, harder thing is to put the test suite in RAM automatically. Maybe the
|
||||
whole test directory, which includes fixtures, gets copied to a tmpfs if one
|
||||
exists.
|
||||
|
||||
Hmm or maybe there's a compromise: Tell people to mount /tmp as a tmpfs so
|
||||
that temp files are fast. Maybe allow people to set some other directory as
|
||||
the temporary file place, in case they want a different tmpfs location.
|
||||
|
||||
In order to run things in parallel, we have to change how we do the
|
||||
stdout_file. I think it's easiest to create separate files for each test and
|
||||
to save them in testroot/.urchin/stdout/$filename. The test root would be
|
||||
defined as the closest ancestor containing a .urchin directory.
|
||||
|
||||
Options
|
||||
-------------
|
||||
I want long options. For example, there's presently -f and -e.
|
||||
I want to make them -f|--force and -e|--exit.
|
||||
|
||||
Environment variables
|
||||
-------------
|
||||
Do something to make it easier to debug environment variables, because that is
|
||||
|
|
@ -108,26 +80,6 @@ This is as far as I have gotten with contemplating license changes. For now
|
|||
we're sticking with the original MIT-style license, but it's easy to change
|
||||
licenses later.
|
||||
|
||||
Nagios plugins
|
||||
-----------------
|
||||
It would be cool to run Nagios plugins with Urchin. This is already possible,
|
||||
actually, but it might be worth giving some special thought to it.
|
||||
https://nagios-plugins.org/doc/guidelines.html
|
||||
|
||||
Source setup and teardown
|
||||
--------------------
|
||||
If setup and teardown are sourced instead of executed, maybe we can more
|
||||
cleanly create and teardown temporary files.
|
||||
|
||||
(
|
||||
. ./setup
|
||||
./$thetestfile
|
||||
. ./teardown
|
||||
)
|
||||
|
||||
On the other hand, this could just be sourced explicitly in the test file,
|
||||
without the special setup and teardown feature.
|
||||
|
||||
Run on a file
|
||||
----------------
|
||||
Presently you can run urchin only on a directory.
|
||||
|
|
|
|||
Loading…
Reference in New Issue