Syntax highlight in readme.md

HISTORY

@@ -1,10 +1,8 @@
 HISTORY
 =======
 
-Version 0.1.0
+Version 0.2.0 (unstable)
 ---------------------
-This release includes breaking changes.
-
 ### Cross-OS testing
 I have started testing Urchin across multiple operating systems.
 This gives access to more shells, as some shels are easier to install on
@@ -13,6 +11,10 @@ certain operating systems.
 With this cross-OS test suite, I have extended support to more shells.
 A later version of Urchin could include a remote testing feature.
 
+Version 0.1.0 (stable)
+---------------------
+This release includes breaking changes.
+
 ### Test root directory
 We introduce a concept of the root directory of a test suite.
 Such a concept is important in case you want to run subsets of your

TODO

@@ -278,3 +278,17 @@ NetBSD
 
 
 
+Things I can use to make things better
+------------------------
+${x##*blah}
+$IFS and set --
+Redirection, especiall <<-
+Maybe fifo
+for x in "$@"
+until
+readonly
+getopts
+
+Variable assignments specified with special built-in utilities remain in
+effect after the built-in completes; this shall not be the case with a
+regular built-in or other utility.

docs/execution-flow

@@ -0,0 +1,109 @@
+Here I discuss Urchin's general execution flow and how it is handled
+specifically when tests are run on remote environments.
+
+
+Steps of an Urchin run
+----------------------
+When Urchin runs a directory of files, it goes through the following
+steps.
+
+1. Head
+2. Test
+3. Foot
+4. Reporting
+
+Urchin stores files in a temporary directory, creating a new directory
+on each invocation. The directory contains these things.
+
+* head (file)
+* test (file)
+* foot (file)
+* stdout (directory)
+
+When run on remotes, the temporary directory corresponding to the local
+master process additionally has these files.
+
+* remote-test
+
+Messages from the head, test, and foot steps go in the corresponding
+files. In the head and foot phases, messages are just simple prints.
+Messages from the test phase always correspond to a particular test
+file, and they are written to the test file in a delimiter-separated
+format.
+
+Stdout and stderr from test runs are written to files in the stdout
+directory, one file per test file per shell that the file is run in.
+
+The reporting phase
+----------------------
+In most cases Urchin begins printing to the screen only during the
+reporting phase. The only case where anything is printed beforehand is
+when Urchin is run with -vvvv; that sets "+x", so the commands are
+printed as they run, though all other output is still suppressed.
+
+Test results are reported in the reporting phase. Four output formats
+are available.
+
+1. Urchin's human-readable format (default)
+2. Test Anything Protocol
+3. Delimiter-separated values (used internally)
+4. Remote Urchin worker output
+
+Most of the output is generated based on the delimiter-separated values
+in the test log file. The first two formats also include stdout and
+stderr from the tests, depending on verbosity level flags; when it needs
+these, Urchin reads them from appropriate files in the temporary
+directory.
+
+I could discuss the further details of each format elsewhere.
+
+Remotes
+----------------------
+When Urchin runs tests on a remote, it copies tests to the remote and
+then calls Urchin on the remote with "--format=remote". This specifies
+the following.
+
+* The temporary directory should be kept, rather than deleted, after
+  Urchin runs.
+* The path of the temporary directory should be printed as output.
+* No other output should be printed to stdout.
+
+After the remote Urchin finishes running, the local urchin downloads
+the remote Urchin's test log file from the temporary directory.
+It modifies the file to include the remote's name and then concatenates
+the result to the "remote-test" file in the local temporary directory.
+For example, the file from the remote might look like this,
+
+    :sh:Counting tests/.test/faila:0:not_ok
+
+and the result might look like this.
+
+    nsa:sh:Counting tests/.test/faila:0:not_ok
+
+This gets processed in the reporting step like usual, according to
+whatever format is specified. Instead of printing just "sh" as the
+environment in which the particular test was run, the report will print
+"sh on nsa".
+
+When it needs the stdout files, it prints them over ssh.
+
+New flags
+----------
+In making this remotes feature, I wound up adding some others.
+
+-r, --remote      SSH host to use as a remote
+-F, --format      Output format, one of "urchin", "tap", "dsv", "remote"
+
+Urchin runs only locally by default. If you pass at least one --remote
+flag, Urchin runs tests only on the specified remotes; it can't run both
+locally and remotely in the same run. If you want to do that, you could
+wait until I add that feature, or you can add "localhost" as a remote.
+
+Settings that I'm thinking about
+
+* Port for rsync/ssh
+* SSH protocol version
+* --rsync-path
+
+Can those all be set in ssh_config? Probably not --rsync-path, but
+I guess I could just fix it on the remote.

environments/nixos-container/configuration.nix

@@ -0,0 +1,39 @@
+# A NixOS container to protect against accidental fork bombs
+#
+# Put this in /var/lib/containers/test/etc/nixos/configuration.nix
+# See https://nixos.org/wiki/NixOS:Containers
+{ config, lib, pkgs, ... }:
+   
+with lib;
+ 
+{ boot.isContainer = true;
+  networking.hostName = mkDefault "urchin";
+  networking.useDHCP = false;
+
+  environment.systemPackages = with pkgs; [
+    # Urchin
+    bash dash mksh zsh
+    busybox
+
+    # Other
+    vim git rsync tmux
+  ];
+  security.pam.loginLimits = [
+    # Prevent accidental fork bombs.
+    { domain = "*"; item = "nproc"; type = "hard"; value = "200"; }
+  ];
+  services.openssh = {
+    enable = true;
+    passwordAuthentication = false;
+  };
+  users.extraUsers.user = {
+    name = "tlevine";
+    uid = 1000;
+    isNormalUser = true;
+    home = "/home/tlevine";
+    extraGroups = [ "users" "wheel" ];
+    openssh.authorizedKeys.keys = [
+      "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGvQyzr42/96acUTUedaeM2ee+DMt9bkxeurdeXji9sNE10MjjAUFtxPmSI8/BUZW2/a9ByblfaJEI+H+kFVPjVr+QGKXZluxcFMj2BLbH53fi9xLgoQRjb2aAXutb2Bp74/E8R1K+CuFfRRGQ5Spdnv44SLt04D6JbBLcLIcWTpQ4v5RaYr2U27jfiF9z0m+/opxvowEy2gnqlEXFxFk8jZHT4K0uLWm2ENjT6OpyOx8hWcKeAN2vRVRex3pJfSzswn0LpuCrM1rUZ4DRE+FABi8N21Q3MBaMRkwnZPwaZwKzv06q8bu23jYTqK5BrUPtOXeeVuroQXMc12H/6/Nh laptop"
+    ];
+  };
+}

environments/nixos-container/run.sh

@@ -0,0 +1,36 @@
+#!/bin/sh
+set -e
+
+# Create the container.
+if ! nixos-container list | grep ^urchin$ > /dev/null; then
+  sudo nixos-container create urchin
+fi
+
+# Configure the container.
+sudo cp configuration.nix \
+  /var/lib/containers/urchin/etc/nixos/configuration.nix
+sudo nixos-container update urchin
+sudo nixos-container start urchin
+
+# Create the git repository.
+host="tlevine@$(nixos-container show-ip urchin)"
+ssh "${host}" 'if mkdir urchin 2> /dev/null; then
+    cd urchin
+    git init
+    git config --add receive.denyCurrentBranch ignore
+  fi
+'
+
+# Push to the git repository
+git push "${host}":urchin
+
+# Print information
+echo "Log in:
+
+  ssh ${host}
+
+Add git remote
+
+  git remote add ${host} container
+
+"

readme.md

@@ -21,8 +21,10 @@ is like. Clone the repository
 
 Run the tests
 
-    cd urchin
-    ./urchin tests
+```sh
+cd urchin
+./urchin tests
+```
 
 ## Dependencies
 Urchin depends on the following programs.
@@ -50,9 +52,11 @@ of these programs.
 Urchin is contained in a single file, so you can install it by copying it to a
 directory in your `PATH`. For example, you can run the following as root.
 
-    cd /usr/local/bin
-    wget https://raw.githubusercontent.com/tlevine/urchin/v0.1.0-rc3/urchin
-    chmod +x urchin
+```sh
+cd /usr/local/bin
+wget https://raw.githubusercontent.com/tlevine/urchin/v0.1.0-rc3/urchin
+chmod +x urchin
+```
 
 Urchin can be installed with npm too.
 
@@ -110,9 +114,11 @@ directory tree. The test passes if the file exits 0; otherwise, it fails.
 
 urchin looks for files within a directory in the following manner,
 
-    for file in *; do
-      do_something_with_test_file $file
-    done
+```sh
+for file in *; do
+  do_something_with_test_file $file
+done
+```
 
 so files are run in whatever order `*` produces. The order is
 configured in your environment, at least in
@@ -125,18 +131,20 @@ order the tests ran in.
 
 Below you can see how the locale can affect the order.
 
-    $ printf '!c\n@a\n~b\n' | LC_COLLATE=C sort
-    !c
-    @a
-    ~b
-    $ printf '!c\n@a\n~b\n' | LC_COLLATE=en_US.UTF-8 sort
-    @a
-    ~b
-    !c
-    $ printf '!c\n@a\n~b\n' | sort -d
-    @a
-    ~b
-    !c
+```sh
+$ printf '!c\n@a\n~b\n' | LC_COLLATE=C sort
+!c
+@a
+~b
+$ printf '!c\n@a\n~b\n' | LC_COLLATE=en_US.UTF-8 sort
+@a
+~b
+!c
+$ printf '!c\n@a\n~b\n' | sort -d
+@a
+~b
+!c
+```
 
 ### Writing cross-shell compatibility tests for testing shell code
 

tests/Internals/has_shebang_line/.run

@@ -1,3 +1,3 @@
 echo "$1" > $tmp
-TESTING_URCHIN_INTERNALS=true . ../../../urchin
+NO_MAIN= . ../../../urchin
 has_shebang_line $tmp