aboutsummaryrefslogtreecommitdiff
path: root/doc/testscript.cli
diff options
context:
space:
mode:
authorBoris Kolpackov <boris@codesynthesis.com>2017-01-10 13:45:28 +0200
committerBoris Kolpackov <boris@codesynthesis.com>2017-01-10 13:45:28 +0200
commit1108597507c86ee58cc18dc3e9bd69fefb647f27 (patch)
tree8624ac99957c7f5750564a29bd73fbc5d6f25027 /doc/testscript.cli
parent17a6183e8fdc35025483b06d305e4ecb3a6e1021 (diff)
Testscript doc fixes
Diffstat (limited to 'doc/testscript.cli')
-rw-r--r--doc/testscript.cli70
1 files changed, 44 insertions, 26 deletions
diff --git a/doc/testscript.cli b/doc/testscript.cli
index 1f6599d..9fefd45 100644
--- a/doc/testscript.cli
+++ b/doc/testscript.cli
@@ -238,8 +238,8 @@ Let's explain what's going here: to use a regex here-string or here-document
we add the \c{~} redirect modifier. In this case the here-document end marker
must start and end with the regex introducer character of your choice (\c{/}
in our case). Any line inside the here-document fragment that begins with this
-introducer is then treated as a regular expression rather than a literal (@@
-ref to regex).
+introducer is then treated as a regular expression rather than a literal (see
+\l{#syntax-regex Output Regex} for details).
While this was a fairly deep rabbit hole for a first example, it is a good
illustration of how quickly things get complicated when testing real-world
@@ -411,7 +411,7 @@ But what is \c{cat} exactly? While most POSIX systems will have a program with
this name, there is no such thing in vanilla Windows. To help with this
Testscript provides a subset (both in terms of the number and supported
features) of POSIX utilities, such as, \c{echo}, \c{touch}, \c{cat},
-\c{mkdir}, \c{rm}, and so on (@@ ref builtins).
+\c{mkdir}, \c{rm}, and so on (see \l{#builtins Builtins} for details).
Besides explicit group scopes each test is automatically placed in its own
implicit test scope. However, we can make the test scope explicit, for
@@ -568,12 +568,34 @@ usage: $0 <name>
EOE
\
-The execution of these tests happens in parallel. Testscript will start
-running all script-level tests as well as the \c{config} group immediately.
-Inside \c{config}, once the setup command (\c{cat}) is performed, the two
-inner tests are executed in parallel as well.
+Testscript isolates tests from each other by running each test in its own
+temporary working directory under \c{out_base}. For the above \c{testscript}
+the working directory structure will be as follows:
-@@ Show directory structure and give a test of execution (ref to details)!
+\
+$out_base/
+└── test-hello/
+ ├── command-name/
+ ├── command-names/
+ ├── stdin-names/
+ ├── config/
+ │ ├── hello.conf
+ │ ├── custom-greet/
+ │ └── default-greet/
+ └── missing-name/
+\
+
+If all the test succeed then this working directory structure is automatically
+removed. In case of a failure, however, it is left behind in case you need to
+examine the output of the failed tests. It will also be automatically cleaned
+on the subsequent run, before executing any tests.
+
+The execution of tests happens in parallel. In the above case Testscript will
+start running all the top-level tests as well as the \c{config} group
+immediately. Inside \c{config}, once the setup command (\c{cat}) is
+completed, the two inner tests are executed in parallel as well. Refer to
+\l{#model Model and Execution} for details on the test directory structure and
+execution.
\h1#integration|Build System Integration|
@@ -726,14 +748,14 @@ If this variable is not set explicitly, then it default to \c{build.host}
native testing will be supported.
All the testscripts for a particular test target are executed in a
-subdirectory of \c{out_root} (or, more precisely, in subdirectories of this
+subdirectory of \c{out_base} (or, more precisely, in subdirectories of this
subdirectory, as discussed below). If the test target is a directory, then the
subdirectory is called \c{test}. Otherwise, it is the name of the target
prefixed with\c{test-}. For example:
\
-./: test{foo} # $out_root/test/
-exe{hello}: test{bar} # $out_root/test-hello/
+./: test{foo} # $out_base/test/
+exe{hello}: test{bar} # $out_base/test-hello/
\
@@ -755,9 +777,9 @@ recursively.
A scope (both group and test) has an \i{id}. If not specified explicitly, it
is automatically derived from the group/test location in the testscript file
-(@@ ref description). The id of the implicit outermost scope is the script
-file name without the \c{.test} extension. If the file name is \c{testscript},
-then the id is empty.
+(see \l{#syntax-description Description} for details). The id of the
+implicit outermost scope is the script file name without the \c{.test}
+extension. If the file name is \c{testscript}, then the id is empty.
Based on the ids each nested group and test has an \i{id path} that uniquely
identifies it. It starts with the id of the implied outermost group (unless
@@ -854,7 +876,7 @@ test $foo &out-test
Below is its annotated version:
\
-{ # $~ = $out_root/test/basics/
+{ # $~ = $out_base/test/basics/
{ # $~ = .../test/basics/group/
foo = FOO
bar = BAR
@@ -991,7 +1013,7 @@ $* foo : foo
}
\
-The id paths for the three test will then be:
+The id paths for these three test will then be:
\
basics/foo
@@ -1003,16 +1025,12 @@ To only run individual tests, test groups, or testscript files we can specify
their id paths in the \c{config.test} variable, for example:
\
-$ b test config.test=basics # Run all tests in basics.test.
-$ b test config.test=basics/fox # Run bar and baz.
-$ b test config.test=basics/foo # Run foo.
-$ b test \"config.test=basics/foo basics/fox/bar\" # Run fox and bar.
+$ b test config.test=basics # All tests in basics.test.
+$ b test config.test=basics/fox # All tests in fox (bar and baz).
+$ b test config.test=basics/foo # Test foo.
+$ b test \"config.test=basics/foo basics/fox/bar\" # Tests foo and bar.
\
-@@ Maybe test.id instead of config.test?
-
-@@ What about running from root, with multiple basics.test?
-
\h1#lexical|Lexical Structure|
@@ -2042,8 +2060,8 @@ dir/**/ - all sub-directories recursively (which must be empty)
dir/*** - all files and sub-directories recursively and dir/
\
-Registering a cleanup outside testscript working directory is an error
-(@@ is that correct).
+Registering a path for cleanup that is outside the script working directory is
+an error.
\h#syntax-description|Description|