Add feature: Test description language (testscript)

1 files changed, 111 insertions, 0 deletions

diff --git a/build2/test/testscript b/build2/test/testscript
new file mode 100644
index 0000000..c2e2365
--- /dev/null
+++ b/build2/test/testscript
@@ -0,0 +1,111 @@
+- Test description language (testscript) [feature]
+
+Requirements
+============
+
+* Multiple testscript files (to group related test)
+
+* Run multiple tests
+
+* Specify exit status (optional, expect 0 by default).
+
+* Run pre/post test actions (setups/teardowns, e.g., database schema creation).
+
+* Test input (optional)
+
+* Test stdout output (to compare to, optional)
+
+* Test stderr output (to compare to, optional)
+
+* Test description
+
+* Create (and automatically clean up) input files
+
+? Variable expansion, looked up on target in buildfile
+
+* Ability to pass arguments from build files (how for multiple scripts?)
+  Could use $0, $1, $* if have variable expansion.
+
+Questions
+=========
+
+* What is the testcript attached at the buildfile level? An executable? What
+  if the test is not for executable. I guess we always have the directory
+  target.
+
+  We could reuse the test variable to specify (list of) scripts:
+
+  exe{hello}: test = test1 test2
+
+* Do we want a notion of multi-command test (e.g., with pre/post commands)?
+  Maybe {}? Will need it not to clean up output files too early.
+
+Notes
+=====
+
+* If expected exist status is not 0, then probably makes sense to suppress
+  stderr output if there is no comparison.
+
+Syntax
+======
+
+The goal is to be as concise as possible at least for the common cases.
+
+$0 World       # hello World, 0 exit expected
+$0 "John Doe"  # hello "John Doe"
+$0 == 1        # hello, 1 exit expected
+$0 != 0        # hello, non-0 exit expected
+
+$0 World >>EOO
+Hello, World!
+EOO
+
+$0 World >"Hello, World!"
+
+$0 2>>EOE
+Usage: $0 <name>
+EOE
+
+$0 --trace <<EOI >>EOO 2>>EOE # The data should appear in the order specified.
+World
+EOI
+Hello, World!
+EOO
+13 characters written
+EOE
+
+~ Built in cat command with auto-cleanup? To create files for tests, etc.
+
+~ Output file redirects should be relative to out_base. Input? Could actually
+  track created out files and use that if exists, and src otherwise. What if
+  the test create a file without a redirect? Will need to "register" it
+  somehow for auto-cleanup. Maybe &<file>?
+
+~ Will there be a way to combine stderr and stdout, could be useful sometimes
+  (e.g., order of output relative to diagnostics). 2>& syntax?
+
+~ Do we support command piping?
+
+~ What if we actually do want the output to go to a file, e.g., for a second
+  command to act on it? Maybe keep shell syntax since familiar?  Or reverse it
+  (> - inline, >> - multiline, >>> to file). Simplest thing is the shortest, I
+  like it.
+
+~ $0 is target path (works out in case of a directory, but will be out).
+
+~ #-comment (also for mid-line)
+
+~ quoting & escaping (need to think and specify). Would be nice to avoid shell
+  madness. Perhaps only support escaping only inside quoted strings).
+
+~ If we support variable expansion inside "here doc", what if we need to
+  specify literal '$'? Perhaps treat heredoc as a kind of string? Maybe only
+  do effective escaping? Otherwise will have to escape backslashes...
+
+~ Will probably need to support both $foo and $(foo) syntax. Will need to
+  replicate var name lexing mode form build2. Perhaps use it to avoid
+  duplication?
+
+~ line continuation with \
+
+~ == !=