Restyle manifest documentation to be consistent with bpkg manual

1 files changed, 368 insertions, 237 deletions

diff --git a/doc/manual.cli b/doc/manual.cli
index c565278..aa0f92d 100644
--- a/doc/manual.cli
+++ b/doc/manual.cli
@@ -155,388 +155,519 @@ windows-vc_14-32-debug
 linux-gcc_6-cross-arm-eabi
 \
 
-\h#arch-machine-header-manifest|Machine Header Manifest|
+\h#arch-machine-header|Machine Header Manifest|
 
-\
-SYNOPSIS
+@@ TODO: need ref to general manifest overview in bpkg, or, better yet,
+move it to libbutl and ref to that from both places.
 
-id:      <machine-id>
-name:    <machine-name>
+The build machine header manifest contains basic information about a build
+machine on the build host. A list of machine header manifests is sent by
+\c{bbot} agents to controllers. The manifest synopsis is presented next
+followed by the detailed description of each value in subsequent sections.
+
+\
+id: <machine-id>
+name: <machine-name>
 summary: <string>
 \
 
-The build machine header manifest contains basic information about a build
-machine on the build host. A list of machine header manifests is sent by
-\c{bbot} agents to controllers.
+For example:
 
-\dl|
+\
+id: windows_10-msvc_14-1.3
+name: windows_10-msvc_14
+summary: Windows 10 build 1607 with VC 14 update 3
+\
 
-\li|\n\c{id: <machine-id>}\n
+\h2#arch-machine-header-id|\c{id}|
 
-  The \i{machine-id} uniquely identifies a machine version/revision/build.
-  For virtual machines this can be the disk image checksum. For a container
-  this can be UUID that is re-generated every time a container filesystem
-  is altered.|
+\
+id: <machine-id>
+\
 
-\li|\n\c{name: <machine-name>}\n
+The uniquely machine version/revision/build identifies. For virtual machines
+this can be the disk image checksum. For a container this can be UUID that is
+re-generated every time a container filesystem is altered.
 
-  The machine name as described above.|
 
-\li|\n\c{summary: <string>}\n
+\h2#arch-machine-header-name|\c{name}|
 
-  A one-line description of the machine. For example:
+\
+name: <machine-name>
+\
 
-  \
-  id: windows_10-msvc_14-1.3
-  name: windows_10-msvc_14
-  summary: Windows 10 build 1607 with VC 14 update 3
-  \
+The machine name.
 
-||
 
-\h#arch-machine-manifest|Machine Manifest|
+\h2#arch-machine-header-summary|\c{summary}|
 
 \
-SYNOPSIS
-
-id:      <machine-id>
-name:    <machine-name>
 summary: <string>
-
-type:    <machine-type>
-mac:     <macaddr>
-options: <machine-options>
 \
 
+The one-line description of the machine.
+
+
+\h#arch-machine|Machine Manifest|
+
 The build machine manifest contains the complete description of a build
 machine on the build host (see the Build OS documentation for their origin and
-location). The machine manifest starts with the machine manifest header. All
-the header values must appear before any non-header values.
+location). The machine manifest starts with the machine manifest header with
+all the header values appearing before any non-header values. The non-header
+part of manifest synopsis is presented next followed by the detailed
+description of each value in subsequent sections.
 
-\dl|
+\
+type: kvm|nspawn
+[mac]: <addr>
+[options]: <machine-options>
+\
 
-\li|\n\c{type: <machine-type>}\n
 
-  The machine type. Valid values are \c{kvm} (QEMU/KVM virtual machine) and
-  \c{nspawn} (\c{systemd-nspawn} container).|
+\h2#arch-machine-type|\c{type}|
 
-\li|\n\c{mac: <macaddr>}\n
+\
+type: kvm|nspawn
+\
 
-  Optional fixed MAC address for the machine in the hexadecimal,
-  comma-separated format. For example:
+The machine type. Valid values are \c{kvm} (QEMU/KVM virtual machine) and
+\c{nspawn} (\c{systemd-nspawn} container).
 
-  \
-  mac: de:ad:be:ef:de:ad
-  \
 
-  If it is not specified, then a random address is generated on the first
-  machine bootstrap which is then reused for each build/re-bootstrap. Note
-  that it you specify a fixed address, then the machine can only be used by a
-  single \c{bbot} agent.|
+\h2#arch-machine-mac|\c{mac}|
 
+\
+[mac]: <addr>
+\
 
-\li|\n\c{options: <machine-options>}\n
+The fixed MAC address for the machine. Must be in the hexadecimal,
+comma-separated format. For example:
 
-  Optional list of machine options. The exact semantics is machine
-  type-dependent (see below). A single level of quotes (either single or
-  double) is removed in each option before being passed on. Options can be
-  separated with spaces or newlines.
+\
+mac: de:ad:be:ef:de:ad
+\
 
-  For \c{kvm} machines, if this value is present, then it replaces the
-  default network and disk configuration when starting the QEMU/KVM
-  hypervisor The options are pre-processed by replacing the question
-  mark in \c{ifname=?} and \c{mac=?} strings with the network interface
-  and MAC address, respectively.||
+If it is not specified, then a random address is generated on the first
+machine bootstrap which is then reused for each build/re-bootstrap. Note that
+if you specify a fixed address, then the machine can only be used by a single
+\c{bbot} agent.
 
 
-\h#arch-task-manifest|Task Manifest|
+\h2#arch-machine-options|\c{options}|
 
 \
-SYNOPSIS
+[options]: <machine-options>
+\
 
-name:       <package-name>
-version:    <package-version>
-#location:  <package-url>
-repository: <repository-url>
-trust:      <repository-fp>
+The list of machine options. The exact semantics is machine type-dependent
+(see below). A single level of quotes (either single or double) is removed in
+each option before being passed on. Options can be separated with spaces or
+newlines.
 
-machine:       <machine-name>
-target:        <target-triplet>
-config:        <config-vars>
-warning-regex: <warning-regexes>
-\
+For \c{kvm} machines, if this value is present, then it replaces the default
+network and disk configuration when starting the QEMU/KVM hypervisor. The
+options are pre-processed by replacing the question mark in \c{ifname=?} and
+\c{mac=?} strings with the network interface and MAC address, respectively.
+
+
+\h#arch-task|Task Manifest|
 
 The task manifest describes a build task. It consists of two groups of values.
 The first group defines the package to build. The second group defines the
-build configuration to use for building the package.
+build configuration to use for building the package. The manifest synopsis is
+presented next followed by the detailed description of each value in
+subsequent sections.
 
-\dl|
+\
+name: <package-name>
+version: <package-version>
+#location: <package-url>
+repository: <repository-url>
+[trust]: <repository-fp>
 
-\li|\n\c{name: <package-name>}\n
+machine: <machine-name>
+target: <target-triplet>
+[config]: <config-vars>
+[warning-regex]: <warning-regex>
+\
 
-  Package name to test.|
 
-\li|\n\c{version: <package-version>}\n
+\h2#arch-task-name|\c{name}|
 
-  Package version to test.|
+\
+name: <package-name>
+\
 
-\li|\n\c{repository: <repository-url>}\n
+The package name to build.
 
-  The \c{bpkg} repository that contains the package and its dependencies.|
 
-\li|\n\c{trust: <repository-fp>}\n
+\h2#arch-task-version|\c{version}|
 
-  The SHA256 repository certificate fingerprint to trust (see the \c{bpkg}
-  \c{--trust} option for details). This value may be specified multiple times
-  to establish the authenticity of multiple certificates. If the special
-  \c{yes} value is specified, then all repositories will be trusted without
-  authentication (see the \c{bpkg} \c{--trust-yes} option).
+\
+version: <package-version>
+\
 
-  Note that while the controller may return a task with \c{trust} values,
-  whether they will be used is up to the agent's configuration. For example,
-  some agents may only trust their internally-specified fingerprints to
-  prevent the \"man in the middle\" type of attacks.|
+The package version to build.
 
-\li|\n\c{machine: <machine-name>}\n
 
-  The name of the build machine to use.|
+\h2#arch-task-repository|\c{repository}|
 
-\li|\n\c{target: <target-triplet>}\n
+\
+repository: <repository-url>
+\
 
-  The target triplet to build for.
+The \c{bpkg} repository that contains the package and its dependencies.
 
-  Compared to the autotools terminology, the \c{machine} value corresponds
-  to \c{--build} (the machine we are building on) and \c{target} \- to
-  \c{--host} (the machine we are building for). While we use essentially
-  the same \i{target triplet} format as autotools for \c{target}, it is
-  not flexible enough for \c{machine}.|
 
-\li|\n\c{config: <config-vars>}\n
+\h2#arch-task-trust|\c{trust}|
+
+\
+[trust]: <repository-fp>
+\
+
+The SHA256 repository certificate fingerprint to trust (see the \c{bpkg}
+\c{--trust} option for details). This value may be specified multiple times to
+establish the authenticity of multiple certificates. If the special \c{yes}
+value is specified, then all repositories will be trusted without
+authentication (see the \c{bpkg} \c{--trust-yes} option).
+
+Note that while the controller may return a task with \c{trust} values,
+whether they will be used is up to the agent's configuration. For example,
+some agents may only trust their internally-specified fingerprints to prevent
+the \"man in the middle\" attacks.
 
-  Additional build system configuration variables.
 
-  A single level of quotes (either single or double) is removed in each
-  variable before being passed to \c{bpkg}. For example, the following value:
+\h2#arch-task-machine|\c{machine}|
 
-  \
-  config: config.cc.coptions=\"-O3 -stdlib='libc++'\"
-  \
+\
+machine: <machine-name>
+\
 
-  Will be passed to \c{bpkg} as the following (single) argument:
+The name of the build machine to use.
 
-  \
-  config.cc.coptions=-O3 -stdlib='libc++'
-  \
 
-  Variables can be separated with spaces or newlines.|
+\h2#arch-task-target|\c{target}|
 
-\li|\n\c{warning-regex: <warning-regexes>}\n
+\
+target: <target-triplet>
+\
 
-  Additional regular expressions that should be used to detect warnings in
-  the logs. Note that only the first 512 bytes of each log line is considered.
+The target to build for.
 
-  A single level of quotes (either single or double) is removed in each
-  expression before being used for search. For example, the following value:
+Compared to the autotools terminology, the \c{machine} value corresponds to
+\c{--build} (the machine we are building on) and \c{target} \- to \c{--host}
+(the machine we are building for). While we use essentially the same \i{target
+triplet} format as autotools for \c{target}, it is not flexible enough for
+\c{machine}.
 
-  \
-  warning-regex: \"warning C4\d{3}: \"
-  \
 
-  Will be treated as the following (single) regular expression (with a
-  trailing space):
+\h2#arch-task-config|\c{config}|
+
+\
+[config]: <config-vars>
+\
+
+The additional build system configuration variables. A single level of quotes
+(either single or double) is removed in each variable before being passed to
+\c{bpkg}. For example, the following value:
+
+\
+config: config.cc.coptions=\"-O3 -stdlib='libc++'\"
+\
+
+Will be passed to \c{bpkg} as the following (single) argument:
+
+\
+config.cc.coptions=-O3 -stdlib='libc++'
+\
+
+Variables can be separated with spaces or newlines.
+
+\h2#arch-task-warning-regex|\c{warning-regex}|
+
+\
+[warning-regex]: <warning-regex>
+\
 
-  \
-  warning C4\d{3}:
-  \
+Additional regular expressions that should be used to detect warnings in the
+build logs. Note that only the first 512 bytes of each log line is considered.
 
-  Expressions can be separated with spaces or newlines. They will be added to
-  the following default list of regular expressions that detect the \c{build2}
-  toolchain warnings:
+A single level of quotes (either single or double) is removed in each
+expression before being used for search. For example, the following value:
 
-  \
-  ^warning:
-  ^.+: warning:
-  \
+\
+warning-regex: \"warning C4\d{3}: \"
+\
 
-  Note that this built-in list also covers GCC and Clang warnings (for the
-  English locale).||
+Will be treated as the following (single) regular expression (with a trailing
+space):
 
+\
+warning C4\d{3}:
+\
 
-\h#arch-result-manifest|Result Manifest|
+Expressions can be separated with spaces or newlines. They will be added to
+the following default list of regular expressions that detect the \c{build2}
+toolchain warnings:
 
 \
-SYNOPSIS
+^warning:
+^.+: warning:
+\
 
-name:     <package-name>
-version:  <package-version>
+Note that this built-in list also covers GCC and Clang warnings (for the
+English locale).
 
-status:                <status>
-configure-status:      <status>
-update-status:         <status>
-test-status:           <status>
-install-status:        <status>
-test-installed-status: <status>
-uninstall-status:      <status>
 
-configure-log:      <text>
-update-log:         <text>
-test-log:           <text>
-install-log:        <text>
-test-installed-log: <text>
-uninstall-log:      <text>
+\h#arch-result|Result Manifest|
+
+The result manifest describes a build result. The manifest synopsis is
+presented next followed by the detailed description of each value in
+subsequent sections.
+
 \
+name: <package-name>
+version: <package-version>
 
-The result manifest describes a build result.
+status:                  <status>
+[configure-status]:      <status>
+[update-status]:         <status>
+[test-status]:           <status>
+[install-status]:        <status>
+[test-installed-status]: <status>
+[uninstall-status]:      <status>
 
-\dl|
+[configure-log]:      <text>
+[update-log]:         <text>
+[test-log]:           <text>
+[install-log]:        <text>
+[test-installed-log]: <text>
+[uninstall-log]:      <text>
+\
 
-\li|\n\c{name: <package-name>}\n
 
-  Package name from the task manifest.|
+\h2#arch-result-name|\c{name}|
 
-\li|\n\c{version: <package-version>}\n
+\
+name: <package-name>
+\
+
+The package name from the task manifest.
+
+
+\h2#arch-result-version|\c{version}|
 
-  Package version from the task manifest.|
+\
+version: <package-version>
+\
 
-\li|\n\c{status: <status>}\n
+The package version from the task manifest.
 
-  An overall (cumulative) build result status. Valid values are:
 
-  \
-  success    # All operations completed successfully.
-  warning    # One or more operations completed with warnings.
-  error      # One or more operations completed with errors.
-  abort      # One or more operations were aborted.
-  abnormal   # One or more operations terminated abnormally.
-  \
+\h2#arch-result-status|\c{status}|
+
+\
+status: <status>
+\
 
-  The \c{abort} status indicates that the operation has been aborted by
-  \c{bbot}, for example, because it was consuming too many resources and/or
-  was taking too long. Note that a task can be aborted both by the \c{bbot}
-  worker as well as the agent. In the later case the whole machine is shut
-  down and no operation-specific status or logs will be included (@@ Maybe
-  we should just include 'log:' with commands that start VM, for
-  completeness?).
+The overall (cumulative) build result status. Valid values are:
 
-  The \c{abnormal} status indicates that the operation has terminated
-  abnormally, for example, due to the package manager or build system crash.
+\
+success    # All operations completed successfully.
+warning    # One or more operations completed with warnings.
+error      # One or more operations completed with errors.
+abort      # One or more operations were aborted.
+abnormal   # One or more operations terminated abnormally.
+\
 
-  Note that the overall \c{status} value should appear before any
-  per-operation \c{*-status} values.|
+The \c{abort} status indicates that the operation has been aborted by
+\c{bbot}, for example, because it was consuming too many resources and/or was
+taking too long. Note that a task can be aborted both by the \c{bbot} worker
+as well as the agent. In the later case the whole machine is shut down and no
+operation-specific status or logs will be included (@@ Maybe we should just
+include 'log:' with commands that start VM, for completeness?).
 
-\li|\n\c{*-status: <status>}\n
+The \c{abnormal} status indicates that the operation has terminated
+abnormally, for example, due to the package manager or build system crash.
 
-  A per-operation result status. Note that the \c{*-status} values should
-  appear in the same order as the corresponding operations were performed
-  and for each \c{*-status} there should be a corresponding \c{*-log}.|
+Note that the overall \c{status} value should appear before any per-operation
+\c{*-status} values.
 
-\li|\n\c{*-log: <text>}\n
 
-  A per-operation result log. Note that the \c{*-log} values should appear
-  last and in the same order as the corresponding \c{*-status} values.||
+\h2#arch-result-x-status|\c{*-status}|
 
+\
+[*-status]: <status>
+\
 
-\h#arch-task-req-manifest|Task Request Manifest|
+The per-operation result status. Note that the \c{*-status} values should
+appear in the same order as the corresponding operations were performed
+and for each \c{*-status} there should be the corresponding \c{*-log}
+value. Currently supported operation names:
 
 \
-SYNOPSIS
+configure
+update
+test
+install
+test-installed
+uninstall
+\
+
+
+\h2#arch-result-x-log|\c{*-log}|
 
-agent:             <agent-name>
-toolchain-name:    <name>
-toolchain-version: <standard-version>
-fingerprint:       <agent-fingerprint>
 \
+[*-log]: <text>
+\
+
+The per-operation result log. Note that the \c{*-log} values should appear
+last and in the same order as the corresponding \c{*-status} values. For
+the list of supported operation names refer to the \c{*-status} value
+description.
+
+
+\h#arch-task-req|Task Request Manifest|
 
 An agent (or controller acting as an agent) sends a task request to its
 controller via HTTP/HTTPS POST method (@@ URL/API endpoint). The task request
 starts with the task request manifest followed by a list of machine manifests.
+The task request manifest synopsis is presented next followed by the detailed
+description of each value in subsequent sections.
 
-\dl|
+\
+agent: <name>
+toolchain-name: <name>
+toolchain-version: <standard-version>
+[fingerprint]: <agent-fingerprint>
+\
 
-\li|\n\c{agent: <agent-name>}\n
 
-  The name of the agent host (\c{hostname}). This should be unique in a
-  particular \c{bbot} deployment.|
+\h2#arch-task-req-agent|\c{agent}|
 
-\li|\n\c{toolchain-name: <name>}\n
+\
+agent: <name>
+\
 
-  The \c{build2} toolchain name being used by the agent.|
+The name of the agent host (\c{hostname}). The name should be unique in a
+particular \c{bbot} deployment.
 
-\li|\n\c{toolchain-version: <standard-version>}\n
 
-  The \c{build2} toolchain version being used by the agent.|
+\h2#arch-task-req-toolchain-name|\c{toolchain-name}|
 
-\li|\n\c{fingerprint: <agent-fingerprint>}\n
+\
+toolchain-name: <name>
+\
 
-  The SHA256 fingerprint of the agent's public key. An agent may be configured
-  not to use the public key-based authentication in which case it does not
-  include this value. However, the controller may be configured to require
-  the authentication in which case it will respond with the
-  401 (unauthorized) HTTP status code.||
+The \c{build2} toolchain name being used by the agent.
 
-\h#arch-task-res-manifest|Task Response Manifest|
+
+\h2#arch-task-req-toolchain-version|\c{toolchain-version}|
 
 \
-SYNOPSIS
+toolchain-version: <standard-version>
+\
+
+The \c{build2} toolchain version being used by the agent.
+
+
+\h2#arch-task-req-fingerprint|\c{fingerprint}|
 
-session:    <session-id>
-challenge:  <text>
-result-url: <url>
 \
+[fingerprint]: <agent-fingerprint>
+\
+
+The SHA256 fingerprint of the agent's public key. An agent may be configured
+not to use the public key-based authentication in which case it does not
+include this value. However, the controller may be configured to require the
+authentication in which case it should respond with the 401 (unauthorized)
+HTTP status code.
+
+
+\h#arch-task-res|Task Response Manifest|
 
 A controller sends the task response manifest in response to the task request
 initiated by an agent. The response is delivered as a result of the POST
 method. The task response starts with the task response manifest optionally
-followed by a task manifest.
+followed by the task manifest. The task response manifest synopsis is
+presented next followed by the detailed description of each value in
+subsequent sections.
 
-\dl|
+\
+session: <id>
+[challenge]: <text>
+[result-url]: <url>
+\
 
-\li|\n\c{session: <session-id>}\n
 
-  An identifier assigned to this session by the controller. An empty value
-  indicates that the controller has no tasks at this time in which case all
-  the following values as well as the task manifest are absent.|
+\h2#arch-task-res-session|\c{session}|
 
-\li|\n\c{challenge: <string>}\n
+\
+session: <id>
+\
 
-  Random 64-character string (nonce) used to challenge the agent's private
-  key. If present, then the agent must sign this string and include the
-  signature in the result request.
+The identifier assigned to this session by the controller. An empty value
+indicates that the controller has no tasks at this time in which case all the
+following values as well as the task manifest are absent.
 
-  The signature should be calculated by encrypting the string with the agent's
-  private key and then base64-encoding the result.|
 
-\li|\n\c{result-url: <url>}\n
+\h2#arch-task-res-challenge|\c{challenge}|
 
-  The URL to post the result (upload) request to.||
+\
+[challenge]: <text>
+\
+
+The random, 64 characters long string (nonce) used to challenge the agent's
+private key. If present, then the agent must sign this string and include the
+signature in the result request (see below).
+
+The signature should be calculated by encrypting the string with the agent's
+private key and then \c{base64}-encoding the result.
 
 
-\h#arch-result-req-manifest|Result Request Manifest|
+\h2#arch-task-res-result-url|\c{result-url}|
 
 \
-SYNOPSIS
+[result-url]: <url>
+\
+
+The URL to POST (upload) the result request to.
+
+
+\h#arch-result-req|Result Request Manifest|
 
-session:   <session-id>
-challenge: <text>
+On completion of a task an agent (or controller acting as an agent) sends the
+result (upload) request to the controller via the POST method using the URL
+returned in the task response (see above). The result request starts with the
+result request manifest followed by the result manifest. Note that there is no
+result response and only a successful but empty POST result is returned. The
+result request manifest synopsis is presented next followed by the detailed
+description of each value in subsequent sections.
+
+\
+session: <id>
+[challenge]: <text>
 \
 
-On completion of a task an agent (or controller acting as an agent) sends a
-result (upload) request to the controller via HTTP/HTTPS POST method using the
-URL returned in the task response. The result request starts with the
-result request manifest followed by a result manifest. Note that there is no
-result response and only a successful but empty POST result is returned.
 
-\dl|
+\h2#arch-result-req-session|\c{session}|
+
+\
+session: <session-id>
+\
+
+The session id as returned by the controller in the task response.
 
-\li|\n\c{session: <session-id>}\n
 
-  The session id as returned by the controller in the task response.|
+\h2#arch-result-req-challenge|\c{challenge}|
 
-\li|\n\c{challenge: <text>}\n
+\
+[challenge]: <text>
+\
 
-  The answer to the private key challenge as posed by the controller in the
-  task response. Must be present only if the challenge value was present in
-  the task response.||
+The answer to the private key challenge as posed by the controller in the task
+response. It must be present only if the challenge value was present in the
+task response.
 
 
 \h#arch-worker|Worker Logic|