aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorBoris Kolpackov <boris@codesynthesis.com>2018-07-07 19:14:01 +0300
committerKaren Arutyunov <karen@codesynthesis.com>2018-07-10 17:04:39 +0300
commit026377d0c145277b24b3af5fdcf707222e854bd3 (patch)
tree7ebfc0f7d3e92c8a0d23ac2d9d176a08ea509407
parent2aa02defe392a209fe9f11231c82d607813389b9 (diff)
Spec submit logic
-rwxr-xr-xdoc/cli.sh6
-rw-r--r--doc/manual.cli147
-rw-r--r--mod/options.cli93
3 files changed, 238 insertions, 8 deletions
diff --git a/doc/cli.sh b/doc/cli.sh
index 4852af0..a8c42c9 100755
--- a/doc/cli.sh
+++ b/doc/cli.sh
@@ -38,12 +38,14 @@ function compile ()
cli -I .. -v project="brep" -v version="$version" -v date="$date" \
--include-base-last "${o[@]}" --generate-html --html-prologue-file \
-man-prologue.xhtml --html-epilogue-file man-epilogue.xhtml --html-suffix \
-.xhtml ../$n.cli
+man-prologue.xhtml --html-epilogue-file man-epilogue.xhtml --html-suffix .xhtml \
+--link-regex '%brep(#.+)?%build2-repository-interface-manual.xhtml$1%' \
+../$n.cli
cli -I .. -v project="brep" -v version="$version" -v date="$date" \
--include-base-last "${o[@]}" --generate-man --man-prologue-file \
man-prologue.1 --man-epilogue-file man-epilogue.1 --man-suffix .1 \
+--link-regex '%brep(#.+)?%$1%' \
../$n.cli
}
diff --git a/doc/manual.cli b/doc/manual.cli
index 37ab7a5..1aa01b6 100644
--- a/doc/manual.cli
+++ b/doc/manual.cli
@@ -17,11 +17,146 @@
This document describes \c{brep}, the \c{build2} package repository web
interface. For the command line interface of \c{brep} utilities refer to the
\l{brep-load(1)}, \l{brep-clean(1)}, and \l{brep-migrate(1)} man pages.
-"
-/*
-"
-\h Installation
+\h1#submit|Package Submission|
+
+The package submission functionality allows uploading of package archives as
+well as additional, repository-specific information via the HTTP \c{POST}
+method using the \c{multipart/form-data} content type. The implementation in
+\c{brep} only handles uploading as well as basic verification (checksum,
+duplicates) expecting the rest of the submission and publishing logic to be
+handled by a separate entity according to the repository policy. Such an
+entity can be notified by \c{brep} about a new submission as an invocation of
+the \i{handler program} (as part of the submission request) or via email. It
+could also be a separate process that monitors the upload data directory.
+
+The submission request without any parameters is treated as the submission
+form request. If \c{submit-form} is configured, then such a form is generated
+and returned. Otherwise, such a request is treated as an invalid submission
+(missing parameters).
+
+For each submission request \c{brep} performs the following steps.
+
+\ol|
+
+\li|Verify submission size limit.
+
+The submission form-data payload size must not exceed \c{submit-max-size}.|
+
+\li|Verify the required \c{archive} and \c{sha256sum} parameters are present.
+
+The \c{archive} parameter must be the package archive upload while
+\c{sha256sum} must be its 64 characters SHA256 checksum calculated in the
+binary mode.|
+
+\li|Verify other parameters are valid manifest name/value pairs.
+
+The value can only contain printable ASCII characters as well as tab
+(\c{\\t}), carriage return (\c{\\r}), and line feed (\c{\\n}).|
+
+\li|Check for a duplicate submission.
+
+Each submission is saved as a subdirectory in the \c{submit-data} directory
+with a 12-character abbreviated checksum as its name.|
+
+\li|Save the package archive into a temporary directory and verify its
+checksum.
+
+A temporary subdirectory is created in the \c{submit-temp} directory, the
+package archive is saved into it using the submitted name, and its checksum
+is calculated and compared to the submitted checksum.|
+
+\li|Save the submission request manifest into the temporary directory.
+
+The submission request manifest is saved as \c{request.manifest} into the
+temporary subdirectory next to the archive.|
+
+\li|Make the temporary submission directory permanent.
+
+Move/rename the temporary submission subdirectory to \c{submit-data} as an
+atomic operation using the 12-character abbreviated checksum as its new
+name. If such a directory already exist, then this is a duplicate submission.
+
+Note also that once the directory is successfully moved, it is never removed
+by \c{brep}, even in case of a subsequent error.|
+
+\li|Invoke the submission handler program.
+
+If \c{submit-handler} is configured, invoke the handler program passing to it
+additional arguments specified with \c{submit-handler-argument} (if any)
+followed by the absolute path to the submission directory.
+
+The handler program is expected to write the submission result manifest to
+\c{stdout} and terminate with the zero exit status. A non-zero exit status is
+treated as an internal error. The handler program's \c{stderr} is logged.
+
+The handler program assumes ownership of the submission directory and can
+move/remove it (for example, in case of an invalid submission). If after the
+handler program terminates the submission directory still exists, the
+submission result manifest is saved as \c{result.manifest} into this
+directory, next to the request manifest and archive.
+
+If the handler program is not specified, then the following submission result
+manifest is implied (note that it is not saved):
+
+\
+status: 200
+message: submission is queued
+reference: <abbrev-checksum>
+\
+
+|
+
+
+\li|Send the submission email.
+
+If \c{submit-email} is configured, send an email to this address containing
+the submission request manifest and, unless implied, the submission result
+manifest.|
+
+\li|Respond to the client.
+
+Respond to the client with the submission result manifest and its \c{status}
+value as the HTTP status code.|
+
+|
+
+Check violations (max size, duplicate submissions, etc) that are explicitly
+mentioned above are always reported with the submission result manifest. Other
+errors (for example, internal server errors) might be reported with
+unformatted text, including HTML.
+
+
+\h#submit-request-manifest|Submission Request Manifest|
+
+The submission request manifest starts with the below values and in that order
+optionally followed by additional values in the unspecified order
+corresponding to the custom request parameters.
+
+\
+archive: <name>
+sha256sum: <sum>
+timestamp: <datetime>
+[client-ip]: <string>
+[user-agent]: <string>
+\
+
+The \c{timestamp} value is in the ISO-8601 \c{<YYYY>-<MM>-<DD>T<hh>:<mm>:<ss>Z}
+form (always UTC). Note also that \c{client-ip} can be IPv4 or IPv6.
+
+
+\h#submit-result-manifest|Submission Result Manifest|
+
+The submission result manifest starts with the below values and in that order
+optionally followed by additional values if returned by the handler program.
+If the submission is successful, then the \c{reference} value must be present
+and contain a string that can be used to identify this submission (for
+example, the abbreviated checksum).
+
+\
+status: <http-code>
+message: <string>
+[reference]: <string>
+\
+
"
-source "../INSTALL.cli";
-*/
diff --git a/mod/options.cli b/mod/options.cli
index 7fff334..e6b0840 100644
--- a/mod/options.cli
+++ b/mod/options.cli
@@ -388,6 +388,78 @@ namespace brep
}
};
+ class submit: page, module
+ {
+ dir_path submit-data
+ {
+ "<dir>",
+ "The directory to save final submission data to. If unspecified, the
+ package submission functionality will be disabled. If specified,
+ then \cb{submit-temp} must be specified as well. See \l{brep The
+ \cb{build2} Repository Interface Manual} for more information on
+ package submission.
+
+ Note that the directory path must be absolute and the directory
+ itself must exist and have read, write, and execute permissions
+ granted to the user that runs the web server."
+ }
+
+ dir_path submit-temp
+ {
+ "<dir>",
+ "The directory to save temporary submission data to. Must be specified
+ if the package submission functionality is enabled.
+
+ Note that this directory must be on the same filesystem and satisfy
+ the same requirements as \cb{submit-data}. Its contents are
+ automatically cleaned up on each web server startup."
+ }
+
+ size_t submit-max-size = 10485760
+ {
+ "<bytes>",
+ "The maximum size of the submission data accepted. Note that currently
+ the entire submission request is read into memory. The default is
+ 10M."
+ }
+
+ path submit-form
+ {
+ "<file>",
+ "The package submission form fragment. If specified, then its contents
+ are treated as an XHTML5 fragment that is inserted into the <body>
+ element of the submission page. If unspecified, then no submission
+ page will be displayed. Note that the file path must be absolute."
+ }
+
+ string submit-email
+ {
+ "<email>",
+ "The package submission email. If specified, the submission request
+ and result manifests will be sent to this address. See \l{brep The
+ \cb{build2} Repository Interface Manual} for more information."
+ }
+
+ path submit-handler
+ {
+ "<path>",
+ "The handler program to be executed on package submission. The handler
+ is executed as part of the submission request and is passed additional
+ arguments that can be specified with \cb{submit-handler-argument}
+ followed by the absolute path to the submission directory. See
+ \l{brep The \cb{build2} Repository Interface Manual} for more
+ information. Note that the program path must be absolute."
+ }
+
+ strings submit-handler-argument
+ {
+ "<arg>",
+ "Additional arguments to be passed to the submission handler program
+ (see \cb{submit-handler} for details). Repeat this option to specify
+ multiple arguments."
+ }
+ };
+
class repository_root: module
{
};
@@ -536,5 +608,26 @@ namespace brep
//
string result | rs = "*";
};
+
+ // Parameters must either be all present (actual submission) or absent
+ // (submission form request).
+ //
+ // Note also that besides these parameters there can be others. We don't
+ // recognize their semantics and just save them to the submission request
+ // manifest.
+ //
+ class submit
+ {
+ // Package archive file name. Must be <input type="file"/>.
+ //
+ // Note that it can potentially be not just a name but a file path and
+ // in the client's form (e.g., Windows).
+ //
+ string archive;
+
+ // Package archive file SHA256 checksum.
+ //
+ string sha256sum;
+ };
}
}