From bc5ab1cad93c5b2ca059ee18a4b520e3376ebaa2 Mon Sep 17 00:00:00 2001 From: Boris Kolpackov Date: Mon, 27 Mar 2017 10:38:17 +0200 Subject: Write architecture overview --- doc/manual.cli | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) (limited to 'doc') diff --git a/doc/manual.cli b/doc/manual.cli index 66666f5..9d0d4f2 100644 --- a/doc/manual.cli +++ b/doc/manual.cli @@ -17,4 +17,63 @@ This document describes \c{bbot}, the \c{build2} build bot. \h1#intro|Introduction| + +\h1#arch|Architecture| + +The \c{bbot} architecture includes several layers for security and +manageability. At the top we have a \c{bbot} running in the \i{controller} +mode. The controller monitors various \i{build sources} for \i{build +tasks}. For example, a controller may poll a \c{brep} instances for any new +packages to built as well as monitor a \c{git} repository for any new commits +to test. There can be several layers of controllers with \c{brep} being just a +special kind. A machine running a \c{bbot} instance in the controller mode is +called a \i{controller host}. + +Below the controllers we have a \c{bbot} running in the \i{agent} mode +normally on Build OS. The agent polls its controllers for \i{build tasks} to +perform. A machine running a \c{bbot} instance in the agent mode is called a +\i{build host}. + +The actual building is performed in the virtual machines and/or containers +that are executed on the build host. Inside virtual machines/containers, +\c{bbot} is running in the \i{worker mode} and receives build tasks from its +agent. Virtual machines and containers running a \c{bbot} instance in the +worker mode are collectively called \i{build machines}. + +Let's now examine the workflow in the other direction, that is, from a worker +to a controller. Once a build machine is booted (by the agent), the worker +inside connects to the TFTP server running on the build host and downloads the +\i{build task manifest}. It then proceeds to perform the build task and +uploads the \i{build result manifest} (which includes build logs) to the TFTP +server. + +Once an agent receives a build task for a specific build machine, it goes +through the following steps. First, it creates a directory on its TFTP server +with the \i{build machine id} as its name and places the build task manifest +inside. Next, it makes a throw-away snapshot of the build machine and boots +it. After booting the build machine, the agent monitors the machine directory +on its TFTP server for the build result manifest (uploaded by the worker once +the build has completed). Once the result manifest is obtained, the agent +shuts down the build machine and discards its snapshot. + +To obtains a build task the agent polls via HTTP/HTTPS one or more +controllers. Before each poll request the agent enumerates the available build +machines and sends this information as part of the request. The controller +responds with a build task manifest that identifies a specific build machine +to use. + +If the controller has higher-level controllers (for example, \c{brep}), then +it aggregates the available build machines from its agents and polls these +controllers (just as an agent would), forwarding build tasks to suitable +agents. The controller may also be configured to monitor build sources, such +as SCM repositories, directly in which case it generates build tasks itself. + +In this architecture the build results are propagated up the chain: from a +worker, to its agent, to its controller, and so on. A controller that is the +final destination of a build result uses email to notify interested parties of +the outcome. For example, \c{brep} would send a notification to the package +owner if the build failed. Similarly, a \c{bbot} controller that monitors a +\c{git} repository would send an email to a committer if their commit caused a +build failure. The email would include a link (normally HTTP/HTTPS) to the +build logs hosted by the controller. " -- cgit v1.1