1 files changed, 335 insertions, 0 deletions
diff --git a/etc/private/README b/etc/private/README
new file mode 100644
index 0000000..d9a702e
--- /dev/null
+++ b/etc/private/README
@@ -0,0 +1,335 @@
+This directory contains a virtual machine (VM) with a build2 repository web
+interface (brep) installed and configured for a private package repository.
+It also includes a number of scripts and configuration files for running
+this VM as a systemd service.
+
+A brep installation consists of a web server (Apache2), a database server
+(PostgreSQL), and a number of auxiliary processes (repository loader,
+submission handler, etc). While all this can be installed and configured
+manually (as described in brep/INSTALL), this VM has everything pre-installed
+and pre-configured which makes it possible to quickly get a private
+repository up and running.
+
+Note that the configuration offered by this VM is only suitable for a
+private/trusted environment, normally either for personal use (host-local)
+or for use within an organization's private network. Specifically:
+
+  - The repository is accessed via HTTP.
+
+  - The repository is not signed.
+
+  - Submitted packages are published directly and without ownership
+    authentication.
+
+  - The VM does not auto-update (since it does not assume the presence of an
+    Internet connection) and therefore may not have the latest security
+    patches.
+
+The below setup instructions are for host machines that run systemd-based
+Linux distributions. Note, however, that it should be possible to use other
+distributions or operating systems provided they are capable of running
+QEMU/KVM virtual machines. The following utilities are expected to be
+available on the host machine:
+
+  - systemd >= 229    (systemd --version)
+  - bash    >= 4.3    (bash --version)
+  - qemu    >= 2.5.0  (qemu-system-x86_64 --version)
+  - screen, socat     (screen --version, socat -V)
+
+Consult your distribution's package manager if any of these utilities are
+missing.
+
+The host machine is also expected to have KVM virtualization support as
+well as at least 1G or RAM (2G recommended) and at least 5G of disk space
+(4G for VM image and the rest for package storage) that can be dedicated
+to the VM.
+
+Commands shown in this guide use several prompts with the following meaning:
+
+  # -- must be executed as root on the host machine
+  $ -- must be execute as user brep on the host machine
+  > -- can be executed for testing on any other machine with build2 installed
+
+
+1. Create the brep user and group
+---------------------------------
+
+In this setup, the VM image, scripts, etc., as well as the repository packages
+are all kept in the home directory of the special user brep. In particular,
+the packages are stored on the host machine (as opposed to inside the VM
+image) and are shared with the VM (using the virtio-9p filesystem). As a
+result, if necessary, you can manipulate the package repository from the host
+machine (but see Step 6 below for the rules). This setup also makes it easier
+to upgrade VM images by simply replacing the old image with a new (see Step 7
+below for details).
+
+However, to make this arrangement work reliably, the brep user/group IDs on
+the host machine must match those inside the VM. As a result, we create the
+brep user/group with specific IDs:
+
+# groupadd --gid 63700 brep
+# useradd --uid 63700 --gid 63700 --create-home brep
+# usermod --lock brep # disable password login (if desired)
+
+Additionally, if your distribution requires users that are allowed to use KVM
+to belong to a special group (normally kvm), then add the brep user to this
+group:
+
+# usermod -G kvm brep
+
+If unsure whether this is required, skip this step and come back to it if you
+get the 'KVM: permission denied' error on Step 4.
+
+
+2. Download and unpack the VM archive into the brep user's home directory
+-------------------------------------------------------------------------
+
+# su - brep
+$ pwd
+/home/brep
+
+$ curl -fO https://download.build2.org/X.Y.Z/linux-debian-N-brep-X.Y.Z.tar.xz
+$ sha256sum -b linux-debian-N-brep-X.Y.Z.tar.xz
+
+Verify the checksum matches the one from https://build2.org/download.xhtml
+
+$ tar -xf linux-debian-N-brep-X.Y.Z.tar.xz --strip-components=1
+$ ls
+bin/ etc/ vm/ vm-brep@.service README NEWS
+
+
+3. Configure networking for the VM
+----------------------------------
+
+This setup expects the VM to use bridged networking with a persistent tap
+interface. This allows for a wide variety of configurations ranging between
+host-local (private bridge without routing), subnet (private bridge with
+routing/NAT), and local area network (public bridge over host's Ethernet
+adapter). In particular, the last configuration would make the repository
+accessible from other machines on the same local network.
+
+The exact steps on how to setup bridged networking and create a persistent tap
+interface depend the network manager used thus consult your distribution's
+documentation for details. The guide found in etc/systemd-networkd/README
+shows how to setup the local area network configuration mentioned above using
+the systemd-networkd network manager available on most systemd-based
+distributions.
+
+In the rest of this guide we assume that tap interface called tap0 is
+appropriately configured and is owned by user/group brep.
+
+
+4. Generate a MAC address and start the VM for testing
+------------------------------------------------------
+
+The recommended way to obtain a MAC address for the VM is to generate it based
+on the address of the host's Ethernet adapter (see inside vm-gen-macaddress
+for details):
+
+$ ~/bin/vm-gen-macaddress xx:yy:yy:yy:yy:yy 0
+
+Where xx:yy:yy:yy:yy:yy is the MAC address of the host's Ethernet adapter
+which can can be viewed with the following command:
+
+# ip link show
+
+The address printed by vm-gen-macaddress will be in the 02:yy:yy:yy:yy:yy
+form.
+
+If you are using a local network configuration, then now is a good time to
+assign the VM its IP address and hostname. If you need to submit a request to
+your network administrator, then the following text could serve as a template:
+
+"I would like to run a VM on the <host> machine that needs to have its own IP
+ address and domain name (configured via DHCP). It will have fixed MAC address
+ <mac> (which was derived from <host>'s physical Ethernet address; but you are
+ welcome to assign a different MAC address if required). The DHCP client ID is
+ the same as the MAC address. I would like this machine to have the <vm> name
+ if possible.
+
+ FYI, this is a QEMU/KVM virtual machine running as a systemd service. It
+ will use bridged networking with a tap interface."
+
+Where:
+
+  <host>  host machine's name, for example, myserver.lan (run hostname -f)
+  <mac>   the generated mac address (02:yy:yy:yy:yy:yy)
+  <vm>    VM machine's name, for example, mybrep.lan
+
+Note that the VM is configured to receive its hostname from DHCP server (the
+DHCP protocol option 12, "Host Name"). Failed that, the repository URL will
+use the IP address.
+
+Next, create the package repository directory and start the VM for testing
+(replace 02:yy:yy:yy:yy:yy with the actual MAC address):
+
+$ mkdir -p state/bpkg
+$ ~/bin/vm-start --stdio --tap tap0 --mac 02:yy:yy:yy:yy:yy vm/brep.img
+
+After booting, you will be presented with a login. Login as root with password
+123 (VM command prompts are shown indented with two spaces). Then verify IP
+address, hostname, and the network functionality:
+
+  # ip addr show
+  # hostname -f
+  # ping example.org
+
+If everything appears correct, visit the repository web page with a browser
+(for example, http://mybrep.lan). Check the About page to verify the
+repository URL matches the hostname or IP address.
+
+Try to submit a package (for example, from your development machine):
+
+> bdep new hello
+> cd hello
+> git add . && git commit -m test
+> bdep init -C @test cc
+> bdep publish --control=none --repository http://mybrep.lan --force=snapshot
+
+Visit the repository web page and confirm the package is there. Then try to
+consume the submitted package from the repository:
+
+> bpkg create -d test
+> bpkg build -d test hello@http://mybrep.lan/1
+
+If everything is working fine, shut the VM down:
+
+  # shutdown -h now
+
+
+5. Setup the VM to run as a systemd service
+-------------------------------------------
+
+To start the VM as a systemd service on host boot, perform the following
+steps.
+
+First, create the VM configuration file (replace 02:yy:yy:yy:yy:yy with the
+actual MAC address)::
+
+$ cat <<EOF >vm/brep.conf
+RAM=2G
+CPU=1
+TAP=tap0
+MAC=02:yy:yy:yy:yy:yy
+EOF
+
+Then configure the systemd service:
+
+# cp ~brep/vm-brep@.service /etc/systemd/system/
+# chmod 644 /etc/systemd/system/vm-brep@.service
+# systemctl status vm-brep@brep
+# systemctl start vm-brep@brep
+# systemctl status vm-brep@brep
+
+If the VM fails to start, study the logs for a possible cause:
+
+# journalctl -u vm-brep@brep
+
+If the VM has started successfully, perform the same verifications as on Step
+4 above.
+
+To login to the VM running as a systemd service (for example, to verify IP and
+hostname) use the vm-login script (which uses screen(1) to connect to the VM's
+console):
+
+$ ~/bin/vm-login ~/brep-con.sock
+
+Note that the screen may be blank (due to this being a serial console). In
+this case, press Enter to see the login. To close the login, press 'Ctrl-a k'
+(or 'Ctrl-a a k' if already running inside screen).
+
+If everything functions correctly, verify the VM can be stopped:
+
+# systemctl stop vm-brep@brep
+# systemctl status vm-brep@brep
+
+Finally, if desired, enable the VM service to start on boot:
+
+# systemctl enable vm-brep@brep
+
+After this you may also want to reboot the host machine and confirm the VM is
+started on boot.
+
+
+6. Manage the repository state
+------------------------------
+
+While you can submit packages to the repository using bdep-publish(1), they
+can also be added them manually. Also, currently, packages can only be removed
+manually.
+
+The repository packages and metadata are stored in the ~brep/state/bpkg/pkg/
+directory. If you need to make any modifications in this directory, there are
+two rules that you must follow:
+
+  1. You must stop the VM before making any modifications.
+
+  2. You must make any modification only as user brep.
+
+After performing the modifications, remove the 1/packages.manifest file to
+trigger the repository metadata regeneration on the next VM startup. You can
+also customize the repositories.manifest file in the same way. For example,
+you could add cppget.org as a prerequisite repository for your private
+repository.
+
+Putting it all together, the steps could look like this:
+
+# systemctl stop vm-brep@brep
+# su - brep
+$ cd state/bpkg/pkg/1
+$ <make your changes here>
+$ rm packages.manifest
+$ exit
+# systemctl start vm-brep@brep
+
+Note also that it's easy to break the repository with manual modifications.
+For example, you may add a package that has an unmet dependency or remove a
+package that still has some dependents. In this case, the brep service inside
+the VM will fail to start and the repository web interface will be
+unavailable. In this case, you can login into the VM to investigate:
+
+$ ~/bin/vm-login ~/brep-con.sock
+
+  # systemctl status brep-startup
+  # journalctl -u brep-startup
+
+
+7. Upgrade the VM
+-----------------
+
+To upgrade to the new version of the VM, first download and unpack the new
+VM archive similar to Step 2:
+
+$ curl -fO https://download.build2.org/X.Y.Z/linux-debian-N-brep-X.Y.Z.tar.xz
+$ sha256sum -b linux-debian-N-brep-X.Y.Z.tar.xz
+
+Verify the checksum matches the one from https://build2.org/download.xhtml
+
+$ tar -xf linux-debian-N-brep-X.Y.Z.tar.xz
+
+Next read the linux-debian-N-brep-X.Y.Z/NEWS file for changes and potential
+backwards compatibility issues. Unless instructed otherwise by the NEWS file,
+the upgrade procedure is as follows:
+
+# systemctl stop vm-brep@brep
+
+$ cd
+$ mkdir bak
+$ mv -t bak/ bin etc vm vm-brep@.service README NEWS
+$ mv -t ./ linux-debian-N-brep-X.Y.Z/*
+$ cp bak/vm/brep.conf vm/
+$ rm state/bpkg/pkg/1/packages.manifest
+
+# cp ~brep/vm-brep@.service /etc/systemd/system/
+# chmod 644 /etc/systemd/system/vm-brep@.service
+# systemctl daemon-reload
+# systemctl start vm-brep@brep
+# systemctl status vm-brep@brep
+
+If the VM has started successfully, perform the same verifications as on Step
+4 above. If everything is functioning correctly, you can remove the backup
+files:
+
+$ rm -r bak
+
+If there are any issues, investigate as on Step 6.