This directory contains a virtual machine (VM) with a build2 repository web interface (brep) installation for a private package repository plus a number of scripts and configuration files for running this machine as a systemd service. A brep installation consists of a web server (Apache2), a database server (PostgreSQL), and a number of auxiliary processes (repository loader, submission handler, etc). While all this can be installed and configured manually (as described in brep/INSTALL), this VM makes it possible to quickly setup a private repository. Note that the configuration offered by this VM is only suitable for a private/trusted environment, normally either for personal use (host-local) or for use within an organization's private network. Specifically: - The repository is accessed via HTTP. - The repository is not signed. - Submitted packages are published directly and without ownership authentication. - The VM does not perform auto-update (since it does not assume the presence of an Internet connection) and therefore may not have the latest security updates. The below setup instructions are for host machines that run systemd-based Linux distributions. Note, however, that it should be possible to use other distributions or operating systems provided they are capable of running QEMU/KVM virtual machines. The following utilities are expected to be available on the host machine: - systemd >= 229 (systemd --version) - bash >= 4.3 (base --version) - qemu >= 2.5.0 (qemu-system-x86_64 --version) - screen, socat (screen --version, socat -V) Consult your distribution's package manager if any of these utilities are missing. The host machine is also expected to have KVM virtualization support as well as at least 1G or RAM (2G recommended) and at least 5G of disk space (4G for VM image and the rest for package storage) that can be dedicated to the VM. Note: commands that start with the `#` prompt must be executed as root. 1. Create the brep user and group. In this setup, the VM image, scripts, etc., as well as the repository packages are all kept in the home directory of the special brep user. In particular, the packages are stored on the host machine (as opposed to inside the VM image) and are shared with the VM (using the virtio-9p filesystem). As a result, if necessary, you can manipulate the package repository from the host machine (but see Step 6 below for the rules). This setup also makes it easier to upgrade VM images by simply replacing the old one with a new. However, to make this arrangement work reliably, the brep user/group IDs on the host machine must match those inside the VM. As a result, we create the brep user/group with specific IDs: # groupadd --gid 63700 brep # useradd --uid 63700 --gid 63700 --create-home brep # usermod --lock brep # disable password login (if desired) Additionally, if your distribution requires users that are allowed to use KVM to belong to a special group (normally kvm), then add the brep user to this group: # usermod -G kvm brep If unsure whether this is required, skip this step and come back to it if you get 'KVM: permission denied' error on Step 4. 2. Download and unpack the VM archive into the brep user's home directory. # su - brep $ pwd /home/brep $ curl -fO https://download.build2.org/X.Y.Z/linux-debian-N-brep-X.Y.Z.tar.xz $ sha256sum -b linux-debian-N-brep-X.Y.Z.tar.xz Verify the checksum matches the one from https://build2.org/download.xhtml $ tar -xf linux-debian-N-brep-X.Y.Z.tar.xz --strip-components=1 $ ls bin/ etc/ vm/ vm-brep@.service 3. Configure networking for the VM. This setup expects the VM to use bridged networking with a persistent tap interface. This allows for a wide variety of configurations ranging between host-local (private bridge without routing), subnet (private bridge with routing/NAT), and local network (public bridge over host's Ethernet adapter). In particular, the last configuration would make the repository accessible from other machines on the same local network. The exact steps on how to setup bridged networking and create a persistent tap interface depend the network manager used so consult your distribution's documentation for details. The guide found in etc/systemd-networkd/README shows how to setup the local network configuration mentioned above using the systemd-networkd network manager available on most systemd-based distributions. In the rest of this guide we assume that tap interface called tap0 is appropriately configured and is owned by user/group brep. 4. Generate a MAC address and start the VM for testing. The recommended way to obtain a MAC address for the VM is to generate it based on the host's Ethernet adapter address (see inside vm-gen-macaddress for details): $ ~/bin/vm-gen-macaddress xx:yy:yy:yy:yy:yy 0 Where xx:yy:yy:yy:yy:yy is the MAC address of the host's Ethernet adapter that can be viewed with the following command: # ip link show The address printed by vm-gen-macaddress will be in the 02:yy:yy:yy:yy:yy form. If you are using a local network configuration, then now is a good time to assign the VM its IP address and hostname. If you need to submit a request to your network administrator, then the following text could serve as a template: "I would like to run a VM on the machine that needs to have its own IP address and domain name (configured via DHCP). It will have fixed MAC address (which was derived from 's physical Ethernet address; but you are welcome to assign a different MAC address if desired). The DHCP client ID is the same as the MAC address. I would like this machine to have the name if possible. This will be a QEMU/KVM virtual machine running as a systemd service. It will use bridged networking with a tap interface." Where: host machine name, for example, myserver.lan (run hostname -f) the generated mac address (02:yy:yy:yy:yy:yy) VM machine name, for example, mybrep.lan Note that the VM is configured to receive its hostname from DHCP server (the DHCP protocol option 12, "Host Name"). Failed that, the repository URL will use the IP address. Next, create the package repository directory and start the VM for testing (replace 02:yy:yy:yy:yy:yy with the actual MAC address): $ mkdir -p state/bpkg $ ~/bin/vm-start --stdio --tap tap0 --mac 02:yy:yy:yy:yy:yy vm/brep.img After booting, you will be presented with a login. Login as root with password 123 (VM command prompts are shown indented with two spaces). Then verify IP address, hostname, and the network functionality: # ip addr show # hostname -f # ping example.org If everything appears correct, visit the repository web page with a browser (for example, http://mybrep.lan). Check the About page to see the repository URL. Try to submit a package (for example, from your development machine): $ bdep new hello $ cd hello $ git add . && git commit -m test $ bdep init -C @test cc $ bdep publish --control=none --repository http://mybrep.lan --force=snapshot Visit the repository web page and confirm the package is there. Then try to build the submitted package from the repository: $ bpkg create -d test $ bpkg build -d test hello@http://mybrep.lan/1 If everything is working fine, shut the VM down: # shutdown -h now 5. Setup the VM to run as a systemd service. To start the VM as a systemd service on host boot, perform the following steps. First, create the VM configuration file (replace 02:yy:yy:yy:yy:yy with the actual MAC address):: $ cat <vm/brep.conf RAM=2G CPU=1 TAP=tap0 MAC=02:yy:yy:yy:yy:yy EOF Then configure the systemd service: # cp ~brep/vm-brep@.service /etc/systemd/system/ # chmod 644 /etc/systemd/system/vm-brep@.service # systemctl status vm-brep@brep # systemctl start vm-brep@brep # systemctl status vm-brep@brep If the VM fails to start, study the log for a possible cause: # journalctl -u vm-brep@brep If the VM has started successfully, perform the same verifications as on Step 4 above. To login to the VM running as a systemd service (for example, to verify IP and hostname) use the vm-login script (which uses screen to connect to the VM's console): $ ~/bin/vm-login ~/brep-con.sock To close the login, press Ctrl-a k (or Ctrl-a a k if already running inside screen). If everything functions correctly, verify the VM can be stopped: # systemctl stop vm-brep@brep # systemctl status vm-brep@brep Finally, if desired, enable the VM service to start on boot: # systemctl enable vm-brep@brep After this you may also want to reboot the host machine and confirm the VM is started on boot. 6. Manage the repository state. While you can submit packages to the repository using bdep-publish(1), you can also add them manually. Also, packages can only be removed manually. The repository packages and metadata are stored in the ~brep/state/bpkg/pkg/ directory. If you need to make any modifications in this directory, there are two rules that you must follow: 1. You must stop the VM before making any modifications. 2. You must make any modification only as user brep. After performing the modifications, remove the 1/packages.manifest file to trigger the repository metadata regeneration on the next VM startup. You can also customize the repositories.manifest file in the same way. For example, you could add cppget.org as a prerequisite repository for your private repository. Putting it all together, the steps could look like this: # systemctl stop vm-brep@brep # su - brep $ cd state/bpkg/pkg $ $ rm 1/packages.manifest $ exit # systemctl start vm-brep@brep Note also that it's easy to break the repository with manual modifications. For example, you may add a package that has an unmet dependency or remove a package that still has some dependents. In this case, the repository web interface will be unavailable. In this case, you can login into the VM to investigate: $ ~/bin/vm-login ~/brep-con.sock # systemctl status brep-startup # journalctl -u brep-startup 7. Upgrade the VM. To upgrade to the new version of the VM, first download and unpack the new VM archive similar to Step 2: $ curl -fO https://download.build2.org/X.Y.Z/linux-debian-N-brep-X.Y.Z.tar.xz $ sha256sum -b linux-debian-N-brep-X.Y.Z.tar.xz Verify the checksum matches the one from https://build2.org/download.xhtml $ tar -xf linux-debian-N-brep-X.Y.Z.tar.xz Next read the linux-debian-N-brep-X.Y.Z/NEWS file for changes and potential backwards compatibility issues. Unless instructed otherwise by the NEWS file, the upgrade procedure is as follows: # systemctl stop vm-brep@brep $ cd $ mkdir bak $ mv -t bak/ bin etc vm/brep.img vm-brep@.service README NEWS $ mv -t ./ linux-debian-N-brep-X.Y.Z/* $ rm state/bpkg/pkg/1/packages.manifest # cp ~brep/vm-brep@.service /etc/systemd/system/ # chmod 644 /etc/systemd/system/vm-brep@.service # systemctl daemon-reload # systemctl start vm-brep@brep # systemctl status vm-brep@brep If the VM has started successfully, perform the same verifications as on Step 4 above. If everything is functioning correctly, you can remove the backup files: $ rm -r bak If there are any issues, investigate as on Step 6.