diff options
| -rwxr-xr-x | bootstrap | 17 | ||||
| -rw-r--r-- | doc/manual.cli | 116 | ||||
| -rwxr-xr-x | init | 122 |
3 files changed, 188 insertions, 67 deletions
@@ -149,6 +149,7 @@ if [ "$stage" -eq "1" ]; then pkgs+=",net-tools,iproute2,iptables,isc-dhcp-client" pkgs+=",ifupdown,bridge-utils,dnsmasq,ntp,postfix" pkgs+=",iputils-ping,wget,curl" + pkgs+=",openssh-client,openssh-server" pkgs+=",less" @@ -293,8 +294,8 @@ apt-get clean # Clean up /bootstrap. # -rm /usr/lib/systemd/system/default.target.wants/setup.service -rm /usr/lib/systemd/system/setup.service +rm /usr/lib/systemd/system/default.target.wants/buildos-setup.service +rm /usr/lib/systemd/system/buildos-setup.service rm -r /bootstrap # Shutdown the container from within. @@ -408,8 +409,12 @@ fi # Test. # -if [ ! -e /tmp/buildos-disk ]; then - qemu-img create -f raw /tmp/buildos-disk 100M +if [ ! -e /tmp/buildos-state ]; then + qemu-img create -f raw /tmp/buildos-state 20M +fi + +if [ ! -e /tmp/buildos-machines ]; then + qemu-img create -f raw /tmp/buildos-machines 100M fi # To test PXE boot, replace -kernel/-initrd/-append with '-boot n'. @@ -420,7 +425,9 @@ sudo kvm \ -netdev "tap,id=net0,script=./qemu-ifup" \ -device "virtio-scsi-pci,id=scsi" \ -device "scsi-hd,drive=disk1" \ - -drive "if=none,id=disk1,file=/tmp/buildos-disk,format=raw" \ + -drive "if=none,id=disk1,file=/tmp/buildos-state,format=raw" \ + -device "scsi-hd,drive=disk2" \ + -drive "if=none,id=disk2,file=/tmp/buildos-machines,format=raw" \ -boot n # -kernel buildos-image -initrd buildos-initrd \ diff --git a/doc/manual.cli b/doc/manual.cli index eae02d3..eaf8d93 100644 --- a/doc/manual.cli +++ b/doc/manual.cli @@ -36,7 +36,7 @@ mode} and receive \i{build tasks} from their respective agents. \c{buildos} is normally booted from the network using PXE but can also be booted locally from the kernel image and initrd directly. -\h2#boot-reboot|Reboot| +\h#boot-reboot|Reboot| Build OS can detect when the OS build has been updated and automatically reboot the build host. This is achieved by polling the URL specified @@ -44,7 +44,7 @@ with the \c{buildos.buildid_url} kernel command line parameter. It should point to the \c{buildos-buildid} file that comes along the kernel image and initrd. See \l{#boot-net Network Boot} for the usage example. -\h2#boot-net|Network Boot| +\h#boot-net|Network Boot| Here we assume that you have already established your PXE setup using PXELINUX. That is, you have configured a TFTP server that hosts the @@ -100,7 +100,7 @@ $ sudo kvm \ || -\h2#boot-local|Local Boot| +\h#boot-local|Local Boot| During testing it is often useful to boot \c{buildos} directly from the kernel image and initrd files. As an example, here is how this can be done @@ -116,50 +116,33 @@ sudo kvm \ \h1#config|Configuration| -\h2#config-net|Network| +\h#config-storage|Storage| -Network is configured via DHCP. Initially, all Ethernet interfaces that have -carrier are tried in (some) order and the first interface that is successfully -configured via DHCP is used. - -Hostname is configured from the DHCP information. Failed that, a name is -generated based on the MAC address, in the form \c{build-xxxxxxxxxx}. -@@ Maybe also kernel cmdline? - -Based on the discovery of the Ethernet interface, two bridge interfaces are -configured: \c{br0} is a public bridge that includes the Ethernet interface -and is configured via DHCP. \c{br1} is a private interface with NAT to \c{br0} -with \c{dnsmasq} configured as a DHCP on this interface. - -Normally, \c{br0} is used for \c{bslave} virtual machines/container (since -they may need to be accessed directly) and \c{br1} \- for \c{bbot} virtual -machines. You can view the bridge configuration on a booted \c{buildos} -instance by examining \c{/etc/network/interfaces}. - -@@ TODO: private network parameters. +Build OS configures storage based on the labels assigned to disks and +partitions (collectively refered to as disks from now on). Build OS requires +storage for state as well as virtual machines and containers. -\h2#config-email|Email| +\h2#config-storage-state|State| -A \c{buildos} instance sends various notifications (including all messages to -\c{root}) to the admin email address. The admin email is specified with -the \c{buildos.admin_email} kernel command line parameter. +Build OS stores a small amount of state on a disk labeled \c{buildos.state} +(mounted as \c{/state}). This includes random number generator state, SSH +server host keys, and so on. While this state is persistent, it is not +precious. -In order to deliver mail, the \c{postfix} MTA is configured to forward to a -relay. The relay host is specified with the \c{buildos.smtp_relay} kernel -command line parameter. +The stored state is fairly small (hundreds of megabytes) and is not +performance-critical. While one can create a small state partition on the same +physical disk as used for machines (see below), having it on a separate disk +makes it easier to move machine disks around. Based on these requirements, a +small USB flash drive or flash card is a good option. -Note that no authentication of any kind is configured for relaying. This means -that the relay host should accept emails from build hosts either because of -their network location (for example, because they are on your organization's -local network and you are using your organization's relay) or because the -relay host accepts emails send to the admin address from anyone (which is -normally the case if the relay is the final destination for the admin -address, for example, \c{example.org} and \c{admin@example.org}). +While any suitable filesystem can be used, \c{ext4} is a good choice, with +journaling disabled if used on a flash drive/card. For example: -\h2#config-storage|Storage| +\ +mkfs.ext4 -L buildos.machines -O ^has_journal /dev/sdX +\ -Build OS configures storage based on the labels assigned to disks and -partitions (collectively refered to as disks from now on). +\h2#config-storage-machines|Machines| For virtual machine and container storage we can use a single disk, in which case it should be labeled \c{buildos.machines} or multiple disks, in which @@ -206,4 +189,57 @@ confirm over-provisioning, format the disk as \c{btrfs}, and label it as # mkfs.btrfs -L buildos.machines -m single /dev/sda # ^D # Exit shell and reboot. \ + +\h#config-net|Network| + +Network is configured via DHCP. Initially, all Ethernet interfaces that have +carrier are tried in (some) order and the first interface that is successfully +configured via DHCP is used. + +Hostname is configured from the DHCP information. Failed that, a name is +generated based on the MAC address, in the form \c{build-xxxxxxxxxx}. +@@ Maybe also kernel cmdline? + +Based on the discovery of the Ethernet interface, two bridge interfaces are +configured: \c{br0} is a public bridge that includes the Ethernet interface +and is configured via DHCP. \c{br1} is a private interface with NAT to \c{br0} +with \c{dnsmasq} configured as a DHCP on this interface. + +Normally, \c{br0} is used for \c{bslave} virtual machines/container (since +they may need to be accessed directly) and \c{br1} \- for \c{bbot} virtual +machines. You can view the bridge configuration on a booted \c{buildos} +instance by examining \c{/etc/network/interfaces}. + +@@ TODO: private network parameters. + +\h#config-email|Email| + +A \c{buildos} instance sends various notifications (including all messages to +\c{root}) to the admin email address. The admin email is specified with +the \c{buildos.admin_email} kernel command line parameter. + +In order to deliver mail, the \c{postfix} MTA is configured to forward to a +relay. The relay host is specified with the \c{buildos.smtp_relay} kernel +command line parameter. + +Note that no authentication of any kind is configured for relaying. This means +that the relay host should accept emails from build hosts either because of +their network location (for example, because they are on your organization's +local network and you are using your organization's relay) or because the +relay host accepts emails send to the admin address from anyone (which is +normally the case if the relay is the final destination for the admin +address, for example, \c{example.org} and \c{admin@example.org}). + + +\h#config-ssh|SSH| + +Build OS runs an OpenSSH server with password authentication disabled. As a +result, the only way to login remotely is via a public key. To add a public +key into the \c{root} user's \c{authorized_keys} file we can use the +\c{buildos.ssh_key} kernel command line parameter. For example (note the +quotes): + +\ +buildos.ssh_key=\"ssh-rsa AAA...OA0DB user@host\" +\ " @@ -244,25 +244,6 @@ bind-interfaces dhcp-range=${priv_netbase}.10,${priv_netbase}.250,12h EOF -# Configure Postfix. -# -cat <<<"$hname" >/etc/mailname - -sed -r -i \ - -e "s%^(myhostname).*%\1 = $hname%" \ - -e 's%^(mydestination).*%\1 = $myhostname, localhost.localdomain, localhost%' \ - -e 's%^(mynetworks).*%\1 = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128%' \ - -e "s%^(relayhost).*%\1 = $smtp_relay%" \ - /etc/postfix/main.cf - -# Make admin alias for buildos.admin_email, alias root as admin. -# -cat <<EOF >>/etc/aliases -admin: $admin_email -root: admin -EOF -newaliases - # Figure out disk configuration and generate the corresponding /etc/fstab. # fstab=/etc/fstab @@ -271,14 +252,15 @@ fstab=/etc/fstab echo -n '' >$fstab l= +state= machines= while read l || [ -n "$l" ]; do d="$(sed -re 's/.*NAME=\"([^\"]+)\".*/\1/' <<<"$l")" t="$(sed -re 's/.*FSTYPE=\"([^\"]*)\".*/\1/' <<<"$l")" l="$(sed -re 's/.*LABEL=\"([^\"]*)\".*/\1/' <<<"$l")" - # Strip the buildos. prefix from the label. If the result is empty then - # this disk/patition hasn't been labeled for use by buildos. + # Strip the buildos prefix from the label. If the result is empty then this + # disk/patition hasn't been labeled for use by buildos. # l="$(sed -n -re 's/^buildos\.([^ ]+)$/\1/p' <<<"$l")" @@ -286,6 +268,39 @@ while read l || [ -n "$l" ]; do continue fi + # Handle buildos.state. + # + if [ "$l" == "state" ]; then + + if [ -n "$state" ]; then + error "multiple disks labeled with buildos.state" + fi + + if [ -z "$t" ]; then + error "no filesystem on $d labeled with buildos.state" + fi + + info "mounting $d (buildos.state) on /state as $t" + + o="defaults,noatime" + echo "$d /state $t $o 0 0" >>$fstab + + # Check it. + # + if ! fsck -n -t "$t" "$d"; then + info "$d (buildos.state) has errors; run fsck -t $type $d" + error + fi + + # Mount it now since we need it below. + # + mkdir -p "/state" + mount -t "$t" -o "$o" "$d" /state + + state="true" + continue + fi + # Handle buildos.machines and buildos.machines.* mounts. # if [[ "$l" == "machines" ]] || [[ "$l" =~ "machines.".+ ]]; then @@ -317,9 +332,10 @@ while read l || [ -n "$l" ]; do info "mounting $d (buildos.$l) on $m" - echo mkdir -p "$m" + mkdir -p "$m" o="defaults,noatime,nodiratime,user_subvol_rm_allowed" echo "$d $m btrfs $o 0 0" >>$fstab + continue fi done < <(lsblk --pairs --paths --output NAME,FSTYPE,LABEL) #done <<EOF @@ -327,6 +343,13 @@ done < <(lsblk --pairs --paths --output NAME,FSTYPE,LABEL) #NAME="/dev/sdb" FSTYPE="btrfs" LABEL="buildos.machines.vol2" #EOF +if [ -z "$state" ]; then + info "no disks labaled with buildos.state among:" + lsblk --paths --output NAME,TYPE,FSTYPE,SIZE,LABEL,UUID + info "consider formatting and/or labelling a suitable disk" + error +fi + if [ -z "$machines" ]; then info "no disks labaled with buildos.machines* among:" lsblk --paths --output NAME,TYPE,FSTYPE,SIZE,LABEL,UUID @@ -334,6 +357,59 @@ if [ -z "$machines" ]; then error fi +# Configure Postfix. +# +cat <<<"$hname" >/etc/mailname + +sed -r -i \ + -e "s%^(myhostname).*%\1 = $hname%" \ + -e 's%^(mydestination).*%\1 = $myhostname, localhost.localdomain, localhost%' \ + -e 's%^(mynetworks).*%\1 = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128%' \ + -e "s%^(relayhost).*%\1 = $smtp_relay%" \ + /etc/postfix/main.cf + +# Make admin alias for buildos.admin_email, alias root as admin. +# +cat <<EOF >>/etc/aliases +admin: $admin_email +root: admin +EOF +newaliases + +# Configure OpenSSH server. Things that we do: +# +# - Change host key locations to (persistent) /state/etc/ssh/ and remove +# existing keys. If no corresponding key exists in /state, generate it. +# +# - Disable password authentication. +# +sed -r -i \ + -e "s%^#?HostKey +(.+)%HostKey /state\1%" \ + -e "s%^#?PasswordAuthentication.*%PasswordAuthentication no%" \ + /etc/ssh/sshd_config + +# Generate missing keys. +# +mkdir -p /state/etc/ssh +for k in $(echo /etc/ssh/ssh_host_*_key | \ + sed -re 's%/etc/ssh/ssh_host_([^_]+)_key%\1%g'); do + if [ ! -e "/state/etc/ssh/ssh_host_${k}_key" ]; then + ssh-keygen -N "" -t "$k" -f "/state/etc/ssh/ssh_host_${k}_key" + fi +done +rm -f /etc/ssh/ssh_host_*_key* + +# Add buildos.ssh_key to root's authorized_keys. +# +if [ -n "$ssh_key" ]; then + info "adding buildos.ssh_key to ~root/.ssh/authorized_keys" + mkdir -p /root/.ssh + chmod 700 /root/.ssh + + echo "$ssh_key" >>/root/.ssh/authorized_keys + chmod 600 /root/.ssh/authorized_keys +fi + # Hand off to systemd. But first arrange to keep console output (which # becomes tty1). # @@ -343,6 +419,8 @@ cat <<EOF >/etc/systemd/system/getty@tty1.service.d/noclear.conf TTYVTDisallocate=no EOF +# Get rid of klibc tools. +# export PATH=/sbin:/usr/sbin:/bin:/usr/bin exec /lib/systemd/systemd \ |