diff options
| -rw-r--r-- | doc/.gitignore | 2 | ||||
| -rw-r--r-- | doc/bash-style.cli | 4 | ||||
| -rwxr-xr-x | doc/cli.sh | 5 | ||||
| -rw-r--r-- | doc/release.cli | 724 | ||||
| -rw-r--r-- | git/modules | 9 | ||||
| -rwxr-xr-x | publish | 90 | ||||
| -rw-r--r-- | release-bugfix-only.txt | 79 | ||||
| -rw-r--r-- | release.txt | 307 | ||||
| -rwxr-xr-x | rep-test | 179 | ||||
| -rwxr-xr-x | review | 2 | ||||
| -rwxr-xr-x | stage | 125 | ||||
| -rwxr-xr-x | stage-pkg | 17 | ||||
| -rwxr-xr-x | upgrade | 131 | ||||
| -rwxr-xr-x | version | 19 |
14 files changed, 981 insertions, 712 deletions
diff --git a/doc/.gitignore b/doc/.gitignore index a5873dc..36afaee 100644 --- a/doc/.gitignore +++ b/doc/.gitignore @@ -1 +1,3 @@ build2-bash-style.xhtml +build2-release.xhtml + diff --git a/doc/bash-style.cli b/doc/bash-style.cli index 9e90e8a..d22cc23 100644 --- a/doc/bash-style.cli +++ b/doc/bash-style.cli @@ -9,7 +9,7 @@ // - Maximum <pre> line is 70 characters. // -"\h1|Contents|" +"\h1|Table of Contents|" "\$TOC$" " @@ -27,7 +27,7 @@ You just need to be clear on why you are doing it. See also \l{https://google.github.io/styleguide/shell.xml Google's Bash Style Guide} as well as \l{https://github.com/progrium/bashstyle Let's do Bash right!}; we agree with quite a few (but not all) items in there. In particular, -it provides a lot more rationale compared to this guide. +the former provides a lot more rationale compared to this guide. \h1#style|Style| @@ -22,3 +22,8 @@ cli --generate-html --html-suffix .xhtml \ --html-prologue-file doc-prologue.xhtml \ --html-epilogue-file doc-epilogue.xhtml \ --output-prefix build2- bash-style.cli + +cli --generate-html --html-suffix .xhtml \ +--html-prologue-file doc-prologue.xhtml \ +--html-epilogue-file doc-epilogue.xhtml \ +--output-prefix build2- release.cli diff --git a/doc/release.cli b/doc/release.cli new file mode 100644 index 0000000..6639371 --- /dev/null +++ b/doc/release.cli @@ -0,0 +1,724 @@ +// file : doc/release.cli +// copyright : Copyright (c) 2014-2018 Code Synthesis Ltd +// license : MIT; see accompanying LICENSE file + +"\title=Release Process" + +// NOTES +// +// - Maximum <pre> line is 70 characters. +// + +"\h1|Table of Contents|" +"\$TOC$" + +" +\h0#process|Process| + +Review the state and services list (currently on paper) for any new additions. +Consider how/when they are updated/tested during the release process. + + +\h1#stage|Stage| + +The staging repository is completely independent which means it must contain +all the \c{build2} dependencies (plus a few extra packages for testing; see +the \c{etc/stage} script for the complete list). This in turn means that if +any of these dependencies are in the unreleased state, then they should go +through the applicable steps in this section (e.g., updating of \c{NEWS}, etc) +and then be queued and published (effectively released) as part of the +\c{build2} release. Generally, however, we should strive to not unnecessarily +bundle the release of dependencies with the release of \c{build2} to keep +the process as streamlined as possible. + +\N|When unbundling the release of a dependency we need to remove its +distribution from \c{etc/stage} and add the pre-distributed packages +(for example, from \c{public}) to \c{staging/repository/1/}.| + +\b{Pre-Conditions:} + +\ul| + +\li|Current \c{stage} build is clean.|| + +\h#copy|Update copyright if new year| + + Use the \c{git/copyright} script. See the script header for instructions. + +\h#etc|Update \c{etc/git/modules}| + + Review for any new modules. Remove \c{etc/} and \c{private/} from + \c{modules} to reduce noise during stat. + +\h#review|Review \c{@@}| + + Review \c{@@} notes: + + \ + ./review.sh | less -R + \ + + At least look for \c{@@\ TMP} + + \ + ./review.sh | grep TMP + \ + +\h#review-db|Review database schema changes| + + Review database schema changelog differences in \c{bpkg}, \c{bdep}, and + \c{brep} compared to the previous release (tag) for any schema/data + migration that may be required (if something is missing, then it would also + have to be tested). + +\h#news|Update \c{NEWS} files| + + Update in all projects (including dependencies) that will have a release + version. + + See \c{etc/stage} for the complete list. + + +\h#packaging|Release \c{packaging/} dependencies| + + Release dependencies that have the snapshot version as described in + \c{private/build2-packaging.txt}. + + \N|Maybe this should be done during queuing? Why do we release (but not + publish) these now and other dependencies later?| + + +\h#dependencies|Finalize all other dependencies| + + Make sure all other unreleased dependencies listed in \c{etc/stage} are + ready to be released. Effectively, the only remaining step should be to + change the version. + + Do this in the dependency order and finish each off with: + + \ + git pull && bdep sync -fura && bdep test -ar + \ + + +\h#upgrade|Upgrade dependencies| + + Upgrade all dependencies in the \c{build2} toolchain build: + + \ + etc/upgrade + \ + + +\h#hello|Update \c{hello/} projects| + + These projects should closely track the output of \c{bdep-new(1)}. The + recommended procedure is to generate a new project with the same name and + then review/resolve the differences with \c{diff\ -ru}. + + Every change in these projects must be accompanied by a revision increment + and a re-tag. This should be managed with \c{bdep-release(1)}: + + \ + # make changes + git add . + bdep release --revision --show-push + # review commit + git push ... + \ + + Once done, run the \c{intro} scripts and review any changes in the output + (this information will be helpful on the next step): + + \ + cd etc + + ./intro2-tldr 2>&1 | tee intro2-tldr.out + diff -u intro2-tldr.orig intro2-tldr.out # Or use gitk. + mv intro2-tldr.out intro2-tldr.orig + + ./intro2-tour 2>&1 | tee intro2-tour.out + diff -u intro2-tour.orig intro2-tour.out # Or use gitk. + mv intro2-tour.out intro2-tour.orig + \ + +\h#doc|Review documentation| + + Review the following documentation for (1) sample output changes and (2) + still being relevant/making sense. + + \N|Ideally this should be done during development but it's easy to forget. + Also, check if there is any new documentation that has been added that is + not on the below list.| + + \ul| + + \li|Install guide: 1 & 2.| + + \li|Toolchain introduction: 1 & 2 (use \c{intro} script output).| + + \li|Introduction in the build system manual: 1 (uses \c{bdep-new(1)} + output).| + + \li|Testscript manual: 1.|| + + +\h#submod|Update all submodules| + + Make sure all working trees are clean. Review \c{sub_modules} in + \c{etc/git/modules} for any missing new modules, then: + + \N|\c{etc/} and \c{private/} need to be committed manually.| + + \ + ./modup.sh + ./commit.sh # If changes. + ./push.sh + + cd build2-toolchain + + git submodule update --remote --checkout + git submodule foreach git submodule update --init --recursive + + git status # There should only be 'new commits'; see README-GIT + + git commit -a -m \"Update submodules\" + git push + \ + +\h#stage-machines|Update \c{stage} \c{buildtab}s and build machines| + + Review \c{stage} \c{buildtab} for any configurations to drop (for example, + an intermediate version of a compiler). + + Based on these changes update \c{stage} CI \c{buildtab}, which is a subset + of the \c{stage} configurations (and is a base for the \c{queue}/\c{public} + configurations). + + Review deployed machines against the updated \c{stage} \c{buildtab} and + remove those that are no longer used: + + \ + cd private/buildos/ + + ./ls-machines -c stage -c devel + + ~/work/buildos/remove-machine <host> <machine> + \ + + Also review deployed machines against the latest available versions and + upgrade those that are not the latest: + + \ + cd private/buildos/ + + ./ls-machines -l \"/btrfs/$(whoami)/machines/default/\" + ./ls-machines -c stage -c devel + + ~/work/build2/buildos/upload-machine <host> .../new-ver .../old-ver + \ + + +\h#restage|Restage| + + Confirm \c{stage} and \c{devel} \c{buildos} images, hardware-specific + configurations are the same. + + Review \c{staging/0/} and \c{staging/repository/1/} for anything stray. + + Update all submodules in \c{build2-toolchain}: + + \ + git submodule update --remote --checkout + \ + + Restage with \c{baseutils}/\c{mingw} regeneration: + + \ + etc/stage -b + \ + + Upgrade \c{brep} on \c{stage} and sync latest \c{buildtab}s. + + Verify \c{stage} build is clean, nothing is unbuilt. + + +\h#install-stage|Test install scripts| + + Test \l{https://stage.build2.org/0/ \c{stage} install scripts}, including + upgrading, as described in \c{private/install/testing.txt}. + + +\h1#queue|Queue| + +\b{Pre-Conditions:} + +\ul| + +\li|Final \c{stage} build is clean.| + +\li|Build with the \c{queue} toolchain is inactive.|| + + +\h#queue-repo|Review \c{queue} repository| + + Pull, review, and clean the \c{queue} \c{git} repository for any stale + packages or anything (ownership, etc) that may conflict with packages being + released. + + Also clean toolchain distribution: + + \ + rm -rf cppget.org/queue/0/* + \ + +\h#version-release|Change to release version| + + Change to the release version in all the packages being released and tag + (see \c{etc/stage} for the list). Use \c{bdep-release(1)} unless a custom + versioning/tagging scripts are used: + + \ + bdep release --no-open --show-push [--alpha|--beta] + # review commit + git push ... + \ + + Do this in the dependency order and finish each off with: + + \ + bdep sync -fura && bdep update + \ + + For the \c{build2} packages (commit but don't push after each step if + required): + + \ul| + + \li|Close schema versions in \c{bpkg}, \c{bdep}, and \c{brep}.| + + \li|Change \c{BUILD2_STAGE} in \c{build2/build2/config.hxx.in} to \c{false}.| + + \li|Change version by updating (including with new modules) and then + executing: + + \ + etc/version + ./commit.sh + git -C build2-toolchain commit --amend # \"Change version to X.Y.Z\" + \ + + | + + \li|Tag by executing \c{tag.sh\ <version>}.| + + \li|Regenerate documentation in each package.| + + \li|Upgrade all dependencies in configure-only mode by executing + \c{etc/upgrade\ -c}.| + + \li|Trigger regeneration of version files (might require several runs + to \"close off\"): + + \ + BDEP_SYNC=0 b --match-only ~/work/build2/builds/gcc7-asan/ + \ + + If using GCC prior to 8 then might also need explicit version target + upgrades: + + \ + BDEP_SYNC=0 b ~/work/build2/builds/gcc7-asan/.../hxx{version} + \ + + | + + \li|Regenerate ODB in relevant packages passing upgraded configuration + path explicitly (\c{bdep} is not runnable): + + \ + ./odb.sh ~/work/build2/builds/gcc7-asan/ + \ + + | + + \li|Finish upgrading all dependencies by executing in the upgraded + configuration: + + \ + BDEP_SYNC=0 b ~/work/build2/builds/gcc7-asan/ + \ + + || + + Verify key tests pass (in particular, the \c{bdep} tests will now be running + against \c{public} services): + + \ + b test: build2/ bpkg/ bdep/ + \ + + \N|We could have queued after this step before preparing + \c{build2-toolchain}. However, splitting this seems to only increase + complexity without any major benefits (it's not hard to update submodules + and regenerated \c{build2-toolchain} if there are any issues with + packages). Plus we can start testing install scripts, etc.| + + Next push the above changes and update all submodules in + \c{build2-toolchain}: + + \ + ./push.sh + + cd build2-toolchain + + git submodule update --remote --checkout + + git status # There should only be 'new commits'; see README-GIT + + git commit -a -m \"Update submodules\" + \ + + As well as (commit after each step if required): + + \ul| + + \li|Regenerate documentation in each package inside as well as in + \c{build2-toolchain} itself.| + + \li|Update ODB by copying relevant files from the previous step (trust + me, this is the easy way).| + + \li|Change \c{BUILD2_REPO} in \c{build2-toolchain} build scripts to + \c{queue}.|| + + Finally, push the changes: + + \ + git push + \ + +\h#queuing|Queue| + + Prepare packages and the toolchain distribution: + + \ + etc/stage -q -b + \ + + Sort non-alpha packages from \c{cppget.org/queue/1/alpha/} into appropriate + sections (we could probably automate this similar to \c{bdep-release(1)}). + + Note also that we assume all the packages already have the corresponding + ownership information either in \c{queue} or \c{public}. However, if any + new packages were added, then that will have to be added as well. + + Commit and push queue repository: + + \ + cd cppget.org/queue/ + git add . + git ci -m \"Queue build2 toolchain X.Y.Z\" + git push + \ + +\h#build-public|Verify queued packages build with \c{public}| + + This makes sure that the new version can be built with the old toolchain. + + While all the packages should build (except perhaps \c{libhello} which, + being based on the latest \c{bdep-new(1)} output, may use new features), + some tests may fail. In particular, we will be running \c{bpkg} tests using + a previous version of the build system and \c{bdep} tests also using a + previous version of \c{bpkg}. None of that is or will be supported. So + for now we manually review all the failed builds for anything suspicious + and in the future we may skip tests if the tool versions mismatch. + + +\h#install-queue|Test install scripts| + + Test \l{https://download.build2.org/queue/ \c{queue} install scripts}. Here + we just smoke-test each script on its \"primary\" platform and make sure + \c{queue} URLs/repositories are used. + + +\h#upgrade-brep-bpub|Upgrade \c{brep} and \c{bpub} on \c{cppget.org}| + + Upgrade \c{brep} and \c{bpub} using the queued toolchain according + to \c{private/brep-bpub-upgrade.txt}. + + After upgrade, drop all package builds for \c{public} and \c{queue} and + verify there are no regressions. This makes sure that the \c{brep} services + in the new version are usable with the old toolchain. If there are issues, + they have to be fixed by publishing/queuing revisions. + + \N|The \c{bdep} tests that exercise CI and publishing services do it in the + simulation mode. To properly test that these services are compatible with + the old version of the toolchain we would need to CI/publish a real test + package manually.| + + +\h#start-queue|Start \c{queue} builds| + + Update \c{queue} \c{buildtab} based on the \c{stage} CI \c{buildtab} + (normally just a copy sans the sanitized toolchain configurations). + + Adjust \c{stage} and \c{devel} build host configurations to enable the + \c{queue} toolchain. Shift most instances from \c{stage} to \c{queue} + in the hardware class-specific configurations. Regenerate affected + configurations and reboot build hosts: + + \ + cd private/buildos/ + + ./gen-config stage <class> + ./gen-config devel <class> + + ./po-hosts -r -c stage -c devel + \ + + Verify both \c{queue} and \c{public} builds with the queued toolchain, + fixing any breakages with revisions, moving to legacy, and/or queuing + replacements. We can only proceed further once we have a \"resolution\" + for every (newly) broken package. + + +\h#stop-queue|Stop \c{queue} builds| + + Adjust \c{stage} and \c{devel} build host configurations to disable the + \c{queue} toolchain (comment out). Regenerate affected configurations and + reboot build hosts as on the previous step. + +\h1#public|Public| + +\b{Pre-Conditions:} + +\ul| + +\li|A \"resolution\" is queued or published for every (newly) broken package.|| + +\h#public-machines|Update \c{public} \c{buildtab}s and build machines| + + Poweroff the old set of \c{public} build hosts: + + \ + ./po-hosts -c public + \ + + Update \c{public} \c{buildtab}s based on the \c{queue} \c{buildtab} + (normally just a copy). + + Adjust build host configurations and add/remove new/old machines. + + Replace the \c{public} \c{buildos} image on \c{build-cache} with the + one for \c{stage}. + + Comment out the \c{public} toolchain in the build host configuration + (effectively making it a no-toolchain configuration) and power on the new + set of \c{public} build hosts. + + Review deployed machines against the updated \c{public} \c{buildtab} and + remove those that are no longer used: + + \ + cd private/buildos/ + + ./ls-machines -c public + + ~/work/buildos/remove-machine <host> <machine> + \ + + Also review deployed machines against the latest available versions and + upgrade those that are not the latest: + + \ + cd private/buildos/ + + ./ls-machines -l \"/btrfs/$(whoami)/machines/default/\" + ./ls-machines -c public + + ~/work/build2/buildos/upload-machine <host> .../new-ver .../old-ver + \ + + Uncomment the \c{public} toolchain in the build host configuration. The + only remaining step is to reboot (not yet): + + \ + ./po-hosts -r -c public + \ + +\h#pub-dist|Publish distribution| + + Change \c{BUILD2_REPO} in \c{build2-toolchain} build scripts to \c{public} + and publish the distribution (this also cleans/disables the \c{queue} + toolchain): + + \ + etc/stage -p + \ + +\h#pub-pkg|Publish packages| + + Move packages (manually for now) from the \c{queue} to \c{public} \c{git} + repository, including ownership information. Move old/replaced/FTB + packages either to legacy or delete. + + Commit everything (see the commit log for procedure @@ this needs + automation) and push (which should trigger auto-publish). Note: commit and + push both \c{queue} and \c{public} \c{git} repositories. + + Note that once published, the existing install instructions/download + links are no longer usable, so do not linger. + +\h#start-public|Start \c{public} builds| + + Reboot \c{public} build hosts. They should next bootstrap and proceed + with building all packages. + +\h#install-public|Test install scripts| + + Test \l{https://download.build2.org/ \c{public} install scripts}. Here + we just smoke-test each script on its \"primary\" platform and make sure + \c{public} URLs/repositories are used. + +\h#web|Update web| + + \ul| + + \li|Write release notes, use placeholder for announcement URL (or guess). + Add link from the \c{doc.cli}. Add blog entry.| + + \li|Add any new FAQ entries or any other updates (main page, etc). Any new + documentation to link from the Doc page?| + + \li|Update the Download page. + + \ + cat `ls -1 cppget.org/public/0/X.Y.Z/*.sha256` + \ + + | + + \li|Update the Install page. + + \ + cat cppget.org/public/0/toolchain.sha256 + \ + + | + + \li|Regenerate documentation in \c{private/} for all hosts: + + \ + cd private/ + + cd <host>/www/ + ./cli.sh + \ + + | + + \li|Test locally, publish, and test remote: + + \ + cd private/ + ./publish --dry-run + ./publish + \ + + || + + +\h#ann|Announce| + + \ul| + + \li|Write and send the announcement to the mailing lists. + + \ + cat `ls -1 cppget.org/public/0/X.Y.Z/*.sha256` + \ + + Add \c{reply-to:} header for \c{users@} announcement.| + + \li|Patch (or verify) the announcement URL in release notes, re-publish.| + + \li|Announce on reddit, slack, and other relevant places.|| + +\h#tag|Commit, tag, and push the rest.| + + Tag \c{build2-toolchain}: + + \ + cd build2-toolchain + git tag -a vX.Y.Z -m \"Tag version X.Y.Z\" + git push --follow-tags + \ + + Add \c{etc/} and \c{private/} back to \c{modules} in \c{etc/git/modules} + (essentially reverse \l{#etc Update \c{etc/git/modules}}). + + Finalize changes, commit, tag, and push \c{style/}, \c{etc/}, and + \c{private/}. + + Snapshot \c{buildos} subvolumes: + + \ + btrfs subvolume snapshot buildos-3 buildos-3-X.Y.Z + btrfs subvolume snapshot buildos-6 buildos-6-X.Y.Z + \ + +\h1#reopen|Reopen| + +\h#version-snapshot|Change to snapshot version| + + Change to the snapshot version in all the released packages. Use + \c{bdep-release(1)} unless a custom versioning script is used: + + \ + bdep release --open --show-push [--open-*] + # review commit + git push ... + \ + + Essentially, the same steps as in \l{#version-release Change to + release version} (but no tagging). + +\h#stage-machines-reopen|Update \c{stage} \c{buildtab}s and build machines| + + Essentially, the same steps as in \l{#public-machines Update \c{public} + \c{buildtab}s and build machines} but for stage. Some differences: + + Clean \c{buildtab}s (both \c{stage} and CI) by removing no longer relevant + configurations, moving some to \c{legacy}, etc. + + More generally, this is the time to do sweeping changes such as renaming + machines/configurations, adjusting classes, etc. This is also the time to + rebalance machines across available hosts. + +\h#restage-reopen|Restage| + + Make symlinks for the new version in \c{private/baseutils} (for both + \c{baseutils} and \c{mingw}; the idea is that we will start with those and + maybe upgrade later). + + Then cleanup and restage: + + \ + rm -r staging/0/* + rm -r staging/repository/1/*/ + + etc/stage -b + \ + +\h#start-stage|Start \c{stage} builds| + + Reboot \c{stage} build hosts. They should next bootstrap and proceed with + building all the staged packages. Make sure all the builds of the new + development snapshot are successful and there is nothing unbuilt. + +\h#commit-reopen|Commit and push \c{etc/} and \c{private/}.| + + Commit and push changes to \c{etc/} and \c{private/}. + +" diff --git a/git/modules b/git/modules index a793829..d138b98 100644 --- a/git/modules +++ b/git/modules @@ -23,10 +23,10 @@ modules="$modules buildos" modules="$modules msvc-linux" modules="$modules openssl-agent" -#modules="$modules etc" -#modules="$modules private" +modules="$modules etc" +modules="$modules private" -# We don't tag git/, etc/, and private/. +# We don't tag git/, and etc/, private/, build2-toolchain/ are tagged manually. # tag_modules=" \ libbutl \ @@ -42,13 +42,12 @@ bbot \ libstd-modules \ msvc-linux \ openssl-agent \ -build2-toolchain \ buildos" # Submodule update in build2-toolchain has to be done manually. -# As well as in private. # sub_modules=" \ +etc \ private \ libbutl \ libbutl.bash \ diff --git a/publish b/publish deleted file mode 100755 index e9f658f..0000000 --- a/publish +++ /dev/null @@ -1,90 +0,0 @@ -#! /usr/bin/env bash - -# Publish build2 to build2.org/cppget.org (and brep.cppget.org). -# -# The distribution is taken from cppget.org/0/ and packages from -# cppget.org/repository/. -# -# -d -# Only publish the distribution. -# -# -p -# Only publish the packages. -# -usage="$0 [options] -- [<rsync-options>]" - -owd=`pwd` -trap "{ cd $owd; exit 1; }" ERR -set -o errtrace # Trap in functions. - -function info () { echo "$*" 1>&2; } -function error () { info "$*"; exit 1; } - -pkg_only= -dist_only= - -while [ $# -gt 0 ]; do - case $1 in - -d) - dist_only=true - shift - ;; - -p) - pkg_only=true - shift - ;; - --) - shift - break - ;; - *) - error "unexpected $1" - ;; - esac -done - -if [ -z "$pkg_only" ]; then - v="$(sed -n -re 's/^version: ([^.]+\.[^.]+\.[^-]+(-[ab]\.[^.+]+)?).*$/\1/p' build2-toolchain/manifest)" - - if [ -z "$v" ]; then - error "unable to extract version build2-toolchain/manifest" - fi -fi - - -function sync () -{ - if [ -z "$pkg_only" ]; then - info "build2.org:" - - rsync -v -rlpt -c --copy-unsafe-links --prune-empty-dirs --delete-after \ -"${@}" "cppget.org/0/$v/" "build2.org:/var/www/download.build2.org/public/$v/" - - rsync -v -rlpt -c --copy-unsafe-links --prune-empty-dirs --delete-after \ -"${@}" "cppget.org/0/toolchain.sha256" "build2.org:/var/www/download.build2.org/public/" - fi - - if [ -z "$dist_only" ]; then - info "cppget.org:" - etc/rep-publish cppget.org/repository/1/ cppget.org:/var/bpkg/1/ "${@}" - - info "brep.cppget.org:" - etc/rep-publish cppget.org/repository/1/ lists.cppget.org:/var/bpkg/1/ \ -"${@}" - fi -} - -sync --dry-run "${@}" - -r= -while [ -z "$r" ]; do - read -r -p "Continue? [yes/no]: " r - - case "$r" in - yes) ;; - no) exit 0 ;; - *) r= ;; - esac -done - -sync --progress "${@}" diff --git a/release-bugfix-only.txt b/release-bugfix-only.txt deleted file mode 100644 index f8bb098..0000000 --- a/release-bugfix-only.txt +++ /dev/null @@ -1,79 +0,0 @@ -- Create X.Y branch if first bugfix release - - git branch X.Y X.Y.0 - git checkout X.Y - -- See if makes sense to cherry-pick other fixes/changes - -- Update NEWS if anything major, commit - -- Update version in manifest, cli.sh, commit and push: - - git ci -a -m "Bump version to X.Y.N" - git push - -- Regenerate odb.sh, cli.sh - -- If need to rebuild build2-toolchain: - - - Create branch as above, then: - - git submodule update --checkout - - - Add the branch value for each entry in .gitmodules: - - branch = X.Y - - - Then checkout and commit updated submodules: - - git submodule update --remote <submodule> - - - Update version in manifest as above - - - Regenerate odb.sh (may need to checkout old version or copy old), - cli.sh as above. - - - Stage and diff with previous version for sanity check. - -- Dist to queue: - - etc/stage-pkg -q -d -c <group> <proj> - - If need to dist build2-toolchain, then rename X.Y.N-1 to X.Y.N in - private/baseutils, then: - - etc/stage -b -p - - Verify build2-toolchain works by building locally. - -- @@ TODO: Test queue. - -- Move package from queue to the appropriate repository, normally - replacing the old package. - -- Regenerate the repository and publish (remove -p if also publishing - build2-toolchain): - - cd cppget.org - git -C repository add . - git -C repository status - ./update - - cd .. - etc/publish -p - -- Update download links/checksums if changed build2-toolchain. - -- Tag the bugfix release: - - git tag -a X.Y.Z -m "Tag version X.Y.Z" - - Also in build2-toolchain if applicable. - -- Commit cppget.org/repository/ (see history for procedure) - -- Commit private if changed build2-toolchain. - -- Write and send announcements, remember to include checksum. - -- Switch back to master and regenerate cli.sh/odb.sh diff --git a/release.txt b/release.txt deleted file mode 100644 index 350bbaf..0000000 --- a/release.txt +++ /dev/null @@ -1,307 +0,0 @@ -@@ Need to sync hello/* projects with latest bdep-new output (and re-tag) - -@@ No upgrade testing - -- Add new bot machines/configurations/options from stage to queue to public - - @@ See paper in Build OS filder. - - @@ Might have to be done at release if things are incompatible. Maybe - better/cleaner to always do this at release. - -- Replace 0.7.0 and 0.7 in this document with the new version. - -- Remove etc/ and private/ from modules in etc/git/modules to reduce - noise during stat. Also, review for any new modules. - -- Review '@@' items [private/ excluded, update with new modules, at least - look for @@ TMP; also check the review script for any new modules]: - - etc/review | less -R - -- Identify packages that will be released (see etc/stage). They should all be - staged and built successfully, including submodule-updated build2-toolchain - and baseutils/mingw. Review stageing/0/ and staging/1/ for anything stray. - -- Update and test local builds: - - b test: build2/ bpkg/ bdep/ brep/ bbot/ libbutl/ libbpkg/ libbbot/ - b msvc-linux/ - -- Stage hello projects and test: - - cd hello - ./foreach-git pull - ./foreach-git stat - ./stage - - cd .. - etc/intro -s `pwd`/hello/repository/1/ - etc/intro -s https://hello.stage.build2.org/1 - - - Make sure all the bot builds are successful (drop build db). - -- Test bootstrap build2-toolchain using scripts/batch files (bot builds - are using makefile). Need to test build.sh, build-*.bat. Now test via - automated install files: see private/install/testing.txt for details. - -- Update NEWS files in all project that will have non-pre-release version. - -- Review documentation for (1) sample output changes and (2) still being - relevant/making sense: - - - Testscript manual (for 1, see hello/hello-testscript). - - - Install guide for (for 1 & 2). - - - Introduction for (for 1 & 2). Use intro script to get output: - - ./intro2-tldr 2>&1 | tee intro2-tldr.out - ./intro2-tour 2>&1 | tee intro2-tour.out - -- Change to final versions for all packages being released, some by hand, - some with version scripts (see etc/stage for list + msvc-linux). - - - Close schema versions. Review schema changlog difference from previous - release (tag) for any data migration that may be required (would also - need to test this). - - - Push and update all submodules in buil2-toolchain - - - Regenerate all odb & docs (in both packages and inside build2-toolchain) - - - Bootstrap then update and test local - - cd build2 - b clean - git cout build2/version.hxx build2/b-options.?xx - ./bootstrap.sh g++-6 - - cd .. - b-boot '{clean update}(libbutl/ build2/)' - - b clean: bpkg/ bdep/ brep/ bbot/ libbpkg/ libbbot/ msvc-linux/ - - Then the local test step above. - - - Cleanup staging repo: - - rm -r staging/0/* - rm -r staging/repository/1/*/ - - - Restage with -b: - - etc/stage -b - - - Check /0/ and /1/ for anything stray - - - restage hello (libstd-modules) - -- Upgrade brep on cppget.org (queue and main repo) using stage - -- Queue packages: - - - Change BUILD2_REPO in build2-toolchain build scripts to queue - - - Change build.version.stage in build2/build2/context.cxx to false. - - - Cleanup queue repo: - - rm -rf cppget.org/0/* - rm -rf cppget.org/repository/1/queue/*/ - - - Queue - - etc/stage -q -b - - - check /0/ and /1/ for anything stray - - @@ Add extra packages (cli, libcutl, mysql/mariadb). - - - Make sure bot builds are successful - -- Upgrade brep on build2.org (hello) using queue - -- Publish and test hello - - cd hello - ./foreach-git stat - ./stage -p - - cd .. - etc/intro -s https://build2.org/pkg/1/hello - - - Drop build database (package versions do not change) - - - Make sure bot builds are successful - -- Publish and test toolchain - - - Change BUILD2_REPO in build2-toolchain build script to public - - - Regenerate build2-toolchain package in cppget.org/0/ - - etc/stage -p - - @@ Remove *-queue install scripts. - - - Move packages (manually for now): - - - old/replaced/FTB packages either to legacy or delete - - - from queue to appropriate repositories (don't override existing) - - - Regenerate the repository - - cd cppget.org - git -C repository add . - git -C repository status # Review changes. - ./update - - - Disable the queue toolchain on bbots (moved packages from queue/), - delete distribution - - echo disabled | ssh build2.org tee >/dev/null \ - /var/www/download.build2.org/public/queue/toolchain.sha256 - - ssh build2.org rm -r /var/www/download.build2.org/public/queue/*/ - - - Publish everything to build2.org/cppget.org - - etc/publish - - - Make sure bot builds are successful, drop hello build database again. - - @@ First test automated install scripts. - -- Release - - - Update submodules (style/) in private/. - - - Write release notes, use placeholder for announcement URL (or guess). - Add link from the doc page. - - - Any new FAQ entries or any other updates (main page, etc)? Any new - documentation to link from the doc page? - - - Update download page. - - cat `ls -1 *.sha256` - - - Update install page. - - cat toolchain.sha256 - - - Regenerate documentation (./cli.sh) for all hosts. - - - Test locally, publish, test production - - cd private/ - ./publish --dry-run - ./publish - - - Write and send the announcement to mailing lists. - - cat `ls -1 *.sha256` - - - Remember upgrade instructions. - - - Add reply-to: header for users@ announcement. - - - Patch/verify the announcement URL in release notes, re-publish. - - - Announce on reddit and other places (see doc/ann/). - - - Commit and tag private/ and style/ - - git tag -a 0.7.0 -m "Tag version 0.7.0" && git push --follow-tags - -- Tag & Commit hello - - cd hello - ./foreach-git stat - - - commit hello/repository/ (see commit history for procedure) - - git tag -a build2-0.7.0 -m "Tag for build2 version 0.7.0" && git push --follow-tags - - -- Tag & Commit - - - commit & push cppget.org/repository/ (see commit history for procedure; - commit big chunks first, then reorder with rebase). - - - Tag and push all released packages, some by hand, some with tag scripts - (see etc/stage for list but also packages added ad hoc with etc/stage-pkg). - - @@ Use vX.Y.Z tag for packages that are bpkg/bdep-ready (i.e., have - suitable repositories.manifest if required). - - cd <proj> - - # X.Y.Z - v="$(sed -n -re 's/version: ([^ ]+)/\1/p' manifest)"; echo $v; read; \ - git tag -a $v -m "Tag version $v" && git push --follow-tags - - # vX.Y.Z - v="$(sed -n -re 's/version: ([^ ]+)/\1/p' manifest)"; echo v$v; read; \ - git tag -a v$v -m "Tag version $v" && git push --follow-tags - - For build2 projects, review tag_modules in etc/git/modules, then: - - ./tag.sh 0.7.0 - ./push.sh - -- Increment versions and open master for business - - - Change to next development versions for all released packages, some by - hand, some with version scripts (see etc/stage for list + msvc-linux). - - cd <proj> - - ~ edit manifest - - v="$(sed -n -re 's/version: ([^ ]+)/\1/p' manifest)"; echo $v; read; \ - git ci -a -m "Bump version to $v, master is open for business" && git push - - - In build2-toolchain, update all submodules (including libodb*, etc), - change BUILD2_REPO to staging. - - - Regenerate odb & cli docs everywhere (packages and build2-toolchain). - - New sequence: - - ~ rebuild ODB compiler - - @@ bdep sync -fura - @@ b hxx{version} # first lib*, then projects, then odb.sh - - - Update/test local (see early steps of this list). - - - Make symlinks for new version in baseutils (for both baseutils and mingw, - the idea is that we will start with those and maybe upgrade during - development). - - - Restage - - rm -r staging/0/* - rm -r staging/repository/1/*/ - - etc/stage -b - - - Make sure bbot builds of the new development snapshot are successful. - -- Commit etc/ - - - Restore changes to etc/git/modules - - - Change all '+ ' back to '- '. - - - Commit and tag - - git tag -a 0.7.0 -m "Tag version 0.7.0" && git push --follow-tags - -- Snapshot buildos subvolumes as buildos-0.7.0 - - btrfs subvolume snapshot buildos-3 buildos-3-0.7.0 - btrfs subvolume snapshot buildos-6 buildos-6-0.7.0 diff --git a/rep-test b/rep-test deleted file mode 100755 index 09541b6..0000000 --- a/rep-test +++ /dev/null @@ -1,179 +0,0 @@ -#! /usr/bin/env bash - -# Test repository. -# -# Usage: update [options] <dir> -# -# First, the script determines the list of repositories/sections. If <dir> -# contains the repositories.manifest file, then it is the only repository to -# be tested. Otherwise, every first-level subdirectory of <dir> that doesn't -# start with '.' and contains the repositories.manifest file is to be tested. -# -# Then, it makes sure that every package in every repository can be built -# in a clean configuration. -# -# -n -# Only test new packages. For this to work, <dir> should be (part of) a -# git repository. Untracked (and changed) files are considered new. -# -# -e <subdir> -# Exclude the specified sub-directory. Currently only one directory can -# be excluded. -# -# -t <toolchain> -# Specify the build2 toolchain install/stage directory. The script will -# use <toolchain>/bin/b and <toolchain>/bin/bpkd instead of just b and -# bpkg. -# -# -c <option>|<module>|<var> -# Specify additional options, modules, or configuration variables to pass -# to the bpkg create command. For example: -# -# -c cxx -c config.cxx.loptions=-L/usr/local/lib -# -usage="usage: $0 [options] <dir>" - -trap 'exit 1' ERR -set -o errtrace # Trap in functions. - -function info () { echo "$*" 1>&2; } -function error () { info "$*"; exit 1; } - -dir= -cfg=/tmp/test-cfg -toolchain= -exclude= -co=() # Use a bash array to handle arguments with spaces (say "-Ifoo -Ibar"). -new=n - -while [ $# -gt 0 ]; do - case $1 in - -e) - shift - exclude=${1%/} - shift - ;; - -t) - shift - toolchain=${1%/} - shift - ;; - -c) - shift - co=("${co[@]}" "$1") - shift - ;; - -n) - new=y - shift - ;; - *) - dir=${1%/} - shift - break - ;; - esac -done - -if [ -z "$dir" ]; then - error $usage -fi - -if [ -z "$toolchain" ]; then - b="b" - bpkg="bpkg" -else - b="$toolchain/bin/b" - if [ ! -x $b ]; then - error "$b does not exist or is not executable" - fi - bo="--build $b" - - bpkg="$toolchain/bin/bpkg" - if [ ! -x $bpkg ]; then - error "$bpkg does not exist or is not executable" - fi -fi - -if [ "$new" = "y" ]; then - new=( `git -C $dir status --untracked-files=all --porcelain . | \ -sed -e 's%^\(??\| M\|M \) .*/\(.*\)$%\2%'` ) - if [ ${#new[@]} -eq 0 ]; then - info "no new packages in $dir" - exit 0 - fi - info "testing only ${new[@]}" -else - new=() -fi - -function check_new () # <pkg> <ver> -{ - if [ ${#new[@]} -eq 0 ]; then - return 0 - fi - - for f in ${new[@]}; do - if [[ $f == $1-$2.?* ]]; then - return 0 - fi - done - - return 1 -} - -# Determine the list of repositories. -# -rep= - -if [ -f "$dir/repositories.manifest" ]; then - rep=$dir -else - sub=`find $dir -mindepth 1 -maxdepth 1 -type d -name '[^.]*'` - if [ -z "$sub" ]; then - error "no subdirectories in $dir/" - fi - - for d in $sub; do - if [ `basename $d` = "$exclude" ]; then - info "skipping excluded $d/" - continue - fi - if [ -f "$d/repositories.manifest" ]; then - rep+=" $d" - else - info "no repositories.manifest file in $d/, skipping" - fi - done - - if [ -z "$rep" ]; then - error "no repositories in $dir/" - fi -fi - -# Test repositories. -# -for r in $rep; do - info "testing $r" - - IFS=$'\n' - pkg=( `$bpkg rep-info --packages $r` ) - IFS=$' \t\n' - - for p in "${pkg[@]}"; do - nv=( $p ) - n=${nv[0]} - v=${nv[1]} - - if ! check_new $n $v; then - continue - fi - - info "testing $n $v" - - $bpkg $bo create -d $cfg --wipe "${co[@]}" - $bpkg add -d $cfg $r - $bpkg fetch -d $cfg - $bpkg $bo build -d $cfg --yes $n/$v - done -done @@ -6,7 +6,7 @@ # usage="usage: $0" -modules="libbutl build2 libbpkg bpkg bdep brep libbbot bbot libstd-modules build2-toolchain" +modules="libbutl libbutl.bash build2 libbpkg bpkg bpkg-rep bdep brep libbbot bbot libstd-modules build2-toolchain" owd=`pwd` trap "{ cd $owd; exit 1; }" ERR @@ -1,6 +1,6 @@ #! /usr/bin/env bash -# Stage or queue build2 packages and distributions. +# Stage or queue build2 packages and the toolchain distribution. # # -b # Packages baseutils/mingw. @@ -9,11 +9,11 @@ # -s # Only prepare the install scripts and regenerate toolchain.sha256. # -q -# Put packages into the queue instead of staging. Used to update for -# queue. +# Put packages and the distribution into the queue instead of staging. # -p -# Regenerate build2-toolchain into queue's 0/ and update the checksums file -# but don't do anything else. Used to update for public. +# Regenerate and publish distribution. Specifically, move baseutils from +# queue, regenerate build2-toolchain, install scripts, and the checksum +# file, and upload everything to public. Also disable the queue toolchain. # usage="usage: etc/stage" @@ -26,6 +26,12 @@ set -o errtrace # Trap in functions. function info () { echo "$*" 1>&2; } function error () { info "$*"; exit 1; } +# Make sure the build2 tools are runnable. +# +b --version >/dev/null +bpkg --version >/dev/null +bdep --version >/dev/null + v="$(sed -n -re 's/^version: ([^.]+\.[^.]+\.[^-]+(-[ab]\.[^.+]+)?).*$/\1/p' build2-toolchain/manifest)" vf="$(echo $v | sed -re 's/^([^.]*)\.([^.]*)\.([^-]*).*$/\1.\2.\3/')" # Final. @@ -41,6 +47,8 @@ script_only= dist_src=staging/0 dist_dst=stage.build2.org:/var/www/stage.build2.org/public/0 +rsync_del="--delete-after" + while [ $# -gt 0 ]; do case $1 in -b) @@ -48,6 +56,7 @@ while [ $# -gt 0 ]; do shift ;; -o) + base=true base_only=true shift ;; @@ -55,16 +64,22 @@ while [ $# -gt 0 ]; do script_only=true shift ;; - -q) - mode=-q - dist_src=cppget.org/0 + -q | -p) + mode="$1" + dist_src=cppget.org/queue/0 dist_dst=build2.org:/var/www/download.build2.org/public/queue - shift - ;; - -p) - mode=-p - dist_src=cppget.org/0 - dist_dst=example.org/not-used + + if [ "$mode" = "-p" ]; then + + queue_dist_src="$dist_src" + queue_dist_dst="$dist_dst" + + dist_src=cppget.org/public/0 + dist_dst=build2.org:/var/www/download.build2.org/public + + rsync_del= # Do not delete anything existing. + fi + shift ;; *) @@ -113,24 +128,42 @@ if [ "$base" = true ]; then --min-s ./pkpost-mingw-mingw cd ../.. +fi + +# Copy baseutils if requested. +# +if [ "$base" = true -o "$mode" = "-p" ]; then + + if [ "$base" = true ]; then + d=/tmp + else + d="$queue_dist_src/$v" + fi - mv "/tmp/build2-baseutils-$v-i686-windows.zip" "$dist_src/$v/" - mv "/tmp/build2-baseutils-$v-i686-windows.zip.sha256" "$dist_src/$v/" + mv "$d/build2-baseutils-$v-i686-windows.zip" "$dist_src/$v/" + mv "$d/build2-baseutils-$v-i686-windows.zip.sha256" "$dist_src/$v/" - mv "/tmp/build2-baseutils-$v-x86_64-windows.zip" "$dist_src/$v/" - mv "/tmp/build2-baseutils-$v-x86_64-windows.zip.sha256" "$dist_src/$v/" + mv "$d/build2-baseutils-$v-x86_64-windows.zip" "$dist_src/$v/" + mv "$d/build2-baseutils-$v-x86_64-windows.zip.sha256" "$dist_src/$v/" - mv "/tmp/build2-mingw-$v-i686-windows.tar.xz" "$dist_src/$v/" - mv "/tmp/build2-mingw-$v-i686-windows.tar.xz.sha256" "$dist_src/$v/" + mv "$d/build2-mingw-$v-i686-windows.tar.xz" "$dist_src/$v/" + mv "$d/build2-mingw-$v-i686-windows.tar.xz.sha256" "$dist_src/$v/" - mv "/tmp/build2-mingw-$v-x86_64-windows.tar.xz" "$dist_src/$v/" - mv "/tmp/build2-mingw-$v-x86_64-windows.tar.xz.sha256" "$dist_src/$v/" + mv "$d/build2-mingw-$v-x86_64-windows.tar.xz" "$dist_src/$v/" + mv "$d/build2-mingw-$v-x86_64-windows.tar.xz.sha256" "$dist_src/$v/" fi if [ "$base_only" = true ]; then exit 0 fi +# Clean and disable queue toolchain. +# +if [ "$mode" = "-p" ]; then + rm -rf "$queue_dist_src/$v" + echo 'disabled' >"$queue_dist_src/toolchain.sha256" +fi + ## ##if false; then @@ -183,19 +216,32 @@ for f in "${scripts[@]}"; do checksum_line "$f" done -if [ "$mode" = "-p" ]; then - exit 0 -fi - # Sync the distribution. # # Note that we do the distribution first so that the toolchain changes are # visible before package changes. This is relied upon by the Build OS monitor. # -info "Press Enter to start distribution upload" -read -rsync -v -rlpt -c --copy-unsafe-links --prune-empty-dirs --delete-after \ - $rsync_ops "$dist_src/" "$dist_dst/" +function sync_dist () # <dist-src> <dist-dst> [<rsync-options>] +{ + local s="$1" + local d="$2" + + shift + shift + + info "Press Enter to start distribution upload to $d" + read + rsync -v -rlpt -c --copy-unsafe-links --prune-empty-dirs \ + $rsync_ops "$@" "$s/" "$d/" +} + +sync_dist "$dist_src" "$dist_dst" $rsync_del + +# Also sync (disable) queue. +# +if [ "$mode" = "-p" ]; then + sync_dist "$queue_dist_src" "$queue_dist_dst" --delete-after +fi ##fi ## @@ -204,12 +250,16 @@ if [ "$script_only" = true ]; then exit 0 fi +if [ "$mode" = "-p" ]; then + exit 0 +fi + # Distribute packages, regenerate and sync the repository. # # These are just for testing (though libodb-mysql needs libmysqlclient). # -etc/stage-pkg -c $mode -d vcpkg packaging/vcpkg/vcpkg +#etc/stage-pkg -c $mode -d vcpkg packaging/vcpkg/vcpkg etc/stage-pkg -c $mode -d mariadb packaging/mariadb/libmariadb etc/stage-pkg -c $mode -d mysql packaging/mysql/libmysqlclient @@ -233,7 +283,7 @@ etc/stage-pkg -c $mode -d libstudxml ../studxml/libstudxml etc/stage-pkg -c $mode -d hello hello/libhello-1.0 -# For now we also stage libodb-mysql. +# For testing we also stage other libodb-*. # etc/stage-pkg -c $mode -d odb \ ../odb/odb \ @@ -242,7 +292,10 @@ etc/stage-pkg -c $mode -d odb \ ../odb/builds/default/libodb-pgsql \ ../odb/builds/default/libodb-mysql \ ../odb/builds/default/libodb-oracle \ - ../odb/builds/default/libodb-mssql + ../odb/builds/default/libodb-mssql \ + ../odb/builds/default/libodb-boost \ + ../odb/builds/default/libodb-qt + etc/stage-pkg -c $mode build2 \ libbutl \ @@ -250,8 +303,12 @@ etc/stage-pkg -c $mode build2 \ build2 \ libbpkg \ bpkg \ + bpkg-rep \ bdep \ libbbot \ brep \ bbot \ - libstd-modules + openssl-agent + +# msvc-linux +# libstd-modules @@ -12,7 +12,7 @@ # Distribute and regenerating only without publishing the repository. # # -q -# Publish packages into the queue instead of staging. +# Publish packages into the queue instead of staging. Implies -d. # usage="usage: etc/stage-pkg [<options>] <group> <dir>..." @@ -25,6 +25,12 @@ set -o errtrace # Trap in functions. function info () { echo "$*" 1>&2; } function error () { info "$*"; exit 1; } +# Make sure the build2 tools are runnable. +# +b --version >/dev/null +bpkg --version >/dev/null +bdep --version >/dev/null + repo_name="STAGE.BUILD2.ORG" repo_dir="staging/repository/1" repo_host1="stage.build2.org:/var/bpkg/1" @@ -51,10 +57,11 @@ while [ $# -gt 0 ]; do shift ;; -q) - repo_name="CPPGET.ORG/QUEUE" - repo_dir="cppget.org/repository/1/queue" - repo_host1="cppget.org:/var/bpkg/1/queue" - repo_host2="queue.cppget.org:/var/bpkg/1/queue" + #repo_name="CPPGET.ORG/QUEUE" + repo_dir="cppget.org/queue/1/alpha" + #repo_host1="cppget.org:/var/bpkg/1/queue" + #repo_host2="queue.cppget.org:/var/bpkg/1/queue" + dist_only=true shift ;; *) @@ -0,0 +1,131 @@ +#! /usr/bin/env bash + +# Upgrade remote packages in a bdep-managed build2 toolchain build, +# essentially as if by executing: +# +# bdep sync -fur && b +# +# In each project of the toolchain. +# +# Note that we can't just do that because as soon as we drop some dependency +# package, the tools (b, bppg, etc) become non-runnable (missing shared +# libraries, etc). +# +# Note also that this script expects the build system (build2) and the rest of +# the toolchain (bpkg, bdep, etc) to be built in separate build configuration +# (for example, tsan for build2 and asan for the rest). +# +# Finally, this script only upgrades the default configurations. To upgrade +# the rest you can use the normal way, for example: +# +# bdep sync -fura +# +# If the -c option is specified, then the second configuration is upgraded +# and configured but is not updated. This can be done at a later stage by +# running the build system in the configuration directory: +# +# BDEP_SYNC=0 b +# +trap "exit 1" ERR +set -o errtrace # Trap in functions. + +function info () { echo "$*" 1>&2; } +function error () { info "$*"; exit 1; } + +# Make sure the build2 tools are runnable. +# +b --version >/dev/null +bpkg --version >/dev/null +bdep --version >/dev/null + +configure_only= + +while [ $# -gt 0 ]; do + case $1 in + -c) + configure_only=--configure-only + shift + ;; + *) + error "unexpected $1" + ;; + esac +done + +# Get the configuration directories. +# +bcfg="$(b info: build2/ | sed -n -re 's/^out_root: (.+)$/\1/p')" +pcfg="$(b info: bpkg/ | sed -n -re 's/^out_root: (.+)$/\1/p')" + +if [ -z "$bcfg" -o -z "$pcfg" ]; then + error "unable to determine build configuration directories" +fi + +bcfg="$(dirname "$bcfg")" +pcfg="$(dirname "$pcfg")" + +if [ "$bcfg" = "$pcfg" ]; then + error "build2 and bpkg build configuration directories are the same" +fi + +# The plan is as follows: +# +# 0. First, make backup copies of both configurations. If something goes wrong, +# it's likely the tools will be left in a non-runnable state. +# +# 1. Next, install the build system on the side and then upgrade the build +# system configuration using that plus the existing bpkg/bdep. +# +# 2. Finally, upgrade the rest. Note that we only have one shot at this since +# we are running bpkg that is being upgraded. +# + +# Step 0. +# +if test -e "$bcfg.bak"; then + error "$bcfg.bak already exist" +fi + +if test -e "$pcfg.bak"; then + error "$pcfg.bak already exist" +fi + +set -x + +cp -rp "$bcfg" "$bcfg.bak" +cp -rp "$pcfg" "$pcfg.bak" + +# Step 1. +# +bpkg install -d "$bcfg" \ + config.install.root=/tmp/build2-install \ + config.bin.rpath=/tmp/build2-install/lib \ + build2 + +/tmp/build2-install/bin/b --version >/dev/null + +bpkg fetch -d "$bcfg" +BDEP_SYNC=0 bpkg build -d "$bcfg" --build /tmp/build2-install/bin/b --keep-out -ur + +b --version >/dev/null + +b build2/ # Should be a noop. + +rm -rf /tmp/build2-install + +# Step 2. +# +bpkg fetch -d "$pcfg" +BDEP_SYNC=0 bpkg build -d "$pcfg" --keep-out -ur "$configure_only" + +if [ -z "$configure_only" ]; then + + bpkg --version >/dev/null + bdep --version >/dev/null + + b bpkg/ bdep/ # Should be a noop. + +fi + +rm -rf "$bcfg.bak" +rm -rf "$pcfg.bak" @@ -13,22 +13,21 @@ usage="usage: etc/version" build2_min=0.8.0- bpkg_min=0.8.0- -ver=0.8.0-a.0.z -#ver=0.7.0 -mods=(libstd-modules libbutl libbutl.bash libbpkg libbbot) +#ver=0.10.0-a.0.z +ver=0.9.0 +mods=(libstd-modules libbutl libbutl.bash libbpkg libbbot bpkg) -odb_ver=2.5.0-b.8.z -#odb_ver=2.5.0-b.7 +#odb_ver=2.5.0-b.12.z +odb_ver=2.5.0-b.11 odb_mods=(libodb libodb-sqlite libodb-pgsql) -xml_ver=1.1.0-b.3.z -#xml_ver=1.1.0-b.3 +#xml_ver=1.1.0-b.6.z +xml_ver=1.1.0-b.5 xml_mods=(libstudxml) -# @@ NOTE: libpkgconf manually. - -all=(libstd-modules libbutl libbutl.bash build2 libbpkg bpkg bdep libbbot bbot brep build2-toolchain msvc-linux buildos) +# @@ NOTE: libpkgconf version constrain is updated manually. +all=(libstd-modules libbutl libbutl.bash build2 libbpkg bpkg bpkg-rep bdep libbbot bbot brep build2-toolchain msvc-linux openssl-agent buildos) # In manifests for alpha/beta we use an exact match. For snapshots -- snapshot # range. Otherwise an X.Y.* range. |