Updates for 0.5.0 release0.5.0

1 files changed, 148 insertions, 254 deletions

diff --git a/release.txt b/release.txt
index ad06109..5bdead3 100644
--- a/release.txt
+++ b/release.txt
@@ -1,363 +1,257 @@
-TODO
-====
+@@ No upgrade testing
 
-@@ No upgrade testing (see old test-upgrade script).
+- Replace 0.5.0 and 0.5 in this document with the new version.
 
-@@ Not saving in previous toolchains (see old install script).
+- Remove etc/ and private/ from modules in etc/git/modules to reduce
+  noise during stat. Also, review for any new modules.
 
-@@ Using FreeBSD for Clang/libc++ testing, both 3.4 (earliest we support)
-   and 3.7.
+- Review '@@' items [private/ excluded, at least look for @@ TMP]
 
-Setup
-=====
-
-* Make symlinks to development b, bpkg in /usr/local/bin/, used as the latest
-  toolchain.
-
-* Set passwordless sudo, logins to freebsd, cppget1, rbook (Mac OS)
-
-  For Mac OS:
-    echo "boris ALL=(ALL) NOPASSWD:ALL" >>/etc/sudoers
-
-- Boot freebsd VM, make sure data/time is correct.
-
-Notes
-=====
-
-* Be very careful with submodules if fixing anything, remember to update
-  build2-toolchain.
-
-* Be very carefult with regenerating documentation, ODB files, both in
-  individual projects and build2-toolchain.
-
-Stage
-=====
-
-- To stage build2
-
-  ~ make sure build2-toolchain is ready (submodules, doc/odb regen)
-  ~ update BUILD2_REPO URL in build2-toolchain/build* scripts if needed
-
-  To stage to stage.build2.org:
-
-    ~ to exclude baseutils/mingw, add -p
-
-    etc/stage
+  etc/review | less -R
 
-  To stage to cppget.org/queue:
+- Identify packages that will be released (see etc/stage). They should all be
+  staged and built successfully, including submodule-updated build2-toolchain
+  and baseutils/mingw. Review stageing/0/ and staging/1/ for anything stray.
 
-    ~ note: assuming brep running on queue can handle it
+- Update and test local builds:
 
-    etc/stage -q
+  b test: build2/ bpkg/ brep/ bbot/ libbutl/ libbpkg/ libbbot/
+  b msvc-linux/
 
-- To stage hello
+- Stage hello projects and test:
 
   cd hello
-  ./stat
+  ./foreach-git pull
+  ./foreach-git stat
   ./stage
 
-  To run intro on staged:
-
-    etc/intro -s https://build2.org/pkg/1/stage
-
-Test :
-
- + Local with cli disables
-
- + Local with Intel icc
-   ~ fails with ICE on build2
-
- + Local with gcc 4.8
-
- + Local brep build:
-
-   bpkg create -d brep cc config.cc.poptions="-I$(apxs -q includedir)"
-   cd brep
-   bpkg add https://stage.build2.org/1
-   bpkg fetch
-   bpkg build brep ?sys:libapr1 ?sys:libpq
-
-Cross:
-
- + Test VC cross-build
-
- + Test MinGW cross-build (build2 build fails on GC 4.8.2, using ODB cross)
-
-
-Bootstrap (with upgrade procedures):
-
- + CentOS GCC 4.8
-
- + FreeBSD 10 Clang 3.4 (clang++)
-
- + FreeBSD 10 Clang 3.7 (clang++37) with -W -Wall (mod build.sh) + brep:
-
-   bpkg create -d brep cc config.cxx=clang++37 \
-     config.cc.poptions="-I$(apxs -q includedir)"
-   cd brep
-   bpkg add https://stage.build2.org/1
-   bpkg fetch
-   bpkg build brep ?sys:libapr1 ?sys:libpq
-
- + Windows7-64 VC14u2
-
- + WindowsXP-32 MinGW (build2-mingw)
-
- + Windows MSYS2
-
- + Mac OS Xcode/Clang (and g++ alias)
-
-Upgrade brep on stage
-
-Prelude
-=======
-
-+ Remove etc/, build2-toolchain from modules in etc/git/modules.
-
-+ Review '@@' items [note: etc, private excluded]
-
-  etc/review | less -R
-
-+ Make sure Intro is still accurate wrt output, etc. Use -s -p switches to
-  etc/intro to compare (after running hello/stage):
-
+  cd ..
   etc/intro -s `pwd`/hello/repository/1/
+  etc/intro -s https://build2.org/pkg/1/stage
 
-+ If new preprequisites/version, need to update manifests. See
-  private/build2-projects.txt to review.
-
-+ Increment version to final.
+- Update NEWS files in all project that will have non-pre-release version.
 
-  ~ update and use etc/version script
+- Review documentation for (1) sample output changes and (2) still being
+  relevant/making sense:
 
-  ~ update versions/dates for man generation (cli.sh in build2, bpkg, brep,
-    and built2-toolchain)
+  - Testscript manual (for 1, see hello/hello-testscript).
 
-+ Update BUILD2_REPO URL in build2-toolchain/build* scripts to alpha, also
-  grep docs for 'stage.' in case got copied.
+  - Install guide for (for 1 & 2).
 
-+ Close schema versions, review schema changlog difference from previous
-  release (tag) for any data migration that may be required (@@ would also
-  need to test this; see upgrade test).
+  - Introduction for (for 1 & 2). Use intro script to get output:
 
-+ Update NEWS files.
+    etc/intro -s -p https://build2.org/pkg/1/stage
 
-+ Need to update submodules? (remember to push first)
+- Change to final versions for all packages being released, some by hand,
+  some with version scripts (see etc/stage for list + msvc-linux).
 
-+ Need to regenerate ODB files, CLI documentation, in build2-toolchin?
+    - Close schema versions. Review schema changlog difference from previous
+      release (tag) for any data migration that may be required (would also
+      need to test this).
 
-Procession
-==========
+    - Push and update all submodules in buil2-toolchain
 
-+ Stage final packages (remove -p if need baseutils/mingw)
+    - Regenerate all odb & docs (in both packages and inside build2-toolchain)
 
-  etc/stage -p
+    - Bootstrap then update and test local
 
-- Upgrade brep on lists.build2, lists.cppget from stage
+      @@ Need to manually clean old the project builds because they
+         pick up old version.hxx (during dep extraction).
 
-  ~ assumming it can handle old repositories (should normally be the case)
+      cd build2
+      b clean
+      git cout build2/version.hxx build2/b-options.?xx
+      ./bootstrap.sh g++-6
 
-  ~ backup config if changed
+      cd ..
+      b-boot '{clean update}(libbutl/ build2/)'
 
-    cd private/
-    scp -r lists.build2.org:/home/brep/config lists.build2.org/home/brep/
-    scp -r lists.cppget.org:/home/brep/config lists.cppget.org/home/brep/
+      b clean: bpkg/ brep/ bbot/ libbpkg/ libbbot/ msvc-linux/
 
-  ~ note: on cppget we have queue:
+      Then the local test step above.
 
-    install/bin/brep-migrate -n brep_queue
-    install/bin/brep-load -n brep_queue config/brep-queue-load.conf
+    - Cleanup staging repo:
 
-+ Queue final packages
+      rm -r staging/0/*
+      rm -r staging/repository/1/*/
 
-  etc/stage -p -q
+    - Restage (step 1 above, with -b):
 
-+ Smoke-test any last-minute changes
+      etc/stage -b
 
-+ Move packages from queue to alpha, cleanup obsolete, regenerate.
+    - Check /0/ and /1/ for anything stray
 
-  cd cppget.org
-  git -C repository status
-  ./update
+- Upgrade brep on cppget.org (queue and main repo) using stage
 
-+ Publish to production (build2.org/cppget.org)
+- Queue packages:
 
-  etc/publish
+  - Change BUILD2_REPO in build2_toolchain build script to queue
 
-+ Smoke-test
+  - Cleanup queue repo:
 
-+ Publish hello repository
+    rm -rf cppget.org/0/*
+    rm -rf cppget.org/repository/1/queue/*/
 
-  cd hello
-  ./stage -p
+  - Queue
 
-  Test intro steps:
+    etc/stage -q -b
 
-  etc/intro -s https://build2.org/pkg/1/hello
+  - check /0/ and /1/ for anything stray
 
-- Commit and push git repositories:
+  - Make sure bot builds are successful
 
-+  ~ cppget.org (commit individually and push)
+- Upgrade brep on build2.org (hello) using queue
 
-   ~ hello project and respository (add & commit individual packages)
+- Publish and test hello
 
     cd hello
     ./foreach-git stat
+    ./stage -p
 
-    ./foreach-git -x libhello-1.0 tag -a build2-X.Y -m "Tag for build2 version X.Y"
-    git -C libhello-1.0 tag -a 1.0/build2-X.Y -m "Tag for build2 version X.Y"
-    ./foreach-git push --tags
+    cd ..
+    etc/intro -s https://build2.org/pkg/1/hello
 
-+ Tag (review tag_modules) and push.
+  - Drop build database (package versions do not change)
 
-  etc/tag
-  ./push.sh
+  - Make sure bot builds are successful
 
-- Tag and increment version for each published package (see
-  private/build2-projects.txt)
+- Publish and test toolchain
 
-=============================================================================
+  - Change BUILD2_REPO in build2_toolchain build script to public
 
-@@ When copying to production need to make sure we don't override existing
-   versions of prerequisite packages that haven't changed (but which we still
-   stage/queue).
+  - Regenerate build2-toolchain package in cppget.org/0/
 
-=============================================================================
+    etc/stage -p
 
-@@ Move up.
+  - Move packages (manually for now):
 
-- Update everything:
+    - old/replaced/FTB packages either to legacy or delete
 
-  b libbutl/ build2/ libbpkg/ bpkg/ brep/ build2-toolchain-default/
+    - from queue to appropriate repositories (don't override existing)
 
-- Run tests for each project:
+  - Regenerate the repository
 
-  * b 'test(libbutl/ libbpkg/)'
+    cd cppget.org
+    git -C repository add .
+    git -C repository status
+    ./update
 
-  * Manually in build2/
+  - Disable the queue toolchain on bbots (moved packages from queue/),
+    delete distribution
 
-  * Manually in bpkg/; test (valgrind/asan), publish, test remote with -v,
-    FreeBSD tools:
+    echo disabled | ssh build2.org tee >/dev/null \
+      /var/www/download.build2.org/public/queue/toolchain.sha256
 
-    ./test.sh -v --remote --fetch fetch --fetch-option --no-verify-peer \
-      --tar bsdtar --sha256 sha256-freebsd
+    ssh build2.org rm -r /var/www/download.build2.org/public/queue/*/
 
-  * Test dev integration of brep/ on hello and cppget repos
+  - Publish everything to build2.org/cppget.org
 
-    First check if Apache2 configuration needs updating (INSTALL-DEV). Then
-    from build2/ work root:
+    etc/publish
 
-    # This will have broken links since we are not running it as /pkg/hello.
-    #
-    brep/migrate/brep-migrate --recreate
-    brep/load/brep-load hello/brep-loader-home.conf
-    sudo /etc/init.d/apache2 restart
+  - Make sure bot builds are successful, drop hello build database again.
 
-    brep/migrate/brep-migrate --recreate
-    brep/load/brep-load cppget.org/brep-loader-home.conf
-    sudo /etc/init.d/apache2 restart
+- Release
 
-    If need to drop the entire database, one way to do it is by executing:
+  - Update submodules (style/) in private/.
 
-    DROP OWNED BY <user>;
+  - Write release notes, use placeholder for announcement URL. Add link from
+    the doc page.
 
-Procession
-==========
+  - Any new FAQ entries or any other updates (main page, etc)? Any new
+    documentation to link from the doc page?
 
-X Generate distribution (use -t for toolchain-only, without brep). The
-  result is placed into the queue.
+  - Update download page.
 
-  etc/dist 2>&1 | tee dist.log
+    cat `ls -1 *.sha256`
 
-X Test upgrade from previous version (replace Y in 0.Y.0).
+  - Regenerate documentation (./cli.sh) for all hosts.
 
-  etc/test-upgrade -c cxx \
-    -c config.cxx.loptions=-L/usr/local/lib \
-    -c config.cxx.poptions="-I/usr/include/apache2 -I/usr/include/apr-1.0" \
-    /home/boris/work/build2/etc1/install/0.Y.0
+  - Test locally, publish, test production
 
-X Update and test hello repository
+    cd private/
+    ./publish --dry-run
+    ./publish
 
-  cd hello
-  ./foreach-git status
-  ./stage -p
+  - Write and send the announcement to mailing lists.
 
-X Test Intro steps on the local hello repository
+    Remember upgrade instructions.
 
-  etc/intro `pwd`/hello/repository/1/
+    cat `ls -1 *.sha256`
 
-X Stage queue to cppget.org
+  - Patch the announcement URL in release notes, re-publish.
 
-  etc/stage --dry-run
-  etc/stage
+  - Announce on reddit and other places (see doc/ann/).
 
-- Determine the earliest supported backwards-compatible toolchain (see
-  requires: in manifests), update etc/test
+  - Commit and tag private/
 
-- Test
+    git tag -a 0.5.0 -m "Tag version 0.5.0" && git push --tags
 
-  etc/test 2>&1 | tee test.log
-  grep -i warning test.log
+- Tag & Commit hello
+
+  - commit hello/repository/ (see commit history for procedure)
+
+  cd hello
+  ./foreach-git stat
 
+  ./foreach-git -x libhello-1.0 tag -a build2-0.5.0 -m "Tag for build2 version 0.5.0"
+  git -C libhello-1.0 tag -a 1.0/build2-0.5.0 -m "Tag for build2 version 0.5.0"
+  ./foreach-git push --tags
 
-- Test production and save a copy of toolchain in etc1/install/X.Y.Z
+- Tag & Commit
 
-  etc/install -t -i "etc1/install/`cat build2-toolchain/version`" \
-    https://download.build2.org/0.Y/build2-toolchain-0.Y.0.tar.gz \
-    https://pkg.cppget.org/1/alpha
+  - commit cppget.org/repository/ (see commit history for procedure)
 
-- Upgrade brep on cppget.org (via package/alpha upgrade if possible),
-  verify works.
+  - Tag and push all released packages, some by hand, some with tag scripts
+    (see etc/stage for list).
 
-  Note that here we also have queue:
+    cd <proj>
 
-  install/bin/brep-migrate -n brep_queue
-  install/bin/brep-load -n brep_queue config/brep-queue-load.conf
+    v="$(sed -n -re 's/version: ([^ ]+)/\1/p' manifest)"; echo $v; read; \
+      git tag -a $v -m "Tag version $v" && git push --tags
 
-  Backup configs if changed:
+    For build2 projects, review tag_modules in etc/git/modules, then:
 
-  cd private/
-  scp -r lists.cppget.org:/home/brep/config lists.cppget.org/home/brep/
+    ./tag.sh 0.5.0
+    ./push.sh
 
-Release
-=======
+- Increment versions and open master for business
 
-+ Update style/ submodule in private/
+  - Change to next development versions for all released packages, some by
+    hand, some with version scripts (see etc/stage for list + msvc-linux).
 
-+ Write release notes, use placeholder for announcement URL. Add link from
-  the doc page.
+    cd <proj>
 
-- Any new FAQ entries or any other updates (main page, etc)?
+    ~ edit manifest
 
-+ Update download page.
+    v="$(sed -n -re 's/version: ([^ ]+)/\1/p' manifest)"; echo $v; read; \
+      git ci -a -m "Bump version to $v, master is open for business" && git push
 
-+ Regenerate (./cli.sh) all hosts.
+  - In build2-toolchain, update all submodules (including libodb*, etc),
+    change BUILD2_REPO to staging.
 
-+ Test local, publish, test production:
+  - Regenerate odb & cli docs everywhere (packages and build2-toolchain).
 
-  cd private/
-  ./publish --dry-run
-  ./publish
+  - Update/test local (see early steps of this list).
 
-+ Write and send announcements
+  - Make symlinks for new version in baseutils (for both baseutils and mingw,
+    the idea is that we will start with those and maybe upgrade during
+    development). Update version file.
 
-  Remember upgrade instructions.
+  - Restage
 
-  cat `ls -1 *.sha256`
+    rm -r staging/0/*
+    rm -r staging/repository/1/*/
 
-+ Patch the announcement URL in release notes, re-publish.
+    etc/stage -b
 
-- Announce on reddit and other places (doc/ann).
+    - Make sure bbot builds of the new development snapshot are successful.
 
-Postlude
-========
+- Commit etc/
 
-- Increment and open schema versions. Maybe not. Maybe we should only do that
-  when there is a model change.
+  - Restore changes to etc/git/modules
 
-+ Increment project versions to alpha1, change BUILD2_REPO to staging,
-  open master for business.
+  - Change all '+ ' back to '- '.
 
-- Commit changes to etc/, add it back to etc/git/modules, and tag:
+  - Commit and tag
 
-  git tag -a X.Y.Z -m "Tag version X.Y.Z"
+    git tag -a 0.5.0 -m "Tag version 0.5.0" && git push --tags