Further work on packaging guide

1 files changed, 173 insertions, 18 deletions

diff --git a/doc/packaging.cli b/doc/packaging.cli
index db3088d..f456479 100644
--- a/doc/packaging.cli
+++ b/doc/packaging.cli
@@ -3271,10 +3271,14 @@ locally: distribution}.|
 If everything looks good, then you are done: the package submission will be
 reviewed and, if there are no problems, moved to \l{https://cppget.org
 cppget.org}. If there are problems, then an issue will be created in the
-package repository with the review feedback. @@ Link to new version section.
+package repository with the review feedback. In this case you will need to
+\l{#core-release-publish-new-revision release and publish a version revision}
+to address these problems. But in both cases you should first read through
+\l{#core-release-publish-management Package version management} to understand
+the recommended \"version lifecycle\" of a third-party package.
 
 
-\h2#core-release-publish-management|Package version management|
+\h#core-release-publish-management|Package version management|
 
 Once we have pushed the release commit, in order to preserve continous
 versioning, no further changes should be made to the package without also
@@ -3286,23 +3290,23 @@ version provided they don't affect the package. For example, you may keep a
 package. Updating such a file without changing the version is ok since the
 package remains unchanged.|
 
-While with our own projects we can change the versions as we see fit, with
+While in our own projects we can change the versions as we see fit, with
 third-party projects the versions are dictated by the upstream and as a result
-we are limited to which versions we can use to fix issues in the package
-itself. It may be tempting (and maybe even conceptually correct) to release a
-patch version for our own fixes, nut we will be in trouble if later upstream
-releases the same patch version but with a different set of changes (plus the
-users of our package may wonder where did this version come from). As a
-result, we should only change the major, minor, or patch components of the
-package version in response to the corresponding upstream releases. For fixes
-to the package itself we should instead use version revisions.
+we are limited to what we can use to fix issues in the package itself. It may
+be tempting (and maybe even conceptually correct) to release a patch version
+for our own fixes, however, we will be in trouble if later upstream releases
+the same patch version but with a different set of changes (plus the users of
+our package may wonder where did this version come from). As a result, we
+should only change the major, minor, or patch components of the package
+version in response to the corresponding upstream releases. For fixes to the
+package itself we should instead use version revisions.
 
 \N|Because a revision replaces the existing version, we should try to limit
 revision changes to bug fixes and preferably only to the package
-\"infrastructure\" (\c{buildfiles}, etc). Fixes to upstream source code should
-be limited to critical bug fixes, preferably backported from upstream. To put
-it another way, changes in a revision should have an even more limited
-scope than a patch release.|
+\"infrastructure\" (\c{buildfiles}, \c{manifest}, etc). Fixes to upstream
+source code should be limited to critical bugs, preferably be backported from
+upstream. To put it another way, changes in a revision should have an even
+more limited scope than a patch release.|
 
 Based on this, the recommended \"version lifecycle\" for a third-party
 package is as follows:
@@ -3331,15 +3335,166 @@ the \c{2.1.Z} release series and make a revision or patch version there. See
 @@ for details.
 
 
-\h2#core-release-publish-revision|Release and publish revision|
+\h2#core-release-publish-new-revision|Release and publish new revision|
 
+As discussed in \l{#core-release-publish-management Package version
+management}, we release revisions to fix issues in the package
+\"infrastructure\" (\c{buildfiles}, \c{manifest}, etc) as well as critical
+bugs in upstream source code.
 
+In the revision phase of the package version lifecycle (i.e., when the version
+does not end with \c{-a.0.z}), every commit must be accompanies by the
+revision increment to maintain continous verisions. As a result, each revision
+release commit also contains the changes in this revision. Below is a typical
+workflow for releasing and publishing the revision:
+
+\
+$ # make changes
+$ # test locally
+$ git add .
+$ bdep release --revision --show-push
+$ # review commit
+$ git push ...
+$ # test with CI
+$ bdep publish
+\
+
+Customarily, the revision commit message has the \c{\"Release version
+X.Y.Z+R\"} summary as generated by \c{bdep-release} followed by the
+description of changes organized in a list of there are several. For example:
+
+\
+Release version 2.1.0+1
+
+- Don't compile port/strlcpy.c on Linux if GNU libc is 2.38 or newer
+  since it now provides the strl*() functions.
+
+- Switch to using -pthread instead of -D_REENTRANT/-lpthread.
+\
+
+\N|The fact that all the changes must be in a single commit is another reason
+to avoid substantial changes in revisions.|
+
+Note also that you can make multiple commits while developing and testing the
+changes for a revision in a separate branch. However, once they are ready for
+a release, they need to be squashed into a single commit. The
+\l{bdep-release(1)} command provides the \c{--amend} and \c{--squash} options
+to automate this. For example, here is what a workflow with a separate branch
+might look like:
+
+\
+$ git checkout -b wip-2.1.0+1
+
+$ # make strl*() changes
+$ # test locally
+$ git commit -a -m \"Omit port/strlcpy.c if glibc 2.38 or newer\"
+$ git push -u
+$ # test with CI
+
+$ # make pthread changes
+$ # test locally
+$ git commit -a -m \"Switch to using -pthread\"
+$ git push
+$ # test with CI
+
+$ git checkout master
+$ git merge --ff-only wip-2.1.0+1
+$ bdep release --revision --show-push --amend --squash 2
+$ # review commit
+$ # test locally
+$ git push ...
+$ # test with CI
+$ bdep publish
+\
 
-@@ What if need to release a revision/patch release for older version
-   (e.g., like in Thrift)?
+
+\h2#core-release-publish-new-version|Release and publish new version|
+
+As discussed in \l{#core-release-publish-management Package version
+management}, we release new versions in reponse to the corresponding upstream
+releases.
+
+The amount or work required to upgrade a package to a new upstream version
+depends on the extend of changes in the new version.
+
+On one extreme you may have a patch release which fixes a couple of bugs in
+the upstream source code without any changes to the set of source files,
+upstream build system, etc. In such cases, upgrading a package is a simple
+matter of creating a new work branch, pointing the \c{upstream} \c{git}
+submodule to the new release, running tests, and releasing and publishing a
+new package version. @@ make list with links.
+
+On the other extreme you may have a new major upstream release which is
+essentially a from-scratch rewrite with new source code layout, different
+upstream build system, etc. In such cases it may be easier to likewise start
+from scratch. Specifically, create a new work branch, point the \c{upstream}
+\c{git} submodule to the new release @@ link, delete the existing package, and
+continue from \l{#core-package Create package and generate \c{buildfile}
+templates}.
+
+Most of the time, however, it will be something in between where you may need
+to tweak a few things here and there, such as adding symlinks to new source
+files (or removing old ones), tweaking the \c{buildfiles} to reflect changes
+in the upstream build system, etc.
+
+The following sections provide a checklist-like sequence of steps that can be
+used to review upstream changes with links to the relevant earlier sections in
+case undjustments are required.
+
+@@ New branch? But can also do on master theoretically. This closes the
+door to stopping/dropping the work so I think create a branch. Also can
+be base for PR. But we don't need to squash.
+
+@@ Maybe in the initial instructions makes sense to identify and note
+the point where to merge the branch to master if working in a branch
+(e.g., because of a new version).
+
+
+\h2#core-release-publish-old-version|Release and publish version/revision in old release series|
+
+As discussed in \l{#core-release-publish-management Package version
+management}, if we have already switched to the next upstream version in the
+\c{master} branch, we cannot go back and release a new version or a revision
+for an older release series on the same branch. Instead, we need to create a
+seperate, long-lived branch for this work.
+
+As an example, let's say we need to release another revision or a patch
+version for an already released \c{2.1.0} while our \c{master} branch has
+already moved on to \c{2.2.0}. In this case we create a new branch, called
+\c{2.1}, to continue with the \c{2.1.Z} release series. The starting point of
+this branch should be the latest released version/revision in the \c{2.1}
+series. Let's say in our case it is \c{2.1.0+2}, meaning we have released two
+revisions for \c{2.1.0} on the \c{master} branch before upgrading to
+\c{2.2.0}. Therefore we use the \c{v2.1.0+2} release tag to start the
+\c{2.1} branch:
+
+\
+$ git checkout -b 2.1 v2.1.0+2
+\
+
+Once this is done, we continue with the same steps as in
+\l{#core-release-publish-new-revision Release and publish new revision} or
+\l{#core-release-publish-new-version Release and publish new version} except
+that we never merge this branch to \c{master}. If we ever need to release
+another revision or version in this release series, then we continue using
+this branch. In a sense, this branch becomes the equivalent of the \c{master}
+branch for this release series and you should treat it as such (once
+published, never delete, rewrite its history, etc).
+
+\N|It is less likely but possible that you may need to release a new minor
+version in an old release series. For example, the master branch may have
+moved on to \c{3.0.0} and you want to release \c{2.2.0} after the already
+released \c{2.1.0}. In this case it makes sense to call the branch \c{2} since
+it corresponds to the \c{2.Y.Z} release series. If you already have the
+\c{2.1} branch, then it makes sense to rename it to \c{2}.|
+
+
+
+@@ Enforce continous versioning?
 
 @@ When do we transfer the repository to build2-packaging? Should not
    publish until then.
+
 @@ GH issue #?? has some notes.
 
 ========