diff options
Diffstat (limited to 'README-DEV')
| -rw-r--r-- | README-DEV | 212 |
1 files changed, 115 insertions, 97 deletions
@@ -1,102 +1,120 @@ -This document describes how libpq was packaged for build2. In particular, -this understanding will be useful when ugrading to a new upstream version. - -The original libpq library is packaged together with the PostgreSQL server and -client utilities. Most of the libpq source files are located in the -src/interfaces/libpq/ directory. Some .c files are symlinked (copied on -Windows) from other src/ subdirectories during make. So run configure script -in the package root, run make in src/interfaces/libpq/ and then copy source -files and symlink targets into libpq/ directory of the build2 package. Note -that to obtain the full set of source files that includes Windows-specific -ones, you should perform these steps in the MSYS2/MinGW environment. Copy -Windows-specific files and strlcpy.c to libpq/win32/ and libpq/non-bsd/ -directories respectively. Also copy src/port/pthread-win32.h to libpq/ (next -to pthread-win32.c). - -Note that the library name in the .def file takes precedence over the one -specified in the linker command line for both VC and MinGW GCC. So we comment -it out in libpq/win32/libpqdll.def. - -Besides libpq-specific files some of the PostgreSQL common headers are also -required. All of them except one (see below) are located in src/include/ -subdirectories. Copy them into the libpq/postgresql/ directory, preserving the -original directory structure, with the following exceptions: - - * src/include/pg_config_os.h - - This is a symlink referencing the target-specific header in the - src/include/port/ directory, created by the configure script. Copy these - headers under postgresql/port/*/pg_config_os.h path names for the supported - target classes. For example, copy src/include/port/linux.h to - libpq/postgresql/port/linux/pg_config_os.h. During compilation the - '-I.../libpq/postgresql/port/linux' option will be passed to the compiler, - so the appropriate pg_config_os.h is picked up. - - * src/port/pg_config_paths.h - - This file is generated by src/port/Makfile and defines several directory - path macros. Only SYSCONFDIR macro is used in libpq source files. Make the - file empty and place it into libpq/postgresql directory. During compilation - the macro will be defined with -DSYSCONFDIR preprocessor option. - - * src/include/pg_config.h.in, - src/include/pg_config.h.win32 - pg_config_ext.h.in - pg_config_ext.h.win32 - - Use as sources for the manual creation of libpq/postgresql/pg_config.h and - libpq/postgresql/pg_config_ext.h, that are used for all target systems. - Also add the source headers to the package appending the '.orig' extension - to their names. - -By default the original package installs the library into the /usr/local/pgsql -directory. There are also several 'unofficial API' headers installed into the -/usr/local/pgsql/include/internal directory. If installing at some custom -location that has no postgresql or pgsql words in its path, then it becomes -<custom-location>/include/postgresql/internal. We currently do not install -unofficial API headers. - -For the record, the PostgreSQL binary and development packages install -libraries and headers into the following directories: - -Debian/Ubuntu: - /usr/lib/x86_64-linux-gnu - /usr/include/postgresql - /usr/include/postgresql/internal - -Fedora/RHEL: - /usr/lib64 - /usr/include - /usr/include/pgsql/internal - -When merge libpq build2 package with a new version of the original package -make sure that all the preprocessor include directives reference the packaged -header files, rather than PostgreSQL headers that are installed into the -system. It's easy to miss some headers in the package if the PostgreSQL -development package is installed on the host. To verify the correctness you -can build the merged project, concatenate the produced .d files, sort the -resulting file removing duplicates and edit the result, leaving only the -system headers. Afterwards grep through the remained headers for the -'PostgreSQL' pattern: +This document describes an approach applied to packaging PostgreSQL for +build2. In particular, this understanding will be useful when upgrading to a +new upstream version. -$ cat `find . -name '*.d'` | sort -u >headers -$ emacs headers # Edit, leaving system headers only. -$ fgrep PostgreSQL `cat headers` +The upstream package contains the PostgreSQL server, libpq library, and client +utilities. Currently, we only package libpq (see libpq/README-DEV for +details). + +We add the upstream package as a git submodule and symlink the required files +and subdirectories into the build2 package subdirectories. Then, when required, +we "overlay" the upstream with our own headers, placing them into the library +directory. + +Note that symlinking upstream submodule subdirectories into a build2 package +subdirectory results in creating intermediate build files (.d, .o, etc) inside +upstream directory while building the package in source tree. That's why we +need to make sure that packages do not share upstream source files via +subdirectory symlinks, not to also share the related intermediate files. If +several packages need to compile the same upstream source file, then only one +of them can symlink it via the parent directory while others must symlink it +directly. We also add the `ignore = untracked` configuration option into +.gitmodules to make sure that git ignores the intermediate build files under +upstream/ subdirectory. + +The upstream package can be configured to contain a specific feature set. We +reproduce the union of features configured for the upstream source package in +Debian and Fedora distributions. The configuration options defining these sets +are specified in the Debian's rules and Fedora's RPM .spec files. Note, +that at the time of this writing the latest packaged versions are different +(12.1 on Debian and 11.6 on Fedora). That's probably ok since 12.1 doesn't +introduce any new configuration options comparing to 11.6. These files can be +obtained as follows: + +$ wget http://deb.debian.org/debian/pool/main/p/postgresql-12/postgresql-12_12.1-1.debian.tar.xz +$ tar xf postgresql-12_12.1-1.debian.tar.xz debian/rules + +$ wget https://kojipkgs.fedoraproject.org/packages/libpq/11.6/1.fc32/src/libpq-11.6-1.fc32.src.rpm +$ rpm2cpio libpq-11.6-1.fc32.src.rpm | cpio -civ '*.spec' + +As a side note, on Debian and Fedora the source, library, and headers are +packaged as follows: + + src libpq headers +Debian/Ubuntu: postgresql-12 libpq5 libpq-dev +Fedora/RHEL: libpq libpq libpq-devel + +Search for the Debian and Fedora packages at https://packages.debian.org/search +and https://apps.fedoraproject.org/packages/. + +Here are the discovered configuration options. + +Debian: + + --with-icu --with-tcl --with-perl --with-python --with-pam --with-openssl + --with-libxml --with-libxslt --enable-nls --enable-integer-datetimes + --with-gssapi --with-ldap --enable-thread-safety + +Fedora: + + --with-ldap --with-openssl --with-gssapi --enable-nls --without-readline + +The union of these feature sets translates into the following options, after +suppressing the defaults: -Also make sure that the macros set in libpq/postgresql/pg_config.h is still up -to date. For that purpose obtain the macros that are used in the new source -base, then obtain the macros (un)defined in the current -libpq/postgresql/pg_config.h and compare the sets. That can be achieved -running the following commands in the build2 project root directory: + --with-icu --with-tcl --with-perl --with-python --with-pam --with-openssl + --with-libxml --with-libxslt --enable-nls --with-gssapi --with-ldap -$ for m in `cat libpq/postgresql/pg_config.h.in.orig libpq/postgresql/pg_config.h.win32.orig | sed -n 's/.*#\s*\(define\|undef\)\s\{1,\}\([_A-Z0-9]\{1,\}\)\(\s.*\)\{0,1\}$/\2/p' | sort -u`; do - if grep -q -e "\b$m\b" `find . -name '*.h' -a ! -name 'pg_config.h' -o -name '*.c'`; then - echo "$m" - fi - done >used-macros +We, however, drop the external dependencies that are not packaged for build2 +and end up with the following options: -$ cat libpq/postgresql/pg_config.h | - sed -n 's/#\s*\(define\|undef\)\s\{1,\}\([_A-Z0-9]\{1,\}\)\(\s.*\)\{0,1\}$/\2/p' | - sort -u >defined-macros + --with-openssl --without-readline --without-zlib -$ diff defined-macros used-macros +See the configuration options description at the "Installation Procedure" page +(https://www.postgresql.org/docs/current/install-procedure.html). + +Normally, when packaging a project, we need to replace some auto-generated +headers with our own implementations and deduce compilation/linking options. +For PostgreSQL we can rely on the configure.in, GNUmakefile.in, and the +manually maintained makefiles for that. In practice, however, that can be +uneasy and error prone, so you may also need to refer to auto-generated +header files or to see the actual compiler and linker command lines in the +build log. If that's the case, you can configure/build the upstream package on +the platform of interest running the following commands in the upstream +project root directory. + +On POSIX and for MinGW GCC: + +$ mkdir build +$ cd build +$ ../configure --with-openssl --without-readline --without-zlib >build.log 2>&1 +$ cd src/interfaces/libpq +$ make >>../../../build.log 2>&1 + +See the "Installation from Source Code" page +(https://www.postgresql.org/docs/current/installation.html) for details. + +For MSVC: + +Add bison and flex to PATH, if building in the git repository. Install +OpenSSL. + +> cd src\tools\msvc + +Edit config_default.pl to enable OpenSSL: + + openssl => 'c:\OpenSSL', # --with-openssl=<path> + +> build libpq >>build.log 2>&1 + +See the "Building with Visual C++ or the Microsoft Windows SDK" page +(https://www.postgresql.org/docs/current/install-windows-full.html) for +details. + +When the packaging is complete, build all the project packages in source tree +and make sure that no PostgreSQL headers are included from the system, running +the following commands from the project root: + +$ cat `find . -name '*.d'` | sort -u >headers +$ emacs headers # Edit, leaving system headers only. +$ fgrep PostgreSQL <headers |