From 1741b21e27945da8c55069fa7cd66e55757be053 Mon Sep 17 00:00:00 2001 From: Boris Kolpackov Date: Wed, 9 Aug 2023 09:36:57 +0200 Subject: Auto-extract function documentation from functions-*.cxx --- doc/manual.cli | 1062 +++++++++++++++++++++++++++++++++----------------------- 1 file changed, 622 insertions(+), 440 deletions(-) (limited to 'doc/manual.cli') diff --git a/doc/manual.cli b/doc/manual.cli index 5ccf351..c07074e 100644 --- a/doc/manual.cli +++ b/doc/manual.cli @@ -3613,8 +3613,9 @@ info $path.directory($src_base) # $src_base info $path.base($path.leaf($src_base)) # foo \ -Note that functions in \c{build2} are \i{pure} in a sense that they do not -alter the build state in any way. +Note that the majority of functions in \c{build2} are \i{pure} in a sense that +they do not alter the build state in any way (see \l{#functions Functions} for +details). \N|Functions in \c{build2} are currently defined either by the build system core or build system modules and are implemented in C++. In the future it will @@ -5510,261 +5511,363 @@ configuration header into two, one public and installed while the other private.| - -\h1#attributes|Attributes| +\h1#targets|Targets and Target Types| \N{This chapter is a work in progress and is incomplete.} -The only currently recognized target attribute is \c{rule_hint} which -specifies the rule hint. Rule hints can be used to resolve ambiguity when -multiple rules match the same target as well as to override an unambiguous -match. For example, the following rule hint makes sure our executable is -linked with the C++ compiler even though it only has C sources: -\ -[rule_hint=cxx] exe{hello}: c{hello} -\ +\h#targets-types|Target Types| +A target type is part of a target's identity. The core idea behind the concept +of target types is to abstract away from file extensions which can vary from +project to project (for example, C++ source files extensions) or from platform +to platform (for example, executable file extensions). It also allows us to +have non-file-based targets. -\h1#name-patterns|Name Patterns| +Target types form a \i{base-derived} inheritance tree. The root of this tree +is the abstract \c{target{\}} type. The \c{build2} core defines a number of +standard target types, such as \c{file{\}}, \c{doc{\}}, and \c{exe{\}}. Build +system modules can define additional target types that are based on the +standard ones (or on types defined by other modules). For example, the \c{c} +module that provides the C compilation support defines the \c{h{\}} and +\c{c{\}} target types. Finally, \c{buildfiles} can derive project-local target +types using the \c{define} directive. -For convenience, in certain contexts, names can be generated with shell-like -wildcard patterns. A name is a \i{name pattern} if its value contains one or -more unquoted wildcard characters or character sequences. For example: +\N|If a target type represents a file type with a well-established extension, +then by convention such an extension is used as the target type name. For +example, the C language header and source files use the \c{.h} and \c{.c} +extensions and the target types are called \c{h{\}} and \c{c{\}}. -\ -./: */ # All (immediate) subdirectories -exe{hello}: {hxx cxx}{**} # All C++ header/source files. -pattern = '*.txt' # Literal '*.txt'. -\ +Speaking of conventions, as you may have noticed, when mentioning a target +type we customarily add \c{{\}} after its name. We found that this helps with +comprehension since target type names are often short (you can also search for +\c{{} to narrow it down to target types). In a way this is a similar +approach to adding \c{()} after a function name except here we use \c{{\}}, +which mimics target type usage in target names, for example \c{c{hello\}} for +\c{hello.c}.| -Pattern-based name generation is not performed in certain contexts. -Specifically, it is not performed in target names where it is interpreted -as a pattern for target type/pattern-specific variable assignments. For -example. +The following listing shows the hierarchy of the standard target types defined +by the \c{build2} core (the abstract target types are marked with \c{*}) while +the following sections describe each standard target type in detail. For +target types defined by a module refer to the respective module documentation. \ -s = *.txt # Variable assignment (performed). -./: cxx{*} # Prerequisite names (performed). -cxx{*}: dist = false # Target pattern (not performed). + .-----target*------------. + | | | + mtime_target*---. alias fsdir + | | | + path_target* group dir + | + .---------file----. + | | | + .----doc-----. exe buildfile + | | | +legal man manifest + | + man \ -In contexts where it is performed, it can be inhibited with quoting, for -example: +While target types replace (potentially variable) extensions, there still +needs to be a mechanism for specifying them since in most cases targets have +to be mapped to files. There are several ways this can be achieved. + +If a target type represents a file type with a well-established extension, +then such an extension is normally used by default and we don't need to take +any extra steps. For example the \c{h{\}} and \c{c{\}} target types for C +header and source files default to the \c{.h} and \c{.c} extensions, +respectively, and if our project follows this convention, then we can simply +write: \ -pat = 'foo*bar' -./: cxx{'foo*bar'} +exe{utility}: c{utility} h{utility} \ -The following wildcards are recognized: +And \c{c{utility\}} will be mapped to \c{utility.c} and \c{h{utility\}} \- +to \c{utility.h}. + +There are two variants of this default extension case: fixed extension and +customizable extension. A target type may choose to fix the default extension +if it's a bad idea to deviate from the default extension. A good example of +such a target is \c{man1{\}}, which fixes the default extension to be +\c{.1}. More commonly, however, a target will have a default extension but +will allow customizing it with the \c{extension} variable. + +A good example where extension customization is often required are the +\c{hxx{\}} and \c{cxx{\}} target types for C++ header and source files, which +default to the \c{.hxx} and \c{.cxx} extensions, respectively. If our project +uses other extensions, for example, \c{.hpp} and \c{.cpp}, then we can adjust +the defaults (typically done in \c{root.build}, after loading the \c{cxx} +module): \ -* - match any number of characters (including zero) -? - match any single character -[...] - match a character with a bracket expression +hxx{*}: extension = hpp +cxx{*}: extension = cpp \ -\N|Currently only literal character and range bracket expressions are -supported. Specifically, no character or equivalence classes, etc., are -supported nor the special characters backslash-escaping. See the \"Pattern -Matching Notation\" section in the POSIX \"Shell Command Language\" -specification for details.| - -Note that some wildcard characters may have special meaning in certain -contexts. For instance, \c{[} at the beginning of a value will be interpreted -as the start of the attribute list while \c{?} and \c{[} in the eval context -are part of the ternary operator and value subscript, respectively. In such -cases the character will need to be escaped in order to be treated as a -wildcard, for example: +Then we can write: \ -x = \[1-9]-foo.txt -y = (foo.\?xx) -z = ($foo\[123].txt) +exe{utility}: cxx{utility} hxx{utility} \ -If a pattern ends with a directory separator, then it only matches -directories. Otherwise, it only matches files. Matches that start with a dot -(\c{.}) are automatically ignored unless the pattern itself also starts with -this character. +And \c{cxx{utility\}} will be mapped to \c{utility.cpp} and \c{hxx{utility\}} +\- to \c{utility.hpp}. -In addition to the above wildcards, \c{**} and \c{***} are recognized as -wildcard sequences. If a pattern contains \c{**}, then it is matched just like -\c{*} but in all the subdirectories, recursively, but excluding directories -that contain the \c{.buildignore} file. The \c{***} wildcard behaves like -\c{**} but also matches the start directory itself. For example: +What about \c{exe{utility\}}, where does its extension come from? This is an +example of a target type with an extension that varies from platform to +platform. In such cases the extension is expected to be assigned by the rule +that matches the target. In the above example, the link rule from the \c{cxx} +module that matches updating \c{exe{utility\}} will assign a suitable +extension based on the target platform of the C++ compiler that it was +instructed to use. + +Finally, it is always possible to specify the file extension explicitly as +part of the target name. For example: \ -exe{hello}: cxx{**} # All C++ source files recursively. +exe{utility}: cxx{utility.cc} hxx{utility.hh} \ -A group-enclosed (\c{{\}}) pattern value may be followed by -inclusion/exclusion patterns/matches. A subsequent value is treated as an -inclusion or exclusion if it starts with a literal, unquoted plus (\c{+}) or -minus (\c{-}) sign, respectively. In this case the remaining group values, if -any, must all be inclusions or exclusions. If the second value doesn't start -with a plus or minus, then all the group values are considered independent -with leading pluses and minuses not having any special meaning. For regularity -as well as to allow patterns without wildcards, the first pattern can also -start with the plus sign. For example: +This is normally only needed if the default extension is not appropriate or if +the target type does not have a default extension, as is the case, for +example, for the \l{#targets-types-file \c{file{\}}} and \l{#targets-types-doc +\c{doc{\}}} target types. This mechanism can also be used to override the +automatically derived extension. For example: \ -exe{hello}: cxx{f* -foo} # Exclude foo if exists. -exe{hello}: cxx{f* +bar} # Include bar if exists. -exe{hello}: cxx{f* -fo?} # Exclude foo and fox if exist. -exe{hello}: cxx{f* +b* -foo -bar} # Exclude foo and bar if exist. -exe{hello}: cxx{+f* +b* -foo -bar} # Same as above. -exe{hello}: cxx{+foo} # Pattern without wildcards. -exe{hello}: cxx{f* b* -z*} # Names matching three patterns. +exe{($cxx.target.class == 'windows' ? utility.com : utility)}: ... \ -Inclusions and exclusions are applied in the order specified and only to the -result produced up to that point. The order of names in the result is -unspecified. However, it is guaranteed not to contain duplicates. The first -pattern and the following inclusions/exclusions must be consistent with -regards to the type of filesystem entry they match. That is, they should all -match either files or directories. For example: +\N|If you need to specify a name that does not have an extension, then end it +with a single dot. For example, for a header \c{utility} you would write +\c{hxx{utility.\}}. If you need to specify a name with an actual trailing dot, +then escape it with a double dot, for example, \c{hxx{utility..\}}. -\ -exe{hello}: cxx{f* -foo +*oo} # Exclusion has no effect. -exe{hello}: cxx{f* +*oo} # Ok, no duplicates. -./: {*/ -build} # Error: exclusion not a directory. -\ +More generally, anywhere in a name, a double dot can be used to specify a dot +that should not be considered the extension separator while a triple dot \- +which should. For example, in \c{obja{foo.a.o\}} the extension is \c{.o} and +if instead we wanted \c{.a.o} to be considered the extension, then we could +rewrite it either as \c{obja{foo.a..o\}} or as \c{obja{foo...a.o\}}.| -As a more realistic example, let's say we want to exclude source files that -reside in the \c{test/} directories (and their subdirectories) anywhere in the -tree. This can be achieved with the following pattern: +To derive a new target type in a \c{buildfile} we use the \c{define} +directive. Such target types are project-local, meaning they cannot be +exported to other projects. Typically this is used to provide a more +meaningful name to a set of files and also avoid having to specify their +extensions explicitly. Compare: \ -exe{hello}: cxx{** -***/test/**} +./: doc{README.md PACKAGE-README.md INSTALL.md} \ -Similarly, if we wanted to exclude all source files that have the \c{-test} -suffix: +To: \ -exe{hello}: cxx{** -**-test} +define md: doc +doc{*}: extension = md + +./: md{README PACKAGE-README INSTALL} \ -In contrast, the following pattern only excludes such files from the top -directory: + +\h2#targets-types-target|\c{target{\}}| + +The \c{target{\}} target type is a root of the target type hierarchy. It is +abstract and is not commonly used directly, except perhaps in patterns (target +type/pattern-specific variable, pattern rules). + + +\h2#targets-types-alias|\c{alias{\}} and \c{dir{\}}| + +The \c{alias{\}} target type is used for non-file-based targets that serve as +aliases for their prerequisite. + +\N|Alias targets in \c{build2} are roughly equivalent to phony targets in +\c{make}.| + +For example: \ -exe{hello}: cxx{** -*-test} +alias{tests}: exe{test1 test2 test3} \ -If many inclusions or exclusions need to be specified, then an -inclusion/exclusion group can be used. For example: - \ -exe{hello}: cxx{f* -{foo bar}} -exe{hello}: cxx{+{f* b*} -{foo bar}} +$ b test: alias{tests} \ -This is particularly useful if you would like to list the names to include or -exclude in a variable. For example, this is how we can exclude certain files -from compilation but still include them as ordinary file prerequisites (so -that they are still included into the source distribution): +An \c{alias{\}} target can also serve as an \"action\" if supplied with an ad +hoc recipe (or matched by an ad hoc pattern rule). For example: \ -exc = foo.cxx bar.cxx -exe{hello}: cxx{+{f* b*} -{$exc}} file{$exc} +alias{strip}: exe{hello} +{{ + diag strip $< + strip $path($<) +}} \ -If we want to specify our pattern in a variable, then we have to use the -explicit inclusion syntax, for example: +The \c{dir{\}} target type is a special kind of alias that represents a +directory. Building it means building everything inside the directory. See +\l{#intro-proj-struct Project Structure} for background. + +A target without a type that ends with a directory separator (\c{/}) is +automatically treated as \c{dir{\}}. For example, the following two lines are +equivalent: \ -pat = 'f*' -exe{hello}: cxx{+$pat} # Pattern match. -exe{hello}: cxx{$pat} # Literal 'f*'. +./: exe{test1 test2} +dir{./}: exe{test1 test2} +\ -pat = '+f*' -exe{hello}: cxx{$pat} # Literal '+f*'. +Omitting the target type in such situations is customary. -inc = 'f*' 'b*' -exc = 'f*o' 'b*r' -exe{hello}: cxx{+{$inc} -{$exc}} -\ -One common situation that calls for exclusions is auto-generated source -code. Let's say we have auto-generated command line parser in \c{options.hxx} -and \c{options.cxx}. Because of the in/out of source builds, our name pattern -may or may not find these files. Note, however, that we cannot just include -them as non-pattern prerequisites. We also have to exclude them from the -pattern match since otherwise we may end up with duplicate prerequisites. As a -result, this is how we have to handle this case provided we want to continue -using patterns to find other, non-generated source files: +\h2#targets-types-fsdir|\c{fsdir{\}}| + +The \c{fsdir{\}} target type represents a filesystem directory. Unlike +\c{dir{\}} above, it is not an alias and listing an \c{fsdir{\}} directory as +a prerequisite of a target will cause that directory to be created on +\c{update} and removed on \c{clean}. + +While we usually don't need to list explicit \c{fsdir{\}} prerequisites for +our targets, one situation where this is necessary is when the target resides +in a subdirectory that does not correspond to an existing source directory. A +typical example of this situation is placing object files into subdirectories. +Compare: \ -exe{hello}: {hxx cxx}{* -options} {hxx cxx}{options} +obj{foo}: c{foo} +sub/obj{bar}: c{bar} fsdir{sub/} \ -If all our auto-generated source files have a common prefix or suffix, then we -can exclude them wholesale with a pattern. For example, if all our generated -files end with the `-options` suffix: + +\h2#targets-types-mtime-path|\c{mtime_target{\}} and \c{path_target{\}}| + +The \c{mtime_target{\}} target type represents a target that uses modification +times to determine if it is out of date. The \c{path_target{\}} target type +represents a target that has a corresponding filesystem entry. It is derived +from \c{mtime_target{\}} and uses the modification time of that filesystem +entry to determine if the target is out of date. + +Both of these target types are abstract and are not commonly used directly, +except perhaps in patterns (target type/pattern-specific variable, pattern +rules). + + +\h2#targets-types-group|\c{group{\}}| + +The \c{group{\}} target type represents a user-defined explicit target group, +that is, a target that has multiple member targets that are all built together +with a single recipe. + +Normally this target type is not used to declare targets or prerequisites but +rather as a base of a derived group. If desired, such a derived group can be +marked with an attribute as \"see-through\", meaning that when the group is +listed as a prerequisite of a target, the matching rule \"sees\" its members, +rather than the group itself. For example: \ -exe{hello}: {hxx cxx}{** -**-options} {hxx cxx}{foo-options bar-options} +define [see_through] thrift_cxx: group \ -If the name pattern includes an absolute directory, then the pattern match is -performed in that directory and the generated names include absolute -directories as well. Otherwise, the pattern match is performed in the -\i{pattern base} directory. In buildfiles this is \c{src_base} while on the -command line \- the current working directory. In this case the generated -names are relative to the base directory. For example, assuming we have the -\c{foo.cxx} and \c{b/bar.cxx} source files: + +\h2#targets-types-file|\c{file{\}}| + +The \c{file{\}} target type represents a generic file. This target type is +used as a base for most of the file-based targets and can also be used to +declare targets and prerequisites when there are no more specific target +types. + +A target or prerequisite without a target type is automatically treated as +\c{file{\}}. However, omitting a target type in such situations is not +customary. + +The \c{file{\}} target type has no default extension and one cannot be +assigned with the \c{extension} variable. As a result, if a \c{file{\}} target +has an extension, then it must be specified explicitly as part of the target +name. For example: \ -exe{hello}: $src_base/cxx{**} # $src_base/cxx{foo} $src_base/b/cxx{bar} -exe{hello}: cxx{**} # cxx{foo} b/cxx{bar} +./: file{example.conf} \ -Pattern matching as well as inclusion/exclusion logic is target -type-specific. If the name pattern does not contain a type, then the -\c{dir{\}} type is assumed if the pattern ends with a directory separator and -\c{file{\}} otherwise. +\h2#targets-types-doc|\c{doc{\}}, \c{legal{\}}, and \c{man{\}}| -For the \c{dir{\}} target type the trailing directory separator is added to -the pattern and all the inclusion/exclusion patterns/matches that do not -already end with one. Then the filesystem search is performed for matching -directories. For example: +The \c{doc{\}} target type represents a generic documentation file. It has +semantics similar to \c{file{\}} (from which it derives): it can be used as a +base or declare targets/prerequisites and there is no default extension. One +notable difference, however, is that \c{doc{\}} targets are by default +installed into the \c{doc/} installation location (see \l{#module-install +\c{install} Module}). For example: \ -./: dir{* -build} # Search for */, exclude build/. +./: doc{README.md ChangeLog.txt} \ -For the \c{file{\}} and \c{file{\}}-based target types the default extension -(if any) is added to the pattern and all the inclusion/exclusion -patterns/matches that do not already contain an extension. Then the filesystem -search is performed for matching files. - -For example, the \c{cxx{\}} target type obtains the default extension from the -\c{extension} variable (see \l{#targets-types Target Types} for background). -Assuming we have the following line in our \c{root.build}: +The \c{legal{\}} target type is derived from \c{doc{\}} and represents a legal +documentation file, such as a license, copyright notice, authorship +information, etc. The main purpose of having a separate target type like this +is to help with installing licensing-related files into a different +location. To this effect, \c{legal{\}} targets are installed into the +\c{legal/} installation location, which by default is the same as \c{doc/} but +can be customized. For example: \ -cxx{*}: extension = cxx +./: legal{COPYRIGHT LICENSE AUTHORS.md} \ -And the following in our \c{buildfile}: +The \c{man{\}} target type is derived from \c{doc{\}} and represents a manual +page. This target type requires an explicit extension specification and is +installed into the \c{man/} installation location + +\N|If you are using the \c{man{\}} target type directly (instead of one of +\c{man{\}} described below), for example, to install a localized version of +a man page, then you will likely need to adjust the installation location +on the per target basis.| + +The \c{man{\}} target types (where \c{} is an integer between 1 and 9) +are derived from \c{man{\}} and represent manual pages in the respective +sections. These target types have fixed default extensions \c{.} (but an +explicit extension can still be specified, for example \c{man1{foo.1p\}}) and +are installed into the \c{man/} installation locations. For example: \ -exe{hello}: {cxx}{* -foo -bar.cxx} +./: man1{foo} \ -The pattern match will first search for all the files matching the \c{*.cxx} -pattern in \c{src_base} and then exclude \c{foo.cxx} and \c{bar.cxx} from the -result. Note also that target type-specific decorations are removed from the -result. So in the above example if the pattern match produces \c{baz.cxx}, -then the prerequisite name is \c{cxx{baz\}}, not \c{cxx{baz.cxx\}}. -If the name generation cannot be performed because the base directory is -unknown, target type is unknown, or the target type is not directory or -file-based, then the name pattern is returned as is (that is, as an ordinary -name). Project-qualified names are never considered to be patterns. +\h2#targets-types-exe|\c{exe{\}}| + +The \c{exe{\}} target type represents an executable file. Executables in +\c{build2} appear in two distinct but sometimes overlapping contexts: We can +build an executable target, for example from C source files. Or we can list an +executable target as a prerequisite in order to execute it as part of a +recipe. And sometimes this can be the same executable target. For example, +one project may build an executable target that is a source code generator and +another project may import this executable target and use it in its recipes in +order to generate some source code. + +To support this semantics the \c{exe{\}} target type has a peculiar default +extension logic. Specifically, if the \c{exe{\}} target is \"output\", then +the extension is expected to be assigned by the matching rule according to the +target platform for which this executable is built. But if it does not, +then we fall back to no extension (for example, a script). If, however, the +\c{exe{\}} target is \"input\" (that is, it's listed as a prerequisite and +there is no corresponding \"output\" target), then the extension of the host +platform is used as the default. + +In all these cases the extension can also be specified explicitly. This, for +example, would be necessary if the executable were a batch file: + +\ +h{generate}: exe{generate.bat} +{{ + diag $< -> $> + $< -o $path($>) +}} +\ + +Here, without the explicit extension, the \c{.exe} extension would have been +used by default. \h1#variables|Variables| @@ -5832,404 +5935,483 @@ info $(file{bar}: y) # s w \ -\h1#directives|Directives| +\h1#functions|Functions| \N{This chapter is a work in progress and is incomplete.} -\h#directives-define|\c{define}| -\ -define : -\ +Functions in \c{build2} are organized into families, such as the +\c{$string.*()} family for manipulating strings or \c{$regex.*()} for working +with regular expressions. Most functions are pure and those that are not, +such as \c{$builtin.getenv()}, are explicitly documented as such. -Define a new target type \c{} by inheriting from existing target type -\c{}. See \l{#targets-types Target Types} for details. +Some functions, such as from the \c{$regex.*()} family, can only be called +fully qualified with their family name. For example: +\ +if $regex.match($name, '(.+)-(.+)') + ... +\ -\h#directives-include|\c{include}| +While other functions can be called without explicit qualification. For +example: \ -include -include +path = $getenv('PATH') \ -Load the specified file (the first form) or \c{buildfile} in the specified -directory (the second form). In both cases the file is loaded in the scope -corresponding to its directory. Subsequent inclusions of the same file are -automatically ignored. See also \l{#directives-source \c{source}}. +There are also functions that can be called unqualified only for certain types +of arguments (this fact will be reflected in their synopsis and/or +documentation). Note, however, that every function can always be called +qualified. +" +// $builtin.*() +// +" +\h#functions-builtin|Builtin Functions| -\h#directives-source|\c{source}| +The \c{$builtin.*()} function family contains fundamental \c{build2} +functions. +" +source ; +// $string.*() +// +" +\h#functions-string|String Functions| +" +source ; -\ -source -\ -Load the specified file in the current scope as if its contents were copied -and pasted in place of the \c{source} directive. Note that subsequent sourcing -of the same file in the same scope are not automatically ignored. See also -\l{#directives-include \c{include}}. +// $integer.*() +// +" +\h#functions-integer|Integer Functions| +" +source ; +// $bool.*() +// +" +\h#functions-bool|Bool Functions| +" +source ; -\h1#targets|Targets and Target Types| -\N{This chapter is a work in progress and is incomplete.} +// $path.*() +// +" +\h#functions-path|Path Functions| +The \c{$path.*()} function family contains function that manipulating +filesystem paths. +" +source ; -\h#targets-types|Target Types| -A target type is part of a target's identity. The core idea behind the concept -of target types is to abstract away from file extensions which can vary from -project to project (for example, C++ source files extensions) or from platform -to platform (for example, executable file extensions). It also allows us to -have non-file-based targets. +// $name.*() +// +" +\h#functions-name|Name Functions| -Target types form a \i{base-derived} inheritance tree. The root of this tree -is the abstract \c{target{\}} type. The \c{build2} core defines a number of -standard target types, such as \c{file{\}}, \c{doc{\}}, and \c{exe{\}}. Build -system modules can define additional target types that are based on the -standard ones (or on types defined by other modules). For example, the \c{c} -module that provides the C compilation support defines the \c{h{\}} and -\c{c{\}} target types. Finally, \c{buildfiles} can derive project-local target -types using the \c{define} directive. +The \c{$name.*()} function family contains function that operate on target and +prerequisite names. See also the \l{#functions-target \c{$target.*()} function +family} for functions that operate on actual targets. +" +source ; -\N|If a target type represents a file type with a well-established extension, -then by convention such an extension is used as the target type name. For -example, the C language header and source files use the \c{.h} and \c{.c} -extensions and the target types are called \c{h{\}} and \c{c{\}}. -Speaking of conventions, as you may have noticed, when mentioning a target -type we customarily add \c{{\}} after its name. We found that this helps with -comprehension since target type names are often short (you can also search for -\c{{} to narrow it down to target types). In a way this is a similar -approach to adding \c{()} after a function name except here we use \c{{\}}, -which mimics target type usage in target names, for example \c{c{hello\}} for -\c{hello.c}.| +// $target.*() +// +" +\h#functions-target|Target Functions| -The following listing shows the hierarchy of the standard target types defined -by the \c{build2} core (the abstract target types are marked with \c{*}) while -the following sections describe each standard target type in detail. For -target types defined by a module refer to the respective module documentation. +The \c{$target.*()} function family contains function that operate on +targets. See also the \l{#functions-name \c{$name.*()} function family} for +functions that operate on target (and prerequisite) names. +" +source ; -\ - .-----target*------------. - | | | - mtime_target*---. alias fsdir - | | | - path_target* group dir - | - .---------file----. - | | | - .----doc-----. exe buildfile - | | | -legal man manifest - | - man -\ -While target types replace (potentially variable) extensions, there still -needs to be a mechanism for specifying them since in most cases targets have -to be mapped to files. There are several ways this can be achieved. +// $regex.*() +// +" +\h#functions-regex|Regex Functions| -If a target type represents a file type with a well-established extension, -then such an extension is normally used by default and we don't need to take -any extra steps. For example the \c{h{\}} and \c{c{\}} target types for C -header and source files default to the \c{.h} and \c{.c} extensions, -respectively, and if our project follows this convention, then we can simply -write: +The \c{$regex.*()} function family contains function that provide +comprehensive regular expression matching and substitution facilities. The +supported regular expression flavor is ECMAScript (more specifically, +ECMA-262-based C++11 regular expressions). + +In the \c{$regex.*()} functions the substitution escape sequences in the +format string (the \ci{fmt} argument) are extended with a subset of the Perl +escape sequences: \c{\\n}, \c{\\u}, \c{\\l}, \c{\\U}, \c{\\L}, \c{\\E}, +\c{\\1} ... \c{\\9}, and \c{\\\\}. Note that the standard ECMAScript escape +sequences (\c{$1}, \c{$2}, \c{$&}, etc) are still supported. + +Note that functions from the \c{$regex.*()} family can only be called fully +qualified with their family name. For example: \ -exe{utility}: c{utility} h{utility} +if $regex.match($name, '(.+)-(.+)') + ... \ -And \c{c{utility\}} will be mapped to \c{utility.c} and \c{h{utility\}} \- -to \c{utility.h}. +" +source ; -There are two variants of this default extension case: fixed extension and -customizable extension. A target type may choose to fix the default extension -if it's a bad idea to deviate from the default extension. A good example of -such a target is \c{man1{\}}, which fixes the default extension to be -\c{.1}. More commonly, however, a target will have a default extension but -will allow customizing it with the \c{extension} variable. -A good example where extension customization is often required are the -\c{hxx{\}} and \c{cxx{\}} target types for C++ header and source files, which -default to the \c{.hxx} and \c{.cxx} extensions, respectively. If our project -uses other extensions, for example, \c{.hpp} and \c{.cpp}, then we can adjust -the defaults (typically done in \c{root.build}, after loading the \c{cxx} -module): +// $process.*() +// +" +\h#functions-process|Process Functions| +" +source ; -\ -hxx{*}: extension = hpp -cxx{*}: extension = cpp -\ -Then we can write: +// $filesystem.*() +// +" +\h#functions-filesystem|Filesystem Functions| +" +source ; -\ -exe{utility}: cxx{utility} hxx{utility} -\ -And \c{cxx{utility\}} will be mapped to \c{utility.cpp} and \c{hxx{utility\}} -\- to \c{utility.hpp}. +// $project_name.*() +// +" +\h#functions-project_name|Project Name Functions| -What about \c{exe{utility\}}, where does its extension come from? This is an -example of a target type with an extension that varies from platform to -platform. In such cases the extension is expected to be assigned by the rule -that matches the target. In the above example, the link rule from the \c{cxx} -module that matches updating \c{exe{utility\}} will assign a suitable -extension based on the target platform of the C++ compiler that it was -instructed to use. +The \c{$project_name.*()} function family contains function that operate on +the \c{project_name} type. +" +source ; -Finally, it is always possible to specify the file extension explicitly as -part of the target name. For example: + +// $process_path.*() +// +" +\h#functions-process-path|Process Path Functions| + +The \c{$process_path.*()} function family contains function that operate on +the \c{process_path} type and its extended \c{process_path_ex} variant. These +types describe a path to an executable that, if necessary, has been found in +\c{PATH}, completed with an extension, etc. The \c{process_path_ex} variant +includes additional metadata, such as the stable process name for diagnostics +and the executable checksum for change tracking. +" +source ; + + +// $target_triplet.*() +// +" +\h#functions-target-triplet|Target Triplet Functions| + +The \c{$target_triplet.*()} function family contains function that operate on +the \c{target_triplet} type that represents the ubiquitous +\c{\i{cpu}-\i{vendor}-\i{os}} target platform triplet. +" +source ; + + +" +\h1#directives|Directives| + +\N{This chapter is a work in progress and is incomplete.} + +\h#directives-define|\c{define}| \ -exe{utility}: cxx{utility.cc} hxx{utility.hh} +define : \ -This is normally only needed if the default extension is not appropriate or if -the target type does not have a default extension, as is the case, for -example, for the \l{#targets-types-file \c{file{\}}} and \l{#targets-types-doc -\c{doc{\}}} target types. This mechanism can also be used to override the -automatically derived extension. For example: +Define a new target type \c{} by inheriting from existing target type +\c{}. See \l{#targets-types Target Types} for details. + + +\h#directives-include|\c{include}| \ -exe{($cxx.target.class == 'windows' ? utility.com : utility)}: ... +include +include \ -\N|If you need to specify a name that does not have an extension, then end it -with a single dot. For example, for a header \c{utility} you would write -\c{hxx{utility.\}}. If you need to specify a name with an actual trailing dot, -then escape it with a double dot, for example, \c{hxx{utility..\}}. +Load the specified file (the first form) or \c{buildfile} in the specified +directory (the second form). In both cases the file is loaded in the scope +corresponding to its directory. Subsequent inclusions of the same file are +automatically ignored. See also \l{#directives-source \c{source}}. -More generally, anywhere in a name, a double dot can be used to specify a dot -that should not be considered the extension separator while a triple dot \- -which should. For example, in \c{obja{foo.a.o\}} the extension is \c{.o} and -if instead we wanted \c{.a.o} to be considered the extension, then we could -rewrite it either as \c{obja{foo.a..o\}} or as \c{obja{foo...a.o\}}.| -To derive a new target type in a \c{buildfile} we use the \c{define} -directive. Such target types are project-local, meaning they cannot be -exported to other projects. Typically this is used to provide a more -meaningful name to a set of files and also avoid having to specify their -extensions explicitly. Compare: +\h#directives-source|\c{source}| + \ -./: doc{README.md PACKAGE-README.md INSTALL.md} +source \ -To: +Load the specified file in the current scope as if its contents were copied +and pasted in place of the \c{source} directive. Note that subsequent sourcing +of the same file in the same scope are not automatically ignored. See also +\l{#directives-include \c{include}}. -\ -define md: doc -doc{*}: extension = md -./: md{README PACKAGE-README INSTALL} -\ +\h1#attributes|Attributes| +\N{This chapter is a work in progress and is incomplete.} -\h2#targets-types-target|\c{target{\}}| +The only currently recognized target attribute is \c{rule_hint} which +specifies the rule hint. Rule hints can be used to resolve ambiguity when +multiple rules match the same target as well as to override an unambiguous +match. For example, the following rule hint makes sure our executable is +linked with the C++ compiler even though it only has C sources: -The \c{target{\}} target type is a root of the target type hierarchy. It is -abstract and is not commonly used directly, except perhaps in patterns (target -type/pattern-specific variable, pattern rules). +\ +[rule_hint=cxx] exe{hello}: c{hello} +\ -\h2#targets-types-alias|\c{alias{\}} and \c{dir{\}}| +\h1#name-patterns|Name Patterns| -The \c{alias{\}} target type is used for non-file-based targets that serve as -aliases for their prerequisite. +For convenience, in certain contexts, names can be generated with shell-like +wildcard patterns. A name is a \i{name pattern} if its value contains one or +more unquoted wildcard characters or character sequences. For example: -\N|Alias targets in \c{build2} are roughly equivalent to phony targets in -\c{make}.| +\ +./: */ # All (immediate) subdirectories +exe{hello}: {hxx cxx}{**} # All C++ header/source files. +pattern = '*.txt' # Literal '*.txt'. +\ -For example: +Pattern-based name generation is not performed in certain contexts. +Specifically, it is not performed in target names where it is interpreted +as a pattern for target type/pattern-specific variable assignments. For +example. \ -alias{tests}: exe{test1 test2 test3} +s = *.txt # Variable assignment (performed). +./: cxx{*} # Prerequisite names (performed). +cxx{*}: dist = false # Target pattern (not performed). \ +In contexts where it is performed, it can be inhibited with quoting, for +example: + \ -$ b test: alias{tests} +pat = 'foo*bar' +./: cxx{'foo*bar'} \ -An \c{alias{\}} target can also serve as an \"action\" if supplied with an ad -hoc recipe (or matched by an ad hoc pattern rule). For example: +The following wildcards are recognized: \ -alias{strip}: exe{hello} -{{ - diag strip $< - strip $path($<) -}} +* - match any number of characters (including zero) +? - match any single character +[...] - match a character with a bracket expression \ -The \c{dir{\}} target type is a special kind of alias that represents a -directory. Building it means building everything inside the directory. See -\l{#intro-proj-struct Project Structure} for background. +\N|Currently only literal character and range bracket expressions are +supported. Specifically, no character or equivalence classes, etc., are +supported nor the special characters backslash-escaping. See the \"Pattern +Matching Notation\" section in the POSIX \"Shell Command Language\" +specification for details.| -A target without a type that ends with a directory separator (\c{/}) is -automatically treated as \c{dir{\}}. For example, the following two lines are -equivalent: +Note that some wildcard characters may have special meaning in certain +contexts. For instance, \c{[} at the beginning of a value will be interpreted +as the start of the attribute list while \c{?} and \c{[} in the eval context +are part of the ternary operator and value subscript, respectively. In such +cases the character will need to be escaped in order to be treated as a +wildcard, for example: \ -./: exe{test1 test2} -dir{./}: exe{test1 test2} +x = \[1-9]-foo.txt +y = (foo.\?xx) +z = ($foo\[123].txt) \ -Omitting the target type in such situations is customary. +If a pattern ends with a directory separator, then it only matches +directories. Otherwise, it only matches files. Matches that start with a dot +(\c{.}) are automatically ignored unless the pattern itself also starts with +this character. +In addition to the above wildcards, \c{**} and \c{***} are recognized as +wildcard sequences. If a pattern contains \c{**}, then it is matched just like +\c{*} but in all the subdirectories, recursively, but excluding directories +that contain the \c{.buildignore} file. The \c{***} wildcard behaves like +\c{**} but also matches the start directory itself. For example: -\h2#targets-types-fsdir|\c{fsdir{\}}| +\ +exe{hello}: cxx{**} # All C++ source files recursively. +\ -The \c{fsdir{\}} target type represents a filesystem directory. Unlike -\c{dir{\}} above, it is not an alias and listing an \c{fsdir{\}} directory as -a prerequisite of a target will cause that directory to be created on -\c{update} and removed on \c{clean}. +A group-enclosed (\c{{\}}) pattern value may be followed by +inclusion/exclusion patterns/matches. A subsequent value is treated as an +inclusion or exclusion if it starts with a literal, unquoted plus (\c{+}) or +minus (\c{-}) sign, respectively. In this case the remaining group values, if +any, must all be inclusions or exclusions. If the second value doesn't start +with a plus or minus, then all the group values are considered independent +with leading pluses and minuses not having any special meaning. For regularity +as well as to allow patterns without wildcards, the first pattern can also +start with the plus sign. For example: -While we usually don't need to list explicit \c{fsdir{\}} prerequisites for -our targets, one situation where this is necessary is when the target resides -in a subdirectory that does not correspond to an existing source directory. A -typical example of this situation is placing object files into subdirectories. -Compare: +\ +exe{hello}: cxx{f* -foo} # Exclude foo if exists. +exe{hello}: cxx{f* +bar} # Include bar if exists. +exe{hello}: cxx{f* -fo?} # Exclude foo and fox if exist. +exe{hello}: cxx{f* +b* -foo -bar} # Exclude foo and bar if exist. +exe{hello}: cxx{+f* +b* -foo -bar} # Same as above. +exe{hello}: cxx{+foo} # Pattern without wildcards. +exe{hello}: cxx{f* b* -z*} # Names matching three patterns. +\ + +Inclusions and exclusions are applied in the order specified and only to the +result produced up to that point. The order of names in the result is +unspecified. However, it is guaranteed not to contain duplicates. The first +pattern and the following inclusions/exclusions must be consistent with +regards to the type of filesystem entry they match. That is, they should all +match either files or directories. For example: \ -obj{foo}: c{foo} -sub/obj{bar}: c{bar} fsdir{sub/} +exe{hello}: cxx{f* -foo +*oo} # Exclusion has no effect. +exe{hello}: cxx{f* +*oo} # Ok, no duplicates. +./: {*/ -build} # Error: exclusion not a directory. \ +As a more realistic example, let's say we want to exclude source files that +reside in the \c{test/} directories (and their subdirectories) anywhere in the +tree. This can be achieved with the following pattern: -\h2#targets-types-mtime-path|\c{mtime_target{\}} and \c{path_target{\}}| +\ +exe{hello}: cxx{** -***/test/**} +\ -The \c{mtime_target{\}} target type represents a target that uses modification -times to determine if it is out of date. The \c{path_target{\}} target type -represents a target that has a corresponding filesystem entry. It is derived -from \c{mtime_target{\}} and uses the modification time of that filesystem -entry to determine if the target is out of date. +Similarly, if we wanted to exclude all source files that have the \c{-test} +suffix: -Both of these target types are abstract and are not commonly used directly, -except perhaps in patterns (target type/pattern-specific variable, pattern -rules). +\ +exe{hello}: cxx{** -**-test} +\ +In contrast, the following pattern only excludes such files from the top +directory: -\h2#targets-types-group|\c{group{\}}| +\ +exe{hello}: cxx{** -*-test} +\ -The \c{group{\}} target type represents a user-defined explicit target group, -that is, a target that has multiple member targets that are all built together -with a single recipe. +If many inclusions or exclusions need to be specified, then an +inclusion/exclusion group can be used. For example: -Normally this target type is not used to declare targets or prerequisites but -rather as a base of a derived group. If desired, such a derived group can be -marked with an attribute as \"see-through\", meaning that when the group is -listed as a prerequisite of a target, the matching rule \"sees\" its members, -rather than the group itself. For example: +\ +exe{hello}: cxx{f* -{foo bar}} +exe{hello}: cxx{+{f* b*} -{foo bar}} +\ + +This is particularly useful if you would like to list the names to include or +exclude in a variable. For example, this is how we can exclude certain files +from compilation but still include them as ordinary file prerequisites (so +that they are still included into the source distribution): \ -define [see_through] thrift_cxx: group +exc = foo.cxx bar.cxx +exe{hello}: cxx{+{f* b*} -{$exc}} file{$exc} \ +If we want to specify our pattern in a variable, then we have to use the +explicit inclusion syntax, for example: -\h2#targets-types-file|\c{file{\}}| +\ +pat = 'f*' +exe{hello}: cxx{+$pat} # Pattern match. +exe{hello}: cxx{$pat} # Literal 'f*'. -The \c{file{\}} target type represents a generic file. This target type is -used as a base for most of the file-based targets and can also be used to -declare targets and prerequisites when there are no more specific target -types. +pat = '+f*' +exe{hello}: cxx{$pat} # Literal '+f*'. -A target or prerequisite without a target type is automatically treated as -\c{file{\}}. However, omitting a target type in such situations is not -customary. +inc = 'f*' 'b*' +exc = 'f*o' 'b*r' +exe{hello}: cxx{+{$inc} -{$exc}} +\ -The \c{file{\}} target type has no default extension and one cannot be -assigned with the \c{extension} variable. As a result, if a \c{file{\}} target -has an extension, then it must be specified explicitly as part of the target -name. For example: +One common situation that calls for exclusions is auto-generated source +code. Let's say we have auto-generated command line parser in \c{options.hxx} +and \c{options.cxx}. Because of the in/out of source builds, our name pattern +may or may not find these files. Note, however, that we cannot just include +them as non-pattern prerequisites. We also have to exclude them from the +pattern match since otherwise we may end up with duplicate prerequisites. As a +result, this is how we have to handle this case provided we want to continue +using patterns to find other, non-generated source files: \ -./: file{example.conf} +exe{hello}: {hxx cxx}{* -options} {hxx cxx}{options} \ -\h2#targets-types-doc|\c{doc{\}}, \c{legal{\}}, and \c{man{\}}| - -The \c{doc{\}} target type represents a generic documentation file. It has -semantics similar to \c{file{\}} (from which it derives): it can be used as a -base or declare targets/prerequisites and there is no default extension. One -notable difference, however, is that \c{doc{\}} targets are by default -installed into the \c{doc/} installation location (see \l{#module-install -\c{install} Module}). For example: +If all our auto-generated source files have a common prefix or suffix, then we +can exclude them wholesale with a pattern. For example, if all our generated +files end with the `-options` suffix: \ -./: doc{README.md ChangeLog.txt} +exe{hello}: {hxx cxx}{** -**-options} {hxx cxx}{foo-options bar-options} \ -The \c{legal{\}} target type is derived from \c{doc{\}} and represents a legal -documentation file, such as a license, copyright notice, authorship -information, etc. The main purpose of having a separate target type like this -is to help with installing licensing-related files into a different -location. To this effect, \c{legal{\}} targets are installed into the -\c{legal/} installation location, which by default is the same as \c{doc/} but -can be customized. For example: +If the name pattern includes an absolute directory, then the pattern match is +performed in that directory and the generated names include absolute +directories as well. Otherwise, the pattern match is performed in the +\i{pattern base} directory. In buildfiles this is \c{src_base} while on the +command line \- the current working directory. In this case the generated +names are relative to the base directory. For example, assuming we have the +\c{foo.cxx} and \c{b/bar.cxx} source files: \ -./: legal{COPYRIGHT LICENSE AUTHORS.md} +exe{hello}: $src_base/cxx{**} # $src_base/cxx{foo} $src_base/b/cxx{bar} +exe{hello}: cxx{**} # cxx{foo} b/cxx{bar} \ -The \c{man{\}} target type is derived from \c{doc{\}} and represents a manual -page. This target type requires an explicit extension specification and is -installed into the \c{man/} installation location - -\N|If you are using the \c{man{\}} target type directly (instead of one of -\c{man{\}} described below), for example, to install a localized version of -a man page, then you will likely need to adjust the installation location -on the per target basis.| +Pattern matching as well as inclusion/exclusion logic is target +type-specific. If the name pattern does not contain a type, then the +\c{dir{\}} type is assumed if the pattern ends with a directory separator and +\c{file{\}} otherwise. -The \c{man{\}} target types (where \c{} is an integer between 1 and 9) -are derived from \c{man{\}} and represent manual pages in the respective -sections. These target types have fixed default extensions \c{.} (but an -explicit extension can still be specified, for example \c{man1{foo.1p\}}) and -are installed into the \c{man/} installation locations. For example: +For the \c{dir{\}} target type the trailing directory separator is added to +the pattern and all the inclusion/exclusion patterns/matches that do not +already end with one. Then the filesystem search is performed for matching +directories. For example: \ -./: man1{foo} +./: dir{* -build} # Search for */, exclude build/. \ +For the \c{file{\}} and \c{file{\}}-based target types the default extension +(if any) is added to the pattern and all the inclusion/exclusion +patterns/matches that do not already contain an extension. Then the filesystem +search is performed for matching files. -\h2#targets-types-exe|\c{exe{\}}| - -The \c{exe{\}} target type represents an executable file. Executables in -\c{build2} appear in two distinct but sometimes overlapping contexts: We can -build an executable target, for example from C source files. Or we can list an -executable target as a prerequisite in order to execute it as part of a -recipe. And sometimes this can be the same executable target. For example, -one project may build an executable target that is a source code generator and -another project may import this executable target and use it in its recipes in -order to generate some source code. +For example, the \c{cxx{\}} target type obtains the default extension from the +\c{extension} variable (see \l{#targets-types Target Types} for background). +Assuming we have the following line in our \c{root.build}: -To support this semantics the \c{exe{\}} target type has a peculiar default -extension logic. Specifically, if the \c{exe{\}} target is \"output\", then -the extension is expected to be assigned by the matching rule according to the -target platform for which this executable is built. But if it does not, -then we fall back to no extension (for example, a script). If, however, the -\c{exe{\}} target is \"input\" (that is, it's listed as a prerequisite and -there is no corresponding \"output\" target), then the extension of the host -platform is used as the default. +\ +cxx{*}: extension = cxx +\ -In all these cases the extension can also be specified explicitly. This, for -example, would be necessary if the executable were a batch file: +And the following in our \c{buildfile}: \ -h{generate}: exe{generate.bat} -{{ - diag $< -> $> - $< -o $path($>) -}} +exe{hello}: {cxx}{* -foo -bar.cxx} \ -Here, without the explicit extension, the \c{.exe} extension would have been -used by default. +The pattern match will first search for all the files matching the \c{*.cxx} +pattern in \c{src_base} and then exclude \c{foo.cxx} and \c{bar.cxx} from the +result. Note also that target type-specific decorations are removed from the +result. So in the above example if the pattern match produces \c{baz.cxx}, +then the prerequisite name is \c{cxx{baz\}}, not \c{cxx{baz.cxx\}}. + +If the name generation cannot be performed because the base directory is +unknown, target type is unknown, or the target type is not directory or +file-based, then the name pattern is returned as is (that is, as an ordinary +name). Project-qualified names are never considered to be patterns. \h1#module-config|\c{config} Module| -- cgit v1.1