aboutsummaryrefslogtreecommitdiff
path: root/doc/manual.cli
blob: 6874ceeb910b2e286af76e14b2e8e0197343f129 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
// file      : doc/manual.cli
// copyright : Copyright (c) 2014-2018 Code Synthesis Ltd
// license   : MIT; see accompanying LICENSE file

"\name=build2-build-system-manual"
"\subject=build system"
"\title=Build System"

// NOTES
//
// - Maximum <pre> line is 70 characters.
//

"
\h0#preface|Preface|

This is the preface.

\h1#name-patterns|Name Patterns|

For convenience, in certain contexts, names can be generated with shell-like
wildcard patterns. A name is a \i{name pattern} if its value contains one or
more unquoted wildcard characters or character sequences. For example:

\
./: */                     # All (immediate) subdirectories
exe{hello}: {hxx cxx}{**}  # All C++ header/source files.
pattern = '*.txt'          # Literal '*.txt'.
\

Pattern-based name generation is not performed in certain contexts.
Specifically, it is not performed in target names where it is interpreted
as a pattern for target type/pattern-specific variable assignments. For
example.

\
s = *.txt             # Variable assignment (performed).
./: cxx{*}            # Prerequisite names (performed).
cxx{*}: dist = false  # Target pattern (not performed).
\

In contexts where it is performed, it can be inhibited with quoting, for
example:

\
pat = 'foo*bar'
./: cxx{'foo*bar'}
\

The following characters are wildcards:

\
*  - match any number of characters (including zero)
?  - match any single character
\

If a pattern ends with a directory separator, then it only matches
directories. Otherwise, it only matches files. Matches that start with a dot
(\c{.}) are automatically ignored unless the pattern itself also starts with
this character.

In addition to the above wildcard characters, \c{**} and \c{***} are
recognized as wildcard character sequences. If a pattern contains \c{**}, then
it is matched just like \c{*} but in all the subdirectories, recursively. The
\c{***} wildcard behaves like \c{**} but also matches the start directory
itself. For example:

\
exe{hello}: cxx{**}  # All C++ source files recursively.
\

A group-enclosed (\c{{\}}) pattern value may be followed by
inclusion/exclusion patterns/matches. A subsequent value is treated as an
inclusion or exclusion if it starts with a literal, unquoted plus (\c{+}) or
minus (\c{-}) sign, respectively. In this case the remaining group values, if
any, must all be inclusions or exclusions. If the second value doesn't start
with a plus or minus, then all the group values are considered independent
with leading pluses and minuses not having any special meaning. For
regularity, the first pattern can also start with the plus sign. For example:

\
exe{hello}: cxx{f* -foo}            # Exclude foo if present.
exe{hello}: cxx{f* +foo}            # Include foo if not present.
exe{hello}: cxx{f* -fo?}            # Exclude foo and fox if present.
exe{hello}: cxx{f* +b* -foo -bar}   # Exclude foo and bar if present.
exe{hello}: cxx{+f* +b* -foo -bar}  # Same as above.
exe{hello}: cxx{f* b* -z*}          # Names matching three patterns.
\

Inclusions and exclusions are applied in the order specified and only to the
result produced up to that point. The order of names in the result is
unspecified. However, it is guaranteed not to contain duplicates. The
pattern and the following inclusions/exclusions must be consistent with
regards to the type of filesystem entry they match. That is, they should
all match either files or directories. For example:

\
exe{hello}: cxx{f* -foo +*oo}  # Exclusion has no effect.
exe{hello}: cxx{f* +*oo}       # Ok, no duplicates.
./: {*/ -build}                # Error: exclusion not a directory.
\

As a more realistic example, let's say we want to exclude source files that
reside in the \c{test/} directories (and their subdirectories) anywhere in the
tree. This can be achieved with the following pattern:

\
exe{hello}: cxx{** -***/test/**}
\

Similarly, if we wanted to exclude all source files that have the \c{-test}
suffix:

\
exe{hello}: cxx{** -**-test}
\

In contrast, the following pattern only excludes such files from the top
directory:

\
exe{hello}: cxx{** -*-test}
\

If many inclusions or exclusions need to be specified, then an
inclusion/exclusion group can be used. For example:

\
exe{hello}: cxx{f* -{foo bar}}
exe{hello}: cxx{+{f* b*} -{foo bar}}
\

This is particularly useful if you would like to list the names to include or
exclude in a variable. For example, this is how we can exclude certain files
from compilation but still include them as ordinary file prerequisites (so
that they are still included into the distribution):

\
exc = foo.cxx bar.cxx
exe{hello}: cxx{+{f* b*} -{$exc}} file{$exc}
\

If we want to specify our pattern in a variable, then we have to use the
explicit inclusion syntax, for example:

\
pat = 'f*'
exe{hello}: cxx{+$pat} # Pattern match.
exe{hello}: cxx{$pat}  # Literal 'f*'.

pat = '+f*'
exe{hello}: cxx{$pat}  # Literal '+f*'.

inc = 'f*'  'b*'
exc = 'f*o' 'b*r'
exe{hello}: cxx{+{$inc} -{$exc}}
\

One common situation that calls for exclusions is auto-generated source
code. Let's say we have auto-generated command line parser in \c{options.hxx}
and \c{options.cxx}. Because of the in-tree builds, our name pattern may or
may not find these files. Note, however, that we cannot just include them as
non-pattern prerequisites. We also have to exclude them from the pattern match
since otherwise we may end up with duplicate prerequisites. As a result, this
is how we have to handle this case provided we want to continue using patterns
to find other, non-generated source files:

\
exe{hello}: {hxx cxx}{* -options} {hxx cxx}{options}
\

If the name pattern includes an absolute directory, then the pattern match is
performed in that directory and the generated names include absolute
directories as well. Otherwise, the pattern match is performed in the
\i{pattern base} directory. In buildfiles this is \c{src_base} while on the
command line \- the current working directory. In this case the generated
names are relative to the base directory. For example, assuming we have the
\c{foo.cxx} and \c{b/bar.cxx} source files:

\
exe{hello}: $src_base/cxx{**}  # $src_base/cxx{foo} $src_base/b/cxx{bar}
exe{hello}:           cxx{**}  #           cxx{foo}           b/cxx{bar}
\

Pattern matching as well as inclusion/exclusion logic is target
type-specific. If the name pattern does not contain a type, then the
\c{dir{\}} type is assumed if the pattern ends with a directory separator and
\c{file{\}} otherwise.

For the \c{dir{\}} target type the trailing directory separator is added to
the pattern and all the inclusion/exclusion patterns/matches that do not
already end with one. Then the filesystem search is performed for matching
directories. For example:

\
./: dir{* -build}  # Search for */, exclude build/.
\

For the \c{file{\}} and \c{file{\}}-based target types the default extension
(if any) is added to the pattern and all the inclusion/exclusion
patterns/matches that do not already contain an extension. Then the filesystem
search is performed for matching files.

For example, the \c{cxx{\}} target type obtains the default extension from the
\c{extension} variable. Assuming we have the following line in our
\c{root.build}:

\
cxx{*}: extension = cxx
\

And the following in our \c{buildfile}:

\
exe{hello}: {cxx}{* -foo -bar.cxx}
\

The pattern match will first search for all the files matching the \c{*.cxx}
pattern in \c{src_base} and then exclude \c{foo.cxx} and \c{bar.cxx} from the
result. Note also that target type-specific decorations are removed from the
result. So in the above example if the pattern match produces \c{baz.cxx},
then the prerequisite name is \c{cxx{baz\}}, not \c{cxx{baz.cxx\}}.

If the name generation cannot be performed because the base directory is
unknown, target type is unknown, or the target type is not directory or
file-based, then the name pattern is returned as is (that is, as an ordinary
name). Project-qualified names are never considered to be patterns.

\h1#grammar|Grammar|

\
eval:         '(' (eval-comma | eval-qual)? ')'
eval-comma:   eval-ternary (',' eval-ternary)*
eval-ternary: eval-or ('?' eval-ternary ':' eval-ternary)?
eval-or:      eval-and ('||' eval-and)*
eval-and:     eval-comp ('&&' eval-comp)*
eval-comp:    eval-value (('=='|'!='|'<'|'>'|'<='|'>=') eval-value)*
eval-value:   value-attributes? (<value> | eval | '!' eval-value)
eval-qual:    <name> ':' <name>

value-attributes: '[' <key-value-pairs> ']'
\

Note that \c{?:} (ternary operator) and \c{!} (logical not) are
right-associative. Unlike C++, all the comparison operators have the same
precedence. A qualified name cannot be combined with any other operator
(including ternary) unless enclosed in parentheses. The \c{eval} option
in the \c{eval-value} production shall contain single value only (no
commas).

\h1#module-test|Test Module|

The targets to be tested as well as the tests/groups from testscripts to be
run can be narrowed down using the \c{config.test} variable. While this
value is normally specified as a command line override (for example, to
quickly re-run a previously failed test), it can also be persisted in
\c{config.build} in order to create a configuration that will only run a
subset of tests by default. For example:

\
b test config.test=foo/exe{driver} # Only test foo/exe{driver} target.
b test config.test=bar/baz         # Only run bar/baz testscript test.
\

The \c{config.test} variable contains a list of \c{@}-separated pairs with the
left hand side being the target and the right hand side being the testscript
id path. Either can be omitted (along with \c{@}). If the value contains a
target type or ends with a directory separator, then it is treated as a target
name. Otherwise \- an id path. The targets are resolved relative to the root
scope where the \c{config.test} value is set. For example:

\
b test config.test=foo/exe{driver}@bar
\

To specify multiple id paths for the same target we can use the pair
generation syntax:

\
b test config.test=foo/exe{driver}@{bar baz}
\

If no targets are specified (only id paths), then all the targets are tested
(with the testscript tests to be run limited to the specified id paths). If no
id paths are specified (only targets), then all the testscript tests are run
(with the targets to be tested limited to the specified targets). An id path
without a target applies to all the targets being considered.

A directory target without an explicit target type (for example, \c{foo/}) is
treated specially. It enables all the tests at and under its directory. This
special treatment can be inhibited by specifying the target type explicitly
(for example, \c{dir{foo/\}}).


\h1#module-version|Version Module|

A project can use any version format as long as it meets the package version
requirements. The toolchain also provides additional functionality for
managing projects that conform to the \c{build2} \i{standard version}
format. If you are starting a new project that uses \c{build2}, you are
strongly encouraged to use this versioning scheme. It is based on much thought
and, often painful, experience. If you decide not to follow this advice, you
are essentially on your own when version management is concerned.

The standard \c{build2} project version conforms to \l{http://semver.org
Semantic Versioning} and has the following form:

\
<major>.<minor>.<patch>[-<prerel>]
\

For example:

\
1.2.3
1.2.3-a.1
1.2.3-b.2
\

The \c{build2} package version that uses the standard project version will
then have the following form (\i{epoch} is the versioning scheme version
and \i{revision} is the package revision):

\
[+<epoch>-]<major>.<minor>.<patch>[-<prerel>][+<revision>]
\

For example:

\
1.2.3
1.2.3+1
+1-1.2.3-a.1+2
\

The \i{major}, \i{minor}, and \i{patch} should be numeric values between \c{0}
and \c{999} and all three cannot be zero at the same time. For initial
development it is recommended to use \c{0} for \i{major}, start with version
\c{0.1.0}, and change to \c{1.0.0} once things stabilize.

In the context of C and C++ (or other compiled languages), you should
increment \i{patch} when making binary-compatible changes, \i{minor} when
making source-compatible changes, and \i{major} when making breaking changes.
While the binary compatibility must be set in stone, the source compatibility
rules can sometimes be bent. For example, you may decide to make a breaking
change in a rarely used interface as part of a minor release (though this is
probably still a bad idea if your library is widely depended upon). Note also
that in the context of C++ deciding whether a change is binary-compatible is a
non-trivial task. There are resources that list the rules but no automated
tooling yet. If unsure, increment \i{minor}.

If present, the \i{prerel} component signifies a pre-release. Two types of
pre-releases are supported by the standard versioning scheme: \i{final} and
\i{snapshot} (non-pre-release versions are naturally always final). For final
pre-releases the \i{prerel} component has the following form:

\
(a|b).<num>
\

For example:

\
1.2.3-a.1
1.2.3-b.2
\

The letter '\c{a}' signifies an alpha release and '\c{b}' \- beta. The
alpha/beta numbers (\i{num}) should be between 1 and 499.

Note that there is no support for release candidates. Instead, it is
recommended that you use later-stage beta releases for this purpose (and, if
you wish, call them \"release candidates\" in announcements, etc).

What version should be used during development? The common approach is to
increment to the next version and use that until the release. This has one
major drawback: if we publish intermediate snapshots (for example, for
testing) they will all be indistinguishable both between each other and, even
worse, from the final release. One way to remedy this is to increment the
pre-release number before each publication. However, unless automated, this
will be burdensome and error-prone. Also, there is a real possibility of
running out of version numbers if, for example, we do continuous integration
by publishing and testing each commit.

To address this, the standard versioning scheme supports \i{snapshot
pre-releases} with the \i{prerel} component having the following extended
form:

\
(a|b).<num>.<snapsn>[.<snapid>]
\

For example:

\
1.2.3-a.1.20180319215815.26efe301f4a7
\

In essence, a snapshot pre-release is after the previous final release but
before the next (\c{a.1} and, perhaps, \c{a.2} in the above example) and
is uniquely identified by the snapshot sequence number (\i{snapsn}) and
optional snapshot id (\i{snapid}).

The \i{num} component has the same semantics as in the final pre-releases
except that it can be \c{0}. The \i{snapsn} component should be either the
special value '\c{z}' or a numeric, non-zero value that increases for each
subsequent snapshot. It must not be longer than 16 decimal digits. The
\i{snapid} component, if present, should be an alpha-numeric value that
uniquely identifies the snapshot. It is not required for version comparison
(\i{snapsn} should be sufficient) and is included for reference. It must not
be longer than 16 characters.

Where do the snapshot number and id come from? Normally from the version
control system. For example, for \c{git}, \i{snapsn} is the commit date in the
\i{YYYYMMDDhhmmss} form and UTC timezone and \i{snapid} is a 12-character
abbreviated commit id. As discussed below, the \c{build2} \c{version} module
extracts and manages all this information automatically (but the use of
\c{git} commit dates is not without limitations; see below for details).

The special '\c{z}' \i{snapsn} value identifies the \i{latest} or
\i{uncommitted} snapshot. It is chosen to be greater than any other possible
\i{snapsn} value and its use is discussed further below.

As an illustration of this approach, let's examine how versions change
during the lifetime of a project:

\
0.1.0-a.0.z     # development after a.0
0.1.0-a.1       # pre-release
0.1.0-a.1.z     # development after a.1
0.1.0-a.2       # pre-release
0.1.0-a.2.z     # development after a.2
0.1.0-b.1       # pre-release
0.1.0-b.1.z     # development after b.1
0.1.0           # release
0.1.1-b.0.z     # development after b.0 (bugfix)
0.2.0-a.0.z     # development after a.0
0.1.1           # release (bugfix)
1.0.0           # release (jumped straight to 1.0.0)
...
\

As shown in the above example, there is nothing wrong with \"jumping\" to a
further version (for example, from alpha to beta, or from beta to release, or
even from alpha to release). We cannot, however, jump backwards (for example,
from beta back to alpha). As a result, a sensible strategy is to start with
\c{a.0} since it can always be upgraded (but not downgrade) at a later stage.

When it comes to the version control systems, the recommended workflow is as
follows: The change to the final version should be the last commit in the
(pre-)release. It is also a good idea to tag this commit with the project
version. A commit immediately after that should change the version to a
snapshot, \"opening\" the repository for development.

The project version without the snapshot part can be represented as a 64-bit
decimal value comparable as integers (for example, in preprocessor
directives). The integer representation has the following form:

\
AAABBBCCCDDDE

AAA - major
BBB - minor
CCC - patch
DDD - alpha / beta (DDD + 500)
E   - final (0) / snapshot (1)
\

If the \i{DDDE} value is not zero, then it signifies a pre-release. In this
case one is subtracted from the \i{AAABBBCCC} value. An alpha number is stored
in \i{DDD} as is while beta \- incremented by \c{500}. If \i{E} is \c{1}, then
this is a snapshot after \i{DDD}.

For example:

\
             AAABBBCCCDDDE
0.1.0        0000010000000
0.1.2        0000010010000
1.2.3        0010020030000
2.2.0-a.1    0020019990010
3.0.0-b.2    0029999995020
2.2.0-a.1.z  0020019990011
\

A project that uses standard versioning can rely on the \c{build2} \c{version}
module to simplify and automate version managements. The \c{version} module
has two primary functions: eliminate the need to change the version anywhere
except in the project's manifest file and automatically extract and propagate
the snapshot information (serial number and id).

The \c{version} module must be loaded in the project's \c{bootstrap.build}.
While being loaded, it reads the project's manifest and extracts its version
(which must be in the standard form). The version is then parsed and presented
as the following build system variables (which can be used in the buildfiles):

\
[string] version                     # +2-1.2.3-b.4.1234567.deadbeef+3

[string] version.project             # 1.2.3-b.4.1234567.deadbeef
[uint64] version.project_number      # 0010020025041
[string] version.project_id          # 1.2.3-b.4.deadbeef

[bool]   version.stub                # false (true for 0[+<revision>])

[uint64] version.epoch               # 2

[uint64] version.major               # 1
[uint64] version.minor               # 2
[uint64] version.patch               # 3

[bool]   version.alpha               # false
[bool]   version.beta                # true
[bool]   version.pre_release         # true
[string] version.pre_release_string  # b.4
[uint64] version.pre_release_number  # 4

[bool]   version.snapshot            # true
[uint64] version.snapshot_sn         # 1234567
[string] version.snapshot_id         # deadbeef
[string] version.snapshot_string     # 1234567.deadbeef
[bool]   version.snapshot_committed  # true

[uint64] version.revision            # 3
\

As a convenience, the \c{version} module also extract the \c{summary} and
\c{url} manifest values and sets them as the following build system variables
(this additional information is used, for example, when generating the
\c{pkg-config} files):

\
[string] project.summary
[string] project.url
\

If the version is the latest snapshot (that is, it's in the \c{.z} form), then
the \c{version} module extracts the snapshot information from the version
control system used by the project. Currently only \c{git} is supported with
the following semantics.

If the project's source directory (\c{src_root}) is clean (that is, it does
not have any changed or untracked files), then the \c{HEAD} commit date and id
are used as the snapshot number and id, respectively.

Otherwise (that is, the project is between commits), the \c{HEAD} commit date
is incremented by one second and is used as the snapshot number with no id.
While we can work with such uncommitted snapshots locally, we should not
distribute or publish them since they are indistinguishable from each other.

Finally, if the project does not have \c{HEAD} (that is, the project has
no commits yet), the special \c{19700101000000} (UNIX epoch) commit date is
used.

The use of \c{git} commit dates for snapshot ordering has its limitations:
they have one second resolution which means it is possible to create two
commits with the same date (but not the same commit id and thus snapshot
id). We also need all the committers to have a reasonably accurate
clock. Note, however, that in case of a commit date clash/ordering issue, we
still end up with distinct versions (because of the commit id) \- they are
just not ordered correctly. As a result, we feel that the risks are justified
when the only alternative is manual version management (which is always an
option, nevertheless).

When we prepare a distribution of a snapshot, the \c{version} module
automatically adjusts the package name to include the snapshot information as
well as patches the manifest file in the distribution with the snapshot number
and id (that is, replacing \c{.z} in the version value with the actual
snapshot information). The result is a package that is specific to this
commit.

Besides extracting the version information and making it available as
individual components, the \c{version} module also provide rules for
installing the manifest file as well as automatically generating version
headers (or other similar version-based files).

By default the project's \c{manifest} file is installed as documentation, just
like other \c{doc{}} targets (thus replacing the \c{version} file customarily
shipped in the project root directory). The manifest installation rule in the
\c{version} module in addition patches the installed manifest file with the
actual snapshot number and id, just like during the preparation of
distributions.

The version header rule pre-processes a template file (which means it can
be used to generate any kinds of files, not just C/C++ headers). It matches a
\c{file}-based target that has a corresponding \c{in} prerequisite and also
depends on the project's \c{manifest} file. As an example, let's assume we
want to auto-generate a header called \c{version.hxx} for our \c{libhello}
library. To acomplish this we add the \c{version.hxx.in} template as well as
something along these lines to our \c{buildfile}:

\
lib{hello}: ... hxx{version}

hxx{version}: in{version} $src_root/file{manifest}
hxx{version}: dist = true
\

The header rule is a line-based pre-processor that substitutes fragments
enclosed between (and including) a pair of dollar signs (\c{$}) with \c{$$}
being the escape sequence. As an example, let's assume our \c{version.hxx.in}
contains the following lines:

\
#ifndef LIBHELLO_VERSION

#define LIBHELLO_VERSION     $libhello.version.project_number$ULL
#define LIBHELLO_VERSION_STR \"$libhello.version.project$\"

#endif
\

If our \c{libhello} is at version \c{1.2.3}, then the generated
\c{version.hxx} will look like this:

\
#ifndef LIBHELLO_VERSION

#define LIBHELLO_VERSION     10020030000ULL
#define LIBHELLO_VERSION_STR \"1.2.3\"

#endif
\

The first component after the opening \c{$} should be either the name of the
project itself (like \c{libhello} above) or a name of one of its dependencies
as listed in the manifest. If it is the project itself, then the rest can
refer to one of the \c{version.*} variables that we discussed earlier (in
reality it can be any variable visible from the project's root scope).

If the name refers to one of the dependecies (that is, projects listed with
\c{depends:} in the manifest), then the following special substitutions are
recognized:

\
$<name>.version$                           - textual version constraint
$<name>.condition(<VERSION>[,<SNAPSHOT>])$ - numeric satisfaction condition
$<name>.check(<VERSION>[,<SNAPSHOT>])$     - numeric satisfaction check
\

Here \i{VERSION} is the version number macro and the optional \i{SNAPSHOT} is
the snapshot number macro. The snapshot is only required if you plan to
include snapshot information in your dependency constraints.

As an example, let's assume our \c{libhello} depends on \c{libprint} which
is reflected with the following line in our manifest:

\
depends: libprint >= 2.3.4
\

We also assume that \c{libprint} provides its version information in the
\c{libprint/version.hxx} header and uses analogous-named macros. Here
is how we can add a version check to our \c{version.hxx.in}:

\
#ifndef LIBHELLO_VERSION

#define LIBHELLO_VERSION     $libhello.version.project_number$ULL
#define LIBHELLO_VERSION_STR \"$libhello.version.project$\"

#include <libprint/version.hxx>

$libprint.check(LIBPRINT_VERSION)$

#endif
\

After the substitution our \c{version.hxx} header will look like this:

\
#ifndef LIBHELLO_VERSION

#define LIBHELLO_VERSION     10020030000ULL
#define LIBHELLO_VERSION_STR \"1.2.3\"

#include <libprint/version.hxx>

#ifdef LIBPRINT_VERSION
#  if !(LIBPRINT_VERSION >= 20030040000ULL)
#    error incompatible libprint version, libprint >= 2.3.4 is required
#  endif
#endif

#endif
\

The \c{version} and \c{condition} substitutions are the building blocks of the
\c{check} substitution. For example, here is how we can implement a check with
a customized error message:

\
#if !($libprint.condition(LIBPRINT_VERSION)$)
#  error bad libprint, need libprint $libprint.version$
#endif
\

The \c{version} module also treats one dependency in a special way: if you
specify the required version of the build system in your manifest, then the
module will automatically check it for you. For example, if we have the
following line in our manifest:

\
depends: * build2 >= 0.5.0
\

And someone tries to build our project with \c{build2} \c{0.4.0}, then they
will see an error like this:

\
build/bootstrap.build:3:1: error: incompatible build2 version
  info: running 0.4.0
  info: required 0.5.0
\

What version constraints should be use when depending on other project. We
start with a simple case where we depend on a release. Let's say \c{libprint}
\c{2.3.0} added a feature that we need in our \c{libhello}. If \c{libprint}
follows the source/binary compatibility guidelines discussed above, then
any \c{2.X.Y} version should work provided \c{X >= 3}. And this how we can
specify it in the manifest:

\
depends: libprint ^2.3.0
\

Let's say we are now working on \c{libhello} \c{2.0.0} and would like to start
using features from \c{libprint} \c{3.0.0}. However, currently, only
pre-releases of \c{3.0.0} are available. If you would like to add a dependency
on a pre-release (most likely from your own pre-release), then the
recommendation is to only allow a specific version, essentially \"expiring\"
the combination as soon as newer versions become available. For example:

\
version: 2.0.0-b.1
depends: libprint == 3.0.0-b.2
\

Finally, let's assume we are feeling adventerous and would like to test
development snapshots of \c{libprint} (most likey from our own snapshots). In
this case the recommendation is to only allow a snapshot range for a specific
pre-release with the understanding and a warning that no compatibility between
snapshot versions is guaranteed. For example:

\
version: 2.0.0-b.1.z
depends: libprint [3.0.0-b.2.1 3.0.0-b.3)
\

\h1#module-cxx|\c{cxx} (C++) Module|

This chapter describes the \c{cxx} build system module which provides the C++
compilation and linking support. Most of its functionality, however, is
provided by the \c{cc} module, a common implementation for the C-family
languages.

\h#cxx-modules|C++ Modules Support|

This section describes the build system support for C++ modules.

\h2#cxx-modules-intro|Modules Introduction|

The goal of this section is to provide a practical introduction to C++ Modules
and to establish key concepts and terminology.

A pre-modules C++ program or library consists of one or more \i{translation
units} which are customarily referred to as C++ source files. Translation
units are compiled to \i{object files} which are then linked together to
form a program or library.

Let's also recap the difference between an \i{external name} and a \i{symbol}:
External names refer to language entities, for example classes, functions, and
so on. The \i{external} qualifier means they are visible across translation
units.

Symbols are derived from external names for use inside object files. They are
the cross-referencing mechanism for linking a program from multiple,
separately-compiled translation units. Not all external names end up becoming
symbols and symbols are often \i{decorated} with additional information, for
example, a namespace. We often talk about a symbol having to be satisfied by
linking an object file or a library that provides it. Similarly, duplicate
symbol issues may arise if more than one object file or library provides
the same symbol.

What is a C++ module? It is hard to give a single but intuitive answer to this
question.  So we will try to answer it from three different perspectives: that
of a module consumer, a module producer, and a build system that tries to make
those two play nice. But we can make one thing clear at the outset: modules
are a \i{language-level} not a preprocessor-level mechanism; it is \c{import},
not \c{#import}.

One may also wonder why C++ modules, what are the benefits? Modules offer
isolation, both from preprocessor macros and other modules' symbols. Unlike
headers, modules require explicit exportation of entities that will be visible
to the consumers. In this sense they are a \i{physical design mechanism} that
forces us to think how we structure our code. Modules promise significant
build speedups since importing a module, unlike including a header, should be
essentially free. Modules are also the first step to not needing the
preprocessor in most translation units. Finally, modules have a chance of
bringing to mainstream reliable and easy to setup distributed C++ compilation,
since with modules build systems can make sure compilers on the local and
remote hosts are provided with identical inputs.

To refer to a module we use a \i{module name}, a sequence of dot-separated
identifiers, for example \c{hello.core}. While the specification does not
assign any hierarchical semantics to this sequence, it is customary to refer
to \c{hello.core} as a submodule of \c{hello}. We discuss submodules and
provide the module naming guidelines below.

From a consumer's perspective, a module is a collection of external names,
called \i{module interface}, that become \i{visible} once the module is
imported:

\
import hello.core
\

What exactly does \i{visible} mean? To quote the standard: \i{An
import-declaration makes exported declarations [...] visible to name lookup in
the current translation unit, in the same namespaces and contexts [...]. [
Note: The entities are not redeclared in the translation unit containing the
module import declaration. -- end note ]} One intuitive way to think about
this visibility is \i{as if} there were only a single translation unit for the
entire program that contained all the modules as well as all their
consumers. In such a translation unit all the names would be visible to
everyone in exactly the same way and no entity would be redeclared.

This visibility semantics suggests that modules are not a name scoping
mechanism and are orthogonal to namespaces. Specifically, a module can export
names from any number of namespaces, including the global namespace. While the
module name and its namespace names need not be related, it usually makes
sense to have a parallel naming scheme, as discussed below. Finally, the
\c{import} declaration does not imply any additional visibility for names
declared inside namespaces. Specifically, to access such names we must
continue using the standard mechanisms, such as qualification or using
declaration/directive.  For example:

\
import hello.core;        // Exports hello::say().

say ();                   // Error.
hello::say ();            // Ok.

using namespace hello;
say ();                   // Ok.
\

Note also that from the consumer's perspective a module does not provide
any symbols, only C++ entity names. If we use names from a module, then we
may have to satisfy the corresponding symbols using the usual mechanisms:
link an object file or a library that provides them. In this respect, modules
are similar to headers and as with headers, module's use is not limited to
libraries; they make perfect sense when structuring programs. Furthermore,
a library may also have private or implementation modules that are not
meant to be consumed by the library's users.

The producer perspective on modules is predictably more complex. In
pre-modules C++ we only had one kind of translation unit (or source
file). With modules there are three kinds: \i{module interface unit},
\i{module implementation unit}, and the original kind which we will
call a \i{non-module translation unit}.

From the producer's perspective, a module is a collection of module translation
units: one interface unit and zero or more implementation units. A simple
module may consist of just the interface unit that includes implementations
of all its functions (not necessarily inline). A more complex module may
span multiple implementation units.

A translation unit is a module interface unit if it contains an \i{exporting
module declaration}:

\
export module hello.core;
\

A translation unit is a module implementation unit if it contains a
\i{non-exporting module declaration}:

\
module hello.core;
\

While module interface units may use the same file extension as normal source
files, we recommend that a different extension be used to distinguish them as
such, similar to header files. While the compiler vendors suggest various (and
predictably different) extensions, our recommendation is \c{.mxx} for the
\c{.hxx/.cxx} source file naming and \c{.mpp} for \c{.hpp/.cpp}. And if you
are using some other naming scheme, then perhaps now is a good opportunity to
switch to one of the above. Continuing using the source file extension for
module implementation units appears reasonable and that's what we recommend.

A module declaration (exporting or non-exporting) starts a \i{module purview}
that extends until the end of the module translation unit. Any name declared
in a module's purview \i{belongs} to said module. For example:

\
#include <string>                // Not in purview.

export module hello.core;        // Start of purview.

void
say_hello (const std::string&);  // In purview.
\

A name that belongs to a module is \i{invisible} to the module's consumers
unless it is \i{exported}. A name can be declared exported only in a module
interface unit, only in the module's purview, and there are several syntactic
ways to accomplish this. We can start the declaration with the \c{export}
specifier, for example:

\
export module hello.core;

export enum class volume {quiet, normal, loud};

export void
say_hello (const char*, volume);
\

Alternatively, we can enclose one or more declarations into an \i{exported
group}, for example:

\
export module hello.core;

export
{
  enum class volume {quiet, normal, loud};

  void
  say_hello (const char*, volume);
}
\

Finally, if a namespace definition is declared exported, then every name
in its body is exported, for example:

\
export module hello.core;

export namespace hello
{
  enum class volume {quiet, normal, loud};

  void
  say (const char*, volume);
}

namespace hello
{
  void
  impl (const char*, volume); // Not exported.
}
\

Up until now we've only been talking about names belonding to a module. What
about the corresponding symbols? For exported names, the resulting symbols
would be the same as if those names were declared outside of a module's
purview (or as if no modules were used at all). Non-exported names, on the
other hand, have \i{module linkage}: their symbols can be resolved from this
module's units but not from other translation units. They also cannot clash
with symbols for identical names from other modules (and non-modules). This is
usually achieved by decorating the non-exported symbols with the module name.

This ownership model has an important backwards compatibility implication: a
library built with modules enabled can be linked to a program that still uses
headers. And even the other way around: we can build and use a module for a
library that was built with headers.

What about the preprocessor? Modules do not export preprocessor macros,
only C++ names. A macro defined in the module interface unit cannot affect
the module's consumers. And macros defined by the module's consumers cannot
affect the module interface they are importing. In other words, module
producers and consumers are isolated from each other when the preprocessor
is concerned. For example, consider this module interface:

\
export module hello;

#ifndef SMALL
#define HELLO
export void say_hello (const char*);
#endif
\

And its consumer:

\
// module consumer
//
#define SMALL       // No effect.
import hello;

#ifdef HELLO        // Not defined.
...
#endif
\

This is not to say that the preprocessor cannot be used by either, it just
doesn't \"leak\" through the module interface. One practical implication of
this model is the insignificance of the import order.

If a module imports another module in its purview, the imported module's
names are not made automatically visible to the consumers of the importing
module. This is unlike headers and can be surprising. Consider this module
interface as an example:

\
export module hello;

import std.core;

export void
say_hello (const std::string&);
\

And its consumer:

\
import hello;

int
main ()
{
  say_hello (\"World\");
}
\

This example will result in a compile error and the diagnostics may
confusingly indicate that there is no known conversion from a C string to
\"something\" called \c{std::string}. But with the understanding of the
difference between \c{import} and \c{#include} the reason should be clear:
while the module interface \"sees\" \c{std::string} (because it imported its
module), we (the consumer) do not (since we did not). So the fix is to
explicitly import \c{std.core}:

\
import std.core;
import hello;

int
main ()
{
  say_hello (\"World\");
}
\

A module, however, can choose to re-export a module it imports. In this case,
all the names from the imported module will also be visible to the importing
module's consumers. For example, with this change to the module interface the
first version of our consumer will compile without errors (note that whether
this is a good design choice is debatable, as discussed below):

\
export module hello;

export import std.core;

export void
say_hello (const std::string&);
\

One way to think of a re-export is \i{as if} an import of a module also
\"injects\" all the imports said module re-exports, recursively. That's
essentially how most compilers implement it.

Module re-export is the mechanism for assembling bigger modules out of
submodules. As an example, let's say we had the \c{hello.core},
\c{hello.basic}, and \c{hello.extra} modules. To make life easier for users
that want to import all of them we can create the \c{hello} module that
re-exports the three:

\
export module hello;

export
{
  import hello.core;
  import hello.basic;
  import hello.extra;
}
\

Besides starting a module purview, a non-exporting module declaration in the
implementation unit makes non-internal linkage names declared or made visible
in the \i{interface purview} also visible in the \i{implementation purview}.
In this sense non-exporting module declaration acts as an extended
\c{import}. For example:

\
import hello.impl;          // Not visible (exports impl()).

void
extra_impl ();              // Not visible.

export module hello.extra;  // Start of interface purview.

import hello.core;          // Visible (exports core()).

void
extra ();                   // Visible.

static void
extra2 ();                  // Not visible (internal linkage).
\

And this is the implementation unit:

\
module hello.extra;         // Start of implementation purview.

void
f ()
{
  impl ();        // Error.
  extra_impl ();  // Error.
  core ();        // Ok.
  extra ();       // Ok.
  extra2 ();      // Error.
}
\

In particular, this means that while the relative order of imports is not
significant, the placement of imports in the module interface unit relative
to the module declaration can be.

The final perspective that we consider is that of the build system. From its
point of view the central piece of the module infrastructure is the \i{binary
module interface}: a binary file that is produced by compiling the module
interface unit and that is required when compiling any translation unit that
imports this module as well as the module's implementation units.

Then, in a nutshell, the main functionality of a build system when it comes to
modules support is figuring out the order in which all the translation units
should be compiled and making sure that every compilation process is able to
find the binary module interfaces it needs.

Predictably, the details are more complex. Compiling a module interface unit
produces two outputs: the binary module interface and the object file. The
latter contains object code for non-inline functions, global variables, etc.,
that the interface unit may define. This object file has to be linked when
producing any binary (program or library) that uses this module.

Also, all the compilers currently implement module re-export as a shallow
reference to the re-exported module name which means that their binary
interfaces must be discoverable as well, recursively. In fact, currently, all
the imports are handled like this, though a different implementation is at
least plausible, if unlikely.

While the details vary between compilers, the contents of the binary module
interface can range from a stream of preprocessed tokens to something fairly
close to object code. As a result, binary interfaces can be sensitive to the
compiler options and if the options used to produce the binary interface (for
example, when building a library) are sufficiently different compared to the
ones used when compiling the module consumers, the binary interface may be
unusable. So while a build system should strive to reuse existing binary
interfaces, it should also be prepared to compile its own versions \"on the
side\".

This also suggests that binary module interfaces are not a distribution
mechanism and should probably not be installed. Instead, we should install and
distribute module interface sources and build systems should be prepared to
compile them, again, on the side.


\h2#cxx-modules-build|Building Modules|

Compiler support for C++ Modules is still experimental. As a result, it is
currently only enabled if the C++ standard is set to \c{experimental}. After
loading the \c{cxx} module we can check if modules are enabled using the
\c{cxx.features.modules} boolean variable. This is what the relevant
\c{root.build} fragment could look like for a modularized project:

\
cxx.std = experimental

using cxx

assert $cxx.features.modules 'compiler does not support modules'

mxx{*}: extension = mxx
cxx{*}: extension = cxx
\

To support C++ modules the \c{cxx} module (build system) defines several
additional target types. The \c{mxx{\}} target is a module interface unit.
As you can see from the above \c{root.build} fragment, in this project we
are using the \c{.mxx} extension for our module interface files. While
you can use the same extension as for \c{cxx{\}} (source files), this is
not recommended since some functionality, such as wildcard patterns, will
become unusable.

The \c{bmi{\}} group and its \c{bmie{\}}, \c{bmia{\}}, and \c{bmis{\}} members
are used to represent binary module interfaces targets. We normally do not
need to mention them explicitly in our buildfiles except, perhaps, to specify
additional, module interface-specific compile options. We will see some
examples of this below.

To build a modularized executable or library we simply list the module
interfaces as its prerequisites, just as we do for source files. As an
example, let's build the \c{hello} program that we have started in the
introduction (you can find the complete project in the
\l{https://build2.org/pkg/hello Hello Repository} under
\c{mhello}). Specifically, we assume our project contains the following files:

\
// file: hello.mxx (module interface)

export module hello;

import std.core;

export void
say_hello (const std::string&);
\

\
// file: hello.cxx (module implementation)

module hello;

import std.io;

using namespace std;

void
say_hello (const string& name)
{
  cout << \"Hello, \" << name << '!' << endl;
}
\

\
// file: driver.cxx

import std.core;
import hello;

int
main ()
{
  say_hello (\"World\");
}
\

To build a \c{hello} executable from these files we can write the following
\c{buildfile}:

\
exe{hello}: cxx{driver} {mxx cxx}{hello}
\

Or, if you prefer to use wildcard patterns:

\
exe{hello}: {mxx cxx}{*}
\

Alternatively, we can package the module into a library and then link the
library to the executable:

\
exe{hello}: cxx{driver} lib{hello}
lib{hello}: {mxx cxx}{hello}
\

As you might have surmised from this example, the modules support in
\c{build2} automatically resolves imports to module interface units that are
specified either as direct prerequisites or as prerequisites of library
prerequisites.

To perform this resolution without a significant overhead, the implementation
delays the extraction of the actual module name from module interface units
(since not all available module interfaces are necessarily imported by all the
translation units). Instead, the implementation tries to guess which interface
unit implements each module being imported based on the interface file
path. Or, more precisely, a two-step resolution process is performed: first a
best match between the desired module name and the file path is sought and
then the actual module name is extracted and the correctness of the initial
guess is verified.

The practical implication of this implementation detail is that our module
interface files must embed a portion of a module name, or, more precisely, a
sufficient amount of \"module name tail\" to unambiguously resolve all the
modules used in a project. Note also that this guesswork is only performed for
direct module interface prerequisites; for those that come from libraries the
module names are known and are therefore matched exactly.

As an example, let's assume our \c{hello} project had two modules:
\c{hello.core} and \c{hello.extra}. While we could call our interface files
\c{hello.core.mxx} and \c{hello.extra.mxx}, respectively, this doesn't look
particularly good and may be contrary to the file naming scheme used in our
project. To resolve this issue the match of module names to file names is
made \"fuzzy\": it is case-insensitive, it treats all separators (dots, dashes,
underscores, etc) as equal, and it treats a case change as an imaginary
separator. As a result, the following naming schemes will all match the
\c{hello.core} module name:

\
hello-core.mxx
hello_core.mxx
HelloCore.mxx
hello/core.mxx
\

We also don't have to embed the full module name. In our case, for example, it
would be most natural to call the files \c{core.mxx} and \c{extra.mxx} since
they are already in the project directory called \c{hello/}. This will work
since our module names can still be guessed correctly and unambiguously.

If a guess turns out to be incorrect, the implementation issues diagnostics
and exits with an error before attempting to build anything. To resolve this
situation we can either adjust the interface file names or we can specify the
module name explicitly with the \c{cxx.module_name} variable. The latter
approach can be used with interface file names that have nothing in common
with module names, for example:

\
mxx{foobar}@./: cxx.module_name = hello
\

Note also that standard library modules (\c{std} and \c{std.*}) are treated
specially: they are not fuzzy-matched and they need not be resolvable to
the corresponding \c{mxx{\}} or \c{bmi{\}} in which case it is assumed
they will be resolved in an ad hoc way by the compiler. This means that if
you want to build your own standard library module (for example, because
your compiler doesn't yet ship one; note that this may not be supported
by all compilers), then you have to specify the module name explicitly.
For example:

\
exe{hello}: cxx{driver} {mxx cxx}{hello} mxx{std-core}

mxx{std-core}@./: cxx.module_name = std.core
\

When C++ modules are enabled and available, the build system makes sure the
\c{__cpp_modules} feature test macro is defined. Currently, its value is
\c{201703} for VC and \c{201704} for GCC and Clang but this will most likely
change in the future.

One major difference between the current C++ modules implementation in VC and
the other two compilers is the use of the \c{export module} syntax to identify
the interface units. While both GCC and Clang have adopted this new syntax,
VC is still using the old one without the \c{export} keyword. We can use the
\c{__cpp_modules} macro to provide a portable declaration:

\
#if __cpp_modules >= 201704
export
#endif
module hello;
\

Note, however, that the modules support in \c{build2} provides temporary
\"magic\" that allows us to use the new syntax even with VC (don't ask how).

\h2#cxx-modules-symexport|Module Symbols Exporting|

When building a shared library, some platforms (notably Windows) require that
we explicitly export symbols that must be accessible to the library users.
If you don't need to support such platforms, you can thank your lucky stars
and skip this section.

When using headers, the traditional way of achieving this is via an \"export
macro\" that is used to mark exported APIs, for example:

\
LIBHELLO_EXPORT void
say_hello (const string&);
\

This macro is then appropriately defined (often in a separate \"export
header\") to export symbols when building the shared library and to import
them when building the library's users.

The introduction of modules changes this in a number of ways, at least as
implemented by VC (hopefully other compilers will follow suit). While we
still have to explicitly mark exported symbols in our module interface
unit, there is no need (and, in fact, no way) to do the same when said
module is imported. Instead, the compiler automatically treats all
such explicitly exported symbols (note: symbols, not names) as imported.

One notable aspect of this new model is the locality of the export macro: it
is only defined when compiling the module interface unit and is not visible to
the consumers of the module. This is unlike headers where the macro has to
have a unique per-library name (that \c{LIBHELLO_} prefix) because a header
from one library can be included while building another library.

We can continue using the same export macro and header with modules and, in
fact, that's the recommended approach when maintaining the dual, header/module
arrangement for backwards compatibility (discussed below). However, for
modules-only codebases, we have an opportunity to improve the situation in two
ways: we can use a single, keyword-like macro instead of a library-specific
one and we can make the build system manage it for us thus getting rid of the
export header.

To enable this functionality in \c{build2} we set the
\c{cxx.features.symexport} boolean variable to \c{true} before loading the
\c{cxx} module. For example:

\
cxx.std = experimental

cxx.features.symexport = true

using cxx

...
\

Once enabled, \c{build2} automatically defines the \c{__symexport} macro to
the appropriate value depending on the platform and the type of library being
built. As library authors, all we have to do is use it in appropriate places
in our module interface units, for example:

\
export module hello;

import std.core;

export __symexport void
say_hello (const std::string&);
\

As an aside, you may be wondering why can't a module export automatically mean
a symbol export? While you will normally want to export symbols of all your
module-exported names, you may also need to do so for some non-module-exported
ones. For example:

\
export module foo;

__symexport void
f_impl ();

export __symexport inline void
f ()
{
  f_impl ();
}
\

Furthermore, symbol exporting is a murky area with many limitations and
pitfalls (such as auto-exporting of base classes). As a result, it would not
be unreasonable to expect such an automatic module exporting to only further
muddy the matter.


\h2#cxx-modules-install|Modules Installation|

As discussed in the introduction, binary module interfaces are not a
distribution mechanism and installing module interface sources appears to be
the preferred approach.

Module interface units are by default installed in the same location as
headers (for example, \c{/usr/include}). However, instead of relying on a
header-like search mechanism (\c{-I} paths, etc.), an explicit list of
exported modules is provided for each library in its \c{.pc} (\c{pkg-config})
file.

Specifically, the library's \c{.pc} file contains the \c{cxx_modules} variable
that lists all the exported C++ modules in the \c{<name>=<path>} form with
\c{<name>} being the module's C++ name and \c{<path>} \- the module interface
file's absolute path. For example:

\
Name: libhello
Version: 1.0.0
Cflags:
Libs: -L/usr/lib -lhello

cxx_modules = hello.core=/usr/include/hello/core.mxx hello.extra=/usr/include/hello/extra.mxx
\

Additional module properties are specified with variables in the
\c{cxx_module_<property>.<name>} form, for example:

\
cxx_module_symexport.hello.core = true
cxx_module_preprocessed.hello.core = all
\

Currently, two properties are defined. The \c{symexport} property with the
boolean value signals whether the module uses the \c{__symexport} support
discussed above.

The \c{preprocessed} property indicates the degree of preprocessing the module
unit requires and is used to optimize module compilation. Valid values are
\c{none} (not preprocessed), \c{includes} (no \c{#include} directives in the
source), \c{modules} (as above plus no module declarations depend on the
preprocessor, for example, \c{#ifdef}, etc.), and \c{all} (the source is fully
preprocessed). Note that for \c{all} the source may still contain comments and
line continuations.


\h2#cxx-modules-guidelines|Modules Design Guidelines|

Modules are a physical design mechanism for structuring and organizing our
code. Their explicit exportation semantics combined with the way they are
built make many aspects of creating and consuming modules significantly
different compared to headers. This section provides basic guidelines for
designing modules. We start with the overall considerations such as module
granularity and partitioning into translation units then continue with the
structure of typical module interface and implementation units. The following
section discusses practical approaches to modularizing existing code and
providing dual, header/module interfaces for backwards-compatibility.

Unlike headers, the cost of importing modules should be negligible. As a
result, it may be tempting to create \"mega-modules\", for example, one per
library. After all, this is how the standard library is modularized with its
fairly large \c{std.core} and \c{std.io} modules.

There is, however, a significant drawback to this choice: every time we make a
change, all consumers of such a mega-module will have to be recompiled,
whether the change affects them or not. And the bigger the module the higher
the chance that any given change does not (semantically) affect a large
portion of the module's consumers. Note also that this is not an issue for the
standard library modules since they are not expected to change often.

Another, more subtle, issue with mega-modules (which does affect the standard
library) is the inability to re-export only specific interfaces, as will be
discussed below.

The other extreme in choosing module granularity is a large number of
\"mini-modules\". Their main drawback is the tediousness of importation by the
consumers.

The sensible approach is then to create modules of conceptually-related and
commonly-used entities possibly complemented with aggregate modules for ease
of importation. This also happens to be generally good design.

As an example, let's consider an XML library that provides support for both
parsing and serialization. Since it is common for applications to only use one
of the functionalities, it makes sense to provide the \c{xml.parser} and
\c{xml.serializer} modules. While it is not too tedious to import both, for
convenience we could also provide the \c{xml} module that re-exports the two.

Once we are past selecting an appropriate granularity for our modules, the
next question is how to partition them into translation units. A module can
consist of just the interface unit and, as discussed above, such a unit can
contain anything an implementation unit can, including non-inline function
definitions. Some may then view this as an opportunity to get rid of the
header/source separation and have everything in a single file.

There are a number of drawbacks with this approach: Every time we change
anything in the module interface unit, all its consumers have to be
recompiled. If we keep everything in a single file, then every time we change
the implementation we trigger recompilations that would have been avoided had
the implementation been factored out into a separate unit. Note that a build
system in cooperation with the compiler could theoretically avoid such
unnecessary recompilations: if the compiler produces identical binary
interface files when the module interface is unchanged, then the build system
could detect this and skip recompiling the module's consumers.

A related issue with single-file modules is the reduction in the build
parallelization opportunities. If the implementation is part of the interface
unit, then the build system cannot start compiling the module's consumers
until both the interface and the implementation are compiled. On the other
hand, had the implementation been split into a separate file, the build system
could start compiling the module's consumers (as well as the implementation
unit) as soon as the module interface is compiled.

Another issues with combining the interface with the implementation is the
readability of the interface which could be significantly reduced if littered
with implementation details. We could keep the interface separate by moving
the implementation to the bottom of the interface file but then we might as
well move it into a separate file and avoid the unnecessary recompilations or
parallelization issues.

The sensible guideline is then to have a separate module implementation unit
except perhaps for modules with a simple implementation that is mostly
inline/template. Note that more complex modules may have several
implementation units, however, based on our granularity guideline, those
should be rare.

Once we start writing our first real module the immediate question that
normally comes up is where to put \c{#include} directives and \c{import}
declarations and in what order. To recap, a module unit, both interface and
implementation, is split into two parts: before the module declaration which
obeys the usual or \"old\" translation unit rules and after the module
declaration which is the module purview. Inside the module purview all
non-exported declarations have module linkage which means their symbols are
invisible to any other module (including the global module). With this
understanding, consider the following module interface:

\
export module hello;

#include <string>
\

Do you see the problem? We have included \c{<string>} in the module purview
which means all its names (as well as all the names in any headers it might
include, recursively) are now declared as having the \c{hello} module linkage.
The result of doing this can range from silent code blot to strange-looking
unresolved symbols.

The guideline this leads to should be clear: including a header in the module
purview is almost always a bad idea. There are, however, a few types of
headers that may make sense to include in the module purview. The first are
headers that only define preprocessor macros, for example, configuration or
export headers. There are also cases where we do want the included
declarations to end up in the module purview. The most common example is
inline/template function implementations that have been factored out into
separate files for code organization reasons. As an example, consider the
following module interface that uses an export header (which presumably sets
up symbols exporting macros) as well as an inline file:

\
#include <string>

export module hello;

#include <libhello/export.hxx>

export namespace hello
{
  ...
}

#include <libhello/hello.ixx>
\

A note on inline/template files: in header-based projects we could include
additional headers in those files, for example, if the included declarations
are only needed in the implementation. For the reasons just discussed, this
does not work with modules and we have to move all the includes into the
interface file, before the module purview. On the other hand, with modules, it
is safe to use namespace-level using-directives (for example, \c{using
namespace std;}) in inline/template files (and, with care, even in the
interface file).

What about imports, where should we import other modules? Again, to recap,
unlike a header inclusion, an \c{import} declaration only makes exported names
visible without redeclaring them. As result, in module implementation
units, it doesn't really matter where we place imports, in or out of the
module purview. There are, however, two differences when it comes to module
interface units: only imports in the purview are visible to implementation
units and we can only re-export an imported module from the purview.

The guideline is then for interface units to import in the module purview
unless there is a good reason not to make the import visible to the
implementation units. And for implementation units to always import in the
purview for consistency. For example:

\
#include <cassert>

export module hello;

import std.core;

#include <libhello/export.hxx>

export namespace hello
{
  ...
}

#include <libhello/hello.ixx>
\

By putting all these guidelines together we can then create a module interface
unit template:

\
// Module interface unit.

<header includes>

export module <name>;      // Start of module purview.

<module imports>

<special header includes>  // Configuration, export, etc.

<module interface>

<inline/template includes>
\

As well as the module implementation unit template:

\
// Module implementation unit.

<header includes>

module <name>;             // Start of module purview.

<extra module imports>     // Only additional to interface.

<module implementation>
\

Let's now discuss module naming. Module names are in a separate \"name plane\"
and do not collide with namespace, type, or function names. Also, as mentioned
earlier, the standard does not assign a hierarchical meaning to module names
though it is customary to assume module \c{hello.core} is a submodule of
\c{hello} and importing the latter also imports the former.

It is important to choose good names for public modules (that is, modules
packaged into libraries and used by a wide range of consumers) since changing
them later can be costly. We have more leeway with naming private modules
(that is, the ones used by programs or internal to libraries) though it's
worth coming up with a consistent naming scheme here as well.

The general guideline is to start names of public modules with the library's
namespace name followed by a name describing the module's functionality. In
particular, if a module is dedicated to a single class (or, more generally,
has a single primary entity), then it makes sense to use its name as the
module name's last component.

As a concrete example, consider \c{libbutl} (the \c{build2} utility library):
All its components are in the \c{butl} namespace so all its module names start
with \c{butl.} One of its components is the \c{small_vector} class template
which resides in its own module called \c{butl.small_vector}. Another
component is a collection of string parsing utilities that are grouped into
the \c{butl::string_parser} namespace with the corresponding module called
\c{butl.string_parser}.

When is it a good idea to re-export a module? The two straightforward cases
are when we are building an aggregate module out of submodules, for example,
\c{xml} out of \c{xml.parser} and \c{xml.serializer}, or when one module
extends or supersedes another, for example, as \c{std.core} extends
\c{std.fundamental}. It is also clear that there is no need to re-export a
module that we only use in the implementation. The case when we use a module
in our interface is, however, a lot less clear cut.

But before considering the last case in more detail, let's understand the
issue with re-export. In other words, why not simply re-export any module we
import in our interface? In essence, re-export implicitly injects another
module import anywhere our module is imported. If we re-export \c{std.core}
then consumers of our module will also automatically \"see\" all the names
exported by \c{std.core}. They can then start using names from \c{std} without
explicitly importing \c{std.core} and everything will compile until one day
they no longer need to import our module or we no longer need to import
\c{std.core}. In a sense, re-export becomes part of our interface and it is
generally good design to keep interfaces minimal.

And so, at the outset, the guideline is then to only re-export the minimum
necessary. This, by the way, is the reason why it may make sense to divide
\c{std.core} into submodules such as \c{std.core.string}, \c{std.core.vector},
etc.

Let's now discuss a few concrete examples to get a sense of when re-export
might or might not be appropriate. Unfortunately, there does not seem to be a
hard and fast rule and instead one has to rely on their good sense of design.

To start, let's consider a simple module that uses \c{std::string} in its
interface:

\
export module hello;

import std.core;

export namespace hello
{
  void say (const std::string&);
}
\

Should we re-export \c{std.core} (or, \c{std.core.string}) in this case? Most
likely not. If consumers of our module want to use \c{std::string} in order to
pass an argument to our function, then it is natural to expect them to
explicitly import the necessary module. In a sense, this is analogous to
scoping: nobody expects to be able to use just \c{string} (without \c{std::})
because of \c{using namespace hello;}.

So it seems that a mere usage of a name in an interface does not generally
warrant a re-export. The fact that a consumer may not even use this part of
our interface further supports this conclusion.

Let's now consider a more interesting case (inspired by real events):

\
export module small_vector;

import std.core;

template <typename T, std::size_t N>
export class small_vector: public std::vector<T, ...>
{
  ...
};
\

Here we have the \c{small_vector} container implemented in terms of
\c{std::vector} by providing a custom allocator and with most of the functions
derived as is. Consider now this innocent-looking consumer code:

\
import small_vector;

small_vector<int, 1> a, b;

if (a == b) // Error.
  ...
\

We don't reference \c{std::vector} directly so presumably we shouldn't need to
import its module. However, the comparison won't compile: our \c{small_vector}
implementation re-uses the comparison operators provided by \c{std::vector}
(via implicit to-base conversion) but they aren't visible.

There is a palpable difference between the two cases: the first merely uses
\c{std.core} interface while the second is \i{based on} and, in a sense,
\i{extends} it which feels like a stronger relationship. Re-exporting
\c{std.core} (or, better yet, \c{std.core.vector}, should it become available)
does not seem unreasonable.

Note also that there is no re-export of headers nor header inclusion
visibility in the implementation units. Specifically, in the previous example,
if the standard library is not modularized and we have to use it via headers,
then the consumers of our \c{small_vector} will always have to explicitly
include \c{<vector>}. This suggest that modularizing a codebase that still
consumes substantial components (like the standard library) via headers can
incur some development overhead compared to the old, headers-only approach.


\h2#cxx-modules-existing|Modularizing Existing Code|

The aim of this section is to provide practical guidelines to modularizing
existing codebases as well as supporting the dual, header/module interface for
backwards-compatibility.

Predictably, a well modularized (in the general sense) set of headers makes
conversion to C++ modules easier. Inclusion cycles will be particularly hard
to deal with (C++ modules do not allow circular interface dependencies).
Furthermore, as we will see below, if you plan to provide the dual
header/module interface, then having a one-to-one header to module mapping
will simplify this task. As a result, it may make sense to spend some time
cleaning and re-organizing your headers prior to attempting modularization.

Let's first discuss why the modularization approach illustrated by the
following example does not generally work:

\
export module hello;

export
{
#include \"hello.hxx\"
}
\

There are several issue that usually make this unworkable. Firstly, the header
we are trying to export most likely includes other headers. For example, our
\c{hello.hxx} may include \c{<string>} and we have already discussed why
including it in the module purview, let alone exporting its names, is a bad
idea. Secondly, the included header may declare more names than what should be
exported, for example, some implementation details. In fact, it may declare
names with internal linkage (uncommon for headers but not impossible) which
are illegal to export. Finally, the header may define macros which will no
longer be visible to the consumers.

Sometimes, however, this can be the only approach available (for example, if
trying to non-intrusively modularize a third-party library). It is possible to
work around the first issue by \i{pre-including} outside of the module purview
headers that should not be exported. Here we rely on the fact that the second
inclusion of the same header will be ignored. For example:

\
#include <string> // Pre-include to suppress inclusion below.

export module hello;

export
{
#include \"hello.hxx\"
}
\

Needless to say this approach is very brittle and usually requires that you
place all the inter-related headers into a single module. As a result, its use
is best limited to exploratory modularization and early prototyping.

When starting modularization of a codebase there are two decisions we have to
make at the outset: the level of the C++ modules support we can assume and the
level of backwards compatibility we need to provide.

The two modules support levels we distinguish are just modules and modules
with the modularized standard library. The choice we have to make then is
whether to support the standard library only as headers, only as modules, or
both. Note that some compiler/standard library combinations may not be usable
in some of these modes.

The possible backwards compatibility levels are \i{modules-only} (consumption
via headers is no longer supported), \i{modules-or-headers} (consumption
either via headers or modules), and \i{modules-and-headers} (as the previous
case but with support for consuming a library built with modules via headers
and vice versa).

What kind of situations call for the last level? We may need to continue
offering the library as headers if we have a large number of existing
consumers that cannot possibly be all modularized at once (or even ever). So
the situation we may end up in is a mixture of consumers trying to use the
same build of our library with some of them using modules and some \-
headers. The case where we may want to consume a library built with headers
via modules is not as far fetched as it may seem: the library might have been
built with an older version of the compiler (for example, it was installed
from a distribution's package) while the consumer is being built with a
compiler version that supports modules. Note also that as discussed earlier
the modules ownership semantics supports both kinds of such \"cross-usage\".

Generally, compiler implementations do not support mixing inclusion and
importation of the same entities in the same translation unit. This makes
migration tricky if you plan to use the modularized standard library because
of its pervasive use. There are two plausible strategies to handling this
aspect of migration: If you are planning to consume the standard library
exclusively as modules, then it may make sense to first change your entire
codebase to do that. Simply replace all the standard library header inclusions
with importation of the relevant \c{std.*} modules.

The alternative strategy is to first complete the modularization of our entire
project (as discussed next) while continuing consuming the standard library as
headers. Once this is done, we can normally switch to using the modularized
standard library quite easily. The reason for waiting until the complete
modularization is to eliminate header inclusions between components which
would often result in conflicting styles of the standard library consumption.

Note also that due to the lack of header re-export and include visibility
support discussed earlier, it may make perfect sense to only support the
modularized standard library when modules are enabled even when providing
backwards compatibility with headers. In fact, if all the compiler/standard
library implementations that your project caters to support the modularized
standard library, then there is little sense not to impose such a restriction.

The overall strategy for modularizing our own components is to identify and
modularize inter-dependent sets of headers one at a time starting from the
lower-level components. This way any newly modularized set will only depend on
the already modularized ones. After converting each set we can switch its
consumers to using imports keeping our entire project buildable and usable.

While ideally we would want to be able to modularize just a single component
at a time, this does not seem to work in practice because we will have to
continue consuming some of the components as headers. Since such headers can
only be imported out of the module purview, it becomes hard to reason (both
for us and often the compiler) what is imported/included and where. For
example, it's not uncommon to end up importing the module in its
implementation unit which is not something that all the compilers can handle
gracefully.

Let's now explore how we can provide the various levels of backwards
compatibility discussed above. Here we rely on two feature test macros to
determine the available modules support level: \c{__cpp_modules} (modules are
available) and \c{__cpp_lib_modules} (standard library modules are available,
assumes \c{__cpp_modules} is also defined).

If backwards compatibility is not necessary (the \i{modules-only} level), then
we can use the module interface and implementation unit templates presented
earlier and follow the above guidelines. If we continue consuming the standard
library as headers, then we don't need to change anything in this area. If we
only want to support the modularized standard library, then we simply replace
the standard library header inclusions with the corresponding module
imports. If we want to support both ways, then we can use the following
templates. The module interface unit template:

\
// C includes, if any.

#ifndef __cpp_lib_modules
<std includes>
#endif

// Other includes, if any.

export module <name>;

#ifdef __cpp_lib_modules
<std imports>
#endif

<module interface>
\

The module implementation unit template:

\
// C includes, if any.

#ifndef __cpp_lib_modules
<std includes>

<extra std includes>
#endif

// Other includes, if any.

module <name>;

#ifdef __cpp_lib_modules
<extra std imports>        // Only additional to interface.
#endif

<module implementation>
\

For example:

\
// hello.mxx (module interface)

#ifndef __cpp_lib_modules
#include <string>
#endif

export module hello;

#ifdef __cpp_lib_modules
import std.core;
#endif

export void say_hello (const std::string& name);
\

\
// hello.cxx (module implementation)

#ifndef __cpp_lib_modules
#include <string>

#include <iostream>
#endif

module hello;

#ifdef __cpp_lib_modules
import std.io;
#endif

using namespace std;

void say_hello (const string& n)
{
  cout << \"Hello, \" << n << '!' << endl;
}
\

If we need support for symbol exporting in this setup (that is, we are
building a library and need to support Windows), then we can use the
\c{__symexport} mechanism discussed earlier, for example:

\
// hello.mxx (module interface)

...

export __symexport void say_hello (const std::string& name);
\

The consumer code in the \i{modules-only} setup is straightforward: they
simply import the desired modules.

To support consumption via headers when modules are unavailable (the
\i{modules-or-headers} level) we can use the following setup. Here we also
support the dual header/modules consumption for the standard library (if this
is not required, replace \c{#ifndef __cpp_lib_modules} with \c{#ifndef
__cpp_modules} and remove \c{#ifdef __cpp_lib_modules}). The module interface
unit template:

\
#ifndef __cpp_modules
#pragma once
#endif

// C includes, if any.

#ifndef __cpp_lib_modules
<std includes>
#endif

// Other includes, if any.

#ifdef __cpp_modules
export module <name>;

#ifdef __cpp_lib_modules
<std imports>
#endif
#endif

<module interface>
\

The module implementation unit template:

\
#ifndef __cpp_modules
#include <module interface file>
#endif

// C includes, if any.

#ifndef __cpp_lib_modules
<std includes>

<extra std includes>
#endif

// Other includes, if any

#ifdef __cpp_modules
module <name>;

#ifdef __cpp_lib_modules
<extra std imports>        // Only additional to interface.
#endif
#endif

<module implementation>
\

Notice the need to repeat \c{<std includes>} in the implementation file due to
the lack of include visibility discussed above. This is necessary when modules
are enabled but the standard library is not modularized since in this case the
implementation does not \"see\" any of the headers included in the interface.

Besides these templates we will most likely also need an export header that
appropriately defines a module export macro depending on whether modules are
used or not. This is also the place where we can handle symbol exporting. For
example, here is what it could look like for our \c{libhello} library:

\
// export.hxx (module and symbol export)

#pragma once

#ifdef __cpp_modules
#  define LIBHELLO_MODEXPORT export
#else
#  define LIBHELLO_MODEXPORT
#endif

#if   defined(LIBHELLO_SHARED_BUILD)
#  ifdef _WIN32
#    define LIBHELLO_SYMEXPORT __declspec(dllexport)
#  else
#    define LIBHELLO_SYMEXPORT
#  endif
#elif defined(LIBHELLO_SHARED)
#  ifdef _WIN32
#    define LIBHELLO_SYMEXPORT __declspec(dllimport)
#  else
#    define LIBHELLO_SYMEXPORT
#  endif
#else
#  define LIBHELLO_SYMEXPORT
#endif
\

And this is the module that uses it and provides the dual header/module
support:

\
// hello.mxx (module interface)

#ifndef __cpp_modules
#pragma once
#endif

#ifndef __cpp_lib_modules
#include <string>
#endif

#ifdef __cpp_modules
export module hello;

#ifdef __cpp_lib_modules
import std.core;
#endif
#endif

#include <libhello/export.hxx>

LIBHELLO_MODEXPORT namespace hello
{
  LIBHELLO_SYMEXPORT void say (const std::string& name);
}
\

\
// hello.cxx (module implementation)

#ifndef __cpp_modules
#include <libhello/hello.mxx>
#endif

#ifndef __cpp_lib_modules
#include <string>

#include <iostream>
#endif

#ifdef __cpp_modules
module hello;

#ifdef __cpp_lib_modules
import std.io;
#endif
#endif

using namespace std;

namespace hello
{
  void say (const string& n)
  {
    cout << \"Hello, \" << n << '!' << endl;
  }
}
\

The consumer code in the \i{modules-or-headers} setup has to use either
inclusion or importation depending on the modules support availability, for
example:

\
#ifdef __cpp_modules
import hello;
#else
#include <libhello/hello.mxx>
#endif
\

Predictably, the final backwards compatibility level (\i{modules-and-headers})
is the most onerous to support. Here existing consumers have to continue
working with the modularized version of our library which means we have to
retain all the existing header files. We also cannot assume that just because
modules are available they are used (a consumer may still prefer headers),
which means we cannot rely on (only) the \c{__cpp_modules} and
\c{__cpp_lib_modules} macros to make the decisions.

One way to arrange this is to retain the headers and adjust them according to
the \i{modules-or-headers} template but with one important difference: instead
of using the standard module macros we use our custom ones (and we can also
have unconditional \c{#pragma once}). For example:

\
// hello.hxx (module header)

#pragma once

#ifndef LIBHELLO_LIB_MODULES
#include <string>
#endif

#ifdef LIBHELLO_MODULES
export module hello;

#ifdef LIBHELLO_LIB_MODULES
import std.core;
#endif
#endif

#include <libhello/export.hxx>

LIBHELLO_MODEXPORT namespace hello
{
  LIBHELLO_SYMEXPORT void say (const std::string& name);
}
\

Now if this header is included (for example, by an existing consumer) then
none of the \c{LIBHELLO_*MODULES} macros will be defined and the header will
act as, well, a plain old header. Note that we will also need to make the
equivalent change in the export header.

We also provide the module interface files which appropriately define the two
custom macros and then simply includes the corresponding headers:

\
// hello.mxx (module interface)

#ifdef __cpp_modules
#define LIBHELLO_MODULES
#endif

#ifdef __cpp_lib_modules
#define LIBHELLO_LIB_MODULES
#endif

#include <libhello/hello.hxx>
\

The module implementation unit can remain unchanged. In particular, we
continue including \c{hello.mxx} if modules support is unavailable. However,
if you find the use of different macros in the header and source files
confusing, then instead it can be adjusted as follows (note also that now we
are including \c{hello.hxx}):

\
// hello.cxx (module implementation)

#ifdef __cpp_modules
#define LIBHELLO_MODULES
#endif

#ifdef __cpp_lib_modules
#define LIBHELLO_LIB_MODULES
#endif

#ifndef LIBHELLO_MODULES
#include <libhello/hello.hxx>
#endif

#ifndef LIBHELLO_LIB_MODULES
#include <string>

#include <iostream>
#endif

#ifdef LIBHELLO_MODULES
module hello;

#ifdef LIBHELLO_LIB_MODULES
import std.io;
#endif
#endif

...
\

In this case it may also make sense to factor the \c{LIBHELLO_*MODULES} macro
definitions into a common header.

In the \i{modules-and-headers} setup the existing consumers that would like to
continue using headers don't require any changes. And for those that would
like to use modules if available the arrangement is the same as for the
\i{modules-or-headers} compatibility level.

If our module needs to \"export\" macros then the recommended approach is to
simply provide an additional header that the consumer includes. While it might
be tempting to also wrap the module import into this header, some may prefer
to explicitly import the module and include the header, especially if the
macros may not be needed by all consumers. This way we can also keep the
header macro-only which means it can be included freely, in or out of module
purviews.
"