aboutsummaryrefslogtreecommitdiff
path: root/doc/manual.cli
blob: e941b6ba843edfe7e68f7119f34fdcb65258b13f (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
2529
2530
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
2541
2542
2543
2544
2545
2546
2547
2548
2549
2550
2551
2552
2553
2554
2555
2556
2557
2558
2559
2560
2561
2562
2563
2564
2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649
2650
2651
2652
2653
2654
2655
2656
2657
2658
2659
2660
2661
2662
2663
2664
2665
2666
2667
2668
2669
2670
2671
2672
2673
2674
2675
2676
2677
2678
2679
2680
2681
2682
2683
2684
2685
2686
2687
2688
2689
2690
2691
2692
2693
2694
2695
2696
2697
2698
2699
2700
2701
2702
2703
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
2719
2720
2721
2722
2723
2724
2725
2726
2727
2728
2729
2730
2731
2732
2733
2734
2735
2736
2737
2738
2739
2740
2741
2742
2743
2744
2745
2746
2747
2748
2749
2750
2751
2752
2753
2754
2755
2756
2757
2758
2759
2760
2761
2762
2763
2764
2765
2766
2767
2768
2769
2770
2771
2772
2773
2774
2775
2776
2777
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
2818
2819
2820
2821
2822
2823
2824
2825
2826
2827
2828
2829
2830
2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
2856
2857
2858
2859
2860
2861
2862
2863
2864
2865
2866
2867
2868
2869
2870
2871
2872
2873
2874
2875
2876
2877
2878
2879
2880
2881
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
2906
2907
2908
2909
2910
2911
2912
2913
2914
2915
2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
2927
2928
2929
2930
2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
2971
2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986
2987
2988
2989
2990
2991
2992
2993
2994
2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
3040
3041
3042
3043
3044
3045
3046
3047
3048
3049
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
3063
3064
3065
3066
3067
3068
3069
3070
3071
3072
3073
3074
3075
3076
3077
3078
3079
3080
3081
3082
3083
3084
3085
3086
3087
3088
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181
3182
3183
3184
3185
3186
3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
3204
3205
3206
3207
3208
3209
3210
3211
3212
3213
3214
3215
3216
3217
3218
3219
3220
3221
3222
3223
3224
3225
3226
3227
3228
3229
3230
3231
3232
3233
3234
3235
3236
3237
3238
3239
3240
3241
3242
3243
3244
3245
3246
3247
3248
3249
3250
3251
3252
3253
3254
3255
3256
3257
3258
3259
3260
3261
3262
3263
3264
3265
3266
3267
3268
3269
3270
3271
3272
3273
3274
3275
3276
3277
3278
3279
3280
3281
3282
3283
3284
3285
3286
3287
3288
3289
3290
3291
3292
3293
3294
3295
3296
3297
3298
3299
3300
3301
3302
3303
3304
3305
3306
3307
3308
3309
3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324
3325
3326
3327
3328
3329
3330
3331
3332
3333
3334
3335
3336
3337
3338
3339
3340
3341
3342
3343
3344
3345
3346
3347
3348
3349
3350
3351
3352
3353
3354
3355
3356
3357
3358
3359
3360
3361
3362
3363
3364
3365
3366
3367
3368
3369
3370
3371
3372
3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408
3409
3410
3411
3412
3413
3414
3415
3416
3417
3418
3419
3420
3421
3422
3423
3424
3425
3426
3427
3428
3429
3430
3431
3432
3433
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462
3463
3464
3465
3466
3467
3468
3469
3470
3471
3472
3473
3474
3475
3476
3477
3478
3479
3480
3481
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491
3492
3493
3494
3495
3496
3497
3498
3499
3500
3501
3502
3503
3504
3505
3506
3507
3508
3509
3510
3511
3512
3513
3514
3515
3516
3517
3518
3519
3520
3521
3522
3523
3524
3525
3526
3527
3528
3529
3530
3531
3532
3533
3534
3535
3536
3537
3538
3539
3540
3541
3542
3543
3544
3545
3546
3547
3548
3549
3550
3551
3552
3553
3554
3555
3556
3557
3558
3559
3560
3561
3562
3563
3564
3565
3566
3567
3568
3569
3570
3571
3572
3573
3574
3575
3576
3577
3578
3579
3580
3581
3582
3583
3584
3585
3586
3587
3588
3589
3590
3591
3592
3593
3594
3595
3596
3597
3598
3599
3600
3601
3602
3603
3604
3605
3606
3607
3608
3609
3610
3611
3612
3613
3614
3615
3616
3617
3618
3619
3620
3621
3622
3623
3624
3625
3626
3627
3628
3629
3630
3631
3632
3633
3634
3635
3636
3637
3638
3639
3640
3641
3642
3643
3644
3645
3646
3647
3648
3649
3650
3651
3652
3653
3654
3655
3656
3657
3658
3659
3660
3661
3662
3663
3664
3665
3666
3667
3668
3669
3670
3671
3672
3673
3674
3675
3676
3677
3678
3679
3680
3681
3682
3683
3684
3685
3686
3687
3688
3689
3690
3691
3692
3693
3694
3695
3696
3697
3698
3699
3700
3701
3702
3703
3704
3705
3706
3707
3708
3709
3710
3711
3712
3713
3714
3715
3716
3717
3718
3719
3720
3721
3722
3723
3724
3725
3726
3727
3728
3729
3730
3731
3732
3733
3734
3735
3736
3737
3738
3739
3740
3741
3742
3743
3744
3745
3746
3747
3748
3749
3750
3751
3752
3753
3754
3755
3756
3757
3758
3759
3760
3761
3762
3763
3764
3765
3766
3767
3768
3769
3770
3771
3772
3773
3774
3775
3776
3777
3778
3779
3780
3781
3782
3783
3784
3785
3786
3787
3788
3789
3790
3791
3792
3793
3794
3795
3796
3797
3798
3799
3800
3801
3802
3803
3804
3805
3806
3807
3808
3809
3810
3811
3812
3813
3814
3815
3816
3817
3818
3819
3820
3821
3822
3823
3824
3825
3826
3827
3828
3829
3830
3831
3832
3833
3834
3835
3836
3837
3838
3839
3840
3841
3842
3843
3844
3845
3846
3847
3848
3849
3850
3851
3852
3853
3854
3855
3856
3857
3858
3859
3860
3861
3862
3863
3864
3865
3866
3867
3868
3869
3870
3871
3872
3873
3874
3875
3876
3877
3878
3879
3880
3881
3882
3883
3884
3885
3886
3887
3888
3889
3890
3891
3892
3893
3894
3895
3896
3897
3898
3899
3900
3901
3902
3903
3904
3905
3906
3907
3908
3909
3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
3920
3921
3922
3923
3924
3925
3926
3927
3928
3929
3930
3931
3932
3933
3934
3935
3936
3937
3938
3939
3940
3941
3942
3943
3944
3945
3946
3947
3948
3949
3950
3951
3952
3953
3954
3955
3956
3957
3958
3959
3960
3961
3962
3963
3964
3965
3966
3967
3968
3969
3970
3971
3972
3973
3974
3975
3976
3977
3978
3979
3980
3981
3982
3983
3984
3985
3986
3987
3988
3989
3990
3991
3992
3993
3994
3995
3996
3997
3998
3999
4000
4001
4002
4003
4004
4005
4006
4007
4008
4009
4010
4011
4012
4013
4014
4015
4016
4017
4018
4019
4020
4021
4022
4023
4024
4025
4026
4027
4028
4029
4030
4031
4032
4033
4034
4035
4036
4037
4038
4039
4040
4041
4042
4043
4044
4045
4046
4047
4048
4049
4050
4051
4052
4053
4054
4055
4056
4057
4058
4059
4060
4061
4062
4063
4064
4065
4066
4067
4068
4069
4070
4071
4072
4073
4074
4075
4076
4077
4078
4079
4080
4081
4082
4083
4084
4085
4086
4087
4088
4089
4090
4091
4092
4093
4094
4095
4096
4097
4098
4099
4100
4101
4102
4103
4104
4105
4106
4107
4108
4109
4110
4111
4112
4113
4114
4115
4116
4117
4118
4119
4120
4121
4122
4123
4124
4125
4126
4127
4128
4129
4130
4131
4132
4133
4134
4135
4136
4137
4138
4139
4140
4141
4142
4143
4144
4145
4146
4147
4148
4149
4150
4151
4152
4153
4154
4155
4156
4157
4158
4159
4160
4161
4162
4163
4164
4165
4166
4167
4168
4169
4170
4171
4172
4173
4174
4175
4176
4177
4178
4179
4180
4181
4182
4183
4184
4185
4186
4187
4188
4189
4190
4191
4192
4193
4194
4195
4196
4197
4198
4199
4200
4201
4202
4203
4204
4205
4206
4207
4208
4209
4210
4211
4212
4213
4214
4215
4216
4217
4218
4219
4220
4221
4222
4223
4224
4225
4226
4227
4228
4229
4230
4231
4232
4233
4234
4235
4236
4237
4238
4239
4240
4241
4242
4243
4244
4245
4246
4247
4248
4249
4250
4251
4252
4253
4254
4255
4256
4257
4258
4259
4260
4261
4262
4263
4264
4265
4266
4267
4268
4269
4270
4271
4272
4273
4274
4275
4276
4277
4278
4279
4280
4281
4282
4283
4284
4285
4286
4287
4288
4289
4290
4291
4292
4293
4294
4295
4296
4297
4298
4299
4300
4301
4302
4303
4304
4305
4306
4307
4308
4309
4310
4311
4312
4313
4314
4315
4316
4317
4318
4319
4320
4321
4322
4323
4324
4325
4326
4327
4328
4329
4330
4331
4332
4333
4334
4335
4336
4337
4338
4339
4340
4341
4342
4343
4344
4345
4346
4347
4348
4349
4350
4351
4352
4353
4354
4355
4356
4357
4358
4359
4360
4361
4362
4363
4364
4365
4366
4367
4368
4369
4370
4371
4372
4373
4374
4375
4376
4377
4378
4379
4380
4381
4382
4383
4384
4385
4386
4387
4388
4389
4390
4391
4392
4393
4394
4395
4396
4397
4398
4399
4400
4401
4402
4403
4404
4405
4406
4407
4408
4409
4410
4411
4412
4413
4414
4415
4416
4417
4418
4419
4420
4421
4422
4423
4424
4425
4426
4427
4428
4429
4430
4431
4432
4433
4434
4435
4436
4437
4438
4439
4440
4441
4442
4443
4444
4445
4446
4447
4448
4449
4450
4451
4452
4453
4454
4455
4456
4457
4458
4459
4460
4461
4462
4463
4464
4465
4466
4467
4468
4469
4470
4471
4472
4473
4474
4475
4476
4477
4478
4479
4480
4481
4482
4483
4484
4485
4486
4487
4488
4489
4490
4491
4492
4493
4494
4495
4496
4497
4498
4499
4500
4501
4502
4503
4504
4505
4506
4507
4508
4509
4510
4511
4512
4513
4514
4515
4516
4517
4518
4519
4520
4521
4522
4523
4524
4525
4526
4527
4528
4529
4530
4531
4532
4533
4534
4535
4536
4537
4538
4539
4540
4541
4542
4543
4544
4545
4546
4547
4548
4549
4550
4551
4552
4553
4554
4555
4556
4557
4558
4559
4560
4561
4562
4563
4564
4565
4566
4567
4568
4569
4570
4571
4572
4573
4574
4575
4576
4577
4578
4579
4580
4581
4582
4583
4584
4585
4586
4587
4588
4589
4590
4591
4592
4593
4594
4595
4596
4597
4598
4599
4600
4601
4602
4603
4604
4605
4606
4607
4608
4609
4610
4611
4612
4613
4614
4615
4616
4617
4618
4619
4620
4621
4622
4623
4624
4625
4626
4627
4628
4629
4630
4631
4632
4633
4634
4635
4636
4637
4638
4639
4640
4641
4642
4643
4644
4645
4646
4647
4648
4649
4650
4651
4652
4653
4654
4655
4656
4657
4658
4659
4660
4661
4662
4663
4664
4665
4666
4667
4668
4669
4670
4671
4672
4673
4674
4675
4676
4677
4678
4679
4680
4681
4682
4683
4684
4685
4686
4687
4688
4689
4690
4691
4692
4693
4694
4695
4696
4697
4698
4699
4700
4701
4702
4703
4704
4705
4706
4707
4708
4709
4710
4711
4712
4713
4714
4715
4716
4717
4718
4719
4720
4721
4722
4723
4724
4725
4726
4727
4728
4729
4730
4731
4732
4733
4734
4735
4736
4737
4738
4739
4740
4741
4742
4743
4744
4745
4746
4747
4748
4749
4750
4751
4752
4753
4754
4755
4756
4757
4758
4759
4760
4761
4762
4763
4764
4765
4766
4767
4768
4769
4770
4771
4772
4773
4774
4775
4776
4777
4778
4779
4780
4781
4782
4783
4784
4785
4786
4787
4788
4789
4790
4791
4792
4793
4794
4795
4796
4797
4798
4799
4800
4801
4802
4803
4804
4805
4806
4807
4808
4809
4810
4811
4812
4813
4814
4815
4816
4817
4818
4819
4820
4821
4822
4823
4824
4825
4826
4827
4828
4829
4830
4831
4832
4833
4834
4835
4836
4837
4838
4839
4840
4841
4842
4843
4844
4845
4846
4847
4848
4849
4850
4851
4852
4853
4854
4855
4856
4857
4858
4859
4860
4861
4862
4863
4864
4865
4866
4867
4868
4869
4870
4871
4872
4873
4874
4875
4876
4877
4878
4879
4880
4881
4882
4883
4884
4885
4886
4887
4888
4889
4890
4891
4892
4893
4894
4895
4896
4897
4898
4899
4900
4901
4902
4903
4904
4905
4906
4907
4908
4909
4910
4911
4912
4913
4914
4915
4916
4917
4918
4919
4920
4921
4922
4923
4924
4925
4926
4927
4928
4929
4930
4931
4932
4933
4934
4935
4936
4937
4938
4939
4940
4941
4942
4943
4944
4945
4946
4947
4948
4949
4950
4951
4952
4953
4954
4955
4956
4957
4958
4959
4960
4961
4962
4963
4964
4965
4966
4967
4968
4969
4970
4971
4972
4973
4974
4975
4976
4977
4978
4979
4980
4981
4982
4983
4984
4985
4986
4987
4988
4989
4990
4991
4992
4993
4994
4995
4996
4997
4998
4999
5000
5001
5002
5003
5004
5005
5006
5007
5008
5009
5010
5011
5012
5013
5014
5015
5016
5017
5018
5019
5020
5021
5022
5023
5024
5025
5026
5027
5028
5029
5030
5031
5032
5033
5034
5035
5036
5037
5038
5039
5040
5041
5042
5043
5044
5045
5046
5047
5048
5049
5050
5051
5052
5053
5054
5055
5056
5057
5058
5059
5060
5061
5062
5063
5064
5065
5066
5067
5068
5069
5070
5071
5072
5073
5074
5075
5076
5077
5078
5079
5080
5081
5082
5083
5084
5085
5086
5087
5088
5089
5090
5091
5092
5093
5094
5095
5096
5097
5098
5099
5100
5101
5102
5103
5104
5105
5106
5107
5108
5109
5110
5111
5112
5113
5114
5115
5116
5117
5118
5119
5120
5121
5122
5123
5124
5125
5126
5127
5128
5129
5130
5131
5132
5133
5134
5135
5136
5137
5138
5139
5140
5141
5142
5143
5144
5145
5146
5147
5148
5149
5150
5151
5152
5153
5154
5155
5156
5157
5158
5159
5160
5161
5162
5163
5164
5165
5166
5167
5168
5169
5170
5171
5172
5173
5174
5175
5176
5177
5178
5179
5180
5181
5182
5183
5184
5185
5186
5187
5188
5189
5190
5191
5192
5193
5194
5195
5196
5197
5198
5199
5200
5201
5202
5203
5204
5205
5206
5207
5208
5209
5210
5211
5212
5213
5214
5215
5216
5217
5218
5219
5220
5221
5222
5223
5224
5225
5226
5227
5228
5229
5230
5231
5232
5233
5234
5235
5236
5237
5238
5239
5240
5241
5242
5243
5244
5245
5246
5247
5248
5249
5250
5251
5252
5253
5254
5255
5256
5257
5258
5259
5260
5261
5262
5263
5264
5265
5266
5267
5268
5269
5270
5271
5272
5273
5274
5275
5276
5277
5278
5279
5280
5281
5282
5283
5284
5285
5286
5287
5288
5289
5290
5291
5292
5293
5294
5295
5296
5297
5298
5299
5300
5301
5302
5303
5304
5305
5306
5307
5308
5309
5310
5311
5312
5313
5314
5315
5316
5317
5318
5319
5320
5321
5322
5323
5324
5325
5326
5327
5328
5329
5330
5331
5332
5333
5334
5335
5336
5337
5338
5339
5340
5341
5342
5343
5344
5345
5346
5347
5348
5349
5350
5351
5352
5353
5354
5355
5356
5357
5358
5359
5360
5361
5362
5363
5364
5365
5366
5367
5368
5369
5370
5371
5372
5373
5374
5375
5376
5377
5378
5379
5380
5381
5382
5383
5384
5385
5386
5387
5388
5389
5390
5391
5392
5393
5394
5395
5396
5397
5398
5399
5400
5401
5402
5403
5404
5405
5406
5407
5408
5409
5410
5411
5412
5413
5414
5415
5416
5417
5418
5419
5420
5421
5422
5423
5424
5425
5426
5427
5428
5429
5430
5431
5432
5433
5434
5435
5436
5437
5438
5439
5440
5441
5442
5443
5444
5445
5446
5447
5448
5449
5450
5451
5452
5453
5454
5455
5456
5457
5458
5459
5460
5461
5462
5463
5464
5465
5466
5467
5468
5469
5470
5471
5472
5473
5474
5475
5476
5477
5478
5479
5480
5481
5482
5483
5484
5485
5486
5487
5488
5489
5490
5491
5492
5493
5494
5495
5496
5497
5498
5499
5500
5501
5502
5503
5504
5505
5506
5507
5508
5509
5510
5511
5512
5513
5514
5515
5516
5517
5518
5519
5520
5521
5522
5523
5524
5525
5526
5527
5528
5529
5530
5531
5532
5533
5534
5535
5536
5537
5538
5539
5540
5541
5542
5543
5544
5545
5546
5547
5548
5549
5550
5551
5552
5553
5554
5555
5556
5557
5558
5559
5560
5561
5562
5563
5564
5565
5566
5567
5568
5569
5570
5571
5572
5573
5574
5575
5576
5577
5578
5579
5580
5581
5582
5583
5584
5585
5586
5587
5588
5589
5590
5591
5592
5593
5594
5595
5596
5597
5598
5599
5600
5601
5602
5603
5604
5605
5606
5607
5608
5609
5610
5611
5612
5613
5614
5615
5616
5617
5618
5619
5620
5621
5622
5623
5624
5625
5626
5627
5628
5629
5630
5631
5632
5633
5634
5635
5636
5637
5638
5639
5640
5641
5642
5643
5644
5645
5646
5647
5648
5649
5650
5651
5652
5653
5654
5655
5656
5657
5658
5659
5660
5661
5662
5663
5664
5665
5666
5667
5668
5669
5670
5671
5672
5673
5674
5675
5676
5677
5678
5679
5680
5681
5682
5683
5684
5685
5686
5687
5688
5689
5690
5691
5692
5693
5694
5695
5696
5697
5698
5699
5700
5701
5702
5703
5704
5705
5706
5707
5708
5709
5710
5711
5712
5713
5714
5715
5716
5717
5718
5719
5720
5721
5722
5723
5724
5725
5726
5727
5728
5729
5730
5731
5732
5733
5734
5735
5736
5737
5738
5739
5740
5741
5742
5743
5744
5745
5746
5747
5748
5749
5750
5751
5752
5753
5754
5755
5756
5757
5758
5759
5760
5761
5762
5763
5764
5765
5766
5767
5768
5769
5770
5771
5772
5773
5774
5775
5776
5777
5778
5779
5780
5781
5782
5783
5784
5785
5786
5787
5788
5789
5790
5791
5792
5793
5794
5795
5796
5797
5798
5799
5800
5801
5802
5803
5804
5805
5806
5807
5808
5809
5810
5811
5812
5813
5814
5815
5816
5817
5818
5819
5820
5821
5822
5823
5824
5825
5826
5827
5828
5829
5830
5831
5832
5833
5834
5835
5836
5837
5838
5839
5840
5841
5842
5843
5844
5845
5846
5847
5848
5849
5850
5851
5852
5853
5854
5855
5856
5857
5858
5859
5860
5861
5862
5863
5864
5865
5866
5867
5868
5869
5870
5871
5872
5873
5874
5875
5876
5877
5878
5879
5880
5881
5882
5883
5884
5885
5886
5887
5888
5889
5890
5891
5892
5893
5894
5895
5896
5897
5898
5899
5900
5901
5902
5903
5904
5905
5906
5907
5908
5909
5910
5911
5912
5913
5914
5915
5916
5917
5918
5919
5920
5921
5922
5923
5924
5925
5926
5927
5928
5929
5930
5931
5932
5933
5934
5935
5936
5937
5938
5939
5940
5941
5942
5943
5944
5945
5946
5947
5948
5949
5950
5951
5952
5953
5954
5955
5956
5957
5958
5959
5960
5961
5962
5963
5964
5965
5966
5967
5968
5969
5970
5971
5972
5973
5974
5975
5976
5977
5978
5979
5980
5981
5982
5983
5984
5985
5986
5987
5988
5989
5990
5991
5992
5993
5994
5995
5996
5997
5998
5999
6000
6001
6002
6003
6004
6005
6006
6007
6008
6009
6010
6011
6012
6013
6014
6015
6016
6017
6018
6019
6020
6021
6022
6023
6024
6025
6026
6027
6028
6029
6030
6031
6032
6033
6034
6035
6036
6037
6038
6039
6040
6041
6042
6043
6044
6045
6046
6047
6048
6049
6050
6051
6052
6053
6054
6055
6056
6057
6058
6059
6060
6061
6062
6063
6064
6065
6066
6067
6068
6069
6070
6071
6072
6073
6074
6075
6076
6077
6078
6079
6080
6081
6082
6083
6084
6085
6086
6087
6088
6089
6090
6091
6092
6093
6094
6095
6096
6097
6098
6099
6100
6101
6102
6103
6104
6105
6106
6107
6108
6109
6110
6111
6112
6113
6114
6115
6116
6117
6118
6119
6120
6121
6122
6123
6124
6125
6126
6127
6128
6129
6130
6131
6132
6133
6134
6135
6136
6137
6138
6139
6140
6141
6142
6143
6144
6145
6146
6147
6148
6149
6150
6151
6152
6153
6154
6155
6156
6157
6158
6159
6160
6161
6162
6163
6164
6165
6166
6167
6168
6169
6170
6171
6172
6173
6174
6175
6176
6177
6178
6179
6180
6181
6182
6183
6184
6185
6186
6187
6188
6189
6190
6191
6192
6193
6194
6195
6196
6197
6198
6199
6200
6201
6202
6203
6204
6205
6206
6207
6208
6209
6210
6211
6212
6213
6214
6215
6216
6217
6218
6219
6220
6221
6222
6223
6224
6225
6226
6227
6228
6229
6230
6231
6232
6233
6234
6235
6236
6237
6238
6239
6240
6241
6242
6243
6244
6245
6246
6247
6248
6249
6250
6251
6252
6253
6254
6255
6256
6257
6258
6259
6260
6261
6262
6263
6264
6265
6266
6267
6268
6269
6270
6271
6272
6273
6274
6275
6276
6277
6278
6279
6280
6281
6282
6283
6284
6285
6286
6287
6288
6289
6290
6291
6292
6293
6294
6295
6296
6297
6298
6299
6300
6301
6302
6303
6304
6305
6306
6307
6308
6309
6310
6311
6312
6313
6314
6315
6316
6317
6318
6319
6320
6321
6322
6323
6324
6325
6326
6327
6328
6329
6330
6331
6332
6333
6334
6335
6336
6337
6338
6339
6340
6341
6342
6343
6344
6345
6346
6347
6348
6349
6350
6351
6352
6353
6354
6355
6356
6357
6358
6359
6360
6361
6362
6363
6364
6365
6366
6367
6368
6369
6370
6371
6372
6373
6374
6375
6376
6377
6378
6379
6380
6381
6382
6383
6384
6385
6386
6387
6388
6389
6390
6391
6392
6393
6394
6395
6396
6397
6398
6399
6400
6401
6402
6403
6404
6405
6406
6407
6408
6409
6410
6411
6412
6413
6414
6415
6416
6417
6418
6419
6420
6421
6422
6423
6424
6425
6426
6427
6428
6429
6430
6431
6432
6433
6434
6435
6436
6437
6438
6439
6440
6441
6442
6443
6444
6445
6446
6447
6448
6449
6450
6451
6452
6453
6454
6455
6456
6457
6458
6459
6460
6461
6462
6463
6464
6465
6466
6467
6468
6469
6470
6471
6472
6473
6474
6475
6476
6477
6478
6479
6480
6481
6482
6483
6484
6485
6486
6487
6488
6489
6490
6491
6492
6493
6494
6495
6496
6497
6498
6499
6500
6501
6502
6503
6504
6505
6506
6507
6508
6509
6510
6511
6512
6513
6514
6515
6516
6517
6518
6519
6520
6521
6522
6523
6524
6525
6526
6527
6528
6529
6530
6531
6532
6533
6534
6535
6536
6537
6538
6539
6540
6541
6542
6543
6544
6545
6546
6547
6548
6549
6550
6551
6552
6553
6554
6555
6556
6557
6558
6559
6560
6561
6562
6563
6564
6565
6566
6567
6568
6569
6570
6571
6572
6573
6574
6575
6576
6577
6578
6579
6580
6581
6582
6583
6584
6585
6586
6587
6588
6589
6590
6591
6592
6593
6594
6595
6596
6597
6598
6599
6600
6601
6602
6603
6604
6605
6606
6607
6608
6609
6610
6611
6612
6613
6614
6615
6616
6617
6618
6619
6620
6621
6622
6623
6624
6625
6626
6627
6628
6629
6630
6631
6632
6633
6634
6635
6636
6637
6638
6639
6640
6641
6642
6643
6644
6645
6646
6647
6648
6649
6650
6651
6652
6653
6654
6655
6656
6657
6658
6659
6660
6661
6662
6663
6664
6665
6666
6667
6668
6669
6670
6671
6672
6673
6674
6675
6676
6677
6678
6679
6680
6681
6682
6683
6684
6685
6686
6687
6688
6689
6690
6691
6692
6693
6694
6695
6696
6697
6698
6699
6700
6701
6702
6703
6704
6705
6706
6707
6708
6709
6710
6711
6712
6713
6714
6715
6716
6717
6718
6719
6720
6721
6722
6723
6724
6725
6726
6727
6728
6729
6730
6731
6732
6733
6734
6735
6736
6737
6738
6739
6740
6741
6742
6743
6744
6745
6746
6747
6748
6749
6750
6751
6752
6753
6754
6755
6756
6757
6758
6759
6760
6761
6762
6763
6764
6765
6766
6767
6768
6769
6770
6771
6772
6773
6774
6775
6776
6777
6778
6779
6780
6781
6782
6783
6784
6785
6786
6787
6788
6789
6790
6791
6792
6793
6794
6795
6796
6797
6798
6799
6800
6801
6802
6803
6804
6805
6806
6807
6808
6809
6810
6811
6812
6813
6814
6815
6816
6817
6818
6819
6820
6821
6822
6823
6824
6825
6826
6827
6828
6829
6830
6831
6832
6833
6834
6835
6836
6837
6838
6839
6840
6841
6842
6843
6844
6845
6846
6847
6848
6849
6850
6851
6852
6853
6854
6855
6856
6857
6858
6859
6860
6861
6862
6863
6864
6865
6866
6867
6868
6869
6870
6871
6872
6873
6874
6875
6876
6877
6878
6879
6880
6881
6882
6883
6884
6885
6886
6887
6888
6889
6890
6891
6892
6893
6894
6895
6896
6897
6898
6899
6900
6901
6902
6903
6904
6905
6906
6907
6908
6909
6910
6911
6912
6913
6914
6915
6916
6917
6918
6919
6920
6921
6922
6923
6924
6925
6926
6927
6928
6929
6930
6931
6932
6933
6934
6935
6936
6937
6938
6939
6940
6941
6942
6943
6944
6945
6946
6947
6948
6949
6950
6951
6952
6953
6954
6955
6956
6957
6958
6959
6960
6961
6962
6963
6964
6965
6966
6967
6968
6969
6970
6971
6972
6973
6974
6975
6976
6977
6978
6979
6980
6981
6982
6983
6984
6985
6986
6987
6988
6989
6990
6991
6992
6993
6994
6995
6996
6997
6998
6999
7000
7001
7002
7003
7004
7005
7006
7007
7008
7009
7010
7011
7012
7013
7014
7015
7016
7017
7018
7019
7020
7021
7022
7023
7024
7025
7026
7027
7028
7029
7030
7031
7032
7033
7034
7035
7036
7037
7038
7039
7040
7041
7042
7043
7044
7045
7046
7047
7048
7049
7050
7051
7052
7053
7054
7055
7056
7057
7058
7059
7060
7061
7062
7063
7064
7065
7066
7067
7068
7069
7070
7071
7072
7073
7074
7075
7076
7077
7078
7079
7080
7081
7082
7083
7084
7085
7086
7087
7088
7089
7090
7091
7092
7093
7094
7095
7096
7097
7098
7099
7100
7101
7102
7103
7104
7105
7106
7107
7108
7109
7110
7111
7112
7113
7114
7115
7116
7117
7118
7119
7120
7121
7122
7123
7124
7125
7126
7127
7128
7129
7130
7131
7132
7133
7134
7135
7136
7137
7138
7139
7140
7141
7142
7143
7144
7145
7146
7147
7148
7149
7150
7151
7152
7153
7154
7155
7156
7157
7158
7159
7160
7161
7162
7163
7164
7165
7166
7167
7168
7169
7170
7171
7172
7173
7174
7175
7176
7177
7178
7179
7180
7181
7182
7183
7184
7185
7186
7187
7188
7189
7190
7191
7192
7193
7194
7195
7196
7197
7198
7199
7200
7201
7202
7203
7204
7205
7206
7207
7208
7209
7210
7211
7212
7213
7214
7215
7216
7217
7218
7219
7220
7221
7222
7223
7224
7225
7226
7227
7228
7229
7230
7231
7232
7233
7234
7235
7236
7237
7238
7239
7240
7241
7242
7243
7244
7245
7246
7247
7248
7249
7250
7251
7252
7253
7254
7255
7256
7257
7258
7259
7260
7261
7262
7263
7264
7265
7266
7267
7268
7269
7270
7271
7272
7273
7274
7275
7276
7277
7278
7279
7280
7281
7282
7283
7284
7285
7286
7287
7288
7289
7290
7291
7292
7293
7294
7295
7296
7297
7298
7299
7300
7301
7302
7303
7304
7305
7306
7307
7308
7309
7310
7311
7312
7313
7314
7315
7316
7317
7318
7319
7320
7321
7322
7323
7324
7325
7326
7327
7328
7329
7330
7331
7332
7333
7334
7335
7336
7337
7338
7339
7340
7341
7342
7343
7344
7345
7346
7347
7348
7349
7350
7351
7352
7353
7354
7355
7356
7357
7358
7359
7360
7361
7362
7363
7364
7365
7366
7367
7368
7369
7370
7371
7372
7373
7374
7375
7376
7377
7378
7379
7380
7381
7382
7383
7384
7385
7386
7387
7388
7389
7390
7391
7392
7393
7394
7395
7396
7397
7398
7399
7400
7401
7402
7403
7404
7405
7406
7407
7408
7409
7410
7411
7412
7413
7414
7415
7416
7417
7418
7419
7420
7421
7422
7423
7424
7425
7426
7427
7428
7429
7430
7431
7432
7433
7434
7435
7436
7437
7438
7439
7440
7441
7442
7443
7444
7445
7446
7447
7448
7449
7450
7451
7452
7453
7454
7455
7456
7457
7458
7459
7460
7461
7462
7463
7464
7465
7466
7467
7468
7469
7470
7471
7472
7473
7474
7475
7476
7477
7478
7479
7480
7481
7482
7483
7484
7485
7486
7487
7488
7489
7490
7491
7492
7493
7494
7495
7496
7497
7498
7499
7500
7501
7502
7503
7504
7505
7506
7507
7508
7509
7510
7511
7512
7513
7514
7515
7516
7517
7518
7519
7520
7521
7522
7523
7524
7525
7526
7527
7528
7529
7530
7531
7532
7533
7534
7535
7536
7537
7538
7539
7540
7541
7542
7543
7544
7545
7546
7547
7548
7549
7550
7551
7552
7553
7554
7555
7556
7557
7558
7559
7560
7561
7562
7563
7564
7565
7566
7567
7568
7569
7570
7571
7572
7573
7574
7575
7576
7577
7578
7579
7580
7581
7582
7583
7584
7585
7586
7587
7588
7589
7590
7591
7592
7593
7594
7595
7596
7597
7598
7599
7600
7601
7602
7603
7604
7605
7606
7607
7608
7609
7610
7611
7612
7613
7614
7615
7616
7617
7618
7619
7620
7621
7622
7623
7624
7625
7626
7627
7628
7629
7630
7631
7632
7633
7634
7635
7636
7637
7638
7639
7640
7641
7642
7643
7644
7645
7646
7647
7648
7649
7650
7651
7652
7653
7654
7655
7656
7657
7658
7659
7660
7661
7662
7663
7664
7665
7666
7667
7668
7669
7670
7671
7672
7673
7674
7675
7676
7677
7678
7679
7680
7681
7682
7683
7684
7685
7686
7687
7688
7689
7690
7691
7692
7693
7694
7695
7696
7697
7698
7699
7700
7701
7702
7703
7704
7705
7706
7707
7708
7709
7710
7711
7712
7713
7714
7715
7716
7717
7718
7719
7720
7721
7722
7723
7724
7725
7726
7727
7728
7729
7730
7731
7732
7733
7734
7735
7736
7737
7738
7739
7740
7741
7742
7743
7744
7745
7746
7747
7748
7749
7750
7751
7752
7753
7754
7755
7756
7757
7758
7759
7760
7761
7762
7763
7764
7765
7766
7767
7768
7769
7770
7771
7772
7773
7774
7775
7776
7777
7778
7779
7780
7781
7782
7783
7784
7785
7786
7787
7788
7789
7790
7791
7792
7793
7794
7795
7796
7797
7798
7799
7800
7801
7802
7803
7804
7805
7806
7807
7808
7809
7810
7811
7812
7813
7814
7815
7816
7817
7818
7819
7820
7821
7822
7823
7824
7825
7826
7827
7828
7829
7830
7831
7832
7833
7834
7835
7836
7837
7838
7839
7840
7841
7842
7843
7844
7845
7846
7847
7848
7849
7850
7851
7852
7853
7854
7855
7856
7857
7858
7859
7860
7861
7862
7863
7864
7865
7866
7867
7868
7869
7870
7871
7872
7873
7874
7875
7876
7877
7878
7879
7880
7881
7882
7883
7884
7885
7886
7887
7888
7889
7890
7891
7892
7893
7894
7895
7896
7897
7898
7899
7900
7901
7902
7903
7904
7905
7906
7907
7908
7909
7910
7911
7912
7913
7914
7915
7916
7917
7918
7919
7920
7921
7922
7923
7924
7925
7926
7927
7928
7929
7930
7931
7932
7933
7934
7935
7936
7937
7938
7939
7940
7941
7942
7943
7944
7945
7946
7947
7948
7949
7950
7951
7952
7953
7954
7955
7956
7957
7958
7959
7960
7961
7962
7963
7964
7965
7966
7967
7968
7969
7970
7971
7972
7973
7974
7975
7976
7977
7978
7979
7980
7981
7982
7983
7984
7985
7986
7987
7988
7989
7990
7991
7992
7993
7994
7995
7996
7997
7998
7999
8000
8001
8002
8003
8004
8005
8006
8007
8008
8009
8010
8011
8012
8013
8014
8015
8016
8017
8018
8019
8020
8021
8022
8023
8024
8025
8026
8027
8028
8029
8030
8031
8032
8033
8034
8035
8036
8037
8038
8039
8040
8041
8042
8043
8044
8045
8046
8047
8048
8049
8050
8051
8052
8053
8054
8055
8056
8057
8058
8059
8060
8061
8062
8063
8064
8065
8066
8067
8068
8069
8070
8071
8072
8073
8074
8075
8076
8077
8078
8079
8080
8081
8082
8083
8084
8085
8086
8087
8088
8089
8090
8091
8092
8093
8094
8095
8096
8097
8098
8099
8100
8101
8102
8103
8104
8105
8106
8107
8108
8109
8110
8111
8112
8113
8114
8115
8116
8117
8118
8119
8120
8121
8122
8123
8124
8125
8126
8127
8128
8129
8130
8131
8132
8133
8134
8135
8136
8137
8138
8139
8140
8141
8142
8143
8144
8145
8146
8147
8148
8149
8150
8151
8152
8153
8154
8155
8156
8157
8158
8159
8160
8161
8162
8163
8164
8165
8166
8167
8168
8169
8170
8171
8172
8173
8174
8175
8176
8177
8178
8179
8180
8181
8182
8183
8184
8185
8186
8187
8188
8189
8190
8191
8192
8193
8194
8195
8196
8197
8198
8199
8200
8201
8202
8203
8204
8205
8206
8207
8208
8209
8210
8211
8212
8213
8214
8215
8216
8217
8218
8219
8220
8221
8222
8223
8224
8225
8226
8227
8228
8229
8230
8231
8232
8233
8234
8235
8236
8237
8238
8239
8240
8241
8242
8243
8244
8245
8246
8247
8248
8249
8250
8251
8252
8253
8254
8255
8256
8257
8258
8259
8260
8261
8262
8263
8264
8265
8266
8267
8268
8269
8270
8271
8272
8273
8274
8275
8276
8277
8278
8279
8280
8281
8282
8283
8284
8285
8286
8287
8288
8289
8290
8291
8292
8293
8294
8295
8296
8297
8298
8299
8300
8301
8302
8303
8304
8305
8306
8307
8308
8309
8310
8311
8312
8313
8314
8315
8316
8317
8318
8319
8320
8321
8322
8323
8324
8325
8326
8327
8328
8329
8330
8331
8332
8333
8334
8335
8336
8337
8338
8339
8340
8341
8342
8343
8344
8345
8346
8347
8348
8349
8350
8351
8352
8353
8354
8355
8356
8357
8358
8359
8360
8361
8362
8363
8364
8365
8366
8367
8368
8369
8370
8371
8372
8373
8374
8375
8376
8377
8378
8379
8380
8381
8382
8383
8384
8385
8386
8387
8388
8389
8390
8391
8392
8393
8394
8395
8396
8397
8398
8399
8400
8401
8402
8403
8404
8405
8406
8407
8408
8409
8410
8411
8412
8413
8414
8415
8416
8417
8418
8419
8420
8421
8422
8423
8424
8425
8426
8427
8428
8429
8430
8431
8432
8433
8434
8435
8436
8437
8438
8439
8440
8441
8442
8443
8444
8445
8446
8447
8448
8449
8450
8451
8452
8453
8454
8455
8456
8457
8458
8459
8460
8461
8462
8463
8464
8465
8466
8467
8468
8469
8470
8471
8472
8473
8474
8475
8476
8477
8478
8479
8480
8481
8482
8483
8484
8485
8486
8487
8488
8489
8490
8491
8492
8493
8494
8495
8496
8497
8498
8499
8500
8501
8502
8503
8504
8505
8506
8507
8508
8509
8510
8511
8512
8513
8514
8515
8516
8517
8518
8519
8520
8521
8522
8523
8524
8525
8526
8527
8528
8529
8530
8531
8532
8533
8534
8535
8536
8537
8538
8539
8540
8541
8542
8543
8544
8545
8546
8547
8548
8549
8550
8551
8552
8553
8554
8555
8556
8557
8558
8559
8560
8561
8562
8563
8564
8565
8566
8567
8568
8569
8570
8571
8572
8573
8574
8575
8576
8577
8578
8579
8580
8581
8582
8583
8584
8585
8586
8587
8588
8589
8590
8591
8592
8593
8594
8595
8596
8597
8598
8599
8600
8601
8602
8603
8604
8605
8606
8607
8608
8609
8610
8611
8612
8613
8614
8615
8616
8617
8618
8619
8620
8621
8622
8623
8624
8625
8626
8627
8628
8629
8630
8631
8632
8633
8634
8635
8636
8637
8638
8639
8640
8641
8642
8643
8644
8645
8646
8647
8648
8649
8650
8651
8652
8653
8654
8655
8656
8657
8658
8659
8660
8661
8662
8663
8664
8665
8666
8667
8668
8669
8670
8671
8672
8673
8674
8675
8676
8677
8678
8679
8680
8681
8682
8683
8684
8685
8686
8687
8688
8689
8690
8691
8692
8693
8694
8695
8696
8697
8698
8699
8700
8701
8702
8703
8704
8705
8706
8707
8708
8709
8710
8711
8712
8713
8714
8715
8716
8717
8718
8719
8720
8721
8722
8723
8724
8725
8726
8727
8728
8729
8730
8731
8732
8733
8734
8735
8736
8737
8738
8739
8740
8741
8742
8743
8744
8745
8746
8747
8748
8749
8750
8751
8752
8753
8754
8755
8756
8757
8758
8759
8760
8761
8762
8763
8764
8765
8766
8767
8768
8769
8770
8771
8772
8773
8774
8775
8776
8777
8778
8779
8780
8781
8782
8783
8784
8785
8786
8787
8788
8789
8790
8791
8792
8793
8794
8795
8796
8797
8798
8799
8800
8801
8802
8803
8804
8805
8806
8807
8808
8809
8810
8811
8812
8813
8814
8815
8816
8817
8818
8819
8820
8821
8822
8823
8824
8825
8826
8827
8828
8829
8830
8831
8832
8833
8834
8835
8836
8837
8838
8839
8840
8841
8842
8843
8844
8845
8846
8847
8848
8849
8850
8851
8852
8853
8854
8855
8856
8857
8858
8859
8860
8861
8862
8863
8864
8865
8866
8867
8868
8869
8870
8871
8872
8873
8874
8875
8876
8877
8878
8879
8880
8881
8882
// file      : doc/manual.cli
// license   : MIT; see accompanying LICENSE file

"\name=build2-build-system-manual"
"\subject=build system"
"\title=Build System"

// NOTES
//
// - Maximum <pre> line is 70 characters.
//

// @@ backlink variable ref (build system core variables reference?)
// @@ installation of dependencies

/*
@@ Redo synopsis in directives once enable p-code-box.css.

@@ info (where? in scopes? could show some? separate section?)
@@ other meta-ops: create (anything else?)

@@ all tree output needs extra space (review with mc) (also dir/ suffix)

@@ Need to mention ixx/txx files somewhere since used in bdep-new-generated
   projects.

@@ establish a chapter for each module
@@ module synopsis idea

@@ - style guide for quoting. What's naturally reversed (paths, options)
     should not be quoted?). Also indentation (two spaces).

@@ Copy/expand variable prepend/append/replace assignment note to Variables
   section. Add ref from the note.

@@ Synthesized dependencies (where did that obj{} target come form?)

*/

"
\h0#preface|Preface|

This document describes the \c{build2} build system. For the build system
driver command line interface refer to the \l{b(1)} man pages. For other tools
in the \c{build2} toolchain (package and project managers, etc) see the
\l{https://build2.org/doc.xhtml Documentation} index.

\h1#intro|Introduction|

The \c{build2} build system is a native, cross-platform build system with a
terse, mostly declarative description language, a conceptual model of build,
and a uniform interface with consistent behavior across platforms and
compilers.

Those familiar with \c{make} will see many similarities, though mostly
conceptual rather than syntactic. This is not by accident since \c{build2}
borrows the fundamental DAG-based build model from original \c{make} and many
of its conceptual extensions from GNU \c{make}. We believe, paraphrasing a
famous quote, that \i{those who do not understand \c{make} are condemned to
reinvent it, poorly.} So our goal with \c{build2} was to reinvent \c{make}
\i{well} while handling the demands and complexity of modern cross-platform
software development.

Like \c{make}, \c{build2} is an \i{\"honest\"} build system without magic or
black boxes. You can expect to understand what's going on underneath and be
able to customize most of its behavior to suit your needs. This is not to say
that it's not an \i{opinionated} build system and if you find yourself
\"fighting\" some of its fundamental design choices, it would probably be
wiser to look for alternatives.

We believe the importance and complexity of the problem warranted the design
of a new purpose-built language and will hopefully justify the time it takes
for you to master it. In the end we hope \c{build2} will make creating and
maintaining build infrastructure for your projects a pleasant task.

Also note that \c{build2} is not specific to C/C++ or even to compiled
languages; its build model is general enough to handle any DAG-based
operations. See the \l{#module-bash \c{bash}} module for a good example.

While the build system is part of a larger, well-integrated build toolchain
that includes the package and project dependency managers, it does not depend
on them and its standalone usage is the only subject of this manual.

We begin with a tutorial introduction that aims to show the essential elements
of the build system on real examples but without getting into too much
detail. Specifically, we want to quickly get to the point where we can build
useful executable and library projects.


\h#intro-hello|Hello, World|

Let's start with the customary \i{\"Hello, World\"} example: a single source
file from which we would like to build an executable:

\
$ tree hello/
hello/
└── hello.cxx

$ cat hello/hello.cxx

#include <iostream>

int main ()
{
  std::cout << \"Hello, World!\" << std::endl;
}
\

While this very basic program hardly resembles what most software projects
look like today, it is useful for introducing key build system concepts
without getting overwhelmed. In this spirit we will also use the \c{build2}
\i{simple project} structure, which, similarly, should only be used for
basic needs.

To turn our \c{hello/} directory into a simple project all we need to do
is add a \c{buildfile}:

\
$ tree hello/
hello/
├── hello.cxx
└── buildfile

$ cat hello/buildfile

using cxx

exe{hello}: cxx{hello.cxx}
\

Let's start from the bottom: the second line is a \i{dependency declaration}.
On the left hand side of \c{:} we have a \i{target}, the \c{hello} executable,
and on the right hand side \- a \i{prerequisite}, the \c{hello.cxx} source
file. Those \c{exe} and \c{cxx} in \c{exe{...\}} and \c{cxx{...\}} are called
\i{target types}. In fact, for clarity, target type names are always mentioned
with trailing \c{{\}}, for example, \"the \c{exe{\}} target type denotes an
executable\".

Notice that the dependency declaration does not specify \i{how} to build an
executable from a C++ source file \- this is the job of a \i{rule}. When the
build system needs to update a target, it tries to \i{match} a suitable rule
based on the types of the target and its prerequisites. The \c{build2} core
has a number of predefined fundamental rules with the rest coming from
\i{build system modules}. For example, the \c{cxx} module defines a number of
rules for compiling C++ source code as well as linking executables and
libraries.

It should now be easy to guess what the first line of our \c{buildfile} does:
it loads the \c{cxx} module which defines the rules necessary to build our
program (it also registers the \c{cxx{\}} target type).

Let's now try to build and run our program (\c{b} is the build system driver):

\
$ cd hello/  # Change to project root.

$ b
c++ cxx{hello}
ld exe{hello}

$ ls -1
buildfile
hello.cxx
hello
hello.d
hello.o
hello.o.d

$ ./hello
Hello, World!
\

Or, if we are on Windows and using Visual Studio:

\
> cd hello

> b
c++ cxx{hello}
ld exe{hello}

> dir /b
buildfile
hello.cxx
hello.exe
hello.exe.d
hello.exe.obj
hello.exe.obj.d

> .\hello.exe
Hello, World!
\

By default \c{build2} uses the same C++ compiler it was built with and without
passing any extra options, such as debug or optimization, target architecture,
etc. To change these defaults we use \i{configuration variables}. For example,
to specify a different C++ compiler we use \c{config.cxx}:

\
$ b config.cxx=clang++
\

\N|For Visual Studio, \c{build2} by default will use the latest available
version and build for the \c{x86_64} target (\c{x64} in the Microsoft's
terminology). You can, however, override these defaults by either running from
a suitable Visual Studio development command prompt or by specifying an
absolute path to \c{cl} that you wish to use. For example (notice the use
of inner quotes):

\
> b \"config.cxx='...\VC\Tools\MSVC\14.23.28105\bin\Hostx64\x86\cl'\"
\

See \l{#cc-msvc MSVC Compiler Toolchain} for details.|

Similarly, for additional compile options, such as debug information or
optimization level, there is \c{config.cxx.coptions}. For example:

\
$ b config.cxx=clang++ config.cxx.coptions=-g
\

\N|These and other configuration variables will be discussed in more detail
later. We will also learn how to make our configuration persistent so that we
don't have to repeat such long command lines on every build system invocation.

Similar to \c{config.cxx}, there is also \c{config.c} for specifying the C
compiler. Note, however, that if your project uses both C and C++, then you
normally only need to specify one of them \- \c{build2} will determine the
other automatically.|

Let's discuss a few points about the build output. Firstly, to reduce the
noise, the commands being executed are by default shown abbreviated and with
the same target type notation as we used in the \c{buildfile}. For example:

\
c++ cxx{hello}
ld exe{hello}
\

If, however, you would like to see the actual command lines, you can pass
\c{-v} (to see even more, there is the \c{-V} as well as \c{--verbose}
options; see \l{b(1)} for details). For example:

\
$ b -v
g++ -o hello.o -c hello.cxx
g++ -o hello hello.o
\

Most of the files produced by the build system should be self-explanatory: we
have the object file (\c{hello.o}, \c{hello.obj}) and executable (\c{hello},
\c{hello.exe}). For each of them we also have the corresponding \c{.d} files
which store the \i{auxiliary dependency information}, things like compile
options, header dependencies, etc.

To remove the build system output we use the \c{clean} \i{operation} (if no
operation is specified, the default is \c{update}):

\
$ b clean
rm exe{hello}
rm obje{hello}

$ ls -1
buildfile
hello.cxx
\

One of the main reasons behind the \i{target type} concept is the
platform/compiler-specified variances in file names as illustrated by the
above listings. In our \c{buildfile} we refer to the executable target as
\c{exe{hello\}}, not as \c{hello.exe} or \c{hello$EXT}. The actual file
extension, if any, will be determined based on the compiler's target platform
by the rule doing the linking. In this sense, target types are a
platform-independent replacement of file extensions (though they do have other
benefits, such as allowing non-file targets as well as being hierarchical).

Let's revisit the dependency declaration line from our \c{buildfile}:

\
exe{hello}: cxx{hello.cxx}
\

In light of target types replacing file extensions this looks tautological:
why do we need to specify both the \c{cxx{\}} target type \i{and} the \c{.cxx}
file extension? In fact, we don't have to if we specify the default file
extension for the \c{cxx{\}} target type. Here is our updated \c{buildfile} in
its entirety:

\
using cxx

cxx{*}: extension = cxx

exe{hello}: cxx{hello}
\

Let's unpack the new line. What we have here is a \i{target
type/pattern-specific variable}. It only applies to targets of the \c{cxx{\}}
type whose names match the \c{*} wildcard pattern. The \c{extension} variable
name is reserved by the \c{build2} core for specifying target type
extensions.

Let's see how all these pieces fit together. When the build system needs to
update \c{exe{hello\}}, it searches for a suitable rule. A rule from the
\c{cxx} module matches since it knows how to build a target of type \c{exe{\}}
from a prerequisite of type \c{cxx{\}}. When the matched rule is \i{applied},
it searches for a target for the \c{cxx{hello\}} prerequisite. During this
search, the \c{extension} variable is looked up and its value is used to end
up with the \c{hello.cxx} file.

Here is our new dependency declaration again:

\
exe{hello}: cxx{hello}
\

It has the canonical form: no extensions, only target types. Sometimes
explicit extension specification is still necessary, for example, if your
project uses multiple extensions for the same file type. But if unnecessary,
it should be omitted for brevity.

\N|If you prefer the \c{.cpp} file extension and your source file is called
\c{hello.cpp}, then the only line in our \c{buildfile} that needs changing is
the \c{extension} variable assignment:

\
cxx{*}: extension = cpp
\

|

Let's say our \c{hello} program got complicated enough to warrant moving some
functionality into a separate source/header module (or a real C++ module).
For example:

\
$ tree hello/
hello/
├── hello.cxx
├── utility.hxx
├── utility.cxx
└── buildfile
\

This is what our updated \c{buildfile} could look like:

\
using cxx

hxx{*}: extension = hxx
cxx{*}: extension = cxx

exe{hello}: cxx{hello} hxx{utility} cxx{utility}
\

Nothing really new here: we've specified the default extension for the
\c{hxx{\}} target type and listed the new header and source files as
prerequisites. If you have experience with other build systems, then
explicitly listing headers might seem strange to you. As will be discussed
later, in \c{build2} we have to explicitly list all the prerequisites of a
target that should end up in a distribution of our project.

\N|You don't have to list \i{all} headers that you include, only the ones
belonging to your project. Like all modern C/C++ build systems, \c{build2}
performs automatic header dependency extraction.|

In real projects with a substantial number of source files, repeating target
types and names will quickly become noisy. To tidy things up we can use
\i{name generation}. Here are a few examples of dependency declarations
equivalent to the above:

\
exe{hello}: cxx{hello utility} hxx{utility}
exe{hello}: cxx{hello} {hxx cxx}{utility}
\

The last form is probably the best choice if your project contains a large
number of header/source pairs. Here is a more realistic example:

\
exe{hello}: {    cxx}{hello}               \
            {hxx    }{forward types}       \
            {hxx cxx}{format print utility}
\

Manually listing a prerequisite every time we add a new source file to our
project is both tedious and error prone. Instead, we can automate our
dependency declarations with \i{wildcard name patterns}. For example:

\
exe{hello}: {hxx cxx}{*}
\

Based on the previous discussion of default extensions, you can probably guess
how this works: for each target type the value of the \c{extension} variable
is added to the pattern and files matching the result become prerequisites.
So, in our case, we will end up with files matching the \c{*.hxx} and
\c{*.cxx} wildcard patterns.

In more complex projects it is often convenient to organize source code into
subdirectories. To handle such projects we can use the recursive wildcard:

\
exe{hello}: {hxx cxx}{**}
\

\N|Using wildcards is somewhat controversial. Patterns definitely make
development more pleasant and less error prone: you don't need to update your
\c{buildfile} every time you add, remove, or rename a source file and you
won't forget to explicitly list headers, a mistake that is often only detected
when trying to build a distribution of a project. On the other hand, there is
the possibility of including stray source files into your build without
noticing. And, for more complex projects, name patterns can become fairly
complex (see \l{#name-patterns Name Patterns} for details). Note also that on
modern hardware the performance of wildcard searches hardly warrants a
consideration.

In our experience, when combined with modern version control systems like
\c{git(1)}, stray source files are rarely an issue and generally the benefits
of wildcards outweigh their drawbacks. But, in the end, whether to use them or
not is a personal choice and, as shown above, \c{build2} supports both
approaches.|

And that's about all there is to our \c{hello} example. To summarize, we've
seen that to build a simple project we need a single \c{buildfile} which
itself doesn't contain much more than a dependency declaration for what we
want to build. But we've also mentioned that simple projects are only really
meant for basics. So let's convert our \c{hello} example to the \i{standard
project} structure which is what we will be using for most of our real
development.

\N|Simple projects have so many restrictions and limitations that they are
hardly usable for anything but, well, \i{really} simple projects.

Specifically, such projects cannot be imported by other projects nor can they
use build system modules that require bootstrapping. Notably, this includes
the \c{dist} and \c{config} modules (the \c{test} and \c{install} modules are
loaded implicitly). And without the \c{config} module there is no support for
persistent configurations.

As a result, you should only use a simple project if you are happy to always
build in the source directory and with the default build configuration or
willing to specify the output directory and/or custom configuration on every
invocation. In other words, expect an experience similar to a plain
\c{Makefile}.

One notable example where simple projects are handy is a \i{glue
\c{buildfiles}} that \"pulls\" together several other projects, usually for
convenience of development. See \l{#intro-import Target Importation} for
details.|


\h#intro-proj-struct|Project Structure|

A \c{build2} \i{standard project} has the following overall layout:

\
hello/
├── build/
│   ├── bootstrap.build
│   └── root.build
├── ...
└── buildfile
\

Specifically, the project's root directory should contain the \c{build/}
subdirectory as well as the root \c{buildfile}. The \c{build/} subdirectory
contains project-wide build system information.

\N|The \l{bdep-new(1)} command is an easy way to create the standard layout
executable (\c{-t\ exe}) and library (\c{-t\ lib}) projects. To change the C++
file extensions to \c{.hpp/.cpp}, pass \c{-l c++,cpp}. For example:

\
$ bdep new --no-init -l c++,cpp -t exe hello
\

|

\N|It is also possible to use an alternative build file/directory naming
scheme where every instance of the word \i{build} is replaced with \i{build2},
for example:

\
hello/
├── build2/
│   ├── bootstrap.build2
│   └── root.build2
├── ...
└── build2file
\

Note that the naming must be consistent within a project with all the
filesystem entries either following \i{build} or \i{build2} scheme. In
other words, we cannot call the directory \c{build2/} while still using
\c{buildfile}.

The alternative naming scheme is primarily useful when adding \c{build2}
support to an existing project along with other build systems. In this case,
the fairly generic standard names might already be in use. For example, it is
customary to have \c{build/} in \c{.gitignore}. Plus more specific naming will
make it easier to identify files and directories as belonging to the
\c{build2} support. For new projects as well as for existing projects that are
switching exclusively to \c{build2} the standard naming scheme is recommended.

To create a project with the alternative naming using \l{bdep-new(1)} pass
the \c{alt-naming} project type sub-option. For example:

\
$ bdep new -t exe,alt-naming ...
\

|

To support lazy loading of subprojects (discussed later), reading of the
project's build information is split into two phases: bootstrapping and
loading. During bootstrapping the project's \c{build/bootstrap.build} file is
read. Then, when (and if) the project is loaded completely, its
\c{build/root.build} file is read followed by the \c{buildfile} (normally from
the project root but possibly from a subdirectory).

The \c{bootstrap.build} file is required. Let's see what it would look like
for a typical project using our \c{hello} as an example:

\
project = hello

using version
using config
using test
using install
using dist
\

The first non-comment line in \c{bootstrap.build} should be the assignment of
the project name to the \c{project} variable. After that, a typical
\c{bootstrap.build} file loads a number of build system modules. While most
modules can be loaded during the project load phase in \c{root.build}, certain
modules have to be loaded early, while bootstrapping (for example, because
they define new operations).

Let's examine briefly the modules loaded by our \c{bootstrap.build}: The
\l{#module-version \c{version}} module helps with managing our project
versioning. With this module we only maintain the version in a single place
(the project's \c{manifest} file) and it is automatically made available in
various convenient forms throughout our project (\c{buildfiles}, header files,
etc). The \c{version} module also automates versioning of snapshots between
releases.

The \c{manifest} file is what makes our build system project a \i{package}.
It contains all the metadata that a user of a package might need to know:
name, version, dependencies, etc., all in one place. However, even if you
don't plan to package your project, it is a good idea to create a basic
\c{manifest} if only to take advantage of the version management offered by
the \c{version} module. So let's go ahead and add it next to our root
\c{buildfile}:

\
$ tree hello/
hello/
├── build/
│   └── ...
├── ...
├── buildfile
└── manifest

$ cat hello/manifest
: 1
name: hello
version: 0.1.0
summary: hello C++ executable
\

The \c{config} module provides support for persistent configurations. While
build configuration is a large topic that we will be discussing in more detail
later, in a nutshell \c{build2} support for configuration is an integral part
of the build system with the same mechanisms available to the build system
core, modules, and your projects. However, without \c{config}, the
configuration information is \i{transient}. That is, whatever configuration
information was automatically discovered or that you have supplied on the
command line is discarded after each build system invocation. With the
\c{config} module, however, we can \i{configure} a project to make the
configuration \i{persistent}. We will see an example of this shortly.

Next up are the \c{test}, \c{install}, and \c{dist} modules. As their names
suggest, they provide support for testing, installation and preparation of
distributions. Specifically, the \c{test} module defines the \c{test}
operation, the \c{install} module defines the \c{install} and \c{uninstall}
operations, and the \c{dist} module defines the \c{dist}
(meta-)operation. Again, we will try them out in a moment.

Moving on, the \c{root.build} file is optional though most projects will have
it. This is the place where we define project's configuration variables
(subject of \l{#proj-config Project Configuration}), establish project-wide
settings, as well as load build system modules that provide support for the
languages/tools that we use. Here is what it could look like for our \c{hello}
example:

\
cxx.std = latest

using cxx

hxx{*}: extension = hxx
cxx{*}: extension = cxx
\

As you can see, we've moved the loading of the \c{cxx} modules and setting of
the default file extensions from the root \c{buildfile} in our simple project
to \c{root.build} when using the standard layout. We've also set the
\c{cxx.std} variable to tell the \c{cxx} module to select the latest C++
standard available in any particular C++ compiler this project might be built
with.

\N|Selecting the C++ standard for our project is a messy issue. If we don't
specify the standard explicitly with \c{cxx.std}, then the default standard in
each compiler will be used, which, currently, can range from C++98 to
C++14. So unless you carefully write your code to work with any standard, this
is probably not a good idea.

Fixing the standard (for example, to \c{c++11}, \c{c++14}, etc) should work
theoretically. In practice, however, compilers add support for new standards
incrementally and many versions, while perfectly usable, are not
feature-complete. As a result, a better practical strategy is to specify the
set of minimum supported compiler versions rather than the C++ standard.

There is also the issue of using libraries that require a newer standard in
old code. For example, headers from a library that relies on C++14 features
will not compile when included in a project that is built as C++11.  And, even
if the headers compile (that is, C++14 features are only used in the
implementation), strictly speaking, there is no guarantee that codebases
compiled with different C++ standards are ABI compatible (in fact, some
changes to the C++ language leave the implementations no choice but to break
the ABI).

As result, our recommendation is to set the standard to \c{latest} and specify
the minimum supported compilers and versions in your project's documentation
(see package manifest \l{bpkg#manifest-package-requires \c{requires}} value
for one possible place). Practically, this should allow you to include and
link any library, regardless of the C++ standard that it uses.|

Let's now take a look at the root \c{buildfile}:

\
./: {*/ -build/}
\

In plain English, this \c{buildfile} declares that building this directory
(and, since it's the root of our project, building this entire project) means
building all its subdirectories excluding \c{build/}. Let's now try to
understand how this is actually achieved.

We already know this is a dependency declaration, \c{./} is the target, and
what's after \c{:} are its prerequisites, which seem to be generated with some
kind of a name pattern (the wildcard character in \c{*/} should be the
giveaway). What's unusual about this declaration, however, is the lack of any
target types plus that strange-looking \c{./}.

Let's start with the missing target types. In fact, the above \c{buildfile}
can be rewritten as:

\
dir{.}: dir{* -build}
\

So the trailing slash (always forward, even on Windows) is a special shorthand
notation for \c{dir{\}}. As we will see shortly, it fits naturally with other
uses of directories in \c{buildfiles} (for example, in scopes).

The \c{dir{\}} target type is an \i{alias} (and, in fact, is derived from more
general \c{alias{\}}). Building it means building all its prerequisites.

\N|If you are familiar with \c{make}, then you can probably see the similarity
with the ubiquitous \c{all} pseudo-target. In \c{build2} we instead use
directory names as more natural aliases for the \"build everything in this
directory\" semantics.

Note also that \c{dir{\}} is purely an alias and doesn't have anything to do
with the filesystem. In particular, it does not create any directories. If you
do want explicit directory creation (which should be rarely needed), use the
\c{fsdir{\}} target type instead.|

The \c{./} target is a special \i{default target}. If we run the build system
without specifying the target explicitly, then this target is built by
default. Every \c{buildfile} has the \c{./} target. If we don't declare it
explicitly, then its declaration is implied with the first target in the
\c{buildfile} as its prerequisite. Recall our \c{buildfile} from the simple
\c{hello} project:

\
exe{hello}: cxx{hello}
\

It is equivalent to:

\
./: exe{hello}
exe{hello}: cxx{hello}
\

If, however, we had several targets in the same directory that we wanted built
by default, then we would need to explicitly list them as prerequisites of the
default target. For example:

\
./: exe{hello}
exe{hello}: cxx{hello}

./: exe{goodby}
exe{goodby}: cxx{goodby}
\

While straightforward, this is somewhat inelegant in its repetitiveness. To
tidy things up we can use \i{dependency declaration chains} that allow us to
chain together several target-prerequisite declarations in a single line.
For example:

\
./: exe{hello}: cxx{hello}

./: exe{goodby}: cxx{goodby}
\

With dependency chains a prerequisite of the preceding target becomes a
target itself for the following prerequisites.

Let's get back to our root \c{buildfile}:

\
./: {*/ -build/}
\

The last unexplained bit is the \c{{*/\ -build/\}} name pattern. All it does
is exclude \c{build/} from the subdirectories to build. See \l{#name-patterns
Name Patterns} for details.

Let's take a look at a slightly more realistic root \c{buildfile}:

\
./: {*/ -build/} doc{README.md LICENSE} manifest
\

Here we have the customary \c{README.md} and \c{LICENSE} files as well as the
package \c{manifest}. Listing them as prerequisites achieves two things: they
will be installed if/when our project is installed and, as mentioned earlier,
they will be included into the project distribution.

The \c{README.md} and \c{LICENSE} files use the \c{doc{\}} target type. We
could have used the generic \c{file{\}} but using the more precise \c{doc{\}}
makes sure that they are installed into the appropriate documentation
directory. The \c{manifest} file doesn't need an explicit target type since it
has a fixed name (\c{manifest{manifest\}} is valid but redundant).

Standard project infrastructure in place, where should we put our source code?
While we could have everything in the root directory of our project, just like
we did with the simple layout, it is recommended to instead place the source
code into a subdirectory named the same as the project. For example:

\
hello/
├── build/
│   └── ...
├── hello/
│   ├── hello.cxx
│   └── buildfile
├── buildfile
├── manifest
└── README.md
\

\N|There are several reasons for this layout: It implements the canonical
inclusion scheme where each header is prefixed with its project name. It also
has a predictable name where users can expect to find our project's source
code. Finally, this layout prevents clutter in the project's root directory
which usually contains various other files. See \l{intro#proj-struct Canonical
Project Structure} for details.

Note, however, that this layout is not mandatory and \c{build2} is flexible
enough to support various arrangements used in today's C and C++ projects.
Furthermore, the \l{bdep-new(1)} command provides a number of customization
options and chances are you will be able to create your preferred layout
automatically. See \l{bdep-new.xhtml#src-layout SOURCE LAYOUT} for more
information and examples.

Note also that while we can name our header and source files however we like
(but, again, see \l{intro#proj-struct Canonical Project Structure} for some
sensible guidelines), C++ module interface files need to embed a sufficient
amount of the module name suffix in their names to unambiguously resolve all
the modules within a project. See \l{#cxx-modules-build Building Modules} for
details.|

The source subdirectory \c{buildfile} is identical to that of the simple project
minus the parts moved to \c{root.build}:

\
exe{hello}: {hxx cxx}{**}
\

Let's now build our project and see where the build system output ends up
in this new layout:

\
$ cd hello/  # Change to project root.
$ b
c++ hello/cxx{hello}
ld hello/exe{hello}

$ tree ./
./
├── build/
│   └── ...
├── hello/
│   ├── hello.cxx
│   ├── hello
│   ├── hello.d
│   ├── hello.o
│   ├── hello.o.d
│   └── buildfile
├── buildfile
└── manifest

$ hello/hello
Hello, World!
\

If we don't specify a target to build (as in the example above), then \c{build2}
will build the current directory or, more precisely, the default target in the
\c{buildfile} in the current directory. We can also build a directory other
than the current, for example:

\
$ b hello/
\

\N|Note that the trailing slash is required. In fact, \c{hello/} in the above
command line is a target and is equivalent to \c{dir{hello\}}, just like in
the \c{buildfiles}.|

Or we can build a specific target:

\
$ b hello/exe{hello}
\

Naturally, nothing prevents us from building multiple targets or even projects
in the same build system invocation. For example, if we had the \c{libhello}
project next to our \c{hello/}, then we could build both at once:

\
$ ls -1
hello/
libhello/

$ b hello/ libhello/
\

Speaking of libraries, let's see what the standard project structure looks
like for one, using \c{libhello} created by \l{bdep-new(1)} as an example:

\
$ bdep new --no-init -l c++ -t lib libhello

$ tree libhello/
libhello/
├── build/
│   ├── bootstrap.build
│   ├── root.build
│   └── export.build
├── libhello/
│   ├── hello.hxx
│   ├── hello.cxx
│   ├── export.hxx
│   ├── version.hxx.in
│   └── buildfile
├── tests/
│   └──  ...
├── buildfile
├── manifest
└── README.md
\

The overall layout (\c{build/}, \c{libhello/} source directory) as well as the
contents of the root files (\c{bootstrap.build}, \c{root.build}, root
\c{buildfile}) are exactly the same. There is, however, the new file
\c{export.build} in \c{build/}, the new subdirectory \c{tests/}, and the
contents of the project's source subdirectory \c{libhello/} look quite a bit
different. We will examine all of these differences in the coming sections, as
we learn more about the build system.

\N|Again, this layout is not mandatory and \l{bdep-new(1)} can create a number
of alternative library structures. For example, if you prefer the
\c{include/src} split, try:

\
$ bdep new --no-init -l c++ -t lib,split libhello
\

See \l{bdep-new.xhtml#src-layout SOURCE LAYOUT} for more examples.|


\N|The standard project structure is not type (executable, library, etc) or
even language specific. In fact, the same project can contain multiple
executables and/or libraries (for example, both \c{hello} and \c{libhello}).
However, if you plan to package your projects, it is a good idea to keep them
as separate build system projects (they can still reside in the same version
control repository, though).

Speaking of projects, this term is unfortunately overloaded to mean two
different things at different levels of software organization. At the bottom
we have \i{build system projects} which, if packaged, become \i{packages}. And
at the top, related packages are often grouped into what is also commonly
referred to as \i{projects}. At this point both usages are probably too well
established to look for alternatives.|

And this completes the conversion of our simple \c{hello} project to the
standard structure. Earlier, when examining \c{bootstrap.build}, we mentioned
that modules loaded in this file usually provide additional operations. So we
still need to discuss what exactly the term \i{build system operation} means
and see how to use operations that are provided by the modules we have loaded.
But before we do that, let's see how we can build our projects \i{out of
source} tree and learn about another cornerstone \c{build2} concept:
\i{scopes}.


\h#intro-dirs-scopes|Output Directories and Scopes|

Two common requirements placed on modern build systems are the ability to
build projects out of the source directory tree (referred to as just \i{out of
source} vs \i{in source}) as well as isolation of \c{buildfiles} from each
other when it comes to target and variable names. In \c{build2} these
mechanisms are closely-related, integral parts of the build system.

\N|This tight integration has advantages, like being always available and
working well with other build system mechanisms, as well as disadvantages,
like the inability to implement a completely different out of source
arrangement and/or isolation model. In the end, if you find yourself
\"fighting\" this aspect of \c{build2}, it will likely be easier to use a
different build system than subvert it.|

Let's start with an example of an out of source build for our \c{hello}
project. To recap, this is what we have:

\
$ ls -1
hello/

$ tree hello/
hello/
├── build/
│   └── ...
├── hello/
│   └── ...
├── buildfile
└── manifest
\

To start, let's build it in the \c{hello-out/} directory next to the project:

\
$ b hello/@hello-out/
mkdir fsdir{hello-out/}
mkdir hello-out/fsdir{hello/}
c++ hello/hello/cxx{hello}@hello-out/hello/
ld hello-out/hello/exe{hello}

$ ls -1
hello/
hello-out/

$ tree hello-out/
hello-out/
└── hello/
    ├── hello
    ├── hello.d
    ├── hello.o
    └── hello.o.d
\

This definitely requires some explaining. Let's start from the bottom, with
the \c{hello-out/} layout. It is \i{parallel} to the source directory. This
mirrored side-by-side listing (of the relevant parts) should illustrate this
clearly:

\
hello/             ~~>  hello-out/
└── hello/         ~~>  └── hello/
    └── hello.cxx  ~~>      └── hello.o
\

In fact, if we copy the contents of \c{hello-out/} over to \c{hello/}, we will
end up with exactly the same result as in the in source build. And this is not
accidental: an in source build is just a special case of an out of source
build where the \i{out} directory is the same as \i{src}.

\N|In \c{build2} this parallel structure of the out and src directories is a
cornerstone design decision and is non-negotiable, so to speak. In particular,
out cannot be inside src. And while we can stash the build system output
(object files, executables, etc) into (potentially different) subdirectories,
this is not recommended. As will be shown later, \c{build2} offers better
mechanisms to achieve the same benefits (like reduced clutter, ability to run
executables) but without the drawbacks (like name clashes).|

Let's now examine how we invoked the build system to achieve this out of
source build. Specifically, if we were building in source, our command line
would have been:

\
$ b hello/
\

but for the out of source build, we have:

\
$ b hello/@hello-out/
\

In fact, that strange-looking construct, \c{hello/@hello-out/} is just a more
elaborate target specification that explicitly spells out the target's src and
out directories. Let's add an explicit target type to make it clearer:

\
$ b hello/@hello-out/dir{.}
\

What we have on the right of \c{@} is the target in the out directory and on
the left \- its src directory. In plain English, this command line says
\"build me the default target from \c{hello/} in the \c{hello-out/}
directory\".

As an example, if instead we wanted to build only the \c{hello} executable out
of source, then the invocation would have looked like this:

\
$ b hello/hello/@hello-out/hello/exe{hello}
\

We could have also specified out for an in source build, but that's redundant:

\
$ b hello/@hello/
\

There is another example of this elaborate target specification in the build
diagnostics:

\
c++ hello/hello/cxx{hello}@hello-out/hello/
\

Notice, however, that now the target (\c{cxx{hello\}}) is on the left of
\c{@}, that is, in the src directory. It does, however, make sense if you
think about it: our \c{hello.cxx} is a \i{source file}, it is not built and it
resides in the project's source directory. This is in contrast, for example,
to the \c{exe{hello\}} target which is the output of the build system and goes
to the out directory. So in \c{build2} targets can be either in src or in out
(there can also be \i{out of any project} targets, for example, installed
files).

The elaborate target specification can also be used in \c{buildfiles}. We
haven't encountered any so far because targets mentioned without explicit
src/out default to out and, naturally, most of the targets we mention in
\c{buildfiles} are things we want built. One situation where you may encounter
an src target mentioned explicitly is when specifying its installability
(discussed in the next section). For example, if our project includes the
customary \c{INSTALL} file, it probably doesn't make sense to install it.
However, since it is a source file, we have to use the elaborate target
specification when disabling its installation:

\
doc{INSTALL}@./: install = false
\

Note also that only targets and not prerequisites have this notion of src/out
directories. In a sense, prerequisites are relative to the target they are
prerequisites of and are resolved to targets in a manner that is specific to
their target types. For \c{file{\}}-based prerequisites the corresponding
target in out is first looked up and, if found, used. Otherwise, an existing
file in src is searched for and, if found, the corresponding target (now in
src) is used. In particular, this semantics gives preference to generated code
over static.

\N|More precisely, a prerequisite is relative to the scope (discussed below)
in which the dependency is declared and not to the target that it is a
prerequisite of. However, in most practical cases, this means the same thing.|

And this pretty much covers out of source builds. Let's summarize the key
points we have established so far: Every build has two parallel directory
trees, src and out, with the in source build being just a special case where
they are the same. Targets in a project can be either in the src or out
directory though most of the time targets we mention in our \c{buildfiles}
will be in out, which is the default. Prerequisites are relative to targets
they are prerequisites of and \c{file{\}}-based prerequisites are first
searched for as declared targets in out and then as existing files in src.

Note also that we can have as many out of source builds as we want and we can
place them anywhere we want (but not inside src), say, on a RAM-backed
disk/filesystem. As an example, let's build our \c{hello} project with two
different compilers:

\
$ b hello/@hello-gcc/    config.cxx=g++
$ b hello/@hello-clang/  config.cxx=clang++
\

In the next section we will see how to permanently configure our out of
source builds so that we don't have to keep repeating these long command
lines.

\N|While technically you can have both in source and out of source builds
at the same time, this is not recommended. While it may work for basic
projects, as soon as you start using generated source code (which is fairly
common in \c{build2}), it becomes difficult to predict where the compiler will
pick generated headers. There is support for remapping mis-picked headers but
this may not always work with older C/C++ compilers. Plus, as we will see in
the next section, \c{build2} supports \i{forwarded configurations} which
provide most of the benefits of an in source build but without the drawbacks.|

Let's now turn to \c{buildfile} isolation. It is a common, well-established
practice to organize complex software projects in directory hierarchies. One
of the benefits of this organization is isolation: we can use the same, short
file names in different subdirectories. In \c{build2} the project's directory
tree is used as a basis for its \i{scope} hierarchy. In a sense, scopes are
like C++ namespaces that automatically track the project's filesystem
structure and use directories as their names. The following listing
illustrates the parallel directory and scope hierarchies for our \c{hello}
project. \N{The \c{build/} subdirectory is special and does not have a
corresponding scope.}

\
hello/                   hello/
│                        {
└── hello/                 hello/
    │                      {
    └── ...                  ...
                           }
                         }
\

Every \c{buildfile} is loaded in its corresponding scope, variables set in a
\c{buildfile} are set in this scope and relative targets mentioned in a
\c{buildfile} are relative to this scope's directory. Let's \"load\" the
\c{buildfile} contents from our \c{hello} project to the above listing:

\
hello/                   hello/
│                        {
├── buildfile              ./: {*/ -build/}
│
└── hello/                 hello/
    │                      {
    └── buildfile            exe{hello}: {hxx cxx}{**}
                           }
                         }
\

In fact, to be absolutely precise, we should also add the contents of
\c{bootstrap.build} and \c{root.build} to the project's root scope (module
loading is omitted for brevity):

\
hello/                   hello/
│                        {
├── build/
│   ├── bootstrap.build    project = hello
│   │
│   └── root.build         cxx.std = latest
│                          hxx{*}: extension = hxx
│                          cxx{*}: extension = cxx
│
├── buildfile              ./: {*/ -build/}
│
└── hello/                 hello/
    │                      {
    └── buildfile            exe{hello}: {hxx cxx}{**}
                           }
                         }
\

The above scope structure is very similar to what you will see (besides a lot
of other things) if you build with \c{--dump\ match}. With this option the
build system driver dumps the build state after matching rules to targets (see
\l{#intro-diag-debug Diagnostics and Debugging} for more information). Here is
an abbreviated output of building our \c{hello} with \c{--dump} (assuming an in
source build in \c{/tmp/hello}):

\
$ b --dump match

/
{
  [target_triplet] build.host = x86_64-linux-gnu
  [string] build.host.class = linux
  [string] build.host.cpu = x86_64
  [string] build.host.system = linux-gnu

  /tmp/hello/
  {

    [dir_path] src_root = /tmp/hello/
    [dir_path] out_root = /tmp/hello/

    [dir_path] src_base = /tmp/hello/
    [dir_path] out_base = /tmp/hello/

    [project_name] project = hello
    [string] project.summary = hello executable
    [string] project.url = https://example.org/hello

    [string] version = 1.2.3
    [uint64] version.major = 1
    [uint64] version.minor = 2
    [uint64] version.patch = 3

    [string] cxx.std = latest

    [string] cxx.id = gcc
    [string] cxx.version = 8.1.0
    [uint64] cxx.version.major = 8
    [uint64] cxx.version.minor = 1
    [uint64] cxx.version.patch = 0

    [target_triplet] cxx.target = x86_64-w64-mingw32
    [string] cxx.target.class = windows
    [string] cxx.target.cpu = x86_64
    [string] cxx.target.system = mingw32

    hxx{*}: [string] extension = hxx
    cxx{*}: [string] extension = cxx

    hello/
    {
      [dir_path] src_base = /tmp/hello/hello/
      [dir_path] out_base = /tmp/hello/hello/

      dir{./}: exe{hello}
      exe{hello.}: cxx{hello.cxx}
    }

    dir{./}: dir{hello/} manifest{manifest}
  }
}
\

This is probably quite a bit more information than what you've expected to see
so let's explain a couple of things. Firstly, it appears there is another
scope outer to our project's root. In fact, \c{build2} extends scoping outside
of projects with the root of the filesystem (denoted by the special \c{/})
being the \i{global scope}. This extension becomes useful when we try to build
multiple unrelated projects or import one project into another. In this model
all projects are part of a single scope hierarchy with the global scope at its
root.

The global scope is read-only and contains a number of pre-defined
\i{build-wide} variables such as the build system version, host platform
(shown in the above listing), etc.

Next, inside the global scope, we see our project's root scope
(\c{/tmp/hello/}). Besides the variables that we have set ourselves (like
\c{project}), it also contains a number of variables set by the build system
core (for example, \c{out_base}, \c{src_root}, etc) as well by build system
modules (for example, \c{project.*} and \c{version.*} variables set by the
\c{version} module and \c{cxx.*} variables set by the \c{cxx} module).

The scope for our project's source directory (\c{hello/}) should look
familiar. We again have a few special variables (\c{out_base}, \c{src_base}).
Notice also that the name patterns in prerequisites have been expanded to
the actual files.

As you can probably guess from their names, the \c{src_*} and \c{out_*}
variables track the association between scopes and src/out directories. They
are maintained automatically by the build system core with the
\c{src/out_base} pair set on each scope within the project and an additional
\c{src/out_root} pair set on the project's root scope so that we can get the
project's root directories from anywhere in the project. Note that directory
paths in these variables are always absolute and normalized.

In the above example the corresponding src/out variable pairs have the same
values because we were building in source. As an example, this is what the
association will look like for an out of source build:

\
hello/  ~~>      hello-out/                   <~~  hello-out/
│                {                                 │
│                  src_root = .../hello/           │
│                  out_root = .../hello-out/       │
│                                                  │
│                  src_base = .../hello/           │
│                  out_base = .../hello-out/       │
│                                                  │
└── hello/  ~~>    hello/                     <~~  └── hello/
                   {
                     src_base = .../hello/hello/
                     out_base = .../hello-out/hello/
                   }
                 }
\

Now that we have some scopes and variables to play with, it's a good time to
introduce variable expansion. To get the value stored in a variable we use
\c{$} followed by the variable's name. The variable is first looked up in the
current scope (that is, the scope in which the expansion was encountered) and,
if not found, in the outer scopes all the way to the global scope.

\N|To be precise, this is for the default \i{variable visibility}. Variables,
however, can have more limited visibilities, such as \i{project}, \i{scope},
\i{target}, or \i{prerequisite}.|

To illustrate the lookup semantics, let's add the following line to each
\c{buildfile} in our \c{hello} project:

\
$ cd hello/  # Change to project root.

$ cat buildfile
...
info \"src_base: $src_base\"

$ cat hello/buildfile
...
info \"src_base: $src_base\"
\

And then build it:

\
$ b
buildfile:3:1: info: src_base: /tmp/hello/
hello/buildfile:8:1: info: src_base: /tmp/hello/hello/
\

In this case \c{src_base} is defined in each of the two scopes and we get
their respective values. If, however, we change the above line to print
\c{src_root} instead of \c{src_base}, we will get the same value from the
root scope:

\
buildfile:3:1: info: src_root: /tmp/hello/
hello/buildfile:8:1: info: src_root: /tmp/hello/
\

\N|In this section we've only scratched the surface when it comes to
variables. In particular, variables and variable values in \c{build2} are
optionally typed (those \c{[string]}, \c{[uint64]} we've seen in the build
state dump). And in certain contexts the lookup semantics actually starts from
the target, not from the scope (target-specific variables; there are also
prerequisite-specific). These and other variable-related topics will be
covered in subsequent sections.|

One typical place to find \c{src/out_root} expansions is in the include search
path options. For example, the source directory \c{buildfile} generated by
\l{bdep-new(1)} for an executable project actually looks like this
(\c{poptions} stands for \i{preprocessor options}):

\
exe{hello}: {hxx cxx}{**}

cxx.poptions =+ \"-I$out_root\" \"-I$src_root\"
\

\N|The strange-looking \c{=+} line is a \i{prepend} variable assignment. It
adds the value on the right hand side to the beginning of the existing
value. So, in the above example, the two header search paths will be added
before any of the existing preprocessor options (and thus will be considered
first).

There are also the \i{append} assignment, \c{+=}, which adds the value on the
right hand side to the end of the existing value, as well as, of course, the
normal or \i{replace} assignment, \c{=}, which replaces the existing value
with the right hand side. One way to remember where the existing and new
values end up in the \c{=+} and \c{+=} results is to imagine the new value
taking the position of \c{=} and the existing value \- of \c{+}.|

The above \c{buildfile} allows us to include our headers using the project's
name as a prefix, inline with the \l{intro#proj-struct Canonical Project
Structure} guidelines. For example, if we added the \c{utility.hxx} header to
our \c{hello} project, we would include it like this:

\
#include <iostream>

#include <hello/utility.hxx>

int main ()
{
...
}
\

\N|Besides \c{poptions}, there are also \c{coptions} (compile options),
\c{loptions} (link options), \c{aoptions} (archive options) and \c{libs}
(extra libraries to link). If you are familiar with \c{make}, these are
roughly equivalent to \c{CPPFLAGS}, \c{CFLAGS}/\c{CXXFLAGS}, \c{LDFLAGS},
\c{ARFLAGS}, and \c{LIBS}/\c{LDLIBS}, respectively. Here they are again in the
tabular form:

\
*.poptions   preprocess        CPPFLAGS
*.coptions   compile           CFLAGS/CXXFLAGS
*.loptions   link              LDFLAGS
*.aoptions   archive           ARFLAGS
*.libs       extra libraries   LIBS/LDLIBS
\

More specifically, there are three sets of these variables: \c{cc.*} (stands
for \i{C-common}) which applies to all C-like languages as well as \c{c.*} and
\c{cxx.*} which only apply during the C and C++ compilation, respectively. We
can use these variables in our \c{buildfiles} to adjust the compiler/linker
behavior. For example:

\
if ($cc.class == 'gcc')
{
  cc.coptions  += -fno-strict-aliasing  # C and C++
  cxx.coptions += -fno-exceptions       # only C++
}

if ($c.target.class != 'windows')
  c.libs += -lpthread  # only C
\

Additionally, as we will see in \l{#intro-operations-config Configuring},
there are also the \c{config.cc.*}, \c{config.c.*}, and \c{config.cxx.*} sets
which are used by the users of our projects to provide external configuration.
The initial values of the \c{cc.*}, \c{c.*}, and \c{cxx.*} variables are taken
from the corresponding \c{config.*.*} values.

And, as we will learn in \l{#intro-lib Library Exportation}, there are also
the \c{cc.export.*}, \c{c.export.*}, and \c{cxx.export.*} sets that are used
to specify options that should be exported to the users of our library.

If we adjust the \c{cc.*}, \c{c.*}, and \c{cxx.*} variables at the scope
level, as in the above fragment, then the changes will apply when building
every target in this scope (as well as in the nested scopes, if any). Usually
this is what we want but sometimes we may need to pass additional options only
when compiling certain source files or linking certain libraries or
executables. For that we use the target-specific variable assignment. For
example:

\
exe{hello}: {hxx cxx}{**}

obj{utility}: cxx.poptions += -DNDEBUG
exe{hello}: cxx.loptions += -static
\

Note that we set these variables on targets which they affect. In particular,
those with a background in other build systems may, for example, erroneously
expect that setting \c{poptions} on a library target will affect compilation
of its prerequisites. For example, the following does not work:

\
exe{hello}: cxx.poptions += -DNDEBUG
\

The recommended way to achieve this behavior in \c{build2} is to organize your
targets into subdirectories, in which case we can just set the variables on
the scope. And if this is impossible or undesirable, then we can use target
type/pattern-specific variables (if there is a common pattern) or simply list
the affected targets explicitly. For example:

\
obj{*.test}: cxx.poptions += -DDEFINE_MAIN
obj{main utility}: cxx.poptions += -DNDEBUG
\

The first line covers compilation of source files that have the \c{.test}
second-level extension (see \l{#intro-unit-test Implementing Unit Testing}
for background) while the second simply lists the targets explicitly.

It is also possible to specify different options when producing different
types of object files (\c{obje{\}} \- executable, \c{obja{\}} \- static
library, or \c{objs{\}} \- shared library) or when linking different libraries
(\c{liba{\}} \- static library or \c{libs{\}} \- shared library). See
\l{#intro-lib Library Exportation and Versioning} for an example.|

As mentioned above, each \c{buildfile} in a project is loaded into its
corresponding scope. As a result, we rarely need to open scopes explicitly.
In the few cases that we do, we use the following syntax:

\
<directory>/
{
  ...
}
\

If the scope directory is relative, then it is assumed to be relative to the
current scope. As an exercise for understanding, let's reimplement our
\c{hello} project as a single \c{buildfile}. That is, we move the contents of
the source directory \c{buildfile} into the root \c{buildfile}:

\
$ tree hello/
hello/
├── build/
│   └── ...
├── hello/
│   └── hello.cxx
└── buildfile

$ cat hello/buildfile

./: hello/

hello/
{
  ./: exe{hello}: {hxx cxx}{**}
}
\

\N|While this single \c{buildfile} setup is not recommended for new projects,
it can be useful for non-intrusive conversion of existing projects to
\c{build2}. One approach is to place the unmodified original project into a
subdirectory (potentially automating this with a mechanism such as \c{git(1)}
submodules) then adding the \c{build/} subdirectory and the root \c{buildfile}
which explicitly opens scopes to define the build over the upstream project's
subdirectory structure.|

Seeing this merged \c{buildfile} may make you wonder what exactly caused the
loading of the source directory \c{buildfile} in our normal setup. In other
words, when we build our \c{hello} from the project root, who loads
\c{hello/buildfile} and why?

Actually, in the earlier days of \c{build2}, we had to explicitly load
\c{buildfiles} that define targets we depend on with the \c{include}
directive. In fact, we still can (and have to if we are depending on
targets other than directories). For example:

\
./: hello/

include hello/buildfile
\

We can also omit \c{buildfile} for brevity and have just:

\
include hello/
\

This explicit inclusion, however, quickly becomes tiresome as the number of
directories grows. It also makes using wildcard patterns for subdirectory
prerequisites a lot less appealing.

To overcome this the \c{dir{\}} target type implements an interesting
prerequisite to target resolution semantics: if there is no existing target
with this name, a \c{buildfile} that (presumably) defines this target is
automatically loaded from the corresponding directory. In fact, this mechanism
goes a step further and, if the \c{buildfile} does not exist, then it assumes
one with the following contents was implied:

\
./: */
\

That is, it simply builds all the subdirectories. This is especially handy
when organizing related tests into directory hierarchies.

\N|As mentioned above, this automatic inclusion is only triggered if the
target we depend on is \c{dir{\}} and we still have to explicitly include the
necessary \c{buildfiles} for other targets. One common example is a project
consisting of a library and an executable that links it, each residing in a
separate directory next to each other (as noted earlier, this is not
recommended for projects that you plan to package). For example:

\
hello/
├── build/
│   └── ...
├── hello/
│   ├── main.cxx
│   └── buildfile
├── libhello/
│   ├── hello.hxx
│   ├── hello.cxx
│   └── buildfile
└── buildfile
\

In this case the executable \c{buildfile} would look along these lines:

\
include ../libhello/ # Include lib{hello}.

exe{hello}: {hxx cxx}{**} ../libhello/lib{hello}
\

Note also that \c{buildfile} inclusion should only be used for accessing
targets within the same project. For cross-project references we use
\l{#intro-import Target Importation}.|


\h#intro-operations|Operations|

Modern build systems have to perform operations other than just building:
cleaning the build output, running tests, installing/uninstalling the build
results, preparing source distributions, and so on. And, if the build system has
integrated configuration support, configuring the project would naturally
belong to this list as well.

\N|If you are familiar with \c{make}, you should recognize the parallel with
the common \c{clean} \c{test}, \c{install}, and \c{dist}, \"operation\"
pseudo-targets.|

In \c{build2} we have the concept of a \i{build system operation} performed on
a target. The two pre-defined operations are \c{update} and \c{clean} with
other operations provided by build system modules.

Operations to be performed and targets to perform them on are specified on the
command line. As discussed earlier, \c{update} is the default operation and
\c{./} in the current directory is the default target if no operation and/or
target is specified explicitly. And, similar to targets, we can specify
multiple operations (not necessarily on the same target) in a single build
system invocation. The list of operations to perform and targets to perform
them on is called a \i{build specification} or \i{buildspec} for short (see
\l{b(1)} for details). Here are a few examples:

\
$ cd hello        # Change to project root.

$ b               # Update current directory.
$ b ./            # Same as above.
$ b update        # Same as above.
$ b update: ./    # Same as above.

$ b clean update  # Rebuild.

$ b clean:  hello/             # Clean specific target.
$ b update: hello/exe{hello}   # Update specific target

$ b update: libhello/ tests/   # Update two targets.
\

Let's revisit \c{build/bootstrap.build} from our \c{hello} project:

\
project = hello

using version
using config
using test
using install
using dist
\

Other than \c{version}, all the modules we load define new operations. Let's
examine each of them starting with \c{config}.


\h2#intro-operations-config|Configuring|

As mentioned briefly earlier, the \l{#module-config \c{config}} module
provides support for persisting configurations by having us \i{configure} our
projects. At first it may feel natural to call \c{configure} an operation.
There is, however, a conceptual problem: we don't really configure a
target. And, perhaps after some meditation, it should become clear that what
we are really doing is configuring operations on targets. For example,
configuring updating a C++ project might involve detecting and saving
information about the C++ compiler while configuring installing it may require
specifying the installation directory.

In other words, \c{configure} is an operation on operation on targets \- a
meta-operation.  And so in \c{build2} we have the concept of a \i{build system
meta-operation}.  If not specified explicitly (as part of the buildspec), the
default is \c{perform}, which is to simply perform the operation.

Back to \c{config}, this module provides two meta-operations: \c{configure}
which saves the configuration of a project into the \c{build/config.build}
file as well as \c{disfigure} which removes it.

\N|While the common meaning of the word \i{disfigure} is somewhat different to
what we make it mean in this context, we still prefer it over the commonly
suggested alternative (\i{deconfigure}) for the symmetry of their Latin
\i{con-} (\"together\") and \i{dis-} (\"apart\") prefixes.|

Let's say for the in source build of our \c{hello} project we want to use
\c{Clang} and enable debug information. Without persistence we would have to
repeat this configuration on every build system invocation:

\
$ cd hello/  # Change to project root.

$ b config.cxx=clang++ config.cxx.coptions=-g
\

Instead, we can configure our project with this information once and from
then on invoke the build system without any arguments:

\
$ b configure config.cxx=clang++ config.cxx.coptions=-g

$ tree ./
./
├── build/
│   ├── ...
│   └── config.build
└── ...

$ b
$ b clean
$ b
...
\

Let's take a look at \c{config.build}:

\
$ cat build/config.build

config.cxx = clang++
config.cxx.poptions = [null]
config.cxx.coptions = -g
config.cxx.loptions = [null]
config.cxx.aoptions = [null]
config.cxx.libs = [null]
...
\

As you can see, it's just a buildfile with a bunch of variable assignments. In
particular, this means you can tweak your build configuration by modifying
this file with your favorite editor. Or, alternatively, you can adjust the
configuration by reconfiguring the project:

\
$ b configure config.cxx=g++

$ cat build/config.build

config.cxx = g++
config.cxx.poptions = [null]
config.cxx.coptions = -g
config.cxx.loptions = [null]
config.cxx.aoptions = [null]
config.cxx.libs = [null]
...
\

Any variable value specified on the command line overrides those specified in
the \c{buildfiles}. As a result, \c{config.cxx} was updated while the value of
\c{config.cxx.coptions} was preserved.

Command line variable overrides are also handy to adjust the configuration for
a single build system invocation. For example, let's say we want to quickly
check that our project builds with optimization but without permanently
changing the configuration:

\
$ b config.cxx.coptions=-O3  # Rebuild with -O3.
$ b                          # Rebuild with -g.
\

\N|Besides the various \c{*.?options} variables, we can also specify the
\"compiler mode\" options as part of the compiler executable in \c{config.c}
and \c{config.cxx}. Such options cannot be modified by buildfiles and they
will appear last on the command lines. For example:

\
$ b configure config.cxx=\"g++ -m32\"
\

The compiler mode options are also the correct place to specify
\i{system-like} header (\c{-I}) and library (\c{-L}, \c{/LIBPATH}) search
paths. Where by system-like we mean common installation directories like
\c{/usr/include} or \c{/usr/local/lib} which may contain older versions of the
libraries we are trying to build and/or use. By specifying these paths as part
of the mode options (as opposed to \c{config.*.poptions} and
\c{config.*.loptions}) we make sure they will be considered last, similar to
the compiler's build-in search paths. For example:

\
$ b configure config.cxx=\"g++ -L/opt/install\"
\

|


If we would like to prevent subsequent changes to the environment from
affecting our build configuration, we can make it \i{hermetic} (see
\l{#module-config-hermetic Hermetic Build Configurations} for details):

\
$ b configure config.config.hermetic=true ...
\

\N|One prominent use of hermetic configurations is to preserve the build
environment of the Visual Studio development command prompt. That is,
hermetically configuring our project in a suitable Visual Studio command
prompt makes us free to build it from any other prompt or shell, IDE, etc.|

We can also configure out of source builds of our projects. In this case,
besides \c{config.build}, \c{configure} also saves the location of the source
directory so that we don't have to repeat that either. Remember, this is how
we used to build our \c{hello} out of source:

\
$ b hello/@hello-gcc/   config.cxx=g++
$ b hello/@hello-clang/ config.cxx=clang++
\

And now we can do:

\
$ b configure: hello/@hello-gcc/   config.cxx=g++
$ b configure: hello/@hello-clang/ config.cxx=clang++

$ hello-clang/
hello-clang/
└── build/
    ├── bootstrap/
    │   └── src-root.build
    └── config.build

$ b hello-gcc/
$ b hello-clang/
$ b hello-gcc/ hello-clang/
\

One major benefit of an in source build is the ability to run executables as
well as examine build and test output (test results, generated source code,
documentation, etc) without leaving the source directory. Unfortunately, we
cannot have multiple in source builds and as was discussed earlier, mixing in
and out of source builds is not recommended.

To overcome this limitation \c{build2} has a notion of \i{forwarded
configurations}. As the name suggests, we can configure a project's source
directory to forward to one of its out of source builds. Once done,
whenever we run the build system from the source directory, it will
automatically build in the corresponded forwarded output
directory. Additionally, it will \i{backlink} (using symlinks or another
suitable mechanism) certain \"interesting\" targets (\c{exe{\}}, \c{doc{\}})
to the source directory for easy access. As an example, let's configure our
\c{hello/} source directory to forward to the \c{hello-gcc/} build:

\
$ b configure: hello/@hello-gcc/,forward

$ cd hello/  # Change to project root.
$ b
c++ hello/cxx{hello}@../hello-gcc/hello/
ld ../hello-gcc/hello/exe{hello}
ln ../hello-gcc/hello/exe{hello} -> hello/
\

Notice the last line in the above listing: it indicates that \c{exe{hello}}
from the out directory was backlinked in our project's source subdirectory:

\
$ tree ./
./
├── build/
│   ├── bootstrap/
│   │   └── out-root.build
│   └── ...
├── hello/
│   ├── ...
│   └── hello -> ../../hello-gcc/hello/hello*
└── ...

$ ./hello/hello
Hello World!
\

\N|By default only \c{exe{\}} and \c{doc{\}} targets are backlinked. This,
however, can be customized with the \c{backlink} target-specific variable.|


\h2#intro-operations-test|Testing|

The next module we load in \c{bootstrap.build} is \l{#module-test \c{test}}
which defines the \c{test} operation. As the name suggests, this module
provides support for running tests.

There are two types of tests that we can run with the \c{test} module: simple
and scripted.

A simple test is just an executable target with the \c{test} target-specific
variable set to \c{true}. For example:

\
exe{hello}: test = true
\

A simple test is executed once and in its most basic form (typical for unit
testing) doesn't take any inputs nor produce any output, indicating success
via the zero exit status. If we test our \c{hello} project with the above
addition to the \c{buildfile}, then we will see the following output:

\
$ b test
test hello/exe{hello}
Hello, World!
\

While the test passes (since it exited with zero status), we probably don't
want to see that \c{Hello, World!} every time we run it (this can, however, be
quite useful when running examples). More importantly, we don't really test its
functionality and if tomorrow our \c{hello} starts swearing rather than
greeting, the test will still pass.

Besides checking its exit status we can also supply some basic information to
a simple test (more common for integration testing). Specifically, we can pass
command line options (\c{test.options}) and arguments (\c{test.arguments}) as
well as input (\c{test.stdin}, used to supply test's \c{stdin}) and output
(\c{test.stdout}, used to compare to test's \c{stdout}).

Let's see how we can use this to fix our \c{hello} test by making sure our
program prints the expected greeting. First, we need to add a file that will
contain the expected output, let's call it \c{test.out}:

\
$ ls -1 hello/
hello.cxx
test.out
buildfile

$ cat hello/test.out
Hello, World!
\

Next, we arrange for it to be compared to our test's \c{stdout}. Here is the
new \c{hello/buildfile}:

\
exe{hello}: {hxx cxx}{**}
exe{hello}: file{test.out}: test.stdout = true
\

The last line looks new. What we have here is a \i{prerequisite-specific
variable} assignment. By setting \c{test.stdout} for the \c{file{test.out\}}
prerequisite of target \c{exe{hello\}} we mark it as expected \c{stdout}
output of \i{this} target (theoretically, we could have marked it as
\c{test.input} for another target). Notice also that we no longer need the
\c{test} target-specific variable; it's unnecessary if one of the other
\c{test.*} variables is specified.

Now, if we run our test, we won't see any output:

\
$ b test
test hello/exe{hello}
\

And if we try to change the greeting in \c{hello.cxx} but not in \c{test.out},
our test will fail printing the \c{diff(1)} comparison of the expected and
actual output:

\
$ b test
c++ hello/cxx{hello}
ld hello/exe{hello}
test hello/exe{hello}
--- test.out
+++ -
@@ -1 +1 @@
-Hello, World!
+Hi, World!
error: test hello/exe{hello} failed
\

Notice another interesting thing: we have modified \c{hello.cxx} to change the
greeting and our test executable was automatically rebuilt before testing.
This happened because the \c{test} operation performs \c{update} as its
\i{pre-operation} on all the targets to be tested.

Let's make our \c{hello} program more flexible by accepting the name to
greet on the command line:

\
#include <iostream>

int main (int argc, char* argv[])
{
  if (argc < 2)
  {
    std::cerr << \"error: missing name\" << std::endl;
    return 1;
  }

  std::cout << \"Hello, \" << argv[1] << '!' << std::endl;
}
\

We can exercise its successful execution path with a simple test fairly
easily:

\
exe{hello}: test.arguments = 'World'
exe{hello}: file{test.out}: test.stdout = true
\

What if we also wanted to test its error handling? Since simple tests are
single-run, this won't be easy. Even if we could overcome this, having
expected output for each test in a separate file will quickly become untidy.
And this is where script-based tests come in. Testscript is \c{build2}'s
portable language for running tests. It vaguely resembles Bash and is
optimized for concise test implementation and fast, parallel execution.

Just to give you an idea (see \l{testscript#intro Testscript Introduction} for
a proper introduction), here is what testing our \c{hello} program with
Testscript would look like:

\
$ ls -1 hello/
hello.cxx
testscript
buildfile

$ cat hello/buildfile

exe{hello}: {hxx cxx}{**} testscript
\

And this is the contents of \c{hello/testscript}:

\
: basics
:
$* 'World' >'Hello, World!'

: missing-name
:
$* 2>>EOE != 0
error: missing name
EOE
\

A couple of key points: The \c{test.out} file is gone with all the test inputs
and expected outputs incorporated into \c{testscript}. To test an executable
with Testscript, all we have to do is list the corresponding \c{testscript}
file as its prerequisite (and which, being a fixed name, doesn't need an
explicit target type, similar to \c{manifest}).

To see Testscript in action, let's say we've made our program more forgiving
by falling back to a default name if one wasn't specified:

\
#include <iostream>

int main (int argc, char* argv[])
{
  const char* n (argc > 1 ? argv[1] : \"World\");
  std::cout << \"Hello, \" << n << '!' << std::endl;
}
\

If we forget to adjust the \c{missing-name} test, then this is what we could
expect to see when running the tests:

\
b test
c++ hello/cxx{hello}
ld hello/exe{hello}
test hello/testscript{testscript} hello/exe{hello}
hello/testscript:7:1: error: hello/hello exit code 0 == 0
  info: stdout: hello/test-hello/missing-name/stdout
\

Testscript-based integration testing is the default setup for executable
(\c{-t\ exe}) projects created by \l{bdep-new(1)}. Here is the recap of the
overall layout:

\
hello/
├── build/
│   └── ...
├── hello/
│   ├── hello.cxx
│   ├── testscript
│   └── buildfile
├── buildfile
└── manifest
\

For libraries (\c{-t\ lib}), however, the integration testing setup is a bit
different. Here are the relevant parts of the layout:

\
libhello/
├── build/
│   └── ...
├── libhello/
│   ├── hello.hxx
│   ├── hello.cxx
│   ├── export.hxx
│   ├── version.hxx.in
│   └── buildfile
├── tests/
│   ├── build/
│   │   ├── bootstrap.build
│   │   └── root.build
│   ├── basics/
│   │   ├── driver.cxx
│   │   └── buildfile
│   └── buildfile
├── buildfile
└── manifest
\

Specifically, there is no \c{testscript} in \c{libhello/}, the project's
source directory. Instead, we have the \c{tests/} subdirectory which itself
looks like a project: it contains the \c{build/} subdirectory with all the
familiar files, etc. In fact, \c{tests} is a \i{subproject} of our
\c{libhello} project.

While we will be examining \c{tests} in greater detail later, in a nutshell,
the reason it is a subproject is to be able to test an installed version of
our library. By default, when \c{tests} is built as part of its parent project
(called \i{amalgamation}), the locally built \c{libhello} library will be
automatically imported. However, we can also configure a build of \c{tests}
out of its amalgamation, in which case we can import an installed version of
\c{libhello}. We will learn how to do all that as well as the underlying
concepts (\i{subproject}/\i{amalgamation}, \i{import}, etc) in the coming
sections.

Inside \c{tests/} we have the \c{basics/} subdirectory which contains a simple
test for our library's API. By default it doesn't use Testscript but if you
want to, you can. You can also rename \c{basics/} to something more meaningful
and add more tests next to it. For example, if we were creating an XML parsing
and serialization library, then our \c{tests/} could have the following
layout:

\
tests/
├── build/
│   └── ...
├── parser/
│   └── ...
├── serializer/
│   └── ...
└── buildfile
\

\N|Nothing prevents us from having the \c{tests/} subdirectory for executable
projects. And it can be just a subdirectory or a subproject, the same as for
libraries. Making it a subproject makes sense if your program has complex
installation, for example, if its execution requires configuration and/or data
files that need to be found, etc. For simple programs, however, testing the
executable before installing it is usually sufficient.

For a general discussion of functional/integration and unit testing refer to
the \l{intro#proj-struct-tests Tests} section in the toolchain introduction.
For details on the unit test support implementation see \l{#intro-unit-test
Implementing Unit Testing}.|


\h2#intro-operations-install|Installing|

The \l{#module-install \c{install}} module defines the \c{install} and
\c{uninstall} operations.  As the name suggests, this module provides support
for project installation.

\N|Installation in \c{build2} is modeled after UNIX-like operation systems
though the installation directory layout is highly customizable. While
\c{build2} projects can import \c{build2} libraries directly, installation is
often a way to \"export\" them in a form usable by other build systems.|

The root installation directory is specified with the \c{config.install.root}
configuration variable. Let's install our \c{hello} program into
\c{/tmp/install}:

\
$ cd hello/  # Change to project root.

$ b install config.install.root=/tmp/install/
\

And see what we've got (executables are marked with \c{*}):

\
$ tree /tmp/install/

/tmp/install/
├── bin/
│   └── *hello
└── share/
    └── doc/
        └── hello/
            └── manifest
\

Similar to the \c{test} operation, \c{install} performs \c{update} as a
pre-operation for targets that it installs.

\N|We can also configure our project with the desired \c{config.install.*}
values so that we don't have to repeat them on every install/uninstall. For
example:

\
$ b configure config.install.root=/tmp/install/
$ b install
$ b uninstall
\

|

Now let's try the same for \c{libhello} (symbolic link targets are shown with
\c{->} and actual static/shared library names may differ on your operating
system):

\
$ rm -r /tmp/install

$ cd libhello/  # Change to project root.

$ b install config.install.root=/tmp/install/

$ tree /tmp/install/

/tmp/install/
├── include/
│   └── libhello/
│       ├── hello.hxx
│       ├── export.hxx
│       └── version.hxx
├── lib/
│   ├── pkgconfig/
│   │   ├── libhello.pc
│   │   ├── libhello.shared.pc
│   │   └── libhello.static.pc
│   ├── libhello.a
│   ├── libhello.so -> libhello-0.1.so
│   └── libhello-0.1.so
└── share/
    └── doc/
        └── libhello/
            └── manifest
\

As you can see, the library headers go into the customary \c{include/}
subdirectory while static and shared libraries (and their \c{pkg-config(1)}
files) \- into \c{lib/}. Using this installation we should be able to import
this library from other build systems or even use it in a manual build:

\
$ g++ -I/tmp/install/include -L/tmp/install/lib greet.cxx -lhello
\

If we want to install into a system-wide location like \c{/usr} or
\c{/usr/local}, then we most likely will need to specify the \c{sudo(1)}
program:

\
$ b config.install.root=/usr/local/ config.install.sudo=sudo
\

\N|In \c{build2} only actual install/uninstall commands are executed with
\c{sudo(1)}. And while on the topic of sensible implementations, \c{uninstall}
can be generally trusted to work reliably.|

The default installability of a target as well as where it is installed is
determined by its target type. For example, \c{exe{\}} is by default installed
into \c{bin/}, \c{doc{\}} \- into \c{share/doc/<project>/}, and \c{file{\}} is
not installed.

We can, however, override these defaults with the \c{install} target-specific
variable.  Its value should be either special \c{false} indicating that the
target should not be installed or the directory to install the target to. As
an example, here is what the root \c{buildfile} from our \c{libhello} project
looks like:

\
./: {*/ -build/} manifest

tests/: install = false
\

The first line we have already seen and the purpose of the second line should
now be clear: it makes sure we don't try to install anything in the \c{tests/}
subdirectory.

If the value of the \c{install} variable is not \c{false}, then it is normally
a relative path with the first path component being one of these names:

\
name        default                         override
----        -------                         --------
root                                        config.install.root

data_root   root/                           config.install.data_root
exec_root   root/                           config.install.exec_root

bin         exec_root/bin/                  config.install.bin
sbin        exec_root/sbin/                 config.install.sbin
lib         exec_root/lib/                  config.install.lib
libexec     exec_root/libexec/<project>/    config.install.libexec
pkgconfig   lib/pkgconfig/                  config.install.pkgconfig

etc         data_root/etc/                  config.install.etc
include     data_root/include/              config.install.include
share       data_root/share/                config.install.share
data        share/<project>/                config.install.data

doc         share/doc/<project>/            config.install.doc
legal       doc/                            config.install.legal
man         share/man/                      config.install.man
man<N>      man/man<N>/                     config.install.man<N>
\

Let's see what's going on here: The default install directory tree is derived
from the \c{config.install.root} value but the location of each node in this
tree can be overridden by the user that installs our project using the
corresponding \c{config.install.*} variables. In our \c{buildfiles}, in turn,
we use the node names instead of actual directories. As an example, here is a
\c{buildfile} fragment from the source directory of our \c{libhello} project:

\
hxx{*}:
{
  install         = include/libhello/
  install.subdirs = true
}
\

Here we set the installation location for headers to be the \c{libhello/}
subdirectory of the \c{include} installation location. Assuming
\c{config.install.root} is \c{/usr/}, the \c{install} module will perform the
following steps to resolve this relative path to the actual, absolute
installation directory:

\
include/libhello/
data_root/include/libhello/
root/include/libhello/
/usr/include/libhello/
\

In the above \c{buildfile} fragment we also see the use of the
\c{install.subdirs} variable. Setting it to \c{true} instructs the \c{install}
module to recreate subdirectories starting from this point in the project's
directory hierarchy.  For example, if our \c{libhello/} source directory had
the \c{details/} subdirectory with the \c{utility.hxx} header, then this
header would have been installed as
\c{.../include/libhello/details/utility.hxx}.


\h2#intro-operations-dist|Distributing|

The last module that we load in our \c{bootstrap.build} is \c{dist} which
provides support for the preparation of distributions and defines the \c{dist}
meta-operation. Similar to \c{configure}, \c{dist} is a meta-operation rather
than an operation because, conceptually, we are preparing a distribution for
performing operations (like \c{update}, \c{test}) on targets rather than
targets themselves.

The preparation of a correct distribution requires that all the necessary
project files (sources, documentation, etc) be listed as prerequisites in the
project's \c{buildfiles}.

\N|You may wonder why not just use the export support offered by many version
control systems? The main reason is that in most real-world projects version
control repositories contain a lot more than what needs to be distributed. In
fact, it is not uncommon to host multiple build system projects/packages in a
single repository. As a result, with this approach we seem to inevitably end
up maintaining an exclusion list, which feels backwards: why specify all the
things we don't want in a new list instead of making sure the already existing
list of things that we do want is complete? Also, once we have the complete
list, it can be put to good use by other tools, such as editors, IDEs, etc.|

The preparation of a distribution also requires an out of source build. This
allows the \c{dist} module to distinguish between source and output
targets. By default, targets found in src are included into the distribution
while those in out are excluded. However, we can customize this with the
\c{dist} target-specific variable.

As an example, let's prepare a distribution of our \c{hello} project using the
out of source build configured in \c{hello-out/}. We use \c{config.dist.root}
to specify the directory to write the distribution to:

\
$ b dist: hello-out/ config.dist.root=/tmp/dist

$ ls -1 /tmp/dist
hello-0.1.0/

$ tree /tmp/dist/hello-0.1.0/
/tmp/dist/hello-0.1.0/
├── build/
│   ├── bootstrap.build
│   └── root.build
├── hello/
│   ├── hello.cxx
│   ├── testscript
│   └── buildfile
├── buildfile
└── manifest
\

As we can see, the distribution directory includes the project version (from
the \c{version} variable which, in our case, is extracted from
\c{manifest} by the \c{version} module). Inside the distribution directory we
have our project's source files (but, for example, without any \c{.gitignore}
files that we may have had in \c{hello/}).

We can also ask the \c{dist} module to package the distribution directory into
one or more archives and generate their checksum files for us. For example:

\
$ b dist: hello-out/ \
  config.dist.root=/tmp/dist \
  config.dist.archives=\"tar.gz zip\" \
  config.dist.checksums=sha256

$ ls -1 /tmp/dist
hello-0.1.0/
hello-0.1.0.tar.gz
hello-0.1.0.tar.gz.sha256
hello-0.1.0.zip
hello-0.1.0.zip.sha256
\

\N|We can also configure our project with the desired \c{config.dist.*} values
so we don't have to repeat them every time. For example:

\
$ b configure: hello-out/ config.dist.root=/tmp/dist ...
$ b dist
\

|

Let's now take a look at an example of customizing what gets distributed.
Most of the time you will be using this mechanism to include certain targets
from out. Here is a fragment from the \c{libhello} source directory
\c{buildfile}:

\
hxx{version}: in{version} $src_root/manifest
{
  dist = true
}
\

Our library provides the \c{version.hxx} header that the users can include to
obtain its version. This header is generated by the \c{version} module from
the \c{version.hxx.in} template. In essence, the \c{version} module takes the
version value from our \c{manifest}, splits it into various components (major,
minor, patch, etc) and then preprocesses the \c{in{\}} file substituting these
values (see the \l{#module-version \c{version}} module documentation for
details). The end result is an automatically maintained version header.

One problem with auto-generated headers is that if one does not yet exist,
then the compiler may still find it somewhere else. For example, we may have
an older version of a library installed somewhere where the compiler searches
for headers by default (for example, \c{/usr/local/include/}). To overcome
this problem it is a good idea to ship pre-generated headers in our
distributions.  But since they are output targets, we have to explicitly
request this with \c{dist=true}.


\h#intro-import|Target Importation|

Recall that if we need to depend on a target defined in another \c{buildfile}
within our project, then we simply include said \c{buildfile} and reference
the target.  For example, if our \c{hello} included both an executable and a
library in separate subdirectories next to each other:

\
hello/
├── build/
│   └── ...
├── hello/
│   ├── ...
│   └── buildfile
└── libhello/
    ├── ...
    └── buildfile
\

Then our executable \c{buildfile} could look like this:

\
include ../libhello/ # Include lib{hello}.

exe{hello}: {hxx cxx}{**} ../libhello/lib{hello}
\

What if instead \c{libhello} were a separate project? The inclusion approach
would no longer work for two reasons: we don't know the path to \c{libhello}
(after all, it's an independent project and can reside anywhere) and we can't
assume the path to the \c{lib{hello\}} target within \c{libhello} (the project
directory layout can change).

To depend on a target from a separate project we use \i{importation} instead
of inclusion. This mechanism is also used to depend on targets that are not
part of any project, for example, installed libraries.

The importing project's side is pretty simple. This is what the above
\c{buildfile} will look like if \c{libhello} were a separate project:

\
import libs = libhello%lib{hello}

exe{hello}: {hxx cxx}{**} $libs
\

The \c{import} directive is a kind of variable assignment that resolves a
\i{project-qualified} relative target (\c{libhello%lib{hello\}})
to an unqualified absolute target and stores it in the variable (\c{libs} in
our case). We can then expand the variable (\c{$libs}), normally
in the dependency declaration, to get the imported target.

If we needed to import several libraries, then we simply repeat the \c{import}
directive, usually accumulating the result in the same variable, for example:

\
import libs  = libformat%lib{format}
import libs += libprint%lib{print}
import libs += libhello%lib{hello}

exe{hello}: {hxx cxx}{**} $libs
\

Let's now try to build our \c{hello} project that uses imported \c{libhello}:

\
$ b hello/
error: unable to import target libhello%lib{hello}
  info: use config.import.libhello command line variable to specify
        its project out_root
\

While that didn't work out well, it does make sense: the build system cannot
know the location of \c{libhello} or which of its builds we want to use.
Though it does helpfully suggest that we use \c{config.import.libhello} to
specify its out directory (\c{out_root}). Let's point it to \c{libhello}
source directory to use its in source build (\c{out_root\ ==\ src_root}):

\
$ b hello/ config.import.libhello=libhello/
c++ libhello/libhello/cxx{hello}
ld libhello/libhello/libs{hello}
c++ hello/hello/cxx{hello}
ld hello/hello/exe{hello}
\

And it works. Naturally, the importation mechanism works the same for out of
source builds and we can persist the \c{config.import.*} variables in the
project's configuration. As an example, let's configure Clang builds of the
two projects out of source:

\
$ b configure: libhello/@libhello-clang/ config.cxx=clang++
$ b configure: hello/@hello-clang/ config.cxx=clang++ \
  config.import.libhello=libhello-clang/

$ b hello-clang/
c++ libhello/libhello/cxx{hello}@libhello-clang/libhello/
ld libhello-clang/libhello/libs{hello}
c++ hello/hello/cxx{hello}@hello-clang/hello/
ld hello-clang/hello/exe{hello}
\

If the corresponding \c{config.import.*} variable is not specified, \c{import}
searches for a project in a couple of other places. First, it looks in the list
of subprojects starting from the importing project itself and then continuing
with its outer amalgamations and their subprojects (see \l{#intro-subproj
Subprojects and Amalgamations} for details on this subject).

\N|We've actually seen an example of this search step in action: the \c{tests}
subproject in \c{libhello}. The test imports \c{libhello} which is
automatically found as an amalgamation containing this subproject.|

If the project being imported cannot be located using any of these methods,
then \c{import} falls back to the rule-specific search. That is, a rule that
matches the target may provide support for importing certain target types
based on rule-specific knowledge. Support for importing installed libraries by
the C++ link rule is a good example of this. Internally, the \c{cxx} module
extracts the compiler's library search paths (that is, paths that would be
used to resolve \c{-lfoo}) and then the link rule uses them to search for
installed libraries. This allows us to use the same \c{import} directive
regardless of whether we import a library from a separate build, from a
subproject, or from an installation directory.

\N|Importation of an installed library will work even if it is not a
\c{build2} project. Besides finding the library itself, the link rule will
also try to locate its \c{pkg-config(1)} file and, if present, extract
additional compile/link flags from it. The link rule also automatically
produces \c{pkg-config(1)} files for libraries that it installs.|

\N|A common problem with importing and using third-party C/C++ libraries is
compiler warnings. Specifically, we are likely to include their headers into
our project's source files which means we may see warnings in such headers
(which we cannot always fix) mixed with warnings in our code (which we should
normally be able to fix). See \l{#cc-internal-scope Compilation Internal
Scope} for a mechanism to deal with this problem.|

Let's now examine the exporting side of the importation mechanism. While a
project doesn't need to do anything special to be found by \c{import}, it does
need to handle locating the exported target (or targets; there could be
several) within the project as well as loading their \c{buildfiles}. And this
is the job of an \i{export stub}, the \c{build/export.build} file that you
might have noticed in the \c{libhello} project:

\
libhello
├── build/
│   └── export.build
└── ...
\

Let's take a look inside:

\
$out_root/
{
  include libhello/
}

export $out_root/libhello/$import.target
\

An export stub is a special kind of \c{buildfile} that bridges from the
importing project into exporting. It is loaded in a special temporary scope
outside of any project, in a \"no man's land\" so to speak. The only variables set
on the temporary scope are \c{src_root} and \c{out_root} of the project being
imported as well as \c{import.target} containing the name of the target being
imported (without project qualification; that is, \c{lib{hello\}} in our
example).

Typically, an export stub will open the scope of the exporting project, load
the \c{buildfile} that defines the target being exported and finally
\"return\" the absolute target name to the importing project using the
\c{export} directive. And this is exactly what the export stub in our
\c{libhello} does.

We now have all the pieces of the importation puzzle in place and you can
probably see how they all fit together. To summarize, when the build system
sees the \c{import} directive, it looks for a project with the specified
name. If found, it creates a temporary scope, sets the \c{src/out_root}
variables to point to the project and \c{import.target} \- to the target name
specified in the \c{import} directive. And then it load the project's export
stub in this scope. Inside the export stub we switch to the project's root
scope, load its \c{buildfile} and then use the \c{export} directive to return
the exported target. Once the export stub is processed, the build system
obtains the exported target and assigns it to the variable specified in the
\c{import} directive.

\N|Our export stub is quite \"loose\" in that it allows importing any target
defined in the project's source subdirectory \c{buildfile}. While we found it
to be a good balance between strictness and flexibility, if you would like to
\"tighten\" your export stubs, you can. For example:

\
if ($import.target == lib{hello})
  export $out_root/libhello/$import.target
\

If no \c{export} directive is executed in an export stub then the build system
assumes that the target is not exported by the project and issues appropriate
diagnostics.|

Let's revisit the executable \c{buildfile} with which we started this section.
Recall that it is for an executable that depends on a library which resides
in the same project:

\
include ../libhello/ # Include lib{hello}.

exe{hello}: {hxx cxx}{**} ../libhello/lib{hello}
\

If \c{lib{hello\}} is exported by this project, then instead of manually
including its \c{buildfile} we can use \i{project-local importation}:

\
import lib = lib{hello}

exe{hello}: {hxx cxx}{**} $lib
\

The main advantage of project-local importation over inclusion is the ability
to move things around without having to adjust locations in multiple places
(the only place we need to do it is the export stub). This advantage becomes
noticeable in more complex projects with a large number of components.

\N|An import is project-local if the target being imported has no project
name. Note that the target must still be exported in the project's export
stub. In other words, project-local importation use the same mechanism as the
normal import.|

Another special type of importation is \i{ad hoc importation}. It is triggered
if the target being imported has no project name and is either absolute or is
a relative directory (in which case it is interpreted as relative to the
importing scope). Semantically this is similar a normal import but with the
location of the project being imported hard-coded into the \c{buildfile}.
While this would be a bad idea in most case, sometimes we may want to create a
special \i{glue \c{buildfile}} that \"pulls\" together several projects,
usually for convenience of development.

One typical case that calls for such a glue \c{buildfile} is a multi-package
project. For example, we may have a \c{hello} project (in a more general
sense, as in a version control repository) that contains the \c{libhello}
library and \c{hello} executable packages (which are independent build system
projects):

\
hello/
├── .git/
├── hello/
│   ├── build/
│   │   └── ...
│   ├── hello/
│   │   └── ...
│   ├── buildfile
│   └── manifest
└── libhello/
    ├── build/
    │   └── ...
    ├── libhello/
    │   └── ...
    ├── buildfile
    └── manifest
\

Notice that the root of this repository is not a build system project and we
cannot, for example, just run the build system driver without any arguments to
update all the packages. Instead we have to list them explicitly:

\
$ b hello/ libhello/
\

And that's inconvenient. To overcome this shortcoming we can turn the
repository root into a simple build system project by adding a glue
\c{buildfile} that imports (using ad hoc importation) and builds all the
packages:

\
import pkgs = */

./: $pkgs
\

\N|Unlike other import types, ad hoc importation does not rely (or require) an
export stub. Instead, it directly loads a \c{buildfile} that could plausibly
declare the target being imported.

In the unlikely event of a project-local importation of a directory target,
it will have to be spelled with an explicit \c{dir{\}} target type, for
example:

\
import d = dir{tests/}
\

|


\h#intro-lib|Library Exportation and Versioning|

By now we have examined and explained every line of every \c{buildfile} in our
\c{hello} executable project. There are, however, still a few lines to be
covered in the source subdirectory \c{buildfile} in \c{libhello}. Here it is
in its entirety:

\
intf_libs = # Interface dependencies.
impl_libs = # Implementation dependencies.

lib{hello}: {hxx ixx txx cxx}{** -version} hxx{version} \
  $impl_libs $intf_libs

# Include the generated version header into the distribution (so that
# we don't pick up an installed one) and don't remove it when cleaning
# in src (so that clean results in a state identical to distributed).
#
hxx{version}: in{version} $src_root/manifest
{
  dist  = true
  clean = ($src_root != $out_root)
}

# Build options.
#
cxx.poptions =+ \"-I$out_root\" \"-I$src_root\"

obja{*}: cxx.poptions += -DLIBHELLO_STATIC_BUILD
objs{*}: cxx.poptions += -DLIBHELLO_SHARED_BUILD

# Export options.
#
lib{hello}:
{
  cxx.export.poptions = \"-I$out_root\" \"-I$src_root\"
  cxx.export.libs = $intf_libs
}

liba{hello}: cxx.export.poptions += -DLIBHELLO_STATIC
libs{hello}: cxx.export.poptions += -DLIBHELLO_SHARED

# For pre-releases use the complete version to make sure they cannot
# be used in place of another pre-release or the final version. See
# the version module for details on the version.* variable values.
#
if $version.pre_release
  lib{hello}: bin.lib.version = \"-$version.project_id\"
else
  lib{hello}: bin.lib.version = \"-$version.major.$version.minor\"

# Install into the libhello/ subdirectory of, say, /usr/include/
# recreating subdirectories.
#
{hxx ixx txx}{*}:
{
  install         = include/libhello/
  install.subdirs = true
}
\

Let's start with all those \c{cxx.export.*} variables. It turns out that
merely exporting a library target is not enough for the importers of the
library to be able to use it. They also need to know where to find its
headers, which other libraries to link, etc. This information is carried in a
set of target-specific \c{cxx.export.*} variables that parallel the \c{cxx.*}
set and that together with the library's prerequisites constitute the
\i{library metadata protocol}. Every time a source file that depends on a
library is compiled or a binary is linked, this information is automatically
extracted by the compile and link rules from the library dependency chain,
recursively. And when the library is installed, this information is carried
over to its \c{pkg-config(1)} file.

\N|Similar to the \c{c.*} and \c{cc.*} sets discussed earlier, there are also
\c{c.export.*} and \c{cc.export.*} sets.|

Here are the parts relevant to the library metadata protocol in the above
\c{buildfile}:

\
intf_libs = # Interface dependencies.
impl_libs = # Implementation dependencies.

lib{hello}: ... $impl_libs $intf_libs

lib{hello}:
{
  cxx.export.poptions = \"-I$out_root\" \"-I$src_root\"
  cxx.export.libs = $intf_libs
}

liba{hello}: cxx.export.poptions += -DLIBHELLO_STATIC
libs{hello}: cxx.export.poptions += -DLIBHELLO_SHARED
\

As a first step we classify all our library dependencies into \i{interface
dependencies} and \i{implementation dependencies}. A library is an interface
dependency if it is referenced from our interface, for example, by including
(importing) one of its headers (modules) from one of our (public) headers
(modules) or if one of its functions is called from our inline or template
functions. Otherwise, it is an implementation dependency.

To illustrate the distinction between interface and implementation
dependencies, let's say we've reimplemented our \c{libhello} to use
\c{libformat} to format the greeting and \c{libprint} to print it.  Here is
our new header (\c{hello.hxx}):

\
#include <libformat/format.hxx>

namespace hello
{
  void
  say_hello_formatted (std::ostream&, const std::string& hello);

  inline void
  say_hello (std::ostream& o, const std::string& name)
  {
    say_hello_formatted (o, format::format_hello (\"Hello\", name));
  }
}
\

And this is the new source file (\c{hello.cxx}):

\
#include <libprint/print.hxx>

namespace hello
{
  void
  say_hello_formatted (ostream& o, const string& h)
  {
    print::print_hello (o, h);
  }
}
\

In this case, \c{libformat} is our interface dependency since we both include
its header in our interface and call it from one of our inline functions. In
contrast, \c{libprint} is only included and used in the source file and so we
can safely treat it as an implementation dependency. The corresponding
\c{import} directives in our \c{buildfile} will therefore look like this:

\
import intf_libs = libformat%lib{format}
import impl_libs = libprint%lib{print}
\

The preprocessor options (\c{poptions}) of an interface dependency must be
made available to our library's users. The library itself should also be
explicitly linked whenever our library is linked. All this is achieved by
listing the interface dependencies in the \c{cxx.export.libs} variable:

\
lib{hello}:
{
  cxx.export.libs = $intf_libs
}
\

\N|More precisely, the interface dependency should be explicitly linked if a
user of our library may end up with a direct call to the dependency in one of
their object files. Not linking such a library is called \i{underlinking}
while linking a library unnecessarily (which can happen because we've included
its header but are not actually calling any of its non-inline/template
functions) is called \i{overlinking}. Underlinking is an error on some
platforms while overlinking may slow down the process startup and/or waste its
memory.

Note also that this only applies to shared libraries. In case of static
libraries, both interface and implementation dependencies are always linked,
recursively.|

The remaining lines in the library metadata fragment are:

\
lib{hello}:
{
  cxx.export.poptions = \"-I$out_root\" \"-I$src_root\"
}

liba{hello}: cxx.export.poptions += -DLIBHELLO_STATIC
libs{hello}: cxx.export.poptions += -DLIBHELLO_SHARED
\

The first line makes sure the users of our library can locate its headers by
exporting the relevant \c{-I} options. The last two lines define the library
type macros that are relied upon by the \c{export.hxx} header to properly
setup symbol exporting.

\N|The \c{liba{\}} and \c{libs{\}} target types correspond to the static and
shared libraries, respectively. And \c{lib{\}} is actually a target group that
can contain one, the other, or both as its members.

Specifically, when we build a \c{lib{\}} target, which members will be built
is determined by the \c{config.bin.lib} variable with the \c{static},
\c{shared}, and \c{both} (default) possible values. So to only build a shared
library we can run:

\
$ b config.bin.lib=shared
\

When it comes to linking \c{lib{\}} prerequisites, which member is picked is
controlled by the \c{config.bin.{exe,liba,libs\}.lib} variables for the
executable, static library, and shared library targets, respectively. Each
contains a list of \c{shared} and \c{static} values that determine the linking
preferences. For example, to build both shared and static libraries but to
link executable to static libraries we can run:

\
$ b config.bin.lib=both config.bin.exe.lib=static
\

See the \l{#module-bin \c{bin}} module documentation for more information.|

Note also that we don't need to change anything in the above \c{buildfile} if
our library is header-only. In \c{build2} this is handled dynamically and
automatically based on the absence of source file prerequisites. In fact, the
same library can be header-only on some platforms or in some configuration and
\"source-ful\" in others.

\N|In \c{build2} a header-only library (or a module interface-only library) is
not a different kind of library compared to static/shared libraries but is
rather a binary-less, or \i{binless} for short, static or shared library. So,
theoretically, it is possible to have a library that has a binless static and
a binary-ful (\i{binful}) shared variants. Note also that binless libraries
can depend on binful libraries and are fully supported where the
\c{pkg-config(1)} functionality is concerned.

If you are creating a new library with \l{bdep-new(1)} and are certain that it
will always be binless and in all configurations, then you can produce a
simplified \c{buildfile} by specifying the \c{binless} option, for example:

\
$ bdep new -l c++ -t lib,binless libheader-only
\

|

Let's now turn to the second subject of this section and the last unexplained
bit in our \c{buildfile}: shared library versioning. Here is the relevant
fragment:

\
if $version.pre_release
  lib{hello}: bin.lib.version = \"-$version.project_id\"
else
  lib{hello}: bin.lib.version = \"-$version.major.$version.minor\"
\

Shared library versioning is a murky, platform-specific area. Instead of
trying to come up with a unified versioning scheme that few are likely to
comprehend (similar to \c{autoconf}), \c{build2} provides a
platform-independent versioning scheme as well as the ability to specify
platform-specific versions in a native format.

The library version is specified with the \c{bin.lib.version} target-specific
variable. Its value should be a sequence of \c{@}-pairs with the left hand
side (key) being the platform name and the right hand side (value) being the
version. An empty key (in which case \c{@} can be omitted) signifies the
platform-independent version (see the \l{#module-bin \c{bin}} module
documentation for the exact semantics). For example:

\
lib{hello}: bin.lib.version = -1.2 linux@3
\

\N{While the interface for platform-specific versions is defined, their
support is currently only implemented on Linux.}

A platform-independent version is embedded as a suffix into the library name
(and into its \c{soname} on relevant platforms) while platform-specific
versions are handled according to the platform. Continuing with the above
example, these would be the resulting shared library names on select
platforms:

\
libhello.so.3       # Linux
libhello-1.2.dll    # Windows
libhello-1.2.dylib  # Mac OS
\

With this background we can now explain what's going in our \c{buildfile}:

\
if $version.pre_release
  lib{hello}: bin.lib.version = \"-$version.project_id\"
else
  lib{hello}: bin.lib.version = \"-$version.major.$version.minor\"
\

Here we only use platform-independent library versioning. For releases we
embed both major and minor version components assuming that patch releases are
binary compatible. For pre-releases, however, we use the complete version to
make sure it cannot be used in place of another pre-release or the final
version.

\N|The \c{version.project_id} variable contains the project's (as opposed to
package's), shortest \"version id\". See the \l{#module-version \c{version}}
module documentation for details.|

\h#intro-subproj|Subprojects and Amalgamations|

In \c{build2} projects can contain other projects, recursively. In this
arrangement the outer project is called an \i{amalgamation} and the inner \-
\i{subprojects}. In contrast to importation where we merely reference a
project somewhere else, amalgamation is physical containment. It can be
\i{strong} where the src directory of a subproject is within the amalgamating
project or \i{weak} where only the out directory is contained.

There are several distinct use cases for amalgamations. We've already
discussed the \c{tests/} subproject in \c{libhello}. To recap: traditionally,
it is made a subproject rather than a subdirectory to support building it as a
standalone project in order to test library installations.

As discussed in \l{#intro-import Target Importation}, subprojects and
amalgamations (as well as their subprojects, recursively) are automatically
considered when resolving imports. As a result, amalgamation can be used to
\i{bundle} dependencies to produce an external dependency-free distribution.
For example, if our \c{hello} project imports \c{libhello}, then we could copy
the \c{libhello} project into \c{hello}, for example:

\
$ tree hello/
hello/
├── build/
│   └── ...
├── hello/
│   ├── hello.cxx
│   └── ...
├── libhello/
│   ├── build/
│   │   └── ...
│   ├── libhello/
│   │   ├── hello.hxx
│   │   ├── hello.cxx
│   │   └── ...
│   ├── tests/
│   │   └── ...
│   └── buildfile
└── buildfile

$ b hello/
c++ hello/libhello/libhello/cxx{hello}
ld hello/libhello/libhello/libs{hello}
c++ hello/hello/cxx{hello}
ld hello/hello/exe{hello}
\

Note, however, that while project bundling can be useful in certain cases, it
does not scale as a general dependency management solution. For that,
independent packaging and proper dependency management are the appropriate
mechanisms.

\N|By default \c{build2} looks for subprojects only in the root directory of a
project. That is, every root subdirectory is examined to see if it itself is a
project root. If you need to place a subproject somewhere else in your
project's directory hierarchy, then you will need to specify its location (and
of all other subprojects) explicitly with the \c{subprojects} variable in
\c{bootstrap.build}. For example, if above we placed \c{libhello} into the
\c{extras/} subdirectory of \c{hello}, then our \c{bootstrap.build} would need
to start like this:

\
project = hello
subprojects = extras/libhello/
...
\

Note also that while importation of specific targets from subprojects is
always performed, whether they are loaded and built as part of the overall
project build is controlled using the standard subdirectories inclusion and
dependency mechanisms. Continuing with the above example, if we adjust the
root \c{buildfile} in \c{hello} to exclude the \c{extras/} subdirectory from
the build:

\
./: {*/ -build/ -extras/}
\

Then while we can still import \c{libhello} from any \c{buildfile} in our
project, the entire \c{libhello} (for example, its tests) will never be built
as part of the \c{hello} build.

Similar to subprojects we can also explicitly specify the project's
amalgamation with the \c{amalgamation} variable (again, in
\c{bootstrap.build}). This is rarely necessary except if you want to prevent
the project from being amalgamated, in which case you should set it to the
empty value.

If either of these variables is not explicitly set, then they will contain
the automatically discovered values.|

Besides affecting importation, another central property of amalgamation is
configuration inheritance. As an example, let's configure the above bundled
\c{hello} project in its src directory:

\
$ b configure: hello/ config.cxx=clang++ config.cxx.coptions=-g

$ b tree
hello/
├── build/
│   ├── config.build
│   └── ...
├── libhello/
│   ├── build/
│   │   ├── config.build
│   │   └── ...
│   └── ...
└── ...
\

As you can see, we now have the \c{config.build} files in both projects'
\c{build/} subdirectories. If we examine the amalgamation's \c{config.build},
we will see the familiar picture:

\
$ cat hello/build/config.build

config.cxx = clang++
config.cxx.poptions = [null]
config.cxx.coptions = -g
config.cxx.loptions = [null]
config.cxx.aoptions = [null]
config.cxx.libs = [null]

...

\

The subproject's \c{config.build}, however, is pretty much empty:

\
$ cat hello/libhello/build/config.build

# Base configuration inherited from ../
\

As the comment suggests, the base configuration is inherited from the outer
project. We can, however, override some values if we need to. For example
(note that we are re-configuring the \c{libhello} subproject):

\
$ b configure: hello/libhello/ config.cxx.coptions=-O2

$ cat hello/libhello/build/config.build

# Base configuration inherited from ../

config.cxx.coptions = -O2
\

This configuration inheritance combined with import resolution is behind the
most common use of amalgamations in \c{build2} \- shared build
configurations. Let's say we are developing multiple projects, for example,
\c{hello} and \c{libhello} that it imports:

\
$ ls -1
hello/
libhello/
\

And we want to build them with several compilers, let's say GCC and Clang. As
we have already seen in \l{#intro-operations-config Configuring}, we can
configure several out of source builds for each compiler, for example:

\
$ b configure: libhello/@libhello-gcc/   config.cxx=g++
$ b configure: libhello/@libhello-clang/ config.cxx=clang++

$ b configure: hello/@hello-gcc/   \
               config.cxx=g++      \
               config.import.libhello=libhello-gcc/
$ b configure: hello/@hello-clang/ \
               config.cxx=clang++  \
               config.import.libhello=libhello-clang/

$ ls -l
hello/
hello-gcc/
hello-clang/
libhello/
libhello-gcc/
libhello-clang/
\

Needless to say, this is a lot of repetitive typing. Another problem is
future changes to the configurations. If, for example, we need to adjust
compile options in the GCC configuration, then we will have to (remember to)
do it in both places.

You can probably sense where this is going: why not create two shared build
configurations (that is, amalgamations), one for GCC and one for Clang, within
each of which we build both of our projects (as their subprojects)? This is
how we can do that:

\
$ b create: build-gcc/,cc   config.cxx=g++
$ b create: build-clang/,cc config.cxx=clang++

$ b configure: libhello/@build-gcc/libhello/
$ b configure: hello/@build-gcc/hello/

$ b configure: libhello/@build-clang/libhello/
$ b configure: hello/@build-clang/hello/

$ ls -l
hello/
libhello/
build-gcc/
build-clang/
\

Let's explain what's going on here. First, we create two build configurations
using the \c{create} meta-operation. These are real \c{build2} projects just
tailored for housing other projects as subprojects. In \c{create}, after the
directory name, we specify the list of modules to load in the project's
\c{root.build}. In our case we specify \c{cc} which is a common module for
C-based languages (see \l{b(1)} for details on \c{create} and its parameters).

\N|When creating build configurations it is a good idea to get into the habit
of using the \c{cc} module instead of \c{c} or \c{cxx} since with more complex
dependency chains we may not know whether every project we build only uses C
or C++. In fact, it is not uncommon for a C++ project to have C implementation
details and even the other way around (yes, really, there are C libraries with
C++ implementations).|

Once the configurations are ready we simply configure our \c{libhello} and
\c{hello} as subprojects in each of them. Note that now we neither need to
specify \c{config.cxx}, because it will be inherited from the amalgamation,
nor \c{config.import.*}, because the import will be automatically resolved to
a subproject.

Now, to build a specific project in a particular configuration we simply build
the corresponding subdirectory. We can also build the entire build
configuration if we want to. For example:

\
$ b build-gcc/hello/

$ b build-clang/
\

\N|In case you've already looked into \l{bpkg(1)} and/or \l{bdep(1)}, their
build configurations are actually these same amalgamations (created underneath
with the \c{create} meta-operation) and their packages are just subprojects.
And with this understanding you are free to interact with them directly using
the build system interface.|


\h#intro-lang|Buildfile Language|

By now we should have a good overall sense of what writing \c{buildfiles}
feels like. In this section we will examine the language in slightly more
detail and with more precision.

Buildfile is primarily a declarative language with support for variables, pure
functions, repetition (\c{for}-loop), conditional inclusion/exclusion
(\c{if-else}), and pattern matching (\c{switch}). At the lexical level,
\c{buildfiles} are UTF-8 encoded text restricted to the Unicode graphic
characters, tabs (\c{\\t}), carriage returns (\c{\\r}), and line feeds
(\c{\\n}).

Buildfile is a line-oriented language. That is, every construct ends at the
end of the line unless escaped with line continuation (trailing \c{\\}). For
example:

\
exe{hello}: {hxx cxx}{**} \\
  $libs
\


Some lines may start a \i{block} if followed by \c{{} on the next line. Such a
block ends with a closing \c{\}} on a separate line. Some types of blocks can
nest. For example:

\
if ($cxx.target.class == 'windows')
{
  if ($cxx.target.system == 'ming32')
  {
    ...
  }
}
\

A comment starts with \c{#} and everything from this character and until the
end of the line is ignored. A multi-line comment starts with \c{#\\} on a
separate line and ends with the same character sequence, again on a separate
line. For example:

\
# Single line comment.

info 'Hello, World!' # Trailing comment.

#\
Multi-
line
comment.
#\
\

The three primary Buildfile constructs are dependency declaration, directive,
and variable assignment. We've already used all three but let's see another
example:

\
include ../libhello/                              # Directive.

exe{hello}: {hxx cxx}{**} ../libhello/lib{hello}  # Dependency.

cxx.poptions += -DNDEBUG                          # Variable.
\

There is also the scope opening (we've seen one in \c{export.build}) as well
as target-specific and prerequisite-specific variable assignment blocks. The
latter two are used to assign several entity-specific variables at once. For
example:

\
details/                          # Scope.
{
  hxx{*}: install = false
}

hxx{version}:                     # Target-specific.
{
  dist  = true
  clean = ($src_root != $out_root)
}

exe{test}: file{test.roundtrip}:  # Prerequisite-specific.
{
  test.stdin  = true
  test.stdout = true
}
\

Variable assignment blocks can be combined with dependency declarations, for
example:

\
h{config}: in{config}
{
  in.symbol = '@'
  in.substitution = lax

  SYSTEM_NAME = $c.target.system
  SYSTEM_PROCESSOR = $c.target.cpu
}
\

In case of a dependency chain, if the chain ends with a colon (\c{:}), then
the block applies to the last set of prerequisites. Otherwise, it applies to
the last set of targets. For example:

\
./: exe{test}: cxx{main}
{
  test = true        # Applies to the exe{test} target.
}

./: exe{test}: libue{test}:
{
  bin.whole = false  # Applies to the libue{test} prerequisite.
}
\

\N|All prerequisite-specific variables must be assigned at once as part of the
dependency declaration since repeating the same dependency again duplicates
the prerequisite rather than references the already existing one.|

There is also the target type/pattern-specific variable assignment block,
for example:

\
exe{*.test}:
{
  test = true
  install = false
}
\

See \l{#variables Variables} for a more detailed discussion of variables.

Each \c{buildfile} is processed linearly with directives executed and
variables expanded as they are encountered. However, certain variables, for
example \c{cxx.poptions}, are also expanded by rules during execution in which
case they will \"see\" the final value set in the \c{buildfile}.

\N|Unlike GNU \c{make(1)}, which has deferred (\c{=}) and immediate (\c{:=})
variable assignments, all assignments in \c{build2} are immediate. For
example:

\
x = x
y = $x
x = X
info $y # Prints 'x', not 'X'.
\

|


\h2#intro-lang-expand|Expansion and Quoting|

While we've discussed variable expansion and lookup earlier, to recap, to get
the variable's value we use \c{$} followed by its name. The variable name is
first looked up in the current scope (that is, the scope in which the
expansion was encountered) and, if not found, in the outer scopes,
recursively.

There are two other kinds of expansions: function calls and evaluation
contexts, or \i{eval contexts} for short. Let's start with the latter since
function calls are built on top of eval contexts.

An eval context is essentially a fragment of a line with additional
interpretations of certain characters to support value comparison, logical
operators, and a few other constructs. Eval contexts begin with \c{(}, end
with \c{)}, and can nest. Here are a few examples:

\
info ($src_root != $out_root)                 # Prints true or false.
info ($src_root == $out_root ? 'in' : 'out')  # Prints in or out.

macos = ($cxx.target.class == 'macos')  # Assigns true or false.
linux = ($cxx.target.class == 'linux')  # Assigns true or false.

if ($macos || $linux)  # Also eval context.
  ...
\

\N|Below is the eval context grammar that shows supported operators and their
precedence.

\
eval:         '(' (eval-comma | eval-qual)? ')'
eval-comma:   eval-ternary (',' eval-ternary)*
eval-ternary: eval-or ('?' eval-ternary ':' eval-ternary)?
eval-or:      eval-and ('||' eval-and)*
eval-and:     eval-comp ('&&' eval-comp)*
eval-comp:    eval-value (('=='|'!='|'<'|'>'|'<='|'>=') eval-value)*
eval-value:   value-attributes? (<value> | eval | '!' eval-value)
eval-qual:    <name> ':' <name>

value-attributes: '[' <key-value-pairs> ']'
\

Note that \c{?:} (ternary operator) and \c{!} (logical not) are
right-associative. Unlike C++, all the comparison operators have the same
precedence. A qualified name cannot be combined with any other operator
(including ternary) unless enclosed in parentheses. The \c{eval} option in the
\c{eval-value} production shall contain a single value only (no commas).

Additionally, the \c{`} (backtick) and \c{|} (bitwise or) tokens are reserved
for future support of arithmetic evaluation contexts and evaluation pipelines,
respectively.|

A function call starts with \c{$} followed by its name and an eval context
listing its arguments. Note that there is no space between the name and
\c{(}. For example:

\
x =
y = Y

info $empty($x)  # true
info $empty($y)  # false

if $regex.match($y, '[A-Z]')
  ...

p = $src_base/foo.txt

info $path.leaf($src_base)              # foo.txt
info $path.directory($src_base)         # $src_base
info $path.base($path.leaf($src_base))  # foo
\

Note that functions in \c{build2} are \i{pure} in a sense that they do not
alter the build state in any way.

\N|Functions in \c{build2} are currently defined either by the build system
core or build system modules and are implemented in C++. In the future it will
be possible to define custom functions in \c{buildfiles} (also in C++).|

Variable and function names follow the C identifier rules. We can also group
variables into namespaces and functions into families by combining multiple
identifiers with \c{.}. These rules are used to determine the end of the
variable name in expansions. If, however, a name is recognized as being longer
than desired, then we can use the eval context to explicitly specify its
boundaries. For example:

\
base = foo
name = $(base).txt
\

What is the structure of a variable value? Consider this assignment:

\
x = foo bar
\

The value of \c{x} could be a string, a list of two strings, or something else
entirely. In \c{build2} the fundamental, untyped value is a \i{list of
names}. A value can be typed to something else later but it always starts as a
list of names. So in the above example we have a list of two names, \c{foo}
and \c{bar}, the same as in this example (notice the extra spaces):

\
x = foo    bar
\

\N|The motivation behind going with a list of names instead of a string or a
list of strings is that at its core we are dealing with targets and their
prerequisites and it would be natural to make the representation of their
names (that is, the way we refer to them) the default. Consider the following
two examples; it would be natural for them to mean the same thing:

\
exe{hello}: {hxx cxx}{**}
\

\
prereqs = {hxx cxx}{**}
exe{hello}: $prereqs
\

Note also that the name semantics was carefully tuned to be \i{reversible} to
its syntactic representation for common non-name values, such as paths,
command line options, etc., that are usually found in \c{buildfiles}.|

To get to individual elements of a list, an expansion can be followed by a
subscript. Note that subscripts are only recognize inside evaluation contexts
and there should be no space between the expansion and \c{[}. For example:

\
x = foo bar

info ($x[0])                                # foo
info ($regex.split('foo bar', ' ', '')[1])  # bar
\

Names are split into a list at whitespace boundaries with certain other
characters treated as syntax rather than as part of the value. Here are
a few examples:

\
x = $y          # expansion
x = (a == b)    # eval context
x = {foo bar}   # name generation
x = [null]      # attributes
x = name@value  # pairs
x = # start of a comment
\

The complete set of syntax characters is \c{$(){\}@#\"'} plus space and tab, as
well as \c{[]}, but only in certain contexts (see \l{#attributes Attributes}
for details). If instead we need these characters to appear literally as part
of the value, then we either have to \i{escape} or \i{quote} them.

\N|Additionally, \c{*?[} will be treated as wildcards in name patterns (see
\l{#name-patterns Name Patterns} for details). Note that this treatment can
only be inhibited with quoting and not escaping.

While name patterns are recognized inside evaluation contexts, in certain
cases the \c{?[} characters are treated as part of the ternary operator and
value subscript, respectively. In such case, to be treat as wildcards rather
than as syntax, these characters have to be escaped, for example:

\
x = (foo.\?xx)
y = ($foo\[123].txt)
\

|

To escape a special character, we prefix it with a backslash (\c{\\}; to
specify a literal backslash, double it). For example:

\
x = \$
y = C:\\\\Program\ Files
\

Similar to UNIX shells, \c{build2} supports single (\c{''}) and double
(\c{\"\"}) quoting with roughly the same semantics. Specifically, expansions
(variable, function call, and eval context) and escaping are performed inside
double-quoted strings but not in single-quoted. Note also that quoted strings
can span multiple lines with newlines treated literally (unless escaped in
double-quoted strings). For example:

\
x = \"(a != b)\"  # true
y = '(a != b)'  # (a != b)

x = \"C:\\\\Program Files\"
y = 'C:\Program Files'

t = 'line one
line two
line three'
\

Since quote characters are also part of the syntax, if you need to specify
them literally in the value, then they will either have to be escaped or
quoted. For example:

\
cxx.poptions += -DOUTPUT='\"debug\"'
cxx.poptions += -DTARGET=\\\"$cxx.target\\\"
\

An expansion can be one of two kinds: \i{spliced} or \i{concatenated}. In a
spliced expansion the variable, function, or eval context is separated from
other text with whitespaces. In this case, as the name suggests, the resulting
list of names is spliced into the value. For example:

\
x = 'foo fox'
y = bar $x baz  # Three names: 'bar' 'foo fox' 'baz'.
\

\N|This is an important difference compared to the semantics of UNIX shells
where the result of expansion is re-parsed. In particular, this is the reason
why you won't see quoted expansions in \c{buildfiles} as often as in
(well-written) shell scripts.|

In a concatenated expansion the variable, function, or eval context are
combined with unseparated text before and/or after the expansion. For example:

\
x = 'foo fox'
y = bar$(x)foz  # Single name: 'barfoo foxbaz'
\

A concatenated expansion is typed unless it is quoted. In a typed concatenated
expansion the parts are combined in a type-aware manner while in an untyped \-
literally, as string. To illustrate the difference, consider this
\c{buildfile} fragment:

\
info $src_root/foo.txt
info \"$src_root/foo.txt\"
\

If we run it on a UNIX-like operating system, we will see two identical
lines, along these lines:

\
/tmp/test/foo.txt
/tmp/test/foo.txt
\

However, if we run it on Windows (which uses backslashes as directory
separators), we will see the output along these lines:

\
C:\test\foo.txt
C:\test/foo.txt
\

The typed concatenation resulted in a native directory separator because
\c{dir_path} (the \c{src_root} type) did the right thing.

Not every typed concatenation is well defined and in certain situations we may
need to force untyped concatenation with quoting. Options specifying header
search paths (\c{-I}) are a typical case, for example:

\
cxx.poptions =+ \"-I$out_root\" \"-I$src_root\"
\

If we were to remove the quotes, we would see the following error:

\
buildfile:6:20: error: no typed concatenation of <untyped> to dir_path
  info: use quoting to force untyped concatenation
\


\h2#intro-if-else|Conditions (\c{if-else})|

The \c{if} directive can be used to conditionally exclude \c{buildfile}
fragments from being processed. The conditional fragment can be a single
(separate) line or a block with the initial \c{if} optionally followed by a
number of \c{elif} directives and a final \c{else}, which together form the
\c{if-else} chain. An \c{if-else} block can contain nested \c{if-else}
chains. For example:

\
if ($cxx.target.class == 'linux')
  info 'linux'
elif ($cxx.target.class == 'windows')
{
  if ($cxx.target.system == 'mingw32')
    info 'windows-mingw'
  elif ($cxx.target.system == 'win32-msvc')
    info 'windows-msvc'
  else
    info 'windows-other'
}
else
  info 'other'
\

The \c{if} and \c{elif} directive names must be followed by an expression that
expands to a single, literal \c{true} or \c{false}. This can be a variable
expansion, a function call, an eval context, or a literal value. For example:

\
if $version.pre_release
  ...

if $regex.match($x, '[A-Z]')
  ...

if ($cxx.target.class == 'linux')
  ...

if false
{
  # disabled fragment
}

x = X
if $x  # Error, must expand to true or false.
  ...
\

There are also \c{if!} and \c{elif!} directives which negate the condition
that follows (note that there is no space before \c{!}). For example:

\
if! $version.pre_release
  ...
elif! $regex.match($x, '[A-Z]')
  ...
\

Note also that there is no notion of variable locality in \c{if-else} blocks
and any value set inside is visible outside. For example:

\
if true
{
  x = X
}

info $x  # Prints 'X'.
\

The \c{if-else} chains should not be used for conditional dependency
declarations since this would violate the expectation that all of the
project's source files are listed as prerequisites, irrespective of the
configuration.  Instead, use the special \c{include} prerequisite-specific
variable to conditionally include prerequisites into the build. For example:

\
# Incorrect.
#
if ($cxx.target.class == 'linux')
  exe{hello}: cxx{hello-linux}
elif ($cxx.target.class == 'windows')
  exe{hello}: cxx{hello-win32}

# Correct.
#
exe{hello}: cxx{hello-linux}: include = ($cxx.target.class == 'linux')
exe{hello}: cxx{hello-win32}: include = ($cxx.target.class == 'windows')
\


\h2#intro-switch|Pattern Matching (\c{switch})|

The \c{switch} directive is similar to \c{if-else} in that it allows us to
conditionally exclude \c{buildfile} fragments from being processed. The
difference is in the way the conditions are structured: while in \c{if-else}
we can do arbitrary tests, in \c{switch} we match one or more values against a
set of patterns. For instance, this is how we can reimplement the first
example from \l{#intro-if-else Conditionals (\c{if-else})} using \c{switch}:

\
switch $cxx.target.class, $cxx.target.system
{
  case 'linux'
    info 'linux'

  case 'windows', 'mingw32'
    info 'windows-mingw'

  case 'windows', 'win32-msvc'
    info 'windows-msvc'

  case 'windows'
    info 'windows-other'

  default
    info 'other'
}
\

Similar to \c{if-else}, the conditional fragment can be a single (separate)
line or a block with a zero or more \c{case} lines/blocks optionally followed
by \c{default}. A \c{case-default} block can contain nested \c{switch}
directives (though it is often more convenient to use multiple values
in a single \c{switch}, as shown above). For example:

\
switch $cxx.target.class
{
  ...
  case 'windows'
  {
    switch $cxx.target.system
    {
      case 'mingw32'
        info 'windows-mingw'

      case 'win32-msvc'
        info 'windows-msvc'

      default
        info 'windows-other'
    }
  }
  ...
}
\

All the \c{case} fragments are tried in the order specified with the first
that matches evaluated and all the others ignored (that is, there is no
explicit \c{break} nor the ability to fall through). If none of the \c{case}
patterns matched and there is the \c{default} fragment, then it is evaluated.
Multiple \c{case} lines can be specified for a single conditional fragment.
For example:

\
switch $cxx.target.class, $cxx.id
{
  case 'windows', 'msvc'
  case 'windows', 'clang'
    info 'msvcrt'
}
\

The \c{switch} directive name must be followed by one or more \i{value
expressions} separated with a comma (\c{,}). Similarly, the \c{case} directive
name must be followed by one or more \i{pattern expressions} separated with a
comma (\c{,}). These expressions can be variable expansions, function calls,
eval contexts, or literal values.

If multiple values/patterns are specified, then all the \c{case} patterns must
match in order for the corresponding fragment to be evaluated. However, if
some trailing patterns are omitted, then they are considered as matching. For
example:

\
switch $cxx.target.class, $cxx.target.system
{
  case 'windows', 'mingw32'
    info 'windows-mingw'

  case 'windows', 'win32-msvc'
    info 'windows-msvc'

  case 'windows'
    info 'windows-other'
}
\

The first pattern in the pattern expression can be optionally followed by one
or more alternative patterns separated by a vertical bar (\c{|}). Only one of
the alternatives need to match in order for the whole pattern expression to be
considered as matching. For example:

\
switch $cxx.id
{
  case 'clang' | 'clang-apple'
    ...
}
\

The value in the value expression can be optionally followed by a colon
(\c{:}) and a \i{match function}. If the match function is not specified, then
equality is used by default. For example:

\
switch $cxx.target.cpu: regex.match
{
  case 'i[3-6]86'
    ...

  case 'x86_64'
    ...
}
\

The match function name can be optionally followed by additional values that
are passed as the third argument to the match function. This is normally used
to specify additional match flags, for example:

\
switch $cxx.target.cpu: regex.match icase
{
  ...
}
\

Other commonly used match functions are \c{regex.search()} (similar to
\c{regex.match()} but searches for any match rather than matching the whole
value), \c{path.match()} (match using shell wildcard patterns) and
\c{string.icasecmp()} (match using equality but ignoring case). Additionally,
any other function that takes the value as its first argument, the pattern as
its second, and returns \c{bool} can be used as a match function.

\N|Note that there is no special wildcard or match-anything pattern at the
syntax level. In most common cases the desired semantics can be achieved with
\c{default} and/or by omitting trailing patterns. If you do need it, then we
recommend using \c{path.match()} and its \c{*} wildcard. For example:

\
switch $cxx.target.class: path.match, \
       $cxx.target.system: path.match, \
       $cxx.id: path.match
{
  case 'windows', '*', 'clang'
    ...
}
\

|

Note also that similar to \c{if-else}, there is no notion of variable locality
in the \c{switch} and \c{case-default} blocks and any value set inside is
visible outside. Additionally, the same considerations about conditional
dependency declarations apply.


\h2#intro-for|Repetitions (\c{for})|

The \c{for} directive can be used to repeat the same \c{buildfile} fragment
multiple times, once for each element of a list. The fragment to repeat can be
a single (separate) line or a block, which together form the \c{for} loop. A
\c{for} block can contain nested \c{for} loops. For example:

\
for n: foo bar baz
{
  exe{$n}: cxx{$n}
}
\

The \c{for} directive name must be followed by the variable name (called
\i{loop variable}) that on each iteration will be assigned the corresponding
element, \c{:}, and an expression that expands to a potentially empty list of
values. This can be a variable expansion, a function call, an eval context, or
a literal list as in the above fragment. Here is a somewhat more realistic
example that splits a space-separated environment variable value into names
and then generates a dependency declaration for each of them:

\
for n: $regex.split($getenv(NAMES), ' +', '')
{
  exe{$n}: cxx{$n}
}
\

Note also that there is no notion of variable locality in \c{for} blocks and
any value set inside is visible outside. At the end of the iteration the loop
variable contains the value of the last element, if any. For example:

\
for x: x X
{
  y = Y
}

info $x  # Prints 'X'.
info $y  # Prints 'Y'.
\


\h#intro-unit-test|Implementing Unit Testing|

As an example of how many of these features fit together to implement more
advanced functionality, let's examine a \c{buildfile} that provides support
for unit testing. This support is added by the \l{bdep-new(1)} command if we
specify the \c{unit-tests} option when creating executable (\c{-t\
exe,unit-tests}) or library (\c{-t\ lib,unit-tests}) projects. Here is the
source subdirectory \c{buildfile} of an executable created with this option:

\
./: exe{hello}: libue{hello}: {hxx cxx}{** -**.test...}

# Unit tests.
#
exe{*.test}
{
  test = true
  install = false
}

for t: cxx{**.test...}
{
  d = $directory($t)
  n = $name($t)...

  ./: $d/exe{$n}: $t $d/hxx{+$n} $d/testscript{+$n}
  $d/exe{$n}: libue{hello}: bin.whole = false
}

cxx.poptions =+ \"-I$out_root\" \"-I$src_root\"
\

The basic idea behind this unit testing arrangement is to keep unit tests next
to the source code files that they test and automatically recognize and build
them into test executables without having to manually list each in the
\c{buildfile}. Specifically, if we have \c{hello.hxx} and \c{hello.cxx},
then to add a unit test for this module all we have to do is drop the
\c{hello.test.cxx} source file next to them and it will be automatically
picked up, built into an executable, and run during the \c{test} operation.

As an example, let's say we've renamed \c{hello.cxx} to \c{main.cxx} and
factored the printing code into the \c{hello.hxx/hello.cxx} module that we
would like to unit-test. Here is the new layout:

\
hello/
├── build
│   └── ...
├── hello
│   ├── hello.cxx
│   ├── hello.hxx
│   ├── hello.test.cxx
│   ├── main.cxx
│   └── buildfile
└── ...
\

Let's examine how this support is implemented in our \c{buildfile}, line by
line. Because now we link \c{hello.cxx} object code into multiple executables
(unit tests and the \c{hello} program itself), we have to place it into a
\i{utility library}. This is what the first line does (it has to explicitly
list \c{exe{hello\}} as a prerequisite of the default targets since we now
have multiple targets that should be built by default):

\
./: exe{hello}: libue{hello}: {hxx cxx}{** -**.test...}
\

A utility library (\cb{u} in \c{lib\b{u}e}) is a static library that is built
for a specific type of a \i{primary target} (\cb{e} in \c{libu\b{e}} for
executable). If we were building a utility library for a library then we would
have used the \c{libul{\}} target type instead. In fact, this would be the
only difference in the above unit testing implementation if it were for a
library project instead of an executable:

\
./: lib{hello}: libul{hello}: {hxx cxx}{** -**.test...}

...

# Unit tests.
#
...

for t: cxx{**.test...}
{
  ...

  $d/exe{$n}: libul{hello}: bin.whole = false
}
\

Going back to the first three lines of the executable \c{buildfile}, notice
that we had to exclude source files in the \c{*.test.cxx} form from the
utility library. This makes sense since we don't want unit testing code (each
with its own \c{main()}) to end up in the utility library.

The exclusion pattern, \c{-**.test...}, looks a bit cryptic. What we have here
is a second-level extension (\c{.test}) which we use to classify our source
files as belonging to unit tests. Because it is a second-level extension, we
have to indicate this fact to the pattern matching machinery with the trailing
triple dot (meaning \"there are more extensions coming\"). If we didn't do
that, \c{.test} would have been treated as a first-level extension explicitly
specified for our source files.

\N|If you need to specify a name that does not have an extension, then end it
with a single dot. For example, for a header \c{utility} you would write
\c{hxx{utility.\}}. If you need to specify a name with an actual trailing
dot, then escape it with a double dot, for example, \c{hxx{utility..\}}.|

The next couple of lines set target type/pattern-specific variables to treat
all unit test executables as tests that should not be installed:

\
exe{*.test}:
{
  test = true
  install = false
}
\

\N|You may be wondering why we had to escape the second-level \c{.test}
extension in the name pattern above but not here. The answer is that these are
different kinds of patterns in different contexts. In particular, patterns in
the target type/pattern-specific variables are only matched against target
names without regard for extensions. See \l{#name-patterns Name Patterns} for
details.|

Then we have the \c{for}-loop that declares an executable target for each unit
test source file. The list of these files is generated with a name pattern
that is the inverse of what we've used for the utility library:

\
for t: cxx{**.test...}
{
  d = $directory($t)
  n = $name($t)...

  ./: $d/exe{$n}: $t $d/hxx{+$n} $d/testscript{+$n}
  $d/exe{$n}: libue{hello}: bin.whole = false
}
\

In the loop body we first split the test source file into the directory
(remember, we can have sources, including tests, in subdirectories) and name
(which contains the \c{.test} second-level extension and which we immediately
escape with \c{...}). And then we use these components to declare a dependency
for the corresponding unit test executable. There is nothing here that we
haven't already seen except for using variable expansions instead of literal
names.

By default utility libraries are linked in the \"whole archive\" mode where
every object file from the static library ends up in the resulting executable
or library. This behavior is what we want when linking the primary target but
can normally be relaxed for unit tests to speed up linking. This is what the
last line in the loop does using the \c{bin.whole} prerequisite-specific
variable.

\N|You can easily customize this and other aspects on a test-by-test basis
by excluding the specific test(s) from the loop and then providing a custom
implementation. For example:

\
for t: cxx{**.test... -special.test...}
{
  ...
}

./: exe{special.test...}: cxx{special.test...} libue{hello}
\

Note also that if you plan to link any of your unit tests in the whole archive
mode, then you will also need to exclude the source file containing the
primary executable's \c{main()} from the utility library. For example:

\
./: exe{hello}: cxx{main} libue{hello}
libue{hello}: {hxx cxx}{** -main -**.test...}
\

|



\h#intro-diag-debug|Diagnostics and Debugging|

Sooner or later we will run into a situation where our \c{buildfiles} don't do
what we expect them to. In this section we examine a number of techniques and
mechanisms that can help us understand the cause of a misbehaving build.

To perform a build the build system goes through several phases. During the
\i{load} phase the \c{buildfiles} are loaded and processed. The result of this
phase is the in-memory \i{build state} that contains the scopes, targets,
variables, etc., defined by the \c{buildfiles}. Next is the \i{match} phase
during which rules are matched to the targets that need to be built,
recursively. Finally, during the \i{execute} phase the matched rules are
executed to perform the build.

The load phase is always serial and stops at the first error. In contrast, by
default, both match and execute are parallel and continue in the presence of
errors (similar to the \"keep going\" \c{make} mode). While beneficial in
normal circumstances, during debugging this can lead to both interleaved
output that is hard to correlate as well as extra noise from cascading
errors. As a result, for debugging, it is usually helpful to run serially and
stop at the first error, which can be achieved with the \c{--serial-stop|-s}
option.

\N|The match phase can be temporarily switched to either (serial) load or
(parallel) execute. The former is used, for example, to load additional
\c{buildfiles} during the \c{dir{\}} prerequisite to target resolution, as
described in \l{#intro-dirs-scopes Output Directories and Scopes}. While the
latter is used to update generated source code (such as headers) that is
required to complete the match.|

Debugging issues in each phase requires different techniques. Let's start with
the load phase. As mentioned in \l{#intro-lang Build Language}, \c{buildfiles}
are processed linearly with directives executed and variables expanded as they
are encountered. As we have already seen, to print a variable value we can use
the \c{info} directive. For example:

\
x = X
info $x
\

This will print something along these lines:

\
buildfile:2:1: info: X
\

Or, if we want to clearly see where the value begins and ends (useful when
investigating whitespace-related issues):

\
x = \" X \"
info \"'$x'\"
\

Which prints:

\
buildfile:2:1: info: ' X '
\

Besides the \c{info} directive, there are also \c{text}, which doesn't print
the \c{info:} prefix, \c{warn}, which prints a warning, as well as \c{fail}
which prints an error and causes the build system to exit with an error. Here
is an example of using each:

\
text 'note: we are about to get an error'
warn 'the error is imminent'
fail 'this is the end'
info 'we will never get here'
\

This will produce the following output:

\
buildfile:1:1: note: we are about to get an error
buildfile:2:1: warning: the error is imminent
buildfile:3:1: error: this is the end
\

If you find yourself writing code like this:

\
if ($cxx.target.class == 'windows')
  fail 'Windows is not supported'
\

Then the \c{assert} directive is a more concise way to express the same:

\
assert ($cxx.target.class != 'windows') 'Windows is not supported'
\

The assert condition must be an expression that evaluates to \c{true} or
\c{false}, similar to the \c{if} directive (see \l{#intro-if-else Conditions
(\c{if-else})} for details). The description after the condition is optional
and, similar to \c{if}, there is also the \c{assert!} variant, which fails if
the condition is \c{true}.

All the diagnostics directives write to \c{stderr}. If instead we need to
write something to \c{stdout} to, for example, send some information back to
our caller, then we can use the \c{print} directive. For example, this will
print the C++ compiler id and its target:

\
print \"$cxx.id $cxx.target\"
\

\N|To query the value of a target-specific variable we use the qualified name
syntax (the \c{eval-qual} production) of eval context, for example:

\
obj{main}: cxx.poptions += -DMAIN
info $(obj{main}: cxx.poptions)
\

There is no direct way to query the value of a prerequisite-specific variable
since a prerequisite has no identity. Instead, we can use the \c{dump}
directive discussed next to print the entire dependency declaration, including
prerequisite-specific variables for each prerequisite.|

While printing variable values is the most common mechanism for diagnosing
\c{buildfile} issues, sometimes it is also helpful to examine targets and
scopes. For that we use the \c{dump} directive.

Without any arguments, \c{dump} prints (to \c{stderr}) the contents of the
scope it was encountered in and at that point of processing the \c{buildfile}.
Its output includes variables, targets and their prerequisites, as well as
nested scopes, recursively. As an example, let's print the source directory
scope of our \c{hello} executable project. Here is its \c{buildfile} with
the \c{dump} directive at the end:

\
exe{hello}: {hxx cxx}{**}

cxx.poptions =+ \"-I$out_root\" \"-I$src_root\"

dump
\

This will produce the output along these lines:

\
buildfile:5:1: dump:
  /tmp/hello/hello/
  {
    [strings] cxx.poptions = -I/tmp/hello -I/tmp/hello
    [dir_path] out_base = /tmp/hello/hello/
    [dir_path] src_base = /tmp/hello/hello/

    buildfile{buildfile.}:

    exe{hello.?}: cxx{hello.?}
  }
\

\N|The question marks (\c{?}) in the dependency declaration mean that the file
extensions haven't been assigned yet, which happens during the match phase.|

Instead of printing the entire scope, we can also print individual targets by
specifying one or more target names in \c{dump}. To make things more
interesting, let's convert our \c{hello} project to use a utility library,
similar to the unit testing setup (\l{#intro-unit-test Implementing Unit
Testing}). We will also link to the \c{pthread} library to see an example of a
target-specific variable being dumped:

\
exe{hello}: libue{hello}: bin.whole = false
exe{hello}: cxx.libs += -lpthread
libue{hello}: {hxx cxx}{**}

dump exe{hello}
\

The output will look along these lines:

\
buildfile:5:1: dump:
  /tmp/hello/hello/exe{hello.?}:
  {
    [strings] cxx.libs = -lpthread
  }
  /tmp/hello/hello/exe{hello.?}: /tmp/hello/hello/:libue{hello.?}:
  {
    [bool] bin.whole = false
  }
\

The output of \c{dump} might look familiar: in \l{#intro-dirs-scopes Output
Directories and Scopes} we've used the \c{--dump} option to print the entire
build state, which looks pretty similar. In fact, the \c{dump} directive uses
the same mechanism but allows us to print individual scopes and targets.

There is, however, an important difference to keep in mind: \c{dump} prints
the state of a target or scope at the point in the \c{buildfile} load phase
where it was executed. In contrast, the \c{--dump} option can be used to print
the state after the load phase (\c{--dump load}) and/or after the match phase
(\c{--dump match}). In particular, the after match printout reflects the
changes to the build state made by the matching rules, which may include
entering of additional dependencies, setting of additional variables,
resolution of prerequisites to targets, assignment of file extensions, etc. As
a result, while the \c{dump} directive should be sufficient in most cases,
sometimes you may need to use the \c{--dump} option to examine the build state
just before rule execution.

Let's now move from state to behavior. As we already know, to see the
underlying commands executed by the build system we use the \c{-v} options
(which is equivalent to \c{--verbose\ 2}). Note, however, that these are
\i{logical} rather than actual commands. You can still run them and they
should produce the desired result, but in reality the build system may have
achieved the same result in a different way. To see the actual commands we use
the \c{-V} option instead (equivalent to \c{--verbose\ 3}). Let's see the
difference in an example. Here is what building our \c{hello} executable
with \c{-v} might look like:

\
$ b -s -v
g++ -o hello.o -c hello.cxx
g++ -o hello hello.o
\

And here is the same build with \c{-V}:

\
$ b -s -V
g++ -MD -E -fdirectives-only -MF hello.o.t -o hello.o.ii hello.cxx
g++ -E -fpreprocessed -fdirectives-only hello.o.ii
g++ -o hello.o -c -fdirectives-only hello.o.ii
g++ -o hello hello.o
\

From the second listing we can see that in reality \c{build2} first partially
preprocessed \c{hello.cxx} while extracting its header dependency information.
It then preprocessed it fully, which is used to extract module dependency
information, calculate the checksum for ignorable change detection, etc.  When
it comes to producing \c{hello.o}, the build system compiled the partially
preprocessed output rather than the original \c{hello.cxx}. The end result,
however, is the same as in the first listing.

Verbosity level \c{3} (\c{-V}) also triggers printing of the build system
module configuration information. Here is what we would see for the \c{cxx}
module:

\
cxx hello@/tmp/hello/
  cxx        g++@/usr/bin/g++
  id         gcc
  version    7.2.0 (Ubuntu 7.2.0-1ubuntu1~16.04)
  major      7
  minor      2
  patch      0
  build      (Ubuntu 7.2.0-1ubuntu1~16.04)
  signature  gcc version 7.2.0 (Ubuntu 7.2.0-1ubuntu1~16.04)
  checksum   09b3b59d337eb9a760dd028fa0df585b307e6a49c2bfa00b3[...]
  target     x86_64-linux-gnu
  runtime    libgcc
  stdlib     libstdc++
  c stdlib   glibc
...
\

Verbosity levels higher than \c{3} enable build system tracing. In particular,
level \c{4} is useful for understanding why a rule doesn't match a target or
if it does, why it determined the target to be out of date. For example,
assuming we have an up-to-date build of our \c{hello}, let's change a compile
option:

\
$ b -s --verbose 4
info: /tmp/hello/dir{hello/} is up to date

$ b -s --verbose 4 config.cxx.poptions+=-DNDEBUG
trace: cxx::compile_rule::apply: options mismatch forcing update
of /tmp/hello/hello/obje{hello.o}
...
\

Higher verbosity levels result in more and more tracing statements being
printed. These include \c{buildfile} loading and parsing, prerequisite to
target resolution, as well as build system module and rule-specific logic.

Another useful diagnostics option is \c{--mtime-check}. When specified, the
build system performs a number of file modification time sanity checks that
can be helpful in diagnosing spurious rebuilds.

If neither state dumps nor behavior analysis are sufficient to understand the
problem, there is always an option of running the build system under a C++
debugger in order to better understand what's going on. This can be
particularly productive for debugging complex rules.

Finally, to help with diagnosing the build system performance issues, there is
the \c{--stat} option. It causes \c{build2} to print various execution
statistics which can be useful for pin-pointing the bottlenecks. There are
also a number of options for tuning the build system's performance, such as,
the number of jobs to perform in parallel, the stack size, queue depths, etc.
See the \l{b(1)} man pages for details.


\h1#proj-config|Project Configuration|

As discussed in the introduction (specifically, \l{#intro-proj-struct Project
Structure}) support for build configurations is an integral part of \c{build2}
with the same mechanism used by the build system core (for example, for
project importation via the \c{config.import.*} variables), by the build
system modules (for example, for supplying compile options such as
\c{config.cxx.coptions}), as well as by our projects to provide any
project-specific configurability. Project configuration is the topic of this
chapter.

\N|The \c{build2} build system currently provides no support for
\c{autoconf}-style probing of the build environment in order to automatically
discover available libraries, functions, features, etc.

The main reason for omitting this support is the fundamental ambiguity and the
resulting brittleness of such probing due to the reliance on compiler, linker,
or test execution failures. Specifically, in many such tests it is impossible
for a build system to distinguish between a missing feature, a broken test,
and a misconfigured build environment. This leads to requiring a user
intervention in the best case and to a silently misconfigured build in the
worst. Other issues with this approach include portability, speed (compiling
and linking takes time), as well as limited applicability during
cross-compilation (specifically, inability to run tests).

As a result, we recommend using \i{expectation-based} configuration where your
project assumes a feature to be available if certain conditions are
met. Examples of such conditions at the source code level include feature
test macros, platform macros, runtime library macros, compiler macros, etc.,
with the build system modules exposing some of the same information via
variables to allow making similar decisions in \c{buildfiles}. Another
alternative is to automatically adapt to missing features using more advanced
techniques such as C++ SFINAE. And in situations where none of this is
possible, we recommend delegating the decision to the user via a configuration
value.  Our experience with \c{build2} as well as those of other large
cross-platform projects such as Boost show that this is a viable strategy.

Having said that, \c{build2} does provide the ability to extract configuration
information from the environment (\c{$getenv()} function) or other tools
(\c{$process.run*()} family of functions). Note, however, that for this to
work reliably there should be no ambiguity between the \"no configuration
available\" case (if such a case is possible) and the \"something went wrong\"
case. We show a realistic example of this in \l{#proj-config-report
Configuration Report} where we extract the GCC plugin directory while dealing
with the possibility of it being configured without plugin support.|

Before we delve into the technical details, let's discuss the overall need for
project configurability. While it may seem that making one's project more
user-configurable is always a good idea, there are costs: by having a choice
we increase the complexity and open the door for potential incompatibility.
Specifically, we may end up with two projects in the same build needing a
shared dependency with incompatible configurations.

\N|While some languages, such as Rust, support having multiple
differently-configured projects in the same build, this is not something that
is done often in C/C++. This ability is also not without its drawbacks, most
notably code bloat.|

As a result, our recommendation is to strive for simplicity and avoid user
configurability whenever possible. For example, there is a common desire to
make certain functionality optional in order not to make the user pay for
things they don't need. This, however, is often better addressed either by
always providing the optional functionality if it's fairly small or by
factoring it into a separate project if it's substantial. If a configuration
value is to be provided, it should have a sensible default with a bias for
simplicity and compatibility rather than the optimal result. For example, in
the optional functionality case, the default should probably be to provide it.

As discussed in the introduction, the central part of the build configuration
functionality are the \i{configuration variables}. One of the key features
that make them special is support for automatic persistence in the
\c{build/config.build} file provided by the \l{#module-config \c{config}}
module (see \l{#intro-operations-config Configuring} for details).

\N|Another mechanism that can be used for project configuration is environment
variables. While not recommended, sometimes it may be forced on us by external
factors. In such cases, environment variables that affect the build result
should be reported with the \c{config.environment} directive as discussed in
\l{#module-config-hermetic Hermetic Build Configurations}.|

The following example, based on the \c{libhello} project from the introduction,
gives an overview of the project configuration functionality with the
remainder of the chapter providing the detailed explanation of all the parts
shown as well as the alternative approaches.

\
libhello/
├── build/
│   ├── root.build
│   └── ...
├── libhello/
│   ├── hello.cxx
│   ├── buildfile
│   └── ...
└── ...
\

\
# build/root.build

config [string] config.libhello.greeting ?= 'Hello'
\

\
# libhello/buildfile

cxx.poptions += \"-DLIBHELLO_GREETING=\\\"$config.libhello.greeting\\\"\"
\

\
// libhello/hello.cxx

void say_hello (ostream& o, const string& n)
{
  o << LIBHELLO_GREETING \", \" << n << '!' << endl;
}
\

\
$ b configure config.libhello.greeting=Hi -v
config libhello@/tmp/libhello/
  greeting   Hi

$ cat build/config.build
config.libhello.greeting = Hi

$ b -v
g++ ... -DLIBHELLO_GREETING=\"Hi\" ...
\

By (enforced) convention, configuration variables start with \c{config.}, for
example, \c{config.import.libhello}. In case of a build system module, the
second component in its configuration variables should be the module name, for
example, \c{config.cxx}, \c{config.cxx.coptions}. Similarly, project-specific
configuration variables should have the project name as their second
component, for example, \c{config.libhello.greeting}.

\N|More precisely, a project configuration variable must match the
\c{config[.**].<project>.**} pattern where additional components may be
present after \c{config.} in case of subprojects. Overall, the recommendation
is to use hierarchical names, such as \c{config.libcurl.tests.remote} for
subprojects, similar to build system submodules.

If a build system module for a tool (such as a source code generator) and the
tool itself share a name, then they may need to coordinate their configuration
variable names in order to avoid clashes. Note also that when importing an
executable target in the \c{<project>%exe{<project>\}} form, the
\c{config.<project>} variable is treated as an alias for
\c{config.import.<project>.<project>.exe}.

The build system core reserves \c{build} and \c{import} as the second
component in configuration variables as well as \c{configured} as the third
and subsequent components.|

A variable in the \c{config.<project>.develop} form has pre-defined
semantics: it allows a project to distinguish between \i{development} and
\i{consumption} builds. While normally there is no distinction between these
two modes, sometimes a project may need to provide additional functionality
during development. For example, a source code generator which uses its own
generated code in its implementation may need to provide a bootstrap step from
the pre-generated code. Normally, such a step is only needed during
development.

\N|While some communities, such as Rust, believe that building and running
tests is only done during development, we believe its reasonable for an
end-user to want to run tests for all their dependencies. As a result, we
strongly discourage restricting tests to the development mode only. Test are
an integral part of the project and should always be available.|

If used, the \c{config.<project>.develop} variable should be explicitly
defined by the project with the \c{bool} type and the \c{false} default
value. For example:

\
# build/root.build

config [bool] config.libhello.develop ?= false
\

\N|If the \c{config.<project>.develop} variable is specified by the user of
the project but the project does not define it (that is, the project does not
distinguish between development and consumption), then this variable is
silently ignored. By default \l{bdep-init(1)} configures projects being
initialized for development. This can be overridden with explicit
\c{config.<project>.develop=false}.|



\h#proj-config-directive|\c{config} Directive|

To define a project configuration variable we add the \c{config} directive
into the project's \c{build/root.build} file (see \l{#intro-proj-struct
Project Structure}). For example:


\
config [bool]   config.libhello.fancy    ?= false
config [string] config.libhello.greeting ?= 'Hello'
\

\N|The irony does not escape us: these configuration variables are exactly of
the kind that we advocate against. However, finding a reasonable example of
build-time configurability in a \i{\"Hello, World!\"} library is not easy. In
fact, it probably shouldn't have any. So, for this chapter, do as we say, not
as we do.|

Similar to \c{import} (see \l{#intro-import Target Importation}), the
\c{config} directive is a special kind of variable assignment. Let's examine
all its parts in turn.

First comes the optional list of variable attributes inside \c{[\ ]}. The only
attribute that we have in the above example is the variable type, \c{bool} and
\c{string}, respectively. It is generally a good idea to assign static types
to configuration variables because their values will be specified by the users
of our project and the more automatic validation we provide the better (see
\l{#variables Variables} for the list of available types). For example, this
is what will happen if we misspell the value of the \c{fancy} variable:

\
$ b configure config.libhello.fancy=fals
error: invalid bool value 'fals' in variable config.libhello.fancy
\

After the attribute list we have the variable name. The \c{config} directive
will validate that it matches the \c{config[.**].<project>.**} pattern (with
one exception discussed in \l{#proj-config-report Configuration Report}).

Finally, after the variable name comes the optional default value. Note that
unlike normal variables, the default value assignment (\c{?=}) is the only
valid form of assignment in the \c{config} directive.

The semantics of the \c{config} directive is as follows: First an overridable
variable is entered with the specified name, type (if any), and global
visibility. Then, if the variable is undefined and the default value is
specified, it is assigned the default value. After this, if the variable is
defined (either as user-defined or default), it is marked for persistence.
Finally, a defined variable is also marked for reporting as discussed in
\l{#proj-config-report Configuration Report}. Note that if the variable
is user-defined, then the default value is not evaluated.

Note also that if the configuration value is not specified by the user and you
haven't provided the default, the variable will be undefined, not \c{null},
and, as a result, omitted from the persistent configuration
(\c{build/config.build} file). However, \c{null} is a valid default value. It
is traditionally used for \i{optional} configuration values. For example:

\
config [string] config.libhello.fallback_name ?= [null]
\

A common approach for representing an C/C++ enum-like value is to use
\c{string} as a type and pattern matching for validation. In fact, validation
and propagation can often be combined. For example, if our library needed to
use a database for some reason, we could handle it like this:

\
config [string] config.libhello.database ?= [null]

using cxx

switch $config.libhello.database
{
  case [null]
  {
    # No database in use.
  }
  case 'sqlite'
  {
    cxx.poptions += -DLIBHELLO_WITH_SQLITE
  }
  case 'pgsql'
  {
    cxx.poptions += -DLIBHELLO_WITH_PGSQL
  }
  default
  {
    fail \"invalid config.libhello.database value \
'$config.libhello.database'\"
  }
}
\

While it is generally a good idea to provide a sensible default for all your
configuration variables, if you need to force the user to specify its value
explicitly, this can be achieved with an extra check. For example:

\
config [string] config.libhello.database

if! $defined(config.libhello.database)
  fail 'config.libhello.database must be specified'
\

And if you want to also disallow \c{null} values, then the above check should
be rewritten like this: \N{An undefined variable expands into a \c{null}
value.}

\
if ($config.libhello.database == [null])
  fail 'config.libhello.database must be specified'
\

If computing the default value is expensive or requires elaborate logic, then
the handling of a configuration variable can be broken down into two steps
along these lines:

\
config [string] config.libhello.greeting

if! $defined(config.libhello.greeting)
{
  greeting = ... # Calculate default value.

  if ($greeting == [null])
    fail \"unable to calculate default greeting, specify manually \
with config.libhello.greeting\"

  config config.libhello.greeting ?= $greeting
}
\

Other than assigning the default value via the \c{config} directive,
configuration variables should not be modified by the project's
\c{buildfiles}. Instead, if further processing of the configuration value is
necessary, we should assign the configuration value to a different,
non-\c{config.*}, variable and modify that. The two situations where this is
commonly required are post-processing of configuration values to be more
suitable for use in \c{buildfiles} as well as further customization of
configuration values. Let's see examples of both.

To illustrate the first situation, let's say we need to translate the database
identifiers specified by the user:

\
config [string] config.libhello.database ?= [null]

switch $config.libhello.database
{
  case [null]
    database = [null]

  case 'sqlite'
    database = 'SQLITE'

  case 'pgsql'
    database = 'PGSQL'

  case 'mysql'
  case 'mariadb'
    database = 'MYSQL'

  default
    fail \"...\"
  }
}

using cxx

if ($database != [null])
  cxx.poptions += \"-DLIBHELLO_WITH_$database\"
\

For the second situation, the typical pattern looks like this:

\
config [strings] config.libhello.options

options  = # Overridable options go here.
options += $config.libhello.options
options += # Non-overridable options go here.
\

That is, assuming that the subsequently specified options (for example,
command line options) override any previously specified, we first set default
\c{buildfile} options that are allowed to be overridden by options from the
configuration value, then append such options, if any, and finish off by
appending \c{buildfile} options that should always be in effect.

As a concrete example of this approach, let's say we want to make the compiler
warning level of our project configurable (likely a bad idea; also ignores
compiler differences):

\
config [strings] config.libhello.woptions

woptions  = -Wall -Wextra
woptions += $config.libhello.woptions
woptions += -Werror

using cxx

cxx.coptions += $woptions
\

With this arrangement, the users of our project can customize the warning
level but cannot disable the treatment of warnings as errors. For example:

\
$ b -v config.libhello.woptions=-Wno-extra
g++ ... -Wall -Wextra -Wno-extra -Werror ...
\

While we have already seen some examples of how to propagate the configuration
values to our source code, \l{#proj-config-propag Configuration Propagation}
discusses this topic in more detail.


\h#proj-config-report|Configuration Report|

One of the effects of the \c{config} directive is to mark a defined
configuration variable for reporting. The project configuration report is
printed automatically at a sufficiently high verbosity level along with the
build system module configuration. For example (some of the \c{cxx} module
configuration is omitted for brevity):

\
$ b config.libhello.greeting=Hey -v
cxx libhello@/tmp/libhello/
  cxx        g++@/usr/bin/g++
  id         gcc
  version    9.1.0
  ...
config libhello@/tmp/libhello/
  fancy      false
  greeting   Hey
\

\N|The configuration report is printed immediately after loading the project's
\c{build/root.build} file. It is always printed at verbosity level \c{3}
(\c{-V}) or higher. It is also printed at verbosity level \c{2} (\c{-v}) if
any of the reported configuration variables have a \i{new} value. A value is
considered new if it was set to default or was overridden on the command
line.|

The project configuration report header (the first line) starts with the
special \c{config} module name (the \c{config} module itself does not have a
report) followed by the project name and its \c{out_root} path. After the
header come configuration variables with the \c{config[.**].<project>} prefix
removed. The configuration report for each variable can be customized using a
number of \c{config.report*} attributes as discussed next.

The \c{config.report} attribute controls whether the variable is included into
the report and, if so, the format to print its value in. For example, this is
how we can exclude a variable from the report:

\
config [bool, config.report=false] config.libhello.selftest ?= false
\

While we would normally want to report all our configuration variables , if
some of them are internal and not meant to be used by the users of our
project, it probably makes sense to exclude them.

The only currently supported alternative printing format is \c{multiline}
which prints a list value one element per line. \N{Other printing formats may
be supported in the future.} For example:

\
config [dir_paths, config.report=multiline] config.libhello.search_dirs
\

\
$ b config.libhello.search_dirs=\"/etc/default /etc\" -v
config libhello@/tmp/libhello/
  search_dirs
    /etc/default/
    /etc/
\

The \c{config.report} attribute can also be used to include a non-\c{config.*}
variable into a report. This is primarily useful for configuration values
that are always discovered automatically but that are still useful to report
for troubleshooting. Here is a realistic example:

\
using cxx

# Determine the GCC plugin directory.
#
if ($cxx.id == 'gcc')
{
  plugin_dir = [dir_path] $process.run($cxx.path -print-file-name=plugin)

  # If plugin support is disabled, then -print-file-name will print
  # the name we have passed (the real plugin directory will always
  # be absolute).
  #
  if (\"$plugin_dir\" == plugin)
    fail \"$recall($cxx.path) does not support plugins\"

  config [config.report] plugin_dir
}
\

\N|This is the only situation where a variable that does not match the
\c{config[.**].<project>.**} pattern is allowed in the \c{config} directive.
Note also that a value of such a variable is never considered new.|

Note that this mechanism should not be used to report configuration values
that require post-processing because of the loss of the new value status
(unless you are reporting both the original and post-processed values).
Instead, use the \c{config.report.variable} attribute to specify an
alternative variable for the report. For example:

\
config [strings, config.report.variable=woptions] \
  config.libhello.woptions

woptions  = -Wall -Wextra
woptions += $config.libhello.woptions
woptions += -Werror
\

\
$ b config.libhello.woptions=-Wno-extra -v
config libhello@/tmp/libhello/
  woptions   -Wall -Wextra -Wno-extra -Werror
\


\h#proj-config-propag|Configuration Propagation|

Using configuration values in our \c{buildfiles} is straightforward: they are
like any other \c{buildfile} variables and we can access them directly. For
example, this is how we could provide optional functionality in our library by
conditionally including certain source files: \N{See \l{#intro-if-else
Conditions (\c{if-else})} for why we should not use \c{if} to implement
this.}

\
# build/root.build

config [strings] config.libhello.io ?= true
\

\
# libhello/buildfile

lib{hello}: {hxx ixx txx cxx}{** -version -hello-io} hxx{version}
lib{hello}: {hxx cxx}{hello-io}: include = $config.libhello.io
\

On the other hand, it is often required to propagate the configuration
information to our source code. In fact, we have already seen one way to do
it: we can pass this information via C/C++ preprocessor macros defined on the
compiler's command line. For example:

\
# build/root.build

config [bool]   config.libhello.fancy    ?= false
config [string] config.libhello.greeting ?= 'Hello'
\

\
# libhello/buildfile

if $config.libhello.fancy
  cxx.poptions += -DLIBHELLO_FANCY

cxx.poptions += \"-DLIBHELLO_GREETING=\\\"$config.libhello.greeting\\\"\"
\

\
// libhello/hello.cxx

void say_hello (ostream& o, const string& n)
{
#ifdef LIBHELLO_FANCY
  // TODO: something fancy.
#else
  o << LIBHELLO_GREETING \", \" << n << '!' << endl;
#endif
}
\

We can even use the same approach to export certain configuration information
to our library's users (see \l{#intro-lib Library Exportation and Versioning}
for details):

\
# libhello/buildfile

# Export options.
#
if $config.libhello.fancy
  lib{hello}: cxx.export.poptions += -DLIBHELLO_FANCY
\

This mechanism is simple and works well across compilers so there is no reason
not to use it when the number of configuration values passed and their size
are small. However, it can quickly get unwieldy as these numbers grow. For
such cases, it may make sense to save this information into a separate
auto-generated source file with the help of the \l{#module-in \c{in}} module,
similar to how we do it for the version header.

The often-used approach is to generate a header file and include it into
source files that need access to the configuration information. Historically,
this was a C header full of macros called \c{config.h}. However, for C++
projects, there is no reason not to make it a C++ header and, if desired, to
use modern C++ features instead of macros. Which is what we will do here.

As an example of this approach, let's convert the above command line-based
implementation to use the configuration header. We will continue using macros
as a start (or in case this is a C project) and try more modern techniques
later. The \c{build/root.build} file is unchanged except for loading the
\c{in} module:

\
# build/root.build

config [bool]   config.libhello.fancy    ?= false
config [string] config.libhello.greeting ?= 'Hello'

using in
\

The \c{libhello/config.hxx.in} file is new:

\
// libhello/config.hxx.in

#pragma once

#define LIBHELLO_FANCY    $config.libhello.fancy$
#define LIBHELLO_GREETING \"$config.libhello.greeting$\"
\

As you can see, we can reference our configuration variables directly in the
\c{config.hxx.in} substitutions (see the \l{#module-in \c{in}} module
documentation for details on how this works).

\N|With this setup, the way to export configuration information to our
library's users is to make the configuration header public and install it,
similar to how we do it for the version header.|

The rest is changed as follows:

\
# libhello/buildfile

lib{hello}: {hxx ixx txx cxx}{** -version -config} hxx{version config}

hxx{config}: in{config}
{
  install = false
}
\

\
// libhello/hello.cxx

#include <libhello/config.hxx>

void say_hello (ostream& o, const string& n)
{
#if LIBHELLO_FANCY
  // TODO: something fancy.
#else
  o << LIBHELLO_GREETING \", \" << n << '!' << endl;
#endif
}
\

\N|Notice that we had to replace \c{#ifdef\ LIBHELLO_FANCY} with \c{#if\
LIBHELLO_FANCY}. If you want to continue using \c{#ifdef}, then you will need
to make the necessary arrangements yourself (the \c{in} module is a generic
preprocessor and does not provide any special treatment for \c{#define}). For
example:

\
#define LIBHELLO_FANCY $config.libhello.fancy$
#if !LIBHELLO_FANCY
#  undef LIBHELLO_FANCY
#endif
\

|

Now that the macro-based version is working, let's see how we can take
advantage of modern C++ features to hopefully improve on some of their
drawbacks. As a first step, we can replace the \c{LIBHELLO_FANCY} macro with a
compile-time constant and use \c{if\ constexpr} instead of \c{#ifdef} in our
implementation:

\
// libhello/config.hxx.in

namespace hello
{
  inline constexpr bool fancy = $config.libhello.fancy$;
}
\

\
// libhello/hello.cxx

#include <libhello/config.hxx>

void say_hello (ostream& o, const string& n)
{
  if constexpr (fancy)
  {
    // TODO: something fancy.
  }
  else
    o << LIBHELLO_GREETING \", \" << n << '!' << endl;
}
\

\N|Note that with \c{if\ constexpr} the branch not taken must still be valid,
parsable code. This is both one of the main benefits of using it instead of
\c{#if} (the code we are not using is still guaranteed to be syntactically
correct) as well as its main drawback (it cannot be used, for example, for
platform-specific code without extra efforts, such as providing shims for
missing declarations, etc).|

Next, we can do the same for \c{LIBHELLO_GREETING}:

\
// libhello/config.hxx.in

namespace hello
{
  inline constexpr char greeting[] = \"$config.libhello.greeting$\";
}
\

\
// libhello/hello.cxx

#include <libhello/config.hxx>

void say_hello (ostream& o, const string& n)
{
  if constexpr (fancy)
  {
    // TODO: something fancy.
  }
  else
    o << greeting << \", \" << n << '!' << endl;
}
\

\N|Note that for \c{greeting} we can achieve the same result without using
inline variables or \c{constexpr} and which would be usable in older C++ and
even C. All we have to do is add the \c{config.cxx.in} source file next to
our header with the definition of the \c{greeting} variable. For example:

\
// libhello/config.hxx.in

namespace hello
{
  extern const char greeting[];
}
\

\
// libhello/config.cxx.in

#include <libhello/config.hxx>

namespace hello
{
  const char greeting[] = \"$config.libhello.greeting$\";
}
\

\
# libhello/buildfile

lib{hello}: {hxx ixx txx cxx}{** -config} {hxx cxx}{config}

hxx{config}: in{config}
{
  install = false
}

cxx{config}: in{config}
\

As this illustrates, the \c{in} module can produce as many auto-generated
source files as we need. For example, we could use this to split the
configuration header into two, one public and installed while the other
private.|


\h1#attributes|Attributes|

\N{This chapter is a work in progress and is incomplete.}


\h1#name-patterns|Name Patterns|

For convenience, in certain contexts, names can be generated with shell-like
wildcard patterns. A name is a \i{name pattern} if its value contains one or
more unquoted wildcard characters or character sequences. For example:

\
./: */                     # All (immediate) subdirectories
exe{hello}: {hxx cxx}{**}  # All C++ header/source files.
pattern = '*.txt'          # Literal '*.txt'.
\

Pattern-based name generation is not performed in certain contexts.
Specifically, it is not performed in target names where it is interpreted
as a pattern for target type/pattern-specific variable assignments. For
example.

\
s = *.txt             # Variable assignment (performed).
./: cxx{*}            # Prerequisite names (performed).
cxx{*}: dist = false  # Target pattern (not performed).
\

In contexts where it is performed, it can be inhibited with quoting, for
example:

\
pat = 'foo*bar'
./: cxx{'foo*bar'}
\

The following wildcards are recognized:

\
*     - match any number of characters (including zero)
?     - match any single character
[...] - match a character with a bracket expression
\

\N|Currently only literal character and range bracket expressions are
supported. Specifically, no character or equivalence classes, etc., are
supported nor the special characters backslash-escaping. See the \"Pattern
Matching Notation\" section in the POSIX \"Shell Command Language\"
specification for details.|

Note that some wildcard characters may have special meaning in certain
contexts. For instance, \c{[} at the beginning of a value will be interpreted
as the start of the attribute list while \c{?} and \c{[} in the eval context
are part of the ternary operator and value subscript, respectively. In such
cases the wildcard character will need to be escaped, for example:

\
x = \[1-9]-foo.txt
y = (foo.\?xx)
z = ($foo\[123].txt)
\

If a pattern ends with a directory separator, then it only matches
directories. Otherwise, it only matches files. Matches that start with a dot
(\c{.}) are automatically ignored unless the pattern itself also starts with
this character.

In addition to the above wildcards, \c{**} and \c{***} are recognized as
wildcard sequences. If a pattern contains \c{**}, then it is matched just like
\c{*} but in all the subdirectories, recursively, but excluding directories
that contain the \c{.buildignore} file. The \c{***} wildcard behaves like
\c{**} but also matches the start directory itself. For example:

\
exe{hello}: cxx{**}  # All C++ source files recursively.
\

A group-enclosed (\c{{\}}) pattern value may be followed by
inclusion/exclusion patterns/matches. A subsequent value is treated as an
inclusion or exclusion if it starts with a literal, unquoted plus (\c{+}) or
minus (\c{-}) sign, respectively. In this case the remaining group values, if
any, must all be inclusions or exclusions. If the second value doesn't start
with a plus or minus, then all the group values are considered independent
with leading pluses and minuses not having any special meaning. For regularity
as well as to allow patterns without wildcards, the first pattern can also
start with the plus sign. For example:

\
exe{hello}: cxx{f* -foo}            # Exclude foo if exists.
exe{hello}: cxx{f* +bar}            # Include bar if exists.
exe{hello}: cxx{f* -fo?}            # Exclude foo and fox if exist.
exe{hello}: cxx{f* +b* -foo -bar}   # Exclude foo and bar if exist.
exe{hello}: cxx{+f* +b* -foo -bar}  # Same as above.
exe{hello}: cxx{+foo}               # Pattern without wildcards.
exe{hello}: cxx{f* b* -z*}          # Names matching three patterns.
\

Inclusions and exclusions are applied in the order specified and only to the
result produced up to that point. The order of names in the result is
unspecified. However, it is guaranteed not to contain duplicates. The first
pattern and the following inclusions/exclusions must be consistent with
regards to the type of filesystem entry they match. That is, they should all
match either files or directories. For example:

\
exe{hello}: cxx{f* -foo +*oo}  # Exclusion has no effect.
exe{hello}: cxx{f* +*oo}       # Ok, no duplicates.
./: {*/ -build}                # Error: exclusion not a directory.
\

As a more realistic example, let's say we want to exclude source files that
reside in the \c{test/} directories (and their subdirectories) anywhere in the
tree. This can be achieved with the following pattern:

\
exe{hello}: cxx{** -***/test/**}
\

Similarly, if we wanted to exclude all source files that have the \c{-test}
suffix:

\
exe{hello}: cxx{** -**-test}
\

In contrast, the following pattern only excludes such files from the top
directory:

\
exe{hello}: cxx{** -*-test}
\

If many inclusions or exclusions need to be specified, then an
inclusion/exclusion group can be used. For example:

\
exe{hello}: cxx{f* -{foo bar}}
exe{hello}: cxx{+{f* b*} -{foo bar}}
\

This is particularly useful if you would like to list the names to include or
exclude in a variable. For example, this is how we can exclude certain files
from compilation but still include them as ordinary file prerequisites (so
that they are still included into the distribution):

\
exc = foo.cxx bar.cxx
exe{hello}: cxx{+{f* b*} -{$exc}} file{$exc}
\

If we want to specify our pattern in a variable, then we have to use the
explicit inclusion syntax, for example:

\
pat = 'f*'
exe{hello}: cxx{+$pat} # Pattern match.
exe{hello}: cxx{$pat}  # Literal 'f*'.

pat = '+f*'
exe{hello}: cxx{$pat}  # Literal '+f*'.

inc = 'f*'  'b*'
exc = 'f*o' 'b*r'
exe{hello}: cxx{+{$inc} -{$exc}}
\

One common situation that calls for exclusions is auto-generated source
code. Let's say we have auto-generated command line parser in \c{options.hxx}
and \c{options.cxx}. Because of the in-tree builds, our name pattern may or
may not find these files. Note, however, that we cannot just include them as
non-pattern prerequisites. We also have to exclude them from the pattern match
since otherwise we may end up with duplicate prerequisites. As a result, this
is how we have to handle this case provided we want to continue using patterns
to find other, non-generated source files:

\
exe{hello}: {hxx cxx}{* -options} {hxx cxx}{options}
\

If the name pattern includes an absolute directory, then the pattern match is
performed in that directory and the generated names include absolute
directories as well. Otherwise, the pattern match is performed in the
\i{pattern base} directory. In buildfiles this is \c{src_base} while on the
command line \- the current working directory. In this case the generated
names are relative to the base directory. For example, assuming we have the
\c{foo.cxx} and \c{b/bar.cxx} source files:

\
exe{hello}: $src_base/cxx{**}  # $src_base/cxx{foo} $src_base/b/cxx{bar}
exe{hello}:           cxx{**}  #           cxx{foo}           b/cxx{bar}
\

Pattern matching as well as inclusion/exclusion logic is target
type-specific. If the name pattern does not contain a type, then the
\c{dir{\}} type is assumed if the pattern ends with a directory separator and
\c{file{\}} otherwise.

For the \c{dir{\}} target type the trailing directory separator is added to
the pattern and all the inclusion/exclusion patterns/matches that do not
already end with one. Then the filesystem search is performed for matching
directories. For example:

\
./: dir{* -build}  # Search for */, exclude build/.
\

For the \c{file{\}} and \c{file{\}}-based target types the default extension
(if any) is added to the pattern and all the inclusion/exclusion
patterns/matches that do not already contain an extension. Then the filesystem
search is performed for matching files.

For example, the \c{cxx{\}} target type obtains the default extension from the
\c{extension} variable. Assuming we have the following line in our
\c{root.build}:

\
cxx{*}: extension = cxx
\

And the following in our \c{buildfile}:

\
exe{hello}: {cxx}{* -foo -bar.cxx}
\

The pattern match will first search for all the files matching the \c{*.cxx}
pattern in \c{src_base} and then exclude \c{foo.cxx} and \c{bar.cxx} from the
result. Note also that target type-specific decorations are removed from the
result. So in the above example if the pattern match produces \c{baz.cxx},
then the prerequisite name is \c{cxx{baz\}}, not \c{cxx{baz.cxx\}}.

If the name generation cannot be performed because the base directory is
unknown, target type is unknown, or the target type is not directory or
file-based, then the name pattern is returned as is (that is, as an ordinary
name). Project-qualified names are never considered to be patterns.


\h1#variables|Variables|

\N{This chapter is a work in progress and is incomplete.}

The following variable/value types can currently be used in \c{buildfiles}:

\
bool

int64
int64s

uint64
uint64s

string
strings

path
paths
dir_path
dir_paths

name
names
name_pair

project_name
target_triplet
\

Note that while expansions in the target and prerequisite-specific assignments
happen in the corresponding target and prerequisite contexts, respectively,
for type/pattern-specific assignments they happen in the scope context. Plus,
a type/pattern-specific prepend/append is applied at the time of expansion for
the actual target. For example:

\
x = s

file{foo}:              # target
{
  x += t    # s t
  y = $x y  # s t y
}

file{foo}: file{bar}    # prerequisite
{
  x += p    # x t p
  y = $x y  # x t p y
}

file{b*}:               # type/pattern
{
  x += w   # <append w>
  y = $x w # <assign s w>
}

x = S

info $(file{bar}: x) # S w
info $(file{bar}: y) # s w
\


\h1#directives|Directives|

\N{This chapter is a work in progress and is incomplete.}

\h#directives-include|\c{include}|

\
include <file>
include <directory>
\

Load the specified file (the first form) or \c{buildfile} in the specified
directory (the second form). In both cases the file is loaded in the scope
corresponding to its directory. Subsequent inclusions of the same file are
automatically ignored. See also \l{#directives-source \c{source}}.


\h#directives-source|\c{source}|


\
source <file>
\

Load the specified file in the current scope as if its contents were copied
and pasted in place of the \c{source} directive. Note that subsequent sourcing
of the same file in the same scope are not automatically ignored.  See also
\l{#directives-include \c{include}}.


\h1#module-config|\c{config} Module|

\N{This chapter is a work in progress and is incomplete.}

\h#module-config-hermetic|Hermetic Build Configurations|

Hermetic build configurations save environment variables that affect the
project along with other project configuration in the \c{build/config.build}
file. These saved environment variables are then used instead of the current
environment when performing operations on the project, thus making sure the
project \"sees\" exactly the same environment as during configuration.

\N|While currently hermetic configurations only deal with the environment, in
the future this functionality may be extended to also support disallowing
changes to external resources (compilers, system headers and libraries, etc).|

To create a hermetic configuration we use the \c{config.config.hermetic}
configuration variable. For example:

\
$ b configure config.config.hermetic=true
\

\N|Hermetic configurations are not the default because they are not without
drawbacks. Firstly, a hermetic configuration may break if the saved
environment becomes incompatible with the rest of the system. For example, you
may re-install an external program (say, a compiler) into a different location
and update your \c{PATH} to match the new setup. However, a hermetic
configuration will \"see\" the first change but not the second.

Another issue is the commands printed during a hermetic build: they are
executed in the saved environment which may not match the environment in which
the build system was invoked. As a result, we cannot easily re-execute such
commands, which is often handy during build troubleshooting.

It is also important to keep in mind that a non-hermetic build configuration
does not break or produce incorrect results if the environment changes.
Instead, changes to the environment are detected and affected targets are
automatically rebuilt.

The two use-cases where hermetic configurations are really useful are when we
need to save an environment which is not generally available (for example, an
environment of a Visual Studio development command prompt) or when our build
results need to exactly match the specific configuration (for example, because
parts of the overall result have already been built and installed, as is the
case with build system modules).|

If we now examine \c{config.build}, we will see something along these lines:

\
$ cat build/config.build

config.config.hermetic = true
config.config.environment = CPATH CPLUS_INCLUDE_PATH PATH=...
\

\N|Hermetic configuration support is built on top of the low-level
\c{config.config.environment} configuration variable which allows us to
specify custom environment variables and their values. Specifically, it
contains a list of environment variable \"sets\" (\c{\i{name}=\i{value}})
and \"unsets\" (\ci{name}). For example:

\
$ b configure \
  config.config.environment=\"PATH=/bin:/usr/bin LD_LIBRARY_PATH\"
\

Specifying \c{config.config.hermetic=true} simply instructs the \c{config}
module to collect and save in \c{config.config.environment} environment
variables that affect the project. These include:

\ul|

\li|built-in variables (such as \c{PATH} and \c{LD_LIBRARY_PATH} or equivalent),|

\li|variables that affect external programs as reported by build system
modules (such as \c{CPLUS_INCLUDE_PATH} reported by the \c{cxx} module) or
by imported programs via metadata,|

\li|variables reported by the project itself with the \c{config.environment}
directive (discussed below).|||

Reconfiguring a hermetic configuration preserves the saved environment unless
\i{re-hermetization} is explicitly requested with the
\c{config.config.hermetic.reload} configuration variable. For example:

\
$ b configure config.config.hermetic.reload=true
\

\N|Note that \c{config.config.hermetic.reload} is transient and is not stored
in \c{config.build}. In other words, there is no way to create a hermetic
configuration that is re-hermetized by default during reconfiguration.|

To \i{de-hermetize} a hermetic build configuration, reconfigure it with
\c{config.config.hermetic=false}.

\N|The \c{config.config.hermetic} variable has essentially a tri-state value:
\c{true} means keep hermetized (save the environment in
\c{config.config.environment}), \c{false} means keep de-hermetized (clear
\c{config.config.environment}) and \c{null} or undefined means don't touch
\c{config.config.environment}.|

We can adjust the set of environment variables saved in a hermetic
configuration using the \c{config.config.hermetic.environment} configuration
variable. It contains a list of inclusions (\ci{name}) and exclusions
(\c{\i{name}@false}) which are applied to the final set of environment
variables that affect the project. For example:

\
LC_ALL=C b configure \
  config.config.hermetic=true \
  config.config.hermetic.environment=\"LC_ALL PATH@false\"
\

Typically, the set of environment variables that affect the project is
discovered automatically. Specifically, modules that we use (such as \c{cxx})
are expected to report the environment variables that affect the programs they
invoke (such as the C++ compiler). Similarly, programs that we import in our
\c{buildfiles} (for example to use in ad hoc recipes) are expected to report
environment variables that affect them as part of their metadata.

However, there are situations where we need to report an environment variable
manually. These include calling the \c{$getenv()} function from a
\c{buildfile} or invoking a program (either in an ad hoc recipe, the \c{run}
directive, or the \c{$run*()} function family) that either does not provide
the metadata or does not report the environment as part of it. In such cases
we should report the environment variable manually using the
\c{config.environment} directive. For example:

\
config.environment USE_FOO

foo = $getenv(USE_FOO)

if ($foo != [null])
  cxx.poptions += \"-DUSE_FOO=$foo\"
\

Additionally, if invoking a program in an ad hoc recipe that either does not
provide the metadata or does not report the environment as part of it, then we
additionally should track the changes to the relevant environment variables
manually using the \c{depdb env} builtin. For example:

\
import! foo = foo%exe{foo} # Uses FOO and BAR environment variables.

config.environment FOO BAR

file{output}: file{input} $foo
{{
  diag foo $>
  depdb env FOO BAR
  $foo $path($<[0]) >$path($>)
}}
\

\N|Normally, we would want to report variables that affect the build result
rather than build byproducts (for example, diagnostics). This is, for example,
the reason why locale-related environment variables are not saved by default.
Also, sometime environment variables only affect certain modes of a program.
If such modes are not used, then there is no need to report the corresponding
variables.|


\h1#module-test|\c{test} Module|

\N{This chapter is a work in progress and is incomplete.}

The targets to be tested as well as the tests/groups from testscripts to be
run can be narrowed down using the \c{config.test} variable. While this
value is normally specified as a command line override (for example, to
quickly re-run a previously failed test), it can also be persisted in
\c{config.build} in order to create a configuration that will only run a
subset of tests by default. For example:

\
b test config.test=foo/exe{driver} # Only test foo/exe{driver} target.
b test config.test=bar/baz         # Only run bar/baz testscript test.
\

The \c{config.test} variable contains a list of \c{@}-separated pairs with the
left hand side being the target and the right hand side being the testscript
id path. Either can be omitted (along with \c{@}). If the value contains a
target type or ends with a directory separator, then it is treated as a target
name. Otherwise \- an id path. The targets are resolved relative to the root
scope where the \c{config.test} value is set. For example:

\
b test config.test=foo/exe{driver}@bar
\

To specify multiple id paths for the same target we can use the pair
generation syntax:

\
b test config.test=foo/exe{driver}@{bar baz}
\

If no targets are specified (only id paths), then all the targets are tested
(with the testscript tests to be run limited to the specified id paths). If no
id paths are specified (only targets), then all the testscript tests are run
(with the targets to be tested limited to the specified targets). An id path
without a target applies to all the targets being considered.

A directory target without an explicit target type (for example, \c{foo/}) is
treated specially. It enables all the tests at and under its directory. This
special treatment can be inhibited by specifying the target type explicitly
(for example, \c{dir{foo/\}}).

The test execution time can be limited using the \c{config.test.timeout}
variable. Its value has the \c{<operation-timeout>/<test-timeout>} form where
the timeouts are specified in seconds and either of them (but not both) can be
omitted. The left hand side sets the timeout for the whole \c{test} operation
and the right hand side \- for individual tests. The zero value clears the
previously set timeout. For example:

\
b test config.test.timeout=20   # Test operation.
b test config.test.timeout=20/5 # Test operation and individual tests.
b test config.test.timeout=/5   # Individual tests.
\

The test timeout can be specified on multiple nested root scopes. For example,
we can specify a greater timeout for the entire build configuration and lesser
ones for individual projects. The tests must complete before the nearest of
the enclosing scope timeouts. Failed that, the timed out tests are terminated
forcibly causing the entire \c{test} operation to fail. See also the
\l{testscript#builtins-timeout \c{timeout}} builtin for specifying timeouts
from within the tests and test groups.

The programs being tested can be executed via a \i{runner program} by
specifying the \c{config.test.runner} variable. Its value has the \c{<path>
[<options>]} form. For example:

\
b test config.test.runner=\"valgrind -q\"
\

When the runner program is specified, commands of simple and Testscript tests
are automatically adjusted so that the runner program is executed instead,
with the test command passed to it as arguments. For ad hoc test recipes,
the runner program has to be handled explicitly. Specifically, if
\c{config.test.runner} is specified, the \c{test.runner.path} and
\c{test.runner.options} variables contain the runner program path and options,
respectively, and are set to \c{null} otherwise. These variables can be used
by ad hoc recipes to detect the presence of the runner program and, if so,
arrange appropriate execution of desired commands. For example:

\
exe{hello}:
% test
{{
  diag test $>

  cmd = ($test.runner.path == [null] \
    ? $> \
    : $test.runner.path $test.runner.options $path($>))

  $cmd 'World' >>>?'Hello, World!'
}}
\


\h1#module-install|\c{install} Module|

\N{This chapter is a work in progress and is incomplete.}

The \c{install} module provides support for installing and uninstalling
projects.

As briefly discussed in the \l{#intro-operations-install Installing} section
of the Introduction, the \c{install} module defines the following standard
installation locations:

\
name        default                                 config.* override
----        -------                                 -----------------
root                                                install.root

data_root   root/                                   install.data_root
exec_root   root/                                   install.exec_root

bin         exec_root/bin/                          install.bin
sbin        exec_root/sbin/                         install.sbin
lib         exec_root/lib/<private>/                install.lib
libexec     exec_root/libexec/<private>/<project>/  install.libexec
pkgconfig   lib/pkgconfig/                          install.pkgconfig

etc         data_root/etc/                          install.etc
include     data_root/include/<private>/            install.include
share       data_root/share/                        install.share
data        share/<private>/<project>/              install.data

doc         share/doc/<private>/<project>/          install.doc
legal       doc/                                    install.legal
man         share/man/                              install.man
man<N>      man/man<N>/                             install.man<N>
\

The \c{<project>}, \c{<version>}, and \c{<private>} substitutions in these
\c{config.install.*} values are replaced with the project name, version, and
private subdirectory, respectively. If either is empty, then the corresponding
directory component is ignored.

The optional private installation subdirectory (\c{<private>}) mechanism can
be used to hide the implementation details of a project. This is primarily
useful when installing an executable that depends on a bunch of libraries into
a shared location, such as \c{/usr/local/}. By hiding the libraries in the
private subdirectory we can make sure that they will not interfere with
anything that is already installed into such a shared location by the user
and that any further such installations won't interfere with our executable.

The private installation subdirectory is specified with the
\c{config.install.private} variable. Its value must be a relative
directory and may include multiple components. For example:

\
$ b install config.install.root=/usr/local/ config.install.private=hello/
\

\N|If you are relying on your system's dynamic linker defaults to
automatically find shared libraries that are installed with your executable,
then adding the private installation subdirectory will most definitely
cause this to stop working. The recommended way to resolve this problem is
to use \i{rpath}, for example:

\
$ b install                       \
  config.install.root=/usr/local/ \
  config.install.private=hello/   \
  config.bin.rpath=/usr/local/lib/hello/
\

|

\h1#module-version|\c{version} Module|

A project can use any version format as long as it meets the package version
requirements. The toolchain also provides additional functionality for
managing projects that conform to the \c{build2} \i{standard version}
format. If you are starting a new project that uses \c{build2}, you are
strongly encouraged to use this versioning scheme. It is based on much thought
and, often painful, experience. If you decide not to follow this advice, you
are essentially on your own when version management is concerned.

The standard \c{build2} project version conforms to \l{http://semver.org
Semantic Versioning} and has the following form:

\
<major>.<minor>.<patch>[-<prerel>]
\

For example:

\
1.2.3
1.2.3-a.1
1.2.3-b.2
\

The \c{build2} package version that uses the standard project version will
then have the following form (\i{epoch} is the versioning scheme version
and \i{revision} is the package revision):

\
[+<epoch>-]<major>.<minor>.<patch>[-<prerel>][+<revision>]
\

For example:

\
1.2.3
1.2.3+1
+2-1.2.3-a.1+2
\

The \i{major}, \i{minor}, and \i{patch} should be numeric values between \c{0}
and \c{99999} and all three cannot be zero at the same time. For initial
development it is recommended to use \c{0} for \i{major}, start with version
\c{0.1.0}, and change to \c{1.0.0} once things stabilize.

In the context of C and C++ (or other compiled languages), you should
increment \i{patch} when making binary-compatible changes, \i{minor} when
making source-compatible changes, and \i{major} when making breaking changes.
While the binary compatibility must be set in stone, the source compatibility
rules can sometimes be bent. For example, you may decide to make a breaking
change in a rarely used interface as part of a minor release (though this is
probably still a bad idea if your library is widely depended upon). Note also
that in the context of C++ deciding whether a change is binary-compatible is a
non-trivial task. There are resources that list the rules but no automated
tooling yet. If unsure, increment \i{minor}.

If present, the \i{prerel} component signifies a pre-release. Two types of
pre-releases are supported by the standard versioning scheme: \i{final} and
\i{snapshot} (non-pre-release versions are naturally always final). For final
pre-releases the \i{prerel} component has the following form:

\
(a|b).<num>
\

For example:

\
1.2.3-a.1
1.2.3-b.2
\

The letter '\c{a}' signifies an alpha release and '\c{b}' \- beta. The
alpha/beta numbers (\i{num}) should be between 1 and 499.

Note that there is no support for release candidates. Instead, it is
recommended that you use later-stage beta releases for this purpose (and, if
you wish, call them \"release candidates\" in announcements, etc).

What version should be used during development? The common approach is to
increment to the next version and use that until the release. This has one
major drawback: if we publish intermediate snapshots (for example, for
testing) they will all be indistinguishable both between each other and, even
worse, from the final release. One way to remedy this is to increment the
pre-release number before each publication. However, unless automated, this
will be burdensome and error-prone. Also, there is a real possibility of
running out of version numbers if, for example, we do continuous integration
by publishing and testing each commit.

To address this, the standard versioning scheme supports \i{snapshot
pre-releases} with the \i{prerel} component having the following extended
form:

\
(a|b).<num>.<snapsn>[.<snapid>]
\

For example:

\
1.2.3-a.1.20180319215815.26efe301f4a7
\

In essence, a snapshot pre-release is after the previous final release but
before the next (\c{a.1} and, perhaps, \c{a.2} in the above example) and
is uniquely identified by the snapshot sequence number (\i{snapsn}) and
optional snapshot id (\i{snapid}).

The \i{num} component has the same semantics as in the final pre-releases
except that it can be \c{0}. The \i{snapsn} component should be either the
special value '\c{z}' or a numeric, non-zero value that increases for each
subsequent snapshot. It must not be longer than 16 decimal digits. The
\i{snapid} component, if present, should be an alpha-numeric value that
uniquely identifies the snapshot. It is not required for version comparison
(\i{snapsn} should be sufficient) and is included for reference. It must not
be longer than 16 characters.

Where do the snapshot number and id come from? Normally from the version
control system. For example, for \c{git}, \i{snapsn} is the commit date in the
\i{YYYYMMDDhhmmss} form and UTC timezone and \i{snapid} is a 12-character
abbreviated commit id. As discussed below, the \c{build2} \c{version} module
extracts and manages all this information automatically (but the use of
\c{git} commit dates is not without limitations; see below for details).

The special '\c{z}' \i{snapsn} value identifies the \i{latest} or
\i{uncommitted} snapshot. It is chosen to be greater than any other possible
\i{snapsn} value and its use is discussed further below.

As an illustration of this approach, let's examine how versions change
during the lifetime of a project:

\
0.1.0-a.0.z     # development after a.0
0.1.0-a.1       # pre-release
0.1.0-a.1.z     # development after a.1
0.1.0-a.2       # pre-release
0.1.0-a.2.z     # development after a.2
0.1.0-b.1       # pre-release
0.1.0-b.1.z     # development after b.1
0.1.0           # release
0.1.1-b.0.z     # development after b.0 (bugfix)
0.2.0-a.0.z     # development after a.0
0.1.1           # release (bugfix)
1.0.0           # release (jumped straight to 1.0.0)
...
\

As shown in the above example, there is nothing wrong with \"jumping\" to a
further version (for example, from alpha to beta, or from beta to release, or
even from alpha to release). We cannot, however, jump backwards (for example,
from beta back to alpha). As a result, a sensible strategy is to start with
\c{a.0} since it can always be upgraded (but not downgraded) at a later stage.

When it comes to the version control systems, the recommended workflow is as
follows: The change to the final version should be the last commit in the
(pre-)release. It is also a good idea to tag this commit with the project
version. A commit immediately after that should change the version to a
snapshot, \"opening\" the repository for development.

The project version without the snapshot part can be represented as a 64-bit
decimal value comparable as integers (for example, in preprocessor
directives). The integer representation has the following form:

\
AAAAABBBBBCCCCCDDDE

AAAAA - major
BBBBB - minor
CCCCC - patch
DDD   - alpha / beta (DDD + 500)
E     - final (0) / snapshot (1)
\

If the \i{DDDE} value is not zero, then it signifies a pre-release. In this
case one is subtracted from the \i{AAAAABBBBBCCCCC} value. An alpha number is
stored in \i{DDD} as is while beta \- incremented by \c{500}. If \i{E} is
\c{1}, then this is a snapshot after \i{DDD}.

For example:

\
             AAAAABBBBBCCCCCDDDE
0.1.0        0000000001000000000
0.1.2        0000000001000020000
1.2.3        0000100002000030000
2.2.0-a.1    0000200001999990010
3.0.0-b.2    0000299999999995020
2.2.0-a.1.z  0000200001999990011
\

A project that uses standard versioning can rely on the \c{build2} \c{version}
module to simplify and automate version managements. The \c{version} module
has two primary functions: eliminate the need to change the version anywhere
except in the project's manifest file and automatically extract and propagate
the snapshot information (sequence number and id).

The \c{version} module must be loaded in the project's \c{bootstrap.build}.
While being loaded, it reads the project's manifest and extracts its version
(which must be in the standard form). The version is then parsed and presented
as the following build system variables (which can be used in the buildfiles):

\
[string] version                     # +2-1.2.3-b.4.1234567.deadbeef+3

[string] version.project             # 1.2.3-b.4.1234567.deadbeef
[uint64] version.project_number      # 0000100002000025041
[string] version.project_id          # 1.2.3-b.4.deadbeef

[bool]   version.stub                # false (true for 0[+<revision>])

[uint64] version.epoch               # 2

[uint64] version.major               # 1
[uint64] version.minor               # 2
[uint64] version.patch               # 3

[bool]   version.alpha               # false
[bool]   version.beta                # true
[bool]   version.pre_release         # true
[string] version.pre_release_string  # b.4
[uint64] version.pre_release_number  # 4

[bool]   version.snapshot            # true
[uint64] version.snapshot_sn         # 1234567
[string] version.snapshot_id         # deadbeef
[string] version.snapshot_string     # 1234567.deadbeef
[bool]   version.snapshot_committed  # true

[uint64] version.revision            # 3
\

As a convenience, the \c{version} module also extracts the \c{summary} and
\c{url} manifest values and sets them as the following build system variables
(this additional information is used, for example, when generating the
\c{pkg-config} files):

\
[string] project.summary
[string] project.url
\

If the version is the latest snapshot (that is, it's in the \c{.z} form), then
the \c{version} module extracts the snapshot information from the version
control system used by the project. Currently only \c{git} is supported with
the following semantics.

If the project's source directory (\c{src_root}) is clean (that is, it does
not have any changed or untracked files), then the \c{HEAD} commit date and id
are used as the snapshot number and id, respectively.

Otherwise (that is, the project is between commits), the \c{HEAD} commit date
is incremented by one second and is used as the snapshot number with no id.
While we can work with such uncommitted snapshots locally, we should not
distribute or publish them since they are indistinguishable from each other.

Finally, if the project does not have \c{HEAD} (that is, the project has
no commits yet), the special \c{19700101000000} (UNIX epoch) commit date is
used.

The use of \c{git} commit dates for snapshot ordering has its limitations:
they have one second resolution which means it is possible to create two
commits with the same date (but not the same commit id and thus snapshot
id). We also need all the committers to have a reasonably accurate
clock. Note, however, that in case of a commit date clash/ordering issue, we
still end up with distinct versions (because of the commit id) \- they are
just not ordered correctly. As a result, we feel that the risks are justified
when the only alternative is manual version management (which is always an
option, nevertheless).

When we prepare a distribution of a snapshot, the \c{version} module
automatically adjusts the package name to include the snapshot information as
well as patches the manifest file in the distribution with the snapshot number
and id (that is, replacing \c{.z} in the version value with the actual
snapshot information). The result is a package that is specific to this
commit.

Besides extracting the version information and making it available as
individual components, the \c{version} module also provides rules for
installing the manifest file as well as automatically generating version
headers (or other similar version-based files).

By default the project's \c{manifest} file is installed as documentation, just
like other \c{doc{\}} targets (thus replacing the \c{version} file customarily
shipped in the project root directory). The manifest installation rule in the
\c{version} module in addition patches the installed manifest file with the
actual snapshot number and id, just like during the preparation of
distributions.

The version header rule is based on the \l{#module-in \c{in}} module rule and
can be used to preprocess a template file with version information. While it
is usually used to generate C/C++ version headers (thus the name), it can
really generate any kind of files.

The rule matches a \c{file}-based target that has the corresponding \c{in}
prerequisite and also depends on the project's \c{manifest} file. As an
example, let's assume we want to auto-generate a header called \c{version.hxx}
for our \c{libhello} library. To accomplish this we add the \c{version.hxx.in}
template as well as something along these lines to our \c{buildfile}:

\
lib{hello}: ... hxx{version}

hxx{version}: in{version} $src_root/file{manifest}
{
  dist = true
}
\

The header rule is a line-based preprocessor that substitutes fragments
enclosed between (and including) a pair of dollar signs (\c{$}) with \c{$$}
being the escape sequence (see the \l{#module-in \c{in}} module for
details). As an example, let's assume our \c{version.hxx.in} contains the
following lines:

\
#ifndef LIBHELLO_VERSION

#define LIBHELLO_VERSION     $libhello.version.project_number$ULL
#define LIBHELLO_VERSION_STR \"$libhello.version.project$\"

#endif
\

If our \c{libhello} is at version \c{1.2.3}, then the generated
\c{version.hxx} will look like this:

\
#ifndef LIBHELLO_VERSION

#define LIBHELLO_VERSION     100002000030000ULL
#define LIBHELLO_VERSION_STR \"1.2.3\"

#endif
\

The first component after the opening \c{$} should be either the name of the
project itself (like \c{libhello} above) or a name of one of its dependencies
as listed in the manifest. If it is the project itself, then the rest can
refer to one of the \c{version.*} variables that we discussed earlier (in
reality it can be any variable visible from the project's root scope).

If the name refers to one of the dependencies (that is, projects listed with
\c{depends:} in the manifest), then the following special substitutions are
recognized:

\
$<name>.version$                           - textual version constraint
$<name>.condition(<VERSION>[,<SNAPSHOT>])$ - numeric satisfaction condition
$<name>.check(<VERSION>[,<SNAPSHOT>])$     - numeric satisfaction check
\

Here \i{VERSION} is the version number macro and the optional \i{SNAPSHOT} is
the snapshot number macro. The snapshot is only required if you plan to
include snapshot information in your dependency constraints.

As an example, let's assume our \c{libhello} depends on \c{libprint} which
is reflected with the following line in our manifest:

\
depends: libprint >= 2.3.4
\

We also assume that \c{libprint} provides its version information in the
\c{libprint/version.hxx} header and uses analogous-named macros. Here
is how we can add a version check to our \c{version.hxx.in}:

\
#ifndef LIBHELLO_VERSION

#define LIBHELLO_VERSION     $libhello.version.project_number$ULL
#define LIBHELLO_VERSION_STR \"$libhello.version.project$\"

#include <libprint/version.hxx>

$libprint.check(LIBPRINT_VERSION)$

#endif
\

After the substitution our \c{version.hxx} header will look like this:

\
#ifndef LIBHELLO_VERSION

#define LIBHELLO_VERSION     100002000030000ULL
#define LIBHELLO_VERSION_STR \"1.2.3\"

#include <libprint/version.hxx>

#ifdef LIBPRINT_VERSION
#  if !(LIBPRINT_VERSION >= 200003000040000ULL)
#    error incompatible libprint version, libprint >= 2.3.4 is required
#  endif
#endif

#endif
\

The \c{version} and \c{condition} substitutions are the building blocks of the
\c{check} substitution. For example, here is how we can implement a check with
a customized error message:

\
#if !($libprint.condition(LIBPRINT_VERSION)$)
#  error bad libprint, need libprint $libprint.version$
#endif
\

The \c{version} module also treats one dependency in a special way: if you
specify the required version of the build system in your manifest, then the
module will automatically check it for you. For example, if we have the
following line in our manifest:

\
depends: * build2 >= 0.5.0
\

And someone tries to build our project with \c{build2} \c{0.4.0}, then they
will see an error like this:

\
build/bootstrap.build:3:1: error: incompatible build2 version
  info: running 0.4.0
  info: required 0.5.0
\

What version constraints should be used when depending on another project? We
start with a simple case where we depend on a release. Let's say \c{libprint}
\c{2.3.0} added a feature that we need in our \c{libhello}. If \c{libprint}
follows the source/binary compatibility guidelines discussed above, then
any \c{2.X.Y} version should work provided \c{X >= 3}. And this how we can
specify it in the manifest:

\
depends: libprint ^2.3.0
\

Let's say we are now working on \c{libhello} \c{2.0.0} and would like to start
using features from \c{libprint} \c{3.0.0}. However, currently, only
pre-releases of \c{3.0.0} are available. If you would like to add a dependency
on a pre-release (most likely from your own pre-release), then the
recommendation is to only allow a specific version, essentially \"expiring\"
the combination as soon as newer versions become available. For example:

\
version: 2.0.0-b.1
depends: libprint == 3.0.0-b.2
\

Finally, let's assume we are feeling adventurous and would like to test
development snapshots of \c{libprint} (most likely from our own snapshots). In
this case the recommendation is to only allow a snapshot range for a specific
pre-release with the understanding and a warning that no compatibility between
snapshot versions is guaranteed. For example:

\
version: 2.0.0-b.1.z
depends: libprint [3.0.0-b.2.1 3.0.0-b.3)
\

\h1#module-bin|\c{bin} Module|

\N{This chapter is a work in progress and is incomplete.}


\h1#module-cc|\c{cc} Module|

\N{This chapter is a work in progress and is incomplete.}

This chapter describes the \c{cc} build system module which provides the
common compilation and linking support for C-family languages.

\h#cc-config|C-Common Configuration Variables|

\
config.c
config.cxx
  cc.id

  c.target
  c.target.cpu
  c.target.vendor
  c.target.system
  c.target.version
  c.target.class

config.cc.poptions
  cc.poptions

config.cc.coptions
  cc.coptions

config.cc.loptions
  cc.loptions

config.cc.aoptions
  cc.aoptions

config.cc.libs
  cc.libs

config.cc.internal.scope
  cc.internal.scope
\

Note that the compiler mode options are \"cross-hinted\" between \c{config.c}
and \c{config.cxx} meaning that if we specify one but not the other, mode
options, if any, will be added to the absent. This may or may not be the
desired behavior, for example:

\
# Ok: config.c=\"gcc -m32\"
$ b config.cxx=\"g++ -m32\"

# Not OK: config.c=\"clang -stdlib=libc++\"
$ b config.cxx=\"clang++ -stdlib=libc++\"
\

\h#cc-internal-scope|Compilation Internal Scope|

\N|While this section uses the \c{cxx} module and C++ compilation as an
example, the same functionality is available for C compilation \- simply
replace \c{cxx} with \c{c} in the module and variable names.|

The \c{cxx} module has a notion of a project's internal scope. During
compilation of a project's C/C++ translation units a header search path
(\c{-I}) exported by a library that is outside of the internal scope is
considered external and, if supported by the compiler, the corresponding
\c{-I} option is translated to an appropriate \"external header search path\"
option (\c{-isystem} for GCC/Clang, \c{/external:I} for MSVC 16.10 and
later). In particular, this suppresses compiler warnings in such external
headers (\c{/external:W0} is automatically added unless a custom
\c{/external:Wn} is specified).

\N|While the aim of this functionality is to control warnings in external
libraries, the underlying mechanisms currently provided by compilers have
limitations and undesirable side effects. In particular, \c{-isystem} paths
are searched after \c{-I} so translating \c{-I} to \c{-isystem} alters the
search order. This should normally be harmless when using a development build
of a library but may result in a change of semantics for installed
libraries. Also, marking the search path as system has additional (to warning
suppression) effects, see
\l{https://gcc.gnu.org/onlinedocs/cpp/System-Headers.html System Headers} in
the GCC documentation for details. On the MSVC side, \c{/external:W0}
currently does not suppress some warnings (refer to the MSVC documentation for
details).

Another issue is warnings in template instantiations. Each such warning could
be either due to a (potential) issue in the template itself or due to the
template arguments we are instantiating it with. By default, all such warnings
are suppressed and there is currently no way to change this with GCC/Clang
\c{-isystem}. While MSVC provides \c{/external:templates-}, it cannot be
applied on the library by library basis, only globally for the entire
compilation. See MSVC \c{/external:templates-} documentation for more
background on this issue.|

\N|In the future this functionality will be extended to side-building BMIs for
external module interfaces and header units.|

The internal scope can be specified by the project with the
\c{cxx.internal.scope} variable and overridden by the user with the
\c{config.cxx.internal.scope} variable. Note that \c{cxx.internal.scope} must
be specified before loading the \c{cxx} module (\c{cxx.config}, more
precisely) and after which it contains the effective value (see below). For
example:

\
# root.build

cxx.std = latest
cxx.internal.scope = current

using cxx
\

Valid values for \c{cxx.internal.scope} are:

\
current  -- current root scope (where variable is assigned)
base     -- target's base scope
root     -- target's root scope
bundle   -- target's bundle amalgamation
strong   -- target's strong amalgamation
weak     -- target's weak amalgamation
global   -- global scope (everything is internal)
\

Valid values for \c{config.cxx.internal.scope} are the same except for
\c{current}.

Note also that there are \c{[config.]cc.internal.scope} variables that can be
used to specify the internal scope for all the \c{cc}-based modules.

The project's effective internal scope is chosen based on the following
priority list:

\ol|

\li|\c{config.cxx.internal.scope}|

\li|\c{config.cc.internal.scope}|

\li|effective scope from bundle amalgamation|

\li|\c{cxx.internal.scope}|

\li|\c{cc.internal.scope}||

In particular, item #3 allows an amalgamation that bundles a project to
override its internal scope.

\N|If no \c{*.internal.scope} is specified by the project, user, or bundle,
then this functionality is disabled and all libraries are treated as internal
regardless of their location.

While it may seem natural to have this enabled by default, the limitations and
side effects of the underlying mechanisms as well as cases where it would be
undesirable (such as in separate \c{*-tests} projects, see below) all suggest
that explicit opt-in is probably the correct choice.|

The recommended value for a typical project is \c{current}, meaning that only
headers inside the project will be considered internal. The \c{tests}
subproject, if present, will inherit its value from the project (which acts as
a bundle amalgamation), unless it is being built out of source (for example,
to test an installed library).

A project can also whitelist specific libraries using the
\c{cxx.internal.libs} variable. If a library target name (that is, the name
inside \c{lib{\}}) matches any of the wildcard patterns listed in this
variable, then the library is considered internal regardless of its
location. For example (notice that the pattern is quoted):

\
# root.build

cxx.std = latest
cxx.internal.scope = current
cxx.internal.libs = foo 'bar-*'

using cxx
\

Note that this variable should also be set before loading the \c{cxx} module
and there is the common \c{cc.internal.libs} equivalent. However, there are
no \c{config.*} versions nor the override by the bundle amalgamation
semantics.

Typically you would want to whitelist libraries that are developed together
but reside in separate build system projects. In particular, a separate
\c{*-tests} project for a library should whitelist the library being tested if
the internal scope functionality is in use. Another reason to whitelist is to
catch warnings in instantiations of templates that belong to a library that is
otherwise warning-free (see the MSVC \c{/external:templates-} option for
background).

Note also that if multiple libraries are installed into the same location (or
otherwise share the same header search paths, for example, as a family of
libraries), then the whitelist may not be effective.


\h#cc-auto-symexport|Automatic DLL Symbol Exporting|

The \c{bin.def} module (automatically loaded by the \c{c} and \c{cxx} modules
for the \c{*-win32-msvc} targets) provides a rule for generating
symbol-exporting \c{.def} files. This allows automatically exporting all
symbols for all the Windows targets/compilers using the following arrangement
(showing for \c{cxx} in this example):

\
lib{foo}: libul{foo}: {hxx cxx}{**} ...

lib{foo}: def{foo}: include = ($cxx.target.system == 'win32-msvc')
def{foo}: libul{foo}

if ($cxx.target.system == 'mingw32')
  cxx.loptions += -Wl,--export-all-symbols
\

That is, we use the \c{.def} file approach for MSVC (including when building
with Clang) and the built-in support (\c{--export-all-symbols}) for MinGW.

Note that it is also possible to use the \c{.def} file approach for MinGW. In
this case we need to explicitly load the \c{bin.def} module (which should be
done after loading \c{c} or \c{cxx}) and can use the following arrangement:

\
# root.build

using cxx
using bin.def
\

\
lib{foo}: libul{foo}: {hxx cxx}{**} ...

lib{foo}: def{foo}: include = ($cxx.target.class == 'windows')
def{foo}: libul{foo}
\

Note also that this only deals with exporting of the symbols from a DLL. In
order to work, code that uses such a DLL should be able to import the symbols
without explicit \c{__declspec(dllimport)} declarations. This works thanks
to the symbol auto-importing support in Windows linkers. Note, however, that
auto-importing only works for functions and not for global variables.


\h#cc-gcc|GCC Compiler Toolchain|

The GCC compiler id is \c{gcc}.


\h#cc-clang|Clang Compiler Toolchain|

The vanilla Clang compiler id is \c{clang} (including when targeting the MSVC
runtime), Apple Clang compiler id is \c{clang-apple}, and Clang's \c{cl}
compatibility driver (\c{clang-cl}) id is \c{msvc-clang}.

\h2#cc-clang-msvc|Clang Targeting MSVC|

There are two common ways to obtain Clang on Windows: bundled with the MSVC
installation or as a separate installation. If you are using the separate
installation, then the Clang compiler is most likely already in the \c{PATH}
environment variable. Otherwise, if you are using Clang that is bundled with
MSVC, the \c{cc} module will attempt various search strategies described
below. Note, however, that in both cases once the Clang compiler binary
located, the mode (32 or 64-bit) and the rest of the environment (locations of
binary utilities as well as the system headers and libraries) are obtained
by querying Clang.

\N|Normally, if Clang is invoked from one of the Visual Studio command
prompts, then it will use the corresponding Visual Studio version and
environment (it is, however, still up to you to match the mode with the
\c{-m32}/\c{-m64} options, if necessary). Otherwise, Clang will try to locate
the latest version of Visual Studio and Platform SDK and use that (in this
case it matches the environment to the \c{-m32}/\c{-m64} options).  Refer to
Clang documentation for details.|

If you specify the compiler as just \c{config.c=clang} or
\c{config.cxx=clang++} and it is found in the \c{PATH} environment variable or
if you specify it as an absolute path, then the \c{cc} module will use that.

Otherwise, if you are building from one of the Visual Studio development
command prompts, the \c{cc} module will look for the corresponding bundled
Clang (\c{%VCINSTALLDIR%\\Tools\\Llvm\\bin}).

Finally, the \c{cc} module will attempt to locate the latest installed version
of Visual Studio and look for a bundled Clang in there.

The default mode (32 or 64-bit) depends on the Clang configuration and can
be overridden with the \c{-m32}/\c{-m64} options. For example:

\
> b \"config.cxx=clang++ -m64\"
\

The default MSVC runtime selected by the \c{cc} module is multi-threaded
shared (the \c{/MD} option in \c{cl}). Unfortunately, the Clang driver does
not yet provide anything equivalent to the \c{cl} \c{/M*} options (see
\l{https://bugs.llvm.org/show_bug.cgi?id=33273 Clang bug #33273}) and
selection of an alternative runtime has to be performed manually:

\
> rem /MD  - multi-threaded shared (default)
> rem
> b \"config.cxx=clang++ -nostdlib -D_MT -D_DLL\" ^
    config.cc.libs=/DEFAULTLIB:msvcrt

> rem /MDd - multi-threaded debug shared
> rem
> b \"config.cxx=clang++ -nostdlib -D_MT -D_DLL -D_DEBUG\" ^
    config.cc.libs=/DEFAULTLIB:msvcrtd

> rem /MT  - multi-threaded static
> rem
> b \"config.cxx=clang++ -nostdlib -D_MT\" ^
    config.cc.libs=/DEFAULTLIB:libcmt

> rem /MTd - multi-threaded debug static
> rem
> b \"config.cxx=clang++ -nostdlib -D_MT -D_DEBUG\" ^
    config.cc.libs=/DEFAULTLIB:libcmtd
\

By default the MSVC's binary utilities (\c{link} and \c{lib}) are used when
compiling with Clang. It is, however, possible to use LLVM's versions instead,
for example:

\
> b config.cxx=clang++     ^
    config.bin.ld=lld-link ^
    config.bin.ar=llvm-lib
\

In particular, one benefit of using \c{llvm-lib} is support for thin archives
which, if available, is automatically enabled for utility libraries.

\N|While there is basic support for Clang's \c{cl} compatibility driver
(\c{clang-cl}), its use is not recommended. This driver is a very thin wrapper
over the standard Clang interface that does not always recreate the \c{cl}'s
semantics exactly. Specifically, its diagnostics in the \c{/showIncludes} mode
does not match that of \c{cl} in the presence of missing headers. As a result,
\c{clang-cl}'s use, if any, should be limited to projects that do not have
auto-generated headers.

If you need to link with other projects that use \c{clang-cl}, then the
recommended approach is to discover any additional \c{cc1} options passed by
\c{clang-cl} by comparing the \c{-v} output of a test compilation with
\c{clang-cl} and \c{clang}/\c{clang++} and then passing them explicitly
to \c{clang}/\c{clang++}, potentially prefixed with \c{-Xclang}. For example:

\
b \"config.cxx=clang++ -Xclang -fms-volatile ...\"
\

Relevant additional options that are passed by \c{clang-cl} at the time of
this writing:

\
-fno-strict-aliasing
-fstack-protector-strong
-Xclang -fms-volatile
-ffunction-sections
\

|


\h#cc-msvc|MSVC Compiler Toolchain|

The Microsoft VC (MSVC) compiler id is \c{msvc}.

There are several ways to specify the desired MSVC compiler and mode (32 or
64-bit) as well as the corresponding environment (locations of binary
utilities as well as the system headers and libraries).

\N|Unlike other compilers, MSVC compiler (\c{cl}) binaries are
target-specific, that is, there are no \c{-m32}/\c{-m64} options nor something
like the \c{/MACHINE} option available in \c{link}.|

If the compiler is specified as just \c{cl} in \c{config.{c,cxx}} and it is
found in the \c{PATH} environment variable, then the \c{cc} module assumes the
build is performed from one of the Visual Studio development command prompts
and expects the environment (the \c{PATH}, \c{INCLUDE}, and \c{LIB}
environment variables) to already be setup.

If, however, \c{cl} is not found in \c{PATH}, then the \c{cc} module will
attempt to locate the latest installed version of Visual Studio and Platform
SDK and use that in the 64-bit mode.

Finally, if the compiler is specified as an absolute path to \c{cl}, then the
\c{cc} module will attempt to locate the corresponding Visual Studio
installation as well as the latest Platform SDK and use that in the mode
corresponding to the specified \c{cl} executable. Note that to specify an
absolute path to \c{cl} (which most likely contains spaces) we have to
use two levels of quoting:

\
> b \"config.cxx='...\VC\Tools\MSVC\14.23.28105\bin\Hostx64\x86\cl'\"
\

\N|The latter two methods are only available for Visual Studio 15 (2017) and
later and for earlier versions the development command prompt must be used.|

The default MSVC runtime selected by the \c{cc} module is multi-threaded
shared (the \c{/MD} \c{cl} option). An alternative runtime can be selected
by passing one of the \c{cl} \c{/M*} options, for example:

\
> b \"config.cxx=cl /MT\"
\

\h1#module-c|\c{c} Module|

\N{This chapter is a work in progress and is incomplete.}

This chapter describes the \c{c} build system module which provides the C
compilation and linking support. Most of its functionality, however, is
provided by the \l{#module-cc \c{cc}} module, a common implementation for the
C-family languages.

\h#c-config|C Configuration Variables|

The following listing summarizes the \c{c} module configuration variables as
well as the corresponding module-specific variables that are derived from
their values. See also \l{#cc-config C-Common Configuration Variables}.

\
config.c
  c.path
  c.mode

config.c.id
  c.id
  c.id.type
  c.id.variant
  c.class

config.c.version
  c.version
  c.version.major
  c.version.minor
  c.version.patch
  c.version.build

config.c.target
  c.target
  c.target.cpu
  c.target.vendor
  c.target.system
  c.target.version
  c.target.class

config.c.std
  c.std

config.c.poptions
  c.poptions

config.c.coptions
  c.coptions

config.c.loptions
  c.loptions

config.c.aoptions
  c.aoptions

config.c.libs
  c.libs

config.c.internal.scope
  c.internal.scope
\


\h1#module-cxx|\c{cxx} Module|

\N{This chapter is a work in progress and is incomplete.}

This chapter describes the \c{cxx} build system module which provides the C++
compilation and linking support. Most of its functionality, however, is
provided by the \l{#module-cc \c{cc}} module, a common implementation for the
C-family languages.

\h#cxx-config|C++ Configuration Variables|

The following listing summarizes the \c{cxx} module configuration variables as
well as the corresponding module-specific variables that are derived from
their values. See also \l{#cc-config C-Common Configuration Variables}.

\
config.cxx
  cxx.path
  cxx.mode

config.cxx.id
  cxx.id
  cxx.id.type
  cxx.id.variant
  cxx.class

config.cxx.version
  cxx.version
  cxx.version.major
  cxx.version.minor
  cxx.version.patch
  cxx.version.build

config.cxx.target
  cxx.target
  cxx.target.cpu
  cxx.target.vendor
  cxx.target.system
  cxx.target.version
  cxx.target.class

config.cxx.std
  cxx.std

config.cxx.poptions
  cxx.poptions

config.cxx.coptions
  cxx.coptions

config.cxx.loptions
  cxx.loptions

config.cxx.aoptions
  cxx.aoptions

config.cxx.libs
  cxx.libs

config.cxx.internal.scope
  cxx.internal.scope

config.cxx.translate_include
  cxx.translate_include
\


\h#cxx-modules|C++ Modules Support|

This section describes the build system support for C++ modules.

\h2#cxx-modules-intro|Modules Introduction|

The goal of this section is to provide a practical introduction to C++ Modules
and to establish key concepts and terminology.

A pre-modules C++ program or library consists of one or more \i{translation
units} which are customarily referred to as C++ source files. Translation
units are compiled to \i{object files} which are then linked together to
form a program or library.

Let's also recap the difference between an \i{external name} and a \i{symbol}:
External names refer to language entities, for example classes, functions, and
so on. The \i{external} qualifier means they are visible across translation
units.

Symbols are derived from external names for use inside object files. They are
the cross-referencing mechanism for linking a program from multiple,
separately-compiled translation units. Not all external names end up becoming
symbols and symbols are often \i{decorated} with additional information, for
example, a namespace. We often talk about a symbol having to be satisfied by
linking an object file or a library that provides it. Similarly, duplicate
symbol issues may arise if more than one object file or library provides
the same symbol.

What is a C++ module? It is hard to give a single but intuitive answer to this
question.  So we will try to answer it from three different perspectives: that
of a module consumer, a module producer, and a build system that tries to make
those two play nice. But we can make one thing clear at the outset: modules
are a \i{language-level} not a preprocessor-level mechanism; it is \c{import},
not \c{#import}.

One may also wonder why C++ modules, what are the benefits? Modules offer
isolation, both from preprocessor macros and other modules' symbols. Unlike
headers, modules require explicit exportation of entities that will be visible
to the consumers. In this sense they are a \i{physical design mechanism} that
forces us to think how we structure our code. Modules promise significant
build speedups since importing a module, unlike including a header, should be
essentially free. Modules are also the first step to not needing the
preprocessor in most translation units. Finally, modules have a chance of
bringing to mainstream reliable and easy to setup distributed C++ compilation,
since with modules build systems can make sure compilers on the local and
remote hosts are provided with identical inputs.

To refer to a module we use a \i{module name}, a sequence of dot-separated
identifiers, for example \c{hello.core}. While the specification does not
assign any hierarchical semantics to this sequence, it is customary to refer
to \c{hello.core} as a submodule of \c{hello}. We discuss submodules and
provide the module naming guidelines below.

From a consumer's perspective, a module is a collection of external names,
called \i{module interface}, that become \i{visible} once the module is
imported:

\
import hello.core
\

What exactly does \i{visible} mean? To quote the standard: \i{An
import-declaration makes exported declarations [...] visible to name lookup in
the current translation unit, in the same namespaces and contexts [...]. [
Note: The entities are not redeclared in the translation unit containing the
module import declaration. -- end note ]} One intuitive way to think about
this visibility is \i{as if} there were only a single translation unit for the
entire program that contained all the modules as well as all their
consumers. In such a translation unit all the names would be visible to
everyone in exactly the same way and no entity would be redeclared.

This visibility semantics suggests that modules are not a name scoping
mechanism and are orthogonal to namespaces. Specifically, a module can export
names from any number of namespaces, including the global namespace. While the
module name and its namespace names need not be related, it usually makes
sense to have a parallel naming scheme, as discussed below. Finally, the
\c{import} declaration does not imply any additional visibility for names
declared inside namespaces. Specifically, to access such names we must
continue using the standard mechanisms, such as qualification or using
declaration/directive.  For example:

\
import hello.core;        // Exports hello::say().

say ();                   // Error.
hello::say ();            // Ok.

using namespace hello;
say ();                   // Ok.
\

Note also that from the consumer's perspective a module does not provide
any symbols, only C++ entity names. If we use names from a module, then we
may have to satisfy the corresponding symbols using the usual mechanisms:
link an object file or a library that provides them. In this respect, modules
are similar to headers and as with headers, module's use is not limited to
libraries; they make perfect sense when structuring programs. Furthermore,
a library may also have private or implementation modules that are not
meant to be consumed by the library's users.

The producer perspective on modules is predictably more complex. In
pre-modules C++ we only had one kind of translation unit (or source
file). With modules there are three kinds: \i{module interface unit},
\i{module implementation unit}, and the original kind which we will
call a \i{non-module translation unit}.

From the producer's perspective, a module is a collection of module translation
units: one interface unit and zero or more implementation units. A simple
module may consist of just the interface unit that includes implementations
of all its functions (not necessarily inline). A more complex module may
span multiple implementation units.

A translation unit is a module interface unit if it contains an \i{exporting
module declaration}:

\
export module hello.core;
\

A translation unit is a module implementation unit if it contains a
\i{non-exporting module declaration}:

\
module hello.core;
\

While module interface units may use the same file extension as normal source
files, we recommend that a different extension be used to distinguish them as
such, similar to header files. While the compiler vendors suggest various (and
predictably different) extensions, our recommendation is \c{.mxx} for the
\c{.hxx/.cxx} source file naming and \c{.mpp} for \c{.hpp/.cpp}. And if you
are using some other naming scheme, then perhaps now is a good opportunity to
switch to one of the above. Continuing using the source file extension for
module implementation units appears reasonable and that's what we recommend.

A module declaration (exporting or non-exporting) starts a \i{module purview}
that extends until the end of the module translation unit. Any name declared
in a module's purview \i{belongs} to said module. For example:

\
#include <string>                // Not in purview.

export module hello.core;        // Start of purview.

void
say_hello (const std::string&);  // In purview.
\

A name that belongs to a module is \i{invisible} to the module's consumers
unless it is \i{exported}. A name can be declared exported only in a module
interface unit, only in the module's purview, and there are several syntactic
ways to accomplish this. We can start the declaration with the \c{export}
specifier, for example:

\
export module hello.core;

export enum class volume {quiet, normal, loud};

export void
say_hello (const char*, volume);
\

Alternatively, we can enclose one or more declarations into an \i{exported
group}, for example:

\
export module hello.core;

export
{
  enum class volume {quiet, normal, loud};

  void
  say_hello (const char*, volume);
}
\

Finally, if a namespace definition is declared exported, then every name
in its body is exported, for example:

\
export module hello.core;

export namespace hello
{
  enum class volume {quiet, normal, loud};

  void
  say (const char*, volume);
}

namespace hello
{
  void
  impl (const char*, volume); // Not exported.
}
\

Up until now we've only been talking about names belonging to a module. What
about the corresponding symbols? For exported names, the resulting symbols
would be the same as if those names were declared outside of a module's
purview (or as if no modules were used at all). Non-exported names, on the
other hand, have \i{module linkage}: their symbols can be resolved from this
module's units but not from other translation units. They also cannot clash
with symbols for identical names from other modules (and non-modules). This is
usually achieved by decorating the non-exported symbols with the module name.

This ownership model has an important backwards compatibility implication: a
library built with modules enabled can be linked to a program that still uses
headers. And even the other way around: we can build and use a module for a
library that was built with headers.

What about the preprocessor? Modules do not export preprocessor macros,
only C++ names. A macro defined in the module interface unit cannot affect
the module's consumers. And macros defined by the module's consumers cannot
affect the module interface they are importing. In other words, module
producers and consumers are isolated from each other when the preprocessor
is concerned. For example, consider this module interface:

\
export module hello;

#ifndef SMALL
#define HELLO
export void say_hello (const char*);
#endif
\

And its consumer:

\
// module consumer
//
#define SMALL       // No effect.
import hello;

#ifdef HELLO        // Not defined.
...
#endif
\

This is not to say that the preprocessor cannot be used by either, it just
doesn't \"leak\" through the module interface. One practical implication of
this model is the insignificance of the import order.

If a module imports another module in its purview, the imported module's
names are not made automatically visible to the consumers of the importing
module. This is unlike headers and can be surprising. Consider this module
interface as an example:

\
export module hello;

import std.core;

export void
say_hello (const std::string&);
\

And its consumer:

\
import hello;

int
main ()
{
  say_hello (\"World\");
}
\

This example will result in a compile error and the diagnostics may
confusingly indicate that there is no known conversion from a C string to
\"something\" called \c{std::string}. But with the understanding of the
difference between \c{import} and \c{#include} the reason should be clear:
while the module interface \"sees\" \c{std::string} (because it imported its
module), we (the consumer) do not (since we did not). So the fix is to
explicitly import \c{std.core}:

\
import std.core;
import hello;

int
main ()
{
  say_hello (\"World\");
}
\

A module, however, can choose to re-export a module it imports. In this case,
all the names from the imported module will also be visible to the importing
module's consumers. For example, with this change to the module interface the
first version of our consumer will compile without errors (note that whether
this is a good design choice is debatable, as discussed below):

\
export module hello;

export import std.core;

export void
say_hello (const std::string&);
\

One way to think of a re-export is \i{as if} an import of a module also
\"injects\" all the imports said module re-exports, recursively. That's
essentially how most compilers implement it.

Module re-export is the mechanism for assembling bigger modules out of
submodules. As an example, let's say we had the \c{hello.core},
\c{hello.basic}, and \c{hello.extra} modules. To make life easier for users
that want to import all of them we can create the \c{hello} module that
re-exports the three:

\
export module hello;

export
{
  import hello.core;
  import hello.basic;
  import hello.extra;
}
\

Besides starting a module purview, a non-exporting module declaration in the
implementation unit makes non-internal linkage names declared or made visible
in the \i{interface purview} also visible in the \i{implementation purview}.
In this sense non-exporting module declaration acts as an extended
\c{import}. For example:

\
import hello.impl;          // Not visible (exports impl()).

void
extra_impl ();              // Not visible.

export module hello.extra;  // Start of interface purview.

import hello.core;          // Visible (exports core()).

void
extra ();                   // Visible.

static void
extra2 ();                  // Not visible (internal linkage).
\

And this is the implementation unit:

\
module hello.extra;         // Start of implementation purview.

void
f ()
{
  impl ();        // Error.
  extra_impl ();  // Error.
  core ();        // Ok.
  extra ();       // Ok.
  extra2 ();      // Error.
}
\

In particular, this means that while the relative order of imports is not
significant, the placement of imports in the module interface unit relative
to the module declaration can be.

The final perspective that we consider is that of the build system. From its
point of view the central piece of the module infrastructure is the \i{binary
module interface}: a binary file that is produced by compiling the module
interface unit and that is required when compiling any translation unit that
imports this module as well as the module's implementation units.

Then, in a nutshell, the main functionality of a build system when it comes to
modules support is figuring out the order in which all the translation units
should be compiled and making sure that every compilation process is able to
find the binary module interfaces it needs.

Predictably, the details are more complex. Compiling a module interface unit
produces two outputs: the binary module interface and the object file. The
latter contains object code for non-inline functions, global variables, etc.,
that the interface unit may define. This object file has to be linked when
producing any binary (program or library) that uses this module.

Also, all the compilers currently implement module re-export as a shallow
reference to the re-exported module name which means that their binary
interfaces must be discoverable as well, recursively. In fact, currently, all
the imports are handled like this, though a different implementation is at
least plausible, if unlikely.

While the details vary between compilers, the contents of the binary module
interface can range from a stream of preprocessed tokens to something fairly
close to object code. As a result, binary interfaces can be sensitive to the
compiler options and if the options used to produce the binary interface (for
example, when building a library) are sufficiently different compared to the
ones used when compiling the module consumers, the binary interface may be
unusable. So while a build system should strive to reuse existing binary
interfaces, it should also be prepared to compile its own versions \"on the
side\".

This also suggests that binary module interfaces are not a distribution
mechanism and should probably not be installed. Instead, we should install and
distribute module interface sources and build systems should be prepared to
compile them, again, on the side.


\h2#cxx-modules-build|Building Modules|

Compiler support for C++ Modules is still experimental. As a result, it is
currently only enabled if the C++ standard is set to \c{experimental}. After
loading the \c{cxx} module we can check if modules are enabled using the
\c{cxx.features.modules} boolean variable. This is what the relevant
\c{root.build} fragment could look like for a modularized project:

\
cxx.std = experimental

using cxx

assert $cxx.features.modules 'compiler does not support modules'

mxx{*}: extension = mxx
cxx{*}: extension = cxx
\

To support C++ modules the \c{cxx} module (build system) defines several
additional target types. The \c{mxx{\}} target is a module interface unit.
As you can see from the above \c{root.build} fragment, in this project we
are using the \c{.mxx} extension for our module interface files. While
you can use the same extension as for \c{cxx{\}} (source files), this is
not recommended since some functionality, such as wildcard patterns, will
become unusable.

The \c{bmi{\}} group and its \c{bmie{\}}, \c{bmia{\}}, and \c{bmis{\}} members
are used to represent binary module interfaces targets. We normally do not
need to mention them explicitly in our buildfiles except, perhaps, to specify
additional, module interface-specific compile options. We will see some
examples of this below.

To build a modularized executable or library we simply list the module
interfaces as its prerequisites, just as we do for source files. As an
example, let's build the \c{hello} program that we have started in the
introduction (you can find the complete project in the
\l{https://build2.org/pkg/hello Hello Repository} under
\c{mhello}). Specifically, we assume our project contains the following files:

\
// file: hello.mxx (module interface)

export module hello;

import std.core;

export void
say_hello (const std::string&);
\

\
// file: hello.cxx (module implementation)

module hello;

import std.io;

using namespace std;

void
say_hello (const string& name)
{
  cout << \"Hello, \" << name << '!' << endl;
}
\

\
// file: driver.cxx

import std.core;
import hello;

int
main ()
{
  say_hello (\"World\");
}
\

To build a \c{hello} executable from these files we can write the following
\c{buildfile}:

\
exe{hello}: cxx{driver} {mxx cxx}{hello}
\

Or, if you prefer to use wildcard patterns:

\
exe{hello}: {mxx cxx}{*}
\

Alternatively, we can package the module into a library and then link the
library to the executable:

\
exe{hello}: cxx{driver} lib{hello}
lib{hello}: {mxx cxx}{hello}
\

As you might have surmised from this example, the modules support in
\c{build2} automatically resolves imports to module interface units that are
specified either as direct prerequisites or as prerequisites of library
prerequisites.

To perform this resolution without a significant overhead, the implementation
delays the extraction of the actual module name from module interface units
(since not all available module interfaces are necessarily imported by all the
translation units). Instead, the implementation tries to guess which interface
unit implements each module being imported based on the interface file
path. Or, more precisely, a two-step resolution process is performed: first a
best match between the desired module name and the file path is sought and
then the actual module name is extracted and the correctness of the initial
guess is verified.

The practical implication of this implementation detail is that our module
interface files must embed a portion of a module name, or, more precisely, a
sufficient amount of \"module name tail\" to unambiguously resolve all the
modules used in a project. Note also that this guesswork is only performed for
direct module interface prerequisites; for those that come from libraries the
module names are known and are therefore matched exactly.

As an example, let's assume our \c{hello} project had two modules:
\c{hello.core} and \c{hello.extra}. While we could call our interface files
\c{hello.core.mxx} and \c{hello.extra.mxx}, respectively, this doesn't look
particularly good and may be contrary to the file naming scheme used in our
project. To resolve this issue the match of module names to file names is
made \"fuzzy\": it is case-insensitive, it treats all separators (dots, dashes,
underscores, etc) as equal, and it treats a case change as an imaginary
separator. As a result, the following naming schemes will all match the
\c{hello.core} module name:

\
hello-core.mxx
hello_core.mxx
HelloCore.mxx
hello/core.mxx
\

We also don't have to embed the full module name. In our case, for example, it
would be most natural to call the files \c{core.mxx} and \c{extra.mxx} since
they are already in the project directory called \c{hello/}. This will work
since our module names can still be guessed correctly and unambiguously.

If a guess turns out to be incorrect, the implementation issues diagnostics
and exits with an error before attempting to build anything. To resolve this
situation we can either adjust the interface file names or we can specify the
module name explicitly with the \c{cxx.module_name} variable. The latter
approach can be used with interface file names that have nothing in common
with module names, for example:

\
mxx{foobar}@./: cxx.module_name = hello
\

Note also that standard library modules (\c{std} and \c{std.*}) are treated
specially: they are not fuzzy-matched and they need not be resolvable to
the corresponding \c{mxx{\}} or \c{bmi{\}} in which case it is assumed
they will be resolved in an ad hoc way by the compiler. This means that if
you want to build your own standard library module (for example, because
your compiler doesn't yet ship one; note that this may not be supported
by all compilers), then you have to specify the module name explicitly.
For example:

\
exe{hello}: cxx{driver} {mxx cxx}{hello} mxx{std-core}

mxx{std-core}@./: cxx.module_name = std.core
\

When C++ modules are enabled and available, the build system makes sure the
\c{__cpp_modules} feature test macro is defined. Currently, its value is
\c{201703} for VC and \c{201704} for GCC and Clang but this will most likely
change in the future.

One major difference between the current C++ modules implementation in VC and
the other two compilers is the use of the \c{export module} syntax to identify
the interface units. While both GCC and Clang have adopted this new syntax,
VC is still using the old one without the \c{export} keyword. We can use the
\c{__cpp_modules} macro to provide a portable declaration:

\
#if __cpp_modules >= 201704
export
#endif
module hello;
\

Note, however, that the modules support in \c{build2} provides temporary
\"magic\" that allows us to use the new syntax even with VC (don't ask how).

\h2#cxx-modules-symexport|Module Symbols Exporting|

When building a shared library, some platforms (notably Windows) require that
we explicitly export symbols that must be accessible to the library users.
If you don't need to support such platforms, you can thank your lucky stars
and skip this section.

When using headers, the traditional way of achieving this is via an \"export
macro\" that is used to mark exported APIs, for example:

\
LIBHELLO_EXPORT void
say_hello (const string&);
\

This macro is then appropriately defined (often in a separate \"export
header\") to export symbols when building the shared library and to import
them when building the library's users.

The introduction of modules changes this in a number of ways, at least as
implemented by VC (hopefully other compilers will follow suit). While we
still have to explicitly mark exported symbols in our module interface
unit, there is no need (and, in fact, no way) to do the same when said
module is imported. Instead, the compiler automatically treats all
such explicitly exported symbols (note: symbols, not names) as imported.

One notable aspect of this new model is the locality of the export macro: it
is only defined when compiling the module interface unit and is not visible to
the consumers of the module. This is unlike headers where the macro has to
have a unique per-library name (that \c{LIBHELLO_} prefix) because a header
from one library can be included while building another library.

We can continue using the same export macro and header with modules and, in
fact, that's the recommended approach when maintaining the dual, header/module
arrangement for backwards compatibility (discussed below). However, for
modules-only codebases, we have an opportunity to improve the situation in two
ways: we can use a single, keyword-like macro instead of a library-specific
one and we can make the build system manage it for us thus getting rid of the
export header.

To enable this functionality in \c{build2} we set the
\c{cxx.features.symexport} boolean variable to \c{true} before loading the
\c{cxx} module. For example:

\
cxx.std = experimental

cxx.features.symexport = true

using cxx

...
\

Once enabled, \c{build2} automatically defines the \c{__symexport} macro to
the appropriate value depending on the platform and the type of library being
built. As library authors, all we have to do is use it in appropriate places
in our module interface units, for example:

\
export module hello;

import std.core;

export __symexport void
say_hello (const std::string&);
\

As an aside, you may be wondering why can't a module export automatically mean
a symbol export? While you will normally want to export symbols of all your
module-exported names, you may also need to do so for some non-module-exported
ones. For example:

\
export module foo;

__symexport void
f_impl ();

export __symexport inline void
f ()
{
  f_impl ();
}
\

Furthermore, symbol exporting is a murky area with many limitations and
pitfalls (such as auto-exporting of base classes). As a result, it would not
be unreasonable to expect such an automatic module exporting to only further
muddy the matter.


\h2#cxx-modules-install|Modules Installation|

As discussed in the introduction, binary module interfaces are not a
distribution mechanism and installing module interface sources appears to be
the preferred approach.

Module interface units are by default installed in the same location as
headers (for example, \c{/usr/include}). However, instead of relying on a
header-like search mechanism (\c{-I} paths, etc.), an explicit list of
exported modules is provided for each library in its \c{.pc} (\c{pkg-config})
file.

Specifically, the library's \c{.pc} file contains the \c{cxx_modules} variable
that lists all the exported C++ modules in the \c{<name>=<path>} form with
\c{<name>} being the module's C++ name and \c{<path>} \- the module interface
file's absolute path. For example:

\
Name: libhello
Version: 1.0.0
Cflags:
Libs: -L/usr/lib -lhello

cxx_modules = hello.core=/usr/include/hello/core.mxx hello.extra=/usr/include/hello/extra.mxx
\

Additional module properties are specified with variables in the
\c{cxx_module_<property>.<name>} form, for example:

\
cxx_module_symexport.hello.core = true
cxx_module_preprocessed.hello.core = all
\

Currently, two properties are defined. The \c{symexport} property with the
boolean value signals whether the module uses the \c{__symexport} support
discussed above.

The \c{preprocessed} property indicates the degree of preprocessing the module
unit requires and is used to optimize module compilation. Valid values are
\c{none} (not preprocessed), \c{includes} (no \c{#include} directives in the
source), \c{modules} (as above plus no module declarations depend on the
preprocessor, for example, \c{#ifdef}, etc.), and \c{all} (the source is fully
preprocessed). Note that for \c{all} the source may still contain comments and
line continuations.


\h2#cxx-modules-guidelines|Modules Design Guidelines|

Modules are a physical design mechanism for structuring and organizing our
code. Their explicit exportation semantics combined with the way they are
built make many aspects of creating and consuming modules significantly
different compared to headers. This section provides basic guidelines for
designing modules. We start with the overall considerations such as module
granularity and partitioning into translation units then continue with the
structure of typical module interface and implementation units. The following
section discusses practical approaches to modularizing existing code and
providing dual, header/module interfaces for backwards-compatibility.

Unlike headers, the cost of importing modules should be negligible. As a
result, it may be tempting to create \"mega-modules\", for example, one per
library. After all, this is how the standard library is modularized with its
fairly large \c{std.core} and \c{std.io} modules.

There is, however, a significant drawback to this choice: every time we make a
change, all consumers of such a mega-module will have to be recompiled,
whether the change affects them or not. And the bigger the module the higher
the chance that any given change does not (semantically) affect a large
portion of the module's consumers. Note also that this is not an issue for the
standard library modules since they are not expected to change often.

Another, more subtle, issue with mega-modules (which does affect the standard
library) is the inability to re-export only specific interfaces, as will be
discussed below.

The other extreme in choosing module granularity is a large number of
\"mini-modules\". Their main drawback is the tediousness of importation by the
consumers.

The sensible approach is then to create modules of conceptually-related and
commonly-used entities possibly complemented with aggregate modules for ease
of importation. This also happens to be generally good design.

As an example, let's consider an XML library that provides support for both
parsing and serialization. Since it is common for applications to only use one
of the functionalities, it makes sense to provide the \c{xml.parser} and
\c{xml.serializer} modules. While it is not too tedious to import both, for
convenience we could also provide the \c{xml} module that re-exports the two.

Once we are past selecting an appropriate granularity for our modules, the
next question is how to partition them into translation units. A module can
consist of just the interface unit and, as discussed above, such a unit can
contain anything an implementation unit can, including non-inline function
definitions. Some may then view this as an opportunity to get rid of the
header/source separation and have everything in a single file.

There are a number of drawbacks with this approach: Every time we change
anything in the module interface unit, all its consumers have to be
recompiled. If we keep everything in a single file, then every time we change
the implementation we trigger recompilations that would have been avoided had
the implementation been factored out into a separate unit. Note that a build
system in cooperation with the compiler could theoretically avoid such
unnecessary recompilations: if the compiler produces identical binary
interface files when the module interface is unchanged, then the build system
could detect this and skip recompiling the module's consumers.

A related issue with single-file modules is the reduction in the build
parallelization opportunities. If the implementation is part of the interface
unit, then the build system cannot start compiling the module's consumers
until both the interface and the implementation are compiled. On the other
hand, had the implementation been split into a separate file, the build system
could start compiling the module's consumers (as well as the implementation
unit) as soon as the module interface is compiled.

Another issues with combining the interface with the implementation is the
readability of the interface which could be significantly reduced if littered
with implementation details. We could keep the interface separate by moving
the implementation to the bottom of the interface file but then we might as
well move it into a separate file and avoid the unnecessary recompilations or
parallelization issues.

The sensible guideline is then to have a separate module implementation unit
except perhaps for modules with a simple implementation that is mostly
inline/template. Note that more complex modules may have several
implementation units, however, based on our granularity guideline, those
should be rare.

Once we start writing our first real module the immediate question that
normally comes up is where to put \c{#include} directives and \c{import}
declarations and in what order. To recap, a module unit, both interface and
implementation, is split into two parts: before the module declaration which
obeys the usual or \"old\" translation unit rules and after the module
declaration which is the module purview. Inside the module purview all
non-exported declarations have module linkage which means their symbols are
invisible to any other module (including the global module). With this
understanding, consider the following module interface:

\
export module hello;

#include <string>
\

Do you see the problem? We have included \c{<string>} in the module purview
which means all its names (as well as all the names in any headers it might
include, recursively) are now declared as having the \c{hello} module linkage.
The result of doing this can range from silent code blot to strange-looking
unresolved symbols.

The guideline this leads to should be clear: including a header in the module
purview is almost always a bad idea. There are, however, a few types of
headers that may make sense to include in the module purview. The first are
headers that only define preprocessor macros, for example, configuration or
export headers. There are also cases where we do want the included
declarations to end up in the module purview. The most common example is
inline/template function implementations that have been factored out into
separate files for code organization reasons. As an example, consider the
following module interface that uses an export header (which presumably sets
up symbols exporting macros) as well as an inline file:

\
#include <string>

export module hello;

#include <libhello/export.hxx>

export namespace hello
{
  ...
}

#include <libhello/hello.ixx>
\

A note on inline/template files: in header-based projects we could include
additional headers in those files, for example, if the included declarations
are only needed in the implementation. For the reasons just discussed, this
does not work with modules and we have to move all the includes into the
interface file, before the module purview. On the other hand, with modules, it
is safe to use namespace-level using-directives (for example, \c{using
namespace std;}) in inline/template files (and, with care, even in the
interface file).

What about imports, where should we import other modules? Again, to recap,
unlike a header inclusion, an \c{import} declaration only makes exported names
visible without redeclaring them. As result, in module implementation
units, it doesn't really matter where we place imports, in or out of the
module purview. There are, however, two differences when it comes to module
interface units: only imports in the purview are visible to implementation
units and we can only re-export an imported module from the purview.

The guideline is then for interface units to import in the module purview
unless there is a good reason not to make the import visible to the
implementation units. And for implementation units to always import in the
purview for consistency. For example:

\
#include <cassert>

export module hello;

import std.core;

#include <libhello/export.hxx>

export namespace hello
{
  ...
}

#include <libhello/hello.ixx>
\

By putting all these guidelines together we can then create a module interface
unit template:

\
// Module interface unit.

<header includes>

export module <name>;      // Start of module purview.

<module imports>

<special header includes>  // Configuration, export, etc.

<module interface>

<inline/template includes>
\

As well as the module implementation unit template:

\
// Module implementation unit.

<header includes>

module <name>;             // Start of module purview.

<extra module imports>     // Only additional to interface.

<module implementation>
\

Let's now discuss module naming. Module names are in a separate \"name plane\"
and do not collide with namespace, type, or function names. Also, as mentioned
earlier, the standard does not assign a hierarchical meaning to module names
though it is customary to assume module \c{hello.core} is a submodule of
\c{hello} and importing the latter also imports the former.

It is important to choose good names for public modules (that is, modules
packaged into libraries and used by a wide range of consumers) since changing
them later can be costly. We have more leeway with naming private modules
(that is, the ones used by programs or internal to libraries) though it's
worth coming up with a consistent naming scheme here as well.

The general guideline is to start names of public modules with the library's
namespace name followed by a name describing the module's functionality. In
particular, if a module is dedicated to a single class (or, more generally,
has a single primary entity), then it makes sense to use its name as the
module name's last component.

As a concrete example, consider \c{libbutl} (the \c{build2} utility library):
All its components are in the \c{butl} namespace so all its module names start
with \c{butl.} One of its components is the \c{small_vector} class template
which resides in its own module called \c{butl.small_vector}. Another
component is a collection of string parsing utilities that are grouped into
the \c{butl::string_parser} namespace with the corresponding module called
\c{butl.string_parser}.

When is it a good idea to re-export a module? The two straightforward cases
are when we are building an aggregate module out of submodules, for example,
\c{xml} out of \c{xml.parser} and \c{xml.serializer}, or when one module
extends or supersedes another, for example, as \c{std.core} extends
\c{std.fundamental}. It is also clear that there is no need to re-export a
module that we only use in the implementation. The case when we use a module
in our interface is, however, a lot less clear cut.

But before considering the last case in more detail, let's understand the
issue with re-export. In other words, why not simply re-export any module we
import in our interface? In essence, re-export implicitly injects another
module import anywhere our module is imported. If we re-export \c{std.core}
then consumers of our module will also automatically \"see\" all the names
exported by \c{std.core}. They can then start using names from \c{std} without
explicitly importing \c{std.core} and everything will compile until one day
they no longer need to import our module or we no longer need to import
\c{std.core}. In a sense, re-export becomes part of our interface and it is
generally good design to keep interfaces minimal.

And so, at the outset, the guideline is then to only re-export the minimum
necessary. This, by the way, is the reason why it may make sense to divide
\c{std.core} into submodules such as \c{std.core.string}, \c{std.core.vector},
etc.

Let's now discuss a few concrete examples to get a sense of when re-export
might or might not be appropriate. Unfortunately, there does not seem to be a
hard and fast rule and instead one has to rely on their good sense of design.

To start, let's consider a simple module that uses \c{std::string} in its
interface:

\
export module hello;

import std.core;

export namespace hello
{
  void say (const std::string&);
}
\

Should we re-export \c{std.core} (or, \c{std.core.string}) in this case? Most
likely not. If consumers of our module want to use \c{std::string} in order to
pass an argument to our function, then it is natural to expect them to
explicitly import the necessary module. In a sense, this is analogous to
scoping: nobody expects to be able to use just \c{string} (without \c{std::})
because of \c{using namespace hello;}.

So it seems that a mere usage of a name in an interface does not generally
warrant a re-export. The fact that a consumer may not even use this part of
our interface further supports this conclusion.

Let's now consider a more interesting case (inspired by real events):

\
export module small_vector;

import std.core;

template <typename T, std::size_t N>
export class small_vector: public std::vector<T, ...>
{
  ...
};
\

Here we have the \c{small_vector} container implemented in terms of
\c{std::vector} by providing a custom allocator and with most of the functions
derived as is. Consider now this innocent-looking consumer code:

\
import small_vector;

small_vector<int, 1> a, b;

if (a == b) // Error.
  ...
\

We don't reference \c{std::vector} directly so presumably we shouldn't need to
import its module. However, the comparison won't compile: our \c{small_vector}
implementation re-uses the comparison operators provided by \c{std::vector}
(via implicit to-base conversion) but they aren't visible.

There is a palpable difference between the two cases: the first merely uses
\c{std.core} interface while the second is \i{based on} and, in a sense,
\i{extends} it which feels like a stronger relationship. Re-exporting
\c{std.core} (or, better yet, \c{std.core.vector}, should it become available)
does not seem unreasonable.

Note also that there is no re-export of headers nor header inclusion
visibility in the implementation units. Specifically, in the previous example,
if the standard library is not modularized and we have to use it via headers,
then the consumers of our \c{small_vector} will always have to explicitly
include \c{<vector>}. This suggest that modularizing a codebase that still
consumes substantial components (like the standard library) via headers can
incur some development overhead compared to the old, headers-only approach.


\h2#cxx-modules-existing|Modularizing Existing Code|

The aim of this section is to provide practical guidelines to modularizing
existing codebases as well as supporting the dual, header/module interface for
backwards-compatibility.

Predictably, a well modularized (in the general sense) set of headers makes
conversion to C++ modules easier. Inclusion cycles will be particularly hard
to deal with (C++ modules do not allow circular interface dependencies).
Furthermore, as we will see below, if you plan to provide the dual
header/module interface, then having a one-to-one header to module mapping
will simplify this task. As a result, it may make sense to spend some time
cleaning and re-organizing your headers prior to attempting modularization.

Let's first discuss why the modularization approach illustrated by the
following example does not generally work:

\
export module hello;

export
{
#include \"hello.hxx\"
}
\

There are several issue that usually make this unworkable. Firstly, the header
we are trying to export most likely includes other headers. For example, our
\c{hello.hxx} may include \c{<string>} and we have already discussed why
including it in the module purview, let alone exporting its names, is a bad
idea. Secondly, the included header may declare more names than what should be
exported, for example, some implementation details. In fact, it may declare
names with internal linkage (uncommon for headers but not impossible) which
are illegal to export. Finally, the header may define macros which will no
longer be visible to the consumers.

Sometimes, however, this can be the only approach available (for example, if
trying to non-intrusively modularize a third-party library). It is possible to
work around the first issue by \i{pre-including} outside of the module purview
headers that should not be exported. Here we rely on the fact that the second
inclusion of the same header will be ignored. For example:

\
#include <string> // Pre-include to suppress inclusion below.

export module hello;

export
{
#include \"hello.hxx\"
}
\

Needless to say this approach is very brittle and usually requires that you
place all the inter-related headers into a single module. As a result, its use
is best limited to exploratory modularization and early prototyping.

When starting modularization of a codebase there are two decisions we have to
make at the outset: the level of the C++ modules support we can assume and the
level of backwards compatibility we need to provide.

The two modules support levels we distinguish are just modules and modules
with the modularized standard library. The choice we have to make then is
whether to support the standard library only as headers, only as modules, or
both. Note that some compiler/standard library combinations may not be usable
in some of these modes.

The possible backwards compatibility levels are \i{modules-only} (consumption
via headers is no longer supported), \i{modules-or-headers} (consumption
either via headers or modules), and \i{modules-and-headers} (as the previous
case but with support for consuming a library built with modules via headers
and vice versa).

What kind of situations call for the last level? We may need to continue
offering the library as headers if we have a large number of existing
consumers that cannot possibly be all modularized at once (or even ever). So
the situation we may end up in is a mixture of consumers trying to use the
same build of our library with some of them using modules and some \-
headers. The case where we may want to consume a library built with headers
via modules is not as far fetched as it may seem: the library might have been
built with an older version of the compiler (for example, it was installed
from a distribution's package) while the consumer is being built with a
compiler version that supports modules. Note also that as discussed earlier
the modules ownership semantics supports both kinds of such \"cross-usage\".

Generally, compiler implementations do not support mixing inclusion and
importation of the same entities in the same translation unit. This makes
migration tricky if you plan to use the modularized standard library because
of its pervasive use. There are two plausible strategies to handling this
aspect of migration: If you are planning to consume the standard library
exclusively as modules, then it may make sense to first change your entire
codebase to do that. Simply replace all the standard library header inclusions
with importation of the relevant \c{std.*} modules.

The alternative strategy is to first complete the modularization of our entire
project (as discussed next) while continuing consuming the standard library as
headers. Once this is done, we can normally switch to using the modularized
standard library quite easily. The reason for waiting until the complete
modularization is to eliminate header inclusions between components which
would often result in conflicting styles of the standard library consumption.

Note also that due to the lack of header re-export and include visibility
support discussed earlier, it may make perfect sense to only support the
modularized standard library when modules are enabled even when providing
backwards compatibility with headers. In fact, if all the compiler/standard
library implementations that your project caters to support the modularized
standard library, then there is little sense not to impose such a restriction.

The overall strategy for modularizing our own components is to identify and
modularize inter-dependent sets of headers one at a time starting from the
lower-level components. This way any newly modularized set will only depend on
the already modularized ones. After converting each set we can switch its
consumers to using imports keeping our entire project buildable and usable.

While ideally we would want to be able to modularize just a single component
at a time, this does not seem to work in practice because we will have to
continue consuming some of the components as headers. Since such headers can
only be imported out of the module purview, it becomes hard to reason (both
for us and often the compiler) what is imported/included and where. For
example, it's not uncommon to end up importing the module in its
implementation unit which is not something that all the compilers can handle
gracefully.

Let's now explore how we can provide the various levels of backwards
compatibility discussed above. Here we rely on two feature test macros to
determine the available modules support level: \c{__cpp_modules} (modules are
available) and \c{__cpp_lib_modules} (standard library modules are available,
assumes \c{__cpp_modules} is also defined).

If backwards compatibility is not necessary (the \i{modules-only} level), then
we can use the module interface and implementation unit templates presented
earlier and follow the above guidelines. If we continue consuming the standard
library as headers, then we don't need to change anything in this area. If we
only want to support the modularized standard library, then we simply replace
the standard library header inclusions with the corresponding module
imports. If we want to support both ways, then we can use the following
templates. The module interface unit template:

\
// C includes, if any.

#ifndef __cpp_lib_modules
<std includes>
#endif

// Other includes, if any.

export module <name>;

#ifdef __cpp_lib_modules
<std imports>
#endif

<module interface>
\

The module implementation unit template:

\
// C includes, if any.

#ifndef __cpp_lib_modules
<std includes>

<extra std includes>
#endif

// Other includes, if any.

module <name>;

#ifdef __cpp_lib_modules
<extra std imports>        // Only additional to interface.
#endif

<module implementation>
\

For example:

\
// hello.mxx (module interface)

#ifndef __cpp_lib_modules
#include <string>
#endif

export module hello;

#ifdef __cpp_lib_modules
import std.core;
#endif

export void say_hello (const std::string& name);
\

\
// hello.cxx (module implementation)

#ifndef __cpp_lib_modules
#include <string>

#include <iostream>
#endif

module hello;

#ifdef __cpp_lib_modules
import std.io;
#endif

using namespace std;

void say_hello (const string& n)
{
  cout << \"Hello, \" << n << '!' << endl;
}
\

If we need support for symbol exporting in this setup (that is, we are
building a library and need to support Windows), then we can use the
\c{__symexport} mechanism discussed earlier, for example:

\
// hello.mxx (module interface)

...

export __symexport void say_hello (const std::string& name);
\

The consumer code in the \i{modules-only} setup is straightforward: they
simply import the desired modules.

To support consumption via headers when modules are unavailable (the
\i{modules-or-headers} level) we can use the following setup. Here we also
support the dual header/modules consumption for the standard library (if this
is not required, replace \c{#ifndef __cpp_lib_modules} with \c{#ifndef
__cpp_modules} and remove \c{#ifdef __cpp_lib_modules}). The module interface
unit template:

\
#ifndef __cpp_modules
#pragma once
#endif

// C includes, if any.

#ifndef __cpp_lib_modules
<std includes>
#endif

// Other includes, if any.

#ifdef __cpp_modules
export module <name>;

#ifdef __cpp_lib_modules
<std imports>
#endif
#endif

<module interface>
\

The module implementation unit template:

\
#ifndef __cpp_modules
#include <module interface file>
#endif

// C includes, if any.

#ifndef __cpp_lib_modules
<std includes>

<extra std includes>
#endif

// Other includes, if any

#ifdef __cpp_modules
module <name>;

#ifdef __cpp_lib_modules
<extra std imports>        // Only additional to interface.
#endif
#endif

<module implementation>
\

Notice the need to repeat \c{<std includes>} in the implementation file due to
the lack of include visibility discussed above. This is necessary when modules
are enabled but the standard library is not modularized since in this case the
implementation does not \"see\" any of the headers included in the interface.

Besides these templates we will most likely also need an export header that
appropriately defines a module export macro depending on whether modules are
used or not. This is also the place where we can handle symbol exporting. For
example, here is what it could look like for our \c{libhello} library:

\
// export.hxx (module and symbol export)

#pragma once

#ifdef __cpp_modules
#  define LIBHELLO_MODEXPORT export
#else
#  define LIBHELLO_MODEXPORT
#endif

#if   defined(LIBHELLO_SHARED_BUILD)
#  ifdef _WIN32
#    define LIBHELLO_SYMEXPORT __declspec(dllexport)
#  else
#    define LIBHELLO_SYMEXPORT
#  endif
#elif defined(LIBHELLO_SHARED)
#  ifdef _WIN32
#    define LIBHELLO_SYMEXPORT __declspec(dllimport)
#  else
#    define LIBHELLO_SYMEXPORT
#  endif
#else
#  define LIBHELLO_SYMEXPORT
#endif
\

And this is the module that uses it and provides the dual header/module
support:

\
// hello.mxx (module interface)

#ifndef __cpp_modules
#pragma once
#endif

#ifndef __cpp_lib_modules
#include <string>
#endif

#ifdef __cpp_modules
export module hello;

#ifdef __cpp_lib_modules
import std.core;
#endif
#endif

#include <libhello/export.hxx>

LIBHELLO_MODEXPORT namespace hello
{
  LIBHELLO_SYMEXPORT void say (const std::string& name);
}
\

\
// hello.cxx (module implementation)

#ifndef __cpp_modules
#include <libhello/hello.mxx>
#endif

#ifndef __cpp_lib_modules
#include <string>

#include <iostream>
#endif

#ifdef __cpp_modules
module hello;

#ifdef __cpp_lib_modules
import std.io;
#endif
#endif

using namespace std;

namespace hello
{
  void say (const string& n)
  {
    cout << \"Hello, \" << n << '!' << endl;
  }
}
\

The consumer code in the \i{modules-or-headers} setup has to use either
inclusion or importation depending on the modules support availability, for
example:

\
#ifdef __cpp_modules
import hello;
#else
#include <libhello/hello.mxx>
#endif
\

Predictably, the final backwards compatibility level (\i{modules-and-headers})
is the most onerous to support. Here existing consumers have to continue
working with the modularized version of our library which means we have to
retain all the existing header files. We also cannot assume that just because
modules are available they are used (a consumer may still prefer headers),
which means we cannot rely on (only) the \c{__cpp_modules} and
\c{__cpp_lib_modules} macros to make the decisions.

One way to arrange this is to retain the headers and adjust them according to
the \i{modules-or-headers} template but with one important difference: instead
of using the standard module macros we use our custom ones (and we can also
have unconditional \c{#pragma once}). For example:

\
// hello.hxx (module header)

#pragma once

#ifndef LIBHELLO_LIB_MODULES
#include <string>
#endif

#ifdef LIBHELLO_MODULES
export module hello;

#ifdef LIBHELLO_LIB_MODULES
import std.core;
#endif
#endif

#include <libhello/export.hxx>

LIBHELLO_MODEXPORT namespace hello
{
  LIBHELLO_SYMEXPORT void say (const std::string& name);
}
\

Now if this header is included (for example, by an existing consumer) then
none of the \c{LIBHELLO_*MODULES} macros will be defined and the header will
act as, well, a plain old header. Note that we will also need to make the
equivalent change in the export header.

We also provide the module interface files which appropriately define the two
custom macros and then simply includes the corresponding headers:

\
// hello.mxx (module interface)

#ifdef __cpp_modules
#define LIBHELLO_MODULES
#endif

#ifdef __cpp_lib_modules
#define LIBHELLO_LIB_MODULES
#endif

#include <libhello/hello.hxx>
\

The module implementation unit can remain unchanged. In particular, we
continue including \c{hello.mxx} if modules support is unavailable. However,
if you find the use of different macros in the header and source files
confusing, then instead it can be adjusted as follows (note also that now we
are including \c{hello.hxx}):

\
// hello.cxx (module implementation)

#ifdef __cpp_modules
#define LIBHELLO_MODULES
#endif

#ifdef __cpp_lib_modules
#define LIBHELLO_LIB_MODULES
#endif

#ifndef LIBHELLO_MODULES
#include <libhello/hello.hxx>
#endif

#ifndef LIBHELLO_LIB_MODULES
#include <string>

#include <iostream>
#endif

#ifdef LIBHELLO_MODULES
module hello;

#ifdef LIBHELLO_LIB_MODULES
import std.io;
#endif
#endif

...
\

In this case it may also make sense to factor the \c{LIBHELLO_*MODULES} macro
definitions into a common header.

In the \i{modules-and-headers} setup the existing consumers that would like to
continue using headers don't require any changes. And for those that would
like to use modules if available the arrangement is the same as for the
\i{modules-or-headers} compatibility level.

If our module needs to \"export\" macros then the recommended approach is to
simply provide an additional header that the consumer includes. While it might
be tempting to also wrap the module import into this header, some may prefer
to explicitly import the module and include the header, especially if the
macros may not be needed by all consumers. This way we can also keep the
header macro-only which means it can be included freely, in or out of module
purviews.


\h1#module-in|\c{in} Module|

The \c{in} build system module provides support for \c{.in} (input) file
preprocessing. Specifically, the \c{.in} file can contain a number of
\i{substitutions} \- build system variable names enclosed with the
substitution symbol (\c{$} by default) \- which are replaced with the
corresponding variable values to produce the output file. For example:

\
# build/root.build

using in
\

\
// config.hxx.in

#define TARGET \"$cxx.target$\"
\

\
# buildfile

hxx{config}: in{config}
\

The \c{in} module defines the \c{in{\}} target type and implements the \c{in}
build system rule.

While we can specify the \c{.in} extension explicitly, it is not necessary
because the \c{in{\}} target type implements \i{target-dependent search} by
taking into account the target it is a prerequisite of. In other words, the
following dependency declarations produce the same result:

\
hxx{config}:     in{config}
hxx{config.hxx}: in{config}
hxx{config.hxx}: in{config.hxx.in}
\

By default the \c{in} rule uses \c{$} as the substitution symbol. This can be
changed using the \c{in.symbol} variable. For example:

\
// data.cxx.in

const char data[] = \"@data@\";
\

\
# buildfile

cxx{data}: in{data}
{
  in.symbol = '@'
  data = 'Hello, World!'
}
\

Note that the substitution symbol must be a single character.

The default substitution mode is strict. In this mode every substitution
symbol is expected to start a substitution with unresolved (to a variable
value) names treated as errors. The double substitution symbol (for example,
\c{$$}) serves as an escape sequence.

The substitution mode can be relaxed using the \c{in.substitution} variable.
Its valid values are \c{strict} (default) and \c{lax}. In the lax mode a pair
of substitution symbols is only treated as a substitution if what's between
them looks like a build system variable name (that is, it doesn't contain
spaces, etc). Everything else, including unterminated substitution symbols, is
copied as is. Note also that in this mode the double substitution symbol is
not treated as an escape sequence.

The lax mode is mostly useful when trying to reuse existing \c{.in} files from
other build systems, such as \c{autoconf}. Note, however, that the lax mode is
still stricter than \c{autoconf}'s semantics which also leaves unresolved
substitutions as is. For example:

\
# buildfile

h{config}: in{config} # config.h.in
{
  in.symbol = '@'
  in.substitution = lax

  CMAKE_SYSTEM_NAME = $c.target.system
  CMAKE_SYSTEM_PROCESSOR = $c.target.cpu
}
\

The \c{in} rule tracks changes to the input file as well as the substituted
variable values and automatically regenerates the output file if any were
detected. Substituted variable values are looked up starting from the
target-specific variables. Typed variable values are converted to string
using the corresponding \c{builtin.string()} function overload before
substitution.

Using an undefined variable in a substitution is an error. Using a \c{null}
value in a substitution is also an error unless the fallback value is
specified with the \c{in.null} variable. For example:

\
# buildfile

h{config}: in{config}
{
  in.null = '' # Substitute null values with empty string.
}
\

A number of other build system modules, for example, \l{#module-version
\c{version}} and \l{#module-bash \c{bash}}, are based on the \c{in} module and
provide extended functionality. The \c{in} preprocessing rule matches any
\c{file{\}}-based target that has the corresponding \c{in{\}} prerequisite
provided none of the extended rules match.


\h1#module-bash|\c{bash} Module|

The \c{bash} build system module provides modularization support for \c{bash}
scripts. It is based on the \l{#module-in \c{in}} build system module and
extends its preprocessing rule with support for \i{import substitutions} in
the \c{@import\ <module>@} form. During preprocessing, such imports are
replaced with suitable \c{source} builtin calls. For example:

\
# build/root.build

using bash
\

\
# hello/say-hello.bash

function say_hello ()
{
  echo \"Hello, $1!\"
}
\

\
#!/usr/bin/env bash

# hello/hello.in

@import hello/say-hello@

say_hello 'World'
\

\
# hello/buildfile

exe{hello}: in{hello} bash{say-hello}
\

By default the \c{bash} preprocessing rule uses the lax substitution mode and
\c{@} as the substitution symbol but this can be overridden using the standard
\c{in} module mechanisms.

In the above example, \c{say-hello.bash} is a \i{module}. By convention,
\c{bash} modules have the \c{.bash} extension and we use the \c{bash{\}}
target type (defined by the \c{bash} build system module) to refer to them in
buildfiles.

The \c{say-hello.bash} module is \i{imported} by the \c{hello} script with the
\c{@import\ hello/say-hello@} substitution. The \i{import path}
(\c{hello/say-hello} in our case) is a relative path to the module file within
the project. Its first component (\c{hello} in our case) must be the project
base name and the \c{.bash} module extension can be omitted. \N{The constraint
placed on the first component of the import path is required to implement
importation of installed modules, as discussed below.}

During preprocessing, the import substitution will be replaced with a
\c{source} builtin call and the import path resolved to one of the \c{bash{\}}
prerequisites from the script's dependency declaration. The actual module path
used in \c{source} depends on whether the script is preprocessed for
installation. If it's not (development build), then the absolute path to the
module file is used. Otherwise, a path relative to the sourcing script's
directory is derived. This allows installed scripts and their modules to be
moved around.

\N|The derivation of the sourcing script's directory works even if the script
is executed via a symbolic link from another directory. Implementing this,
however, requires \c{readlink(1)} with support for the \c{-f} option. One
notable platform that does not provide such \c{readlink(1)} by default is Mac
OS. The script, however, can provide a suitable implementation as a function.
See the \c{bash} module tests for a sample implementation of such a function.|

By default, \c{bash} modules are installed into a subdirectory of the \c{bin/}
installation directory named as the project base name. For instance, in the
above example, the script will be installed as \c{bin/hello} and the module as
\c{bin/hello/say-hello.bash} with the script sourcing the module relative to
the \c{bin/} directory. Note that currently it is assumed the script and all
its modules are installed into the same \c{bin/} directory.

Naturally, modules can import other modules and modules can be packaged into
\i{module libraries} and imported using the standard build system import
mechanism. For example, we could factor the \c{say-hello.bash} module into a
separate \c{libhello} project:

\
# build/export.build

$out_root/
{
  include libhello/
}

export $src_root/libhello/$import.target
\

\
# libhello/say-hello.bash

function hello_say_hello ()
{
  echo \"Hello, $1!\"
}
\

And then import it in a module of our \c{hello} project:

\
# hello/hello-world.bash.in

@import libhello/say-hello@

function hello_world ()
{
  hello_say_hello 'World'
}
\

\
#!/usr/bin/env bash

# hello/hello.in

@import hello/hello-world@

hello_world
\

\
# hello/buildfile

import mods = libhello%bash{say-hello}

exe{hello}:        in{hello}       bash{hello-world}
bash{hello-world}: in{hello-world} $mods
\

The \c{bash} preprocessing rule also supports importation of installed modules
by searching in the \c{PATH} environment variable.

By convention, \c{bash} module libraries should use the \c{lib} name prefix,
for example, \c{libhello}. If there is also a native library (that is, one
written in C/C++) that provides the same functionality (or the \c{bash}
library is a language binding for said library), then it is customary to add
the \c{.bash} extension to the \c{bash} library name, for example,
\c{libhello.bash}. Note that in this case the project base name is
\c{libhello}.

Modules can be \i{private} or \i{public}. Private modules are implementation
details of a specific project and are not expected to be imported from other
projects. The \c{hello/hello-world.bash.in} module above is an example of a
private module. Public modules are meant to be used by other projects and are
normally packaged into libraries, like the \c{libhello/say-hello.bash} module
above.

Public modules must take care to avoid name clashes. Since \c{bash} does not
have a notion of namespaces, the recommended way is to prefix all module
functions (and global variables, if any) with the library name (without the
\c{lib} prefix), like in the \c{libhello/say-hello.bash} module above.

While using such decorated function names can be unwieldy, it is relatively
easy to create wrappers with shorter names and use those instead. For example:

\
@import libhello/say-hello@

function say_hello () { hello_say_hello \"$@\"; }
\

A module should normally also prevent itself from being sourced multiple
times. The recommended way to achieve this is to begin the module with a
\i{source guard}. For example:

\
# libhello/say-hello.bash

if [ \"$hello_say_hello\" ]; then
  return 0
else
  hello_say_hello=true
fi

function hello_say_hello ()
{
  echo \"Hello, $1!\"
}
\

The \c{bash} preprocessing rule matches \c{exe{\}} targets that have the
corresponding \c{in{\}} and one or more \c{bash{\}} prerequisites as well as
\c{bash{\}} targets that have the corresponding \c{in{\}} prerequisite (if you
need to preprocess a script that does not depend on any modules, you can use
the \c{in} module's rule).
"