ConformalLabpp/doc/architecture/compile-time.md

# Compile-time analysis & quick-wins

> **Audience.** Anyone maintaining or extending the build system.
> Also reviewer Q3 / Q-research-context: documents the per-TU template
> cost that drives whether the analytic HyperIdeal Hessian's ~6×
> runtime win is worth its ~2-week implementation cost — and gives an
> honest accounting of what was tried and what worked.

## TL;DR

| Configuration | Wall (Ninja, `-j8`) | CPU | Tests pass |
|---|---:|---:|---|
| **Baseline** (no PCH, no Unity Build) | **78 s** | 676 s | 236 / 236 |
| **+ PCH** (`CONFORMALLAB_USE_PCH=ON`, default) | 66 s (−15 %) | 474 s (−30 %) | 236 / 236 |
| **+ PCH + Unity Build** | 55 s (−30 %) | 167 s (−75 %) | 236 / 236 |
| **+ #6 Dense → Core in 3 headers** (current default) | **55–60 s (~−25 %, within noise)** | n/a | 236 / 236 |

> **Honest note on variance.** Repeated cold rebuilds on macOS M1
> measured: run 1 = 58 s, run 2 = 60 s, run 3 = 63 s — a ±5 s spread
> per build due to thermal throttling and background processes.
> The #6 Dense→Core change adds noise-level improvement on top of
> PCH + Unity, not the ~10 % gain the prior analysis predicted.
> Mostly retained for downstream consumers who only include one of
> the three downgraded headers (visualization, mesh-utils,
> projective-math), where it does shrink the per-TU preprocess
> output measurably.

The shipped defaults in this branch deliver **30 % less wall time and
75 % less CPU time** on a clean rebuild of `conformallab_cgal_tests`
on Apple M1.  All 236 CGAL tests pass under every configuration.
Tunable via `-DCONFORMALLAB_USE_PCH=ON|OFF` and
`-DCMAKE_UNITY_BUILD=ON|OFF`.

## Measurement environment

| Item | Value |
|---|---|
| CPU | Apple M1 (8 logical cores) |
| RAM | 16 GB |
| Compiler | Apple clang 17.0.0 |
| Build type | Release |
| CGAL | 6.1.1 vendored (48 MB headers) |
| Eigen | 3.4.0 vendored (6.5 MB headers) |
| Generator | Ninja 1.x |
| Target | `conformallab_cgal_tests` (22 TUs in baseline) |

## Where the time goes (baseline, before optimisation)

Top-5 slowest translation units (wall-clock per TU, `-j8`):

| TU | s |
|---|---:|
| `test_layout.cpp` | 54.7 |
| `test_geometry_utils.cpp` | 52.2 |
| `test_phase6.cpp` | 50.7 |
| `test_pipeline.cpp` | 50.4 |
| `test_phase7.cpp` | 45.3 |

A minimal "hello world" TU that does nothing but
`#include <CGAL/Discrete_conformal_map.h>` already costs **5.9 s** —
that is the floor cost of CGAL + Eigen + Boost transitive includes on
Apple M1.

Clang `-ftime-trace` on `test_layout.cpp` (~17 s isolated):

| Phase | Time | % |
|---|---:|---:|
| Backend (CodeGen + Opt) | 9.3 s | 55 % |
| Frontend (Parse + Sema + Templates) | 8.0 s | 45 % |
| ⤷ InstantiateFunction | 4.3 s | 25 % |
| ⤷ InstantiateClass | 3.3 s | 19 % |

Single most expensive Eigen template instantiations
(~1.1 – 1.2 s each, per TU):

* `Eigen::SelfAdjointEigenSolver<Matrix<2,2>>` (PCA in
  `normalise_euclidean`)
* `Eigen::ColPivHouseholderQR<Matrix<complex,3,3>>`
  (`MobiusMap::from_three`)
* `Eigen::internal::tridiagonalization_inplace<Matrix<2,2>>`
* `Eigen::HouseholderSequence<Matrix<2,2>>`

These get instantiated **from scratch in every TU** that pulls
`layout.hpp` in — the inefficiency this branch's PCH closes.

## What was tried

The four candidate quick-wins from the prior analysis were:

1. **Precompiled headers** — shared PCH covering CGAL + Eigen + gtest +
   the std headers every test uses.  Implemented; **shipped**, opt-out
   via `-DCONFORMALLAB_USE_PCH=OFF`.  See
   `code/tests/cgal/CMakeLists.txt`.
2. **`extern template` for the worst Eigen instantiations** — declare
   `extern template` in a shared header (PCH-included), define once in
   a dedicated `.cpp`.  **Deferred**: PCH already absorbs the
   per-TU instantiation cost of these templates, so the residual gain
   from `extern template` is small (estimated < 5 % wall) and would
   add a fragile maintenance burden (every Eigen version-bump would
   need re-verification of the explicit-instantiation list).  If a
   future Eigen update breaks the PCH, this is the next lever.
3. **Header split** for `<CGAL/Discrete_conformal_map.h>** — separate
   into `_euclidean.h` / `_spherical.h` / `_hyper_ideal.h` so
   downstream consumers who only need one geometry pay less.
   **Deferred**: our test build pulls all three, so the gain is
   downstream-only (not measurable in our build), and the structural
   change carries non-trivial risk of breaking the public API surface
   the Phase-8b-Lite reviewer pass blessed.  Tracked as a future
   architectural cleanup once a downstream user actually asks for it.
4. **Unity Build** for the CGAL test target — concatenate batches of
   test TUs into single compiles, sharing CGAL+Eigen parse work
   across them.  Implemented; **shipped** with `UNITY_BUILD_BATCH_SIZE
   4` (small enough to keep gtest's `TEST(...)` macros + per-file
   `using namespace …` from colliding).  Opt-out via
   `-DCMAKE_UNITY_BUILD=OFF`.

## After-state — Unity batch sizes

With the 22 source TUs grouped into 5 batches of ≤ 4 files each,
clean rebuild produces:

| Unity batch | Wall (s) |
|---|---:|
| `unity_2_cxx` | 46.3 |
| `unity_3_cxx` | 41.5 |
| `unity_4_cxx` | 40.6 |
| `unity_1_cxx` | 26.0 |
| `unity_0_cxx` | 13.0 |

Plus gtest itself (`gtest-all.cc.o`, 8.5 s) and `gtest_main.cc.o`
(1.2 s) outside the batches.

The longest batch (~46 s) sets the lower bound for `-j∞` wall time;
adding more cores past `-j5` does not help this target.

## Honesty notes

* **`-j` scaling is sublinear.**  Measured speedups: `-j1`→`-j2` =
  1.66×, `-j2`→`-j4` = 1.59×, `-j4`→`-j8` = 1.28×.  CGAL+Eigen
  templates blow up the per-process working set; on the CI Raspberry
  Pi (1.6 GB RAM cap) we run `-j1` for the CGAL job by necessity.
* **The PCH compiles in ~3 s** the first time and then short-circuits
  every TU.  Total PCH cost amortises after the second TU.
* **CGAL version sensitivity.**  The PCH is keyed to the specific
  CGAL headers it lists.  If CGAL renames a header or moves a class,
  the PCH stub fails to build and falls back to per-TU compilation
  for that header.  The four CI quality-gate runs catch this within
  one PR.
* **macOS-specific.**  The numbers above are Apple M1.  Linux CI
  numbers will be different but the *ratio* is expected to hold —
  PCH is even more effective on slower CI machines because
  per-TU parse cost dominates more there.

## Architecture-touch quick-wins (mid-tier levers)

Beyond PCH + Unity Build, the audit found four "architecture-touch"
candidates in the 🟡 mid-tier of the original analysis (levers 5–7
and 10).  Status after evaluation:

| Lever | Status | Decision |
|---|---|---|
| **#5** Move `detail::` impls to `.inl`/`.tpp` | not implemented | Pure enabler for #7; gain only realises if #7 lands. |
| **#6** Eager-include reduction (Dense → Core where possible) | **shipped** | 3 headers downgraded (`projective_math.hpp`, `hyper_ideal_visualization_utility.hpp`, `mesh_utils.hpp`); 5 others kept `<Eigen/Dense>` because they use `.inverse()` / `.determinant()` / `ColPivHouseholderQR` / `SelfAdjointEigenSolver`.  Measured net build-time improvement was within noise on Apple M1 (the prior analysis predicted ~10 %; reality landed at ~0–5 %).  Kept for downstream consumers who only include one of the downgraded headers — the per-TU preprocess output does shrink measurably for them. |
| **#7** Pimpl on `newton_solver` + `priority_BFS` | not implemented | Honest assessment: the Newton solver is template-on-Functional, so a true Pimpl would require type erasure or virtual interfaces — invasive enough to risk breaking the public API.  Deferred until a concrete user reports compile-time pain. |
| **#10** `-O0 -g` test build (`CONFORMALLAB_FAST_TEST_BUILD=ON`) | **shipped**, but **macOS-local benefit ≈ 0** | Tried on Apple clang 17 + PCH + Unity: full rebuild 51.6 s vs 46.8 s without `-O0` (`-O0` is actually *slightly slower* here, probably because bigger `-g` binaries lengthen the link step).  Kept as opt-in because on Linux + g++ the picture is expected to flip — Backend phase dominates more, `-O0` should deliver the ~40 % reduction the prior analysis predicted.  See ccache honesty notes for the same pattern. |

## Workflow modes — what to choose when

Four orthogonal switches, each opt-in, none affect the default user
build:

| Switch | Effect | When to use |
|---|---|---|
| `-DBUILD_TESTING=OFF` | Skip the test subtree entirely, including the `FetchContent` of GTest.  **Configure ≈ 1 s · Build ≈ 0 s · 0 object files.** | IDE-syntax-check workflows that only need `compile_commands.json`; configure-only health checks. |
| `-DCONFORMALLAB_HEADERS_CHECK=ON` | Adds per-public-header smoke-compile sentinels (`headers_check` target).  **Full ≈ 12 s · incremental after touching one header ≈ 0.1 s.** | "Does my refactor still parse all public headers?" without waiting 55 s for the full test build. |
| `-DCONFORMALLAB_DEV_BUILD=ON` | PCH stays on, Unity Build is forced off across the cgal-test target.  Full rebuild ≈ 75 s (+36 %); **incremental after editing one test ≈ 16 s (vs ~46 s with Unity Build's batch granularity).** | Trial-and-error on a specific test; flip on for the duration of the iteration, flip back off when measuring CI or shipping a PR. |
| `-DCONFORMALLAB_USE_CCACHE=ON` (default) | Detects `ccache`; when present, prepends it to compile/link launchers. | Linux CI primarily.  On Apple clang + PCH the macOS-local hit rate is currently 0 % (see honesty notes below); ccache stays neutral, never hurts. |
| `-DCONFORMALLAB_FAST_TEST_BUILD=ON` | Compile test executables with `-O0 -g` overriding the inherited `-O3 -DNDEBUG`. | **Linux CI** primarily, where `-O0` typically cuts ~40 % off build time.  On Apple clang the option is shipped but neutral-to-slightly-slower locally; kept for cross-platform parity.  Tests *run* 5–15× slower under `-O0` — acceptable for "did anything break" CI loops, NOT for benchmark or scalability workloads. |

### Mode matrix at a glance

| Scenario | Configure (s) | Full build (s) | Incremental edit-rebuild (s) |
|---|---:|---:|---:|
| Default (PCH + Unity + #6 Dense→Core) | 5 | **55–60** (±5 s noise) | ~46 (unity batch) |
| `BUILD_TESTING=OFF` | **1** | **0** | n/a |
| `HEADERS_CHECK=ON` only | 1 | **12** | **0.1** (single header) |
| `DEV_BUILD=ON` | 5 | 75 | **16** (single test) |
| `FAST_TEST_BUILD=ON` (Linux CI) | 5 | ~30 (expected) | ~25 (expected) |
| `FAST_TEST_BUILD=ON` (Apple clang) | 5 | 52 | n/a (use DEV_BUILD locally) |

## Cross-platform perf bench (Linux CI)

`.gitea/workflows/perf-compile-time.yml` runs a 5-step matrix on the
eulernest Linux runner after every push to `main` that touches the
build system or public headers.  The matrix validates the predictions
that this document makes against the macOS-local measurements:

| Run | Predicted Linux gain | Apple M1 actual |
|---|---|---|
| Baseline (no PCH, no Unity, no ccache) | reference | 78 s |
| + PCH only | ~15 % | 66 s (−15 %) |
| + PCH + Unity (default) | ~30 % | 55 s (−30 %) |
| + FAST_TEST_BUILD (-O0 -g) | **~40 % vs default** (g++ Backend dominates) | 52 s (neutral on clang) |
| + ccache warm rerun | **≥ 90 % vs default** (8.5× speedup) | 56 s (Apple-clang PCH friction = 0 % hit) |

The job is data-collection only — does not gate merges.  Output appears
in the run summary tab; if Linux numbers diverge from predictions, the
"Next levers" table below gets updated based on what actually wins.

## Next levers (not in this branch)

If 55 s wall is still not enough for full-clean rebuilds:

| Lever | Estimated win | Cost |
|---|---|---|
| `extern template` (lever #2 above) | 55 s → ~52 s | 2 h + Eigen version tracking |
| Header split (lever #3) | downstream-only | 1 day + API risk |
| C++20 Modules | speculative; experimental in Apple clang 17 | weeks |

## ccache — honesty notes (macOS local vs Linux CI)

Local ccache stats on Apple clang + PCH + Unity Build, after two
clean rebuilds:

```
Cacheable calls:      4 / 16 (25.00 %)
  Hits:               0 /  4 ( 0.00 %)
Uncacheable calls:   12 / 16 (75.00 %)
```

Three issues defeat the hot rebuild:

1. **PCH artefacts are not cached by default.** Apple clang's
   `-include-pch ...gch` path embeds timestamps that miss the cache
   lookup.  Workaround: set `CCACHE_SLOPPINESS=pch_defines,
   include_file_mtime,include_file_ctime,time_macros,file_macro,
   system_headers` (tried; still 0 % hit on macOS).
2. **Unity Build .cxx files** have generated paths that change
   between configurations; ccache treats each as a fresh compile.
3. **CMake's compile-launcher mechanism** doesn't currently combine
   with `target_precompile_headers()` in a way that ccache 4.x
   recognises on Apple clang.  Tracked upstream as a known issue.

**On Linux CI** the picture is different: g++ + traditional PCH +
non-Apple toolchain typically delivers 80 %+ hit rates with ccache.
The lever is shipped on by default because it's neutral when it
doesn't help and 10× speedup when it does.

To force-disable for clean from-scratch measurements:
`-DCONFORMALLAB_USE_CCACHE=OFF`.