docs: centralise test counts + add release-policy + remove stale stub references

Two complementary improvements aimed at reducing recurring maintenance
overhead:

1. **Test-count centralisation** — `doc/api/tests.md` is now the
   single source of truth for the test counts.  All other docs
   (README, CLAUDE.md, doc/contributing.md, doc/getting-started.md,
   doc/math/validation.md, doc/math/validation-protocol.md,
   scripts/try_it.sh) use qualitative phrasing + a link instead of
   hardcoded numbers.  The previous regime had eight places with
   "227 CGAL tests, 23 non-CGAL tests" that drifted apart across
   releases (the v0.9.0 release-prep needed to touch nine files).

2. **Versioning policy** — `doc/release-policy.md` (new, ~250 lines)
   formalises:
   * SemVer rules for the pre-1.0 and post-1.0 phases.
   * Phase-milestone → MINOR-bump mapping (v0.10.0 → Phase 9c, …).
   * Single-source-of-truth table for moving numbers (test counts,
     version, date).
   * Step-by-step release process (the recipe that worked for v0.9.0
     after the false-start with PR #11/#12).
   * Hotfix policy + post-1.0 deprecation policy.
   * Known failure modes and how to recover from them.

Plus a small CI gate:

3. **scripts/check-test-counts.sh** — verifies the totals in
   doc/api/tests.md match `ctest` output.  Re-uses existing build-cgal/
   if present.  Exit 0 on match, 1 on divergence with recovery hints.
   Cheap enough (~30 s) to run on every PR.

Other cleanups
──────────────
* code/tests/cgal/CMakeLists.txt — stale "Test 7 (genus-2 homology)
  as GTEST_SKIP stub until Phase 8" comment removed; that test landed
  as HomologyGenerators.Genus2_FourCutEdges in Phase 7.
* CLAUDE.md — "test-fast also runs stubs" Known Quirks entry updated
  to reflect the v0.9.0 stub cleanup (no GTEST_SKIPs remain).
* CLAUDE.md doc map — new entry for doc/release-policy.md.

Stubs audit
───────────
Zero GTEST_SKIP() calls remain in the codebase as of this commit.
The only references to stubs are in historical documentation
(CHANGELOG.md v0.7.0 entry, doc/roadmap/* "deferred to research-track"
notes) — those are intended.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Tarik Moussa
2026-05-22 07:59:10 +02:00
parent e67ccd6b9d
commit 0f78d181e1
10 changed files with 326 additions and 17 deletions

90
scripts/check-test-counts.sh Executable file
View File

@@ -0,0 +1,90 @@
#!/usr/bin/env bash
# scripts/check-test-counts.sh
#
# CI gate that verifies the totals in `doc/api/tests.md` match the
# actual `ctest` output. Catches the canonical test-count document
# going stale even if no other doc hardcodes the number.
#
# Usage:
# bash scripts/check-test-counts.sh
#
# Exit codes:
# 0 totals match
# 1 totals diverge — prints diff and which file to fix
# 2 prerequisites missing (cmake not found, tests.md missing, …)
#
# This script is meant to be cheap enough to run on every PR (~30 s on
# a typical CI runner). It re-uses the existing build-cgal/ directory
# if one is present; otherwise it builds a throwaway build-counts/.
set -euo pipefail
cd "$(dirname "${BASH_SOURCE[0]}")/.." # repo root
TESTS_MD="doc/api/tests.md"
if [ ! -f "$TESTS_MD" ]; then
echo "FAIL: $TESTS_MD not found" >&2
exit 2
fi
command -v cmake >/dev/null || { echo "FAIL: cmake not in PATH" >&2; exit 2; }
command -v ctest >/dev/null || { echo "FAIL: ctest not in PATH" >&2; exit 2; }
# ── Build ───────────────────────────────────────────────────────────────────
# Re-use existing build-cgal/ if it exists with the right flags; otherwise
# create a throwaway build-counts/ directory.
BUILD_DIR=""
if [ -f build-cgal/CMakeCache.txt ] && grep -q "WITH_CGAL_TESTS:.*=ON\|WITH_CGAL:.*=ON" build-cgal/CMakeCache.txt; then
BUILD_DIR=build-cgal
else
BUILD_DIR=build-counts
cmake -S code -B "$BUILD_DIR" -DWITH_CGAL_TESTS=ON -DCMAKE_BUILD_TYPE=Release >/dev/null
fi
cmake --build "$BUILD_DIR" -j"$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 2)" >/dev/null
# ── Get actual counts ───────────────────────────────────────────────────────
cd "$BUILD_DIR"
actual_cgal=$(ctest -R "^cgal" 2>&1 | grep "tests passed" | sed -E 's/.*out of ([0-9]+).*/\1/')
actual_fast=$(ctest -E "^cgal" 2>&1 | grep "tests passed" | sed -E 's/.*out of ([0-9]+).*/\1/')
cd - >/dev/null
# ── Get claimed counts from tests.md ────────────────────────────────────────
# Expected format (in this order):
# **Total: <N1> tests, 0 skipped.** (non-CGAL section)
# ...
# **Total: <N2> tests, 0 skipped.** (CGAL section)
claimed_fast=$(grep -E "\*\*Total: [0-9]+ tests, 0 skipped\.\*\*" "$TESTS_MD" \
| head -n 1 | sed -E 's/.*Total: ([0-9]+) tests.*/\1/')
claimed_cgal=$(grep -E "\*\*Total: [0-9]+ tests, 0 skipped\.\*\*" "$TESTS_MD" \
| tail -n 1 | sed -E 's/.*Total: ([0-9]+) tests.*/\1/')
# ── Compare ────────────────────────────────────────────────────────────────
ok=1
if [ "${actual_fast}" != "${claimed_fast}" ]; then
echo "MISMATCH: non-CGAL — $TESTS_MD claims ${claimed_fast}, ctest reports ${actual_fast}"
ok=0
fi
if [ "${actual_cgal}" != "${claimed_cgal}" ]; then
echo "MISMATCH: CGAL — $TESTS_MD claims ${claimed_cgal}, ctest reports ${actual_cgal}"
ok=0
fi
if [ "$ok" -eq 0 ]; then
cat <<EOF
FAIL: $TESTS_MD totals out of sync with actual test counts.
Recovery:
1. Edit $TESTS_MD: update the two
'**Total: N tests, 0 skipped.**' lines.
2. Add/remove suite rows in the per-suite tables if test
suites were added/removed.
3. Re-run this script to confirm.
EOF
exit 1
fi
echo "OK: test counts in $TESTS_MD match ctest output"
echo " non-CGAL: ${actual_fast} CGAL: ${actual_cgal} total: $((actual_fast + actual_cgal))"
exit 0

View File

@@ -10,7 +10,8 @@
# bash scripts/try_it.sh
#
# Expected output (last lines):
# [PASS] 227 CGAL tests pass, 0 skipped
# [PASS] full CGAL test suite passes, 0 skipped
# (current counts: doc/api/tests.md)
# [PASS] 23 non-CGAL tests pass
# [EXAMPLE] Converged in N iterations. ||G||_inf < 1e-9