docs: centralise test counts + add release-policy + remove stale stub references
Two complementary improvements aimed at reducing recurring maintenance
overhead:
1. **Test-count centralisation** — `doc/api/tests.md` is now the
single source of truth for the test counts. All other docs
(README, CLAUDE.md, doc/contributing.md, doc/getting-started.md,
doc/math/validation.md, doc/math/validation-protocol.md,
scripts/try_it.sh) use qualitative phrasing + a link instead of
hardcoded numbers. The previous regime had eight places with
"227 CGAL tests, 23 non-CGAL tests" that drifted apart across
releases (the v0.9.0 release-prep needed to touch nine files).
2. **Versioning policy** — `doc/release-policy.md` (new, ~250 lines)
formalises:
* SemVer rules for the pre-1.0 and post-1.0 phases.
* Phase-milestone → MINOR-bump mapping (v0.10.0 → Phase 9c, …).
* Single-source-of-truth table for moving numbers (test counts,
version, date).
* Step-by-step release process (the recipe that worked for v0.9.0
after the false-start with PR #11/#12).
* Hotfix policy + post-1.0 deprecation policy.
* Known failure modes and how to recover from them.
Plus a small CI gate:
3. **scripts/check-test-counts.sh** — verifies the totals in
doc/api/tests.md match `ctest` output. Re-uses existing build-cgal/
if present. Exit 0 on match, 1 on divergence with recovery hints.
Cheap enough (~30 s) to run on every PR.
Other cleanups
──────────────
* code/tests/cgal/CMakeLists.txt — stale "Test 7 (genus-2 homology)
as GTEST_SKIP stub until Phase 8" comment removed; that test landed
as HomologyGenerators.Genus2_FourCutEdges in Phase 7.
* CLAUDE.md — "test-fast also runs stubs" Known Quirks entry updated
to reflect the v0.9.0 stub cleanup (no GTEST_SKIPs remain).
* CLAUDE.md doc map — new entry for doc/release-policy.md.
Stubs audit
───────────
Zero GTEST_SKIP() calls remain in the codebase as of this commit.
The only references to stubs are in historical documentation
(CHANGELOG.md v0.7.0 entry, doc/roadmap/* "deferred to research-track"
notes) — those are intended.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
90
scripts/check-test-counts.sh
Executable file
90
scripts/check-test-counts.sh
Executable file
@@ -0,0 +1,90 @@
|
||||
#!/usr/bin/env bash
|
||||
# scripts/check-test-counts.sh
|
||||
#
|
||||
# CI gate that verifies the totals in `doc/api/tests.md` match the
|
||||
# actual `ctest` output. Catches the canonical test-count document
|
||||
# going stale even if no other doc hardcodes the number.
|
||||
#
|
||||
# Usage:
|
||||
# bash scripts/check-test-counts.sh
|
||||
#
|
||||
# Exit codes:
|
||||
# 0 totals match
|
||||
# 1 totals diverge — prints diff and which file to fix
|
||||
# 2 prerequisites missing (cmake not found, tests.md missing, …)
|
||||
#
|
||||
# This script is meant to be cheap enough to run on every PR (~30 s on
|
||||
# a typical CI runner). It re-uses the existing build-cgal/ directory
|
||||
# if one is present; otherwise it builds a throwaway build-counts/.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "${BASH_SOURCE[0]}")/.." # repo root
|
||||
|
||||
TESTS_MD="doc/api/tests.md"
|
||||
|
||||
if [ ! -f "$TESTS_MD" ]; then
|
||||
echo "FAIL: $TESTS_MD not found" >&2
|
||||
exit 2
|
||||
fi
|
||||
command -v cmake >/dev/null || { echo "FAIL: cmake not in PATH" >&2; exit 2; }
|
||||
command -v ctest >/dev/null || { echo "FAIL: ctest not in PATH" >&2; exit 2; }
|
||||
|
||||
# ── Build ───────────────────────────────────────────────────────────────────
|
||||
# Re-use existing build-cgal/ if it exists with the right flags; otherwise
|
||||
# create a throwaway build-counts/ directory.
|
||||
BUILD_DIR=""
|
||||
if [ -f build-cgal/CMakeCache.txt ] && grep -q "WITH_CGAL_TESTS:.*=ON\|WITH_CGAL:.*=ON" build-cgal/CMakeCache.txt; then
|
||||
BUILD_DIR=build-cgal
|
||||
else
|
||||
BUILD_DIR=build-counts
|
||||
cmake -S code -B "$BUILD_DIR" -DWITH_CGAL_TESTS=ON -DCMAKE_BUILD_TYPE=Release >/dev/null
|
||||
fi
|
||||
cmake --build "$BUILD_DIR" -j"$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 2)" >/dev/null
|
||||
|
||||
# ── Get actual counts ───────────────────────────────────────────────────────
|
||||
cd "$BUILD_DIR"
|
||||
actual_cgal=$(ctest -R "^cgal" 2>&1 | grep "tests passed" | sed -E 's/.*out of ([0-9]+).*/\1/')
|
||||
actual_fast=$(ctest -E "^cgal" 2>&1 | grep "tests passed" | sed -E 's/.*out of ([0-9]+).*/\1/')
|
||||
cd - >/dev/null
|
||||
|
||||
# ── Get claimed counts from tests.md ────────────────────────────────────────
|
||||
# Expected format (in this order):
|
||||
# **Total: <N1> tests, 0 skipped.** (non-CGAL section)
|
||||
# ...
|
||||
# **Total: <N2> tests, 0 skipped.** (CGAL section)
|
||||
claimed_fast=$(grep -E "\*\*Total: [0-9]+ tests, 0 skipped\.\*\*" "$TESTS_MD" \
|
||||
| head -n 1 | sed -E 's/.*Total: ([0-9]+) tests.*/\1/')
|
||||
claimed_cgal=$(grep -E "\*\*Total: [0-9]+ tests, 0 skipped\.\*\*" "$TESTS_MD" \
|
||||
| tail -n 1 | sed -E 's/.*Total: ([0-9]+) tests.*/\1/')
|
||||
|
||||
# ── Compare ────────────────────────────────────────────────────────────────
|
||||
ok=1
|
||||
if [ "${actual_fast}" != "${claimed_fast}" ]; then
|
||||
echo "MISMATCH: non-CGAL — $TESTS_MD claims ${claimed_fast}, ctest reports ${actual_fast}"
|
||||
ok=0
|
||||
fi
|
||||
if [ "${actual_cgal}" != "${claimed_cgal}" ]; then
|
||||
echo "MISMATCH: CGAL — $TESTS_MD claims ${claimed_cgal}, ctest reports ${actual_cgal}"
|
||||
ok=0
|
||||
fi
|
||||
|
||||
if [ "$ok" -eq 0 ]; then
|
||||
cat <<EOF
|
||||
|
||||
FAIL: $TESTS_MD totals out of sync with actual test counts.
|
||||
|
||||
Recovery:
|
||||
1. Edit $TESTS_MD: update the two
|
||||
'**Total: N tests, 0 skipped.**' lines.
|
||||
2. Add/remove suite rows in the per-suite tables if test
|
||||
suites were added/removed.
|
||||
3. Re-run this script to confirm.
|
||||
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "OK: test counts in $TESTS_MD match ctest output"
|
||||
echo " non-CGAL: ${actual_fast} CGAL: ${actual_cgal} total: $((actual_fast + actual_cgal))"
|
||||
exit 0
|
||||
Reference in New Issue
Block a user