v2.6.0 harness control: the move reproduces every committed cell¶
← Benchmark index · Comparison harness · ADR-0027
v2.6.0 moved the comparison harness out of the anthocnet module into
routing-bench, and turned every arm-specific branch into a hook of the arm
(#767,
#768). ADR-0027 §4
accepts the move only if no published number moves. This page is the control
that checks it (#769).
Plan and rule¶
- Plan:
tools/bench/campaigns/v2.6.0-harness-control.yaml, generationpc769. - It re-runs one cheap block (seeds 1–2) of every MANET-family cell committed in the v2.4.0 and v2.5.0 generations.
- Every run goes through
routing-compare, at the branch that moved the harness. - Each cell keeps its committed plan's own inputs and protocol list. Lists are never merged, because the identity-matched
##COMMON##set depends on which arms share an invocation. - Rule: every
##RUN##,##ROUTE##and##COMMON##row for seeds 1–2 must equal the committed one. The check istools/bench/harness_control.py. - The runs are deterministic (#129) and seed-pinned (#352), so any difference would be a defect in the move, not noise.
Result¶
37 of 37 cells PASS: 726 rows, byte for byte. No cell's
##CONFIG## differs from its committed one beyond runs/firstRun.
| control cell | committed file | rows compared | verdict |
|---|---|---|---|
grid-rwp-tworay |
grid-rwp-tworay-v240.txt |
36 | PASS |
grid-rwp-nakagami |
grid-rwp-nakagami-v240.txt |
36 | PASS |
grid-ssrwp-tworay |
grid-ssrwp-tworay-v240.txt |
36 | PASS |
grid-ssrwp-nakagami |
grid-ssrwp-nakagami-v240.txt |
36 | PASS |
grid-gaussmarkov-tworay |
grid-gaussmarkov-tworay-v240.txt |
36 | PASS |
grid-gaussmarkov-nakagami |
grid-gaussmarkov-nakagami-v240.txt |
36 | PASS |
static-mesh |
static-mesh-v240.txt |
24 | PASS |
fanet-main |
fanet-main-v240.txt |
30 | PASS |
fanet-sparse |
fanet-sparse-v240.txt |
30 | PASS |
vanet-main |
vanet-main-v240.txt |
30 | PASS |
vanet-sparse |
vanet-sparse-v240.txt |
30 | PASS |
disaster-main |
disaster-main-v240.txt |
24 | PASS |
scale-50 |
scale-50-v240.txt |
24 | PASS |
scale-100 |
scale-100-v240.txt |
24 | PASS |
scale-200 |
scale-200-v240.txt |
24 | PASS |
scale-500 |
scale-500-v240.txt |
12 | PASS |
scale-500-olsr |
scale-500-olsr-v240.txt |
6 | PASS |
scale-500-oracle |
scale-500-oracle-v240.txt |
6 | PASS |
anchor-ara-p0 |
anchor-ara-p0-v250c.txt |
12 | PASS |
anchor-ara-p30 |
anchor-ara-p30-v250c.txt |
12 | PASS |
anchor-ara-p120 |
anchor-ara-p120-v250c.txt |
12 | PASS |
anchor-ara-p600 |
anchor-ara-p600-v250c.txt |
12 | PASS |
anchor-termite-s1 |
anchor-termite-s1-v250c.txt |
12 | PASS |
anchor-termite-s5 |
anchor-termite-s5-v250c.txt |
12 | PASS |
anchor-termite-s10 |
anchor-termite-s10-v250c.txt |
12 | PASS |
anchor-termite-s20 |
anchor-termite-s20-v250c.txt |
12 | PASS |
anchor-qrouting-low |
anchor-qrouting-low-v250c.txt |
12 | PASS |
anchor-qrouting-high |
anchor-qrouting-high-v250c.txt |
12 | PASS |
grid-rwp-tworay-cmp |
grid-rwp-tworay-v250c.txt |
18 | PASS |
grid-rwp-nakagami-cmp |
grid-rwp-nakagami-v250c.txt |
18 | PASS |
grid-ssrwp-tworay-cmp |
grid-ssrwp-tworay-v250c.txt |
18 | PASS |
grid-ssrwp-nakagami-cmp |
grid-ssrwp-nakagami-v250c.txt |
18 | PASS |
grid-gaussmarkov-tworay-cmp |
grid-gaussmarkov-tworay-v250c.txt |
18 | PASS |
grid-gaussmarkov-nakagami-cmp |
grid-gaussmarkov-nakagami-v250c.txt |
18 | PASS |
scale-50-s2 |
scale-50-v250s2.txt |
6 | PASS |
scale-200-s2 |
scale-200-v250s2.txt |
6 | PASS |
scale-500-s2 |
scale-500-v250s2.txt |
6 | PASS |
Coverage:
| generation | cells |
|---|---|
| v2.4.0 | the six grid cells, the static mesh (hwmp), both FANET and both VANET cells, the disaster scene, the scale ladder (50–500 nodes, including the 500-node OLSR and oracle singles) |
| v2.5.0 | the ten comparator anchor cells, the six comparator family cells (ara, termite, qrouting), the scale stage 2 |
Every registered arm appears in at least one cell.
The one red run. grid-rwp-nakagami went red in GitHub Actions. Its
scenario_check results step FAILs on the known oracle-adjacency approximation
on a fading channel: the oracle's identity-matched hops are 1.189 against dsdv's
1.033 on seed 2. The committed grid-rwp-nakagami-v240.txt carries the same
FAIL with the same numbers, so the control reproduces it like every other row.
Provenance¶
- Runs: 39 runs of
benchmark-cell.ymlon the3.42-optimage (release profile), dispatched bycampaign.py dispatch. - Where to find them: each
results/cells/<cell>-pc769.txtnames its run ids in its first line, and its##PROV##lines name the commit.
python3 tools/bench/harness_control.py tools/bench/campaigns/v2.6.0-harness-control.yaml