The v2.4.0 control campaign — plan

Status: complete (2026-10-21). The results are at the end of this page. Above them, unchanged, is the pre-registration: it fixed the cells, the protocols each family is compared against, and how the results are read, before the first run existed, as the v2.3.0 campaign did. The plan file is tools/bench/campaigns/v2.4.0.yaml, and the jobs are dispatched and collected with tools/bench/campaign.py (#662).

← Benchmark index · Methodology · Roadmap

What this campaign decides

No default. It has a single arm, control, at the shipped defaults: v2.3.0's, with quiet mode on. What it changes is what AntHocNet is compared against. ADR-0026 makes baselines family-native: each family is compared against the protocols its own literature uses, plus its oracle. This campaign re-measures the five published families on those sets. Its cells become the v2.4.0 generation every family page quotes, and the input to the cross-family view (#711).

It is also where the four new comparators get their 20-seed cells:

• HWMP (#708);
• GPCR (#709);
• 3-D GPSR (#710);
• OPSPF (#707).

Their PRs merged them with an anchor and the delivery smoke, on short runs only.

The protocols per family

family	cells	control protocols	retired from v2.3.0's control
MANET grid	the 6 grid cells	AntHocNet, AODV, OLSR, DSDV, AOMDV, oracle	—
static Wi-Fi mesh	static-mesh	AntHocNet, OLSR, HWMP, oracle	AODV, DSDV
FANET	fanet-main, fanet-sparse	AntHocNet, AODV, OLSR, gpsr3d, oracle	DSDV, AOMDV
VANET	vanet-main, vanet-sparse	AntHocNet, AODV, GPSR, GPCR, oracle	OLSR, DSDV, AOMDV
satellite ISL grid	sat-base, -failcell, -corridor, -seam	AntHocNet, oracle	AODV, OLSR
LEO shells	walker16, Starlink S1, S1 storm	AntHocNet, OPSPF, geo-greedy, oracle, oracle-delay	AODV

Bold marks a protocol new to that family's control. Retired rows stay on their pages as the dated record of their campaigns, marked retired, never deleted (ADR-0026).

Three choices the table does not show, each the maintainer's:

• AOMDV joins the MANET grid (2026-10-20). ADR-0026 lists it as native there, but no grid cell has ever published it. The grid is re-run in full, not carried forward, so the v2.4.0 generation is one consistent set.
• OPSPF on the ISL grid is deferred (2026-10-20). The arm exists in leo-walker only (#707). On sat-base, sat-corridor and sat-seam nothing fails unscheduled, so OPSPF would be the oracle by construction. Only sat-failcell would differ, and porting the arm to isl-grid for that one cell is follow-up work. Until it lands, the ISL grid's comparison is AntHocNet against its bound.
• The static mesh changes radio and one HWMP parameter. These are below.

The static-mesh cell

The field, load and seeds are v2.3.0's. Two things change, for every arm in the cell:

• 802.11a at a fixed 6 Mbit/s (--standard=80211a --rateManager=constant6; maintainer decision 2026-10-19, #708). ns-3's 802.11s tunes every mesh interface to 5 GHz channel 100, so HWMP cannot run on the corpus's 802.11b radio. The disk channel ignores frequency, so the topology is v2.3.0's. The rate is not: these rows are not paired with the 802.11b record.
• --ns3::dot11s::PeerLink::MaxBeaconLoss=20 (maintainer decision 2026-10-20). The finding on #708 has the evidence. At ns-3's default of 2, a peer link closes after two missed beacons. On this 50-node field the path-selection floods lose beacons, links churn, each closure floods more path selection, and HWMP delivers 56.5 % over 900 s (seed 1). With the parameter at 20 the churn stops (peering frames 203 k → 1.7 k) and HWMP delivers 88.9 %. IEEE 802.11s does not fix this value; it is ns-3's choice. The attribute reaches only dot11s peer links, so it is inert for the other arms.

The HWMP gates (#716) run 16 nodes for at most 120 s. They prove the arm routes, not how it scales; this cell is where the scaling shows.

Seed blocks

No job may run past the 6 h runner limit:

• The grid cells run as two 10-seed blocks. v2.3.0's Gauss-Markov/Nakagami control took 4 h 43 min for five protocols, and AOMDV is a sixth.
• The static mesh runs as four 5-seed blocks. One HWMP seed costs about 15 CPU-minutes locally, four times AntHocNet's.
• VANET keeps v2.3.0's blocks: main 5 × 4 seeds, sparse 2 × 10.
• Everything else runs 20 seeds in one job. One S1 storm seed runs OPSPF and the oracle together in under 6 min (#721).

campaign.py collect concatenates the blocks into one file per cell, and bench_parse.py pairs them by seed. The plan is 44 jobs.

How the results are read

Each reading is fixed now, before any number exists.

1. The family gap to the oracle. AntHocNet minus the oracle, PDR in pp, with a paired 95 % CI per cell. This is the family-neutral yardstick (#711).
2. AntHocNet against the best native baseline. The best native baseline is the one with the highest mean PDR over the 20 seeds, with ties broken by lower delay99. It is chosen per cell from the data, by this rule only. The comparison is paired per seed (bench_parse.py --ab style, PDR, delay99 and NRL), with the verdict vocabulary of the earlier campaigns.
3. The comparators' own acceptance criteria:
4. HWMP, the #708 criteria;
5. GPCR above GPSR on the VANET cells, which is what #709 measures;
6. gpsr3d on the two FANET cells (#710);
7. OPSPF on the three shells, with AntHocNet − OPSPF as the satellite headline (#707). Each issue gets its verdict, with the numbers, when its cells land (ADR-0013).
8. The OPSPF / oracle reading. OPSPF beats the 1 s hop oracle on failure cells by design. The oracle there is not a delivery bound (#720). The pages say so wherever both appear, and no AntHocNet gap on the LEO failure cells is called a gap to "the bound".

Validity checks, run before any reading

• Attribution. AntHocNet's core and model code have not changed since v2.3.0. bench_parse.py --control against v2.3.0's published cells should therefore show these rows identical, seed for seed:
• every AntHocNet row, everywhere except the static mesh;
• every kept baseline row (AODV, OLSR, DSDV, oracle; geo-greedy and the two oracles on LEO). Two exceptions are expected:
• stock AODV on multi-interface satellite nodes may read ORDER-ONLY (#675);
• GPSR on VANET will differ: the #717 fix (#718) moved GPSR's first hello after stream pinning, which changes its draws. That is the reason for the difference, recorded on #717. Any other difference means the harness moved. That family's readings wait until it is explained.
• The static mesh has no attribution control: every arm's radio changed. Its consistency check is the oracle's mean hop count (the ##ROUTE## line). It depends on the topology, not on the rate, so it must stay within 1 % of v2.3.0's on every seed. Seed 1 reads 2.9242 against 2.9241, with PDR 99.3 % on both.
• Every arm reached its run. On the Wi-Fi cells, the ##CONFIG## line shows the protocols and the static-mesh radio. On the static mesh, the # hwmp diag line shows peering frames in the low thousands, not hundreds of thousands. If it does not, the MaxBeaconLoss attribute did not reach the run, and the cell is void.
• scenario_check.py results on every cell. OPSPF's rule (#721) FAILs a run that floods in a failure-free cell or never floods in a failure cell. The #431 oracle-hop findings on the Nakagami cells are known.
• Preflight. scenario_check.py preflight on the cells this plan changes:
• static mesh: WARN only, the 802.11a and HWMP notes;
• FANET with gpsr3d: WARN only;
• VANET main with GPCR: WARN only;
• grid with AOMDV: WARN only. vanet-sparse FAILs the degree rule, as it has since its definition: it is the deliberately partitioned street cell. Its protocol set does not change that.

What would make this plan wrong

• A kept baseline row moves beyond the two expected exceptions. Stop: the harness moved.
• OPSPF floods zero LSAs on walker16, or floods on a cell without failures. The arm is mis-wired; the results rule catches both.
• HWMP's peering frames climb into the hundreds of thousands on any seed. The attribute did not take, or the churn has a second cause; diagnose on #708 before publishing.
• A comparator falls far below its short-run numbers on the 20-seed cell. Two examples: gpsr3d under 80 % on fanet-main (it read 89.2 % on 2 seeds), or GPCR under GPSR on vanet-main. Check the cell before reading it as a result.

Phase order

1. Code, no runs. Done: #716 (HWMP), #718 (GPCR), #719 (gpsr3d), #721 (OPSPF). Merge this plan.
2. Campaign. All 44 jobs at main, at most 12 runs active.
3. Collect and check (the validity checks above), then post each comparator's verdict on its issue.
4. Restate. Each family page publishes its v2.4.0 native set, with migrant rows marked retired. #711 lands with the new cross-family view. The roadmap's v2.4.0 row records the comparators as done.

Results

Every validity check passed as pre-registered, and every family page now reads its native set. The 44 jobs ran at main (f0aedd3, the plan's merge) from 2026-10-20 16:03 UTC to 23:37 UTC, at most 12 active. None failed. The cells are results/cells/<cell>-v240.txt, one per cell, and leo-<shell>-<protocol>-v240.txt on the LEO shells.

Validity

• Attribution against v2.3.0. Every AntHocNet row and every kept baseline row is byte-identical to v2.3.0's published cells, seed for seed, in all 30 cells. That covers AODV, OLSR, DSDV and the oracle on the MANET grid and FANET, AODV and the oracle on VANET, the oracle on the ISL grid, and geo-greedy and both oracles on the LEO shells. Exactly the two pre-registered exceptions differ:
• GPSR on VANET: 0 of 20 seeds identical, from the #717 fix (#718).
• The static mesh, every arm: the 802.11a radio.
• The static-mesh topology check. The oracle's mean hop count stays within 1 % of v2.3.0's on every seed (the largest difference is 0.99 %).
• Every arm reached its run. The ##CONFIG## lines show each family's protocol set and the static mesh's radio. HWMP's peering frames read 1575–2413 per seed in the first block, so MaxBeaconLoss=20 took.
• scenario_check.py results. 0 FAIL on the static mesh, FANET, VANET, ISL-grid and LEO cells. On the MANET grid, two Nakagami cells keep 2 and 3 FAILs of the known #431 oracle-hop class; v2.3.0's same cells had 8 and 15, and ssrwp/Nakagami goes from 12 to 0. The drop comes from the cell, not from the checker (main's checker gives the same counts). The identity-matched hop rule compares arms on the packets that every arm in the run delivered. AOMDV, new here and the lowest deliverer, shrinks that common set: on Gauss-Markov/Nakagami seed 4 it falls from 10 364 to 8 702 packets. Fewer of the fading-path packets that trip the rule remain.
• A checker fix this campaign needed. scenario_check.py put the summary tables of a cell collected from seed blocks into one group. It then compared block 1's oracle against the other blocks' arms, and reported 3 false FAILs on the static mesh. It now groups each block by its firstRun, with a test case.

The readings

AntHocNet − oracle, and AntHocNet − the best native baseline (highest mean PDR, ties by delay99), both paired per seed, n = 20, 95 % CIs (tools/bench/native_view.py):

family	cell	AntHocNet PDR %	oracle PDR %	AntHocNet − oracle (pp)	best native	its PDR %	dPDR (pp)	d delay99 (ms)	d NRL	verdict
MANET grid	grid-rwp-tworay	98.47	97.94	+0.54 [+0.29, +0.78]	olsr	88.30	+10.17 [+9.60, +10.73]	+155.35 [+151.90, +158.80]	+5.04 [+4.90, +5.18]	PAIRED-MIXED
MANET grid	grid-rwp-nakagami	91.90	96.09	-4.19 [-5.39, -2.99]	olsr	85.72	+6.18 [+5.27, +7.10]	-1610.70 [-1650.80, -1564.25]	+12.15 [+11.63, +12.67]	PAIRED-MIXED
MANET grid	grid-ssrwp-tworay	98.47	98.02	+0.46 [+0.15, +0.77]	olsr	89.16	+9.31 [+8.70, +9.92]	+149.85 [+146.95, +152.95]	+4.80 [+4.64, +4.95]	PAIRED-MIXED
MANET grid	grid-ssrwp-nakagami	90.66	95.55	-4.89 [-6.05, -3.73]	olsr	84.91	+5.76 [+4.73, +6.78]	-1603.70 [-1651.25, -1555.05]	+12.38 [+11.74, +13.03]	PAIRED-MIXED
MANET grid	grid-gaussmarkov-tworay	97.11	96.55	+0.57 [+0.17, +0.97]	aodv	85.84	+11.28 [+10.64, +11.92]	-282.90 [-295.45, -270.90]	-23.86 [-24.36, -23.35]	PAIRED-IMPROVED
MANET grid	grid-gaussmarkov-nakagami	86.86	94.82	-7.96 [-9.16, -6.76]	olsr	78.28	+8.57 [+7.58, +9.57]	-1306.95 [-1385.95, -1225.80]	+17.05 [+15.75, +18.35]	PAIRED-MIXED
static mesh	static-mesh	99.34	99.94	-0.60 [-0.90, -0.30]	olsr	99.98	-0.65 [-0.91, -0.39]	+0.20 [+0.05, +0.40]	+2.40 [+2.19, +2.61]	PAIRED-WORSE
FANET	fanet-main	94.47	96.88	-2.41 [-2.68, -2.15]	gpsr3d	88.27	+6.20 [+5.72, +6.67]	+27.00 [-30.40, +76.05]	+2.74 [+2.65, +2.84]	PAIRED-MIXED
FANET	fanet-sparse	52.57	60.27	-7.70 [-8.12, -7.28]	aodv	50.05	+2.52 [+1.80, +3.23]	-4670.20 [-5084.65, -4324.05]	+6.16 [+5.74, +6.57]	PAIRED-MIXED
VANET	vanet-main	42.09	81.70	-39.61 [-40.20, -39.01]	aodv	42.26	-0.16 [-0.59, +0.26]	-1899.15 [-1949.00, -1852.55]	+13.16 [+11.40, +14.91]	PAIRED-MIXED
VANET	vanet-sparse	23.93	30.79	-6.85 [-7.02, -6.68]	aodv	23.79	+0.15 [-0.18, +0.47]	-7933.60 [-8041.95, -7829.40]	+20.13 [+19.35, +20.90]	PAIRED-MIXED
ISL grid	sat-base	100.00	100.00	+0.00 [+0.00, +0.00]	—	—	—	—	—	—
ISL grid	sat-failcell	100.00	100.00	+0.00 [+0.00, +0.00]	—	—	—	—	—	—
ISL grid	sat-corridor	100.00	100.00	+0.00 [+0.00, +0.00]	—	—	—	—	—	—
ISL grid	sat-seam	100.00	100.00	+0.00 [+0.00, +0.00]	—	—	—	—	—	—
LEO	leo-walker16	92.74	98.35	-5.62 [-6.01, -5.22]	opspf	98.90	-6.17 [-7.42, -4.91]	+18.10 [+9.25, +27.60]	-401.03 [-411.70, -390.35]	PAIRED-MIXED
LEO	leo-starlink1	79.34	95.27	-15.94 [-16.64, -15.23]	opspf	96.82	-17.48 [-19.08, -15.88]	+81.95 [+75.50, +88.25]	-18425.79 [-18691.83, -18159.75]	PAIRED-MIXED
LEO	leo-starlink1-storm	79.17	95.08	-15.91 [-16.80, -15.01]	opspf	96.12	-16.94 [-19.27, -14.61]	+74.75 [+70.35, +79.70]	-15422.35 [-15955.05, -14889.66]	PAIRED-MIXED

Drawn by ns3/tools/family-charts.py from the committed v2.4.0 cells (#711). Filled: the CI excludes 0. The LEO oracle is the 1 s hop oracle, which is not a delivery bound on those failure cells (#720).

What the table says, family by family:

• MANET grid. AntHocNet delivers 5.8–11.3 pp more than the best native baseline in all six cells, and on two-ray it delivers more than the link-blind oracle. OLSR keeps a far shorter tail on two-ray (about 21 ms delay99 against AntHocNet's 171–218 ms) and less overhead everywhere. AOMDV, new to the grid, is last in every cell (55–72 %).
• Static mesh. OLSR leads: 99.98 % against AntHocNet's 99.34 %, at NRL 1.78 against 4.17. HWMP is third at 91.0 %. AntHocNet − HWMP is +8.34 pp, delay99 −1097 ms, NRL −45.3, all paired-significant.
• FANET. AntHocNet leads both cells. On FANET main the best native baseline is 3-D GPSR (88.3 %), +2.16 pp over AODV. On the sparse cell AODV leads the natives, and 3-D GPSR trails it by 7.9 pp: voids are geographic routing's weakness.
• VANET. AntHocNet ties AODV on delivery in both cells (−0.16 and +0.15 pp, both CIs straddle 0), with a tail 1.9–7.9 s shorter at 13–20 more NRL. Both geographic arms trail AODV by 7–16 pp. The gap to the oracle on VANET main (39.6 pp) is still the largest of any cell.
• ISL grid. AntHocNet equals the oracle's delivery on all four cells. No native baseline is measured here yet (OPSPF on isl-grid is deferred).
• LEO shells. OPSPF beats AntHocNet by 6.2 pp on walker16 and by 16.9–17.5 pp on Starlink S1. That is the headline satellite comparison #707 asked for, and it goes against AntHocNet. OPSPF buys it with 5–20× the control: NRL 499 against 98 on walker16, and 16 357–19 399 against 934–974 on S1. Its floods are also only accounted, not transmitted (three simplifications, all disclosed on leo-walker.md). OPSPF also sits at or above the 1 s hop oracle (+0.55, +1.54 and +1.04 pp), as #720 predicted.

The comparators' own criteria

comparator	criterion	20-seed result	issue
HWMP	routes on the static-mesh cell; the 20-seed cell published	91.00 % ± 1.15, NRL 49.5; −8.99 pp against OLSR, paired-worse on all three metrics	#708
GPCR	above GPSR on the VANET cells	+2.91 pp on main, +0.59 pp on sparse, paired-improved in both	#709
3-D GPSR	the 20-seed FANET cells published	88.27 % on main (above AODV by 2.16 pp), 42.17 % on sparse	#710
OPSPF	the 20-seed arm on the three shells, AntHocNet − OPSPF as the headline	−6.17, −17.48 and −16.94 pp	#707

Provenance

item	value
plan	tools/bench/campaigns/v2.4.0.yaml, merged in #722 (f0aedd3)
runs	the label → run-id log on #715, and each cell file's # header
images	ns3:3.42-opt, except leo-walker on ns3:3.48
seeds	1–20 in every cell
collected	rescue-artifacts run 39518211831, campaign.py collect --from