Colony Arena swarm-routing ⌥

Archive · Data · swarm-routing/benchmarks

benchmarks

Immutable, reproducible benchmark datasets: every number traceable to its image, commits and seeds.

Locked: Phase 4: dataset format first

RepositoryFormat benchmarks#3Leaderboard site#12

What it does

benchmarks will hold the org's evidence: 20-seed ##RUN## / ##BENCH## data, the paired statistical verdicts recomputed from it, and a manifest for every run pinning the immutable ns3-dev:<ver>-<profile>-<sha7> image, each protocol's commit, the scenario, the seeds and the exact command line. No manifest, no merge.

Merged data is immutable: a correction is a new run that names the one it supersedes. Campaigns run on the optimized image; CI legs use debug and sanitized. The Colony Arena's leaderboard (site#12) will be regenerated from these verdicts by a script, never typed by hand.

Diagrams

benchmarks: from a campaign to the leaderboardA campaign runs on the immutable optimized image with pinned protocol commits; the raw marker lines and a manifest land in runs; a script recomputes the paired verdicts; this site's CI regenerates the leaderboard from those verdicts.campaignns3-dev:<ver>-optimized-<sha7>runs/marker linesmanifest.jsonverdicts/recomputedby scriptleaderboardthis site's CI(site#12)
Every number on the leaderboard traces back to an image digest, protocol commits, a scenario and its seeds. Dashed: planned (benchmarks#3, site#12).
benchmarks: what a run's manifest pins (UML class diagram)Class diagram of a planned run manifest: an image reference by immutable tag and digest, one commit per protocol, the scenario, the seeds (at least twenty) and the exact command line; a verdict refers to two runs paired by seed.dataset«GPL-2.0-only · data»Manifest+ image : ns3-dev:<ver>-<profile>-<sha7>+ digest : sha256+ commits : {protocol: sha}+ scenario : string+ seeds : int[≥ 20]+ command : stringRun+ markers : ##RUN## / ##BENCH##+ manifest : ManifestVerdict+ a, b : Run (paired by seed)+ delta : per-seed Δ+ noise : verdictSupersedes+ old : Run+ issue : why12
Planned shape (the schema is benchmarks#3); every field is required: no manifest, no merge.

Package structure

The repository's layout, its licence per directory (LICENSING.md) and what exists today.

Directories of swarm-routing/benchmarks
PathWhatLicenceState
runs/<yyyy-mm-dd>-<campaign>/raw marker lines / CSVs + manifest.json, immutable once mergedGPL-2.0-onlyplanned
verdicts/paired comparison tables produced by ns3-toolGPL-2.0-onlyplanned
schemas/the marker-line schema versions the data usesGPL-2.0-onlyplanned

Where it sits in the colony

The swarm-routing organisation: who depends on whomDependency graph of the swarm-routing organisation. The five protocols link against swarm-core and register as arms in ns3-tools; swarm-core and ns3-tools build and run inside the ns3-containers images; ns3-tools writes verdicts into benchmarks; the Colony Arena site builds AntHocNet's core to WebAssembly and will render the benchmarks leaderboard; the .github repository provides every repository's reusable CI.linkslinkslinkslinksextractsarmarmarmbuilds inruns inverdictscore/ → WASMleaderboardCIAntHocNetprotocol · liveARAprotocolBeeAdHocprotocolTermiteprotocolHOPNETprotocolswarm-coreshared modulens3-toolsharness · readersns3-containersns3-dev imagesbenchmarksdatasets · verdictsColony Arenathis site.githubreusable CI · templatesdashed arrow: «use» dependency (UML); highlighted: this project and its neighbours
benchmarks and its neighbours in the organisation.

Quests

Sources

Facts as of 2026-11-08, from the repository's AGENTS.md and the org ROADMAP.