Archive · Data · swarm-routing/benchmarks

benchmarks

Immutable, reproducible benchmark datasets: every number traceable to its image, commits and seeds.

Under construction: v0.1.0 released, no epic yet Computed from its 0 epics · release v0.1.0, as they stood on .

RepositoryFormat benchmarks#3Leaderboard site#12

What it does

benchmarks will hold the org's evidence: 20-seed ##RUN## / ##BENCH## data, the paired statistical verdicts recomputed from it, and a manifest for every run pinning the immutable ns3-dev:<ver>-<profile>-<sha7> image, each protocol's commit, the scenario, the seeds and the exact command line. No manifest, no merge.

Merged data is immutable: a correction is a new run that names the one it supersedes. Campaigns run on the optimized image; CI legs use debug and sanitized. The Colony Arena's leaderboard (site#12) will be regenerated from these verdicts by a script, never typed by hand.

Diagrams

benchmarks: from a campaign to the leaderboardA campaign runs on the immutable optimized image with pinned protocol commits; the raw marker lines and a manifest land in runs; a script recomputes the paired verdicts; this site's CI regenerates the leaderboard from those verdicts.campaignns3-dev:<ver>-optimized-<sha7>runs/marker linesmanifest.jsonverdicts/recomputedby scriptleaderboardthis site's CI(site#12)
Every number on the leaderboard traces back to an image digest, protocol commits, a scenario and its seeds. Dashed: planned (benchmarks#3, site#12).
benchmarks: what a run's manifest pins (UML class diagram)Class diagram of a planned run manifest: an image reference by immutable tag and digest, one commit per protocol, the scenario, the seeds (at least twenty) and the exact command line; a verdict refers to two runs paired by seed.dataset«GPL-2.0-only · data»Manifest+ image : ns3-dev:<ver>-<profile>-<sha7>+ digest : sha256+ commits : {protocol: sha}+ scenario : string+ seeds : int[≥ 20]+ command : stringRun+ markers : ##RUN## / ##BENCH##+ manifest : ManifestVerdict+ a, b : Run (paired by seed)+ delta : per-seed Δ+ noise : verdictSupersedes+ old : Run+ issue : why12
Planned shape (the schema is benchmarks#3); every field is required: no manifest, no merge.

Package structure

The repository's layout, its licence per directory (LICENSING.md) and what exists today.

Directories of swarm-routing/benchmarks
PathWhatLicenceState
runs/<yyyy-mm-dd>-<campaign>/raw marker lines / CSVs + manifest.json, immutable once mergedGPL-2.0-onlypresent
verdicts/paired comparison tables produced by ns3-toolGPL-2.0-onlyplanned
schemas/the marker-line schema versions the data usesGPL-2.0-onlypresent

Where it sits in the colony

The swarm-routing organisation: who depends on whomDependency graph of the swarm-routing organisation. ARA, BeeAdHoc, Termite and HOPNET link against swarm-core, which was extracted from AntHocNet, and register as arms in ns3-tools; AntHocNet runs its benchmarks on ns3-tools' harness; swarm-core and ns3-tools build and run inside the ns3-containers images; ns3-tools writes verdicts into benchmarks; the Colony Arena site builds AntHocNet's core to WebAssembly and will render the benchmarks leaderboard; the .github repository's reusable CI runs inside the ns3-containers images.linkslinkslinkslinksextractsarmarmarmarmbenchbuilds inruns inverdictscore/ → WASMleaderboard (planned)CI imagesAntHocNetprotocol · liveARAprotocolBeeAdHocprotocolTermiteprotocolHOPNETprotocolswarm-coreshared modulens3-toolsharness · readersns3-containersns3-dev imagesbenchmarksdatasets · verdictsColony Arenathis site.githubreusable CI · templatesdashed arrow: «use» dependency (UML); highlighted: this project and its neighbours
benchmarks and its neighbours in the organisation.

Quests

No epic yet: the work here is tracked as single issues (as of 2026-11-13).

Cite this project

Using benchmarks in a paper? Cite the project as below, and the works it rests on from its references.

Text

Joppi, Daniel Henrique (n.d.). benchmarks (version 0.1.0) [Data set]. https://doi.org/10.5281/zenodo.24145189

BibTeX

@misc{benchmarks,
  author  = {Joppi, Daniel Henrique},
  title   = {benchmarks},
  version = {0.1.0},
  doi     = {10.5281/zenodo.24145189},
  url     = {https://github.com/swarm-routing/benchmarks}
}

Version
0.1.0
Author
Daniel Henrique Joppi · ORCID
DOI
10.5281/zenodo.24145189
Licence
GPL-2.0-only

Read from CITATION.cff at main when this site was built.

References

The 3 works docs/references.bib lists, read at main when this site was built. Each links its DOI; a work with none links its publisher's or institution's page.

Software it builds on

Sources

The status and the quests come from the repository's issues and releases (2026-11-13); the other facts on this page are kept by hand from its AGENTS.md (as of 2026-11-08).