Problem
Performance is a headline property of corosio and we have no automated
signal for it. perf/bench already measures nine categories
(socket_throughput, socket_latency, local_socket_*, timer,
http_server, fan_out, accept_churn, io_context) and writes
JSON, but it only runs when someone remembers to run it locally.
Regressions are found late, by hand, and improvements are asserted in
PR descriptions without evidence anyone can check.
What we need
- Every PR gets a performance verdict: PR head vs merge-base, per
category, per backend, on Linux (epoll, io_uring, select), Windows
(iocp) and macOS (kqueue), corosio only. IOCP and kqueue are
separate implementations, not ports; a Linux-only signal says
nothing about them.
- The verdict is trustworthy. A flagged regression must be real,
and a clean result must mean "no change larger than the measured
noise floor". The method has to state its noise floor and only flag
deltas outside it.
- The result is visible where the review happens: a PR comment,
updated in place on each push, with one table — category, metric,
base, head, delta, and a marker on significant rows. Full JSON as
a workflow artifact.
- Advisory, not blocking. A flagged row is a conversation starter
on the PR, not a red check.
Dedicated hardware
Benchmarks run on dedicated hardware, one machine per platform, that
does nothing else. Hosted runners are too noisy to trust.
Method: build base and head, alternate base/head for N iterations,
report the paired delta. A row is flagged when its delta exceeds the
variance observed in base-vs-base runs on the same machine.
The trigger is gated so untrusted PRs cannot run arbitrary code on the
machine: automatic for maintainers, label-approved for everyone else.
Scope
Acceptance
- A PR touching
include/ or src/ gets a benchmark comment covering
all three platforms, updated on each push.
Problem
Performance is a headline property of corosio and we have no automated
signal for it.
perf/benchalready measures nine categories(
socket_throughput,socket_latency,local_socket_*,timer,http_server,fan_out,accept_churn,io_context) and writesJSON, but it only runs when someone remembers to run it locally.
Regressions are found late, by hand, and improvements are asserted in
PR descriptions without evidence anyone can check.
What we need
category, per backend, on Linux (epoll, io_uring, select), Windows
(iocp) and macOS (kqueue), corosio only. IOCP and kqueue are
separate implementations, not ports; a Linux-only signal says
nothing about them.
and a clean result must mean "no change larger than the measured
noise floor". The method has to state its noise floor and only flag
deltas outside it.
updated in place on each push, with one table — category, metric,
base, head, delta, and a marker on significant rows. Full JSON as
a workflow artifact.
on the PR, not a red check.
Dedicated hardware
Benchmarks run on dedicated hardware, one machine per platform, that
does nothing else. Hosted runners are too noisy to trust.
Method: build base and head, alternate base/head for N iterations,
report the paired delta. A row is flagged when its delta exceeds the
variance observed in base-vs-base runs on the same machine.
The trigger is gated so untrusted PRs cannot run arbitrary code on the
machine: automatic for maintainers, label-approved for everyone else.
Scope
that is each platform's noise floor
platform, one PR comment with a section per platform; gated
trigger as above
Acceptance
include/orsrc/gets a benchmark comment coveringall three platforms, updated on each push.