VoxelBench 1.8.0 - Deadlines that hold when your server doesn't

This release fixes one defect that ran through the whole bench, though its symptoms looked nothing alike: runs that stopped without a message, tests cut short on slow hosts, and entire reports thrown away while most of their measurements were perfectly good.

Scores remain comparable with 1.7.x. No formula changed, and a report produced by 1.7.0 still compares directly to one produced by 1.8.0.

A deadline counted in the unit the failure destroys

Between two tests, VoxelBench waits eight seconds for the machine to settle. That delay was expressed in ticks — the server's internal clock. A tick lasts 50 ms when all is well, and a great deal longer when the server is struggling.

So the window stretched exactly in proportion to how useless it had become: the previous test had just saturated the machine, and that was precisely why we were waiting.

Measured on a Folia server during the mob-AI test, at 0.26 ticks per second — one tick every 3.8 seconds — the old window's 160 ticks would have taken 615 seconds: more than ten minutes for an eight-second pause. It now takes 8064 milliseconds.

The change only touches waits whose subject does not depend on the server's rhythm. TNT fuses, redstone timing, fluid propagation and per-tick chunk budgets stay in ticks: that is their correct unit, and converting them would have distorted the measurements instead of protecting them.

Runs no longer stall between two tests

A reported log showed a benchmark stopping dead after the chunk-loading test. No error, no message — and the global lock held until the server was restarted.

The cause: nothing watched the gap between two tests. One test's watchdog is disarmed the moment it finishes; the next one's is only armed when it starts. In between sit the teardown of thousands of chunks and the entire setup of the test to come.

A watchdog now covers the run as a whole, and its thresholds are derived from the limits each test declares — a hand-picked threshold would have cut the memory test short, which legitimately asks for up to fifteen minutes.

A test that proves it is progressing earns more time

The memory test had a fixed 300-second limit. On a slow host its final pass needed 377: the test was cut, and the whole report refused.

A test can now request a reprieve. Each completed unit of work pushes the deadline back, while an absolute ceiling stays in place — the point is not to remove the protection, but to tell a slow test from a stuck one.

The reprieve renews on evidence: a pass that genuinely finished. Never on a mere sign of life, or the watchdog would push back its own deadline forever and stop being worth anything.

Messages now say what happened, and what to do

A user read this in their chat:

⊘ Test ignoré: Mémoire — memoryPressure: projected peak heap 107.6% after pre-flight GC

A developer's sentence, in English, in the middle of a translated message. It names a cause and never a remedy.

Every test that does not run to completion now carries a structured reason: a translated cause, and above all a computed remedy. No longer "raise -Xmx", which leaves you guessing, but the amount of memory you would have needed, worked out:

⊘ Test skipped: Memory — not enough memory — projected peak would reach
                          107.6% of the heap (1670 MB available)
→ Raise -Xmx (about 2304 MB for the standard profile) or run
  /bench start low-memory.

Two additions complete this. A warning before the run when memory plainly will not be enough: the verdict was computable in the first second, and the user learned it in the eleventh minute. And an end-of-run summary that ties the skipped tests together — three notices six minutes apart do not connect themselves in a scrolling chat.

A partial run is no longer lost

This is the most visible change for anyone measuring a small server.

When memory runs short, VoxelBench refuses to start a test rather than bring the server down. That is the right decision. But until now, a report missing a required test was rejected outright: a host with 1670 MB of heap lost twelve correctly measured tests because three had been skipped — that is, because the plugin had correctly protected itself.

The report now carries, for each test, a status and a cause code. The service can therefore tell a deliberate skip from a crash, and the report is kept and viewable.

It is not scored, and that will not change: putting a number on a run with a missing required test would mean scoring something nobody measured. It does not appear in the leaderboards — but you no longer lose eleven minutes of measurement.

What the screen claimed without any way of being wrong

The /bench test disk command reported "✓ Test passed", "TPS 20.00" and "MSPT 50.00 ms". Those three values were hard-coded: they appeared regardless of the actual result.

Worse, all six throughput metrics displayed 0 MB/s on an NVMe drive that had just measured 2621 MB/s sequential read. The measurement was correct — it reached the report intact — only the display lied, and in the costliest direction for someone whose whole reason for running it was to diagnose their disk.

A missing metric now shows "—" rather than "0". Zero is a plausible throughput: confusing the two reads as a dead disk where there is only a missing value.

Quieter fixes

• A red error line (No key layers in MapLike[{}]) was written to the console on every run, by VoxelBench's own bench-world creation. The fix existed but had only been applied to Folia; three Paper logs said otherwise. The generated terrain itself was always correct.
• The memory collection performed before each test abandoned its loop on the first attempt, mistaking "there is nothing left to free" for "collection has not started yet". It now waits for proof that a collection happened, and says so when that proof never arrives.
• The memory-leak detector reported leaks that did not exist: it compared a test's end to its start, and so measured what the test had just created and not yet cleaned up. A leak is what survives cleanup.
• A test now waits for the previous one's chunks to actually unload, rather than for a delay to elapse. When the wait reaches its limit, the log says so instead of starting silently on a cluttered world.
• The pre-flight check logs that it is waiting. When it opened its confirmation screen, the command stopped without leaving a trace: the server answered "no test running" and nobody could tell why.

Compatibility

Validated on three platforms, full run and report submitted: Paper 1.21.11, Spigot 26.1.2 and Canvas 26.1.2 (Folia family). Continuous integration additionally covers Spigot from 1.17.1 to 26.2.

The new report fields are optional: a service that does not know about them ignores them, and reports produced by earlier versions are read exactly as before.