Auto-bench on the Website

How a Run Goes

1. A run is due on its schedule, or you click Run now: VoxelBench queues a job.
2. A bench agent picks the job up, and its bot connects to your server's verified address, as a Minecraft: Java Edition player.
3. The bot types /vbautobot followed by a one-time code and the mode: it is the only command the bot uses to start a run. The plugin asks voxelbench.com whether that code is valid for this server, and receives what to run: mode, test or profile, warm-up, number of runs.
4. The plugin runs the benchmark and reports its progress to the bot. Each measured run sends its report to voxelbench.com, attached to the job.
5. The bot disconnects.

What happens on the server, and what the server needs, is described in Auto-bench in the plugin documentation.

Before You Start

• A plan that includes auto-bench: Pro, Enterprise, or a hosting provider account verified by VoxelBench. On the Free plan, the Auto-bench page only offers to upgrade.
• A linked and verified server. Only verified servers can receive a target, and the bot connects to the address you verified (see Linking and Verifying a Server). Linking the server again removes its verification: verify it again afterwards.
• On the server: the official, up-to-date VoxelBench jar, outgoing HTTPS to voxelbench.com, and a way in for the bot through whitelist, online mode, anti-bot plugins and proxies (see Letting the Bot In). A pinned benchmark world is strongly recommended (see Where Runs Take Place).

An auto-bench run loads the server like any benchmark: players online will feel it. Schedule it for quiet hours.

Microsoft Accounts

How the bot signs in depends on your server:

• Offline mode (online-mode=false): no account is needed. The bot joins under a generated name starting with vb_. Choose Offline (cracked server) as the target's auth mode.
• Online mode: the bot must sign in with a genuine Minecraft: Java Edition account. You lend one in the Microsoft accounts card of the Auto-bench page, then choose BYO Microsoft account as the target's auth mode.

To add an account, click Add Microsoft account. A window gives you a Microsoft address and a short code: open the address, type the code and sign in with the Microsoft account that owns Minecraft: Java Edition. The page notices the sign-in by itself. An account without Java Edition is refused.

VoxelBench never sees your password: you sign in on Microsoft's own page. It keeps the sign-in token Microsoft returns, stored encrypted, and lends it to the bench agent only for the runs that use this account. The page's own warning applies: Mojang may suspend accounts used by automated clients, so only add an account you are willing to lose.

Each account shows its state: available, leased (in use by a run), banned or invalid. After a failed sign-in (auth_failed), the account is marked banned and is no longer used until you remove it and add it again. Removing an account deletes its stored credentials; it is refused while a target still uses the account.

Creating a Target

A target is one server, one way of benchmarking it and one schedule. On the Auto-bench page, click New target:

Creating a target is only possible from the dashboard: the API and agents cannot do it, because a target engages a Microsoft account you lend.

Editing and Deleting

Edit (in the target's row) changes the name, the schedule (a preset or any cron expression), the Minecraft version, the mode, the arguments, the warm-up and the number of runs. The server and the auth mode cannot be edited: create a new target to change them. The Enabled switch pauses and resumes a target. Delete removes it for good, from the dashboard only.

Scheduling

The presets are Daily at 02:00 UTC (the default), Every 6 hours, Twice daily (02h/14h UTC), Weekly (Sunday 02:00 UTC) and Monthly (1st 02:00 UTC). When editing a target, you can also type any five-field cron expression — minute, hour, day of month, month, day of week — always in UTC, with *, */N, lists (2,14) and ranges (1-5).

• At most once an hour. A schedule that fires more often is refused when you save it.
• Around the time, not on the minute. Each run is moved by a random offset of up to 20% of the interval, 30 minutes at most, earlier or later, so that nobody can tune a server just for a known time. The scheduler checks every 5 minutes for runs that are due.
• One run at a time. If the previous run of the target is still going when the next one is due, the new one waits until the target is free.
• Within the quota. When your daily run quota is used up, a scheduled run is skipped and the target moves on to its next time.

An API token or a connected agent with the auto-bench:write scope can change a target's schedule, name and notes, and switch it on or off (see API and Tokens).

Launching and Cancelling

Run now, in the target's row, queues a run at once; a bench agent picks it up shortly after.

• One run in flight per target. While a run is queued or in progress, launching again is refused (through the API: already_running, with the id of the run in progress).
• 60 seconds at least between two launches of the same target.
• The run quota must cover the target's number of runs.

Cancel, in the Live runs panel, asks the bot to stop: it tells the plugin to stop the run, which the plugin does at once, and disconnects. The run first shows cancelling, then cancelled. If the agent does not confirm within 5 minutes, the run is closed anyway (cancel_timeout). A cancelled run is never counted as a failure of the target, but it stays counted in the run quota.

On the server, /bench stop also stops an auto-bench run; VoxelBench records it as stopped on the server (aborted_in_game), not as a failure. A run of a multi-run job that ends without its report is different: the plugin abandons the remaining runs, and VoxelBench records a failure of the target (agent_crash, see Other Errors).

Quotas

The Free plan has no auto-bench.

The run quota counts every job created in the last 24 hours with its number of runs (the warm-up does not count), whether it succeeded, failed or was cancelled, and scheduled runs included. It is a rolling window: runs come back as earlier ones age past 24 hours. The Auto-bench page shows how many runs you used in the last 24 hours.

Following Runs

The Auto-bench page shows:

• Live runs, updated every 5 seconds: each run in progress, with its run number (Run 2/5), a warmup badge during the warm-up, the current test and the Cancel button;
• Recent runs: completed, failed and timed-out runs, with their status, target, mode, score, duration and start, and a link to the report or the test result. Click a failed run to see what went wrong, what to do, and the agent's raw message;
• in the targets table: the Next run, the Last success, and the Failures in a row, with the last error.

The same runs can be read through the API (GET /api/v1/auto-bench/jobs), each failure with the same explanation as the English dashboard.

Run Statuses

Failures and Automatic Pause

A failed run (status failed) sends you a notification with the number of failures in a row. After 5 failures in a row, the target is switched off automatically, and you are notified. Switching it back on resets the count, and so does a successful run.

Some endings are not the server's fault and are not counted: a run interrupted by a VoxelBench update (agent_shutdown), a run stopped on the server with /bench stop (aborted_in_game), and cancellations, including a certification run that VoxelBench's moderation rescheduled or cancelled (rescheduled, user_cancelled). Every other failure counts, including one caused by the target's settings, such as a wrong test ID, and a multi-run job abandoned because one of its runs failed.

Errors and What to Do

A failed run shows a label, its code, an explanation and what to do. The codes below are the ones you will see. Codes that come from the plugin are also listed, with their fixes, in Auto-bench in the plugin documentation.

Before the Plugin Answers

On the Server

Codes and Server Identity

Runs Ended on Purpose

None of these counts as a failure of the target.

Other Errors