Extending VoxelBench

The extension API is in preview: it is complete and versioned, but not yet announced for general third-party use. Its compatibility policy is docs/API_STABILITY.md in the plugin repository, private today (see below).

Your tests will appear in:

• /bench test <id> (and tab completion),
• custom YAML profiles (/bench custom run my-profile),
• the benchmark GUI, in the Extensions category of the tests menu,
• the backend report payload tagged with your plugin name.

Prerequisites

• JDK 17+ to run Gradle (the plugin itself can target Java 16, like VoxelBench)
• A Bukkit/Paper plugin you control
• VoxelBench installed on the target server

---

1. Add the API dependency

The API is published through JitPack, built from a VoxelBench release tag. The version is the git tag exactly as written, leading v included (v1.9.0, not 1.9.0). Pin the release your server runs.

Not available yet: the VoxelBench repository is private today, so JitPack answers 401 for these coordinates until it is made public. If you have access to the repository, you can build the API jar yourself with ./gradlew assembleApi (output in build/api/) and add it as a local compileOnly file dependency meanwhile.

Gradle

repositories {
    maven { url 'https://jitpack.io' }
}

dependencies {
    compileOnly 'com.github.Wasabules:voxelbench-plugin:v1.9.0'
}

The dependency is compileOnly — the API classes already exist inside the running VoxelBench plugin JAR; bundling them again would cause class loader conflicts.

Maven

<repositories>
    <repository>
        <id>jitpack.io</id>
        <url>https://jitpack.io</url>
    </repository>
</repositories>

<dependencies>
    <dependency>
        <groupId>com.github.Wasabules</groupId>
        <artifactId>voxelbench-plugin</artifactId>
        <version>v1.9.0</version>
        <scope>provided</scope>
    </dependency>
</dependencies>

Or scaffold a ready-to-build project

From a checkout of the VoxelBench repository:

./gradlew newExtension -PextName=MyBench -PtestId=mybench.cacheLatency

This creates extensions/MyBench/, a standalone Gradle project that builds on its own:

• settings.gradle and the Gradle wrapper (./gradlew)
• build.gradle with the Spigot API 1.17.1, the VoxelBench API pinned to the latest release tag of your checkout, the Shadow plugin, and JUnit 6
  • Mockito test dependencies (plugin compiled for Java 16, tests for Java 17)
• plugin.yml with depend: [VoxelBench]
• a main class registering a sample test (typed parameter and metric specs, TestCompletion), a .gitignore and a README.md

Pass -PapiTag=v1.9.0 to pin another release (required when no release tag can be found, e.g. outside a git checkout). Without -PtestId, the ID defaults to myext. followed by the class name (myext.myBench). Build the plugin with ./gradlew shadowJar in the new folder. The API dependency resolves through JitPack, so the note above applies.

---

2. Declare the dependency in plugin.yml

name: MyExtension
version: 1.0.0
main: com.example.MyExtension
api-version: '1.17'

# Hard depend — Bukkit guarantees VoxelBench loads before your plugin.
# Use `softdepend` instead if you want to degrade gracefully when
# VoxelBench is absent.
depend: [VoxelBench]

---

3. Implement a BenchmarkTest

import fr.wasabii.voxelBench.api.*;
import java.util.*;
import java.util.concurrent.CompletableFuture;
import java.util.function.Consumer;

public class MyDatabaseLatencyTest implements BenchmarkTest {

    @Override public String getId()           { return "myext.dbLatency"; }
    @Override public String getDisplayName()  { return "Database Latency"; }
    @Override public TestCategory getCategory() { return TestCategory.EXTENSION; }
    @Override public int getTimeoutSeconds()  { return 60; }

    @Override
    public void run(BenchmarkContext ctx, Consumer<TestResult> callback) {
        // Single-call completion that hops back to the server thread by itself.
        TestCompletion done = TestCompletion.of(ctx, callback);

        int queries = ParamHelpers.paramInt(
                ctx.getOwnerPlugin(), ctx.getParameters(),
                "queries", 100, 1, 10_000, ctx.getSender());

        // Blocking I/O goes to a plain Java thread, never to a server thread.
        // Do NOT use Bukkit.getScheduler(): it is unavailable on Folia.
        CompletableFuture.runAsync(() -> {        // or your own ExecutorService
            double[] latencies = new double[queries];
            long startNs = System.nanoTime();

            for (int i = 0; i < queries; i++) {
                if (ctx.isStopRequested()) {
                    done.failure("stopped");
                    return;
                }
                long t0 = System.nanoTime();
                executeOneQuery();                          // your code
                latencies[i] = (System.nanoTime() - t0) / 1_000_000.0;
            }

            double durationSec = (System.nanoTime() - startNs) / 1e9;
            double mean = Arrays.stream(latencies).average().orElse(0);

            Map<String, Object> metrics = new LinkedHashMap<>();
            metrics.put("meanLatency",
                    RichMetric.of(mean)
                            .unit("ms").higherIsBetter(false)
                            .label("Mean query latency").precision(2).build());
            metrics.put("perQuery",
                    RichMetricSeries.of(latencies)
                            .unit("ms").higherIsBetter(false)
                            .label("Per-query latency").precision(2).build());

            done.success(durationSec, metrics);
        });
    }
}

Key contracts:

• run() is called on a server thread: the main thread on Spigot/Paper, the region thread that owns the test zone on Folia. Never block it; move blocking work to a thread of your own.
• The result must be delivered exactly once. TestCompletion.of(ctx, callback) (marked experimental) enforces this and hands the result back to the server thread from any thread. If you call the raw callback yourself, do it once, from a server thread.
• Never use Bukkit.getScheduler() in a test: Folia does not support it. Use the context methods below for world work.
• Poll ctx.isStopRequested() regularly in long-running loops.
• Override cleanup() if your test allocates blocks / entities / files.

Working in the world (Folia-safe)

On Folia, blocks and entities may only be touched from the region thread that owns them. BenchmarkContext (API 1.2+) runs your code in the right place on every platform, so the same test works on Spigot, Paper and Folia:

@Override
public void run(BenchmarkContext ctx, Consumer<TestResult> callback) {
    TestCompletion done = TestCompletion.of(ctx, callback);
    long startNs = System.nanoTime();
    java.util.concurrent.atomic.AtomicInteger changed =
            new java.util.concurrent.atomic.AtomicInteger();

    ctx.forEachZone(zone -> {
        // Only touch blocks/entities around THIS zone, synchronously.
        changed.addAndGet(placeBlocksAround(zone));    // your code
    }, () -> {
        double sec = (System.nanoTime() - startNs) / 1e9;
        done.success(sec, Map.of("blocksPerSecond", changed.get() / sec));
    });
}

The per-zone task is considered finished when it returns, so do its work synchronously. Track what you create and remove it in cleanup().

---

4. Register your test in onEnable

public class MyExtension extends JavaPlugin {

    @Override
    public void onEnable() {
        VoxelBenchAPI api = VoxelBenchAPI.getInstance();

        api.getTestRegistry().register(
                TestDescriptor.builder()
                        .id("myext.dbLatency")
                        .displayName("Database Latency")
                        .description("Round-trip latency to the primary database")
                        .category(TestCategory.EXTENSION)
                        .owner(getName())                          // MUST match plugin.yml
                        .builder(ctx -> new MyDatabaseLatencyTest())
                        .build(),
                this);
    }
}

You do not need to unregister in onDisable — VoxelBench listens to PluginDisableEvent and prunes your tests automatically when your plugin disables (server stop, /reload, plugin unload, or even a crash inside your own onDisable).

---

5. Build, deploy, test

Build your plugin (./gradlew jar or mvn package), drop the JAR in plugins/, restart the server. The VoxelBench startup log will say:

[VoxelBench] Loaded 1 extension test(s) from MyExtension: myext.dbLatency

Then in-game (parameters are passed as key=value pairs):

/bench test myext.dbLatency
/bench test myext.dbLatency queries=500

Or in a custom YAML profile (running a profile requires a linked server):

# plugins/VoxelBench/custom_benchmarks/storage-suite.yml
name: "Storage Suite"
tests:
  - id: disk
  - id: myext.dbLatency
    params:
      queries: 500

/bench custom run storage-suite

VoxelBench reads the profiles when it starts, before your plugin (which depends on it) has registered its tests. A profile that uses an extension test is therefore rejected at every server start, with references unknown test 'myext.dbLatency' in the console: run /bench custom reload once the server is up, then run the profile.

Unit-test your test off-server

The fr.wasabii.voxelBench.api.testing package (experimental) lets you call run() from JUnit without a Minecraft server:

• MockBenchmarkContext — a BenchmarkContext built with MockBenchmarkContext.builder(), where you set parameters (.param("queries", 50)), the stop flag, the remaining timeout, and optionally the plugins, world, zone or sender.
• CapturingCompletion — a callback that captures the result: getResult() for synchronous tests, awaitResult(Duration) for asynchronous ones, and getInvocationCount() to catch double completion.

@Test
void dbLatency_reportsMeanLatency() throws Exception {
    BenchmarkContext ctx = MockBenchmarkContext.builder()
            .param("queries", 10)
            .build();

    CapturingCompletion done = new CapturingCompletion();
    new MyDatabaseLatencyTest().run(ctx, done);

    TestResult result = done.awaitResult(Duration.ofSeconds(10));
    assertEquals(TestResult.Status.SUCCESS, result.getStatus());
    assertTrue(result.getMetrics().containsKey("meanLatency"));
    assertEquals(1, done.getInvocationCount());
}

The mock does not simulate worlds, chunks or entities: test world logic on a real server.

---

ID conventions

Pick a namespace prefix to avoid colliding with built-ins or other extensions:

The host normalises by stripping hyphens / underscores and lowercasing, so myext.dbLatency, MyExt.DbLatency, myext-db-latency all resolve to the same descriptor. Collisions throw at register-time with a clear error message.

---

Versioning

The VoxelBenchAPI.API_VERSION constant is the API version your plugin was compiled against; the current API is 1.2.0 (the region-aware methods of BenchmarkContext arrived in 1.2). api.getApiVersion() returns the version of the VoxelBench installed on the server. Check it at runtime if you depend on recent additions:

VoxelBenchAPI api = VoxelBenchAPI.getInstance();
if (!api.getApiVersionStructured().isAtLeast(1, 2)) {
    getLogger().warning("VoxelBench " + api.getPluginVersion()
            + " is too old for this extension (need API 1.2+).");
    getServer().getPluginManager().disablePlugin(this);
    return;
}

isAtLeast(major, minor) also returns false when the major version differs, since a new major version may break compatibility.

Stability contract:

• STABLE (default) interfaces don't break across minor/patch releases. Major bumps may break, with a changelog entry.
• EXPERIMENTAL marked types may change in any release.
• INTERNAL marked types are not part of the public contract.

---

Backend submission

By default, custom benchmark profile runs (including extension tests) are kept local — no backend submission. If a profile sets submit: true, the report payload's tests[] array includes a provider: "<your plugin name>" field on every test entry your extension contributed. The backend uses this to keep extension results out of the canonical leaderboard while still archiving them for the operator's dashboards.

The single-test path (/bench test myext.dbLatency) submits as a unit test report with the same provider field, when the server has unit-test sync enabled (reports.backend.unit-tests) and is linked. The backend may also exclude these from public leaderboards depending on policy.

---

See also

These files live in the plugin repository, which is private today; they will be linked here once it is public.

• docs/EXTENSION_API_REFERENCE.md — full API reference, every public class
• docs/API_STABILITY.md — versioning and stability policy
• docs/examples/SampleExtensionPlugin.java — complete one-file walkthrough

Troubleshooting