For AI agents: the complete documentation index is available at https://scriptedalchemy.github.io/agent-bundle/llms.txt, the full documentation bundle is available at https://scriptedalchemy.github.io/agent-bundle/llms-full.txt, and this page is available as Markdown at https://scriptedalchemy.github.io/agent-bundle/guide/development/testing.md.
  • English
  • Testing

    Route modules are tested through the framework, not through a hand-written bundler configuration. Two subpaths ship that harness and both are opt-in: @rstest/core and react are optional peer dependencies, so a project that never tests routes installs neither. Rendering also needs @agent-bundle/runtime, which the project already owns whenever it has route modules — the generated entries import it the same way.

    The configuration helper

    agent-bundle/rstest compiles the project once — the same route-graph compilation the build performs, with no artifact build — and returns a plain Rstest configuration object carrying the test manifest, the route loaders, React's react-server resolution, and the automatic JSX runtime:

    // rstest.route-unit.config.ts
    import { defineConfig } from '@rstest/core';
    import { agentBundleRstest } from 'agent-bundle/rstest';
    
    export default defineConfig(await agentBundleRstest());

    Route-unit tests default to tests/route-unit/**/*.test.{ts,tsx} and need their own Rstest run, because rendering a route requires Node's react-server condition for the whole worker process. Keep them out of the project's ordinary rstest run.

    Rendering a route

    agent-bundle/test holds the helpers. renderRoute executes a route — by compiled route id, or by importing the module directly — through the real renderer and the real request store, and resolves to the final Agent Document:

    import { 
    const expectDocument: (subject: DocumentSubject) => DocumentAssertions
    expectDocument
    ,
    const renderRoute: (target: RenderRouteTarget, options?: RenderRouteOptions | undefined) => Promise<RenderedRoute>

    Renders one route through the real Agent renderer and returns its final Agent Document. The route component executes inside a real request scope, its output is encoded as React Flight, and the runtime's own final-only dispatcher decodes it — the harness owns no second rendering path.

    This is the route-unit proof level: no transport is opened, no browser surface is compiled, and no host artifact is built.

    renderRoute
    } from 'agent-bundle/test';
    export const
    const summarizes: () => Promise<void>
    summarizes
    = async ():
    interface Promise<T>

    Represents the completion of an asynchronous operation

    Promise
    <void> => {
    const {
    const document: AgentDocument

    The final Agent Document the real renderer produced.

    document
    } = await
    function renderRoute(target: RenderRouteTarget, options?: RenderRouteOptions | undefined): Promise<RenderedRoute>

    Renders one route through the real Agent renderer and returns its final Agent Document. The route component executes inside a real request scope, its output is encoded as React Flight, and the runtime's own final-only dispatcher decodes it — the harness owns no second rendering path.

    This is the route-unit proof level: no transport is opened, no browser surface is compiled, and no host artifact is built.

    renderRoute
    ('tool:library/summarize', {
    RenderRouteOptionsBase.input?: unknown

    The route's input: tool input, event payload, or script input.

    input
    : {
    title: string
    title
    : 'Dune' },
    });
    function expectDocument(subject: DocumentSubject): DocumentAssertions
    expectDocument
    (
    const document: AgentDocument

    The final Agent Document the real renderer produced.

    document
    )
    .
    DocumentAssertions.toHaveStatus: (status: AgentDocumentStatus) => DocumentAssertions
    toHaveStatus
    ('success')
    .
    DocumentAssertions.toContainMarkdown: (text: string) => DocumentAssertions

    Asserts a Markdown node contains text.

    toContainMarkdown
    ('Dune')
    .
    DocumentAssertions.toHaveValue: (value: unknown) => DocumentAssertions

    Asserts the document's structured value equals value (JSON structural equality). undefined asserts the document emitted no value at all, which null does not satisfy.

    toHaveValue
    ({
    chapters: number
    chapters
    : 24 });
    };

    renderRoute accepts input, args (CLI routes), request-context overrides — including a context.progress reporter — render limits, and a signal. It returns the document, the request-scoped progress the route reported, the resolved provenance, and the route's own resultSchema-parsed value. Progress is recorded whether or not the caller supplies a reporter of its own.

    testManifest() exposes the compiled route inventory, so a suite can iterate every route in process rather than paying for a build per route. Every failure — an unknown route, a refused route kind, a rejected input, a render error — names the route id, the target kind, and the module provenance.

    Matchers over the Agent Document contracts: toHaveStatus, toContainMarkdown, toContainText, toHaveValue, toHaveError, and toHaveNodeKinds.

    This is the route-unit proof level, and only that: it proves a route module renders to the document it claims. It is not evidence about the MCP transport, a packed artifact, or a browser surface.

    Proof levels

    The levels are separate on purpose. Each helper stamps the level it carried into its provenance and prints it in every failure, because a pass at one level is never a receipt for another.

    LevelHelpersWhat it proves
    route-unitrenderRoute, renderRouteEventsA route module renders to the document — and render-event stream — it claims.
    mcp-in-memoryopenInMemoryMcpServer, invokeMcpTool, readMcpResource, getMcpPrompt, listMcpSurface, runContractMatrixThe real generated MCP server's protocol contract, over the SDK's in-memory transport.
    dev-epochrunDevEpochContractMatrixAn epoch-pinned generated stdio process opened through the Workbench session service; the caller owns the epoch lease and process lifetime, and MCP App routes are covered (surface plus ui:// sweep).
    cli-dispatchinvokeCli, cliJson, cliNdjsonA plain or rendered argv vector resolved and run through the routed CLI's own shell — including rendered Markdown, explicit TTY, JSON, and NDJSON modes — in-process.
    packed-stdioopenPackedMcpServer, runPackedContractMatrixA built artifact's generated entry running as a real process over stdio.
    packed-deleted-sourceremoveProjectSource, openPackedMcpServer({ deletedSource }), runPackedContractMatrixThe packed stdio process still runs after project source and configuration are removed and verified absent.
    host-installopenInstalledHostMcpServer, runInstalledHostContractMatrixA built bundle staged into an isolated host root, discovered in the emitted host format, and spawned from the installed layout.

    Two further levels sit alongside these seven, for nine in all. agent-bundle/test/browser supplies mountBrowserApp for the browser-safe browser-app level — production-compiled MCP App HTML mounted over the product bridge in a real browser page — and simulated reuses the installed-host helper openInstalledHostMcpServer without sessionEvidence: an emitted bundle staged directly into an isolated host-shaped root and spawned without a host-owned install, which is weaker than host-install.

    import { 
    const cliJson: (invocation: CliInvocation) => unknown

    The parsed canonical JSON line a successful command wrote to stdout.

    cliJson
    ,
    const cliNdjson: (invocation: CliInvocation) => readonly CliRenderedEvent[]

    The ordered render events a successful --ndjson invocation wrote to stdout.

    cliNdjson
    ,
    const invokeCli: (argv: readonly string[], options?: InvokeCliOptions | undefined) => Promise<CliInvocation>

    Dispatches one argv vector through the routed CLI shell in this process and returns its exit code, streams, and validated result.

    This is the cli-dispatch proof level. Nothing is spawned.

    invokeCli
    ,
    const invokeMcpTool: (tool: string, options?: McpInvocationOptions | undefined) => Promise<McpToolInvocation>

    Calls one compiled tool through the real protocol and returns the projected result. mcp-in-memory level: protocol contract proof, not process proof.

    invokeMcpTool
    } from 'agent-bundle/test';
    export const
    const proofs: () => Promise<Record<string, unknown>>
    proofs
    = async ():
    interface Promise<T>

    Represents the completion of an asynchronous operation

    Promise
    <
    type Record<K extends keyof any, T> = { [P in K]: T; }

    Construct a type with a set of properties K of type T

    Record
    <string, unknown>> => {
    // mcp-in-memory: the generated server projects the document to protocol content. const
    const call: McpToolInvocation
    call
    = await
    function invokeMcpTool(tool: string, options?: McpInvocationOptions | undefined): Promise<McpToolInvocation>

    Calls one compiled tool through the real protocol and returns the projected result. mcp-in-memory level: protocol contract proof, not process proof.

    invokeMcpTool
    ('summarize', {
    input?: unknown
    input
    : {
    title: string
    title
    : 'Dune' } });
    // cli-dispatch, plain .ts route: resolve argv, execute, and map the exit code. const
    const run: CliInvocation
    run
    = await
    function invokeCli(argv: readonly string[], options?: InvokeCliOptions | undefined): Promise<CliInvocation>

    Dispatches one argv vector through the routed CLI shell in this process and returns its exit code, streams, and validated result.

    This is the cli-dispatch proof level. Nothing is spawned.

    invokeCli
    (['library', 'audit', './books', '--max-files', '8']);
    // cli-dispatch, rendered .tsx route: exercise the shell's rendered output modes. const
    const rendered: CliInvocation
    rendered
    = await
    function invokeCli(argv: readonly string[], options?: InvokeCliOptions | undefined): Promise<CliInvocation>

    Dispatches one argv vector through the routed CLI shell in this process and returns its exit code, streams, and validated result.

    This is the cli-dispatch proof level. Nothing is spawned.

    invokeCli
    (['library', 'report', './books', '--ndjson']);
    const
    const events: readonly CliRenderedEvent[]
    events
    =
    function cliNdjson(invocation: CliInvocation): readonly CliRenderedEvent[]

    The ordered render events a successful --ndjson invocation wrote to stdout.

    cliNdjson
    (
    const rendered: CliInvocation
    rendered
    );
    // An explicit TTY proves the in-place progress path rather than the piped one. const
    const tty: CliInvocation
    tty
    = await
    function invokeCli(argv: readonly string[], options?: InvokeCliOptions | undefined): Promise<CliInvocation>

    Dispatches one argv vector through the routed CLI shell in this process and returns its exit code, streams, and validated result.

    This is the cli-dispatch proof level. Nothing is spawned.

    invokeCli
    (['library', 'report', './books'], {
    InvokeCliOptionsBase.tty?: boolean | undefined

    Selects interactive rendered output explicitly. Generated binaries use process.stdout.isTTY; the in-process harness defaults to piped output.

    tty
    : true });
    return {
    exitCode: number
    exitCode
    :
    const run: CliInvocation
    run
    .
    CliInvocation.exitCode: number

    The process exit code the routed shell mapped: 0 success (or the result's exitCode under config.exitCode: 'result'), 1 execution failure, 2 usage or input-validation failure.

    exitCode
    ,
    finalEvent: "shell" | "progress" | "replace" | "error" | "complete" | undefined
    finalEvent
    :
    const events: readonly CliRenderedEvent[]
    events
    .
    ReadonlyArray<CliRenderedEvent>.at(index: number): CliRenderedEvent | undefined

    Returns the item located at the specified index.

    @paramindex The zero-based index of the desired code unit. A negative index will count back from the last item.
    at
    (-1)?.
    type: "shell" | "progress" | "replace" | "error" | "complete" | undefined
    type
    ,
    inPlaceProgress: boolean
    inPlaceProgress
    :
    const tty: CliInvocation
    tty
    .
    CliInvocation.stdout: string

    Everything the shell wrote to stdout, including rendered Markdown, TTY, JSON, or NDJSON output.

    stdout
    .
    String.includes(searchString: string, position?: number): boolean

    Returns true if searchString appears as a substring of the result of converting this object to a String, at one or more positions that are greater than or equal to position; otherwise, returns false.

    @paramsearchString search string@paramposition If position is undefined, 0 is assumed, so as to search all of the String.
    includes
    ('\r\u001B[2K'),
    scanned: unknown
    scanned
    :
    function cliJson(invocation: CliInvocation): unknown

    The parsed canonical JSON line a successful command wrote to stdout.

    cliJson
    (
    const run: CliInvocation
    run
    ),
    structured: unknown
    structured
    :
    const call: McpToolInvocation
    call
    .
    McpToolInvocation.structuredContent?: unknown
    structuredContent
    ,
    }; };

    expectEvents asserts over a render-event stream. toContainSequence is sequence-tolerant — an extra progress or replace frame is legal and cannot turn a passing render red — while a missing frame, a reordering, or a regressed ordinal still fails. toHaveMonotonicSequence, toCompleteOnce, toHaveProgress, and toHaveNoErrors cover the rest of the contract.

    Process evidence is deliberately expensive

    Among the packed levels, only packed-stdio and its strictly stronger packed-deleted-source upgrade are process evidence: pack once, install once, build once, remove and verify source once, spawn once, and iterate every per-route assertion inside that one session. dev-epoch is process evidence of a different shape — the Workbench's own epoch-pinned generated stdio process, not a packed artifact — so a dev-epoch pass says nothing about what a pack ships. The deleted-source journey also reads the embedded MCP App resource from the generated server; it does not prove native-host install or dispatch, or an install mode that copies the artifact elsewhere.

    host-install is separate installed-layout process evidence. Its deterministic adapter-simulator lane is unconditional, available Claude and Codex binaries also prove their public install paths, and Cursor records its unavailable non-interactive host-session surface explicitly.

    The contract matrix

    The contract matrix is the framework-owned generated-plugin wire-contract suite. Three entry points share one implementation; boundary differences are explicit capability flags, not forked check logic. The project supplies only fixtures — valid inputs, a declared resultCompat policy for every in-memory tool route, optional previousResults payloads, optional cancellation cases, and an optional deterministic lifecycle transition driver with declarative expectations.

    runContractMatrix (mcp-in-memory) opens one real MCP client against the real generated server over the SDK's in-memory transport and runs the full matrix. It proves wire-surface completeness against the compiler manifest, fixture coverage, successful-path invocation sweeps, JSON serialized round-trip through each tool route's own resultSchema, declared additive or closed compat behavior on serialized payloads, acceptance of previous-server payloads under the current schema, rejection of negative inputs derived from the advertised listTools input JSON Schema, and mid-flight cancellation hygiene. In-memory transport may pass structured values without serialization, so the matrix closes that gap with an explicit JSON.parse(JSON.stringify(...)) round-trip before validation. MCP Apps are reported as not-applicable for surface registration, because the in-memory level does not register them.

    runPackedContractMatrix (packed-stdio / packed-deleted-source) runs against an already-open packed session — the single packed journey owns session open and close. It proves process stdio evidence for surface completeness (including compiled MCP App resource URIs in listResources), fixture coverage, successful-path sweeps, advertised input-schema rejection, and client-side cancellation hygiene. It cannot load project route modules, because source may be deleted and verified absent, so serialized-round-trip, compat-probe, and version-skew checks — including their per-lifecycle-phase variants — are reported not-applicable with an honest reason. The packed server validates every tool result through its bundled resultSchema before returning; a successful sweep invocation is that evidence.

    runInstalledHostContractMatrix (host-install) runs against an already-open session from openInstalledHostMcpServer. The opener reads the host's emitted MCP document from the installed root, verifies the manifest, the component, resource, and hook paths, and the artifact file digests, spawns that installed command, and observes the running version from the live MCP initialize result. Its report records source, built-artifact, installed-artifact, and running-process versions separately, and fails closed when any value is missing or differs. Metadata records the host binary version when observed, the adapter revision, the manifest and schema digest, and the framework version. Module-backed checks remain honestly not-applicable, because loading project modules would cross back into the source and build tree.

    import { 
    const runContractMatrix: (options: ContractMatrixOptions) => Promise<ContractMatrixReport>

    Runs the contract matrix against one compiled MCP server at the mcp-in-memory proof level. Returns a per-route report; throws one aggregated AgentTestError with code contract-violation when any check failed (never first-failure-only).

    runContractMatrix
    } from 'agent-bundle/test';
    export const
    const matrix: () => Promise<void>
    matrix
    = async ():
    interface Promise<T>

    Represents the completion of an asynchronous operation

    Promise
    <void> => {
    await
    function runContractMatrix(options: ContractMatrixOptions): Promise<ContractMatrixReport>

    Runs the contract matrix against one compiled MCP server at the mcp-in-memory proof level. Returns a per-route report; throws one aggregated AgentTestError with code contract-violation when any check failed (never first-failure-only).

    runContractMatrix
    ({
    ContractMatrixOptions.fixtures: Readonly<Record<string, ContractRouteFixture>>

    Route id -> fixture. Every compiled tool, prompt, and resource route on the server must be covered. App routes are not registered at mcp-in-memory; entries for them are accepted and ignored.

    fixtures
    : {
    'tool:library/summarize': {
    ContractRouteFixture.input?: unknown

    Valid input for the sweep invocation (tools/prompts; resources need none).

    input
    : {
    title: string
    title
    : 'Dune' },
    ContractRouteFixture.previousResults?: readonly unknown[] | undefined

    Serialized payloads captured from previous server versions; each must parse under the CURRENT resultSchema (previous-server + current-client skew).

    previousResults
    : [{
    chapters: number
    chapters
    : 24 }],
    ContractRouteFixture.resultCompat?: ResultCompatPolicy | undefined

    Declared serialized-result compatibility policy. REQUIRED for tool routes.

    resultCompat
    : 'additive',
    }, }, }); };

    The packed and installed-host entry points take the same fixture shape plus the session they run against: runPackedContractMatrix needs the open packed session and the manifest compiled before source removal, and runInstalledHostContractMatrix needs the session openInstalledHostMcpServer returned along with that manifest.

    Lifecycle fixtures

    Lifecycle fixtures replay unknown → queued → running → first-progress → repeated-progress → terminal over the matrix's one open client. The framework validates every phase's structured content and rendered output, additive and closed compatibility, live progress before settlement, journal accumulation, declared notices, idempotent commit replay, and typed budget rejection.

    A caller-supplied same-store restart callback adds durability evidence at that boundary; without one the check is honestly not-applicable. Packed callers should wire that callback into the existing packed journey's restart rather than creating a second pack, build, and install path. A lifecycle fixture's optional state.catalog assertion pins its declared id and lifetime to the compiler manifest used by that same mounted-state replay.

    Event routes and runtime identity

    When the compiled manifest contains event routes, the packed and installed-host boundaries sample the read-only event-runtime status before and throughout sequential matrix events. The runtime-instance-identity check fails if the warm instanceId changes, the artifact epoch drifts, or availability degrades to runtime-restarted or runtime-unavailable. In-memory runs and compiled artifacts without event routes report runtime identity as honestly not-applicable.

    No matrix boundary proves browser App HTML or artifact-rebuild replay.

    When the advertised input schema declares additionalProperties: false, plain z.object tool routes may still strip unknown keys without a protocol failure. The negative-inputs check records that tolerance when other generated negatives still prove rejection paths.

    A failing matrix throws one aggregated AgentTestError with code contract-violation, naming every failing route, every failing check, and the proof-level label the run actually carried.

    Next