Otter Engine Contributor Guide

This book is the contributor-facing guide for Otter's active new engine. It explains how to extend the runtime without copying internals from task files, parked crates, or raw GC adapters.

Use this book for stable contributor workflows and examples. Repository rules for coding agents live in AGENTS.md.

Historical task and ADR files are intentionally excluded from the living docs. When a contributor-facing API stabilizes, its workflow belongs here.

Local Build

Build the book with:

mdbook build docs/book

The docs examples that exercise current GC APIs are backed by Rust tests in crates/otter-gc; run them with the normal task gates:

cargo test -p otter-gc

If mdbook is not installed, install it with normal Rust tooling outside this repository.

Contributing Overview

Otter's active engine and product crates live under crates/.

Start with:

• Architecture for active crate ownership;
• Frontend And Compilation before changing parser, module detection, or TypeScript behavior;
• ES_CONFORMANCE.md before changing ECMAScript behavior;
• AGENTS.md for repository-specific workflow rules.

This book is the contributor-facing source of truth for stable workflows. Task files are implementation plans and closeout history; when a workflow stabilizes, document how to use it here.

Choosing A Crate

• GC storage, tracing, handles, weak/finalization, heap stats, snapshots, and external-memory accounting belong in crates/otter-gc.
• Value representation, object model, bytecode execution, intrinsics, native callable dispatch, and source/compiler integration belong in crates/otter-vm.
• Public embedding, capabilities, event-loop handles, worker/isolate runners, and host-operation scheduling belong in crates/otter-runtime.
• CLI behavior belongs in crates/otter-cli.
• New Web/API/module/product crates belong under crates/*.

Do not introduce a parallel runtime stack or copied compatibility modules.

Working Rules

• Keep changes vertical and reviewable.
• Prefer improving active APIs over preserving unsafe, slow, startup-heavy, or confusing compatibility shims.
• Do not add thread-local heap lookup or context-free GC access.
• Keep unsafe code inside otter-gc. Other active crates keep #![forbid(unsafe_code)]; audited VM adapters may call doc-hidden raw GC APIs but must not make them contributor-facing.
• Update runtime behavior, TypeScript declarations, docs, and tests together when a public surface changes.
• High-level APIs are welcome only when they compile down to the same runtime shape as handwritten code. Add benchmarks when changing native dispatch, bootstrap, or startup.
• Use AST tooling such as oxc/SWC for JS/TS parsing or transforms. Do not regex-parse JavaScript or TypeScript.
• Preserve deterministic output order where observable. Prefer ordered maps or explicit sorting for JSON, object-key snapshots, tests, and iterator output.

Common Commands

cargo build
cargo test --all --all-features
cargo fmt --all
cargo clippy --all-targets --all-features -- -D warnings
mdbook build docs/book

Fast loops:

cargo test -p otter-gc
cargo test -p otter-vm
cargo test -p otter-runtime

For feature work covered by Test262, establish a targeted baseline first, fix by failure category, then record the before/after pass rate in the task closeout or PR notes. If a change significantly changes conformance, regenerate the conformance report through the repository-approved Test262 commands before closing the task.

Tests And Examples

Match test depth to risk:

• narrow helper behavior: focused unit or integration tests;
• GC/worker/session misuse: compile-fail tests;
• public JS-visible behavior: engine fixtures and targeted Test262;
• contributor APIs: rustdoc examples or book-backed integration tests.

Book examples for active APIs should either compile through normal cargo gates or point at the exact test file that backs them. Design-only snippets are shown as ignore.

Closing Work

Close implementation work only after its validation gates are actually green. Record what shipped, note command output or blockers, and keep the book aligned with any workflow that changed.

Porting Process

This process applies when moving behavior from parked compatibility crates, reference implementations, or external sources into the active engine.

Use durable markers for uncertain migrations:

• TODO(port): <reason> when behavior is not fully understood yet.
• PERF(port): <original invariant> - profile before optimizing when replacing a known hot-path trick with idiomatic Rust.
• PORT NOTE: <why shape changed> when GC rooting, scheduling, ownership, or borrow-checker constraints require a control-flow change.

Do not use todo!() or unimplemented!() in reachable runtime code.

Keep JS-visible ports as vertical slices. Update runtime behavior, TypeScript declarations, docs, examples, and targeted tests together when the surface is observable by users.

Preserve active-stack boundaries:

• VM/runtime work belongs in crates/otter-gc, crates/otter-vm, and crates/otter-runtime.
• Standards-facing Web APIs belong in crates/otter-web.
• Otter-hosted modules belong in crates/otter-modules.
• Do not add active path-dependencies on code outside the active crates/* stack.

For large ports, add a short status block when it helps review:

#![allow(unused)]
fn main() {
// PORT STATUS
//   source:     <parked crate/file or reference area>
//   confidence: high | medium | low
//   todos:      N
//   tests:      <targeted tests or reason omitted>
//   notes:      <one line for reviewers>
}

Test Harness

The active otter test harness lives in crates/otter-test and is exposed through the CLI. It discovers JavaScript and TypeScript fixtures, runs each fixture in a fresh runtime, and reports structured outcomes.

Supported suites:

• engine: first-party engine fixtures under tests/engine;
• smoke: short release smoke tests under tests/smoke;
• test262: curated Test262 fixtures under tests/test262-curated.

Each fixture may carry TOML metadata in the source header. The runner uses that metadata for expected exit codes and other fixture-level expectations. When --json is enabled, output is newline-delimited JSON records followed by a summary report using HARNESS_SCHEMA_VERSION.

Discovery skips helper/package directories rather than treating them as standalone tests:

• directory names starting with _;
• node_modules.

Use focused suite/filter runs for local iteration, then run the relevant workspace tests before merging harness behavior changes.

Engine Architecture

The new engine is split into focused crates:

• otter-gc: page-based generational tracing GC;
• otter-bytecode: bytecode representation and disassembly;
• otter-syntax / otter-compiler: frontend and lowering;
• otter-vm: interpreter, object model, intrinsics, and runtime state;
• otter-runtime: embedding/runtime surface;
• otter-cli: command-line entry point.

One JavaScript isolate owns one VM, one runtime state, and one GC heap. Async and worker APIs must preserve that boundary. Values move between workers through structured clone or transferables, not raw GC handles.

Pipeline

Source flows through the active frontend and VM stack:

source file / source string
  -> otter-syntax / otter-compiler
  -> otter-bytecode
  -> otter-vm Interpreter
  -> otter-runtime facade / CLI / embedder

The compiler should use AST APIs for JS/TS analysis and transforms. Bytecode and VM changes should keep debug/disassembly output stable enough for trace and Test262 triage.

Runtime Boundary

RuntimeCx<'_> is the internal VM mutator context. It carries the active interpreter/heap borrow so VM helpers cannot silently use thread-local heap state. NativeCtx<'_> is the public native-binding view exposed to builtin and extension authors.

Both contexts are isolate-local and must not cross .await, worker boundaries, runtime inbox messages, or host-operation futures. Async work copies owned host data out, then re-enters the isolate later with an owned completion.

Async-First Runtime

The active runtime model is async-first. Otter is the public async-capable facade over RuntimeHandle and the isolate runner. CLI execution runs from an async main and awaits Otter execution directly; embedding entry points may expose async or sync caller ergonomics, but observable JavaScript semantics still converge on the same async-capable runtime machinery.

Blocking APIs are convenience adapters. They may block the caller while the same async-capable runtime handle drives the isolate, but they must not grow a second sync-only engine path that bypasses timers, host ops, workers, module loading, or async Web APIs.

Runtime remains the local isolate layer for tests, compile/check/dump workflows, and low-level embedders that deliberately drive the VM in-process. It is not a separate product runtime with different semantics. If behavior is observable from JavaScript, the Otter/RuntimeHandle path and the local Runtime path must converge on the same VM/runtime state machinery.

GC And Handles

The GC is page-based, moving, generational, and isolate-local. Normal engine work uses safe handles and context helpers:

• stack-scoped Local<'gc, T> for temporary roots;
• EscapableHandleScope when one local must leave a nested scope;
• branded Root<'iso, T> for persistent isolate-owned references;
• branded Weak<'iso, T> for weak references upgraded only through a matching session;
• NativeCtx::record_write / GcHeap::record_write for stores;
• ExternalMemory for native/off-object bytes.

Raw collector types live behind otter_gc::raw for audited adapters. They are not a contributor API.

Modules And Permissions

Module loading, hosted modules, Web APIs, and Node-style surfaces must enforce capabilities at the Rust boundary. Type declarations and JS shims are useful ergonomics, not security boundaries. Capability checks should happen before host work is started and before native resources are opened.

Bootstrap

Builtin and extension surfaces should install through a centralized bootstrap registry backed by static specs and mutator-bound builders. This keeps contributor ergonomics high while preserving write barriers, deterministic install order, fast native-call dispatch, and startup benchmark visibility.

The registry lives in otter-vm::bootstrap as a static ordered slice of install entries. Each entry declares a global name, required bootstrap feature bits, and a plain installer function. Installers receive an explicit &mut GcHeap plus globalThis; migrated surfaces use otter-vm::js_surface specs/builders, and unmigrated globals remain small placeholders until their own slices land.

The first migrated namespace is Math. Direct Math.<fn>(...) syntax still uses the existing bytecode fast path, while observable property reads and extracted method calls use the real namespace object installed from math::MATH_SPEC.

Default-off bootstrap telemetry is available for benchmark runs. The plain runtime construction path does not maintain telemetry counters; benches can call the instrumented bootstrap entry point to capture install counts, GC allocation deltas, duplicate-name validation, and per-entry timing.

Debug Workflows

Use the existing machine-readable trace and profiling outputs when debugging engine behavior:

• VM instruction trace for stuck bytecode loops;
• timeout dumps for hangs;
• Chrome/Perfetto async trace for host-op scheduling;
• Chrome/V8 .cpuprofile plus folded stacks for CPU work.

New debug/profiling features should stay default-off and should use standard output formats where possible.

Documentation for stable contributor workflows belongs in this book. Historical task and ADR files are not part of the living contributor docs.

Event Loop And Async Boundary

Otter's public runtime is handle-first and async-friendly, but one isolate still owns one VM, one runtime state, and one GC heap. The public handle may be Send + Sync; the isolate internals are not.

The default product path is async-first. CLI execution starts in an async main and awaits the public Otter/RuntimeHandle stack directly. Blocking wrappers exist only as sync-caller conveniences. Blocking does not mean a separate synchronous runtime: the same event-loop-capable isolate runner must remain available for timers, host ops, dynamic modules, workers, and future async Web APIs.

The production event-loop boundary follows Deno's JsRuntime shape: the runtime itself stays local to one isolate, while embedders drive it with one-tick and run-to-idle style APIs. Boa's job model is the smaller ECMA-262 reference: promise, timeout, native async, and generic jobs run only when no execution context is active and each job runs to completion.

Runtime Layers

The intended shape is:

Otter              // public facade; Clone + Send + Sync
  -> RuntimeHandle // bounded command/completion API
    -> IsolateRunner
      -> RuntimeCore / Interpreter / RuntimeState / GcHeap // !Send + !Sync

Tokio is the default scheduler in otter-runtime, but VM crates must not import Tokio types.

Queues

Do not overload one queue for all async work:

• VM microtask queue: Promise reactions, queueMicrotask, async-function resume, and await resume.
• Runtime inbox: commands, host-op completions, timers, dynamic module completion, interrupts, inspector/debug events, and shutdown.

A runtime turn runs JS work on the mutator, performs a microtask checkpoint, then folds host completions into the runtime inbox according to the selected drive mode.

Microtask checkpointing is VM work. Promise reactions, queueMicrotask, and async-function resumes run only after the current JS execution context unwinds and before the runtime turn is considered complete.

Tokio-specific state belongs in the runtime's internal event-loop and host-service layer. Runtime handles carry owned command payloads and settlement messages; timer callbacks re-enter the isolate by opaque timer token. Handles do not hold VM values, GC handles, or executor locks.

Drive Modes

The runner should support deterministic drive modes:

• poll_one_tick: process at most one event-loop turn and checkpoint;
• run_until_idle: run referenced work until the runtime is idle;
• run_until_promise: drive until a target promise settles or the loop becomes idle with that promise still pending;
• run_until_command: drive until a command completion is delivered;
• shutdown: cancel or drain, then report leaks.

Async Host Ops

Native async APIs must split at the runtime boundary:

1. validate arguments and permissions on the isolate thread;
2. copy owned host data;
3. create a pending promise / operation id;
4. run Rust async work on the event loop without VM references;
5. post an owned completion back to the isolate;
6. resolve or reject the promise on a later mutator turn;
7. run the microtask checkpoint.

Never move RuntimeCx, NativeCtx, Value, Frame, Gc<T>, Local<'gc, T>, or handle scopes into a Rust future.

Host operations should be exposed through narrow runtime-owned services or typed inbox messages. The isolate runner receives only owned completion data on a later turn, then performs the JS-side resolution/checkpoint work on the mutator thread.

Cancellation and backpressure are runtime-handle concerns. Dropping or aborting host work must not leave a JS promise in an untracked state: record the operation id, decrement liveness counters, and settle or report the pending JS work on the isolate turn that observes cancellation.

Liveness And Diagnostics

Timers and host ops have ref/unref liveness. Referenced work keeps run_until_idle alive; unreferenced work may finish if the loop is already being driven but must not keep the runtime alive by itself.

Use ref/unref deliberately:

• Ref for work that the user can observe and that should keep run_until_idle alive;
• Unref for background diagnostics or cache cleanup that may complete opportunistically but must not prevent idle shutdown.

Contributor tests should be able to inspect activity stats: pending commands, timers, host ops, dynamic module jobs, microtasks, cancellations, timeouts, and leaked work at shutdown.

RuntimeHandle::activity_stats() exposes cheap aggregate counters for this purpose. Detailed tracing should stay opt-in so native dispatch and script startup keep their steady-state cost.

Frontend And Compilation

Otter parses JavaScript and TypeScript through OXC. The active frontend stack is:

• crates/otter-syntax: source kind detection, OXC parse options, and parse-once callbacks.
• crates/otter-compiler: AST-to-bytecode lowering and TypeScript erasure.
• crates/otter-bytecode: bytecode module, disassembly, and JSON dump formats.

Do not regex-parse JavaScript or TypeScript source. Consumers that need to inspect module syntax or other AST properties should use otter_syntax::with_program and reuse the parsed OXC program when compiling script sources.

The foundation TypeScript policy is:

• erase type-only constructs such as interface, type, declare, import type, export type, abstract methods, as, satisfies, non-null assertions, and type instantiation syntax;
• reject runtime TypeScript constructs that cannot be erased cleanly in the current engine slice, including enum, runtime namespace, and decorators;
• preserve original source spans through diagnostics and stack traces.

Bytecode dumps are part of the supported CLI/debugging surface:

otter --dump-bytecode path/to/script.js
otter --dump-bytecode=json path/to/script.ts

The text dump starts with:

; otter bytecode dump v1 - module=<specifier> source_kind=<javascript|typescript>

The JSON dump is intended for tools and tests. Keep it stable when possible; when the schema must change, update tests, docs, and any downstream consumers in the same patch.

GC API

Otter's active GC is moving, generational, and isolate-local. Normal engine and extension work should use the safe context API rather than raw collector internals.

The landed contributor API is the safe context surface. Trace derive/macros must generate normal trace code over the safe visitor path; do not invent a macro-first GC API that exposes raw collector internals.

Handle Tiers

• Local<'gc, T> is a stack-scoped root created by a handle scope.
• EscapableHandleScope<'gc> is the explicit way to return one Local<'gc, T> from a nested scope.
• Root<'iso, T> is a persistent isolate-owned root.
• Weak<'iso, T> is a weak handle. It can only be upgraded through a matching GcSession<'iso, '_>.
• Raw Gc<T> handles are VM values, not persistence handles. Do not store them across async, worker, or host-operation boundaries; use a Root and re-enter the owning isolate.

Native Context

Native functions receive NativeCtx<'_>. The public mutable raw heap borrow is intentionally not available to native authors. Use these helpers instead:

fn native(
    ctx: &mut otter_vm::NativeCtx<'_>,
    _args: &[otter_vm::Value],
    _captures: &[otter_vm::Value],
) -> Result<otter_vm::Value, otter_vm::NativeError> {
    let object = ctx.alloc_old(MyBody::default())?;
    ctx.record_write(object, &otter_vm::Value::Undefined);

    let backing = ctx.reserve_external(4096)?;
    drop(backing);

    Ok(otter_vm::Value::Undefined)
}

Current stable helpers:

• alloc / alloc_old: allocate through the owning isolate;
• record_write: record outgoing GC edges after a store;
• reserve_external: account native/off-object backing stores with RAII;
• with_gc_session: enter branded root/weak operations;
• interp_mut: use isolate services such as microtasks or string tables.

NativeCtx::heap() is an immutable diagnostic/read path. The mutable heap borrow is crate-private; contributor code should not need it.

Use NativeCtx::with_gc_session when a native path needs branded root or weak operations:

ctx.with_gc_session(|mut session| {
    let local = session.alloc(MyBody::default())?;
    let root = session.root(local);
    let weak = session.weak(root.get(&session));
    assert!(weak.upgrade(&session).is_some());
    Ok::<_, otter_gc::OutOfMemory>(())
})?;

Mutation

Do not call write barriers directly. Store the value first, then record the store through GcHeap::record_write or NativeCtx::record_write. The stored value implements GcStore, and the heap records every outgoing GC edge without exposing raw slot pointers:

let stored = value.clone();
heap.with_payload(parent, |body| {
    body.field = value;
});
heap.record_write(parent, &stored);

This is the reference pattern used by object properties, array elements, Map/Set entries, promises, generators, upvalues, and finalization registries.

For containers that can hide GC edges, implement or reuse GcStore so the heap sees every outgoing edge without exposing raw slot pointers to the caller.

Escaping Locals

Use EscapableHandleScope when a helper opens a nested handle scope and needs to return one rooted value to the caller's scope:

let escaped = {
    let mut inner = otter_gc::EscapableHandleScope::new(heap.handle_stack());
    let local = inner.local(gc_value);
    inner.escape(&local)
};

The runnable copy of this pattern is covered by crates/otter-gc/tests/book_gc_api_examples.rs and the EscapableHandleScope rustdoc example.

External Memory

Memory outside GC cells must be accounted with an RAII reservation:

let mut backing = heap.reserve_external(16 * 1024)?;
backing.resize(32 * 1024)?;
drop(backing); // releases the reservation

This covers typed-array backing stores, host buffers, large module source caches, and native resources.

The runnable copy of this pattern is covered by crates/otter-gc/tests/book_gc_api_examples.rs and the ExternalMemory rustdoc example.

Worker And Async Boundaries

Never send Gc<T>, Local<'gc, T>, Root<'iso, T>, Weak<'iso, T>, RuntimeCx<'_>, NativeCtx<'_>, or VM Value into worker messages or Rust futures. Use owned structured-clone payloads, transferable metadata, or host-owned byte buffers. Re-enter the isolate with an owned completion and only then touch JS values.

Trace Ergonomics

Today, VM payloads implement the current tracing traits manually or reuse existing wrappers. GcTrace derive macros must generate normal trace code over the safe visitor path. Do not add contributor macros that expose raw trace tables, raw slot visitors, or manual barrier calls.

Internal Only

The following are collector or audited VM-adapter internals:

• RawGc
• TraceTable
• raw slot visitors (*mut RawGc)
• GcHeap::write_barrier_raw
• direct handle-table mutation
• context-free weak upgrades

Raw collector types are not re-exported from the root otter_gc API. Audited VM adapters may import otter_gc::raw::*; contributor code should treat that module as unavailable. Compile-fail gates reject root-level raw imports and direct raw barrier calls.

Package Manager Development Loop

Otter's package-manager loop is intentionally flat:

otter init -y
otter add left-pad
otter install
otter check app.ts
otter run app.ts
otter run --bin fixture-tool
otter test
otter outdated

otter run is the single execution command for files, package scripts, and local package binaries. There is no separate otter exec command and no first-party otter build command in this phase.

Commands

Package-manager commands are explicit first-party CLI operations. Registry network access, project/cache filesystem writes, and install lifecycle subprocesses are not gated by runtime capability flags. Runtime APIs, dynamic imports, and hosted APIs remain capability-gated.

Lockfile Migration

Otter's native lockfile is otter.lock. It is the only lockfile format Otter writes.

For migration, Otter accepts existing npm and pnpm lockfiles when otter.lock is not present:

1. pnpm-lock.yaml
2. npm-shrinkwrap.json
3. package-lock.json

Those files are normalized into the runtime package graph in memory. This lets commands such as otter run, otter check, otter test, otter remove, and otter outdated work against already materialized node_modules during a migration. otter install also consumes the foreign lockfile, materializes the recorded tarballs, records package metadata from extracted manifests, runs available install lifecycle hooks, and writes a native otter.lock.

Lifecycle Hooks

Otter runs package lifecycle scripts during explicit package-manager installs. The install hook subset follows pnpm's dependency install path:

1. preinstall
2. install
3. postinstall

Ordinary package scripts such as build, test, or prepare are not treated as install lifecycle hooks. For extracted package tarballs, Otter also records and runs pnpm/npm-compatible implicit install = "node-gyp rebuild" when a package has binding.gyp and no explicit preinstall or install script. Lifecycle scripts run with the package root as the working directory and project-local node_modules/.bin on PATH.

Outdated

otter outdated compares three versions:

• Current: installed version from otter.lock.
• Wanted: newest registry version satisfying the manifest range.
• Latest: registry latest dist-tag, falling back to the highest semver version when the tag is missing.

The table includes a Bump column:

By default outdated checks dependencies. Use --dev, --peer, or --optional to include the other manifest buckets.

otter outdated --dev --optional

outdated exits with status 1 when at least one dependency is outdated and 0 when everything checked is current.

Resolver Contract

When a package graph is available, it is authoritative for bare package imports from graph-contained packages:

• undeclared bare package imports are rejected even when a matching directory exists under node_modules;
• package self-reference by package name is allowed without a dependency edge;
• optional and peer dependencies have distinct diagnostics when missing;
• peer dependencies may resolve to an already installed package with the same name when the peer range edge points at an unmaterialized placeholder;
• package exports, package imports, main, module, package type, and dependency edge kinds are carried into the runtime through lightweight DTOs.

otter-runtime does not depend on otter-pm; product crates adapt the richer package-manager graph into the runtime DTO.

Extensions Overview

Otter's extension model is layered:

1. hosted modules inside the workspace;
2. native bindings compiled with the engine;
3. source-level plugin packages when a stable extension crate exists;
4. ABI/FFI plugins only with explicit versioning and ownership rules.

All layers must preserve the same runtime rules:

• permissions are deny-by-default;
• no raw GC handle crosses isolate or worker boundaries;
• persistent JS-visible state uses Root;
• weak handles upgrade only through a matching context;
• external memory is accounted;
• async work hops back to the isolate before touching VM state.

JavaScript-visible surfaces should use the production spec/builder flow:

• static specs declare names, arity, attributes, and native targets;
• builders install specs through explicit RuntimeCx / NativeCtx;
• centralized bootstrap owns global/prototype/module install order;
• macros, when available, generate the same static specs rather than a separate runtime registry.

The first stable builder/spec backend lives in otter-vm::js_surface plus the centralized otter-vm::bootstrap registry. New JS-visible surfaces should use that path unless capability checks or delicate install order require a small manual installer that still calls the same builders.

Breaking changes to extension APIs are allowed while the active engine API is pre-stable if they improve safety, startup, or steady-state performance.

Plugin details stay design-only until the API is stable enough to document here fully.

Extension Checklist

For any new JS-visible API:

• choose the active crates/* crate that owns the behavior;
• decide whether the surface is a global, builtin class, namespace, hosted module, or runtime-only helper;
• enforce permissions at the Rust boundary;
• allocate, root, mutate, and account memory through context APIs;
• add runtime tests and TypeScript declarations when the surface is public;
• add docs here when the workflow is stable.

Hosted Modules

Hosted modules expose native Rust functionality to JavaScript through the runtime.

Use hosted modules for Otter-owned APIs such as:

• otter:kv;
• otter:sql;
• otter:ffi;
• standard-facing or runtime-specific modules.

Hosted modules must enforce capabilities at the Rust boundary. Do not trust JavaScript wrappers or TypeScript declarations as the only permission check.

The hosted module crate is crates/otter-modules.

Resolution

Hosted modules are registered on the runtime builder:

let mut runtime = otter_runtime::Runtime::builder()
    .hosted_modules(otter_modules::hosted_modules().iter().copied())
    .build()?;

The module loader resolves registered otter:* specifiers directly to their specifier string. The module graph creates a hosted module namespace instead of reading a source file, so JavaScript can import the surface with normal ESM syntax:

import { openKv } from "otter:kv";
const store = openKv(":memory:");

Unregistered otter:* specifiers fail resolution.

File Layout

Use the repository naming convention:

Rust code owns permissions, resource opening, async dispatch, GC rooting, and external-memory accounting. JavaScript shims may normalize arguments or provide ergonomic exports, but they must not be the only enforcement layer.

Native Boundary

Hosted module native functions should:

1. validate arguments on the isolate thread;
2. check capabilities before opening host resources;
3. allocate through NativeCtx;
4. store persistent JS references as branded roots, not raw Gc<T>;
5. account host buffers with ExternalMemory;
6. for async work, copy owned host data into the future and post an owned completion back to the isolate.

Do not move VM values, handles, contexts, frames, or handle scopes into Rust futures.

Bootstrap And Builders

The production builder/spec flow handles namespace installation. Hosted modules should use runtime-owned specs such as RuntimeNamespaceSpec or the HostedModuleCtx / RuntimeObjectBuilder API when their surface needs capability-aware installation. Keep module registration centralized and easy to audit. If capability enforcement or bootstrap order is delicate, prefer explicit manual code over hiding control flow behind a macro.

Module namespaces that need runtime capabilities install through HostedModuleCtx::method with HostedNativeCall::dynamic(...) closures that capture owned, Send + Sync host data such as a cloned CapabilitySet. Plain namespace exports should use HostedModuleCtx::builtin_method, HostedModuleCtx::property, and HostedModuleCtx::readonly_property. Receiver-backed resource objects should be created with RuntimeObjectBuilder::from_host_data(ctx, data) and accessed through runtime_with_host_data / runtime_with_host_data_mut. This is still a static registration path: the module specifier list is fixed, resolution is centralized, and no per-call metadata parser or hot-path dynamic registry is introduced.

Macros are appropriate when they generate the same static specs and builder calls a manual implementation would write. If a module surface needs new macro ergonomics, add the macro over the builder API rather than bypassing it.

Active Modules

The current active slices are:

• otter:kv: openKv / kv, with in-memory and file-backed JSON stores.
• otter:sql: openSql / sql, backed by SQLite.
• otter:ffi: dlopen, permission-checked library loading metadata.

otter:kv and otter:sql have module-graph tests that import the module specifier and execute the exported native functions.

Native Bindings

Native bindings run on the isolate mutator thread and receive explicit runtime context. They must not reach for thread-local heap state or move VM / GC handles into Rust futures.

The current safe path is:

• use NativeCtx for allocation and mutation;
• use NativeCtx::record_write or higher-level container helpers after storing GC-bearing values;
• use NativeCtx::reserve_external for host buffers and backing stores;
• use NativeCtx::with_gc_session for branded roots and weak handles;
• enforce capabilities at the Rust boundary before starting host work;
• for async host work, copy owned host data, create an operation id and pending promise, run the async phase without VM references, then post a completion back to the isolate.

Specs/builders expose functions, classes, namespaces, and accessors. Production builtins should use runtime-owned helpers such as runtime_method(...), RuntimeObjectBuilder::builtin_method(...), or runtime_native_static(...) by default. Dynamic closures are reserved for embedder cases that need captured Rust state and can still trace explicit JS captures.

Hosted module namespace installers should use HostedModuleCtx and attach long-lived Rust state to receiver objects through the runtime host-object primitive. Namespace-level closures may capture owned configuration such as a cloned capability set, but per-instance state should live on the JS object and be reached through runtime_this_object(...) plus runtime_with_host_data(_mut). Closures must not capture RuntimeCx, NativeCtx, Value, Gc<T>, Local<'gc, T>, frames, or handle scopes.

Source/module loading is separate from filesystem I/O permissions. Following Deno's model, the entrypoint and statically analyzable local module graph are code loading, not fs_read. Runtime APIs that expose arbitrary file reads must still enforce CapabilitySet::read; future non-analyzable dynamic local imports and remote imports should use an explicit import policy rather than piggybacking on ordinary file I/O.

Embedder Console Sink

globalThis.console is installed through the same static namespace spec path as other builtins, but its output target is embedder-overridable. The default runtime config uses StdConsoleSink, which writes log, info, and debug with println!, and warn, error, trace, and failed assert with eprintln!.

Embedders that need structured logging can provide a sink while building the runtime:

use std::sync::Arc;
use otter_runtime::{ConsoleLevel, ConsoleSink};

#[derive(Debug)]
struct TracingConsole;

impl ConsoleSink for TracingConsole {
    fn write(&self, level: ConsoleLevel, fields: &[String]) {
        tracing::info!(?level, message = fields.join(" "));
    }
}

let otter = otter_runtime::Otter::builder()
    .console_sink(Arc::new(TracingConsole))
    .build()?;

The sink receives already-rendered JS argument fields in call order. It must not store VM values, GC handles, or native contexts.

Synchronous Native Shape

use otter_runtime::{
    RuntimeNativeCtx as NativeCtx, RuntimeNativeError as NativeError, RuntimeValue as Value,
    runtime_arg_to_string, runtime_string_value,
};

fn read_flag(
    ctx: &mut NativeCtx<'_>,
    args: &[Value],
) -> Result<Value, NativeError> {
    check_permission(ctx, "env")?;
    let name = runtime_arg_to_string(args, 0);
    let value = read_allowed_env(name)?;
    runtime_string_value(ctx, &value)
}

This snippet is shape-only because string/value helper names continue to move. The stable rule is that permission checks and allocation happen through the explicit native context.

To expose that function as a static builtin, put it behind a spec and let bootstrap or a mutator-bound builder install it:

use otter_runtime::{
    RuntimeMethodSpec as MethodSpec, runtime_method,
};

static READ_FLAG: MethodSpec = runtime_method("readFlag", 1, read_flag);

Async Native Shape

use otter_runtime::RuntimeNativeCtx as NativeCtx;

fn start_async_read(ctx: &mut NativeCtx<'_>, path: PathBuf) -> Result<OpId, Error> {
    check_read_permission(ctx, &path)?;
    let op_id = create_pending_promise(ctx)?;
    queue_owned_host_request("fs.readText", op_id, path);

    Ok(op_id)
}

The host request owns PathBuf, ids, and strings only. It does not capture NativeCtx, VM values, handles, or heap references. Completion must return through a typed runtime inbox message or service result, and promise settlement happens back on the isolate thread.

Macros may eventually reduce boilerplate, but they are syntax sugar over static specs and builders. Manual code is preferred when capability checks, bootstrap order, or async scheduling must stay explicit.

JS Surface Builders

Otter's preferred contributor API for JavaScript-visible surfaces is a static spec plus mutator-bound builder flow.

The examples below document the required generated/runtime shape. Product crates should use the runtime-owned facade in otter_runtime; the VM keeps the static backend and centralized bootstrap internals. Math, JSON, Atomics, console, and active Web API classes are installed through this path.

The goal is high-level ergonomics without runtime overhead:

• exported JavaScript names, arity, and attributes live in static specs;
• builders install those specs through RuntimeCx / NativeCtx during a mutator turn;
• all property writes go through the object model so write barriers fire;
• production builtins use a static native function-pointer path by default;
• dynamic boxed closures are reserved for rare host/embedder cases that need captured Rust state;
• bootstrap install order is centralized and deterministic.

Shape

The intended model is:

#![allow(unused)]
fn main() {
use otter_runtime::{
    RuntimeAttr as Attr, RuntimeConstValue as ConstValue,
    RuntimeConstSpec, RuntimeMethodSpec, RuntimeNamespaceSpec as NamespaceSpec,
    RuntimeNativeCtx as NativeCtx, RuntimeNativeError as NativeError,
    RuntimeValue as Value, runtime_constant, runtime_method, runtime_namespace,
};

fn math_abs(ctx: &mut NativeCtx<'_>, args: &[Value]) -> Result<Value, NativeError> {
    let _ = (ctx, args);
    Ok(Value::Undefined)
}

static MATH_METHODS: [RuntimeMethodSpec; 1] = [runtime_method("abs", 1, math_abs)];

static MATH_CONSTANTS: [RuntimeConstSpec; 1] =
    [runtime_constant("PI", ConstValue::Number(std::f64::consts::PI))];

static MATH_SPEC: NamespaceSpec = runtime_namespace(
    "Math",
    &MATH_METHODS,
    &[],
    &MATH_CONSTANTS,
    Attr::global_binding(),
);
}

Builders are lifetime-bound to the current mutator turn:

let mut object = otter_runtime::RuntimeObjectBuilder::new(ctx)?;
object.builtin_method("abs", 1, math_abs)?;
let namespace = object.build();

Do not store builders, contexts, Value, Gc<T>, Local<'gc, T>, or VM frames in async host futures.

Spec Records

• RuntimeAttr: writable/enumerable/configurable attributes with explicit defaults and helpers such as builtin_function, read_only, and global_binding;
• RuntimeConstValue / RuntimeConstSpec: static primitive values and constant/data properties;
• RuntimePropertySpec: data properties and constants;
• RuntimeMethodSpec: exported name, .length, attributes, and native call target;
• RuntimeAccessorSpec: getter/setter pair and accessor attributes;
• RuntimeConstructorSpec: constructor function, prototype, statics, and prototype methods;
• RuntimeClassSpec: class-shaped constructor/prototype/static surface;
• RuntimeNamespaceSpec: namespace object or hosted module namespace.

Specs contain only static metadata and native targets. They must not hold Gc<T>, Local<'gc, T>, RuntimeCx, NativeCtx, VM frames, or runtime locks.

Builders And Bootstrap

Builders install specs during a mutator turn through explicit context APIs. They may allocate and mutate JS objects, but they must perform those stores through the object model so barriers fire.

The centralized bootstrap registry owns deterministic install order, duplicate-name validation, feature/capability gating, and any lazy/tiered installation choices. Do not scatter ad-hoc global mutation across builtin modules.

Native Calls

Native call storage is split into a static fast path and a dynamic path:

use otter_runtime::{RuntimeNativeCall, RuntimeNativeFastFn, RuntimeNativeFn};
use std::sync::Arc;

pub enum RuntimeNativeCall {
    Static(RuntimeNativeFastFn),
    Dynamic(Arc<RuntimeNativeFn>),
}

Spec-declared builtins and macro-generated builtins should use runtime_method(...), RuntimeObjectBuilder::builtin_method(...), or runtime_native_static(...) by default. Use dynamic closures only when the embedder needs captured Rust state, and keep traced JS captures explicit. Crate-internal VM helpers may still use local unchecked constructors for audited isolate-local payloads.

Current Migration

The first migrated namespaces are Math, JSON, Atomics, and console:

• globalThis.Math is installed from math::MATH_SPEC;
• Math constants use non-writable, non-enumerable, non-configurable descriptors;
• Math methods are Value::NativeFunction values using the static function-pointer path and explicit .length;
• direct Math.abs(...) calls still use the existing Op::MathCall compiler fast path, while method reads such as Math.abs.length and extracted calls use the installed namespace object.
• globalThis.JSON, globalThis.Atomics, and globalThis.console are installed by the centralized bootstrap registry from static namespace specs;
• console output is routed through an embedder-overridable ConsoleSink; the default sink writes with println! / eprintln!.
• Active Web API classes use runtime_class, runtime_constructor, runtime_method, and RuntimeObjectBuilder::from_host_data from otter_runtime. Embedders can enable the active Web globals with otter_web::WebApiBuilderExt::with_web_apis; the CLI enables that preset by default while keeping otter-runtime independent of otter-web.

Performance Rules

High-level API work must preserve the handwritten runtime shape:

• no per-call allocation for static builtins;
• no runtime parsing of metadata;
• no hot-path HashMap<String, Box<dyn Fn...>> registry;
• no hidden global mutation outside centralized bootstrap;
• no hidden permission or async scheduling logic in builders.

Changes that affect builtin installation or native call dispatch need before/after benchmark notes for startup and steady-state native calls.

Plugin System

The plugin system is future work. This page records direction and boundaries so new extension APIs do not block it. It is not a stable ABI promise.

Plugins should eventually be able to add hosted modules, native bindings, and host-owned object surfaces without depending on GC internals.

Design constraints:

• plugin APIs should be safe by default;
• raw collector internals are not part of the normal plugin API;
• long-lived JS references use persistent roots;
• plugin-owned buffers use external-memory accounting;
• async plugin work must re-enter the owning isolate before touching JS values;
• dynamically loaded plugins, if supported, need an explicit ABI and versioning story.

Layering Direction

The supported layers should arrive in this order:

1. in-workspace hosted modules using current native/context APIs;
2. native bindings compiled with the engine and installed through JS surface builders;
3. out-of-tree Rust plugin packages that depend on a stable extension crate;
4. optional dynamic ABI/FFI plugins, only after versioning, safety, and ownership rules are explicit.

The first two layers are source-level Rust APIs. They may change while the engine API is pre-stable. Dynamic plugins require a much stricter compatibility contract and are deferred.

Non-Negotiables

Plugin-facing APIs must not expose raw collector internals by default. Persistent JS-visible state uses roots, weak handles upgrade through a matching branded context, external memory is accounted through RAII, and async work re-enters the owning isolate before touching JS values.

Plugins may request capabilities, but the runtime decides whether those capabilities are granted. Permission checks must remain explicit and testable at the Rust boundary.

Plugin details stay design-only until the plugin API becomes concrete.

Web API Contribution Workflow

Web APIs live in crates/otter-web on the active runtime stack.

Current active slices:

• URL: parsing, relative resolution, and common URL parts.
• Headers: normalized ordered header list.
• Blob: owned byte payloads, size, type, slice, and text decoding.
• Request / Response: owned Fetch-shaped records.

Expose Web API constructors, prototypes, and globals through static ClassSpec / builder data. Keep installation centralized; do not mutate globalThis from unrelated modules and do not add a separate Web runtime stack.

Rules for new Web API work:

• store host state as owned Rust data, not VM contexts or handles;
• validate arguments on the isolate thread;
• enforce net, read, or other capabilities at the Rust boundary before starting host work;
• copy owned request/body data into async futures, then post completions back to the isolate;
• add focused Rust tests for host-side records and JS/module tests for JS-visible behavior.

Macros are appropriate when they generate the same static specs and builder calls a manual implementation would write. Keep manual code when capability checks, bootstrap order, async scheduling, or host-owned object lifetimes are the main behavior.

Startup Performance

Otter treats startup as a production requirement. Contributor-friendly APIs must not silently make RuntimeBuilder::build() or first script execution slower.

The active benchmark suite covers bootstrap install, runtime construction, first execution, static native dispatch through an extracted builtin, and CLI process cold start.

Local Workflow

Run:

cargo bench -p otter-vm --bench bootstrap -- --sample-size 30 --measurement-time 2 --warm-up-time 1
cargo bench -p otter-runtime --bench startup -- --sample-size 30 --measurement-time 2 --warm-up-time 1
cargo bench -p otter-cli --bench cold_start -- --sample-size 10 --measurement-time 2 --warm-up-time 1

Benchmark sources:

• crates/otter-vm/benches/bootstrap.rs;
• crates/otter-runtime/benches/startup.rs;
• crates/otter-cli/benches/cold_start.rs.

Bootstrap telemetry should be opt-in and default-off. Useful counters are:

• objects/functions/prototypes installed;
• strings interned;
• GC allocations and bytes;
• per-bootstrap-phase timing;
• duplicate-name and install-order validation cost.

When changing bootstrap, builders, macro generation, or builtin install order, include a before/after startup table in the PR or closeout notes.

The report should include:

case                              before        after         delta
RuntimeBuilder::build(default)    ...           ...           ...
RuntimeBuilder::build(prod)       ...           ...           ...
first run_script("undefined;")    ...           ...           ...
CLI cold empty JS                 ...           ...           ...
CLI cold tiny TS                  ...           ...           ...

Always note the build profile, machine, command, sample count, and whether the run used bootstrap telemetry.

The runtime startup bench uses a custom Criterion timing loop: each iteration measures only the build/first-run body, then immediately drops the runtime before the next iteration. This keeps the process-global GC cage reused without batching thousands of live heaps or timing teardown. The regression test cargo test -p otter-runtime repeated_otter_build_drop_returns_gc_pages_to_cage guards the same lifecycle at the public Otter handle boundary.

Current Budgets

The local 2026-05-06 ratchet values are:

• default global bootstrap: ~113 us;
• global bootstrap with telemetry: ~121 us;
• RuntimeBuilder::build() default / production-sandbox: ~121 us;
• Otter::builder().build(): ~422 us, including isolate runner startup;
• first run_script("undefined;"): ~126 us;
• first extracted static native Math.abs call: ~122 us;
• CLI cold -e "" / tiny JS / tiny TS: ~25-26 ms.

The CLI cold-start bench also includes bucket cases:

• info: process + clap + dispatch baseline with no runtime/compiler touch;
• dump_tiny_js_file: compile-only first-touch frontend/compiler baseline.

The compile-only path still avoids VM evaluation. otter check and otter --dump-bytecode construct a runtime session so package graph and module resolution match run, but they stop after compilation/linking and do not dispatch bytecode. Ambiguous .js / .ts file execution uses one OXC parse for module-syntax detection and script compilation; .mjs / .mts route directly to the module graph and .cjs / .cts route directly to script execution.

After the cage-reuse fix, cargo bench -p otter-runtime --bench startup -- --sample-size 10 --measurement-time 2 --warm-up-time 1 completed without an iteration cap. Its build-body-only smoke values were:

• RuntimeBuilder::build() default / production-sandbox: ~20 us;
• Otter::builder().build(): ~266 us, including isolate runner startup;
• first run_script("undefined;"): ~22 us;
• first extracted static native Math.abs call: ~26 us.

After the compile-only split and ambiguous-file parse-once routing, cargo bench -p otter-cli --bench cold_start -- --sample-size 10 --measurement-time 2 --warm-up-time 1 produced:

• info: ~3.14-3.18 ms;
• eval_empty: ~25.62-25.79 ms;
• tiny_js_file: ~25.24-25.59 ms;
• dump_tiny_js_file: ~3.19-3.26 ms;
• tiny_ts_file: ~25.85-26.61 ms.

Set OTTER_CLI_STARTUP_TIMINGS=1 on a single CLI invocation to print default-off phase timings to stderr. This is intended for cold-start triage, not for benchmark scoring.

After removing eager zeroing of the full 256 MiB GC cage at process startup, cargo bench -p otter-cli --bench cold_start -- --sample-size 10 --measurement-time 2 --warm-up-time 1 produced:

• info: ~3.17-3.21 ms;
• eval_empty: ~4.31-4.35 ms;
• tiny_js_file: ~4.36-4.51 ms;
• dump_tiny_js_file: ~3.25-3.39 ms;
• tiny_ts_file: ~4.45-4.53 ms.

Bootstrap telemetry budget:

• duplicate registry names: 0;
• bootstrap string interning: 0;
• namespace objects: 4;
• static native functions installed from specs: 57;
• GC allocation delta: <= 160;
• GC live-byte delta: <= 96 KiB.

Regression Policy

High-level contributor APIs must compile down to the same runtime shape as handwritten static specs. Startup regressions need an explicit production justification, a benchmark table, and a follow-up plan when accepted.

Lazy or tiered initialization must preserve spec-observable behavior. If a surface's property enumeration, identity, or initialization timing would change, do not lazy-install that surface without a spec note and tests.

macOS Deployment

Otter release automation can sign macOS artifacts with a hardened runtime. Any future JIT-enabled macOS ARM64 release must include the JIT entitlement.

Manual signing:

codesign --force --options runtime \
  --entitlements release/macos-jit-entitlements.plist \
  --sign "Developer ID Application: <Team Name>" \
  target/release/otter

Release helper:

CODESIGN_IDENTITY="Developer ID Application: <Team Name>" \
  scripts/sign-macos-release.sh target/release/otter

Secrets used by release workflows:

• APPLE_CERTIFICATE_P12_BASE64
• APPLE_CERTIFICATE_PASSWORD
• APPLE_CODESIGN_IDENTITY
• MACOS_KEYCHAIN_PASSWORD

The required entitlement is:

<key>com.apple.security.cs.allow-jit</key>
<true/>

Run a JIT smoke test after signing when JIT is enabled in the release build.

Macro Overview

Macros are contributor ergonomics over the static JS surface backend, not a separate runtime registration system.

Macros sit on top of static specs, mutator-bound builders, centralized bootstrap, and startup/first-run benchmark ratchets.

Macros must be zero-cost at runtime. Generated code should look like handwritten static specs:

static MATH_SPEC: NamespaceSpec = NamespaceSpec {
    name: "Math",
    methods: &[
        MethodSpec {
            name: "abs",
            length: 1,
            attrs: Attr::builtin_function(),
            call: NativeCall::Static(math_abs),
        },
    ],
};

The available macro surface is in otter-macros:

• #[js_namespace] generates namespace specs from inline Rust modules;
• #[js_class] generates constructor/prototype class specs;
• raft! generates grouped namespace specs without helper attributes.

#[js_namespace] example:

use otter_macros::js_namespace;

#[js_namespace(name = "Math", spec = MATH_SPEC)]
mod math {
    #[js_fn(name = "abs", length = 1)]
    pub fn abs(ctx: &mut NativeCtx<'_>, args: &[Value]) -> Result<Value, NativeError> {
        math_abs(ctx, args)
    }
}

The macro emits a public MATH_SPEC: NamespaceSpec and private static method metadata that can be passed to the JS surface builders or installed by the centralized bootstrap registry.

#[js_class] separates ordinary instance methods from JavaScript static methods:

use otter_macros::js_class;

#[js_class(name = "Point", spec = POINT_SPEC)]
mod point {
    #[js_constructor(length = 1)]
    pub fn construct(ctx: &mut NativeCtx<'_>, args: &[Value]) -> Result<Value, NativeError> {
        create_point(ctx, args)
    }

    #[js_method(name = "valueOf", length = 0)]
    pub fn value_of(ctx: &mut NativeCtx<'_>, args: &[Value]) -> Result<Value, NativeError> {
        point_value_of(ctx, args)
    }

    #[js_static_method(name = "from", length = 1)]
    pub fn from(ctx: &mut NativeCtx<'_>, args: &[Value]) -> Result<Value, NativeError> {
        point_from(ctx, args)
    }

    #[js_getter(name = "x")]
    pub fn get_x(ctx: &mut NativeCtx<'_>, args: &[Value]) -> Result<Value, NativeError> {
        point_get_x(ctx, args)
    }
}

Here #[js_method] installs Point.prototype.valueOf, while #[js_static_method] installs Point.from. Both still use the NativeCall::Static Rust function-pointer path.

raft! is for grouped namespace declarations:

otter_macros::raft! {
    pub static MATH_SPEC: namespace("Math") {
        methods: [
            "abs" => math_abs, length = 1;
        ]
    }
}

Remaining planned macro scope:

• macro coverage for constants/data properties where the generated shape is still inspectable;
• broader migration only after benchmark ratchets are in place.

Deferred until their backend APIs are stable:

• #[dive] or equivalent async native binding sugar;
• host-owned object surface macros;
• hosted-module loader macros;
• GC trace derive macros.

Macros must not hide capability enforcement, bootstrap order, async host-op scheduling, or important control flow. Prefer manual code when those concerns are the main behavior.

Generated Shape Contract

Macro expansion must produce:

• static spec records;
• ordinary Rust functions with explicit exported JS names and arity;
• static native call targets for builtins by default;
• builder/bootstrap calls through the centralized registry.

Macro expansion must not produce:

• runtime registries or metadata parsing;
• per-call allocation;
• hidden global mutation;
• hidden permission checks;
• hidden async scheduling;
• captures of RuntimeCx, NativeCtx, Value, Frame, Gc<T>, or Local<'gc, T> across .await.

Example Source And Expansion

Source:

#[js_namespace(name = "Math", spec = MATH_SPEC)]
mod math {
    #[js_fn(name = "abs", length = 1)]
    pub fn abs(ctx: &mut NativeCtx<'_>, args: &[Value]) -> Result<Value, NativeError> {
        math_abs(ctx, args)
    }
}

Required generated shape:

static MATH_SPEC: NamespaceSpec = NamespaceSpec {
    name: "Math",
    methods: &[MethodSpec {
        name: "abs",
        length: 1,
        attrs: Attr::builtin_function(),
        call: NativeCall::Static(math::abs),
    }],
    accessors: &[],
    constants: &[],
    attrs: Attr::global_binding(),
};

Keep manual specs for capability gates, delicate bootstrap order, async scheduling, host-owned object lifetimes, or API shapes not covered by the current macros.

Prefer Manual Code When

• capability enforcement is the main behavior;
• bootstrap or install order needs careful review;
• async scheduling or cancellation behavior is non-trivial;
• the macro would hide object graph, rooting, or external-memory lifetime decisions.