- Zig 78.6%
- TypeScript 17%
- Rust 2.6%
- JavaScript 0.5%
- Shell 0.5%
- Other 0.8%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
SJON is a schema-constrained data language for domain tools and the agents that operate them: deterministic S-expression data plus a pure, bounded expression layer, one front-end, a stable binary wire format, append-only diagnostic codes, and parity hosts in Zig, Node, Rust, and TypeScript.
## 1.5.0 — 2026-10-25
This release keeps binary wire v5 and the diagnostic-code enum unchanged.
The npm packaging fixes described under 1.4.2 first reach registry consumers
with 1.5.0: neither 1.4.2 nor 1.4.3 was published to npm (the previous registry
latest was 1.4.1). Packages ship compiled JavaScript and declarations, including
all documented subpaths; use pnpm to rewrite workspace dependencies when packing.
- **Decode once and query many:** `sjon.prepareBinary`, owning
`PatternQuery.PreparedBinary`, and the four `queryPreparedBinary*` entries
reuse a decoded tree, with fresh per-query environments and tree-path budgets.
Preparation is outside those budgets. `sjon.BinaryDecoder` exposes the
read-only decoder separately. See [the consumer guide](docs/BINARY_PATTERNS.md).
- **Evaluate a captured binary expression view:** `Expr.evalBinaryView` uses a
borrowed cursor and view without whole-buffer root framing; consumers must
honor cursor/input lifetimes. Existing whole-buffer evaluation is unchanged.
- **Carry author subtrees through Zig lowering:** `LoweringInput.subtree` and
`EmittedValue.carry` preserve author syntax and spans while the final forest
validates the destination context. This remains a Zig-only hook surface.
- **Rust distribution:** `sjon-host` is prepared for crates.io with explicit
package contents and a corpus-independent consumer build. It requires
Rust 1.92 and a matching external `sjon.wasm`.
- **Highlight runtime requirement:** the compiled grammar uses JSON import
attributes, so its minimum Node version is corrected to 20.10.
- **Documentation and distribution:** corrected Node/Rust onboarding, eight-op
editing and preserve-layout contracts, explicit host capability boundaries,
current coverage counts, and stronger documentation drift checks.
- **The binary pattern query can now evaluate `(pure <form>)` expression
leaves through new additive entries — the frozen legacy entries are
unchanged.** `PatternQuery.queryBinaryWithEnv` and
`queryBinaryWithBudgetAndEnv` compile a binary pattern with expression
leaves captured as IR views, dry-run each at cycle 0, and evaluate them
per cycle under the pattern-local `cycle` / `tick` / `seed` (which shadow
same-named outer bindings), with the caller's `Expr.Env` chained beneath
— where the legacy `queryBinary` / `queryBinaryWithBudget` silently
degrade every expression leaf to `silence` and keep their byte-for-byte
contract, including in the released read-only `sjon-binary.wasm` (no
artifact, host-ABI, wire, or diagnostic-code change; the wasm closure
allowlist is untouched). The entry, never a null env, selects semantics.
When a leaf exists, the pattern bytes are copied once into the Result
arena (one retained cursor, the compile-time pool index reused over the
layout-identical copy), so a Result outlives the caller's bytes and env;
the retention and pool-index steps preflight checked arithmetic against
`budget.bytes` and refuse with `MemoryBudgetExceeded` before allocating.
Malformed wire anywhere — including under a captured leaf, surfaced
through the cycle-0 dry-run — is a hard, typed `BinaryError` from the
existing error enum; static eval defects are the tree path's diagnostics
at `[pure]`; later-cycle eval failures are counted drops. No new
diagnostic codes, wire still v5, corpus still 393 cases.
A form value in a slot whose value-kind reference cannot resolve is reported
as the reference it is, and the composition that made it unresolvable is
named at preload: the aggregate phase resolves every kind reference every
manifest spells, so a bare name two loaded plugins both declare is a
schema-build diagnostic in every composing host, before any document. Nothing
moved on the stable surfaces: no wire-format change (still v5), no new
diagnostic codes (`ambiguous_element_kind` and `unknown_element_kind` both
exist and are corpus-covered), and the conformance corpus still at 393 cases.
- **The streaming binary cursor consumes the tree-trailing comment block on
root exhaustion, and corrupt flag-set tails are now refused instead of
silently accepted.** The wire appends a `[tree trailing comments]` block
after the roots whenever flag bit 5 is set — the encoder writes the count
varint even for an empty block — but `BinaryCursor.RootIter.next()`
returned null at the last root without touching it, so an exact-end
consumer landed one to eleven bytes short with no way to reach the block.
`RootIter` now mirrors `ChildIter`/`VectorIter`: its null-producing call
consumes the count and its entries through the same reader the mid-tree
comment sites use and reports the count in a new `trailing_count` field.
Error mapping uses existing codes only (`NodeCountExceeded` over-cap or
over-long varint, `InvalidTag` bad kind byte, `Truncated` short read);
entry text indices stay skipped, not pool-validated, matching the
cursor's count-only stance at all four comment sites. This is a
deliberate strictness increase on a host-facing surface: the three
draining `Validator` walkers — and the released `sjon_validate_binary`
wasm export — now surface those errors on a malformed tree-trailing tail
where the walk previously succeeded silently, symmetric with the
mid-tree trailing reads and `init`'s slack rejection. Non-draining
callers (`evalBinary`, `queryPattern`) are unchanged; wire stays 0x05
and no diagnostic codes moved.
### Fixed
- **A form child in a slot whose kind reference is ambiguous or unknown reports
the reference, never a classified mismatch.** When a form lands in a typed
slot whose `.named` reference does not resolve `.found` — two loaded plugins
declare the bare name, or nobody declares it — both walkers used to fall into
expression classification and emit `wrong_underlying` ("form `x` positional
argument expects `y`, got form"), silently losing the head-set narrowing and
burying the honest code. The form fast-path in `matchValueAgainstType`
(`src/Validator.zig`) now falls through to the `.named` arm, and its binary
twin `matchFormAgainstTypeBinary` gained the matching guard between its
head-set step and its expression step, so both paths emit
`ambiguous_element_kind` (naming the claimants and the qualified spelling)
or `unknown_element_kind` at the child's span. A narrowing that cannot
resolve is not a value that failed. A document loading one plugin is
untouched — one claimant, the bare reference resolves, the narrowing runs as
before — and a reference that resolves to a non-form kind keeps today's
classified comparison exactly.
- **The aggregate phase resolves every manifest value-kind reference, so a
bare-referenced kind name two plugins share is named at preload.**
`Schema.validateKindRefs` joins the aggregate validators as the sixth,
appended after cross-refs, unions, forms, lowering and defaults so the
corpus-locked diagnostic-stream order of the other five does not move. It
walks every plugin's forms (each key's `:type`, a form's `:positional` kind,
recursing into slot-local `local_forms` on both carriers and into
per-`:when` variant keys) and every value-kind's vector-shape `:element`,
skips the primitive type shortcuts, and resolves each reference against the
aggregate: `unknown_element_kind` for a miss, `ambiguous_element_kind` for a
cross-plugin collision, span `{0,0}`, path `[<plugin>, <form>, <slot>]`,
stamped `.aggregate`. Declaring the colliding name confers no priority, so a
bare reference inside the declarant's own manifest is named too, which makes
the ambiguity the composition's property, reported when the schema is built
rather than one document's surprise. The two corpus fixtures of that class
(`ambiguous-element-kind`, `unknown-element-kind`) were reviewed and moved to
expect the preload diagnostic ahead of the document's own: their bytes pinned
one phase of a two-phase class and were written before any preload walk
existed, and the diff is the prepended diagnostic and nothing else, with the
corpus still at 393 cases. The TypeScript parity host ports the walk in the
same change (`hosts/typescript-parity/src/plugin.ts`,
`validateKindRefs`), the corpus being the cross-host axis; `hosts/web` and
`hosts/rust` drive the Zig artifact and inherit the walk.
|
||
| conformance | ||
| docs | ||
| editors/vscode | ||
| examples | ||
| fixtures | ||
| hosts | ||
| landing-page | ||
| manifests | ||
| scripts | ||
| src | ||
| tools | ||
| .editorconfig | ||
| .gitignore | ||
| biome.json | ||
| build.zig | ||
| build.zig.zon | ||
| CHANGELOG.md | ||
| coverage-baseline.txt | ||
| LICENSE | ||
| llms.txt | ||
| package.json | ||
| pnpm-lock.yaml | ||
| pnpm-workspace.yaml | ||
| README.md | ||
| SJON.svg | ||
SJON
SJON is a schema-constrained data language for domain tools and the agents that operate them.
Domain tools grow config languages; agents write a growing share of the config. SJON gives both a shared substrate: an S-expression AST with the schema as data, validated identically in Zig, JS, and Rust.
Two layers, parsed by one front-end: deterministic S-expression data, plus a pure, bounded, closed-vocabulary expression layer evaluated by the consumer.
(scene :bpm 130
(canvas :name "main"
(camera :ortho :zoom (* 2 (b 1)))
(stack :mode :mask
(shape :sdf :radius 0.5
:delay (delay :p+s (b 4))
:lifespan (b 16))
(shape :path :points [[0 0] [1 0] [1 1]]))))
Guarantees
Every claim below is machine-enforced, not asserted — each links the gate that checks it.
- Totality — a tree always exists after parse; collection over abort. No parse can crash the caller.
- Determinism — bit-identical eval native vs. WASM (vendored
exp64/log64/pow64), checked by byte-diffed goldens. - Boundedness — no host-stack recursion anywhere in the Parser,
Validator, or Expr evaluator; depth, step, and memory-budget
ceilings, each with a test that trips it (
oom_tests.zig). - Wire stability — Binary IR is versioned and diagnostic codes are append-only, both gated by the conformance corpus. Full layout in docs/DESIGN.md.
- Diagnostics as product — every error carries
(code, severity, span, path), a 130-entry explanations catalogue, and LSP quick-fixes. See it below.
| Field | Size | Value |
|---|---|---|
version |
1 B | 0x05 |
393 fixtures in the shared corpus; each host replays its supported subset
(see the capability matrix below). 25 never-panic fuzz harnesses. 130 explained
diagnostic codes. Zero dependencies in the core library. Every number
here is drift-gated by zig build audit-docs; run zig build verify
to check all of it yourself.
See it catch a mistake
$ printf '(scene :w 800)\n' | sjon validate -
<stdin>:1:2: error: unknown_form: unknown form `scene`
at scene (phase: validation)
1 error
$ sjon explain unknown_form
unknown_form
A form's head name is not declared by any loaded plugin.
...
https://hugodaniel.com/pages/sjon/errors/unknown_form
Every diagnostic is a real error page, not a string — same code in
the CLI, the LSP's Problems panel, and --format=github CI
annotations. See docs/TOOLING.md for the full CLI
and LSP capability list.
Consuming
Every host shares the diagnostic vocabulary, the binary wire format, and the conformance corpus. Supported surfaces agree on the replayed fixtures; intentional boundaries and skip registries are documented in the capability matrix and quickstarts: docs/INTEGRATION.md.
Zig — the reference path, zero runtime dependencies:
// build.zig.zon
.dependencies = .{
.sjon = .{ .path = "../sjon" },
},
// build.zig
const sjon_dep = b.dependency("sjon", .{ .target = target, .optimize = optimize });
exe.root_module.addImport("sjon", sjon_dep.module("sjon"));
const sjon = @import("sjon");
var tree = try sjon.parse(gpa, source);
defer tree.deinit();
const text = try sjon.print(gpa, tree, .{ .mode = .canonical });
defer text.deinit();
Node / Web (WASM) — install npm install @sjon-lang/web@1.5.0.
Build matching artifacts from the 1.5.0 source with Zig 0.16 (zig build wasm-all); the npm package does not bundle WASM. From the source root:
import { SjonEncoder, SjonReader } from "@sjon-lang/web";
const encoder = await SjonEncoder.load("./zig-out/bin/sjon.wasm");
const reader = await SjonReader.load("./zig-out/bin/sjon-binary.wasm");
const binary = encoder.toBinary('(scene :bpm 130)');
const result = reader.validateBinary(binary);
See examples/quickstart-web.mjs and
hosts/web/README.md.
Rust — cargo add sjon-host@1.5.0; wraps the matching external
sjon.wasm via wasmtime (the crate does not bundle it):
use std::path::Path;
use sjon_host::{HostOptions, SjonHost};
let mut host = SjonHost::load(Path::new("zig-out/bin/sjon.wasm"), None)?;
let result = host.validate_document(source, &HostOptions::default())?;
See examples/quickstart-rust.rs and
hosts/rust/README.md.
TypeScript — hosts/typescript-parity/ is a hand-ported reference
for conformance verification, not for production; @sjon-lang/schema
(hosts/schema/) is the typed schema builder for TS-first projects.
See hosts/schema/README.md.
Docs
- docs/AUTHORING.md — hands-on tutorial
- docs/LANGUAGE.md — formal grammar, semantics, JSON bridge
- docs/DESIGN.md — module map, SoA tree, binary wire format
- docs/INTEGRATION.md — embedding from Zig, Node/Web, Rust, TypeScript
- docs/BINARY_PATTERNS.md — decode once and query binary patterns
- docs/TOOLING.md — editor setup, LSP capabilities,
sjonCLI - docs/plugin-model-v1.md — the plugin contract
- examples/ — runnable fixtures and demos, one per host
Building
zig build test # all test suites (all in-source)
zig build verify # every local gate: tests, fuzz, audits, all hosts, biome, clippy
zig build wasm-all # both WASM artifacts
zig build lsp # native sjon-lsp
Requires Zig 0.16.x.
Versioning & releases
The stable surfaces are the binary wire format and the diagnostic-code
enum — versioned, and gated by the conformance corpus plus zig build verify. A change to either is deliberate and version-gated: the wire
bumps its format integer, diagnostic codes only append, and the corpus
lands updated in the same change.
Every package in this repo carries the one version declared in
src/version.zig; zig build audit-format-versions names any
manifest that falls out of step. The @sjon-lang/* packages carry full
registry metadata and are published through scripts/publish-npm.sh.
Use pnpm pack for local tarballs: pnpm rewrites the web package's
workspace:* dependency into the matching registry version. Workspace
and path dependencies remain supported. What each release changed
is in CHANGELOG.md. The repeatable candidate and publication
checks are in docs/RELEASING.md.
License
CC0 — public domain. See LICENSE for the legal text.