Skip to content

Draft: Speculoos pilot - #331

Closed
mnemitz wants to merge 39 commits into
mainfrom
speculoos-pilot
Closed

mnemitz wants to merge 39 commits into
mainfrom
speculoos-pilot

Conversation

@mnemitz

@mnemitz mnemitz commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

mnemitz and others added 24 commits October 27, 2026 10:12
One channel, realtime, carrying all 24 messages, with two operations from the
server's point of view: receiveClientMessages and sendServerMessages. The
server is split into host and pathname. Component names are unchanged, so the
guide pages that pull individual schemas are unaffected.

The live reference page now reads its message lists from the operations
rather than the old channels.publish/subscribe, and renders as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/api-ref/speculoos-realtime renders spec/realtime.yaml through Speculoos and
its Radix Themes map, with the site's own Markdown and a messageExtras
callout matching AsyncAPIMessage's preview and deployment notes.

- build:speculoos-ir writes src/generated/realtime.ir.json (gitignored) and
  runs with start and build.
- Speculoos comes from local tarballs (../speculoos/.packs). After
  `pnpm pack:local` there, run `npm run speculoos:refresh` and restart: npm
  and webpack both cache by an unchanged version otherwise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- reset.css unsets the border and vertical-align Infima leaks into every
  Radix Code, site-wide, replacing CodeInline's inline-style workaround; and
  trims the last paragraph's margin in a DataList value.
- The pilot page is in the API Reference sidebar, temporarily.
- Types are expanded where they are used (expandRefs: "always"), collapsed
  behind the map's toggle, so type names are not links.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Spec (also read by the live page):
- Six messages carry a high-level example as AsyncAPI message examples —
  StartRecognition, RecognitionStarted, AddTranscript, EndOfStream,
  EndOfTranscript, Error. The two that existed moved off their payload
  schemas; AsyncAPIMessage now reads message examples first.
- Info, Warning and Error types are a oneOf of documented consts instead of
  an enum plus a description table that repeated it. Each has a title,
  which the live page's renderer uses to label its tabs.
- AddAudio's payload is an inline, described binary string.

Pilot page:
- Examples sit in a sticky panel beside the reference (no TOC), in the
  order a session runs rather than spec order.
- CSS: codes in description tables stay whole.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- components.securitySchemes: apiKey (Authorization: Bearer) and
  temporaryKey (the jwt query parameter, for browsers), on the server as
  alternatives.
- The channel's ws binding says the upgrade is a GET.

The pilot page renders both as its Handshake block; the handshake failure
responses and retry advice stay prose, since AsyncAPI has no place for them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The spec's server is global.rt.speechmatics.com; its description says how
  to pin a region instead, and the pilot's Handshake card shows it. The
  temporary-key example uses the global host too.
- _realtime-handshake-responses.mdx, passed as handshakeExtras: the upgrade's
  responses as tabs (101 with the example exchange, 400, 401, 405), then the
  client retry advice, which moves to the Error message once close codes
  have a home there. Partials need a description too (docusaurus.config.ts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SessionPanel replaces the examples panel beside a channel: the session as
  a compact sequence diagram — client messages left, server right, the
  audio/transcript loop bracketed, Info/Warning/Error "at any time" — each
  linking to its card and opening to its example. It stands in for the
  protocol overview's Mermaid diagram. The order is the site's; the spec
  cannot express it. Sticky beside the reference from Radix's lg
  breakpoint, scrolling on its own when taller than the window.
- _realtime-connection-close.mdx, after the messages: the close code table
  and the retry advice (moved out of the handshake card). Close codes are
  their own part of the protocol: they do not map one-to-one onto Error
  types, and six types have no documented code yet.
- Two paragraphs above the reference in place of the protocol overview.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both directions are cyan, matching the reference's direction badges; the
arrow says which way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- /api-ref/speculoos-batch-create-job renders POST /jobs from
  static/batch.yaml, converted from Swagger 2.0 to OpenAPI 3 at build time
  (swagger2openapi --anchors) since Speculoos reads OpenAPI 3 only. The
  source spec and live pages are untouched. build:speculoos-ir now runs
  after build:jobs-api-ref, which writes static/batch.yaml, not beside it.
- SpeculoosReference takes `entry` (an entry slug) for a page per entry.
  Beside a REST operation, OperationPanel: the spec's x-codeSamples as tabs,
  then its examples — request, success, then errors — without the
  converter's meaningless "response" names.
- Both pilots hide their front-matter title: the entry's heading and URL is
  the page's h1. The realtime introduction moves into the channel's
  description in the spec, where it renders under that heading.
- spec/batch.yaml: a 201 example for POST /jobs (the live page gets it too).
- Badge colours follow the sidebar's (src/css/badge.css): GET cyan, POST
  green, PUT yellow, DELETE red, WSS and both directions teal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spec/batch.yaml is generated upstream and stays Swagger 2.0 for now, which
cannot say what the Batch pilot needs. scripts/generate-batch-openapi.ts
converts static/batch.yaml with swagger2openapi and applies
spec/batch.overlay.yaml — a standard OpenAPI Overlay 1.0 document — to make
src/generated/batch.openapi.yaml, which Speculoos reads:

- openapi: 3.1.0
- POST /jobs's multipart `config` part is a JobConfig sent as JSON, so the
  page expands JobConfig rather than calling it a string.
- bearerAuth is HTTP Bearer: "Authorization: Bearer <token>".
- CreateJobResponse.status and JobDetails.status are oneOfs of documented
  consts instead of bullets in a description.

The overlay is applied by the script, not a library: the format is a few
lines over jsonpath-plus, and an action that matches nothing fails the build,
so a field renamed upstream cannot silently drop its patch. Actions remove
only what changes, so properties keep their upstream order and prose.
jsonpath-plus and swagger2openapi become explicit devDependencies (both were
already installed through the docs plugin).

The live batch pages still read static/batch.yaml and are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/generate-speculoos-pages.ts writes a stub page per entry in each
IR to docs/api-ref/speculoos/<api>/ (gitignored), named from the title as
the live pages are. Front matter carries the sidebar label, badge and
position, so the sidebar lists each API's directory with autogenerated.
Per-entry config renames a page and passes partials by prop: realtime
gets its handshake responses and connection close.

It runs at the end of build:speculoos-ir and replaces the two hand-made
pilot pages. The batch overlay drops /jobs/{jobid}/log, which the live
reference leaves out too, and notes that success examples are missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/generate-batch-openapi.ts becomes generate-openapi.ts, taking a
name and a Swagger 2.0 source, so Management goes the same way as Batch:
spec/mp.yaml is converted and spec/mp.overlay.yaml adds the server and
the management-token bearer scheme the spec leaves out. The page
generator writes its eight pages; the sidebar gets a Management API
(pilot) category beside Transcription API (pilot).

An operation with no code samples or examples has no panel, and its
reference takes the width.

spec/UPSTREAM-CHANGES.md collates what should change in the Management
and Batch specs at their source, so the overlays can shrink.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UPSTREAM-CHANGES.md records the verdicts on the Management spec and the
plan they led to: the public spec is generated in the upstream repo, by
an overlay for what the auth proxy changes, then dropping unused
components and checking paths against a public allowlist. Content fixes
go into the service spec itself. Until the public spec is published,
the pilot keeps building from spec/mp.yaml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/api_keys still works but is legacy; /api-keys is the documented path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
src/theme/Markdown imported react-markdown without declaring it; it
came only with the OpenAPI docs plugin. Declare it, with the rehype-raw
it is built for (6.1.1, unified 10): the hoisted one was Docusaurus's
7.0.0, the mismatch the @ts-ignore covered, now gone.

react-markdown 8 passes components props of its own (node, inline,
ordered, depth, index, checked), and the MDX components spread them
onto the DOM: React warned on every description with code or a list,
and links, lists and code carried node="[object Object]". Each
component is now wrapped to drop them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spec/management-api-v3.yaml is the public Management API spec, generated
upstream from the service spec and an overlay for what the auth proxy
changes. It is OpenAPI 3 with the server and both bearer schemes, so the
pilot builds from it directly and mp.overlay.yaml goes. The live
reference still reads spec/mp.yaml.

Its content is behind spec/mp.yaml's (descriptions on POST /api-keys,
summaries); UPSTREAM-CHANGES.md now tracks porting that upstream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
batch.overlay.yaml gives every operation a success example, all about
one job, including the 201 on POST /jobs that the pilot had added to
spec/batch.yaml (which now matches upstream again). It also fixes what
validating them turned up: SentimentSegment.confidence's type typo, the
plain-text transcript's schema, and a /usage example with alignment jobs,
which no longer exist. UPSTREAM-CHANGES.md records these, and the stale
schema-level examples upstream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spec/code-samples/batch-api has Python (speechmatics-batch), JavaScript
(@speechmatics/batch-client) and curl samples for every operation, curl
only for /usage, which no SDK wraps. The old samples are rewritten: the
JavaScript ones no longer matched the client.

npm run test:code-samples runs each against a mock built from the spec:
it answers with the operation's success example and checks the method,
path, bearer header, declared query parameters and request body, and
the examples against their schemas. Samples keep their real URLs; the
runner redirects them. Every operation needs a sample.

.github/workflows/code-samples.yml runs it on changes to the spec,
samples or scripts. generate-batch-spec.ts picks up put samples too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The regenerated public spec has the content spec/mp.yaml had and the
service spec lacked: the POST /api-keys description and its type and
body property descriptions, region marked deprecated, and summaries that
match the live pages ("Create a project" for "Post a new project").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spec/code-samples/mp-api has a curl sample per Management operation.
generate-mp-openapi.ts adds them to the public spec for the pilot, and
the live reference picks them up as it does Batch's (now with put).

scripts/code-samples/inject.ts is the one implementation of adding
samples as x-codeSamples, used by both live generators and the pilot:
files sorted, so tabs come in a stable order. The live specs it builds
are unchanged.

The test mock moves to scripts/code-samples/mock.ts and runs per API.
Management samples get the same checks, but a missing response example
is allowed until upstream adds them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The generated pages take the old URLs: docs/api-ref/batch and
docs/api-ref/management as before, and the realtime page at
/api-ref/realtime-transcription-websocket by slug. The sidebar's pilot
categories become Transcription API and Management API.

Removed: the OpenAPI plugin's page generation (its config, scripts and
template), the hand-written realtime page, spec/mp.yaml and
generate-mp-spec.ts (the public spec replaces them), the sidebar
generator's Batch filter, and npm-run-all. The OpenAPI theme stays for
the Agent STT page.

Links to realtime anchors follow the new kebab-case ids; links to
sections that no longer exist point at the page. Redirects cover the
two info pages, the log page and post-a-new-project, now
create-a-project. Speculoos' anchors are registered with Docusaurus's
link checker, which does not see them otherwise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
generate-agent-stt-spec.ts now writes the shape the realtime spec has:
one channel, its description and title from agent-stt.overlay.yaml,
with receive and send operations from the server's point of view, and
a server with host and path apart, its description and security (API
key or temporary key). The overlay's extraMessages are keyed by who
sends them. Regenerated from unchanged upstream sources.

The page generator writes the Agent STT page at
/api-ref/agent-stt-websocket, with partials for the handshake responses
and close codes; the hand-written page goes. The session panel's
sequence is per channel, with Agent STT's turns and segments. Links to
the old page's sections point at the page.

The Markdown wrapper drops react-markdown's isHeader too: the channel
description is the first with a table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
<Schema> renders a type from the same IR and component map as the API
reference: a named type, one property of it (property), or the type
under a key (as), the last two open. An unknown name fails the build.
The nine guide pages that embedded the theme's SchemaNode use it, so
their tables now show the converted, corrected specs rather than raw
Swagger 2.0. The map and options shared with the reference move to
src/components/Speculoos/site.tsx.

With nothing left on the old renderer, the OpenAPI plugin, theme and
both schema loaders go, with the components and CSS that served them:
Schema, SchemaItem, ApiExplorer, AsyncAPIMessage, WebsocketMessageArrow,
the old WebSocket page generator, types.d.ts and the Infima badge rules.
Markdown was a swizzle of the theme's and moves to src/components;
tsconfig takes Docusaurus's own types.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The site depends on the published packages, so it builds anywhere,
preview deployments included. Only what it imports or runs is listed:
core, react and the Radix Themes map, and the CLI, which brings the
loader and the adapters with it.

npm run speculoos:local installs the packs from a ../speculoos checkout
without saving, to try unreleased changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Nov 2, 2026 3:23pm UTC

Request Review

Docusaurus fails a build when CI is set if a partial has front matter,
and the site's description check made every partial carry some. The
check now skips partials (an _ file or folder), which are not pages,
and the four API reference partials drop their front matter.

generate-openapi.ts and generate-mp-openapi.ts create src/generated,
which is ignored by git, so the code samples run on a fresh checkout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread scripts/code-samples/mock.ts Resolved
Comment thread .github/workflows/code-samples.yml Resolved
- The session panel shows a message's example in a Radix hover card
  rather than a toggle. Keyboard and touch readers lose nothing: the
  name links to the message's card, examples included.
- <Schema> tables sit on a Card, as the reference's messages do, so
  they stand apart from the prose in guide pages.
- The code samples workflow asks for a read-only token. It uses no
  secrets, and fork pull requests get a read-only token regardless.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A link to a message, from the session panel or from prose, scrolled its
section's title under the sticky navbar. Speculoos sections get the
scroll margin Docusaurus gives its own headings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From a browser review of the pilot: page width, the session panel on
small screens, the side panel's examples, <Schema> in guides, and
info.description. The reference's own styling is planned in the
Speculoos repo (M7). Working notes: remove before merging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mnemitz and others added 7 commits October 29, 2026 11:58
remark-admonition's buffers are typed, LinkCard's `as` takes the href a
disabled card does not have, and the controlled Tabs root drops a
defaultValue that Docusaurus allows to be null and Radix ignores anyway.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The unified 11 line Docusaurus's MDX already uses, so the old tree goes
from the lockfile. v10 passes custom components only `node`, so that is
all the wrapper strips. Rendered output of admonitions, GFM lists and
tables, raw HTML and code is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The body's Box is a flex child with no width, so it shrank to its
content: most Management pages, and /api-ref, measured 808–945 pixels and
the layout jumped from page to page. Every docs page now takes the full
1000 pixels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under an Examples heading, each labelled by its status's description
("400 Bad request") rather than "Response", so the panel no longer reads
as a repeat of the reference's Responses. A plain-text example shows as
text, not a JSON string. Batch's 201 "OK" is noted for upstream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Below lg it stacked after the whole reference, 15,000 pixels down on a
phone. It now opens the page as a closed card that expands to the
diagram; from lg it stays beside the reference as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each table is headed by its type's name and a link to where the reference
uses it: the message that carries it, or the endpoint nearest it, sent
before returned. The page generator writes src/generated/pages.json so the
component knows each entry's URL. The heading sits above the rows, so the
first row's rule separates them from it rather than doubling the card's
edge.

`open` opens one field and what is in it among the rest, collapsed: the
translation guide shows the whole transcript with `translations` open. An
unknown path fails the build, as an unknown type does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spec's overview of an API showed nowhere. Batch and Management, the
two specs with one, now get an index page at /api-ref/batch/ and
/api-ref/management/: the description, then each endpoint, linked. Batch's
is labelled "Batch API" in the sidebar, since it shares a category with
the realtime APIs. Its upstream wording ("ASR REST API") is noted for
upstream.

The page-URL map moves to site.tsx, shared by <Schema> and the overview.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mnemitz and others added 3 commits November 2, 2026 15:17
/api-ref/batch/ and /api-ref/management/ are not wanted as pages, so the
generator no longer writes an index.mdx per API, and SpeculoosOverview
goes. info.description is back to showing nowhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mnemitz mnemitz closed this Nov 2, 2026
mnemitz added a commit that referenced this pull request Nov 2, 2026
**Stack 1 of 4**, splitting #331 (the Speculoos pilot) for review. Each
PR builds on the one before it:

1. **Site fixes** ← this PR
2. Management reference from the public spec
3. Code samples, tested against a mock
4. API reference on Speculoos

This one is independent of the rest and can merge first.

## Changes
- **The page body fills its width.** Its `Box` was a flex child with no
width, so it shrank to its content: most Management pages and `/api-ref`
measured 808–945px, and the layout jumped from page to page. Every docs
page is now 1000px.
- **Type errors** in `LinkCard`, `Tabs` and `remark-admonition`. The
controlled `Tabs` root drops a `defaultValue` that Docusaurus allows to
be `null` and Radix ignores anyway.
- **Radix `Code`:** Infima's leftover `border` and `vertical-align` are
unset once in `reset.css`, rather than inline in `CodeInline`. Inline
code and admonitions use the `outline` variant.
- **Links:** the temporary key examples use `/v1/api-keys`, the
Management API's path. A guide's link to the Realtime reference drops
its `.mdx` suffix.

`main`'s other `tsc` errors are in the OpenAPI theme and the WebSocket
page generator, which PR 4 removes.

## Checked
- `npm run build` passes, with no broken links.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — fe19f531 Deployed Nov 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant