Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/actions/flybot-tests/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,9 @@ runs:
shell: bash
working-directory: core
run: node scripts/build-client.mjs --check
- name: Offline harness lint
shell: bash
run: npm run lint:interop
# Core's suite (incl. bot-tree: the public flybot.sidebar.structure slot), then each peer
# plugin's own suite and lint (plugins/social, plugins/tree; each also tests against core/).
- name: Unit tests
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,8 @@ jobs:
run: npx --yes -p typescript@5.9.3 tsc --noEmit --strict --skipLibCheck types/*.d.ts
- name: Tests of the repo scripts (risk tier, labels)
run: node --test ".github/scripts/*.test.mjs"
- name: Offline cross-runtime harness (no network or model)
run: npm run test:interop
- name: Changelog has an Unreleased section and well-formed fragments (changelog.d/)
run: node .github/scripts/changelog.mjs check
- name: Lint the workflows (actionlint)
Expand Down
121 changes: 121 additions & 0 deletions dev/interop/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Offline cross-runtime pilot

Implementation slice 1 of [#198](https://github.com/ThinkFlowLab/FlyBot/issues/198),
tracked in [#200](https://github.com/ThinkFlowLab/FlyBot/issues/200), following the
design in [#199](https://github.com/ThinkFlowLab/FlyBot/pull/199). This is a
development-only, deterministic contract simulator. It ships in none of the
FlyBot plugin packages and adds no runtime, task endpoint or permission grant.

## Run

Node 22.19+; no install or model key is needed for the demo and tests:

```sh
npm run demo:interop
npm run test:interop
```

The demo produces a JSON evidence record: `offline-mock`, one session, two
submitted turns, `result-ready` before explicit simulated requester acceptance,
then `accepted`. Real model requests/cost are zero because there is no model
implementation. The synthetic execution's `usage` fields remain null (unknown),
never guessed from those demo-level zeroes.

The demo scripts both agents' responses and the owner's decisions; nobody has
performed real analysis or human acceptance. Tests cover each boundary with
synthetic identities and fake time. Neither command contacts Bot Social or
OpenCode, reads the user's DSH data, uses a real credential or starts a process.

`no-network.mjs` makes accidental fetch/socket/DNS/subprocess calls throw during
the run. This is a regression tripwire for this harness, not a security sandbox
for arbitrary third-party code. Do not load external plugins or real credentials.

For lint/full verification, install the normal core tooling as described in
[AGENTS.md](../../AGENTS.md), or select an existing install with `DSH_HARNESS`:

```sh
npm run lint:interop
npm run verify
```

The required CI `package` job runs the offline suite. The core test composite
runs its lint. Root `verify` runs both after the existing suites.

## Components

| File | Responsibility |
| --- | --- |
| `codec.mjs` | Version-1 typed JSON envelopes, exact field allowlists, bounded UTF-8 input, sorted-key canonical encoding and SHA-256 vectors. This is a local contract, not JCS/A2A or a signature scheme. |
| `journal.mjs` | Single-writer snapshots with a lock, copy-on-transaction, fsynced temporary file, rename and directory fsync. |
| `harness.mjs` | Synthetic task binding, staged approvals, producer sequences, inbox/cursors, turn reservations, execution intent, cancellation observations and reconciliation. |
| `mocks.mjs` | Two-owner grant policy, fake clock, deliberately non-idempotent session/submission methods and controllable lost replies/offline/abort behavior. |
| `runtime-error.mjs` | Explicit recoverable runtime/evidence errors; unexpected assertion, storage and programming errors propagate. |
| `demo.mjs` | Scripted request → question → answer → result → simulated requester acceptance. |

The coordinator's public methods take fixture arguments such as `actor` and a
transport `context`. Those are simulated trusted inputs, **not authentication**.
An owner string in a real network request must never call these methods as if
it established authority. There is no HTTP server or real connector here.

## Boundaries exercised

- Owner A's FlyBot and owner B's OpenCode endpoint are distinct. Both directional
grants are required. Same-owner enrollment is rejected, preserving Bot
Social's design constraint. Display names and a common test operator cannot
combine the two approval authorities.
- Every work message waits for the sender owner's delivery decision. Requests
and answers also wait for the receiving owner's execution decision. Wrong-owner
decisions, changed content under the same event/turn key, grants revoked during
a hold and rebound owners cannot release execution.
- A host-enrolled task fixes selected input, criteria, recipients and budget in
its digest. New tasks get separate sessions; clarification stays in the same
session and must answer that task's outstanding question. Arbitrary chats,
room contexts, unknown fields/versions and unrelated replies cannot start work.
- The runtime records a typed `peer-untrusted` data object. Injection text stays
data; no prompt is assembled and there is no tool executor. Real prompt/fence
integration and a verified no-tools OpenCode policy are future work.
- The inbox/cursor and held event are saved before returning. Before any remote
operation, a turn and execution intent are saved. Lost submit replies are
reconciled against scripted history. Ambiguous/unavailable history and lost
session-creation replies stay `uncertain` without automatic resubmission.
- Cancellation persists a generation/tombstone before best-effort abort. A mock
runtime's explicit `stopped` observation is different from `requested` or
offline. Late messages cannot revive a cancelled task. An authenticated real
runtime stop observation remains a connector obligation.
- Turn/message/progress limits and absolute deadlines survive close/reopen.
There are at most two unfinished tasks and one synchronous work turn at a time
per task. The final allowed turn can finish; no additional turn is admitted.
Cancellation remains available after budgets or grants are exhausted.

`result-ready` tasks await a human decision and intentionally do not consume an
execution slot; they remain subject to the journal's 64-task retention bound.

Call `enforce()` when advancing the fake clock or revoking a grant to simulate
the supervisor tick. It cancels overdue/revoked work; there is deliberately no
real timer. Dispatch, release and reconciliation also recheck the applicable
scope/deadline. At most 64 tasks and 256 inbox records are retained per fixture
journal; reaching capacity fails closed rather than forgetting dedupe keys.

The mock can cancel a returned result. Current FlyBot `tasks.js` cannot; this is
an explicitly proposed host seam, not evidence that that gap is implemented.

## Persistence and remaining evidence

All demo/test data lives under a fresh `os.tmpdir()` directory and is removed
after the run. Journal methods are synchronous; one writer owns the lock. The
durability operations are verified on Linux. A lock left after a killed process
is not stolen automatically; verify the writer is dead before removing it in a
disposable test directory. Never point this harness at personal data.

Tests inject lost remote replies, then close/reopen the coordinator. They do not
claim a SIGKILL/power-loss filesystem test, a durable SSE replay test or a real
remote exactly-once guarantee. The mock's discoverable session/task association
and historical turn digest are deliberate fake capabilities. A pinned OpenCode
version must establish equivalent evidence or remain uncertain; its actual
`messageID`/abort behavior is not proven by this harness.

Still required: the reviewed host task-observation seam, structured social
correlation/approval previews, real grant/authentication enforcement, a pinned
OpenCode interface test, real tool isolation, enforceable model/cost limits and
an approved real trial. Team-mode gates, multi-human deployment and standard
A2A interoperability are unchanged.
66 changes: 66 additions & 0 deletions dev/interop/codec.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import { createHash } from 'node:crypto'

const KINDS = new Set(['request', 'question', 'answer', 'progress', 'result', 'failure', 'cancel', 'cancel-observed'])
const FIELDS = ['version', 'kind', 'eventId', 'taskId', 'turnId', 'generation', 'seq', 'inputVersion', 'inputDigest', 'inReplyTo', 'payload']
const ID = /^[a-zA-Z0-9][a-zA-Z0-9._:-]{0,99}$/
export function requireValue(condition, message) {
if (!condition) throw new Error(message)
}
export function object(value, keys) {
requireValue(value !== null && typeof value === 'object' && !Array.isArray(value), 'expected object')
requireValue(Object.keys(value).every(key => keys.includes(key)), 'unknown field')
}
function text(value, max = 16384) {
requireValue(typeof value === 'string' && value.length > 0 && Buffer.byteLength(value) <= max, 'invalid text size')
}
export function validateInput(value) {
object(value, ['requirement', 'excerpts', 'criteria'])
text(value.requirement); text(value.criteria)
requireValue(Array.isArray(value.excerpts) && value.excerpts.length > 0 && value.excerpts.length <= 16, 'invalid excerpts')
const ids = new Set()
for (const excerpt of value.excerpts) {
object(excerpt, ['id', 'text'])
requireValue(typeof excerpt.id === 'string' && ID.test(excerpt.id) && !ids.has(excerpt.id), 'invalid excerpt id')
ids.add(excerpt.id); text(excerpt.text)
}
requireValue(Buffer.byteLength(JSON.stringify(value)) <= 16384, 'input too large')
}
export function canonical(value, depth = 0) {
requireValue(depth < 16, 'value too deep')
if (value === null || typeof value === 'string' || typeof value === 'boolean') return JSON.stringify(value)
if (typeof value === 'number') {
requireValue(Number.isFinite(value), 'nonfinite number')
return JSON.stringify(value)
}
if (Array.isArray(value)) return `[${value.map(item => canonical(item, depth + 1)).join(',')}]`
object(value, Object.keys(value ?? {}))
return `{${Object.keys(value).sort().map(key => `${JSON.stringify(key)}:${canonical(value[key], depth + 1)}`).join(',')}}`
}
export const digest = value => createHash('sha256').update(canonical(value)).digest('hex')

export function decode(raw) {
requireValue(typeof raw === 'string' && Buffer.byteLength(raw) <= 32768, 'envelope too large')
const event = JSON.parse(raw)
object(event, FIELDS)
requireValue(event.version === 1 && KINDS.has(event.kind), 'unsupported contract')
for (const key of ['eventId', 'taskId', 'turnId']) requireValue(typeof event[key] === 'string' && ID.test(event[key]), `invalid ${key}`)
for (const key of ['generation', 'seq', 'inputVersion']) requireValue(Number.isSafeInteger(event[key]) && event[key] >= (key === 'generation' ? 0 : 1), `invalid ${key}`)
requireValue(typeof event.inputDigest === 'string' && /^[a-f0-9]{64}$/.test(event.inputDigest), 'invalid digest')
if (event.inReplyTo !== undefined) requireValue(typeof event.inReplyTo === 'string' && ID.test(event.inReplyTo), 'invalid reply id')
if (['answer', 'question', 'progress', 'result', 'failure', 'cancel-observed'].includes(event.kind)) requireValue(event.inReplyTo !== undefined, 'missing reply id')
const payload = event.payload
if (event.kind === 'request') {
object(payload, ['input']); validateInput(payload.input)
} else if (event.kind === 'result') {
object(payload, ['summary', 'evidence']); text(payload.summary)
requireValue(Array.isArray(payload.evidence) && payload.evidence.length > 0 && payload.evidence.length <= 16 && payload.evidence.every(id => typeof id === 'string' && ID.test(id)), 'invalid result evidence')
requireValue(Buffer.byteLength(JSON.stringify(payload)) <= 16384, 'result too large')
} else if (event.kind === 'cancel-observed') {
object(payload, [])
} else {
object(payload, ['text']); text(payload.text, event.kind === 'progress' ? 2000 : 16384)
}
return event
}

export const encode = event => canonical(decode(JSON.stringify(event)))
28 changes: 28 additions & 0 deletions dev/interop/demo.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import { mkdtempSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { Pilot } from './harness.mjs'
import { FakeClock, MockPolicy, ScriptedRuntime, input, eventFor } from './mocks.mjs'

const dir = mkdtempSync(join(tmpdir(), 'flybot-interop-demo-'))
const runtime = new ScriptedRuntime()
const pilot = new Pilot({ dir, clock: new FakeClock(), policy: new MockPolicy(), runtime })
try {
const task = pilot.enroll({ id: 'retry-analysis', actor: 'owner-a', from: 'flybot', to: 'opencode', input })
const deliver = event => {
const forward = ['request', 'answer'].includes(event.kind)
pilot.receive(JSON.stringify(event), { server: 'mock-social', sender: forward ? 'flybot' : 'opencode', recipient: forward ? 'opencode' : 'flybot' })
pilot.approve(event.eventId, forward ? 'owner-a' : 'owner-b', 'delivery', true)
if (forward) pilot.approve(event.eventId, 'owner-b', 'execution', true)
}
const request = eventFor(task, 'request')
deliver(request)
const question = eventFor(task, 'question', { inReplyTo: request.eventId, payload: { text: 'Must cancellation stop the next retry?' } })
deliver(question)
const answer = eventFor(task, 'answer', { turnId: 'turn-2', seq: 2, inReplyTo: question.eventId, payload: { text: 'Yes. Never start another attempt after cancellation.' } })
deliver(answer)
deliver(eventFor(task, 'result', { turnId: 'turn-2', seq: 2, inReplyTo: answer.eventId, payload: { summary: 'Cancellation after the pre-check can race the awaited operation. Bound attempts, pass a cancellation signal to attempt, and recheck before each retry.', evidence: ['excerpt-1'] } }))
const beforeAcceptance = pilot.task(task.id).status
pilot.accept(task.id, 'owner-a')
console.log(JSON.stringify({ evidence: 'offline-mock', owners: ['owner-a', 'owner-b'], beforeAcceptance, afterAcceptance: pilot.task(task.id).status, sessions: runtime.sessions.length, submittedTurns: runtime.submissions.length, realModelRequests: 0, realCost: 0, liveCompatibility: 'not tested' }, null, 2))
} finally { pilot.close(); rmSync(dir, { recursive: true, force: true }) }
Loading
Loading