Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,8 @@ QUERY_WORKER_POOL_SIZE=2
QUERY_QUEUE_LIMIT=64

# Register the Schema Commons types (org.haverstack/note, bookmark, task,
# contact, article, place, page, photo) from @haverstack/commons on startup
# (default: false). Off by default because the package is Draft status and
# contact, article, place, page, image, post, site) from @haverstack/commons
# on startup (default: false). Off by default because the package is Draft status and
# registering types is not free — they show up in GET /types and in every
# app's type cache. See docs/deployment.md#schema-commons-seeding.
SEED_COMMONS_TYPES=false
Expand Down
183 changes: 101 additions & 82 deletions docs/api.md

Large diffs are not rendered by default.

7 changes: 4 additions & 3 deletions docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ What bounds the waiting instead is `QUERY_QUEUE_LIMIT` (default 64): the number

A worker that dies _unexpectedly_ — as opposed to one deliberately recycled at a deadline — is respawned with exponential backoff (100ms doubling to 30s), and after five consecutive failures the pool reports itself unhealthy and fails searches fast with the underlying error. This is what keeps a fatal misconfiguration (a `DB_PATH` that has become unreadable, say) from spinning the process respawning threads forever, and makes it surface as itself rather than as a stream of timeouts. The pool recovers on its own once the cause clears.

Every other route (`GET /records/:id`, `POST /records`, `PATCH /records/:id`, etc.) still runs on the main request thread — those are index-bound lookups and writes, not full-index scans, so they aren't the unbounded-cost case this section covers.
Every other route (`GET /records/:id`, `POST /records`, `PATCH /records/:id/content`, etc.) still runs on the main request thread — those are index-bound lookups and writes, not full-index scans, so they aren't the unbounded-cost case this section covers.

Raise `QUERY_TIMEOUT_MS` if your store is large enough that legitimate searches routinely take longer than the default, and raise `QUERY_WORKER_POOL_SIZE` if concurrent searches should run in parallel rather than queue behind each other — each pool worker is a full second (or third, ...) connection to the database, so size it against expected concurrent search load, not total request volume. Raise `QUERY_QUEUE_LIMIT` only if you would rather hold bursts than shed them; lower it to fail fast under overload.

Expand All @@ -172,7 +172,7 @@ Note that `GET /records` and `POST /records/query` are both reachable without au

The `data` volume (`DB_PATH`'s directory) holds more than the database file itself: `stack.db-wal` and `stack.db-shm` are SQLite's write-ahead-log sidecar files, present whenever the process has the database open, and `stack.db.lock` is a storage-ownership lock recording the PID of the process that opened it. Back up or copy the whole directory, not just `stack.db` — a `stack.db` file copied without its `-wal` sidecar can be missing recently-committed data.

The directory also holds `stack.db.tokens` — bearer-token issuance and lookup, kept in its own file rather than inside `stack.db` on purpose: auth material shouldn't travel with the portable stack data. Include it in operational backups (revoking a token should stay revoked across a restore), but exclude it from any "export your data" or stack-sharing flow — a copy of `stack.db` alone is the correct unit for that, and restoring one should never resurrect tokens the owner revoked.
The directory also holds `stack.db.tokens` — bearer-token issuance and lookup, kept in its own file rather than inside `stack.db` on purpose: auth material shouldn't travel with the portable stack data. Include it in operational backups (revoking a token should stay revoked across a restore), but exclude it from any "export your data" or stack-sharing flow — a copy of `stack.db` alone is the correct unit for that, and restoring one should never resurrect tokens the owner revoked. The same holds for the other two sibling stores: `stack.db.nonces` (in-flight DID handshake challenges) and `stack.db.installs` ([install requests](./api.md#installs) awaiting the owner, which are not approvals and so live outside the stack).

A given `DB_PATH` may be held open by only one process at a time. If a second server instance (a botched redeploy, a stuck old container, a manual `pnpm start` alongside a running one) tries to open the same path, it fails at startup with an error naming the PID already holding it, rather than corrupting the database or silently serving alongside it — WAL and real file locking make concurrent _storage_ access safer than the old single-file adapter, but the single-writer topology stands regardless: permissions only exist behind this one server process, so a second writer would be a second, unenforced trust boundary. If you see this error on a routine restart, the previous process likely hasn't exited yet — give it time to shut down cleanly, or confirm it has actually stopped, before starting the replacement.

Expand All @@ -198,6 +198,7 @@ Two consequences worth being deliberate about:

- **The `authenticated` tier is a grant to the public.** A grant whose grantee is `{ "kind": "authenticated" }` resolves for any authenticated entity, and with the handshake open that is anyone at all. See [Permissions](./api.md#permissions). A grant naming a DID (`{ "kind": "entity", "entityId": "did:key:..." }`) is unaffected — that is the vouching mechanism. No tier is reachable by omission: a grant whose grantee is absent or unrecognized is refused, and confers nothing if it reaches storage some other way, so this is a reach you have to ask for by name.
- **Rate limiting is what bounds the token table.** A stranger's handshake writes a token row with a 7-day expiry. Expired rows are reclaimed, but live ones are bounded only by issuance rate × TTL, so the proxy-level rate limit configured above is what actually caps the table. The reverse-proxy examples in this guide already cover `/auth/*`; if you write your own, do not exempt it.
- **Install requests are bounded the same way.** Any authenticated key can ask to be installed, which stores one pending request per key until the owner acts on it or it goes 7 days without being re-sent. Rate-limiting `/auth/*` bounds how many keys can ask.

Read access is unaffected either way: `GET /records`, `POST /records/query`, `GET /records/:id` and `GET /types` already serve anonymous requests, subject to record-level permissions, and a token changes nothing about what they return.

Expand All @@ -207,6 +208,6 @@ Read access is unaffected either way: `GET /records`, `POST /records/query`, `GE

## Schema Commons seeding

`SEED_COMMONS_TYPES=true` registers the [Schema Commons](https://github.com/haverstack/core/blob/main/docs/commons/README.md) types (`org.haverstack/note`, `bookmark`, `task`, `contact`, `article`, `place`, `page`, `photo`) from `@haverstack/commons` on every boot, via `defineType()`, which is idempotent by construction — safe to leave on permanently, and safe to flip on for an already-running stack.
`SEED_COMMONS_TYPES=true` registers the [Schema Commons](https://github.com/haverstack/core/blob/main/docs/commons/README.md) types (`org.haverstack/note`, `bookmark`, `task`, `contact`, `article`, `place`, `page`, `image`, `post`, `site`) from `@haverstack/commons` on every boot, via `defineType()`, which is idempotent by construction — safe to leave on permanently, and safe to flip on for an already-running stack.

It defaults to **off**. Registering a type is not free: it shows up in `GET /types` and gets cached by every app that talks to this stack, whether or not that app uses it. More importantly, `@haverstack/commons` is Draft status — the [governance doc](https://github.com/haverstack/core/blob/main/docs/commons/README.md) explicitly reserves the right to change a Draft type's definition in place, without a version bump, until there's an install base to break. Opting in is the honest posture for a package with that status; turning it on is a statement that you want this reference server to demonstrate commons interop, not a default every deployer should inherit silently.
10 changes: 5 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,18 +41,18 @@
"format:check": "prettier --check ."
},
"dependencies": {
"@haverstack/adapter-local": "^0.37.0",
"@haverstack/commons": "^0.32.0",
"@haverstack/core": "^0.38.0",
"@haverstack/wire-types": "^0.37.0",
"@haverstack/adapter-local": "^0.42.0",
"@haverstack/commons": "^0.38.0",
"@haverstack/core": "^0.43.0",
"@haverstack/wire-types": "^0.42.0",
"@hono/node-server": "^2.1.1",
"hono": "^4.13.7",
"pino": "^10.3.1",
"pino-pretty": "^13.1.3"
},
"devDependencies": {
"@eslint/js": "^10.0.1",
"@haverstack/conformance-fixtures": "^0.32.0",
"@haverstack/conformance-fixtures": "^0.37.0",
"@types/node": "^26.2.0",
"eslint": "^10.8.1",
"eslint-config-prettier": "^10.1.8",
Expand Down
76 changes: 38 additions & 38 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import { attachmentRoutes } from './routes/attachments.js';
import { entityRoutes } from './routes/entity.js';
import { tokenRoutes } from './routes/tokens.js';
import { authRoutes } from './routes/auth.js';
import { installRoutes } from './routes/installs.js';
import { wireError } from './wireError.js';

export type { AppEnv };
Expand Down Expand Up @@ -79,6 +80,7 @@ export function createApp(ctx: StackContext, config: Config, logger: Logger): Ho
app.route('/entity', entityRoutes(ctx));
app.route('/tokens', tokenRoutes(ctx));
app.route('/auth', authRoutes(ctx, config.authOrigin, logger));
app.route('/installs', installRoutes(ctx));

app.notFound((c) => wireError(c, 404, 'not_found', 'Not found'));

Expand Down
18 changes: 18 additions & 0 deletions src/lib/canonicalJson.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/**
* JSON with every object's keys in sorted order, so two values that mean
* the same thing serialize the same way whatever order a client wrote
* their keys in. Array order is kept, since it is part of the value.
*/
export function canonicalJson(value: unknown): string {
return JSON.stringify(sortKeys(value));
}

function sortKeys(value: unknown): unknown {
if (Array.isArray(value)) return value.map(sortKeys);
if (value === null || typeof value !== 'object') return value;
return Object.fromEntries(
Object.keys(value)
.sort()
.map((key) => [key, sortKeys((value as Record<string, unknown>)[key])]),
);
}
26 changes: 26 additions & 0 deletions src/lib/historyPage.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// A record's version history and its journal are the two reads with no
// ceiling when `limit` is omitted: omitting it reads everything by
// contract, so a page size can't be supplied on the caller's behalf
// without silently truncating exactly the caller reading the whole thing.
// This server bounds a page and says so in `cursor` instead, as the spec
// lets it. See docs/spec/wire-format.md § Versions and § Journal.
export const MAX_HISTORY_PAGE = 500;

/**
* Reads one page of a history, one row past it to learn whether anything
* follows, so `cursor` is null exactly when the history is exhausted. A
* `limit` below 1 is passed through as given: whether it is in range is
* `Stack`'s to judge.
*/
export async function readHistoryPage<T>(
limit: number | undefined,
read: (limit: number) => Promise<T[]>,
cursorOf: (row: T) => number,
): Promise<{ rows: T[]; cursor: number | null }> {
const page = Math.min(limit ?? MAX_HISTORY_PAGE, MAX_HISTORY_PAGE);
if (page < 1) return { rows: await read(page), cursor: null };
const rows = await read(page + 1);
if (rows.length <= page) return { rows, cursor: null };
const kept = rows.slice(0, page);
return { rows: kept, cursor: cursorOf(kept[kept.length - 1]!) };
}
14 changes: 14 additions & 0 deletions src/lib/ifMatch.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import type { Context } from 'hono';
import type { IfVersionOptions } from '@haverstack/core';
import { parseIfMatch } from '@haverstack/core/wire';
import type { AppEnv } from '../types.js';

/**
* The `ifVersion` an `If-Match` header carries. Every record write reads
* one and checks it when present; a malformed header is core's 400, never a
* fallthrough to unconditional. See docs/spec/wire-format.md
* § Optimistic concurrency.
*/
export function ifMatchVersion(c: Context<AppEnv>): IfVersionOptions {
return { ifVersion: parseIfMatch(c.req.header('If-Match')) };
}
Loading
Loading