Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ anchor: '@file'
created: 2026-10-20T01:52:57Z
norm: '1'
sig: 021fb596db81e6d0
body_hash: e41afcbcc3225d7f
raw_hash: 76090adc05ae0915
vouched: 2026-10-20T14:14:36Z
body_hash: bd47a44b13e883fa
raw_hash: 535a5365401892a7
vouched: 2026-10-20T14:47:24Z
lines: 1-31
---

Expand Down
16 changes: 15 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,19 @@ Notable changes to Desktop Tools, newest first. Each release's GitHub release no

## [Unreleased]

## [0.17.0] - 2026-10-20

### Changed

- The app runs a pure Swift host in place of the Peekaboo Bridge runtime, and bundles its own MCP server, `Desktop Tools.app/Contents/MacOS/desktop-tools-mcp`. Set agents up with that path; Node is no longer needed. Calls are faster, for example `desktop_windows` at a p50 of 6 ms instead of 115 ms and an inspection of a Ghostty window at 42 ms instead of 551 ms, and results have the same text. The app's binaries are about 6 MB in place of 84 MB. ([#143](https://github.com/githubnext/desktop-tools/issues/143))
- An `npx` setup of `desktop-tools-mcp` at v0.17.0 or later hands over to the app's server. A setup pinned to an earlier tag no longer works with this app: its calls are refused with `CLIENT_OUTDATED`, naming the app's server to set up instead. ([#143](https://github.com/githubnext/desktop-tools/issues/143))
- `desktop-tools-build --native2` builds the Swift host, client, MCP server and app without Peekaboo; embedders switch with it and `createDesktop({ wire: 2 })`. The default build is unchanged. ([#171](https://github.com/githubnext/desktop-tools/issues/171))

### Known gaps

- `desktop_switch_space` is checked only on a Mac with one Space.
- Where MenuBarAgent doesn't draw the menu bar, `desktop_menubar_items` lists and clicks status items through their WindowServer windows. This path is unit-tested but not yet checked live.

## [0.16.1] - 2026-10-19

### Fixed
Expand Down Expand Up @@ -253,7 +266,8 @@ Notable changes to Desktop Tools, newest first. Each release's GitHub release no

- The first release: a signed, notarized Desktop Tools menu bar app, and the desktop tools served over MCP. ([#28](https://github.com/githubnext/desktop-tools/pull/28), [#29](https://github.com/githubnext/desktop-tools/pull/29), [#12](https://github.com/githubnext/desktop-tools/pull/12))

[Unreleased]: https://github.com/githubnext/desktop-tools/compare/v0.16.1...HEAD
[Unreleased]: https://github.com/githubnext/desktop-tools/compare/v0.17.0...HEAD
[0.17.0]: https://github.com/githubnext/desktop-tools/compare/v0.16.1...v0.17.0
[0.16.1]: https://github.com/githubnext/desktop-tools/compare/v0.16.0...v0.16.1
[0.16.0]: https://github.com/githubnext/desktop-tools/compare/v0.15.0...v0.16.0
[0.15.0]: https://github.com/githubnext/desktop-tools/compare/v0.14.1...v0.15.0
Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,23 @@ Native macOS desktop inspection and input for agents: a Swift runtime that a sig

## Set up for an agent

You need macOS 15 or later and Node 22 or later. The MCP server runs from this repository through `npx`; the package isn't on the npm registry.
You need macOS 15 or later. The MCP server ships inside the app.

1. **Install the app.** Download the DMG from the [latest release](https://github.com/githubnext/desktop-tools/releases/latest), drag **Desktop Tools** to Applications, and open it. It runs in the menu bar.
2. **Grant permissions.** From its menu, grant Accessibility, Screen Recording and Keyboard input. The grants belong to the app, not to your terminal or agent.
3. **Add the MCP server** to your client. The `#v…` suffix pins a release tag; leave it off to follow `main`.
3. **Add the MCP server** to your client. It is `desktop-tools-mcp` inside the app.

Claude Code:

```sh
claude mcp add desktop -- npx -y --package=github:githubnext/desktop-tools#v0.16.1 desktop-tools-mcp
claude mcp add desktop -- "/Applications/Desktop Tools.app/Contents/MacOS/desktop-tools-mcp"
```

Codex, in `~/.codex/config.toml`:

```toml
[mcp_servers.desktop]
command = "npx"
args = ["-y", "--package=github:githubnext/desktop-tools#v0.16.1", "desktop-tools-mcp"]
command = "/Applications/Desktop Tools.app/Contents/MacOS/desktop-tools-mcp"
```

Other MCP clients that start stdio servers from JSON, such as Cursor or Claude Desktop:
Expand All @@ -33,13 +32,14 @@ You need macOS 15 or later and Node 22 or later. The MCP server runs from this r
{
"mcpServers": {
"desktop": {
"command": "npx",
"args": ["-y", "--package=github:githubnext/desktop-tools#v0.16.1", "desktop-tools-mcp"]
"command": "/Applications/Desktop Tools.app/Contents/MacOS/desktop-tools-mcp"
}
}
}
```

A setup from before 0.17.0 that runs `npx -y --package=github:githubnext/desktop-tools#v0.17.0 desktop-tools-mcp` (Node 22 or later) keeps working: it hands over to the app's server.

4. **Verify.** Ask the agent to call `desktop_apps`. It should list the running apps with an `inventory_completeness` of `complete`. If it says the socket is missing, open the app.

Any process running as you can use the app's grants while it serves agents. Quit it when you don't need it.
Expand Down
2 changes: 1 addition & 1 deletion docs/app.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Agents find the app only through `desktop.sock`, so the app and its client both
- When the app can't start serving, such as while a copy that is quitting still holds the socket, its menu's status line shows why and it retries after 1, 2, 4… seconds, up to 30.
- While serving, the app watches the socket. If it is removed or replaced, the app publishes it again at once and notes it in its status line and its [log](#approved-agents).
- Opening the app while it runs makes it check its socket at once, and retry a failed start without waiting.
- When a call finds no app serving the socket, the client opens the app in the background, which starts it or makes it check its socket, and waits up to 10 seconds for it to serve. Nothing reached the app the first time, so the call then runs once. Only when that fails does the call fail, with `HOST_NOT_RUNNING` or `HOST_NOT_SERVING` and what the client found. Reconnecting the MCP server never helps: each call starts its own client.
- When a call finds no app serving the socket, the client opens the app in the background, which starts it or makes it check its socket, and waits up to 10 seconds for it to serve. Nothing reached the app the first time, so the call then runs once. Only when that fails does the call fail, with `HOST_NOT_RUNNING` or `HOST_NOT_SERVING` and what the client found. Reconnecting the MCP server never helps: the app's `desktop-tools-mcp` reconnects on the next call after the app restarts.

The client does the same from a shell. Both commands default to the socket beside the `desktop-client` link they are run through:

Expand Down
4 changes: 2 additions & 2 deletions docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
`desktop-tools-mcp` (`dist/mcp.js`) serves the desktop tools over stdio MCP, with Ace's native-first guidance as its instructions. By default it uses the [Desktop Tools app](app.md)'s signed client and socket in `~/.local/state/desktop-tools`. Run it from this repository with `npx`, pinned to a release tag, or from a checkout with `node`:

```sh
claude mcp add desktop -- npx -y --package=github:githubnext/desktop-tools#v0.16.1 desktop-tools-mcp
claude mcp add desktop -- npx -y --package=github:githubnext/desktop-tools#v0.17.0 desktop-tools-mcp
claude mcp add desktop -- node <path>/dist/mcp.js
```

Expand All @@ -25,7 +25,7 @@ For Codex, in `~/.codex/config.toml`:
```toml
[mcp_servers.desktop]
command = "npx"
args = ["-y", "--package=github:githubnext/desktop-tools#v0.16.1", "desktop-tools-mcp"]
args = ["-y", "--package=github:githubnext/desktop-tools#v0.17.0", "desktop-tools-mcp"]
```

The tool names, descriptions, and argument schemas come from `@githubnext/desktop-tools/tools` (`desktopTools`, `desktopRequest`), which Ace's `desktop_*` tools also use. [Tools](tools.md) lists them. The descriptions don't repeat the rules every tool shares: exact targets, single-use snapshots, background and foreground input, what each outcome means, pausing, `desktop_wait` and `expect`, and the recipes. `desktopToolRules(app)` returns those rules, and the server adds them to its instructions; a host with its own agent guidance should include them too.
Expand Down
2 changes: 1 addition & 1 deletion native2/CUTOVER.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ Each step ships on its own and can be reverted.
4. **Harden the runtime.** The socket lease (`<socket>.lock`, as Peekaboo's) and draining calls on stop are in. The dev app signs the dylib, client and MCP binary (with the client's identifier, #169), and the peer check refuses an unsigned client. The approval prompt, pause and SELF_TARGET checks pass through the app. Done.
5. **Bridge the wire.** Done: native2's `desktop-tools-client`, one-shot (`<client> <socket> <tool> [--verbose]`, arguments on stdin, one JSON line out) or serving (`<client> <socket> --serve`, one request line in and one reply line out per call, tagged by id, with `{"cancel": id}` passed to the host), opening its containing app in the background when no host serves; and `createDesktop({ wire: 2 })`, which keeps one serving client per config, restarts it when it exits, and holds the event loop only while a call waits ([#172](https://github.com/githubnext/desktop-tools/issues/172)). Against one release host, `desktop_windows` p50 is 13.0 ms through `dist/mcp.js --wire 2`, 10.8 ms through native2's MCP server, and 16.5 ms for a one-shot client. `createDesktop`, `/protocol`, `/tools`, `dist/mcp.js` and ace keep working against a native2 host by changing one option.
6. **Dogfood in the app.** The switch is in: `npm run dev-app -- --native2` bundles native2's dylib and client in place of `native/`'s, and native2's client answers `desktop-client status` and `repair` with `native/`'s output. The bundled dylib is 4.3 MB and the client 0.3 MB, against 40 MB and 43 MB for `native/`'s. In the dev app, with its own grants, `live-check` through `dist/mcp.js --wire 2` passes the same 26 checks as the standalone host, #114's Finder files paste included. Remaining: run the dev app daily with `DESKTOP_TOOLS_STATE=~/.local/state/desktop-tools-dev node dist/mcp.js --wire 2`.
7. **Ship the Swift MCP server.** Bundled: `npm run dev-app -- --native2` puts `desktop-tools-mcp` in the app, signed with the client's identifier, since the host admits exactly one. Run from the app, it serves the app's own socket (`DESKTOP_TOOLS_STATE`, or the app's state folder), opens the app in the background when nothing serves it, as the client does, and refuses with HOST_NOT_SERVING when a signed host drops it. Through it, the dev app passes the 26 live checks the standalone host does, #114 included. The release builds native2's app, dylib, client and MCP server; run the release workflow with `stack: native` to ship `native/`'s again. Still to do: point the README's setup at the bundled server, keep `dist/mcp.js` as a shim that runs it, so existing `npx` configs keep working, and cut the release.
7. **Ship the Swift MCP server.** Bundled: `npm run dev-app -- --native2` puts `desktop-tools-mcp` in the app, signed with the client's identifier, since the host admits exactly one. Run from the app, it serves the app's own socket (`DESKTOP_TOOLS_STATE`, or the app's state folder), opens the app in the background when nothing serves it, as the client does, and refuses with HOST_NOT_SERVING when a signed host drops it. Through it, the dev app passes the 26 live checks the standalone host does, #114 included. The release builds native2's app, dylib, client and MCP server; run the release workflow with `stack: native` to ship `native/`'s again. `dist/mcp.js`, run with no options, hands stdio over to the app's bundled server, so `npx` setups at v0.17.0 or later keep working; one pinned earlier is answered `CLIENT_OUTDATED` by native2's client, naming the server to set up. 0.17.0 points the README at the bundled server and ships native2.
8. **Delete, after agreeing it with ace:** `native/vendor/Peekaboo`, `native/sources/client` and runtime, and the tool logic in `src/client.ts`, leaving a thin wrapper for the API ace imports. Make native2 the contract's source of truth, so `ContractText` stops being generated from Node. This step is the user's decision.

Steps 2 to 4 can run in parallel. Step 5 was the hinge: with it in, steps 6 to 8 don't break anyone.
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@githubnext/desktop-tools",
"version": "0.16.1",
"description": "Signed macOS desktop inspection and input for agents, built on a pinned, patched Peekaboo Bridge",
"version": "0.17.0",
"description": "Signed macOS desktop inspection and input for agents, with a pure Swift host and MCP server",
"license": "MIT",
"type": "module",
"repository": "github:githubnext/desktop-tools",
Expand Down
Loading