OpenDesk for macOS

A beautiful native SwiftUI front end for OpenDesk — gives Claude (Opus 4.7, Sonnet 4.6, Haiku 4.5) eyes, hands and a keyboard on your Mac.

Everything OpenDesk can do — screenshots, mouse, keyboard, app control, clipboard, OCR, workflows, schedules, encrypted LAN remote control, audit trail — wrapped in a polished GUI you can hand to someone who's never seen a terminal.

What makes this build interesting

• Native MCP server in Swift. OpenDesk.app is its own MCP server (no Python helper to grant permissions to). Claude Code spawns the same signed binary you authorize for Accessibility, so one grant covers everything — same pattern as Karabiner, BetterTouchTool, Rectangle.
• Bring-your-own-subscription auth (default). Drives claude CLI under the hood, so your Pro/Max plan handles login and billing. No Anthropic API key to paste.
• API-key fallback for users without a Claude subscription.
• Per-tool permission system in the GUI — pre-approve screenshot / mouse / keyboard / app etc. so the chat doesn't prompt per call.
• Yolo mode (off by default) — toggles --dangerously-skip-permissions in the background for fully hands-off runs.

---

Prerequisites

macOS	14 Sonoma or newer (built and tested on macOS 26 Tahoe)
Apple Silicon	Required — this is an arm64 build
Xcode Command Line Tools	xcode-select --install (≈1 GB)
Swift	5.9+ — comes with the Command Line Tools
Claude Code	brew install claude or curl -fsSL claude.ai/install.sh | sh

You don't need a paid Apple Developer account to build and run this locally. Ad-hoc signing works for personal use; for distribution to other people, see the Distribution section at the bottom.

---

Quick start

git clone https://github.com/vitalops/opendesk_app.git
cd opendesk_app
./build.sh open

Three things happen:

1. swift build compiles the binary (~1 min cold, ~5 sec warm).
2. The script assembles a real .app bundle, ad-hoc signs it, and installs it to /Applications/OpenDesk.app.
3. The app launches.

On first launch you'll be prompted by macOS for two permissions — Accessibility and Screen Recording. Click Allow on both. Then chat at it: "Take a screenshot and tell me what's on screen."

---

First-launch permissions in detail

OpenDesk needs two macOS-level grants. The native OS prompts will fire the first time the app tries to use the relevant API.

Permission	Why	Where to grant
Accessibility	Posts CGEvents for mouse / keyboard / hotkeys. Reads accessibility trees of focused apps.	System Settings → Privacy & Security → Accessibility
Screen Recording	Captures screenshots so Claude can see what's on screen.	System Settings → Privacy & Security → Screen Recording

The app has a built-in diagnostic so you don't have to guess whether they're working:

1. Open OpenDesk → Settings (sidebar, bottom).
2. Scroll to Desktop control diagnostic.
3. Click Test now.
   • ✓ "Desktop control works." → you're done.
   • ⚠ "OpenDesk isn't trusted by macOS Accessibility yet." → see below.

If the diagnostic stays amber after granting

This is a known macOS quirk specifically affecting ad-hoc-signed builds (which yours is, until you pay for an Apple Developer ID — see Distribution).

macOS TCC stores (bundle id, cdhash) for every permission grant. Each ./build.sh produces a different cdhash, so even though the entry shows ON in System Settings, macOS silently treats it as denied.

The fix is built into the GUI:

1. In the diagnostic's amber banner, click Reset OpenDesk's TCC entries. This runs tccutil reset Accessibility ai.opendesk.macos + tccutil reset ScreenCapture ai.opendesk.macos for you.
2. Then click Quit & Relaunch OpenDesk.
3. On relaunch, macOS will pop fresh native consent dialogs. Click Allow.
4. Click Test now — should be green.

One-shot recovery from the shell

If you ever get into a weirder state:

sudo tccutil reset All ai.opendesk.macos
pkill -f "/Applications/OpenDesk.app"
open /Applications/OpenDesk.app

---

Auth: Subscription vs API Key

Settings → Sign-in method has two cards:

Subscription (default, recommended)

Uses your existing Claude Pro / Max / Team plan via the claude CLI. The chat composer spawns claude -p with OpenDesk registered as the MCP server. No API key to paste; your subscription handles billing.

Setup is a three-step wizard in Settings:

1. Install Claude Code (button auto-detects; if missing, drops you into Terminal with the official installer).
2. Sign in (opens Terminal with claude login — must happen in a TTY).
3. Test the connection — runs claude -p "ok" --max-turns 1 and reports.

API Key

Stored in macOS Keychain under service ai.opendesk.macos, account anthropic_api_key. Never written to disk in plain text. Get a key from console.anthropic.com.

Switch between modes with one click; the chat composer re-routes through whichever engine is active.

---

Repository layout

opendesk_app/
├── README.md             ← you are here
├── LICENSE               ← MIT
├── Package.swift         ← Swift Package Manager spec, macOS 14+
├── build.sh              ← swift build + assemble .app + install to /Applications
├── AppBundle/
│   └── Info.plist        ← bundle id ai.opendesk.macos, permission strings, mDNS service
└── Sources/OpenDesk/
    ├── OpenDeskApp.swift     ← @main; routes --mcp arg to NativeMCPServer, else launches SwiftUI
    ├── Theme/Theme.swift     ← palette, gradients, .odCard, brand buttons
    ├── MCP/                  ← native MCP server (NEW — the polish layer)
    │   ├── NativeMCPServer.swift
    │   ├── NativeTools.swift
    │   ├── NativeScreenshot.swift
    │   ├── NativeMouse.swift
    │   ├── NativeKeyboard.swift
    │   ├── NativeApp.swift
    │   └── NativeClipboard.swift
    ├── Models/MCPTool.swift
    ├── Views/                ← Root + Sidebar + 8 sections
    │   ├── RootView.swift, Sidebar.swift
    │   ├── OverviewView.swift   ← dashboard, status, live screenshot, feature grid
    │   ├── ChatView.swift       ← Computer Use chat with stream rendering
    │   ├── ToolsView.swift      ← per-tool playground
    │   ├── WorkflowsView.swift, SchedulesView.swift, PeersView.swift
    │   ├── AuditView.swift      ← every MCP call this session
    │   └── SettingsView.swift   ← auth, tool perms, permissions, diagnostic
    ├── Services/
    │   ├── ClaudeCodeService.swift   ← claude CLI driver + chat stream
    │   ├── ClaudeService.swift       ← Anthropic API path (API-key mode)
    │   ├── OpenDeskBridge.swift      ← Python opendesk bridge (legacy, Tools playground only)
    │   ├── MCPRegistration.swift     ← writes /tmp MCP config pointing at self
    │   ├── NativeDesktopProbe.swift  ← Test-now diagnostic, Quit&Relaunch, tccutil reset
    │   ├── CancellableProcess.swift  ← Process wrapper with descendants kill
    │   ├── PermissionsService.swift  ← TCC status + deep-links
    │   ├── KeychainStore.swift, SettingsStore.swift
    │   └── ProcessTree.swift         ← Emergency-stop sweep
    └── Resources/
        └── bridge/opendesk_bridge.py ← Python shim (used only by the Tools playground)

---

Architecture (how a chat actually reaches your desktop)

SwiftUI app
   └── User hits Enter in Chat
       └── ChatView.sendViaClaudeCode()
           └── MCPRegistration.writeOpenDeskConfig()  ──▶  /tmp/opendesk-mcp-config.json
                                                              {
                                                                "mcpServers": {
                                                                  "opendesk": {
                                                                    "command": "/Applications/OpenDesk.app/Contents/MacOS/OpenDesk",
                                                                    "args": ["--mcp"]
                                                                  }
                                                                }
                                                              }
           └── ClaudeCodeService.chat()
               └── spawns:  claude -p "<prompt>"
                              --mcp-config /tmp/opendesk-mcp-config.json
                              --model claude-opus-4-7
                              --output-format stream-json
                              --allowedTools mcp__opendesk__{screenshot,mouse,...}
                              [--dangerously-skip-permissions if Yolo on]
                   └── claude launches:  /Applications/OpenDesk.app/Contents/MacOS/OpenDesk --mcp
                       └── NativeMCPServer JSON-RPC loop
                           └── NativeScreenshot / NativeMouse / NativeKeyboard / NativeApp / NativeClipboard
                               └── CGEventPost, CGDisplayCreateImage, NSWorkspace, NSPasteboard
                                   ▲ All running inside the same signed binary
                                     that has Accessibility + Screen Recording.
                                     One macOS grant covers the whole chain.

This is why one Accessibility grant works for everything — there's no Python interpreter or grandchild process for TCC to get confused about.

---

Common commands

./build.sh              # build + install to /Applications (default)
./build.sh open         # build + install + launch
./build.sh debug        # debug build → /Applications
./build.sh local        # build → ./build/OpenDesk.app only (don't touch /Applications)

Inside the app (menu OpenDesk):

• ⌘N — New Chat
• ⌘⇧S — Take Screenshot
• ⌘⇧C — Copy Session as Markdown (transcript + screenshots written to /tmp)
• ⌘⇧E — Export Session… (markdown + screenshot folder)
• ⌘. — Stop Running Chat
• ⌘⇧. — Emergency Stop · Kill all helpers

---

Troubleshooting

Symptom	Cause	Fix
Test now stays "Running test…"	Main thread blocked	Force-quit (pkill -f /Applications/OpenDesk.app) and relaunch
Test now → "not trusted" even after granting	cdhash mismatch (ad-hoc signing)	Click Reset OpenDesk's TCC entries then Quit & Relaunch in the diagnostic banner
Chat sends nothing	Auth mode not configured	Settings → Sign-in method → finish Subscription wizard or paste API key
Claude Code missing in Subscription mode	Not installed	Settings → install button (opens Terminal with the installer)
opendesk-mcp Python process won't die	Stale orphan from a prior crash	OpenDesk menu → "Emergency Stop · Kill all helpers" (⌘⇧.)
App icon shows generic	We don't ship an icon yet	Drop a PNG/icns next to Info.plist and set CFBundleIconFile — TODO

---

Distribution — shipping to other users

The build above produces an ad-hoc-signed bundle. That's fine for personal use but won't pass macOS Gatekeeper on someone else's machine, and the TCC permissions don't persist across rebuilds (cdhash churn).

To ship publicly you need:

1. Apple Developer Program ($99/yr) — gets you a Developer ID Application certificate. Stable signature across builds → TCC grants survive forever, no tccutil reset dance.
2. Codesign with that cert, hardened runtime, entitlements file.
3. Notarize via xcrun notarytool submit … --wait (mandatory since macOS 10.15).
4. Staple the notarization ticket (xcrun stapler staple OpenDesk.app).
5. Package as DMG (use create-dmg from Homebrew).
6. Auto-updates via Sparkle.
7. GitHub Releases for hosted downloads.

A .github/workflows/release.yml doing codesign + notarize + DMG + asset upload on every git tag is the standard pattern — happy to scaffold one; ask if you want it.

---

Develop / contribute

Open in Xcode:

open Package.swift

…or just edit in any editor and swift build from the repo root. The SwiftUI hot path is Sources/OpenDesk/Views/; the native MCP tools live in Sources/OpenDesk/MCP/. Adding a new tool is ~50 lines: write a NativeXXX.swift, register it in NativeTools.list() and NativeTools.call().

---

License

MIT. See LICENSE.

Credit

OpenDesk Python/JS SDKs: https://github.com/vitalops/opendesk

Built with Claude Opus 4.7 in Claude Code.