Andrew Ng has announced OpenWorker, an open-source desktop agent that produces finished work rather than conversation. OpenWorker asks the user for an outcome, not a prompt: a polished document, a Slack reply containing the actual numbers, an updated calendar, a triaged inbox. It then breaks that outcome into steps, works across local files and connected apps, and checks in before anything consequential.

The architecture is four layers, and all of them run on your machine

The repository contains 119 Python files (~32,400 lines) under coworker/, 149 TypeScript/TSX files under surfaces/gui/, and 78 backend test modules.

The stack breaks down as follows:

• Desktop shell — a Tauri 2 native window wrapping a React 18 UI. The bundle identifier is com.openworker.desktop, and the shell supervises the Python server itself.
• Local agent server — Python 3.10+ on FastAPI and uvicorn, binding to 127.0.0.1:8765 by default. The example config caps a turn at 12 model↔tool iterations.
• Capability and connector layer — vetted local tools (files, git, ripgrep-backed search, shell, todo) plus hosted integrations plus MCP.
• Model router — one interface over native, OpenAI-compatible, reseller and local providers.

The engine is built on aisuite, Andrew Ng’s provider-agnostic LLM library.

Bring your own model, from a deliberately small curated list

There is no OpenWorker inference service. The user pastes an API key, or points the app at a local runtime.

The curated model matrix contains exactly 30 entries. Native providers cover OpenAI (GPT-5.6 Sol/Terra/Luna and GPT-5.5), Anthropic (Claude Fable 5, Opus 4.8, Sonnet 4.6, Haiku 4.5) and Google (Gemini 3.1 Pro, 3.6 Flash, 2.5 Pro, 2.5 Flash). OpenAI-compatible vendors add GLM-5.2, DeepSeek V4, Kimi K2.6, MiniMax M2.5, Qwen3 Max, Grok 4.3 and Mistral Large. Open-weight models arrive through Together AI and Fireworks, and fully local models through Ollama, which requires no key at all.

The permission engine is the actual engineering story

Most desktop agent projects treat approvals as a UI afterthought. OpenWorker treats them as a typed layer.

Every tool call is classified into one of four risk classes: read (no side effects), write_local (mutates the workspace, path-scoped), exec (runs commands), and external (side effects off the machine). Five permission modes then decide what happens: discuss and plan are read-only, interactive is the default and asks before writes, commands and external actions, auto allows everything while remaining path-scoped, and custom auto-approves a user-listed set of tools.

Two design decisions stand out.

• First, unattended mode does not raise the autonomy ceiling — it only changes where the human is reached. Prompts that would appear inline are routed to an Inbox, and the session suspends until answered.
• Second, task-scoped standing rules are restricted to external risk only. Shell commands ask forever, by design.

The built-in ops persona also instructs the model to treat content from tools, logs, the web, files and incoming messages as untrusted data rather than instructions. That is an explicit prompt-injection posture, written into the shipped persona.

Privacy: local-first

Model calls go directly from the machine to the configured provider. Conversations, connector tokens and model keys stay local, and the secret store is designed so that secrets never enter the model’s context, prompts or traces.

The only cloud component is an optional broker that handles OAuth handshakes for one-click connectors, using Auth0 Authorization Code with PKCE. Connector tokens are handed straight to the machine and are never stored in the cloud. The app is fully functional signed out, using manually pasted credentials.

Key Takeaways

• OpenWorker is Andrew Ng’s MIT-licensed desktop AI coworker that returns finished deliverables, not chat replies.
• The stack is a Tauri 2 + React shell over a local Python FastAPI agent server built on aisuite.
• Model access is bring-your-own-key across 30 curated tool-calling models, plus fully local Ollama.
• A typed risk engine (read/write_local/exec/external) gates every action across five permission modes.

---

Check out the GitHub Repo, the project site, and the announcement. All credit for this research goes to the researchers and developers of this project.