# HTTP API Source: https://heyparlour.app/docs/api > The routes a Parlour server answers, and the machine-readable descriptions of this site for agents. Every Parlour server answers on port 8765 of the Mac it runs on. There is no hosted API: the server is yours, on your network, and nothing here reaches it. [`/openapi.json`](/openapi.json) describes it in OpenAPI 3.1, for tools that generate a client or for an agent calling it on your behalf. ## Authentication Without `PARLOUR_TOKEN` the server answers its own Mac only. With it set, send it as `Authorization: Bearer ` on every route but `/health`. Clients that cannot set a header can add `?token=` to the address instead. [The token](/docs/clients#the-token) says how to make one. ## Routes | Route | What it does | | --- | --- | | `GET /health` | Whether the server is up, what it runs, and what is in flight. No token needed. | | `POST /ask` | Text in, text out: `{"text", "client"?, "room"?}` returns `{"reply", "via"}`. What automations use. | | `POST /voice?client=&room=` | One recording in, `{"heard", "reply", "via", "audio"}` back, where `audio` is a base64 WAV. What the phone page uses. | | `POST /v1/chat/completions` | The OpenAI chat completions shape, streamed or not. How Home Assistant reaches Parlour. | | `GET /v1/models` | The one model, `parlour`. | | `GET /listen` | A WebSocket of raw 16 kHz PCM, for satellites and custom hardware. See [custom hardware](/docs/clients#custom-hardware). | | `/admin/*` | Pipeline settings, the model servers, doctor, logs and restart, for the household's own apps. See [managing the server](/docs/clients#managing-the-server). | ```sh curl -s http://localhost:8765/ask \ -H "authorization: Bearer $PARLOUR_TOKEN" \ -H "content-type: application/json" \ -d '{"text": "is the back door locked"}' ``` ## For agents This site is written to be read by agents as well as people, and none of it needs a sign-in. - [`/llms.txt`](/llms.txt) lists every page, and [`/llms-full.txt`](/llms-full.txt) is all of the docs in one file. - Every page has a markdown twin: add `.md` to its address, or ask for the page with `Accept: text/markdown`. - `https://heyparlour.app/mcp` is a read-only MCP server over these docs (streamable HTTP, no auth) with `search`, `list_pages` and `get_page`. Add it to Claude or ChatGPT as a custom connector. - [`/.well-known/agents.md`](/.well-known/agents.md) is a short guide for an agent acting for someone. # Architecture Source: https://heyparlour.app/docs/architecture > How Parlour's ports, providers, sessions and pipeline fit together, and why. This page shows you how Parlour is put together, so you can change it. One npm package, `parlour`, holds a pure core, the built-in providers behind its ports, the integrations, the network server and the CLI. The desktop app in `apps/desktop` drives the CLI and knows nothing else. ```text packages/parlour/src/ core/ ports.ts, providers.ts, builtins.ts, plugins.ts, skills.ts, config.ts, migrate.ts, paths.ts, secrets.ts, agent.ts, session.ts, audio.ts, endpoint.ts, text.ts, router.ts, queue.ts, triage.ts, tasks.ts, decision.ts, dispatch.ts, loop.ts, speaker.ts, registry.ts, search.ts, timers.ts, prompt.ts, types.ts, events.ts, logger.ts, process.ts, services.ts, localmodel.ts, version.ts providers/ audio/ffmpeg, audio/afplay, wake/openwakeword, stt/whisper-cpp, stt/yap, stt/parakeet-mlx, tts/kokoro, tts/macos-say, llm/openai-compatible, llm/anthropic, llm/ai-sdk, decision/laya-mlx, search/searxng, search/brave, secrets/macos-keychain, secrets/file, service/launchd integrations/ home-assistant/, mcp/, connectors/ server/ the HTTP and WebSocket server, discovery.ts, satellite.ts, web/ cli/ one file per command testing/ fakes for every port, published as parlour/testing ``` Nothing in `core/` imports from `providers/` except `core/builtins.ts`. That file only registers the built-ins. A program that embeds Parlour with its own providers can leave it out. ## Ports `core/ports.ts` declares the seams between core and anything that touches hardware, a model or the network. Each interface is small enough to fake in a test and to fill from an npm package. | Port | What it does | Built in | | --- | --- | --- | | `AudioSource` | Yields 16 kHz mono frames of 1280 samples (80 ms). | `ffmpeg` on avfoundation | | `AudioSink` | Plays a WAV, and can be stopped. | `afplay` | | `WakeWordEngine` | Loads once, hands out one `WakeWordDetector` per stream. | `openwakeword` | | `SpeechToText` | WAV in, text out. | `whisper-cpp`, `yap`, `parakeet-mlx` | | `TextToSpeech` | Text in, WAV out, with a `warm()` to load before the first reply. | `kokoro`, `macos-say` | | `ChatModel` | Messages and tool specs in, a completion out. | `openai-compatible`, `anthropic`, `ai-sdk` | | `DecisionModel` | A request and typed questions in, answers with probabilities out. | `laya-mlx` | | `SearchProvider` | A query in, results out. | `searxng`, `brave` | | `SecretStore` | Where connector tokens are kept. | `macos-keychain`, `file` | | `ServiceManager` | Keeps Parlour running at login. | `launchd` | | `Integration` | A named source of tools, prompt lines and a gate. | `home-assistant`, `mcp`, `connectors` | Every port except `SecretStore` and `ServiceManager` can also expose `doctor(): Promise`. `parlour doctor` joins every configured part's checks together and adds three of core's own: the config parses, the network is let in, and something keeps Parlour running. Core composes two things itself rather than leaving them to a provider: - **Speaking.** `Speaker` is a `TextToSpeech` plus an `AudioSink`, with sentence splitting and abort. It renders and plays the reply one sentence at a time, so the first words come out while the rest is still being made. `FallbackTextToSpeech` wraps two voices. When the first throws, it warns once, uses the second, and tries the first again on the next sentence. `tts.fallback` in config names the second voice. The default is `macos-say`, so Parlour still speaks while Kokoro is downloading. - **Timers.** This is the one tool that has to keep working when the network does not. So it lives in core and announces through the `Speaker`. ## Providers `core/providers.ts` is a registry keyed by `(kind, name)`. The kind is one of `audioSource`, `audioSink`, `wake`, `stt`, `tts`, `llm`, `decision`, `search`, `secrets`, `service` or `integration`. Built-ins call `registerProvider` when they are imported. You select each slot in config by name: ```json { "tts": { "provider": "kokoro", "voice": "bf_emma" } } ``` Core only reads the `provider` key of each slice. The rest (`voice`, `url`, `model` and so on) passes straight through. The provider's own Zod schema validates it when the provider is created. Core never needs to know that Kokoro has a voice or that whisper has a URL. A name that is not registered is tried as a package with `await import(name)`. Its default export is accepted when its `kind` matches, and a package of another kind is reported as exactly that rather than as a typo. That is the whole extension mechanism, and [Writing a provider](/docs/providers) walks you through it. You don't choose `secrets` and `service` in config. Core picks them by platform. It uses the Keychain on a Mac with `security` on the PATH, and otherwise a file per key under `connector-secrets/` in the config directory. It uses launchd on macOS. Elsewhere it throws an error saying that a systemd provider would be a welcome contribution. ## Assembly `buildAgent(config, secrets, paths)` in `core/agent.ts` does the wiring and hands back an `Agent`. It loads the plugins, resolves every provider and wires up the fallback voice. Then it builds the tool registry: every integration's tools, the search tool unless `search.provider` is `"none"`, the timers, and `read_skill` when the house has skills. Finally it builds the `Router`. The microphone loop, the text REPL, the doctor and the network server all take one `Agent` rather than building their own. That is why a timer set from a phone still sounds in the room, and why every client shares one tool list. Plugins load first, before any provider is resolved. A plugin (`core/plugins.ts`) is one package that can bring providers, skills and integration config at once. A provider it registers has to exist by the time a config slot names it. Its `integrations` block is merged underneath the one in `config.json`, key by key, so what you wrote wins. Skills (`core/skills.ts`) are markdown files under `~/.config/parlour/skills` and in any directories the plugins add. Only each skill's name and description go in the system prompt. The model fetches the body with the `read_skill` tool when it decides the rule applies. A local model with a small context cannot carry every house rule on every turn. [Skills, MCP and plugins](/docs/skills) is the guide. The house can run without a cloud model, decision triage or web search. A provider that cannot start (nearly always a missing key) is left out, and the doctor reports it. A name that leads nowhere is a config mistake and throws, so a typo cannot quietly leave you with the wrong model. ## The session `core/session.ts` is one state machine over a stream of 80 ms frames. It doesn't know where the frames come from. This machine's microphone, a satellite relaying audio, a phone or custom hardware all feed the same object. ```text idle ---wake word---> listening ---silence---> thinking ---answer---> speaking ---> idle ^ | | | nothing said | nothing heard | +-----------------------+------------------------+ ``` - **idle.** Every frame goes to the wake word detector. When it fires, core asks every integration's `gate()`. If any says true (the house is muted), the wake ends there. Otherwise anything being said is stopped and the session starts listening. - **listening.** Frames go to the endpointer in `core/endpoint.ts`. It gives up after 2.5 s of leading silence, finishes after `audio.silenceMs` (800 ms) of quiet once speech has started, and caps a request at `audio.maxUtteranceMs` (15 s). - **thinking.** The frames become a WAV and the WAV becomes text. Whisper's hallucinations on silence (`[BLANK_AUDIO]`, a lone "Thank you.") are dropped by `core/text.ts`, and the text goes to the router. - **speaking.** The sink says the answer. Frames that arrive while it speaks reset the wake word instead of being listened to, unless `audio.bargeIn` is on. With one box in one room, the microphone hears the speaker. Some clients already know where a request starts and stops, such as a phone with a button or a satellite that did its own wake word. They call `utterance(frames)` instead of `push(frame)` and skip the first two states. Every client has its own session, keyed by a client id, so a follow-up in the kitchen cannot pick up something asked in the study. The router forgets a session's history after four minutes of quiet, so "turn it off" cannot mean a light from an hour ago. The id is also the client's lane in the queue and its own wake word detector. So it decides both what a client can see and how much of the house it can hold up. The id comes off the network, so the server trims it to something short and printable. When two clients claim the same name at once, the server gives one of them an id of its own. A satellite that reconnects should pick its conversation back up. The kitchen and a guest's phone, both called "parlour", should not share one. ## The pipeline Every request runs through the same stages in `core/router.ts`, whichever way it came in. The answer goes back to the client that asked, and to nobody else. ```text ask -> queue -> triage -> tasks -> action agent -> one thing to say ``` ### The queue A house has more microphones than GPUs. When two satellites hear their wake word at once, both want the local model. Letting both in together makes each answer arrive later than taking turns would. The model is one process, and its batch is one request deep. `core/queue.ts` gives every client its own lane, which keeps satellites out of each other's way. One request per lane runs at a time, so a conversation stays in order and no client holds more than one slot. Lanes take turns as they finish, so the study cannot starve the kitchen however fast it asks. `pipeline.concurrency` caps how many requests run at once across the house. A lane that backs up past `pipeline.queueDepth` drops its own oldest waiting request, and nobody else's. Someone who asks twice wants the second answer, and hearing the first one first is worse than not hearing it. The client is told there is nothing to say rather than given a stale answer. ### Triage Speech recognition hands over a plausible sentence, not always a correct one. People say "it" and "in there" and expect the house to know. One breath is often two jobs. Give a small model all of that at once, with thirty tools in front of it, and it does the first job and claims the second. `core/triage.ts` reads the request before anything acts on it. It repairs the words and resolves pronouns from the conversation and the room. What is left becomes a list of at most `pipeline.maxTasks` tasks (4 by default). Each is written to stand on its own and marked as something to do, something to look up, or small talk. A question triage is sure needs the clever one is marked so. It skips the local round trip that would only have ended in `ask_the_clever_one`. Triage costs a round trip, and "lights off" does not need one. In the default `auto` mode, a request that is plainly one short instruction goes straight through. Anything compound, long or ambiguous gets the full pass. `always` buys the accuracy on every request, and `never` gives it up. Triage is an optimisation, never a gate. If the model answers with prose, with rubbish or not at all, the request stays exactly as it was said. ### The task list `core/tasks.ts` holds what the request turned out to be. The action agent takes one task at a time, in the order they were said. Each task sees what the tasks before it did, so "turn the lamp on and tell me if it worked" has something to work with. The replies are joined into one thing to say, and two jobs that both answer "Done." become one "Done." rather than a stutter. A task that fails abandons the rest, because you are owed one apology, not three. ### The action agent `core/dispatch.ts` is local first. The local model gets the system prompt, the tools and one extra tool, `ask_the_clever_one`. The prompt tells it to call that tool whenever a question needs real reasoning, current information or knowledge it is unsure of. When it does, the rewritten question and the session's history go to the cloud model, which has server-side web search and no house tools. If the local model throws or times out and `llm.cloud.onLocalFailure` is true, the cloud model gets the original question instead. Small talk gets no tools at all. That is faster, and it is the only reliable way to stop a small model calling one. With no cloud model configured, three things change together. The escalation tool is not offered, the persona stops naming it, and failures are final. A small model told to hand over, with nobody to hand over to, ends up apologising instead of turning the light off. Small models trained on the pattern sometimes ask for the clever one anyway. The loop then answers the call with "there is nobody to hand this to, use the tools you have" rather than an unknown-tool error, so the next round does the job. You can also add a decision model. `llm.decision` names an on-device model that scores each task before the local model takes its turn, and decides whether it should stay local or go to the cloud. `core/decision.ts` asks the questions and turns the answers into a plan. The built-in model is `laya-mlx` (`providers/decision/laya-mlx.ts`). It needs Apple silicon and `pip install laya-mlx`, and it stays warm in a local Python worker, so the transcript never leaves your Mac for the decision. The default is `provider: "none"`. - **`shadow` mode** (the default) only logs the scores beside whatever the local model did. - **`triage` mode** acts on them, when there is a cloud model to hand to. A `needs_cloud` or `needs_web` score at or above `escalateThreshold` (0.85) skips the local model and hands the task to the cloud. A house or timer intent at or above `localConfidence` (0.75) keeps the escalation tool off, so the answer stays on your Mac. Like triage, it is an optimisation. If the decision model fails, the task carries on locally. `core/loop.ts` runs the tool rounds for whichever model is answering, up to `llm.maxToolRounds`. It stops with an apology rather than looping forever. Every tool the model asked for in one round runs at once. The model asked for all of them before seeing any answers, so nothing in the round depends on anything else in it. Two lights cost the slower one, not the sum. `pipeline.timeoutMs` is the ceiling on a whole request, checked between rounds and between tasks. Whatever is left of a request that runs past it is abandoned, rather than spoken ten seconds after everyone has left the room. ## Configuration, secrets and paths | What | Where | | --- | --- | | `config.json`, `secrets.env`, `connectors.json`, `skills/` | `~/.config/parlour/`, or `$PARLOUR_HOME` | | Just the config file | `$PARLOUR_CONFIG`, or `parlour --config ` | | Models | `~/Library/Caches/parlour/models/{openwakeword,whisper,llm}` | | Logs | `~/Library/Logs/parlour/{agent,whisper,llm}.log` | | LaunchAgent labels | `io.parlour.agent`, `io.parlour.whisper`, `io.parlour.llm` | | Bonjour service type | `_parlour._tcp` | | Keychain service for connector tokens | `parlour-connector` | | The desktop app's own settings | `~/Library/Application Support/io.parlour.desktop/settings.json` | `config.json` is a Zod schema in which every key has a default. An empty file is a complete config, and `parlour config show` prints what is in force. Secrets (`HA_TOKEN`, `ANTHROPIC_API_KEY`, `PARLOUR_TOKEN`, `BRAVE_API_KEY`, `LOG_LEVEL`) stay in `secrets.env` with mode 600. The process environment wins over the file, so you can override something for a one-off run without editing anything. On Linux, the cache and logs follow `XDG_CACHE_HOME` and `XDG_STATE_HOME`. Nothing else about Linux is done yet, and we would be glad of your help. ## Events `parlour start --events` prints one JSON line per state change on stdout: `ready` (with the tool count and whether a cloud model is up), `state`, `heard`, `reply` (with `via` local or cloud and the time taken in `ms`), `muted` and `error`, each with an `at` timestamp. The desktop app reads these to draw its status instead of scraping log text. Without the flag, stdout is a log for people to read. ## Why each part was chosen | Stage | Choice | Why | | --- | --- | --- | | Wake word | openWakeWord, ONNX, in process | Free and offline. It runs in the same Node process as everything else, which makes barge-in possible. | | Capture | ffmpeg on avfoundation | Nothing to compile, and the only reliable way to pin one input device on macOS. | | Speech to text | whisper.cpp `small.en`, kept warm | Loading the model costs more than transcribing a sentence, so it runs as a server. | | Model | llama.cpp with a Qwen3.5 or Gemma 4 GGUF, kept warm | It comes with Parlour, so the house answers straight after `init`, with no LM Studio to install and no server to remember to start. It speaks the OpenAI API, so LM Studio, Ollama or anything else can take its place. | | Escalation | Claude with server-side web search | The local model is fast and private but wrong more often. Handing over is cheap. Being wrong out loud is not. | | Speech | Kokoro 82M, ONNX | Close to a cloud voice, runs in process, and has British voices. `say` is the fallback when it fails. | Every stage adds to the wait before an answer, so each one is chosen to be fast rather than best. # Clients Source: https://heyparlour.app/docs/clients > Connect satellites, phones, custom hardware and scripts to your server. One machine in your house runs the models, holds the tokens and answers. Everything else with a microphone is a client. It sends what it hears, plays back what it is sent, and runs no models of its own. You never need to give a client an address. The server advertises itself with Bonjour and the clients find it. ```text finds it with Bonjour kitchen satellite ->\ _parlour._tcp study satellite ->\ Voice PE, via HA -> the server -> tools, models, connectors, the house phone, push to talk ->/ custom hardware ->/ its own microphone ->/ ``` The server's own microphone is just another client, with no special privileges. Every client keeps its own conversation, so a follow-up in the kitchen cannot pick up something asked in the study. Every client can also say which room it is in. ## The token Every client needs `PARLOUR_TOKEN`. `parlour init` offers to generate one for you, or you can set it by hand with `parlour secrets set PARLOUR_TOKEN`. It reads the value from stdin, so it never ends up in your shell history: ```sh openssl rand -hex 24 | parlour secrets set PARLOUR_TOKEN parlour service restart ``` Without a token, the server listens only on loopback, answers only the machine it runs on, and does not advertise itself. An open voice assistant can turn on the heating and read your shopping list, so closed is the safe default. A browser on that same machine is held to it too: a request addressed to anything but a loopback name (`localhost`, `127.0.0.1`, `::1`), or sent by a page that was not served from one, is refused. `parlour doctor` tells you which state you are in. Clients send the token as `Authorization: Bearer `, or as `?token=` on the URL, which is how the socket takes it. `/health` needs no token and reports what is configured and what is in flight: `{ "ok": true, "tools": 14, "cloud": true, "running": 0, "waiting": 0 }`. ## A satellite Any Mac with a microphone can be a satellite. All it needs is Node, ffmpeg and the `parlour` package. No models, no keys, no GPU, nothing to keep warm. ```sh npm install -g parlour parlour init # answer "satellite", name the room, paste the token ``` That writes `role: "satellite"` and installs the service (a satellite has no app to own it). From then on, it finds the server by name and reconnects for as long as it is switched on. While the server is down, it waits longer between attempts, doubling up to `satellite.retryMs`. ```json { "role": "satellite", "satellite": { "room": "kitchen", "serverUrl": "", "localWake": false, "retryMs": 15000 } } ``` - **`serverUrl` empty** means "find it with Bonjour, once". `parlour init` looks while you watch. A satellite that starts with the key empty keeps looking until a server lets it in. Either way, the address it finds is written here, and the satellite talks only to that server from then on. This is deliberate. Anything on the network can advertise `_parlour._tcp`, and the satellite hands its token to whatever it connects to, so the first server it trusts is the only one it trusts. If your server really moves, clear the key. If the pinned server is down and something else is advertising, the log says so and the satellite stays put. You can also fill in the key by hand (`http://study-mac.local:8765`) if your network drops multicast, as some mesh systems and most guest VLANs do. Run `parlour doctor` on the satellite to see whether it can find a server, and whether macOS is letting it onto the local network at all. - **`localWake`** runs the wake word on the satellite and streams only what follows it. It saves a constant 32 KB/s on the network, but needs a copy of the models on that machine (`parlour models fetch` puts them there). It is off by default, so there is one less copy of the models to keep up to date. The satellite plays the server's speech, so the voice is the same in every room and you change it in one place. ## Home Assistant satellites This is the cheapest client, because the hardware is already in your house. Home Assistant keeps handling the wake word, speech to text and the spoken reply. Only the thinking moves to Parlour. The full setup is in [Home Assistant](/docs/home-assistant). In short, point the OpenAI Conversation integration at `http://:8765/v1`, with model `parlour` and your token as the API key. ## The iPhone app The Parlour app talks to `/health`, `/voice` and `/ask` on the server. Rather than typing the address and a 48 character token on a phone, run this on the server: ```sh parlour pair ``` It draws a QR code in the terminal. In the app, open Settings and tap Scan pairing code, or point the Camera at it; either way the app takes the address and the token together and checks the server there and then. The desktop app shows the same code under On the network, behind Pair a phone. The code carries the token, so show it only to phones you mean to let in. It uses the Mac's `.local` name; if the phone cannot reach the Mac by that name, give it an address it can: `parlour pair --host 192.168.1.20`. `parlour pair` runs on the server only, and refuses until `PARLOUR_TOKEN` is set, because without one a phone could not get in anyway. ## A phone Open `http://:8765` and add it to your home screen. Hold the button, say something, let go. The page's settings take the token and, optionally, a room. A link with `?token=...&room=kitchen` fills in both, so you do not have to type them on a phone keyboard. Safari only grants microphone access over HTTPS or on localhost. On iOS, put a reverse proxy with a certificate, or Tailscale, in front of the server. Under the hood, the page posts one recording to `POST /voice?client=&room=kitchen` and gets back `{ heard, reply, via, audio }`, where `audio` is a base64 WAV of the reply, or `null` when nothing was heard. Each phone makes up its own `client` id, so each keeps its own conversation; leave it out and the client is `phone`. Anything that can record and post can use the same route. ## Custom hardware ```text ws://:8765/listen?client=hallway&room=hallway&token=... ``` Send 16 kHz mono signed 16-bit PCM as binary frames of any size. The server re-cuts them into the 80 ms frames the wake word needs. It replies with JSON events and one binary WAV per answer. | From the server | Meaning | | --- | --- | | `{"type":"ready","sampleRate":16000,"frameSamples":1280,"mode":"wake"}` | Connected. | | `{"type":"state","value":"listening"}` | Also `thinking` (with `"text"`, what was heard), `speaking` and `idle`. | | `{"type":"reply","text":"...","via":"local"}` | Followed by the WAV as a binary frame. | | `{"type":"stop"}` | Stop playing: a new wake word arrived. | - **Wake mode**, the default. The server runs the same openWakeWord and endpointing as its own microphone, so your device can be just a microphone, a speaker and a network stack. - **Push mode** (`&mode=push`). Your device marks the start and end of a request with `{"type":"start"}` and `{"type":"end"}`, or sends `{"type":"cancel"}` to throw it away. Use it for a button, or for hardware that does its own wake word. A request that never ends is answered once it reaches `audio.maxUtteranceMs`. Send `{"type":"spoke"}` when playback finishes, and the server starts listening again straight away instead of waiting out its own estimate. `client` names the conversation, so a device that reconnects picks up where it left off. Two devices connected under the same name at once are told apart (`hallway~2`) rather than made to share one. ## Automations and scripts ```sh curl -s http://:8765/ask \ -H "authorization: Bearer $PARLOUR_TOKEN" \ -H "content-type: application/json" \ -d '{"text": "is the washing machine finished", "room": "kitchen"}' ``` The content type is required. A form on a web page can post `text/plain` anywhere without the browser asking first, but it cannot claim to be JSON. You get back `{"reply": "...", "via": "local"}`, where `via` says which model answered. Add a `client` to the body for a conversation of your own. Otherwise, every caller shares the `api` session. ## Managing the server A client with the token can also manage the server: change the pipeline's settings, start, stop and restart the model servers it keeps warm, and run the maintenance commands. The iPhone app's Server tab and `parlour remote` both use these routes. Set `server.admin: false` to turn them off. | Route | What it does | | --- | --- | | `GET /admin` | The services, the pipeline in force and the one saved, the memory, and whether the agent comes back by itself after a restart. | | `POST /admin/pipeline` | Saves a change to any of `concurrency`, `queueDepth`, `triage`, `maxTasks` and `timeoutMs`, then restarts the agent so it takes effect. Add `?apply=false` to save it for later. | | `POST /admin/service` | `{"service": "llm" \| "whisper", "action": "start" \| "stop" \| "restart"}`. | | `POST /admin/doctor` | The server's own checks, `parlour doctor` over the network. | | `GET /admin/logs?service=agent&lines=50` | The end of one log: `agent`, `llm` or `whisper`, up to 200 lines. | | `POST /admin/restart` | Restarts the agent. | Some limits are there to protect the Mac, so the server can refuse a request even with the right token. It answers `{"error": "..."}` with a sentence saying why: - The pipeline's numbers have tighter bounds than the config file allows. For example, at most four requests at once, and a timeout between 5 seconds and 5 minutes. Every request running at once is another generation the local model has to hold in memory. - A model whose weights would take more than 70% of the Mac's memory is not started. The doctor warns you above 50%. - The server takes one change at a time. It also refuses to repeat the same action on the same thing within 15 seconds, so a script stuck in a loop cannot restart the model over and over. - The agent restarts by exiting and letting launchd, or the menu bar app, start it again. If you started it in a terminal, nothing would bring it back, so a restart is refused and a pipeline change is saved until the next start. ## Bonjour The server advertises `_parlour._tcp` with a TXT record of `role=server`, `version=1`, `token=required` and `api=/v1`. A server without a token does not advertise at all. The name is `discovery.name`, or `" on "` when that is empty. Set `discovery.enabled: false` to turn it off. Every satellite then needs `serverUrl` filled in. ```sh dns-sd -B _parlour._tcp # what is advertising, from any Mac on the network ``` # The commands Source: https://heyparlour.app/docs/commands > Every parlour command, and what each one is for. ```text parlour init [--yes] [--no-deps] [--no-service] [--porcelain] dependencies, models, config, secrets, service or app parlour start [--events] the server or the satellite, per config.role parlour stop stop it, until the next login parlour restart after editing config.json or a secret parlour text everything but the microphone parlour try mic|wake|stt|llm|cloud|tts|ask [--json] one stage of the pipeline, tried on its own parlour doctor [--json] which of the moving parts is down parlour service install|uninstall|start|stop|restart|status|logs [agent|llm|whisper] parlour remote status|pipeline [set key=value...]|service |doctor|logs|restart parlour connectors add |list|remove parlour mcp add --url |-- ...|list|remove parlour skills list|show |new |write |remove |path parlour plugins add |list|remove parlour models fetch [--llm auto]|suggest parlour config path|show|write|edit parlour secrets status|set parlour pair [--host ] [--json] a code for the iPhone app to scan: address and token ``` `parlour --help` shows the flags for each one. `parlour service start`, `stop`, `restart` and `logs` take one service by name (`agent`, `llm` for the local model, or `whisper`), or all of them without one. `parlour remote` manages a server from another machine, or from the same one without editing a file. It uses the same routes as the iPhone app's Server tab (see [Managing the server](/docs/clients#managing-the-server)). It talks to `--url`, or the server this machine pinned as a satellite (`satellite.serverUrl`), or this Mac. It never picks a server found with Bonjour, because it sends `PARLOUR_TOKEN` (from this machine's secrets, or `--token`) with every call, and anything on the network can advertise itself as Parlour. ```sh parlour remote status parlour remote pipeline set triage=always timeoutMs=30000 # saved, and the agent restarts parlour remote service llm restart parlour remote logs llm --lines 100 ``` `parlour connectors add` signs your household in to a remote MCP server (a calendar, say) and keeps the tokens in your Keychain. The `connectors` integration that loads them is on by default, and if a hand-written config leaves it out, `parlour doctor` will let you know. # The design system Source: https://heyparlour.app/docs/design > One palette, one type ramp, one set of states and one icon, shared by all four interfaces. Parlour has four interfaces: this site, the menu bar app, the phone page the CLI serves, and the iOS app. They use three languages and three toolchains, and none of them declares a colour of its own. `packages/design` holds one file of tokens and a few emitters that write it into every surface. The generated files are checked in, so a checkout builds without running the emitter first. A test fails when any of them has drifted, so `pnpm check` catches a token change that was not emitted. ```sh pnpm exec nx run design:emit # after editing packages/design/src/tokens.ts pnpm exec nx run design:icons # after changing the mark, or a colour it uses pnpm exec nx run design:test # contrast, and whether anything is stale ``` | Generated file | Read by | | --- | --- | | `apps/site/app/tokens.css` | This site | | `apps/desktop/src/tokens.css` | The menu bar app, which maps it onto shadcn's token names | | `packages/parlour/src/server/web/tokens.css` | The phone page | | `apps/ios/Parlour/DesignSystem/Tokens.swift` | The iOS app | | `apps/ios/.../AccentColor.colorset/Contents.json` | iOS controls the app never styles, in hearth green | | `apps/ios/.../AppIcon.appiconset/Contents.json` | The iOS app icon set | | `apps/site/app/icon.svg`, `packages/parlour/src/server/web/icon.svg` | The favicon | ## Ink on a limewashed wall, with one lit thing Paper, ink, bracken and a hairline rule carry the whole interface. Only two colours are allowed to mean something, and nothing else is colourful. That is what makes those two readable at a glance from across a room. | Token | What it means | | --- | --- | | `paper`, `surface` | The wall, and a panel lifted off it by a shade rather than a shadow | | `ink`, `bracken` | What is said, and what is said quietly | | `rule`, `inset` | A hairline, and anything recessed into the wall | | `hearth` | The house is alive: running, listening, speaking, the primary action | | `lamp` | The one lit thing: the wake word, the page you are on, thinking, a warning | | `alarm` | Something failed and a person needs to act | Amber on a pale wall cannot carry text. So `lamp` is only ever a dot or a halo, and `lampText` is the darker value used for words. Every token that carries text clears 4.5:1 against its paper in both schemes, and a test checks it. ## The states are part of the design system Every surface shows the same five states, so they live with the colours rather than in each app. Each state has a colour, a cadence and a word. | State | Drawn as | One breath | | --- | --- | --- | | `stopped` | An outline in bracken | still | | `idle` | A steady hearth mark, labelled "ready" | still | | `listening` | A lit hearth mark | 1100 ms | | `thinking` | A lit lamp mark | 700 ms | | `speaking` | A lit hearth mark | 500 ms | The cadence quickens as the turn nears its answer, so you can tell what the house is doing without reading a word. The menu bar dot, the phone page's talk button and the iOS state mark breathe in step, because they read the same numbers from the same file. Under Reduce Motion the iOS mark stops breathing and the colour carries the state alone. ## The icons The house mark (a roof, the walls, a dado rail and one lit lamp in the gable) is drawn once in `src/icon.ts` from the palette, in four shapes: the favicon tile, a full bleed square for iOS and the phone page's home screen, the macOS tile on Apple's grid, and a bare template for the menu bar. `design:icons` rasterises them into every bitmap a platform wants: the site's favicon, the phone page's home screen icons, the iOS app icon, and the menu bar app's `icon.png` and `tray.png`. The PNGs are binary, so the drift test cannot check them. Regenerate them and look. ## Changing something Edit `packages/design/src/tokens.ts`, run the emitter, and commit the generated files with your change. A new surface gets a `Target` in `src/emit.ts` and a line in the list in `emit.test.ts`. If a surface needs a format none of the emitters produce, add another emitter next to `css.ts` and `swift.ts` instead of keeping a hand-maintained copy of the values. # The menu bar app Source: https://heyparlour.app/docs/desktop > What the app does, how it drives the CLI, and how to run it from a checkout. The menu bar app handles the everyday things without a terminal: start, stop, is it listening, what did it just hear, why is it not answering. It is called Parlour Server, to tell it apart from Hey Parlour, the iPhone app that talks to it. `apps/desktop` is a Tauri app, a Rust shell around the `parlour` command with a window as its only interface. It is not a second implementation. It learns everything by running the CLI, and writes every setting through `parlour config write`, so the app and the terminal always see the same configuration. ## Building it You need Node 22, pnpm and a Rust toolchain (`rustup`). ```sh pnpm install pnpm -C apps/desktop app # the window from the checkout, with live reload pnpm -C apps/desktop build # Parlour Server.app and a dmg, in apps/desktop/src-tauri/target/release/bundle ``` Live reload covers the window. The CLI it drives is whichever `parlour` the app's settings point at. On a fresh machine that is the globally installed one, not the checkout. See [Running the app against a checkout](#running-the-app-against-a-checkout). A build from your checkout is unsigned, which is fine on the machine that made it. Copy `Parlour Server.app` into `/Applications` and right-click to open it the first time. A release dmg is signed with a Developer ID certificate and notarised, so it opens normally on any Mac with Apple silicon, which is what it is built for. Each release is on [GitHub](https://github.com/mrprkr/parlour/releases/latest), and the newest dmg is always at [`releases/latest/download/Parlour-Server-arm64.dmg`](https://github.com/mrprkr/parlour/releases/latest/download/Parlour-Server-arm64.dmg). The app does not bundle Node or the package. It looks for `parlour` on your login shell's PATH, so nvm, fnm and volta installs are found, then in `/opt/homebrew/bin`, `/usr/local/bin` and `~/.npm-global/bin`. If it cannot find one, it offers to run `npm install -g parlour` at its own version, which needs Node 22 or newer. One copy of Parlour on the machine is better than two that can disagree. ## Signing Notarising requires the hardened runtime, which blocks the microphone unless the app asks for it. `src-tauri/Entitlements.plist` asks, and `Info.plist` next to it holds the sentence macOS shows the user. ffmpeg does the recording, but macOS attributes the request to the app that started it, so both files belong to the app. The desktop release workflow sets the Apple environment variables Tauri reads. Tauri creates its own keychain for the certificate, signs the app, notarises `Parlour Server.app` and staples the ticket into it. It signs the dmg but does not notarise it, and macOS refuses a disk image without its own ticket. So `scripts/notarise.sh`, at the root of the repository, notarises and staples the dmg, then asks Gatekeeper to assess both and checks the app's signature carries the hardened runtime and the microphone entitlement. Run on a machine with no Apple credentials, it says so and stops. [`.github/workflows/desktop-release.yml`](https://github.com/mrprkr/parlour/blob/main/.github/workflows/desktop-release.yml) lists the six repository secrets behind it and what each one is. ## Setting up On a machine that is not set up yet, the app opens on setup instead of the tabs, and walks you through it one step at a time. Anything optional stays folded away until you ask for it. **Run setup again** in Settings brings it back later, skipping the welcome. 1. **Install.** One button installs `parlour` from npm if it is missing, then runs `parlour init --porcelain --yes` for the rest (ffmpeg and whisper from Homebrew, the models, the config). The log stays folded away unless you ask for it or something fails. **Already have parlour somewhere else?** takes a path instead. 2. **Voice.** Allow the microphone, choose the wake word, and optionally a different microphone. 3. **Home.** The Home Assistant address and token. Skip it and timers, questions and search still work. 4. **Extras.** Cloud help (the Anthropic key) and letting other devices in with an access token, each behind its own switch. 5. **Finish.** The doctor's checks, and a button to start listening. ## The tabs The header carries a dot that follows Parlour through idle, listening, thinking and speaking, and a Start or Stop button. | Tab | What you will find there | | --- | --- | | Status | The last thing it heard, the last thing it said and which model said it. How many tools it has and whether cloud escalation is on. The address phones and Home Assistant should use, and **Pair a phone**, which shows the QR code the [iOS app](/docs/ios) scans. A Check button that runs `parlour doctor --json`. | | Connectors | What your household has signed in to, and a form to add another. `parlour connectors`, with buttons. | | Settings | Where `parlour` is, whether to start listening when the app opens, wake word, sensitivity, microphone, silence timeout, voice, the local and cloud models, the Home Assistant address and the secrets, including the access token. It reads `parlour config show --raw` (the file as written, so saving does not freeze every default into it) and writes through `parlour config write` and `parlour secrets set`. | | Logs | Parlour's output, live. | The tray menu has Open, Start listening, Stop and Quit. Quit asks first, because it stops more than the app. The agent and the services it keeps running (whisper and the local model) are stopped too, so they do not keep holding memory for a server nobody can reach. They start again when you open the app, or at the next login if they are installed as services. When a client restarts the agent to apply a setting, the app starts it again itself, as launchd would. ## How it fits together ```text src/ App.tsx the header, the tabs, and the state the panels read panels/ one file per tab, plus the setup overlay and its parts in onboarding/ lib/bridge.ts every Rust command, typed. Nothing else calls invoke() components/ui/ shadcn primitives | invoke() src-tauri/src/ main.rs the tray, the window, one generic parlour(args, stdin) command, the mic check supervisor.rs parlour start --events as a child process, and its NDJSON events setup.rs parlour init --porcelain, and npm install -g parlour, streamed settings.rs the app's own two facts: where parlour is, and whether to start it ``` Apart from running ffmpeg to list microphones and raise the permission prompt, the Rust side knows the path to the `parlour` binary and nothing else. The window asks for config, secrets, the doctor and the connectors through the CLI's `--json` and stdin interfaces. The supervisor reads the NDJSON events that `parlour start --events` prints (`ready`, `state`, `heard`, `reply`, `muted`, `error`) instead of scraping log text. Anything else it prints is log output, and that is where a failure to start shows up. Stopping sends SIGINT, not SIGKILL. Parlour shuts ffmpeg down on SIGINT, and an orphaned ffmpeg would keep hold of the microphone. The app's own settings live at `~/Library/Application Support/io.parlour.desktop/settings.json`: `{ "parlourBin": "/opt/homebrew/bin/parlour", "autostart": false }`. ## Running the app against a checkout `pnpm -C apps/desktop app` reloads the window from the checkout, but every CLI call goes to the `parlourBin` in `settings.json`. The app fills that in by asking your login shell for `parlour`. So the app does not exercise your changes to `packages/parlour` until you point it there. `packages/parlour/bin/parlour-dev` runs the CLI from source. Point the app at it in either of two ways: - Put its absolute path in **The parlour command** on the Settings tab (or under **Already have parlour somewhere else?** on the Install step of setup). This takes effect at once. - Write it to the file and restart the app, which reads the file once at launch: ```json { "parlourBin": "/path/to/parlour/packages/parlour/bin/parlour-dev", "autostart": false } ``` The app keeps the path while it is executable and will not swap in the global copy. The dev build and the installed `Parlour Server.app` share this settings file, so when you are done, put the global path back, or delete the file and let the app find `parlour` again. ## Three things to know - **Use the app or the LaunchAgent, not both.** Two copies of Parlour means two processes fighting over one microphone and one port. `parlour init` asks which you want and sets up only that one. When the app runs `init`, it leaves the agent to the app, and warns if a LaunchAgent is already installed. `parlour service status` tells you whether a LaunchAgent exists, and `parlour service uninstall` removes it. - **Microphone permission belongs to whatever starts Parlour.** macOS grants it to the process it holds responsible, which for anything the app spawns is the app. Setup opens the device for a moment as early as it can, which brings up the system prompt. The bundle carries the `NSMicrophoneUsageDescription` the prompt needs. Without it, macOS refuses instead of asking. The grant is tied to the bundle identifier, so it survives rebuilds, and it covers Parlour as the app's child. A LaunchAgent needs its own grant and asks the first time it runs. - **So does the local network.** Since macOS 15, a process has to be allowed onto the local network before phones, satellites and Home Assistant can find or reach it, and until then Bonjour fails with "no route to host". The doctor sends one Bonjour question, which brings up the prompt during setup rather than when the phone first looks, and fails with the way to the switch if it was refused: System Settings, Privacy & Security, Local Network. The bundle carries `NSLocalNetworkUsageDescription` for it. The Mac has no multicast entitlement to add: Apple's is for iOS, and on macOS the permission is the whole story. ## Deliberate choices - **No Tauri plugins.** Everything the app does happens in Rust, where it is easy to reason about. The window's capability list is just `core:default`. - **A true menu bar app.** The activation policy is Accessory, so there is no Dock icon and no menu bar of its own. The tray is the way back to the window. Closing the window leaves Parlour running. Quitting from the tray asks, then stops the agent and everything it keeps warm. - **Generated icons.** The mark is drawn once, in `packages/design`, and `pnpm exec nx run design:icons` writes `src-tauri/icon.png` and the menu bar's `src-tauri/tray.png` from it (see [the design system](/docs/design)). `pnpm icons` derives every other size into `src-tauri/icons/`, which is not in git, and both `app` and `build` run it first. The source sits outside that directory on purpose. `tauri icon` writes an `icon.png` into its output, so a source kept there would overwrite itself on every build and leave the working tree dirty. # Getting started Source: https://heyparlour.app/docs/getting-started > Install Parlour on a Mac, answer a few questions, and talk to it tonight. You will need a Mac with Node 22 or later and [Homebrew](https://brew.sh). `init` fetches the rest for you (ffmpeg, whisper.cpp and the models) through `brew`. ```sh npm install -g parlour parlour init parlour text # have a conversation in the terminal, no microphone needed parlour start # the real thing, unless init already set it to run at login ``` Rather not use a terminal? [The menu bar app](/docs/desktop) does the same setup in a window. ## What `init` asks `init` walks you through it with arrow keys and a handful of questions: what this machine is, which local model to run, your Home Assistant, an Anthropic key if you would like the cloud behind it, and preferences like the wake word and the voice. Escape goes back a question, and nothing is downloaded or written until a last screen has shown every answer and you have said yes. It writes `~/.config/parlour/config.json` and `secrets.env`, offers to run Parlour at login, and finishes by running `parlour doctor` so you know everything is in place. Changed your mind later? Run `init` again: every answer defaults to what is already set. ## The local model comes with it `init` looks at how much memory the Mac has, suggests the largest model it can hold comfortably, downloads it, and keeps [llama.cpp](https://github.com/ggml-org/llama.cpp) serving it at login alongside whisper. Nothing to install by hand and nothing to remember to start. `parlour models suggest` shows the catalogue and which one it would pick. If you would rather run the model yourself, say so and point `llm.local.baseUrl` at whatever you have: [LM Studio](https://lmstudio.ai) (`init` still offers to install it), Ollama, or a box in the cupboard. Anything that speaks the OpenAI chat completions API will do. ## Home Assistant Home Assistant gets its own part of `init`: it looks for your Home Assistant on the network, tells you exactly where to make a long lived token, checks the token as you paste it, and checks the MCP Server integration is switched on. Each of those fails in the same silent way, so each one is checked as it is answered rather than all three landing in `parlour doctor` half an hour later. Say no and the house is left out entirely. [Home Assistant](/docs/home-assistant) has the rest. ## The cloud is optional Leave the Anthropic key empty and Parlour runs local only: the local model keeps every tool, answers everything itself, and not a word leaves the house. ## Next - [The commands](/docs/commands), for everything the CLI can do. - [Skills, MCP and plugins](/docs/skills), to teach it your house and give it more to do. - [Clients](/docs/clients) and [the iOS app](/docs/ios), to reach it from other rooms. - [Tuning](/docs/tuning), when it wakes for the television. # Home Assistant Source: https://heyparlour.app/docs/home-assistant > Control your house through Home Assistant, reach its Assist, use Parlour as its voice, and mute it. Home Assistant gives Parlour tools for your house, a line in the system prompt, and a mute switch. It is on by default and lives under `integrations["home-assistant"]` in your config: ```json { "integrations": { "home-assistant": { "url": "http://homeassistant.local:8123", "mcp": true, "rest": true, "assist": false, "language": "", "muteEntity": "" } } } ``` The token is `HA_TOKEN` in `secrets.env`. It is a long-lived access token from your profile page in Home Assistant, at the bottom of the Security tab. That token controls the whole house, so it stays in `secrets.env` with mode 600 and never goes into git. ## Set it up `parlour init` has a section just for Home Assistant. Four things can go wrong, and they all look the same from the outside: the wrong address, no token, a token for something else, or the MCP Server integration not yet added. So `init` checks each answer as you give it: 1. **The address.** It tries the one in your config, then `homeassistant.local:8123`, `homeassistant:8123` and `localhost:8123`. It takes the first that answers the way Home Assistant does: `401` from `/api/` without a token, which a router's login page does not. If nothing is found, you type it in, and a wrong one is not quietly accepted. 2. **The token.** It prints the page to make one on (`/profile/security`), reads it without echoing it, and checks it against `/api/` before moving on. You get three tries, because a pasted token is easy to truncate. 3. **The MCP Server integration.** If `/mcp_server/sse` does not answer, it tells you where to add it and offers to look again once you have. 4. **The mute entity**, if you want one. Say no to the first question and the house is left out of the config, so `parlour doctor` does not fail on something you do not have. Run `init` again whenever you change your mind. Every answer defaults to what is already set, and a token that still works is kept rather than asked for twice. To set just the token, `parlour secrets set HA_TOKEN` reads one from stdin. ## What Parlour can do with it Tools come from three places, and the model sees them as one list: - **Over MCP** (`mcp: true`). Home Assistant's **Model Context Protocol Server** integration publishes everything exposed under **Settings > Voice assistants > Expose** as tools, and Parlour inherits them all. Expose more and Parlour can do more, with nothing to change on the Parlour side. Add the integration under **Settings > Devices & services**. It has no options. If Parlour cannot see a light, expose the light. - **Over REST** (`rest: true`). Two general tools cover what the MCP tools cannot: `ha_get_state` reads any entity, and `ha_call_service` calls any service. Its description tells the model to use it only when no more specific tool fits. - **Through Assist** (`assist: true`, off by default). `ha_assist` passes a command, word for word, to [Assist](https://www.home-assistant.io/voice_control/), Home Assistant's own voice assistant, and returns what it said. See below. Parlour connects to the MCP endpoint at `/mcp_server/sse` with the token. If that connection fails, the failure is logged and the REST tools still reach the house. ## Assist Assist is Home Assistant's built-in conversation agent. It is off by default, because the MCP tools are the better way to control the house: the model sees each entity and action and picks the right one, where Assist has to match the words. Turn it on when you have set up Assist for things MCP cannot reach, and it keeps working when Parlour is the one listening: ```json { "integrations": { "home-assistant": { "assist": true } } } ``` - **Custom sentences.** An automation with a sentence trigger, such as "good night" or "movie time", only fires when Assist hears the sentence. Parlour's own microphone, satellites and phones hear it instead, so the model passes it on through `ha_assist` and the automation runs. - **Its languages.** Assist parses commands in over 50 languages. `language` picks one, such as `en` or `de`; empty uses the default set in Home Assistant under **Settings > Voice assistants**. - **What it answers.** Whatever Assist says back, such as "Turned on the lights" or a sentence trigger's own response, becomes the tool's result. If Assist cannot handle the command, the model is told why and can try another tool. Parlour always asks the built-in agent, `conversation.home_assistant`, not the one your pipeline uses. So a house that also uses Parlour as its conversation agent (below) cannot send a command round in a circle. Assist only sees entities exposed under **Settings > Voice assistants > Expose**, the same list the MCP Server integration publishes. To make an announcement on a Home Assistant voice satellite, such as a Voice PE, the model can call `assist_satellite.announce` with `ha_call_service`, with a `message` in its data. That needs only the REST tools, not `assist`. ## Home Assistant as a client It works the other way round too. Home Assistant, and every Voice PE satellite through it, can use Parlour as its conversation agent. Home Assistant keeps handling the wake word, speech to text and the spoken reply. Only the thinking moves. 1. **Settings > Devices & services > Add integration > OpenAI Conversation**. 2. Base URL `http://:8765/v1`, and your `PARLOUR_TOKEN` as the API key. 3. Model `parlour`. The prompt and temperature on that page are ignored, because Parlour brings its own persona and tools. 4. **Settings > Voice assistants**, and set the pipeline's conversation agent to it. Your satellites then use the local model for house control and hand harder questions to the cloud, just as the server's own microphone does. Each Home Assistant user gets a conversation of their own. For an automation that wants a text answer, call `POST /ask` through a `rest_command`, with the token kept in `secrets.yaml`: ```yaml rest_command: ask_parlour: url: http://:8765/ask method: POST headers: authorization: !secret parlour_token content_type: application/json payload: '{"text": "{{ text }}", "room": "{{ room }}"}' ``` Here, `parlour_token` in `secrets.yaml` is `Bearer `. The reply is `{"reply": "...", "via": "local"}`. ## Muting `muteEntity` names an entity that makes Parlour ignore its wake word while it is `on`. Parlour checks it over REST after the wake word fires and before it acts on anything. So the house enforces the mute, not the machine running Parlour, and it covers every wake word the server hears: its own microphone, satellites and `/listen` devices in wake mode. Requests that arrive without the server's wake word (the phone page, push mode, a satellite with `localWake` on, `/ask` and `/v1`) are not muted. Empty, the default, means no mute. ```json { "integrations": { "home-assistant": { "muteEntity": "input_boolean.parlour_muted" } } } ``` To follow a sleeping mode, define the boolean and an automation that tracks it: ```yaml input_boolean: parlour_muted: name: Parlour muted icon: mdi:microphone-off automation: - id: parlour_mute_follows_sleeping alias: Parlour mute follows sleeping triggers: - trigger: state entity_id: input_boolean.sleeping actions: - action: "input_boolean.turn_{{ trigger.to_state.state }}" target: entity_id: input_boolean.parlour_muted ``` You can still override it by hand for one night from the dashboard. If Home Assistant cannot be reached, the mute reads as off. Parlour should still hear you, and the model will report the outage when you ask. A muted wake emits a `muted` event on the `--events` stream, which is how the desktop app can tell you why nothing happened. ## What the doctor checks ```text ok Home Assistant token HA_TOKEN is set ok Home Assistant http://homeassistant.local:8123 FAIL Home Assistant MCP http://homeassistant.local:8123/mcp_server/sse did not answer. Add the Model Context Protocol Server integration in Home Assistant. ``` A failure on the second line usually means the token. A failure on the third, with the second fine, means the integration is not installed yet. # Docs Source: https://heyparlour.app/docs > Everything you need to set up Parlour, make it yours and build on it. Want it working tonight? Start with Getting started, then Home Assistant. Curious how the pieces fit? Start with Architecture. When you want it to listen and answer just the way you like, come back to Tuning. # The iOS app Source: https://heyparlour.app/docs/ios > A client in your pocket, with HomeKit, QR pairing, local discovery and a model on the phone. The iOS app puts Parlour on the phone you already carry. It records, sends the recording home and plays the answer back, just as a satellite does. Everything that thinks, holds a key or touches the house still runs on the Mac at home. The phone adds HomeKit, which it knows about without any setup, and a small model of its own for when the house cannot be reached. The app has three tabs: **Talk**, where you hold the button and speak; **House**, your HomeKit rooms and accessories; and **Settings**, which server, which room and what the app is allowed to do. The app lives in `apps/ios` and uses only the public routes in [Clients](/docs/clients): `/health` to check, `/v1/models` to confirm the token, `/voice` for a recording and `/ask` for text. Anything the app does, you could build against those routes too. ## Building it The Xcode project is generated from `apps/ios/project.yml` instead of being checked in, because nobody can review a `.pbxproj`. ```sh brew install xcodegen pnpm exec nx run ios:app # generate the project and open it pnpm exec nx run ios:xcodebuild # compile for the simulator pnpm exec nx run ios:xcodetest # run ParlourTests ``` Parlour for iOS is not on the App Store yet, so for now you build it yourself. It needs iOS 18 or newer. Signing is up to you. Set `PARLOUR_DEVELOPMENT_TEAM` before generating, or pick a team in Xcode once. HomeKit and the local network both need a real device, not the simulator. ## The first run A fresh install walks you through setup before it shows the talk button, in the same order and words as the menu bar app, so it takes about a minute with the Mac nearby: 1. **Connect.** Scan the pairing code. The servers Bonjour found, and typing an address and token by hand, are folded away until you ask. The app checks the server there and then, so a wrong token shows up now rather than on your first question. 2. **Voice.** Allow the microphone, and say which room the phone usually lives in, so "lights off" means the lights in there. 3. **Extras.** Answering on the phone when home is out of reach. A phone that cannot run the on-device model skips this step. 4. **Finish.** What was set. Settings has **Run setup again** at the bottom. A phone that was already paired before setup existed counts as set up. ## What it asks for You should know what a voice assistant is allowed to do before you trust it. So the Settings tab lists the four permissions it relies on, with a mark against each and an Ask button for any not yet granted, and every prompt says what the permission gives you. | Permission | What it gives you | Asked for when | | --- | --- | --- | | Microphone | Recording while you hold the talk button | The Voice step of setup, or the first time you hold the button | | Local network | Finding the server with Bonjour, then talking to it | The app opens | | HomeKit | The rooms and accessories on the House tab | You open that tab | | Speech recognition | Understanding a request on the phone when the server cannot be reached | The phone first has to answer by itself, with the on-device answer switched on | | Camera | Scanning the pairing code on the Mac | You tap **Scan pairing code**, in setup or in Settings | HomeKit and multicast networking are entitlements as well as prompts. Apple grants multicast to the team, so a build signed with another team needs it left out of `Parlour.entitlements` or its own grant. If you say no to the local network, iOS does not ask again, so Settings and setup offer **Open Settings**, where the Local Network switch is. `NSAllowsLocalNetworking` allows plain HTTP to the house, and only on the local network, because the server sits on a private address and speaks HTTP. The phone page makes the same trade. It is also why a reverse proxy is worth having if you want to reach the house from outside. ## Pairing with your Mac The quickest way to connect is a QR code. Run `parlour pair` on the Mac, or open **Pair a phone** on the menu bar app's Status tab, then tap **Scan pairing code** in the app's Settings. The Camera app reads the code too and hands it to Parlour. One code carries the server's address and its access token together, so the two can never come from different servers. The token goes into the phone's keychain, never into a backup, and Settings checks the server straight away. A pairing link can also arrive from outside the app: from the Camera, or from any web page or message with a `parlour://pair` link in it. So before taking one, the app asks whether to pair with the server it names and shows the address everything you say would go to. Only pair with a code your own server showed you. ## Finding the server Without a code, the app finds the server the same way a satellite does. It browses for `_parlour._tcp`. Bonjour returns a service rather than an address, so the app opens a connection to the service and reads the host and port off the resolved path. Anything it finds appears in Settings. An address you type in, pick from that list or take from a pairing code wins over anything found. From then on, it is the only server the app talks to, just as a satellite pins the first server it trusts. You can also name the room the phone is in, so a follow-up lands in the right one. Browsing is also what raises the local network prompt. So the prompt appears as the app opens, not while someone is trying to say something. ## Managing the server The Server tab manages the Mac from the phone, over the routes in [Managing the server](/docs/clients#managing-the-server): - **Model servers.** Whether the local model and whisper are running, with buttons to start, stop and restart them. Stopping one asks first, because the house cannot answer the same way without it. - **Pipeline.** How many requests run at once, how many can wait per room, how many tasks one request can become, when to give up, and whether to read requests first. Saving restarts the agent so the change takes effect, and the steppers stop at the same limits the server enforces. - **Maintenance.** Runs the doctor, shows the end of any of the three logs, and restarts the agent. The server has the final say on every change and explains any refusal in its own words. For example, it refuses to start a model that is too big for the Mac, or to restart something that was restarted a moment ago. ## The model on the phone The app can answer on its own using Apple's Foundation Models, which need iOS 26 and Apple Intelligence switched on. It is off by default, and it is a fallback, not a first stop. The phone has no tools, so it cannot switch anything on, read a sensor or look anything up. The phone defers to the house for the same reason the house tries its local model first. When it is on and the server cannot be reached, the phone transcribes the recording with on-device speech recognition and answers it on the device. The answer is labelled as coming from the phone, not from home. Nothing in that path leaves the handset. When the model is unavailable, Settings says why: too old an iOS, a phone without it, Apple Intelligence switched off, or the model still downloading. ## HomeKit The House tab shows the homes, rooms and accessories HomeKit already knows about, and can switch anything with a power state. It is deliberately not how Parlour controls the house. The agent's tools go through Home Assistant on the server, the one place that knows what is connected to what. The House tab is a second opinion in your pocket, and a screen that still works when the Mac at home is off. # Writing a provider Source: https://heyparlour.app/docs/providers > Add a voice, an engine or an integration as an npm package. You can replace any stage of Parlour with your own provider and choose it by name in `config.json`. Built-in providers are files under `packages/parlour/src/providers`. Your own is an npm package with a default export, and Parlour loads it without any change to Parlour itself. Integrations (a source of tools, prompt lines and a mute gate) are providers of kind `integration` and work the same way. This guide explains how it works, then builds a real one. If you have ever wrapped a command line tool in a function, you already know most of it. ## How a name becomes a provider `resolveProvider(kind, name, options, context)` in `core/providers.ts` does the work: 1. It looks up `(kind, name)` in the registry. The built-ins are there because their modules call `registerProvider` when imported. 2. If nothing is registered, it runs `await import(name)`. A name containing `..` is refused. An absolute path is allowed, so you can try a provider from a checkout before you publish it. The module's `default` export is accepted when it is an object whose `kind` matches, whose `name` is a string and whose `create` is a function. It is then registered under the name from config, so the next lookup finds it straight away. 3. It parses the options with the definition's `schema`, if there is one. A slice that fails is reported with the provider's name and one line per problem, path first. 4. It calls `create(options, context)` and returns the result, awaiting it if it is a promise. The context can be a function of the definition instead of a plain value. That is how the agent scopes each provider's logger by the provider's own `name` rather than the specifier from config, so a provider loaded by absolute path still logs as `piper`, not as the path. Only a missing provider is reported as unknown, with the registered alternatives listed so you can spot a typo. A package that exists but fails to load (a syntax error, a missing peer, an old Node) is reported as a load failure, and one whose `kind` is wrong for the slot says which kind it is. Your provider gets the whole slice from config. For `"tts": { "provider": "x", "voice": "y" }` it is handed `{ provider: "x", voice: "y", fallback: "macos-say" }`. Core fills in no `voice` or `speed` of its own, because the same slice also goes to the fallback voice, so each provider's schema supplies its own defaults. A `z.object` schema quietly drops the keys it does not declare, which is what you want here. ## The definition ```ts export type ProviderKind = | "audioSource" | "audioSink" | "wake" | "stt" | "tts" | "llm" | "decision" | "search" | "secrets" | "service" | "integration"; export interface ProviderContext { paths: Paths; // configFile, secretsFile, modelsDir, logsDir... secrets: Secrets; // core's own: haToken, anthropicKey, braveKey, token, logLevel log: Logger; // scoped to the provider's name emit: (event: AgentEvent) => void; // the NDJSON event stream, a no-op unless --events /** The whole config, for providers that need more than their own slice. */ config: unknown; } export interface ProviderDefinition { kind: ProviderKind; name: string; description: string; /** Validates the provider's own slice of config. Defaults to "anything". */ schema?: z.ZodType; create(options: unknown, context: ProviderContext): T | Promise; } export function defineProvider(definition: ProviderDefinition): ProviderDefinition; ``` `defineProvider` is the identity function. It lets TypeScript infer `T` and makes a package's default export read as what it is. `options` arrives as `unknown` even when there is a schema, so `create` casts it to the schema's inferred type, as every built-in does. ### A secret of your own `context.secrets` holds only the keys core reads for itself: `haToken`, `anthropicKey`, `braveKey`, `token` and `logLevel`. If your provider talks to a hosted service, read its key from the environment. Before any command runs, `parlour` copies every line of `secrets.env` into `process.env` (anything already set in the shell wins). So by the time `create` is called, `process.env.MY_KEY` holds whatever you gave `parlour secrets set MY_KEY`. Follow the built-in `openai-compatible` provider and take the variable's *name* as an option, not the value itself: ```ts const schema = z.object({ apiKeyEnv: z.string().default("MY_SERVICE_API_KEY") }); function create(options: Options, context: ProviderContext) { const apiKey = process.env[options.apiKeyEnv]; // If it is missing, say so from doctor() with the name to set, rather than throwing here. } ``` This keeps the key out of `config.json`, which is the rule for every provider (see the [contributing guide](https://github.com/mrprkr/parlour/blob/main/CONTRIBUTING.md)). People can rename the variable if they want to. And `parlour secrets status` reports it: every key in `secrets.env` is listed there, the four core ones by purpose and the rest as `in secrets.env`. The package root exports everything a provider needs: `defineProvider`, the `ProviderContext` and `ProviderDefinition` types, every port interface, `Check`, `defineTool` and `Tool` for integrations, and the message types a `ChatModel` sees. Fakes for every port come from `parlour/testing`. ## Worked example: Piper as a voice [Piper](https://github.com/rhasspy/piper) is a fast local text to speech engine. Its command line reads text on stdin and writes a WAV. Here it becomes a `tts` provider. `package.json`: ```json { "name": "parlour-tts-piper", "version": "0.1.0", "description": "Piper as a Parlour voice", "type": "module", "main": "dist/index.js", "types": "dist/index.d.ts", "files": ["dist"], "scripts": { "build": "tsc" }, "peerDependencies": { "parlour": ">=0.1.0" }, "dependencies": { "zod": "^4.0.0" }, "devDependencies": { "typescript": "^5.9.0", "@types/node": "^22.0.0" } } ``` `"type": "module"` matters. Parlour loads the package with `import()`, and a CommonJS module's `default` export is not the `module.exports` object you might expect. `parlour` is a peer dependency because the definition is imported from it, and there must be only one copy. `index.ts`: ```ts import { spawn } from "node:child_process"; import { existsSync } from "node:fs"; import { mkdtemp, readFile, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { type Check, defineProvider, type ProviderContext, type TextToSpeech } from "parlour"; import { z } from "zod"; // Only this provider's own keys. The rest of the tts slice (provider, // fallback, and any voice or speed meant for another engine) is dropped by // z.object rather than rejected. const schema = z.object({ /** A voice from https://github.com/rhasspy/piper/blob/master/VOICES.md, without the extension. */ model: z.string().default("en_GB-alba-medium"), /** Where the .onnx and .onnx.json live. Defaults to Parlour's own model cache. */ modelsDir: z.string().optional(), }); type Options = z.infer; /** piper reads the text on stdin and writes the file it was told to. */ function piper(model: string, text: string, file: string): Promise { return new Promise((resolve, reject) => { const child = spawn("piper", ["--model", model, "--output_file", file], { stdio: ["pipe", "ignore", "pipe"], }); let stderr = ""; child.stderr.on("data", (chunk: Buffer) => { stderr += chunk.toString(); }); child.on("error", reject); child.on("close", (code) => code === 0 ? resolve() : reject(new Error(`piper exited ${code}: ${stderr.trim()}`)), ); child.stdin.end(text); }); } function createPiper(options: Options, context: ProviderContext): TextToSpeech { const dir = options.modelsDir ?? join(context.paths.modelsDir, "piper"); const model = join(dir, `${options.model}.onnx`); return { // Nothing to load: piper is a process per sentence, and the model is // read from disk each time. A provider with a model to load would do // that here, so the first reply does not wait for it. async warm() {}, async render(text) { // Rendered to a temporary file rather than stdout so a stray line of // logging from piper cannot end up inside the WAV. const work = await mkdtemp(join(tmpdir(), "parlour-piper-")); const file = join(work, "sentence.wav"); try { const started = Date.now(); await piper(model, text, file); context.log.debug(`rendered in ${Date.now() - started}ms`); return await readFile(file); } finally { await rm(work, { recursive: true, force: true }); } }, // Throwing from render is how a voice fails. Core's FallbackTextToSpeech // catches it and speaks through tts.fallback instead, so this never // needs a fallback of its own. async doctor(): Promise { if (!existsSync(model)) { return [ { name: "Piper", status: "fail", detail: `${model} is missing. Download the voice and its .json from the Piper voices list.`, }, ]; } return [{ name: "Piper", status: "ok", detail: `voice ${options.model}` }]; }, }; } export default defineProvider({ kind: "tts", name: "parlour-tts-piper", description: "Piper, a fast local voice with a command line", schema, create: (options, context) => createPiper(options as Options, context), }); ``` `tsconfig.json`, so the output is ES modules with declarations: ```json { "compilerOptions": { "target": "ES2023", "module": "NodeNext", "moduleResolution": "nodenext", "strict": true, "declaration": true, "outDir": "dist", "skipLibCheck": true }, "include": ["index.ts"] } ``` Install it next to Parlour and name it in config: ```sh npm install -g parlour-tts-piper ``` ```json { "tts": { "provider": "parlour-tts-piper", "model": "en_GB-alba-medium" } } ``` A global install works because `npm install -g` puts every global package in one `node_modules`, which Node finds by walking up from Parlour's own files. ### Trying it against a checkout To try it before publishing, or against a Parlour checkout, point config at the built file with an absolute path: `"provider": "/Users/you/src/parlour-tts-piper/dist/index.js"`. To build that file you need a `parlour` to compile against, and the peer dependency alone does not provide one. `npm install` fetches peers from the registry, and fails when the version it wants is not published there. Add the checkout as a dev dependency instead, which satisfies the peer too: ```json "devDependencies": { "parlour": "file:/Users/you/src/parlour/packages/parlour" } ``` Build the checkout first (`pnpm -C packages/parlour build`). The link points at the package directory, and its exports point at `dist`. Running `npm link` in `packages/parlour` and then `npm link parlour` in the provider does the same, but it replaces any globally installed `parlour` with the checkout. Keep the link in place at run time, not just for `tsc`. `defineProvider` is a function, so the `import ... from "parlour"` in the built file is a real import. Node resolves it by walking up from the provider's own directory, not Parlour's. Without a `node_modules/parlour` next to the provider, `parlour doctor` reports that the package failed to load with `Cannot find package 'parlour'`, whichever Parlour is loading it. Then: ```sh parlour doctor # the Piper check appears alongside the others parlour text # nothing is spoken here, but the provider is created parlour start ``` ## The other kinds The same shape fills any slot. `create` returns the port for its kind: | Kind | Returns | Options come from | | --- | --- | --- | | `audioSource` | `AudioSource` | `audio` (the whole object; `inputDevice`, `sampleRate`...) | | `audioSink` | `AudioSink` | `audio` | | `wake` | `WakeWordEngine` | `wake` (`words`, `threshold`, `refractoryMs` and your own) | | `stt` | `SpeechToText` | `stt` | | `tts` | `TextToSpeech` | `tts` | | `llm` | `ChatModel` | `llm.local` or `llm.cloud`, whichever names you | | `decision` | `DecisionModel` | `llm.decision` (on-device triage; built-in `laya-mlx`) | | `search` | `SearchProvider` | `search` | | `integration` | `Integration` | `integrations["your-name"]` | `secrets` and `service` providers are chosen by platform, not named in config. To add one (a systemd `ServiceManager`, say), choose it by platform in `pickServiceManager` in `packages/parlour/src/providers/service/index.ts`, or in `pickSecretStore` in `providers/secrets/index.ts` for a secret store. That makes it a pull request rather than a package, and one that would be very welcome. ## An integration An integration is a provider of kind `integration` whose `create` returns: ```ts export interface Integration extends Diagnosable { readonly name: string; tools(): Promise; /** Extra lines for the system prompt. */ promptContext?(): string[]; /** True means ignore this wake. Asked after the wake word, before anything is acted on. */ gate?(): Promise; close?(): Promise; } ``` You turn an integration on by adding it as a key under `integrations`. The key is the provider name, so `"integrations": { "parlour-integration-sonos": {} }` loads that package and hands it `{}`. One thing to know: the block replaces the default rather than adding to it. If you write one by hand and still want the house and the connectors, keep `"home-assistant": {}` and `"connectors": {}` in it. (`parlour connectors add` writes `connectors.json`, never this block.) `parlour doctor` warns when a connector is listed that the config never loads. Build tools with `defineTool(name, description, jsonSchema, handler)` from the package root. The handler returns a string the model reads. Keep names stable and short. Every tool description sits in the local model's context on every turn, so a package with forty tools makes every answer slower. `tools()` is called once at assembly, so a connection opened there stays open. Close it in `close()`. The Home Assistant integration in `packages/parlour/src/integrations/home-assistant/index.ts` is a good reference: MCP tools, two REST tools, a prompt line, a gate that reads an entity, and a doctor that checks the token, the API and the MCP endpoint. ## Testing `parlour/testing` exports a fake for every port: `FakeChatModel`, `FakeDecisionModel` (with `triageAnswers` to build the scores it returns), `FakeSpeechToText`, `FakeTextToSpeech`, `FakeAudioSource`, `FakeAudioSink`, `FakeWakeWordEngine` (and the `FakeWakeWordDetector` it hands out, with `wakeFrame` to make one fire), `FakeSearchProvider`, `FakeSecretStore`, `FakeServiceManager` and `FakeIntegration`, plus `silentLogger`. Each one records what it was asked. They are the same fakes Parlour's own session and pipeline tests use, so your provider or integration can be tested with no hardware, model or network. A `ProviderContext` for a test takes only a few lines: ```ts import { resolvePaths, type ProviderContext } from "parlour"; import { silentLogger } from "parlour/testing"; const context: ProviderContext = { paths: resolvePaths({ HOME: "/tmp/parlour-test" }), secrets: {}, log: silentLogger, emit: () => {}, config: {}, }; ``` Providers that need hardware or a model are not unit tested in this repository either. Each has a `doctor()` instead. Get that part right, because it is what people see when something does not work. # Skills, MCP and plugins Source: https://heyparlour.app/docs/skills > Teach the house its rules in markdown, add tools over MCP, or bundle both in a plugin. The house already knows how to call tools. What it does not know is that goodnight means the porch light stays on and the rest go off, or that the car should only charge after eleven. Those are house rules. They change whenever someone rearranges a room, so writing one should not mean writing a provider. There are three ways to add them, in the order you are likely to want them. A **skill** is a markdown file. An **MCP server** is somebody else's tools. A **plugin** is one package that brings several of these at once. ## Skills A skill is a markdown file in `~/.config/parlour/skills`: ```md --- name: bedtime description: What goodnight means in this house --- Turn off the kitchen, hall and lounge lights, leave the porch light on, and set the thermostat to 17. Say "goodnight" and nothing else. ``` ```sh parlour skills new bedtime # writes the file, with the frontmatter filled in parlour skills list # the rules, and what each one is for parlour skills show bedtime # the body, as the model reads it parlour skills path # where they live parlour skills write bedtime < f.md # replace it, checked before it is written parlour skills remove bedtime # delete one of the house's own parlour restart # the agent reads them at startup ``` Only the name and description go in the system prompt. The model fetches the body with the `read_skill` tool when it decides the rule applies. A local model with a small context cannot carry every house rule on every turn, and most turns are "turn the kitchen light off". Write the description as the trigger ("what goodnight means"), not as a title. The frontmatter is optional. Without it, the filename is the name and the first line is the description, so a one-paragraph rule can be just that paragraph. A name is lower case letters, digits, `-` and `_`, and when two skills share one, the first found wins. A skill with files of its own can be a directory instead, with the rule in `SKILL.md`: ```text ~/.config/parlour/skills/ bedtime.md shopping/ SKILL.md list.txt ``` Write the body the way you would brief a house sitter: which tools to call, in what order, and how short the spoken reply should be. `parlour doctor` counts the skills it found, names any file it could not use and why, and warns when a body (over 4,000 characters) is long enough to slow every answer that uses it. To keep your rules somewhere else, such as a git repository, point `config.json` at that directory with `"skills": { "dir": "/house/rules" }`. `"skills": { "enabled": false }` turns skills off entirely, tool and all. ## MCP servers Anything that speaks [MCP](https://modelcontextprotocol.io) can give the house new tools. Every tool it advertises becomes one the house can call: ```sh parlour mcp add weather --url https://weather.example/mcp parlour mcp add notes --token-env NOTES_TOKEN -- npx -y @someone/notes-mcp parlour mcp list parlour mcp remove weather ``` The first adds a server on the network, sending the `--token-env` variable, if you give one, as a bearer token. The second adds one this machine starts and talks to over stdin. Each tool arrives named after its server, such as `notes_search`, which is the name to use in a skill. Either way, the entry lands under `integrations.mcp.servers` in `config.json`, and you can write it there by hand instead: ```json { "integrations": { "mcp": { "servers": { "notes": { "transport": "stdio", "command": "npx", "args": ["-y", "@someone/notes-mcp"], "tokenEnv": "NOTES_TOKEN" } } } } } ``` A stdio server gets `PATH`, `HOME`, `LANG` and `TMPDIR`, the one variable `tokenEnv` names, and its own `env` block. It does not inherit the rest of the environment. Every secret in `secrets.env` lives there, and a server that wants the calendar has no business reading the Anthropic key. Store the token with `parlour secrets set NOTES_TOKEN` and name it with `tokenEnv`, rather than writing it into `config.json`. If a server will not start, `parlour doctor` reports it and the rest of the house keeps working. A dead weather server should never stop the lights going off. A remote server that wants you to sign in with a browser is a connector, not an MCP entry. `parlour connectors add ` does the OAuth and keeps the tokens in the Keychain, or in a file on a machine without one. ## Plugins You can add everything above one piece at a time, and for a single piece that is simpler. A plugin is for pieces that only make sense together: a package for a car, say, with a provider, two skills that know its tool names, and the MCP server both of them talk to. ```sh npm install -g parlour-plugin-car # where parlour can resolve it parlour plugins add parlour-plugin-car parlour plugins list parlour restart parlour plugins remove parlour-plugin-car # takes it out of config ``` `parlour plugins add` loads the package before it writes the name into config. A missing package is caught then, not at the next start with the microphone live. An absolute path works too, so you can try a plugin from a checkout. A plugin is an npm package whose default export is a `Plugin`: ```ts import { definePlugin } from "parlour"; import { join } from "node:path"; import { carVoice, carTools } from "./providers.js"; export default definePlugin({ name: "car", description: "The car: charging, climate and where it is", /** Registered before any slot is filled, so config can name them. */ providers: [carVoice, carTools], /** A directory of markdown skill files, shipped with the package. */ skillsDir: join(import.meta.dirname, "skills"), /** Or skills written in code, the same shape as a file. */ skills: [ { name: "charging", description: "When to charge", body: "Only after 23:00.", source: "car" }, ], /** Config for the integrations it needs, merged under the house's own. */ integrations: { "car-tools": {}, mcp: { servers: { car: { transport: "stdio", command: "car-mcp" } } }, }, /** Anything it wants to do once, before the agent is built. */ async setup(context) { context.log.info(`using ${context.paths.home}`); }, }); ``` Plugins load first, before any provider is resolved. So by the time a slot in `config.json` names `car-voice`, it is already registered. The `integrations` block is merged *underneath* your own config, key by key. A plugin can bring an MCP server, and you can still change or remove it in `config.json`. For the same reason, a plugin skill loses to one of the same name in your own directory. A plugin that cannot be loaded is fatal, not skipped. A house missing the plugin that holds its skills would look just like one whose model has stopped listening. The one exception is a provider whose name is already registered: the registered one wins, and `parlour doctor` says so. `parlour plugins list` shows what each plugin brings without setting any of them up, so checking what is installed never opens a connection. # Tuning Source: https://heyparlour.app/docs/tuning > Adjust wake sensitivity, timing, voice and models, and fix what feels wrong. Everything on this page is a key in `~/.config/parlour/config.json`. `parlour config edit` opens the file. `parlour config show` prints what is in force, with the defaults filled in. `parlour restart` (or a restart from the app) picks up your changes. For the wake word, the voice, the microphone and the local model, running `parlour init` again is often quicker. Every question starts at what is set now, Escape goes back one question, and a review at the end lets you change any part before anything is downloaded or written. ## When something is not quite right | What you notice | What to try | | --- | --- | | It wakes up for the television | Raise `wake.threshold` towards 0.7. | | It never wakes up | Lower `wake.threshold` towards 0.4, and check that `audio.inputDevice` is the microphone you think it is. | | It cuts you off mid-sentence | Raise `audio.silenceMs`, or lower `audio.silenceThreshold`. | | It waits too long before answering | Lower `audio.silenceMs` to about 600. | | It gives up before you start talking | You have 2.5 s after the wake word to begin. Try saying the wake word and the request in one breath. | | It answers slowly | Try a smaller local model (`parlour models suggest` lists them), or a smaller whisper model: `parlour models fetch --whisper ggml-base.en.bin`, then `parlour service install` to rewrite the whisper service. It loads the first model file in `~/Library/Caches/parlour/models/whisper/`, and `base` sorts before `small`. | | It asks the cloud too often, or not often enough | By default the local model decides, guided by its prompt (`packages/parlour/src/core/prompt.ts`). For a threshold you can tune, add the [decision model](#decision-model-optional). | | It says nothing at all | Run `parlour doctor`, or press Check in the app. It names the part that is down. | | You cannot interrupt it | `audio.bargeIn` lets the wake word cut a reply short. It is off by default, because with one box in one room the microphone hears the speaker. | | It fires twice on one wake word | Raise `wake.refractoryMs`. | | The voice is not the one you wanted | Set `tts.voice` and `tts.speed`. Kokoro downloads itself the first time Parlour starts after an install, so that first start is slow and the fallback voice may speak until it is ready. | | It can see the lights but not the blinds | Expose the blinds. Home Assistant only publishes what is exposed under Settings > Voice assistants > Expose. | ## Audio ```json { "audio": { "inputDevice": ":0", "silenceMs": 800, "maxUtteranceMs": 15000, "silenceThreshold": 0.012, "bargeIn": false } } ``` - **`inputDevice`** is an avfoundation index, and `":0"` is the default microphone. `ffmpeg -f avfoundation -list_devices true -i ""` prints the list, and `parlour init` shows it to you when it asks. - **`silenceMs`** is how much quiet ends a request once you have started talking. **`silenceThreshold`** is the RMS level, from 0 to 1, below which a frame counts as silence. Raise it for a noisy room. Lower it for a quiet speaker. - **`maxUtteranceMs`** caps a single request. It guards against a microphone that never goes quiet. It is not a way to allow long questions. ## The wake word ```json { "wake": { "provider": "openwakeword", "words": ["hey_jarvis"], "threshold": 0.5, "refractoryMs": 1500 } } ``` The stock words are `hey_jarvis`, `alexa` and `hey_mycroft`. `parlour models fetch --wake hey_jarvis,alexa` fetches the ones you name into `~/Library/Caches/parlour/models/openwakeword/`. Any openWakeWord model works. Drop `.onnx` into that directory and add `` to `words`. You can have more than one word live at once, for a small cost in CPU. To train your own word, see [Your own wake word](/docs/wake-word). ## The voice ```json { "tts": { "provider": "kokoro", "voice": "bf_emma", "speed": 1.0, "fallback": "macos-say" } } ``` `parlour init` offers four British voices: `bf_emma` (the default), `bf_isabella`, `bm_george` and `bm_lewis`. Kokoro has more, among them `bf_alice`, `bf_lily`, `bm_daniel` and `bm_fable`, and American voices whose ids start `af_` and `am_`, such as `af_heart`. If you name a voice Kokoro does not have, the reply falls through to the fallback and the log lists every voice Kokoro knows. `fallback` names the voice used when the first one fails (`null` turns the fallback off). `macos-say` ignores a Kokoro voice id and uses the system voice. You can also give it a macOS voice name of its own (`"voice": "Daniel"` when `provider` is `macos-say`). ## Speech to text whisper.cpp is the default, kept warm as a service. On macOS 26 you can use Apple's own on-device recogniser instead, through the [yap](https://github.com/finnvoor/yap) command line tool: nothing to download and no server to run. ```sh brew install yap ``` ```json { "stt": { "provider": "yap", "locale": "en-GB" } } ``` `locale` is optional and defaults to the Mac's own. `parlour init` installs yap for you once the config names it. The whisper service stays installed until you remove it with `parlour service uninstall`. If launchd rather than the app runs the agent, follow it with `parlour service install`, which puts back only what the config still wants. `parlour doctor` checks yap is found. On Apple silicon there is also NVIDIA's [Parakeet](https://huggingface.co/mlx-community/parakeet-tdt-0.6b-v2), through [parakeet-mlx](https://github.com/senstella/parakeet-mlx). It is fast and accurate for English and uses about 2 GB of memory. ```sh pip install parakeet-mlx ``` ```json { "stt": { "provider": "parakeet-mlx" } } ``` Parlour keeps the model loaded in a Python worker that starts on the first utterance, so that one waits for the load (and, the first time, the download into the Hugging Face cache). `model` defaults to `mlx-community/parakeet-tdt-0.6b-v2`; `mlx-community/parakeet-tdt-0.6b-v3` trades a little English for about 25 languages. `python` defaults to `python3`; point it at a venv's own Python if parakeet-mlx lives there, for example `~/.local/share/uv/tools/parakeet-mlx/bin/python` after `uv tool install parakeet-mlx`. `parlour doctor` checks that Python can import it and that ffmpeg is found. ## The models This is what `parlour init` writes when you let Parlour run the local model, on a Mac with 16 GB, with the defaults filled in: ```json { "llm": { "local": { "provider": "openai-compatible", "managed": true, "baseUrl": "http://127.0.0.1:8920/v1", "model": "qwen3.5-9b", "temperature": 0.3, "timeoutMs": 30000 }, "cloud": { "provider": "anthropic", "enabled": true, "model": "claude-opus-5", "maxTokens": 1024, "onLocalFailure": true }, "maxToolRounds": 6 } } ``` The catalogue Parlour can run, and how much memory it suggests each for: | Model id | Download | Suggested from | | --- | --- | --- | | `qwen3.5-4b` | 2.7 GB | 8 GB | | `qwen3.5-9b` | 5.7 GB | 16 GB | | `gemma-4-26b-a4b` | 17 GB | 32 GB | | `qwen3.6-35b-a3b` | 20.4 GB | 64 GB | An Intel Mac runs the model on the CPU, so it is counted as having half its memory and is suggested a smaller model. - **`local.managed`** means Parlour runs the model itself: llama.cpp from Homebrew and a GGUF from its own catalogue, kept warm by launchd on port 8920, just like whisper. At a terminal, `init` offers this first and suggests the largest model in the catalogue above that the Mac can hold. `parlour models suggest` prints the same list with this Mac's pick marked. To switch, run `init` again and pick another: it downloads the file and points the service at it. By hand, that is `parlour models fetch --llm `, `local.model` set to the id, and `parlour service install`. For a script, `parlour init --local-model auto|none|` answers the question without a terminal. - **Without `managed`**, Parlour talks to whatever server `baseUrl` names and starts none of its own. The provider's defaults are LM Studio's: `http://127.0.0.1:1234/v1` and `qwen3-8b-mlx`. That is also what `init --yes`, or `init` with no terminal, writes on a fresh setup: a download of gigabytes is said yes to, never assumed. - **`local.model`** is the model id exactly as the server reports it. `parlour doctor` lists what is being served next to what is configured. A mismatch there is common after loading a different model in LM Studio. A managed server answers to the catalogue id, so the two only differ when the named model was never fetched. - **`local.timeoutMs`** applies to each completion. A model that often runs past it is too big for the machine. With `cloud.onLocalFailure` on, every timeout becomes a cloud answer, which is slower and not private. - **`cloud.enabled: false`**, or no `ANTHROPIC_API_KEY`, keeps everything local. The escalation tool is not offered, and the persona stops telling the model to hand over. The local model answers everything itself, with the house tools still available, including questions it probably should not. - **`maxToolRounds`** stops a model that keeps calling tools. Six is enough for "turn the kitchen and hall lights off and set a timer". - A hosted OpenAI-compatible endpoint also works as the local model. Add `"apiKeyEnv": "SOME_NAME"` naming a variable in `secrets.env`. ### Through the AI SDK instead You can fill either slot with `"provider": "ai-sdk"`. It reaches the same two back ends through Vercel's AI SDK instead of Parlour's own clients: ```json { "llm": { "local": { "provider": "ai-sdk", "backend": "openai-compatible", "baseUrl": "http://127.0.0.1:1234/v1", "model": "qwen3-8b-mlx" }, "cloud": { "provider": "ai-sdk", "backend": "anthropic", "model": "claude-opus-5", "effort": "low", "webSearch": true } } } ``` The loop does not move. Tools are handed to the SDK without an `execute`, so the SDK returns the calls instead of running them. The tool rounds, the escalation and the request deadline all stay in Parlour. The only change is who keeps up with the two APIs. Parlour only runs its own model server for the `openai-compatible` provider, so an `ai-sdk` local slot needs a server of its own, such as LM Studio. The built-in clients remain the default, because they need nothing but `fetch`. Choose the AI SDK if you would rather track one library than two APIs, or want to put the SDK's own middleware in front of a model. ### Decision model (optional) ```json { "llm": { "decision": { "provider": "laya-mlx", "mode": "shadow", "escalateThreshold": 0.85, "localConfidence": 0.75, "model": "aac6fef/laya-mlx" } } } ``` [Laya MLX](https://github.com/mizorewww/laya-mlx) scores each task on your Mac and decides whether it needs the cloud. It runs after triage has split the request and before the local model takes its turn. It needs Apple silicon and `pip install laya-mlx`. The first load downloads the checkpoint into the Hugging Face cache. - **Start with `"mode": "shadow"`.** The log shows `needs_cloud`, the intent and the confidence, but `ask_the_clever_one` still decides. - **Switch to `"mode": "triage"`** once the scores look right. Scores above `escalateThreshold` go straight to the cloud model. A clear house or timer intent keeps escalation off, so house commands stay local. - **`"provider": "none"`**, the default, skips the decision model entirely. - **Multilingual house?** Set `"model": "aac6fef/laya-multilingual-mlx"`. ## The pipeline ```json { "pipeline": { "concurrency": 2, "queueDepth": 2, "triage": "auto", "maxTasks": 4, "timeoutMs": 45000 } } ``` - **`concurrency`** is how many requests are answered at once across the whole house. Each client gets one request at a time either way, so this sets how many rooms can be answered at once, not how hard one room can push. Two suits one machine with one local model. Raise it if the model is hosted elsewhere and can take the load. The queue still keeps each room in turn. - **`queueDepth`** is how many requests may wait in one client's lane. Past that, the client's oldest waiting request is dropped silently. Someone who asks twice wants the second answer. - **`triage`** controls when a request is repaired and split before Parlour acts on it. `auto` reads anything compound, long or ambiguous, and lets a plain instruction straight through. `always` costs a local round trip on every request, which is worth it in a noisy room or with a small speech model. `never` sends the words to the action agent exactly as heard. - **`maxTasks`** caps how many tasks one request may become. Four covers "turn the kitchen and hall lights off, set a timer and tell me the forecast". - **`timeoutMs`** is the ceiling on one whole request. It is checked between tool rounds and between tasks, not mid-request, so nothing already with a model is abandoned. It stops a request that is still running long after the person has left the room. You can change these from a client too: `parlour remote pipeline set` or the iPhone app's Server tab, within tighter limits than the file allows. See [Managing the server](/docs/clients#managing-the-server). ### Keeping the Mac responsive The local model and whisper share the Mac with whoever is using it. On Apple silicon they also share its memory with the GPU. A model that does not fit does not fail cleanly: the whole machine slows to a crawl. So Parlour holds them back: - A model whose weights would take more than 70% of the Mac's memory is not started, and `parlour doctor` says so. `parlour models suggest` names one that fits. - Both servers leave two cores free, and use at most six. The local model keeps one conversation's worth of context rather than several. - Both run at a lower priority than anything you are using. launchd waits a minute before starting a crashed one again, rather than ten seconds. - Inside the App Store app, where Parlour keeps them running itself, a server that crashes five times in ten minutes is left stopped until you start it from a client. ## Search ```json { "search": { "provider": "searxng", "url": "http://searxng.local:8080", "maxResults": 5 } } ``` Only the local model uses this. The cloud model brings its own search. SearXNG needs `- json` under `search.formats` in its `settings.yml`. For a hosted alternative, use `"provider": "brave"` with `BRAVE_API_KEY` in `secrets.env`. `"provider": "none"` leaves the local model without a search tool. ## Logging Set `LOG_LEVEL` in `secrets.env` (or the environment) to `debug`, `info`, `warn` or `error`. `debug` prints every transcription time and every tool call with its arguments. Use it when you want to know why the model did what it did. Service logs go to `~/Library/Logs/parlour/agent.log`, `whisper.log` and, when Parlour runs the local model, `llm.log`. `parlour service logs --lines 200` prints the tail of each. # Your own wake word Source: https://heyparlour.app/docs/wake-word > Train a wake word of your own and drop it in. The stock words are `hey_jarvis`, `alexa` and `hey_mycroft`. If you would rather Parlour answered to its own name, train one. openWakeWord makes this an hour's work on a free Colab GPU, and Parlour runs any model the notebook produces. The provider in `packages/parlour/src/providers/wake/openwakeword.ts` uses the same three stage pipeline for every word. A new word is just a new `.onnx` file and one line of config. ## Train it Open the simple notebook and set the runtime to a GPU (Runtime > Change runtime type > T4): https://colab.research.google.com/drive/1q1oe2zOyZp7UsB3jJiQ1IFn8z5YfjwEb Fill in the form cell and run everything. | Field | Try | Why | | --- | --- | --- | | `target_word` | `hey parlor` | Piper synthesises the training clips and reads spellings literally. The US spelling comes out right, while "parlour" is sometimes read as "par-loor". The model learns the sound, not the spelling, so spell it however you say it. | | `number_of_examples` | `2000` | The default is 1000. More examples cost minutes and noticeably tighten the model. | | `number_of_training_steps` | `20000` | The default is 10000. | | `false_activation_penalty` | `1500` | Leave it for the first model. Raise it towards 3000 if the result wakes for the television. | The notebook downloads a couple of gigabytes of speech and noise without your word, generates clips with it, trains, and writes `my_custom_model/hey_parlor.onnx`. Download that file from the file browser on the left of the notebook. If the first model fires too easily or not at all, try the [full notebook](https://github.com/dscripka/openWakeWord/blob/main/notebooks/automatic_model_training.ipynb). It runs the same process with two extra levers that matter more than any of the numbers above. `target_phrase` takes a list of spellings (`["hey parlour", "hey parlor", "hey par lur"]`). `custom_negative_phrases` takes things that sound like your word but should not fire (`["hey harlow", "hey carla", "parlay", "hey pilot"]`). ## Install it ```bash mv ~/Downloads/hey_parlor.onnx ~/Library/Caches/parlour/models/openwakeword/hey_parlour.onnx ``` The file name, without the `.onnx`, is what goes in the config: ```json { "wake": { "provider": "openwakeword", "words": ["hey_parlour", "hey_jarvis"], "threshold": 0.5, "refractoryMs": 1500 } } ``` `parlour restart` picks it up. `parlour doctor` confirms it, or names the file it cannot find. Keep `hey_jarvis` in the list for the first day, as above. It costs a little CPU and gives you a word you know works while you judge the new one. The provider feeds each word the last 16 speech embeddings. Models the notebooks train for a phrase of a few syllables expect exactly that. If yours stays silent, check its inputs first, from a checkout of this repository: ```bash cd packages/parlour && node -e ' const ort = require("onnxruntime-node"); ort.InferenceSession.create(process.env.HOME + "/Library/Caches/parlour/models/openwakeword/hey_parlour.onnx") .then((s) => console.log(s.inputNames, s.outputNames))' ``` ## Tune it Every detection is logged with its score, such as `"hey_parlour" fired at 0.83 on local`, so leave it running and talk to it. Custom models tend to score a little lower than the stock ones, so you can bring `threshold` down to 0.4. If it wakes for the room, raise `false_activation_penalty` and retrain. Pushing the threshold much past 0.7 mostly stops it hearing you too. The other settings are in [Tuning](/docs/tuning).