AtlatestRepositorycourier

courier / treeDESIGN-relays.md

Read it rendered

1# Courier Relays: Agent Communication via Local Sockets
2
3## Overview
4
5This document describes an enhancement to Courier that adds **relay**
6support — local Unix socket channels that enable real-time bidirectional
7communication between a leader Claude session and worker agent sessions.
8
9Relays extend Courier's existing `send-message` tool with a new transport.
10Instead of building a separate tool for agent coordination, Courier gains the
11ability to route messages to local agents the same way it routes messages to
12humans via Telegram. The leader doesn't think about transports — it just sends
13a message to a recipient.
15Messages on relays are ephemeral — short, human-readable text meant to signal
16state changes and prompt action. Durable context (task details, progress logs,
17design decisions) belongs in folio notes, not relay messages.
19## Problem
21When a leader session spawns worker agents (via tmux), there's no reactive
22communication channel between them. The leader has to poll files to check on
23progress. Workers have no way to signal the leader when they're blocked or
24done. This gap was identified when the Ops session had to be manually prompted
25to check on worker state.
27## Design
29### Concepts
31**Relay**: A named Unix socket that connects exactly two Courier instances —
32a listener and a connector. Messages pass through in real-time with no
33buffering or persistence.
35**Leader**: The Courier instance that creates and listens on relays. This
36is typically the Ops session or any session that spawns workers.
38**Worker**: A Courier instance launched with `--relay <name>`, which
39connects to an existing relay socket. The worker sees the relay as a
40message recipient.
42### Message Flow
44```
45Leader Session Worker Session
46┌──────────────┐ ┌──────────────┐
47│ Claude │ │ Claude │
48│ │ │ │
49│ send-message│ │ send-message│
50│ to: "agent" │ │ to: "leader"│
51│ │ │ │
52│ channel │ │ channel │
53│ notify! ◄───┼────────────────────────┼──── notify! │
54└──────┬───────┘ └──────┬───────┘
55 │ stdio │ stdio
56┌──────┴───────┐ Unix socket ┌──────┴───────┐
57│ Courier │◄══════════════════════►│ Courier │
58│ (leader) │ ~/.courier/relays/ │ (worker) │
59│ │ agent-name.sock │ │
60│ Telegram ◄──┤ │ no Telegram │
61│ Relays │ │ 1 relay │
62└──────────────┘ └──────────────┘
63```
65### Unified send-message
67The existing `send-message` tool gains relay awareness. Courier maintains
68a recipient registry:
70- **Telegram recipients**: identified by chat_id (existing behavior)
71- **Relay recipients**: identified by relay name
73The tool's `to:` parameter accepts either:
74```
75send-message to: "331005009" text: "..." → Telegram (chat ID)
76send-message to: "sigil-agent" text: "..." → relay socket
77send-message to: "leader" text: "..." → relay (worker mode)
78```
80When a message arrives on a relay (from either direction), it's delivered
81as a channel notification to the Claude session, just like Telegram messages:
83```xml
84<channel source="courier" sender="sigil-agent" type="relay">
85 Done with step 2, see folio notes for details
86</channel>
87```
89### Leader Usage
91The leader's Courier exposes a `create-relay` tool:
93```
94create-relay name: "sigil-agent"
95```
97This creates a Unix socket at `~/.courier/relays/sigil-agent.sock` and
98starts listening for a connection. The relay name becomes a valid recipient
99for `send-message`.
101The leader can create multiple relays — one per worker agent it spawns.
103A `list-relays` tool shows active relays and connection status:
105```
106list-relays
107→ sigil-agent: connected (uptime 2h)
108 sc-agent: waiting for connection
109```
111A `close-relay` tool tears down a relay when the worker is done:
113```
114close-relay name: "sigil-agent"
115```
117### Worker Usage
119A worker's Courier is launched with:
121```
122courier --relay sigil-agent
123```
125This connects to `~/.courier/relays/sigil-agent.sock`. The worker's
126Courier:
128- Registers "leader" as a recipient (the other end of the relay)
129- Does NOT start Telegram polling (no token needed)
130- Exposes the same `send-message` tool, but with "leader" as the only
131 non-local recipient
133The worker sends messages to the leader:
135```
136send-message to: "leader" text: "Blocked on test failures, need guidance"
137```
139The leader receives this as a channel notification immediately.
141### Socket Protocol
143Messages on the socket are newline-delimited JSON:
145```json
146{"sender": "sigil-agent", "text": "Step 2 complete, see folio"}
147{"sender": "leader", "text": "Move on to step 3"}
148```
150Simple, no framing complexity. Each side reads lines and delivers them as
151channel notifications to their Claude session.
153### Socket Path Convention
155All relay sockets live in `~/.courier/relays/`. The name maps directly
156to the filename:
158```
159~/.courier/relays/sigil-agent.sock
160~/.courier/relays/sc-agent.sock
161```
163Workers only need the relay name, not the full path.
165## Updated Courier Instructions
167The instructions string sent to Claude in the MCP handshake is updated
168based on mode:
170**Leader mode** (Telegram + relays):
171```
172Messages from Telegram arrive as channel notifications with type="telegram".
173Messages from worker agents arrive with type="relay" and the agent's name
174as the sender.
176Use send-message with a chat_id to reply via Telegram, or with a relay
177name to message a worker agent.
179Use create-relay to set up a communication channel before spawning a
180worker agent. Use list-relays to see active connections.
181```
183**Worker mode** (relay only):
184```
185You are a worker agent connected to a leader session via relay.
186Messages from the leader arrive as channel notifications with type="relay".
188Use send-message to: "leader" to communicate back to the leader session.
189Keep messages short — use folio notes for detailed context.
190```
192## Configuration
194### Leader (existing courier.yaml, extended)
196```yaml
197allowed-senders:
198 - "331005009"
200relays:
201 socket-dir: ~/.courier/relays
202```
204### Worker (CLI args only, no config file needed)
206```
207courier --relay sigil-agent --log logs/courier.log --log-level trace
208```
210No `.env-courier` needed for workers (no Telegram token). The relay name
211is all the config a worker needs.
213## Spawning Workers
215The leader session handles spawning — it knows the project context, worktree
216needs, and MCP config. A typical flow:
2181. Leader calls `create-relay name: "sigil-agent"`
2192. Leader creates a tmux session with Claude, passing MCP config inline:
220 ```bash
221 tmux new-session -d -s sigil-agent \
222 claude --mcp-config '{"mcpServers":{"courier":{"command":"courier","args":["--relay","sigil-agent","--log","logs/courier.log"]}}}' \
223 --project-dir ~/Projects/Code/sigil/sigil \
224 ...
225 ```
226 Or by writing a `.mcp.json` in the project directory.
2273. Worker starts, courier connects to the relay socket
2284. Leader sees "sigil-agent: connected" in `list-relays`
2295. Leader sends initial task: `send-message to: "sigil-agent" text: "..."`
231The spawning logic can be systematized as a shared skill installed across
232repos — a skill that knows how to set up tmux sessions, configure MCP,
233and start Claude with the right flags and permissions.
235## Resilience
237### Worker crash/disconnect
239When a worker's courier disconnects (process dies, tmux session killed):
240- Leader's courier detects the closed socket
241- Delivers a channel notification:
242 ```xml
243 <channel source="courier" sender="sigil-agent" type="relay-disconnect">
244 Agent disconnected
245 </channel>
246 ```
247- Leader can decide whether to respawn or clean up
249### Leader restart
251If the leader's courier restarts:
252- Relay sockets are recreated by the leader
253- Workers using `--relay` retry connection with backoff
254- Short disruption, but recoverable
256### System restart
258Relay sockets are ephemeral (Unix sockets disappear on reboot). After a
259restart:
260- Leader recreates relays as needed
261- Workers are re-spawned (using folio task notes with in-progress status
262 to determine what needs to resume)
263- Workers reconnect via `--relay` and continue
264- `claude --continue` resumes the previous conversation in the worker's
265 project directory
267## Implementation Plan
269### Phase 1: Socket Infrastructure
271- [ ] Unix socket creation/listening in courier
272- [ ] Socket connection (client mode) via `--relay` CLI arg
273- [ ] Newline-delimited JSON message protocol over socket
274- [ ] Deliver incoming socket messages as channel notifications
275- [ ] `~/.courier/relays/` directory convention
277### Phase 2: Leader Tools
279- [ ] `create-relay` tool: create socket, start listening
280- [ ] `close-relay` tool: tear down socket, notify if connected
281- [ ] `list-relays` tool: show active relays and connection status
282- [ ] Update `send-message` to route by recipient type (chat_id vs relay name)
283- [ ] Disconnect detection and notification
285### Phase 3: Worker Mode
287- [ ] `--relay <name>` CLI flag: connect to existing socket
288- [ ] Register "leader" as default recipient
289- [ ] Skip Telegram setup when in worker mode
290- [ ] Updated instructions string for worker context
291- [ ] Connection retry with backoff
293### Phase 4: Resilience
295- [ ] Reconnection on socket drop (worker side)
296- [ ] Relay recreation on leader restart
297- [ ] Integration with folio task notes for resumption after system restart
299## Dependencies
301No new dependencies. Courier already has everything it needs:
302- **sigil-async**: goroutines for socket I/O alongside stdin polling
303- **sigil-json**: message serialization
304- **sigil-log**: structured logging
306Unix socket support may need a small addition to sigil-stdlib or sigil-lib
307if not already exposed.
309## What This Replaces
311This design replaces the earlier Foreman proposal. The key insight: Foreman
312kept narrowing down to "just message passing between leader and workers" —
313which is exactly what Courier already does, just with a different transport.
315Agent orchestration (deciding what to spawn, managing worktrees, tracking
316tasks) stays in the leader session's instructions and skills. Courier
317provides the communication fabric. Folio provides shared knowledge.
319## Open Questions
3211. **Multiple relay connections**: Should a relay support more than two
322 endpoints? Current design is 1:1 (leader <> worker). Broadcast would
323 require a different topology.
3252. **Shared skill for spawning**: The pattern of "create relay, set up
326 tmux, write MCP config, start Claude" could be a shared skill installed
327 across repos. Design TBD.
3293. **Unix socket API in Sigil**: Need to verify that sigil-lib exposes
330 Unix domain socket creation/connection, or add it.