Agent workspace renderer

Planned — nothing built

No agents-workspace-2d exists in the repo; the Babylon spike it sits beside was never deployed either (a fact this plan’s own decision log corrects itself on). Its dashboard-ingest half is largely obviated if Slack work becomes controller-dispatched — see agent-dispatch-unification. The spec-vs-wire drift table remains accurate reference for any firehose consumer.

Status: planned. Nothing built. The read backend (dashboard-backend) is deployed; a Babylon Lite renderer exists as a local spike that has never been deployed. This plan adds a second renderer against the same contract, closes the one animation behaviour the original design named first and nobody has built, and ships the first renderer we actually run.

Depends on: irc-ai-agents steps 8–10 (firehose v1 + dashboard-backend

• SSE) — all shipped. Phase 4 additionally depends on the base/agent-wrapper pi-mono bump to 0.73.1, already executed per vibes/irc/pi-mono-update-plan.md.

Blocks: nothing. This realizes irc-ai-agents step 13, which has been open and optional since 2026-06.

Background: irc-ai-agents §“Frontend progression” item 3 and §“Event vocabulary”; agent-task-console §“Implementation order” step 5 and §“Relationship to plan-irc-ai-agents”; the undocumented vibes/irc/agents-workspace/ prototype (built 2026-06-20→24); review of chaitanyagiri/munder-difflin 2026-08-22.

---

Motivation

Step 13 of plan-irc-ai-agents — “timeline / animated (Godot) frontend” — has sat at ⬜ not started (optional) since the plan was written. That is no longer true, and the drift matters: two thirds of it were built in June and never written down. vibes/irc/agents-workspace/ is a working Babylon Lite scene that runs locally against either a mock sim or the real firehose. It has never been deployed — it is a spike, not a service. plan-agent-task-console.md was written on 2026-06-21 — one day after that prototype’s first commits — and still describes the animated workspace as a hypothetical “flourish” for step 5.

Meanwhile the one behaviour the original design named first is the one still missing. plan-irc-ai-agents §“Frontend progression” specifies it exactly:

Workers rendered as entities in a scene, animations bound to events (tool_call_start → walk to the right tool, tool_call_end → return, status=blocked → question-mark indicator, heartbeat → idle wiggle).

Three of those four shipped. The blocked indicator is there (an MGS-style !), the idle behaviours are there and then some (loitering, pacing, thought-bubble exchanges). But tool_call_start currently routes to a text ticker — pushTool(), agents-workspace/src/main.js:913 — and the agent stays parked at its workstation for the whole task. The scene shows that an agent is busy, never what it is doing. That spatial legibility is the entire reason the event vocabulary emits tool_call_start and tool_call_end as two events rather than one with a duration (§“Event vocabulary”, “Notes on the choice of events”).

Two further gaps:

• Slack agents are invisible. pi-mom runs Socket Mode with no inbound port and no event stream; its only durable state is per-channel JSONL on a PVC. Half our agent fleet cannot appear in any dashboard we have.
• The Babylon client is WebGPU-only. @babylonjs/lite has no WebGL fallback — createEngine() throws outright. Acceptable today, but it is a single point of failure for the only visual we have, and the client may need to move to @babylonjs/core later anyway (audio, and the long list of Lite limitations catalogued in its own README).

munder-difflin — an MIT Electron multi-agent harness reviewed 2026-08-22 — has solved the animation problem well, and its scene layer is portable. This plan takes that layer and nothing else.

---

Current state

What exists (and is not in any plan doc)

Component	State
vibes/irc/dashboard-backend/ (Go)	Shipped and deployed as svc/dashboard per plan-irc-ai-agents step 9 (dashboard.yaml, 81c348f0); access is kubectl port-forward only — the public tasks.rrchnm.internal HTTPRoute is still a noted follow-up there. Joins #firehose anonymously, CHATHISTORY-replays to rebuild state on every (re)connect, folds events into per-task state, serves GET /events as SSE — event: snapshot then event: fire, each event carrying the folded _task — plus a 500-event ring buffer and /healthz.
vibes/irc/agents-workspace/ (Babylon Lite, WebGPU)	Built 2026-06-20→24; never deployed anywhere, and undocumented outside its own README. A local spike: npm run dev, or its compose stack — whose prod profile serves the sim, not dashboard-backend. The README’s moby instructions (“Live at 10.112.113.191:8091”) describe an intended deploy that is not live; don’t read them as current state. Agents walk between a ready room and task workstations, avoid each other and the stations, loiter/pace when idle, trade thought bubbles, get an MGS ! when blocked; stations recolor by status and dissolve on archive. Correct 7-status lifecycle incl. for_review (pod exited, human owed a !complete) and done as resting, not terminal.
vibes/irc/agents-workspace/server/	A sim that stands in for the backend and relays the real one: FIREHOSE_SSE_URL consumes dashboard-backend’s SSE; POST /api/ingest eats single events, arrays, NDJSON, or raw <controller> {…} IRC log lines. One ingestEvent() choke point.
vibes/irc/relay-client/	The purpose-built IRC PWA (Lit + Shoelace). Overlaps plan-agent-task-console.md in intent; out of scope here.

The firehose vocabulary is the contract, and every doc says so. dashboard-backend’s README: “this renderer can be swapped or run alongside others.” That is the licence this plan operates under.

Spec-vs-wire drift found during review

plan-irc-ai-agents §“Event vocabulary” is v1-as-designed; the shipped agent-controller/internal/bot/firehose.go differs in small ways that any renderer must handle. Recording them here so the next person doesn’t rediscover them:

Spec says	Wire actually does
tool_call_start carries args_preview	carries args (the full command)
tool_call_end carries duration_ms	absent
task_assigned	worker_assigned (plus worker_online, task_ready, task_updated, hub_created/hub_updated, which postdate the spec)
—	task ids are strings ("4"); worker nicks are free-form (HAL-9000)
—	a status → in_progress with no worker_spawned (a !reopen rework flip) still needs an agent at the station

Also live: dashboard-backend emits SSE frames named snapshot / fire, while agents-workspace listens for snapshot / firehose / control (src/main.js:1111-1113) — which is why the Babylon client can only reach the real backend through the sim’s relay, never directly.

Why not just adopt munder-difflin

Reviewed in full 2026-08-22. It is an Electron desktop app that spawns local CLI processes through node-pty; its “providers” are the CLIs (claude, codex, grok…), driven by scraping their TUIs. pi-ai’s API providers — Anthropic, OpenAI, OpenRouter, MS Foundry — have nowhere to plug in. Adopting it means replacing agent-controller and pi-mom with its git-based “hive”, and maintaining a fork of a 273 KB src/main/index.ts plus a 160 KB src/main/hive.ts to do it.

Its scene layer, though, is MIT and unusually self-contained:

• ~5,400 LOC over 18 files in src/renderer/src/scene/office/, importing only pixi.js plus a colour-token module and the store’s Agent type.
• Nine window.* refs in the whole directory; three are Electron-specific and get injected instead.
• Diff-driven — it reads a ~8-field view model per agent (character, status, carrying, action, accent, lastPrompt, isGod, plus id/name) and animates the deltas.
• The cast is procedurally drawn (portraitArt.ts, 591 LOC of per-character recipes) — original art, no third-party licence.
• The environment is already pluggable: themeRegistry.ts exists expressly to lift environment constants (station anchors, idle-activity spots, prop positions, palette, tileset URLs, map) out of the scene into a ThemeConfig. The engine underneath — tilemap rendering, BFS pathfinding, camera, sprite animation, bubbles — is already environment-agnostic.

What we must not copy. assets/tilesets/*.png and assets/maps/*.tmj are LimeZu Modern Interiors under a Complete Version licence the munder-difflin author purchased on 2026-08-20. It does not transfer, and it reads: “YOU CAN’T RESELL OR DISTRIBUTE THE ASSET TO OTHERS.” Ship placeholder tiles behind the ThemeConfig seam; buy our own licence later if we want art of that quality. Rename the character roster off its TV-show source for the same reason.

---

Proposed target state

Two renderers, one contract, one backend — which is the design plan-irc-ai-agents §“Dashboard architecture” already called for (“frontends can be swapped or run side-by-side”).

                                          ┌──────────────────────────┐
  agent-controller ──► #firehose ────────► │  dashboard-backend       │
   (sole publisher, JSON per line)         │  • joins #firehose       │
                                           │  • CHATHISTORY replay    │
  base/agent-wrapper ──► POST /ingest ───► │  • in-memory task fold   │
   (IRC workers AND Slack agents)          │  • GET /events  (SSE)    │
   [Phase 4 — new]                         │  • POST /ingest [Phase 4]│
                                           └───────────┬──────────────┘
                                                       │ snapshot + fire
                                    ┌──────────────────┴──────────────────┐
                                    ▼                                     ▼
                       agents-workspace                      agents-workspace-2d
                       (Babylon Lite, 3D, WebGPU)            (Pixi, 2D, WebGL2)
                       existing spike — local only,          NEW — the one that
                       unchanged by this plan                gets deployed
                                    └──────── shared status/tool module ──┘

The new client is a Vite SPA at vibes/irc/agents-workspace-2d/, rendering the ported Pixi engine. It talks to dashboard-backend directly (accepting both fire and firehose frame names, so it also works against the sim in dev) and uses the folded _task the server already computes rather than re-deriving it.

Terminology throughout is environment-agnostic — scene, agent, station, environment — never office/desk/cafeteria. What gets rendered is data and is expected to change.

Why this approach (and not the alternatives)

Option	Effort	Verdict
Adopt munder-difflin wholesale	very high	rejected — Electron + PTY-scraped CLIs; would replace agent-controller and pi-mom, and abandon pi-mono/multi-provider
Extend agents-workspace only	low	considered — cheapest, but leaves us single-renderer on a WebGPU-only engine, and Babylon Lite’s documented limits (no DynamicTexture, no billboardMode, no material swap, no dispose) make rich choreography expensive
Replace the Babylon client with Pixi	medium	rejected, though cheaper than it looks — nothing is deployed, so there is no migration to pay for. Rejected because it discards working collision-avoidance, idle-behaviour, and overlay code that took real effort to get right against Babylon Lite’s constraints
Add a Pixi client alongside	medium	chosen — the contract was designed for parallel renderers; gets munder’s animation quality on WebGL2, hedging the Babylon client’s WebGPU-only constraint; and gives us a prototyping surface for scene behaviour that can feed back into the Babylon client through its existing src/visuals/ seams

The Babylon client’s likely future move from @babylonjs/lite to @babylonjs/core (audio, etc.) is a separate decision on a separate timetable. Nothing here depends on it — the SSE contract is the insulation.

---

Sequencing

Phase 0 — prerequisites

None blocking. Confirm svc/dashboard is reachable (kubectl -n vibes port-forward svc/dashboard 8080:8080) and that the sim in agents-workspace/server/ still runs standalone.

Phase 1 — scaffold + SSE adapter

New Vite app vibes/irc/agents-workspace-2d/, sibling to agents-workspace/ so the two renderers and the shared sim sit together.

• Adapter accepts both fire and firehose frame names — two lines, and it ends the standing confusion that forces the Babylon client through the sim relay.
• Consume the folded _task from each event instead of re-deriving state.
• Factor mapStatus() and WORKER_PRESENT_STATES (agents-workspace/src/main.js:880-897) into a small shared module both renderers import — including their reasoning: only in_progress and blocked have a live pod; for_review means the pod exited and a human is owed a !complete/!reopen; done is resting and reopenable; only archived is terminal and dissolves a station. Two copies will drift.
• Handle the spec-vs-wire drift table above.

Verify: the new client renders a full lifecycle against the existing sim, and separately against a port-forwarded svc/dashboard with no relay in between.

Phase 2 — port the Pixi engine

Copy src/renderer/src/scene/office/* and design/tokens.ts from munder-difflin (MIT — keep the copyright header, record provenance in an ATTRIBUTION.md as they do). Land as src/engine/.

Decouple from React and Electron: OfficeFloor.tsx (1,761 LOC) is essentially one large useEffect building the Pixi app. Turn it into

createScene(container, { getAgents, getTasks, onSelect, theme }): { destroy() }

replacing the 18 useStore call sites, window.cth.hiveTasks(), and useStore.getState().select(id). Drop the window.close() quit-intercept (Electron-only) and leave MessageEnvelope unwired — IRC workers are per-task and isolated, so there is no agent-to-agent traffic to animate yet.

Keep as the generic engine: TiledMapRenderer, Camera, Character, CharacterSprite, SpriteAdapter, pathfinding, ThoughtBubble, ToolBubble, station display, SeatPool, glRecovery, portraitArt, themeLoader.

Rewrite as theme data: the character roster; the ambient-line pool (strip the show-specific quips, keep the solo/two-party exchange machinery); and themeRegistry.ts → an EnvironmentTheme with one starter theme — a hand-authored Tiled map on a generated placeholder 16×16 atlas. Rename its environment-bound vocabulary (ErrandKind’s water/fridge/shelf → generic idleActivity entries with theme-supplied labels).

Render only what the backend sends — progress has no wire equivalent and isGod maps to nothing (IRC workers are peers). No fabricated data.

Verify: an agent walks a path across the placeholder environment, driven by real events. Do this before porting the rest — the ~40 hard-coded tile coordinates in themeRegistry.ts are the bulk of this phase’s risk.

Phase 3 — tool→station choreography

The capability neither existing component has, and the one the original plan named first.

• Normalize tool names: the firehose carries pi-agent-core’s lowercase bash / read / edit / write; munder’s STATION_BY_TOOL is keyed on Claude Code’s Read / Bash / WebFetch / Grep / TodoWrite / MCP. One map, in the shared module from Phase 1.
• tool_call_start → walk to that station, carry the tool icon, float a ToolBubble with args.
• tool_call_end → return, ✓/✗.
• permission_request / permission_resolved → blocked pose + bubble with the command; resume on resolve.
• heartbeat → keep idle agents visibly alive; this is the event’s only purpose (§“Event vocabulary”).
• Keep the text ticker too — it is genuinely useful and it is what the Babylon client shows.

Verify: !new + !ready a scratch task in #tasks; the agent appears on worker_assigned, moves station-to-station per tool call, blocks on [PERM], and leaves on worker_exited. Then run both renderers side by side against the same backend and confirm they agree on every status transition — this is the real test that the shared module holds. (Both run locally for this — neither is deployed until Phase 5.)

Phase 4 — Slack path

Additive, and it lights up both renderers at once.

• dashboard-backend gains POST /ingest — it has none today; it is IRC-in only. Bearer token from a Secret via ESO. Mirror the sim’s /api/ingest semantics (single event, array, NDJSON) so captured logs replay identically, and route it through the same apply() fold so /events output is indistinguishable from IRC-sourced events. Note this gives the backend an authenticated write surface for the first time — see Risks.
• base/agent-wrapper emits. Subscribe to the pi-agent-core Agent event stream — the same .subscribe() src/irc-task.ts already uses to post → tool / ✓ markers — and POST from src/index.ts (the long-running HTTP/Slack path) too. Emit on tool_execution_start / tool_execution_end / message_end / agent_end; those shapes are confirmed unchanged 0.54 → 0.73 in vibes/irc/pi-mono-update-plan.md. Gate the whole emitter on SCENE_HUB_URL being set, so an unconfigured wrapper is byte-identical to today.
• Extend the event shape with origin: 'irc' | 'slack' and provider alongside model. The wrapper already reports provider+model from /health and /manifest; provider support stays a wrapper concern (pi-ai natively; MS Foundry via an OpenAI-compatible base URL) — the scene only needs the field, to badge an agent.
• Slack agents identify by channel_id + capability (wrapper state is already per-channel); IRC workers by nick + task.

Verify: with SCENE_HUB_URL set on the general agent, ask it something in #sysadmin and confirm a Slack-origin agent appears in both renderers. Then confirm IRC is undisturbed — ordered →/✓ telemetry in a task channel, [PERM] prompts still arriving one at a time.

Phase 5 — deploy

This is the first deploy of any renderer. agents-workspace never shipped, so there is no precedent to follow in-cluster and no existing route to reuse — only dashboard-backend is running, and it is port-forward-only.

Static SPA behind Caddy: agents-workspace/Dockerfile + Caddyfile are a usable starting point (they already build a static SPA and reverse-proxy /api/*), but they target a compose stack pointed at the sim — retarget at svc/dashboard. k0s manifests modeled on infra/k0s/vibes/irc/relay-client.yaml (Deployment + Service + HTTPRoute on the internal Gateway). Image to ghcr.io/chnm/systems/vibes/irc/agents-workspace-2d, digest-pinned by CI via docker-images--vibes.yml; Argo syncs.

Deploying the client also forces the question plan-irc-ai-agents left open for dashboard-backend: an in-cluster SPA needs the backend reachable over more than kubectl port-forward. Expect to land the deferred tasks.rrchnm.internal-style HTTPRoute (or an equivalent internal one) as part of this phase.

Verify: the deployed client renders live task activity through the Gateway, from a browser that has never port-forwarded anything.

Phase N — record-keeping

Update plan-irc-ai-agents step 13 and §“Frontend progression” to reflect what was actually built; correct plan-agent-task-console.md step 5; add a changelog entry for agents-workspace (there is none). Fold the spec-vs-wire drift table into plan-irc-ai-agents §“Event vocabulary” so the v1 spec matches the wire.

---

Risks

• Environment-theme retuning. ~40 hard-coded tile coordinates in themeRegistry.ts are tied to the map we’re replacing. Mitigation: Phase 2’s verify gate — one agent walking on the placeholder map before anything else is ported.
• Two renderers drift. Sharpened by the fact that only one will be deployed: the Babylon spike has no users keeping it honest, so it can rot unnoticed. Mitigation: the shared status/tool module, and the side-by-side agreement check in Phase 3’s verify. If maintaining both starts costing more than it returns, that check is the signal to retire the spike — see Open questions.
• Phase 4 gives dashboard-backend a write surface. Today it is anonymous and read-only, which is most of its trust story (plan-irc-ai-agents §“Trust model”). An authenticated /ingest changes that posture. Mitigation: bearer token from ESO, in-cluster Service only, no Gateway route. Consider whether ingest belongs in a separate process — the same read/write split question plan-agent-task-console.md §“Security” leaves open.
• Phase 4 touches the shared wrapper, whose image is a base for six agents, the IRC worker, and slack/bot. Mitigation: strictly opt-in behind SCENE_HUB_URL; no code path changes when unset.
• #firehose integrity is still unhardened (+m / sole-publisher ACL remains an open item in plan-irc-ai-agents). A second consumer raises the value of fixing it, but does not change the exposure.
• Placeholder art may undersell the result. The animation is the value, but good tiles do a lot of the visual work. Decide whether a licensed tileset is worth buying before showing this to anyone.

Rollback

Phases 1–3 add a new service and touch nothing existing — delete the Deployment. Phase 4 is two independent reverts: drop POST /ingest from dashboard-backend, and unset SCENE_HUB_URL on the wrapper (which alone disables the emitter without a rebuild).

---

Open questions

• Does agents-workspace-2d belong under vibes/irc/? Once Phase 4 lands, neither renderer nor dashboard-backend is IRC-specific. A top-level vibes/workspace/ holding both clients and the backend is the honest layout; cheapest answer is to defer the move until Phase 4 actually ships.
• Does this supersede plan-agent-task-console.md, or sit beside it? That plan claims step 13 and demotes the animated workspace to a flourish — but relay-client has since been built and covers much of the console’s own scope. Worth reconciling all three docs in one pass rather than three.
• Does the Babylon spike survive at all? It was never deployed, and this plan deploys the 2D client instead — so it becomes an unused second codebase kept for the WebGPU hedge and as a comparison. Either invest in it (adopt Phase 3’s choreography through its src/visuals/ seams; deploy it beside the 2D client) or retire it and keep the port as a one-renderer story. Decide after Phase 3 reads well in 2D, not before — but decide, rather than letting it drift.
• Buy a LimeZu Complete Version licence (~$25)? Cheapest path to art of munder-difflin’s quality; obliges a credit link to limezu.itch.io in the app and on any public page. Only worth answering once the animation is proven.

---

Decision log

2026-08-22 — reviewed munder-difflin; take the scene, not the harness

Evaluated chaitanyagiri/munder-difflin for adoption. Rejected as a system — Electron, node-pty, CLI-scraping, and its own git-based orchestration would displace agent-controller, pi-mom, and pi-mono. Accepted its scene/office/ layer as a port: MIT, pixi-only, diff-driven off a small view model, procedurally-drawn cast, and an already-extracted environment seam.

2026-08-22 — add a renderer rather than extend or replace

Considered extending agents-workspace in place (cheapest) and replacing its renderer (one client). Chose a parallel Pixi client: the firehose vocabulary was designed for swappable, side-by-side frontends, and a WebGL2 client hedges the Babylon Lite WebGPU-only constraint while the Lite→core question stays open.

2026-08-22 — discovered the prototype the plans don’t mention

vibes/irc/agents-workspace/ was built 2026-06-20→24, but plan-irc-ai-agents step 13 still reads “not started”, plan-agent-task-console.md (written 2026-06-21) treats the animated workspace as hypothetical, and there is no changelog entry. Scope of this plan narrowed accordingly — from “build the animated workspace” to “add a second renderer and close the tool-choreography gap.” Record-keeping folded in as Phase N.

2026-08-22 — correction: agents-workspace is not deployed

The first draft of this plan said it was live on moby :8091. That came from reading its README’s deploy instructions (“Live at 10.112.113.191:8091”) as a statement of current state; it is not, and never shipped. Consequences for this plan: “keeps a deployed thing deployed” was struck from the rationale for adding a renderer rather than replacing one (the remaining reasons — the WebGL2 hedge and not discarding working behaviour code — still hold); Phase 5 is now understood as the first renderer deploy, which drags in the dashboard-backend public-route follow-up that plan-irc-ai-agents deferred.

---

Last updated: 2026-08-22