Show things: placements

A placement is where a surface goes on the screen. The person places one with config or /ui, an extension with ui.show, and the agent with the ui.show tool. The client in use draws it with its own elements. An extension never draws: it says what to show and where, and the client does the rest.

A manifest's default placements may name only band, pane.left, pane.right or footer.

#Show and hide

In Node, from onLoad (it runs once the extension is up), or from any tool:

import { defineExtension } from "@getorrery/ext";

export default defineExtension({
  onLoad: async (ctx) => {
    await ctx.ui.show({
      id: "build-times",
      placement: "pane.right",
      title: "Build times",
      surface: {
        kind: {
          t: "chart",
          kind: "columns",
          unit: "s",
          labels: ["mon", "tue", "wed"],
          series: [{ label: "build", values: [212, 198, 240] }],
        },
      },
      fallback: { kind: { t: "text", value: "build times: mon 212 s, tue 198 s, wed 240 s" } },
    });
  },
});

await ctx.ui.hide({ id: "build-times" });

The same calls in Rust are ui_show and ui_hide on the host handle, with the same fields.

On the wire, a process extension sends the requests itself. Each is a JSON-RPC request, framed with a Content-Length header as the host frames ext/load:

{"jsonrpc":"2.0","id":7,"method":"ui/show","params":{"id":"ci","placement":"pane.right","title":"CI","surface":{"kind":{"t":"text","value":"3 of 5 green"}},"fallback":{"kind":{"t":"text","value":"CI: 3 of 5 green"}}}}

The host answers {"jsonrpc":"2.0","id":7,"result":{}}. ui/hide takes {"id":"ci"}.

• id names the placement. The same id replaces what was there, in place. A new placement moves it.
• An owner changes only what it placed. One extension cannot replace or hide another's surface, and the person changes anything through config or /ui.
• surface is {"kind": …}, where kind is one of the surface kinds.
• title is the label drawn above it.
• Every placed surface is labelled with its owner: ext:<id>, agent or config.

#Components and data (code mode)

Instead of a whole surface, an extension can show a component: a structure over the catalog (orrery/1) and one JSON object of data that the structure reads by JSON Pointer. After the first show, it sends only data. The client binds the data into the structure and draws it. No code of the extension runs on the host or in the client.

ui.show takes exactly one of surface, component or structure:

• component names one the manifest declares in [[provides.components]] (Manifest reference).
• structure is an inline structure. The host checks it against the catalog and registers it as <owner>#<id>.
• data is the first data of a component or a structure: one JSON object. It is refused with a surface.

In Node:

await ctx.ui.show({ id: "ci", placement: "pane.right", title: "CI", component: "runs", data: { failed: 0, runs: [] } });

// The whole object; the host diffs it into the smallest change.
await ctx.ui.data("ci").set({ failed: 2, runs });

// Or RFC 6902 operations, applied in order, all or none.
await ctx.ui.data("ci").patch([{ op: "replace", path: "/failed", value: 3 }]);

// A press, a submit or a choice in a component it showed. `stop()` ends it.
const stop = ctx.ui.onEvent("ci", ({ action, value }) => {
  if (action === "rerun") rerun(value);
});

ctx.ui.onEvent(handler), with no id, hears every component the extension showed. The calls are on onLoad's host too, and on every hook's.

In Rust, orrery_ext_sdk::component(id, placement, name, data) and orrery_ext_sdk::structure(id, placement, structure, data) build the ui_show params. The data goes through the host handle, and actions reach each on_ui_event handler:

use std::sync::Arc;
use orrery_ext_api::Placement;
use serde_json::json;

ctx.ui_show(orrery_ext_sdk::component("ci", Placement::PaneRight, "runs", json!({ "failed": 0 }))).await?;
let host = orrery_ext_sdk::host().ok_or("not connected")?;
host.ui_data("ci").set(json!({ "failed": 2 })).await?;
orrery_ext_sdk::on_ui_event(Arc::new(|e| eprintln!("{} pressed {} with {}", e.id, e.action, e.value)));

On the wire, ui/data takes exactly one of set or patch, and is answered {}:

{"jsonrpc":"2.0","id":8,"method":"ui/data","params":{"id":"ci","patch":[{"op":"replace","path":"/failed","value":3}]}}

The host sends each action as a notification: {"jsonrpc":"2.0","method":"ui/event","params":{"id":"ci","action":"rerun","value":7}}. action is the action id the structure names. value is what came with it: a form's values, the chosen item, a row's key.

Only committed actions reach the owner. Scroll, focus, the open tab and text not yet submitted stay in the client.

Links open through the host. open on a link (ui.intent {node, action: "open", value: <href>}) never reaches the owner, whoever showed the component. The host opens it only when the href is one the component's bound tree shows, and only an http:// or https:// URL or a path inside the workspace. It passes the target to the system's opener (explorer on Windows, open on macOS, xdg-open elsewhere) as one argument, never through a shell, and says what it opened in a notice.

#Limits of components and data

• A refused update keeps the last good data. An update is checked against the component's JSON Schema (its schema, after its defaults are filled), and the refusal names the JSON path, such as the data at `/runs/0/name` does not match. Nothing is half-applied, and a patch is all or none. The first data of a ui.show is checked the same way.
• Only a declared component has a schema. The data of an inline structure is checked only by binding it into a valid catalog tree.
• A schema that is not valid JSON Schema refuses every show and update of that component, naming the problem: fix it in orrery.toml.
• Size: one instance's data is at most limits.dataBytes (256 KiB by default), and a structure at most limits.componentBytes (64 KiB). An update that goes over is refused (Config keys).
• Rate: updates over limits.dataUpdatesPerSec (10 a second by default) are coalesced: the last value wins at the next allowed moment. Nothing is dropped, but a fast value is drawn late.
• No code on the host. A binding only reads, compares and formats. Anything harder is computed by the extension into the data.
• Structures are checked against the catalog orrery/1. A component type outside it is refused, naming the JSON path and the component.
• data must be one JSON object, never an array or a scalar. The host refuses anything else, and the Rust SDK's set refuses it before sending.
• ui.data changes only an id the extension itself showed with a component or a structure.
• Every accepted update is session content and is written to the log. A pane that updates often grows the log.
• ui.onEvent handlers run in the extension's process, on the connection. A slow handler delays the next message: start long work without waiting for it.
• An action whose owner is gone (unloaded or crashed) is refused in the client, with the fix. Use resident = true for a component people act on.
• A link is refused with the fix when it names anything else: file:, javascript:, an absolute path, .., a path that does not exist, or a symlink out of the workspace. The notice says what a link may name.
• A link opens only on the host's own screen, the person at bare orrery. orrery run -p and a client attached to orrery serve never open anything on the serving machine, which may not be the person's: they get a notice naming the target instead, and the link's text form (label (href)) shows it too.
• An open whose href the component does not show is not a link: it goes to the owner like any other action.

#Fallback

A client that cannot draw a surface natively uses, in order:

1. the fallback its sender gave, if any;
2. otherwise the text form, which is the same in every client.

fallback is optional. When you give one, it is a surface too, and it may use core kinds only (text, markdown, table, tree, diff, progress, stream, task, question, form, stack, chart, raster, image). A fallback that uses another kind is refused, and the refusal names the kind. In a tool, build it with ctx.ui.text, ctx.ui.markdown or ctx.ui.table. In onLoad, which has no such helpers, write the object as above.

The text form is not stored. It is worked out from the surface:

orrery run -p prints nothing for a placement. orrery run -p --json emits CUSTOM orrery.placed {id, placement, owner, surface, text} when one is shown and CUSTOM orrery.unplaced {id, owner} when it goes.

#The agent's route

The agent has two core tools, ui.show and ui.hide, with the same fields. Each surface is checked against its element's schema before it is shown, and a surface that fails names the path that failed.

• The tools are searchable, not declared: the model finds them with tools.search when it needs them, so their schema does not sit in every request (Which tools the model sees).

#Components from the agent

The agent's ui.show takes the same three forms as an extension's: a surface, a component with data, or an inline structure with data. It sends data rather than whole trees. What the search answers for ui.show lists the catalog's ids and their props, so the model can write a structure without that list sitting in every prompt.

{"id": "todo", "placement": "pane.right", "title": "Todo",
 "structure": {"type": "text", "value": "${/left} left"}, "data": {"left": 3}}

{"id": "ci", "placement": "pane.right", "structure": {"type": "text", "value": "${/failed} failed"}, "data": {"failed": 0}}

• The agent sends an inline structure; it is registered as agent#<id>. A component an extension declared belongs to that extension: the agent naming one is refused with the fix.
• Every refusal comes back as the tool's error, with the JSON path and the fix: an unknown component, a structure outside the catalog, data that does not match the component's schema, data over limits.dataBytes.

Limits

• The agent has no ui.data: it changes a component by calling ui.show again with the same id and the whole new data. Each call is a tool call in its history, so a component the agent updates often costs tokens every time.
• An action on the agent's component is a note, not a message. A press, a choice or a submit in one the agent showed is added to the conversation as a host message the model reads on its next turn: [ext:ui] <id>: <action> <value>, such as [ext:ui] todo: select "row-3". It is never written as the person's typed message, so the model cannot take a press for words the person said; it is hidden from the thread, where the person already saw the press; and it starts no turn: with the agent idle, the note waits for the person's next message. In session/read it reads as an ext.message from ui.
• The search answer lists the catalog. The agent cannot show an extension's declared components: it composes its own structure.
• The catalog list makes ui.show match many searches (list, table, diff), so it can take one of the ten places a broad tools.search returns.
• The agent's placements stay in its history. Each one costs input tokens on later turns.
• By default the agent may use every placement (ui(*)).

#Config and /ui

A placement can be written in config, so it is there from the next session on:

[ui.placements."build-times"]
placement = "pane.left"

orrery config set 'ui.placements."build-times".placement' pane.left writes the same key. In the TUI, /ui lists what is placed now, with its id, placement, owner and state. /ui move <id> <placement>, /ui hide <id> and /ui save <id> write the key for you (CLI reference). Config keys are listed in Config keys.

A config key moves a surface that exists. It does not create one: the owner still provides the content.

#Permissions

ui(<placement>) rules decide where an agent or an extension may place a surface:

[permissions."ext:build-times"]
deny = ["ui(pane.right)"]

A deny names the rule in its refusal, and the fix is in the same message. With no rule, a placement is allowed. See Permissions and consent.

#Default placements and resident

An extension can declare the surfaces it will fill, so they are drawn at once, in their loading state, before the extension is up:

[extension]
api      = "orrery-ext/1"
name     = "build-times"
version  = "0.1.0"
runtime  = "node"
resident = true

[[provides.placements]]
id        = "build-times"
placement = "pane.right"
title     = "Build times"

• The extension fills each one after ext/load, with the same id.
• resident = true keeps its process for the whole session (Manifest reference).

#Surface kinds

A surface is {"kind": {"t": "<kind>", …}}. The older core kinds are text, table, tree, diff, progress, stream, task, question, form, stack and markdown; ctx.ui.text, .table and .markdown build three of them. Three kinds were added for placements:

Chart kinds: bars, columns, stacked, line, area, scatter, sparkline, donut and gauge.

• scatter reads values as x, y pairs.
• gauge takes values: [value, max]. max defaults to 100.
• stacked draws absolute values only.

#Limits

What is drawn

• One client draws the screen. A placement is drawn by the client in use, with that client's own elements. An extension is never a second client.
• A third-party client that has not added panes and the band shows no panes and no band. Its manifest stays valid, so nothing refuses it, and its author learns it from the conformance fixtures.
• The ADE does not draw placements yet.
• A pane can look like something it is not. The owner label always shows, and input, consent and trust cannot be imitated in place.
• A raster is capped at 64 × 32 cells, and it is coarse in a terminal.
• An image shows only where the terminal has a graphics protocol: PNG on Kitty, any format on iTerm2. Elsewhere, such as over SSH or in tmux without passthrough, it shows its alt. An image is at most 5 MB once decoded.
• Narrow screens: a pane is 25% of the width. It folds into one band row when that would be under 18 columns, or when the thread would be under 40. The band shows at most 3 rows, then +N more.
• Colour is never the only signal. Bars, glyphs, patterns and labels carry the meaning.

What it costs

• A resident extension keeps one process for the whole session. An idle process costs about 1.4 MB in Rust, 7.4 MB in Python, 11.9 MB in Go and 15.5 MB in Node.
• A non-resident extension parks after a quiet spell, and its placements stop changing without a sign. A crash shows stopped (dimmed, with the fix the screen names: /ext restart <id>).
• A placement the agent makes costs input tokens on every later turn, because it stays in history. Keep surfaces small.
• A placement an extension makes is not in the model's context. The model reads a tool's value, not a surface.

What it refuses

• ask rules on ui(<placement>) are refused when the call is made: a placement cannot wait for an answer. Use allow or deny.
• input, consent and trust are the host's alone. Showing under one of those ids is refused.
• A surface that fails its element's schema is refused, naming the path.
• A fallback that uses a non-core kind is refused, naming the kind.
• A manifest may not declare inline or hidden as a default placement.
• A placement that a rule denies is refused, naming the rule.

Lifetime

• A runtime placement is session content. It is recorded in the session, drawn again on resume, and goes with the session. /ui save keeps where a surface is, not what it shows.
• Unloading or uninstalling an extension removes the surfaces it placed.
• A placed surface is not sealed when its turn ends. Its owner may replace it, by id, for as long as the session lives.
• Dismissing a surface (Esc, x or Delete while it has focus, in the TUI; F6 moves focus in) hides it for this client, for this session only. Another attached client still draws it, and the host has no record of the dismissal yet.

Before you rely on it

• Config moves a placement only for an id something has placed. An id nothing has placed does nothing, and says nothing.
• The renderers key in a manifest does nothing. An extension never draws: give an element a fallback instead (Manifest reference).
• ui/* calls made while ext/load is being answered are held and drawn once the session's screen is up, in order (at most 256 wait). Their sender is answered {} at once, so a refusal found then (a denied rule, a bad fallback) comes back as a warning notice from that extension: a toast on a screen, a line on stderr with orrery run -p. Show from ext/load for a first draw, and expect the answer to mean "taken", not "drawn".
• ui/* calls from one extension take effect in the order sent, even when it sends the next without waiting for the answer: ui.show then ui.data under the same id never races. The same holds for state/set, state/get, tools/hide, tools/show, ext/register and ext/unregister, which share that one queue. Its other calls (model calls, session/append, agents, timers) may run concurrently and finish in any order: await one before a call that depends on it. Calls from two extensions are never ordered against each other.
• An extension with [[provides.placements]] starts with the session, like one that subscribes to events, so its declared surfaces are filled without waiting for a first call.
• A crash is noticed at the next heartbeat (up to 15 seconds), so stopped can appear late; a supervisor restart that follows at once may mean stopped is drawn only for a moment. /ext restart <id> starts it again by hand.