Class CompanionClient

The active session id, or null before connect().

Current lifecycle of the receive event stream (0.30.0) — idle before start(), then connecting / connected / reconnecting / degraded_sse (WS errored, SSE fallback carrying delivery) / stopped. Subscribe to transitions with onStreamStateChange.

Canonical URL of the official Pouchy brand icon (square PNG, transparent background) at the given size. Static + public — needs no token — so it's safe to drop straight into an <img src> for "powered by Pouchy" badges, the companion's app tile, etc. Sizes: 256 | 512 | 1024 (default 512).

Tear the session down in one call: stop the event stream and consolidate memory (endSession) exactly once. The single teardown entry point for a NON-voice (text) session — mirror of what a connectCall() handle's close() does for voice. Idempotent. Returns endSession's diagnostic (or null). NOTE: if you drive ping() on your own timer for a background tab, clear that timer yourself — the client doesn't own it.

Resolve a pending confirmation (platform session tokens only — see onConfirmRequest). Show the user what they're approving (the event's summary), collect an explicit tap, then pass their decision here. On approval the recorded action runs server-side; its result comes back as outcome in this response (render it where the user tapped) AND as a normal companion.message on the stream (so the conversation shows it). A denial returns/emits a brief decline the same way. Single-use: re-resolving a settled confirmId fails with 409.

RETRY: if an approved IDEMPOTENT action (a wallet payment) flakes on a transient error, status is 'exec_failed' and retryable is true — you MAY call confirmAction(sameConfirmId, true) again to re-run it (the server dedupes a partial first attempt). A non-idempotent action (a message / skill) stays terminal and is never retryable.

Handshake: start or resume the session for this surface.

Open a live voice call end-to-end: mints credentials (startCall) and opens a WebRTC session directly to the provider — no first-party app code needed. Browser-only. Returns a handle to hang up + inject live context. EL Convai needs the optional '@elevenlabs/client' peer dependency; OpenAI Realtime has no extra deps.

End the session: consolidate what happened into the companion's long-term memory, so it remembers this experience in future first-party conversations. Called automatically when a connectCall() handle is closed; call it directly when a non-voice session ends. Best-effort + idempotent (safe to call twice; uses keepalive so it survives a page unload). Pass the voice transcript when you have it (connectCall does this for you).

Returns the server's consolidation diagnostic — { ok, facts?, skipped? } — or null if there was no session or the (swallowed) keepalive request failed. skipped tells you WHY nothing was written: 'no_session' (you passed the instance id, not the session id), 'no_content' (empty turn log / a voice transcript that never arrived), 'throttled' (a duplicate end beacon — harmless). facts is how many memories were consolidated.

Replay the offline queue: FIFO, sequential, each item a normal buffered input POST reusing its ORIGINAL turnId (the server dedupes completed turnIds — see queueOffline — so a retry can never double-run a turn). Runs automatically after connect() and on every stream reconnect; call it yourself for a manual retry affordance. One flush at a time — concurrent calls share the in-flight run. Per item: success (or a server-side duplicate) → removed; network failure / 5xx / 429 → stop, keep this item and the rest for the next flush; any other 4xx → the item itself is invalid (the server deliberately rejected it) — drop it, console.warn, and keep flushing.

Fetch the user's CURRENT companion avatar so the embedding can render the same virtual human Pouchy shows. The avatar is a VRM 3D model (vrmUrl, load it with a VRM/glTF renderer e.g. three-vrm); imageUrl is a flat 2D portrait when one exists (null today for built-in models). URLs are absolute and the /models asset is CORS-enabled, so a cross-origin renderer can fetch the VRM directly. Reflects the user's live choice (built-in or custom).

Read the companion instance's OWN wallet — nonzero stablecoin balances + total USD. Read-only and receive-only (spends go through the confirm-gated pay tools, never here). Requires the wallet.read scope (or wallet.spend, which subsumes it). Same data the companion gives when asked "what's my balance?".

Fetch this session's recent conversation turns (oldest→newest) so a reconnecting embed can restore its transcript. Distinct from recall (durable memory / facts) — this is the raw exchange log. Returns the token's OWN session turns only; requires a live session (call connect first). limit defaults to 20, capped at 50.

Convenience over ingestKnowledge: hand over a RAW file and Pouchy understands it server-side before ingesting — no parser/transcriber/vision on your side. Pass a data:<mime>;base64,… URL. Supported today: PDF (server-side text extraction), audio/video (Whisper transcript), and image (server-side vision caption — an objective description becomes the material). For other types, extract the text yourself and call ingestKnowledge. Same memory.write:core requirement and "My materials" integration as ingestKnowledge.

Ingest a DOCUMENT into the user's knowledge base. Unlike remember (one short fact), this distils already-extracted text — a PDF body, a meeting transcript, notes — into a headline summary PLUS full-text chunks, each embedded and semantically recallable, and surfaces it in the user's "My materials" list with source attribution, exactly like a first-party upload. So the materials entry point can live in YOUR app, not just Pouchy's.

Extract the text yourself (your own parser, or Pouchy's /api/stt for audio — send your pchy_ token as the bearer; the relay is auth-gated and metered) and pass it here. Writes the user's SHARED knowledge, so the token must hold memory.write:core (the user's consent) — otherwise 403. Embeddings are computed on the user's next first-party open (eventually-consistent semantic recall). kind is a free label for the source type (pdf / audio / note / …) used only for display + the materials icon.

Subscribe to outbound events of a given type, or "*" for all. Returns an unsubscribe function. Since 0.30.0 the envelope payload is NARROWED per event name (OutboundPayloadMap) — e.g. on('companion.typing', (env) => env.payload.active) type-checks without a cast; '*' keeps the untyped envelope. Type-only: runtime behavior is unchanged.

Subscribe to outbound events of a given type, or "*" for all. Returns an unsubscribe function. Since 0.30.0 the envelope payload is NARROWED per event name (OutboundPayloadMap) — e.g. on('companion.typing', (env) => env.payload.active) type-checks without a cast; '*' keeps the untyped envelope. Type-only: runtime behavior is unchanged.

Convenience: subscribe to TTS audio clips for the reply (companion.audio, non-call modality) — play url, or resolve ref your way.

Convenience: subscribe to sensitive-op confirmation requests (companion.confirm_request) — a payment, skill run, or friend message the companion wants the user to approve. Use it to surface the pending approval in your UI.

Whether YOU can resolve it depends on the token:

• Platform session tokens (minted via /v1/sessions for your end users): show your own confirm card and call confirmAction — your end user is the account's human, so the session may approve. This is how confirm-gated custom skills (POST / credentialed) run.
• First-party user tokens (Login-with-Pouchy PATs): observe-only — approval is authed as the Pouchy user (their app / a hosted confirm page), which is also where the biometric (Face ID / passkey) gate lives. confirmAction returns 401 on these tokens.

Subscribe to token-streaming deltas of the reply (P1-2). Registering a subscriber makes sendText stream automatically. Returns an unsubscribe function. Render chunks as they arrive; on meta.reset clear the partial; when onMessage fires, replace the partial with the final text.

Convenience: subscribe to errors the companion surfaces — server-side agent errors and stream failures (e.g. a permanent stream_unauthorized) both arrive as control.error. Since 0.30.0 code is typed as ControlErrorCodeValue (the CONTROL_ERROR_CODES vocabulary + the SDK-synthesized stream codes, open for newer servers) so a switch on it autocompletes. A malformed frame with no string code is delivered with the literal fallback 'error' (not part of the exported vocabulary) — give your switch a default: arm rather than enumerating every code. Returns an unsubscribe fn.

Convenience: subscribe to avatar cues (companion.expression) — viseme / expression / gesture for an embed rendering the companion's body.

Convenience: subscribe to live updates of an already-rendered Instant UI panel (companion.ui_update). Apply each { key, value } to the panel's state bag and re-evaluate bound displays — no rebuild.

Convenience: subscribe to assistant text replies.

Subscribe to offline-queue changes (enqueue / flushed / dropped) — drive a "N pending" badge from items.length. Returns an unsubscribe fn.

Convenience: subscribe to Instant UI render surfaces (companion.ui_action). The host draws payload.interface with its OWN renderer — a web component, a native iOS/Android view tree, a CLI/TUI — all consuming the same platform-neutral genui schema. If the panel set reportChanges, feed the user's edits back via sendText/sendWorldState to continue the turn.

Convenience: subscribe to inbound A2A messages from the user's paired friends (companion.social_message), delivered cross-app to any embed whose token holds social.message. Surface them in your own UI / notify the user.

Subscribe to receive-stream lifecycle transitions (0.30.0) — drive a connection indicator: client.onStreamStateChange((s) => setStatus(s)). Fires only on change, with the previous state as the second argument. Returns an unsubscribe function.

Convenience: subscribe to tool-call requests from the companion. The call carries the wire payload plus argsJson — args pre-parsed as JSON (undefined when empty or malformed), so handlers don't each re-implement the same try/catch around JSON.parse(args).

Convenience: subscribe to the companion's activity indicator (companion.typing). Fires active:true when a turn starts working and active:false when it finishes or pauses for a tool result — spanning the tool-loop / thinking phase before the first text delta. Drive a "typing…" affordance from it: client.onTyping(({ active }) => setTyping(active)).

Convenience: subscribe to the per-token metering echo (control.usage) for a usage / billing view.

Convenience: subscribe to voice-inject cues (companion.voice_inject) — a voiceRelevant world-state line the companion should say aloud during a live call. Route payload.text to your active voice session when payload.speak. A text-only host can ignore it.

Pair this representative session's VISITOR with the owner so their two companions become friends — unlocking the A2A plane (messaging, gifts, visiting) between them. Only valid in a representative session (the client was created with a visitor).

Two-token consent: this client's PAT (the owner's) must hold represent:pair, and you pass the VISITOR's own Pouchy PAT — which must hold social.message — as proof + consent. The visitor must therefore also be a Pouchy user.

The session's still-pending confirmations (display-safe: id, summary, scope, timestamps — never raw args). Use it to rebuild your confirm card after a reload, since confirm_request events are not replayed. Platform session tokens only, like confirmAction.

The sends queued while offline (oldest first), as a defensive copy — see the queueOffline option.

Keepalive: bump this session's last-seen time so a long-idle embed stays "live" within the session TTL (cross-app A2A friend messages keep reaching it). Cheap; call it on a timer for a background tab. Requires a live session.

Recall the memory this token is authorized to see. Without query the facts come back ranked by importance/recency; with query the server runs semantic search over them (embedding-based, falls back to ranked when embeddings are unavailable) — e.g. recall({ query: 'food preferences', limit: 10 }).

Remember a fact (into this app's namespace unless namespace is given and the token has the core scope). Intimate-tier writes are rejected server-side.

Send a user text turn, optionally with images (data URLs) for a multimodal turn — images need the files scope. The reply arrives on the event stream as a companion.message — subscribe via onMessage()/start() to receive it.

TOKEN STREAMING (P1-2): when any onDelta() subscriber is registered (or opts.stream is true), the reply streams token-by-token on this very request — delta subscribers fire as chunks arrive, and onMessage still fires exactly ONCE with the authoritative final text (the streamed copy is emitted locally; the event-stream replay of the same envelope is deduplicated by id). Pass stream: false to force the buffered path.

AWAIT THE REPLY (request/response hosts — a REPL, an HTTP handler, a test): pass awaitReply: true and the promise resolves with the final reply { seq, text, envelope } — no onMessage wiring, no hand-rolled turn gate. It rides the same streaming request (the terminal done frame carries the authoritative envelope), so it works without start(); onMessage/onDelta subscribers still fire as usual. Only when the reply can't ride the request (stream: false, or an old server) does it fall back to the event stream — which then DOES need start() — resolving with the next companion.message, or rejecting with code reply_timeout after replyTimeoutMs (default 60s).

OFFLINE QUEUE (0.34.0, opt-in queueOffline: true): a send that never REACHED the server — fetch rejected at the network level, or navigator.onLine is already false — is queued for replay instead of lost, and resolves { seq: null, queued: true, turnId }. With awaitReply: true there is no reply to await this session, so it rejects with code queued_offline (the item IS queued first). HTTP error responses keep the throw semantics above — they are never queued.

Send a user text turn, optionally with images (data URLs) for a multimodal turn — images need the files scope. The reply arrives on the event stream as a companion.message — subscribe via onMessage()/start() to receive it.

TOKEN STREAMING (P1-2): when any onDelta() subscriber is registered (or opts.stream is true), the reply streams token-by-token on this very request — delta subscribers fire as chunks arrive, and onMessage still fires exactly ONCE with the authoritative final text (the streamed copy is emitted locally; the event-stream replay of the same envelope is deduplicated by id). Pass stream: false to force the buffered path.

AWAIT THE REPLY (request/response hosts — a REPL, an HTTP handler, a test): pass awaitReply: true and the promise resolves with the final reply { seq, text, envelope } — no onMessage wiring, no hand-rolled turn gate. It rides the same streaming request (the terminal done frame carries the authoritative envelope), so it works without start(); onMessage/onDelta subscribers still fire as usual. Only when the reply can't ride the request (stream: false, or an old server) does it fall back to the event stream — which then DOES need start() — resolving with the next companion.message, or rejecting with code reply_timeout after replyTimeoutMs (default 60s).

OFFLINE QUEUE (0.34.0, opt-in queueOffline: true): a send that never REACHED the server — fetch rejected at the network level, or navigator.onLine is already false — is queued for replay instead of lost, and resolves { seq: null, queued: true, turnId }. With awaitReply: true there is no reply to await this session, so it rejects with code queued_offline (the item IS queued first). HTTP error responses keep the throw semantics above — they are never queued.

Report the result of a companion.tool_call this surface performed. When all of the turn's calls are reported, the companion resumes and the continuation (a companion.message or more tool calls) arrives on the event stream.

Push live world-state — a single event or a batch. Retained events (retained:true) coalesce per kind; transient ones carry a salience.

Ergonomic input: specversion / id / source are filled automatically (id → a fresh handle, source → this surface), so callers only supply the meaningful fields: { type, data, retained?, salience?, voiceRelevant? }. A full CloudEvents envelope is still accepted (its fields win).

Change this session's I/O modalities mid-session (e.g. enable/disable voice). The request is intersected with the token's granted modalities server-side — a session can't widen past its key — and the EFFECTIVE set is returned. Requires a live session.

Swap the bearer token used by every subsequent request and stream reconnect. Session tokens expire (default 1h) — call this when your backend re-mints one, or wire onAuthError to do it on demand.

Open the event stream (auto-reconnecting with the resume cursor). Uses the WebSocket plane when opted in + available, else SSE.

Open the voice plane. Returns realtime credentials to connect DIRECTLY to the voice provider (the Pouchy server is out of the audio path). Provider precedence is server-side: ElevenLabs Convai (primary) → OpenAI Realtime (fallback). Requires the call scope.

• elevenlabs-convai: open the EL session with the returned token and pass instructions as overrides.agent.prompt.prompt (the EL agent runs EL's LLM, so the persona/memory snapshot rides in client-side).
• openai-realtime: open WebRTC with the short-lived clientSecret (instructions are baked into the session server-side).

Stop the event stream.