Companion SDK & API reference

Overview

The Companion SDK gives any product its own Agent. The same companion — its memory, personality, voice and avatar — plugs into your surface and can act through tools, reason over your app's live state, and hold a continuous relationship with each user across sessions and devices.

• Standards-first. REST + SSE for messaging, WebRTC for voice, OpenAPI 3.1 for discovery.
• Isomorphic. Runs in the browser and in Node 18+; tree-shakeable, zero required runtime deps.
• Scoped & safe. Every capability — memory, skills, wallet, social, Instant UI — is gated by token scopes; sensitive ops are confirmation-gated, with an optional biometric step-up.
• Beyond the sandbox. The wire contracts are platform-neutral data, so Instant UI renders on web, native iOS/Android or a CLI, and skills can run host-local / device capabilities — not just HTTP.

Installation

# the SDK (isomorphic: browser + Node 18+)
npm i @pouchy_ai/companion-sdk

# optional — only for ElevenLabs Convai voice; OpenAI Realtime needs nothing extra
npm i @elevenlabs/client

No bundler? Load it from the CDN with an import map:

<script type="importmap">
{
  "imports": {
    "@pouchy_ai/companion-sdk": "https://pouchy.ai/sdk/companion-sdk.js",
    "@elevenlabs/client": "https://esm.sh/@elevenlabs/client@1.9.0"
  }
}
</script>
<script type="module">
  import { createCompanion } from '@pouchy_ai/companion-sdk';
  const c = createCompanion({ baseUrl: 'https://pouchy.ai', token: sessionToken });
  await c.connect();
</script>

Production embeds should pin a version instead of riding the floating URL: /sdk/v<version>/companion-sdk.min.js is immutable (1-year cache, never republished), and the import map's integrity field verifies the exact bytes with the sha384 hash below — generated from this release's committed bundle, so the snippet is always in lockstep with what the URL serves. A minified twin of the floating URL also exists at /sdk/companion-sdk.min.js (~10 KB gzipped).

<script type="importmap">
{
  "imports": {
    "@pouchy_ai/companion-sdk": "https://pouchy.ai/sdk/v0.37.0/companion-sdk.min.js"
  },
  "integrity": {
    "https://pouchy.ai/sdk/v0.37.0/companion-sdk.min.js": "sha384-bwaIS4l2+Aant14U2sVjW/C5UFsF+Xz60JhK5EnRROGk3RTYo34q7WQl2gPN3YTK"
  }
}
</script>

Drop-in widget (zero code)

Fastest possible integration: one <iframe> gives you a fully rendered chat UI — bubbles, composer, streaming replies — authenticated by the same session token your backend already mints. Optional query params: theme=dark flips the palette, accent=%23rrggbb recolors the send button and user bubbles (text ink is picked automatically for contrast).

<!-- zero-code embed: a full chat UI in one tag -->
<iframe
  src="https://pouchy.ai/embed?token=SESSION_TOKEN&theme=dark&accent=%23ff6b81"
  style="width: 380px; height: 560px; border: 0; border-radius: 16px"
  allow="clipboard-write"
></iframe>

Prefer not to put the token in a URL? Drive the widget over postMessage: wait for pouchy:loaded, send pouchy:init with the token, then listen for pouchy:ready / message / sent / tool_call / error and push pouchy:send / pouchy:context at any time.

// drive it from the parent page instead of putting the token in the URL
const frame = document.querySelector('iframe');
window.addEventListener('message', (ev) => {
  if (ev.data?.source !== 'pouchy-widget') return;
  // 'pouchy:loaded' | 'pouchy:ready' | 'pouchy:message' | 'pouchy:sent'
  // | 'pouchy:tool_call' | 'pouchy:error'
  if (ev.data.type === 'pouchy:loaded')
    frame.contentWindow.postMessage({ type: 'pouchy:init', token: SESSION_TOKEN }, 'https://pouchy.ai');
  if (ev.data.type === 'pouchy:message') console.log('companion said:', ev.data.text);
});
// send on the user's behalf / push live context at any time:
frame.contentWindow.postMessage({ type: 'pouchy:send', text: 'hi!' }, 'https://pouchy.ai');
frame.contentWindow.postMessage({ type: 'pouchy:context',
  event: { type: 'app.scene', data: 'user is on the checkout page', retained: true } }, 'https://pouchy.ai');

The widget pins the embedding page’s origin (learned from the referrer or the first parent message) and only exchanges messages with it — sibling frames and popups are ignored. When you outgrow the canned UI, everything the widget does is available headless through the SDK below.

Quickstart

Two steps: your backend exchanges the project <strong>Secret Key</strong> (dashboard → Keys) for a per-user <strong>session token</strong>, then the client connects with it — four calls and you are talking. First-seen <code>external_user_id</code>s are provisioned automatically. <strong>Important: <code>external_user_id</code> must be a stable id from your account system</strong> — a browser-generated / device-local id mints a brand-new instance (fresh memory) every time it changes.

# 1) YOUR BACKEND — exchange the project Secret Key (dashboard → Keys)
#    for a per-user session. First-seen users are auto-provisioned.
POST https://pouchy.ai/v1/sessions
Authorization: Bearer pchy_sk_…
{ "agent": "<agentId>", "external_user_id": "user_4211" }
→ { "session_token": "pchy_…", "expires_in": 3600, "instance": { … } }

// 2) YOUR CLIENT — connect with the session token and talk (four calls):
import { createCompanion } from '@pouchy_ai/companion-sdk';

const companion = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: sessionToken        // from step 1 — never ship the Secret Key
});

await companion.connect();                       // handshake → HelloAck
companion.onMessage((text) => render(text));     // stream the reply
companion.start();                               // open the reply channel (SSE)

await companion.sendText('hey — what should I do next?');

React

Prefer hooks over wiring the client by hand? @pouchy_ai/react is a thin, typed layer over the same SDK: wrap your tree in a <CompanionProvider> and read live state with hooks. The provider owns the handshake, the auto-reconnecting stream and teardown.

# React hooks over the same SDK (peer deps: react + companion-sdk)
npm i @pouchy_ai/react @pouchy_ai/companion-sdk react

react and @pouchy_ai/companion-sdk are peer dependencies. Pass the provider the same short-lived session token as the vanilla client — useCompanion() exposes status, session and grantedScopes; useMessages() accumulates replies and optimistic user turns; useTyping() is a turn-spanning indicator; useCall() drives live WebRTC voice.

import { useState } from 'react';
import {
  CompanionProvider,
  useCompanion,
  useMessages,
  useTyping,
  useCall          // voice: { start, hangup, interrupt, inCall, speaking, transcript, ... }
} from '@pouchy_ai/react';

function Chat() {
  const { status } = useCompanion();     // 'idle' | 'connecting' | 'connected' | 'error'
  const { messages, send } = useMessages(); // replies + optimistic user turns
  const typing = useTyping();            // spans the whole turn (incl. tools)
  const [text, setText] = useState('');

  if (status === 'connecting') return <p>connecting…</p>;

  return (
    <>
      {messages.map((m, i) => (<p key={i}><b>{m.role}</b> {m.text}</p>))}
      {typing && <p><i>companion is typing…</i></p>}
      <form onSubmit={(e) => { e.preventDefault(); send(text); setText(''); }}>
        <input value={text} onChange={(e) => setText(e.target.value)} />
      </form>
    </>
  );
}

// The provider owns the handshake, the auto-reconnecting stream and teardown.
// Give it the SAME short-lived session token — never the Secret Key.
export default function App() {
  return (
    <CompanionProvider baseUrl="https://pouchy.ai" token={sessionToken} surface="my-web-app">
      <Chat />
    </CompanionProvider>
  );
}

Anything the hooks do not cover — sendWorldState, recall, onToolCall, setModalities — is on the client returned by useCompanion(), so you never lose access to the full surface documented below.

Svelte & Vue adapters

Svelte and Vue ship as subpath exports of the main package (0.31.0) — thin bindings over one shared, fully-tested view controller. The view tracks streamState, the rolling transcript, the streamed draft, typing, and pendingConfirms as immutable snapshots; sendText appends the user turn optimistically, confirmAction keeps the confirm-card bookkeeping (including the retryable re-approve semantic), and restore: N backfills recent history so a reloaded tab isn't blank.

// Svelte — store contract (Svelte 3 → 5), no extra deps
import { companionStore } from '@pouchy_ai/companion-sdk/svelte';
const companion = companionStore(client, { restore: 20 });
// $companion.transcript / $companion.draft / $companion.typing
// companion.sendText('hey') / companion.confirmAction(id, true)

// Vue 3 — shallowRef snapshot, auto-disposed with the effect scope
import { useCompanion } from '@pouchy_ai/companion-sdk/vue';
const { snapshot, sendText } = useCompanion(client, { restore: 20 });
// snapshot.value.transcript / snapshot.value.streamState

// Any other framework — the same tested controller, bind it yourself
import { createCompanionView } from '@pouchy_ai/companion-sdk';
const view = createCompanionView(client);
const unsub = view.subscribe(() => render(view.getSnapshot()));

You own the client lifecycle in every adapter — create, connect(), start(), and close() it yourself; adapters only observe. On any other framework (or none), bind the same controller via the root export createCompanionView(client). React hosts should use the richer @pouchy_ai/react package above instead.

Python SDK (server-side)

pouchy-companion is the official Python client for the same REST/SSE plane — aimed at server-side integrators (game backends, bots, services). Typed with pydantic v2; the protocol vocabulary is generated from the TypeScript source of truth and drift-tested in CI, so the two SDKs cannot silently diverge. Voice/WebRTC and browser surfaces stay with the JS SDK.

pip install pouchy-companion

from pouchy_companion import CompanionClient, MessagePayload

with CompanionClient(base_url="https://pouchy.ai", token=session_token) as client:
    client.connect()

    # Request/response in one call — streams server-side, returns the reply.
    out = client.send_text("what should I do about the boss?", await_reply=True)
    print(out["text"])

    # Or iterate the event stream (proactive messages, confirms, tool calls):
    for env in client.events():
        if isinstance(env.payload, MessagePayload):
            print("companion:", env.payload.text)

The client covers text turns (plain or streamed await_reply with on_delta, and an optional turn_id for idempotent retries), the SSE event stream with cursor resume, confirms (pending_confirms / confirm_action), app-declared tools (send_tool_result), world state, memory/knowledge, history and the read-only wallet. Errors raise CompanionError with the same .code vocabulary as the JS SDK. Full reference in the package README on PyPI/GitHub.

Unity / C# SDK (preview)

ai.pouchy.companion is the official Unity / .NET client for the same REST/SSE plane — for games and .NET integrators. HttpClient + async/await + Newtonsoft; the wire vocabulary is generated from the TypeScript source of truth and drift-tested in CI. The core CompanionClient has no UnityEngine dependency (unit-testable outside the editor); PouchyCompanionBehaviour is the optional MonoBehaviour that marshals every callback onto the Unity main thread. 0.1.0-preview — the protocol is drift-guarded; the C# awaits its first Unity compile pass.

// Unity Package Manager → Add package from git URL:
//   https://github.com/oviswang/pouchy.git?path=/packages/unity-sdk
using Pouchy.Companion;

var client = new CompanionClient("https://pouchy.ai", sessionToken);
await client.ConnectAsync();

// Request/response in one call — streams server-side, returns the reply.
var reply = await client.SendTextAsync("what should I do about the boss?", awaitReply: true);
Debug.Log(reply.Text);

// Or drop the PouchyCompanionBehaviour on a GameObject and wire OnMessage /
// OnConfirmRequest / OnToolCall in the Inspector — all events fire on the
// Unity main thread.

Covers text turns (plain or streamed awaitReply with onDelta), the SSE event stream with cursor resume, confirms, app-declared tools, world state, memory/knowledge, history and the read-only wallet — the same surface as the other SDKs. Errors throw CompanionError with the shared .Code vocabulary. Voice/WebRTC is out of scope. Full reference in the package README on GitHub.

Authentication & scopes

Every request authenticates with Authorization: Bearer <token>. The standard client credential is the session token your backend mints via POST /v1/sessions (see Quickstart / Platform). A Personal Access Token (PAT), also pchy_-prefixed, is the advanced alternative for personal or single-user integrations — mint it server-side and never ship a long-lived token in client code.

// The client always holds a SHORT-LIVED token: a session token from
// POST /v1/sessions (the standard path), or a server-minted PAT for
// personal/advanced integrations. Never ship a long-lived key in client code.
const companion = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: sessionToken,              // Authorization: Bearer <token> on every call
  modalities: ['text', 'voice'],    // intersected with the token's grant
});

Sensitive capabilities are opt-in per token scope — e.g. wallet.spend, skills.execute, social.message, memory.*, and the representative scopes below. Instant UI rendering rides the non-sensitive ui.render scope. The handshake returns the effective grantedScopes.

Platform: sessions & Admin API

For developers and enterprises building on the dashboard: a project holds agent templates; every end user gets their own instance of an agent (separate memory, relationship progress, wallet). Your end users never handle keys — your backend holds a project Secret Key (pchy_sk_…, or pchy_sk_test_… for the unmetered test environment) and exchanges it per end user for a short-lived session token. First-seen external_user_ids are auto-provisioned. Important: external_user_id must be a stable id from your account system — a browser-generated / device-local id mints a brand-new instance (fresh memory) every time it changes.

# Your BACKEND exchanges the project Secret Key for a per-user session.
# First-seen external_user_ids are auto-provisioned — zero user setup.
POST /v1/sessions
Authorization: Bearer pchy_sk_…
{ "agent": "<agentId>", "external_user_id": "user_4211" }

→ { "session_token": "pchy_…", "expires_in": 3600,
    "instance": { "id": "…", "external_user_id": "user_4211", "created": true } }

// Client: connect the SDK with the session token — same API as a PAT.
const companion = createCompanion({ baseUrl: 'https://pouchy.ai',
  token: sessionToken, modalities: ['text'] });

The returned session_token is an ordinary bearer for the client SDK — pass it as token to createCompanion(). Sessions carry the non-sensitive default scopes and can only be narrowed; agent-level capabilities widen them — setting genui: true on the template adds ui.render (Instant UI) to every session it mints, and a skills list of built-in catalog slugs pre-installs those skills on every instance with skills.execute granted (v1: credential-free public read APIs — dictionary, wikipedia, exchange-rates, …; they auto-run without a confirm round-trip). A social policy ({ pair, crossProject }) opts instances into agent-to-agent pairing + text messaging via the SDK's pairVisitor flow — enforced server-side at pair creation; wallet interactions stay excluded. Templates also carry scripted scenes (story lines with triggers + beats, compiled into a pinned prompt section every turn), per-language voice defaults (voices, ElevenLabs id preferred on calls, OpenAI preset for TTS/fallback) and an initialStage relationship seed applied to new instances only. A wallet flag gives instances a READ-ONLY wallet (balance + own deposit address via wallet.read; end users fund their own instance, paired friends can send to it — instances can never spend). Projects additionally carry a shared knowledge corpus (POST /v1/projects/{id}/knowledge): upload product docs / FAQ / lore once, and every instance recalls the relevant chunks semantically alongside the user's personal memory, cited by document name. Plans cap monthly active users (free tier: 10; a new live user beyond the cap gets 402 limit reached — existing users keep working; test keys are quota-exempt).

Everything the dashboard does is also available programmatically via the Admin API with a project admin key (pchy_admin_…): list/create agents on /v1/admin/agents, read/update a template on /v1/admin/agents/{agentId} (updates bump templateRev; live instances re-apply the persona on their next session). Key types are strictly separated: secret keys can't manage, admin keys can't mint sessions.

# Manage the project from your backend — the dashboard is optional.
# Authorization: Bearer pchy_admin_…   (project implied by the key)
GET/POST/PATCH/DELETE /v1/admin/agents[/{id}]   # templates incl. draft/publish
GET/POST /v1/admin/keys · DELETE /{keyId}       # secret keys (plaintext once)
GET /v1/admin/users · PATCH/DELETE /{iid}       # suspend / GDPR-erase end users
GET/POST /v1/admin/webhooks · DELETE /{whid}    # event push endpoints
GET/POST /v1/admin/channels · PATCH/DELETE /{id}# deploy into Telegram/Slack/…;
                                                #  config.groupMode → one shared
                                                #  brain for a whole GROUP chat
GET /v1/admin/logs · /usage · /billing          # audit, month usage, plan (RO)
GET/PATCH /v1/admin/project                     # rename / archive
GET /v1/admin/openapi                           # OpenAPI 3.1 spec (public)
# Typed client:  npm i @pouchy_ai/admin-sdk  →  createAdminClient({ adminKey })

Industry templates. Creating a project with { "template": "romance" | "game" | "hardware" | "ecommerce" | "life" } seeds 1–2 polished agent presets so the first session works out of the box — pick one when creating your project in the dashboard, too. Agents also carry a publish lifecycle — status: "draft" refuses live-key mints while test keys keep working, so you can tune a persona safely before going live.

Core concepts

Session & surface

Each surface (e.g. "game", "support-widget") is one resumable session. Reconnecting resumes from a cursor, so replies are never lost.

World-state

A stream of CloudEvents-shaped context (retained state + transient events) that grounds the companion in what is happening right now.

Modalities

Text and voice. Requested modalities are intersected with what the token grants.

Tools

Actions your app declares and performs on the companion's request — the bridge from "talk" to "do".

Capabilities

The companion exposes six capabilities over the SDK. Each is gated by a token scope, and each produces (or consumes) a platform-neutral payload — so the web app is just one host: a native iOS/Android app, a game engine, or a CLI can use the same capability by implementing the matching renderer/executor once.

The full scope vocabulary ships typed in the SDK — import COMPANION_SCOPES, SENSITIVE_SCOPES, DEFAULT_SCOPES and check grants with hasScope(ack.grantedScopes, 'skills.execute') instead of copying strings from this page.

createCompanion(options)

Returns a CompanionClient. Options:

Client methods

connect(): Promise<HelloAck>

Opens the session and performs the handshake. Returns the granted scopes, negotiated modalities, a resume cursor, (for representative sessions) the visitor-pairing state, and pendingToolCalls (0.37.0) — the outstanding tool calls of a turn that paused before this handshake (empty array when none), so an embed reloaded mid-pause can complete the turn (see the replayPendingToolCalls option).

start(): void / stop(): void

Open or close the inbound reply channel (SSE by default; stream: "websocket" is accepted but the server WS endpoint is not live yet, so it currently always falls back to SSE). Call start() once after connect().

streamState: CompanionStreamState / onStreamStateChange(handler: (state, prev) => void): () => void

Receive-stream lifecycle (0.30.0): idle | connecting | connected | reconnecting | degraded_sse | stopped. The handler fires on change only, with (state, prev) — drive a connection indicator without hand-rolling one. degraded_sse means the WebSocket transport errored and delivery continues on the SSE fallback; stopped covers stop()/close() and a permanent stream_unauthorized failure.

onMessage(handler: (text, envelope) => void): () => void

Subscribe to streamed companion replies. Returns an unsubscribe function.

sendText(text: string, opts?): Promise<{ seq: number | null; queued?: boolean; turnId?: string }> / sendText(text, { awaitReply: true }): Promise<SendTextReply>

Send a user message. The reply arrives on the stream as a companion.message — subscribe via onMessage(). Or pass { awaitReply: true } and the promise itself resolves with the completed turn's reply { seq, text, envelope } — request/response hosts (a CLI, an HTTP handler, a test) skip the onMessage wiring entirely; works without start().

onDelta(handler: (chunk, { reset? }) => void): () => void

Token streaming: registering makes sendText stream the reply as it is generated — the handler fires per text chunk (reset = clear the partial render), and onMessage still fires exactly once with the authoritative final text. Force per call with sendText(text, { stream: true | false }).

sendWorldState(input: WorldStateInput | WorldStateInput[]): Promise<{ accepted, dropped, injected, reacted }>

Push live context — { type, data, retained? }. Retained values persist for the session; transient ones are one-off signals.

connectCall(opts?): Promise<CompanionCall> / startCall(opts?): Promise<CallCredentials>

Start a live, low-latency voice session over WebRTC. Returns a call handle with .close(); declared tools + host control actions stay available in-call. Pass { bargeIn: true } to keep the mic open while the companion speaks so the user can interrupt mid-utterance (default is half-duplex, which protects speakerphone use from self-echo interruptions).

call.interrupt(): void

Silent cut (0.36.0): stop the utterance currently being spoken WITHOUT generating new speech — for real-time commentary going stale at a beat boundary. Idempotent (no-op when idle). Tiers by call.provider: openai-realtime is guaranteed (response.cancel + output-buffer clear); elevenlabs-convai is best-effort (user_activity signal + guarded internal buffer cut — no public EL stop API; injectEvent(text, true) remains the guaranteed-but-speaking preemption).

onToolCall(handler): () => void / sendToolResult(callId, result)

Receive companion.tool_call for the tools you declared; argsJson is the args pre-parsed as an object (undefined on empty/malformed args; the raw JSON string still rides in args). Reply with sendToolResult(callId, { ok, result }); the turn resumes once every call is reported (the result apply is idempotent per callId). replayed: true (0.37.0) marks the re-delivery of a still-outstanding call after a mid-pause reconnect — treat the call id as an idempotency key for side-effecting tools.

onRender(handler): () => void

Instant UI — receive companion.ui_action. payload.interface is the platform-neutral genui schema; draw it with your own renderer (web / native iOS+Android / CLI). Requires the ui.render scope.

onInterfaceUpdate(handler): () => void

Receive companion.ui_update — a live { panelId?, updates:[{key,value}] } write into an already-rendered panel. Apply it to the panel state; no rebuild.

onSocialMessage(handler): () => void

Receive companion.social_message — an inbound A2A message from a paired friend, delivered cross-app to any embed holding social.message.

onConfirmRequest(handler): () => void

Subscribe to companion.confirm_request — a sensitive op awaiting approval ({ confirmId, scope, summary, stepUp? }). Platform session tokens resolve it with confirmAction; first-party user tokens are observe-only (approval is authed as the Pouchy user, where the biometric/passkey gate lives).

confirmAction(confirmId, approve): Promise<{ status, outcome?, retryable? }>

Resolve a pending confirmation (platform session tokens only). Show the event's summary, collect an explicit tap, pass the decision; on approve the recorded action runs server-side and its result returns as `outcome` in the response (render it where the user tapped) and as a normal companion.message on the stream. Single-use: re-resolving a settled confirmId fails with 409 — EXCEPT an approved idempotent action (a wallet payment) that flaked transiently returns `status:'exec_failed'` + `retryable:true`, which you may re-call with the same confirmId to re-run (deduped).

pendingConfirms(): Promise<PendingConfirm[]>

The session's still-pending confirmations (display-safe: confirmId, scope, summary, createdAt) — rebuild your confirm card after a reload, since confirm_request events are not replayed. Session tokens only.

onAudio / onExpression / onUsage(handler): () => void

Typed subscriptions for companion.audio (TTS clips), companion.expression (avatar cues), and control.usage (per-token metering). All three events are reserved — not emitted yet.

recall(opts?: { query?: string; limit?: number }): Promise<RecalledMemory[]>

Read back the memories relevant to this session — content, kind, importance and namespace. Pass query to run semantic search over them server-side; omit it for the plain importance/recency ranking.

remember(fact) / ingestKnowledge(doc) / ingestFile(file)

Write memory this token is authorized for: <code>remember</code> stores one short fact in the app namespace; <code>ingestKnowledge</code> distils an already-extracted document (PDF body, transcript, notes) into a summary + embedded recallable chunks; <code>ingestFile</code> hands Pouchy a raw <code>data:</code> URL (PDF / audio / image) to understand server-side first. The two ingest calls surface in "My materials" and need the <code>memory.write:core</code> scope.

history(opts?: { limit?: number }): Promise<CompanionTurn[]>

Fetch this session's recent conversation turns ({ user, assistant, ts }, oldest→newest) so a reconnecting embed can restore its transcript. Distinct from recall (durable memory/facts) — this is the raw exchange log. Own session only; limit default 20, max 50.

setModalities(modalities: string[]): Promise<{ modalities }> / ping()

setModalities(modalities): change the live session's I/O modalities mid-session (e.g. toggle voice); intersected with the token's grant server-side, returns the effective set. ping(): keepalive that bumps the session's last-seen time so a long-idle embed stays live within the TTL.

setToken(token: string): void

Swap the bearer token used by every subsequent request and stream reconnect — session tokens expire (1h by default), so call this when your backend re-mints one, or use the onAuthError option to refresh on demand.

getAvatar(): Promise<CompanionAvatar> / brandIconUrl(size?)

Fetch the live avatar (3D VRM URL + 2D portrait, archetype, display name) to render your own front-end, and the Pouchy brand icon URL.

getWallet(): Promise<CompanionWallet>

Read the instance's own wallet — balances + total USD. Read-only and receive-only; needs the wallet.read scope.

pendingOutbox() / onOutboxChange(cb) / flushOutbox(): Promise<{ sent, remaining }>

Offline send queue (0.34.0, opt-in via <code>queueOffline</code>): inspect the durable outbox with <code>pendingOutbox()</code>, subscribe to changes with <code>onOutboxChange()</code>, and force a flush attempt with <code>flushOutbox()</code>.

pairVisitor(visitorToken: string): Promise<{ pairId }>

Representative mode only: pair a visitor who is also a Pouchy user to unlock agent-to-agent (A2A) context. Requires the represent:pair scope.

endSession(opts?: { transcript? }): Promise<EndSessionResult | null>

Cleanly end the session and optionally fold a transcript into long-term memory. Returns the server diagnostic <code>{ ok, facts?, skipped? }</code> (or <code>null</code>) — <code>skipped</code> tells you why nothing was written.

close(): Promise<EndSessionResult | null>

One-call teardown for a text session: stop the stream and run <code>endSession()</code> once (idempotent). Returns the same consolidation diagnostic.

onError(handler): () => void

Subscribe to transport / protocol errors ({ code, message }).

Events

Everything the companion does flows back as typed events on the stream you opened with start(). Subscribe with on(type, fn) (or '*'), or use the typed convenience helpers below. Each returns an unsubscribe function. Unknown event types are ignored, so the protocol is forward-compatible.

companion.onMessage((text) => render(text));                  // streamed reply
companion.onRender(({ interface: ui }) => myRenderer.draw(ui)); // Instant UI panel
companion.onInterfaceUpdate(({ update }) => myRenderer.apply(update));
companion.onSocialMessage(({ fromName, content }) => notify(fromName, content));
companion.onConfirmRequest((req) => showApproval(req));         // req.stepUp ⇒ gate with Face ID
companion.onAudio(({ url }) => play(url));                      // TTS clip (non-call) — reserved, not emitted today
companion.onToolCall(async ({ id, name, args }) => { /* … */ });
companion.start();                                              // open the event stream

World-state

Stream live context with sendWorldState({ type, data, retained? }). Retained values represent current state; transient ones are one-off signals. This is what lets the companion say "boss incoming — heal up" at the right moment.

// Retained state — the latest value persists for the session:
companion.sendWorldState({ type: 'game.player', data: { hp: 12, level: 7 }, retained: true });

// Transient event — a one-off signal:
companion.sendWorldState({ type: 'game.event', data: 'boss_appeared' });

// The companion reasons over this live context (and, in voice, reacts in real time).

Tools & actions

Declare the actions your surface can perform; the companion calls them and you return a result. The same handler serves both text and voice sessions.

const companion = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: PAT,
  tools: [
    {
      name: 'highlight_product',
      description: 'Visually highlight a product card in the host UI',
      parameters: { type: 'object', properties: { id: { type: 'string' } }, required: ['id'] }
    }
  ]
});

// The companion CALLS your tool; you perform it locally and report the result.
// argsJson is the pre-parsed args object (undefined if the args were invalid);
// reply with sendToolResult(callId, …).
companion.onToolCall(async ({ id, name, argsJson }) => {
  if (name === 'highlight_product') {
    highlight(argsJson?.id);
    await companion.sendToolResult(id, { ok: true });
  }
});

// Also available out of the box on connectCall() voice sessions (text turns
// expose only the tools[] you declared — declare these there yourself if you
// want them outside calls; see docs/companion-host-control.md):
// - HOST_CONTROL_TOOLS — universal verbs (invoke_action, set_feature,
//   set_value, navigate, highlight) offered automatically on calls once you
//   declare tools/handles, so the companion can drive generic host actions.
// - AVATAR_VISUAL_TOOLS — play_gesture / play_expression for embeds that
//   render the VRM avatar; DECLARE them to receive the calls (undeclared
//   they no-op silently).

Instant UI

The companion can render an interactive panel on your surface, not just speak text — a form to collect a few values, a summary card with a progress bar, a chart, a set of choices. It arrives as companion.ui_action; you draw payload.interface with your own renderer.

The payload is platform-neutral JSON — a tree of ~20 typed atoms (Text, Slider, Select, Chart, Table, Group…) plus a state bag — so the same panel renders on web, native iOS (SwiftUI), Android (Jetpack Compose), or a terminal. The renderer is the only platform-specific part; the contract never changes. Input atoms write to state; set reportChanges and the user's edits stream back as a turn (the live-form loop), and companion.ui_update applies live value changes without a rebuild.

// The companion renders an interactive panel; YOU draw the platform-neutral
// schema with your own renderer (web component, SwiftUI, Jetpack Compose, CLI).
let panel;
companion.onRender(({ interface: ui }) => {
  panel = renderInterface(ui, mountEl, {
    // If the panel set reportChanges, stream the user's edits back as a turn:
    onReportChanges: (state) => companion.sendText('I adjusted: ' + JSON.stringify(state))
  });
});

// Live, in-place updates — the companion changes a value without a rebuild:
companion.onInterfaceUpdate(({ update }) => panel.applyUpdate(update));

Reference renderers (vanilla web, SwiftUI, Jetpack Compose) and the full atom contract live in the Instant UI renderer guide. Gated by the ui.render scope.

Social (agent-to-agent)

The companion can message the user's paired friends on their behalf (send_friend_message / message_friends, confirmation-gated), read a friend thread (read_friend_messages), and — the cross-app half — surface an inbound friend message to your embed in real time via companion.social_message. So a companion in App A can message a friend whose companion runs in App B, and App B receives it. Requires the social.message scope.

// Inbound A2A: a message from one of the user's PAIRED friends, delivered
// cross-app to any embed whose token holds social.message (not just Pouchy).
companion.onSocialMessage(({ fromUid, fromName, content, createdAt }) => {
  showFriendMessage(fromName, content);
});

// The companion can also SEND on the user's behalf (confirm-gated):
await companion.sendText('tell Mom I will be late');  // → message_friends / send_friend_message

Confirmations & biometric step-up

Sensitive actions — a payment, running a skill, messaging a friend — are never executed inline. The companion emits companion.confirm_request and waits. Who approves depends on the token: a platform session token (an end-user instance minted via /v1/sessions) resolves it directly with confirmAction — your end user is that account's only human, and this is how confirm-gated custom skills (POST / credentialed) run. A first-party user token is observe-only: approval happens on the user's own Pouchy surface (their app or the hosted confirm page, authed with their own login), never in a third-party DOM.

The request carries an advisory stepUp flag (set for irreversible money ops). When true, the first-party approval surface requires a stronger gesture — a passkey / Face ID / Touch ID — before approving, verified server-side via WebAuthn. Users without a passkey, or with step-up disabled, fall back to a tap; the custodial flow is unchanged.

Driving the API with plain REST (no SDK)? The confirm card is session state, not part of the /input response: after a turn, GET /api/companion/session/{sid}/confirm returns the pending cards to render, and POST with { confirmId, approve } resolves one — the executed action's result comes back as outcome. Integrating through a channel connector instead? The channel confirm relay needs zero confirm code: the user approves by replying 确认 / confirm in the chat.

// Sensitive ops (pay, run a skill, message a friend) are NOT executed inline:
// the companion emits companion.confirm_request and waits.
companion.onConfirmRequest(async (req) => {
  // req = { confirmId, scope, summary, stepUp? }

  // PLATFORM SESSION TOKENS (your end users, minted via /v1/sessions):
  // show your own confirm card and resolve it — this is how confirm-gated
  // custom skills (POST / credentialed) run.
  const approved = await showConfirmCard(req.summary);   // your UI
  await companion.confirmAction(req.confirmId, approved);

  // FIRST-PARTY USER TOKENS (Login with Pouchy): observe-only — approval is
  // authed as the Pouchy user (their app / hosted confirm page), where the
  // stepUp === true money ops get a passkey / Face ID gate.
});

// After a reload, rebuild the card from the still-pending list:
const pending = await companion.pendingConfirms();

Voice

connectCall() opens a live, low-latency voice session over WebRTC. Transcripts fold back into the companion’s memory on close(). The platform’s primary voice route is ElevenLabs Convai, which requires the optional @elevenlabs/client peer dependency (that is the npm package name) — without it the call rejects with call_dependency_missing. Only the OpenAI Realtime fallback route needs nothing extra.

const call = await companion.connectCall({
  voice: 'default',
  locale: 'en',
  bargeIn: true,      // optional: let the user talk over the companion
  onTranscript: (line) => console.log(line.role, line.text)
});

// world-state still flows during the call — the companion reacts live:
companion.sendWorldState({ type: 'game.event', data: 'low_health' });

await call.close();  // ends the voice session; transcript folds into memory

Representative mode (on-behalf-of)

Pass a visitor and the session flips from owner-facing to representative: the companion answers that visitor on the owner's behalf (customer-service style) using only screened owner context — never the owner's private memory, system prompt, or PII.

const c = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: OWNER_PAT,                       // must hold the `represent` scope
  surface: 'support-widget',
  appContext: { name: 'AcmeShop', description: 'Order support' },
  visitor: { id: stableVisitorId, displayName: 'Sam' }   // stable per end-user
});

const ack = await c.connect();            // → { representative: true, … }
await c.sendText('do you ship to Canada?');

REST & OpenAPI

The SDK is a thin wrapper over a documented HTTP API. The full, machine-readable contract is published as OpenAPI 3.1 at /api/companion/openapi.json — generate clients in any language.

# Every endpoint is Bearer-authenticated with a PAT.
curl https://pouchy.ai/api/companion/openapi.json        # machine-readable spec (OpenAPI 3.1)

# 1) open a session…
curl -X POST https://pouchy.ai/api/companion/session \
  -H "Authorization: Bearer $POUCHY_PAT" \
  -H "Content-Type: application/json" \
  -d '{ "surface": "default" }'
# → { "session": "sess_…", … }

# 2) …then send a turn into it
curl -X POST https://pouchy.ai/api/companion/session/$SESSION_ID/input \
  -H "Authorization: Bearer $POUCHY_PAT" \
  -H "Content-Type: application/json" \
  -d '{ "text": "hello" }'

Errors

Failed calls throw a CompanionError with a stable code and message; transport/protocol errors also surface via onError(). Local precondition mistakes throw the same type with status: 0 and a code: not_connected (called before connect()), missing_option (no baseUrl/token), not_representative, reply_timeout (awaitReply's deadline expired), needs_event_stream (an awaitReply turn paused on your declared tools with no start() — the post-resume reply can only arrive on the event stream), aborted (a caller-supplied AbortSignal cancelled the request — every network method accepts an optional signal, SDK 0.29.0), queued_offline (the offline queue accepted the send — thrown only by an awaitReply: true call, which has no reply to await; a plain send returns { queued: true } instead of throwing, SDK 0.34.0), request_timeout (the requestTimeoutMs deadline expired before response headers arrived, SDK 0.35.0). Common HTTP codes: 401 (invalid token), 403 (missing scope — e.g. supplying a visitor without represent), 429 (rate limited).

Server-side failures carry a machine-readable code you can switch on: missing_token, invalid_token, missing_scope, invalid_request, session_not_found, turn_pending, no_pending_tools, unknown_call, payload_too_large, rate_limited, forbidden, unavailable, and — API 1.1 — the confirm/step-up codes confirm_not_found, confirm_resolved, step_up_required, step_up_failed. The vocabulary is append-only; fall back to the HTTP status when code is absent. On rate_limited (429 — turn burst ceiling or demo daily budget) CompanionError.retryAfter carries the server's Retry-After in seconds, so back off exactly that long before retrying (SDK 0.27.0). Since 0.28.0 CompanionError.code is typed as the exported CompanionErrorCodeValue union.

The event stream carries its own control.error vocabulary (also append-only), surfaced via onError(): agent_error — a server-side turn failed after the input was accepted (no reply was produced; safe to re-send); call_mint_failed — start_call couldn't mint the voice-provider credential (the HTTP response errors too; retry later or fall back to text); and stream_unauthorized — synthesized by the SDK when the stream hits a permanent 401/403 and reconnects are exhausted (refresh the token via setToken/onAuthError, then start() again).

Voice connect-step failures (SDK 0.26.0) reject connectCall/startCall with the same CompanionError type and client-synthesized codes: call_unsupported — no WebRTC/microphone in this environment; call_connect_failed — the mic-permission request timed out or the SDP exchange failed (carries the HTTP status when the exchange answered non-2xx); call_dependency_missing — the optional @elevenlabs/client peer dependency isn't installed. Browser-native getUserMedia rejections (e.g. NotAllowedError) propagate untouched.

Support

Questions or an integration in progress? Request access and the team will help you plan capabilities, personas and wiring — or email support@pouchy.ai.