Companion SDK & API reference

Overview

The Companion SDK gives any product its own Agent. The same companion — its memory, personality, voice and avatar — plugs into your surface and can act through tools, reason over your app's live state, and hold a continuous relationship with each user across sessions and devices.

• Standards-first. REST + SSE for messaging, WebRTC for voice, OpenAPI 3.1 for discovery.
• Isomorphic. Runs in the browser and in Node 18+; tree-shakeable, zero required runtime deps.
• Scoped & safe. Every capability — memory, skills, wallet, social, Instant UI — is gated by token scopes; sensitive ops are confirmation-gated, with an optional biometric step-up.
• Beyond the sandbox. The wire contracts are platform-neutral data, so Instant UI renders on web, native iOS/Android or a CLI, and skills can run host-local / device capabilities — not just HTTP.

Not sure which channel you need? Start from what you want to do, not from the API surface: to make something happen in your app, declare an app tool and answer companion.tool_call; to keep the companion aware of live business state, push it to the world-state channel; to render a card a skill author designed, just handle companion.skill_card. The full intent-to-channel map lives in docs/companion-channel-map.md.

Installation

# the SDK (isomorphic: browser + Node 18+)
npm i @pouchy_ai/companion-sdk

# optional — only for ElevenLabs Convai voice; OpenAI Realtime needs nothing extra
npm i @elevenlabs/client

No bundler? Load it from the CDN with an import map:

<script type="importmap">
{
  "imports": {
    "@pouchy_ai/companion-sdk": "https://pouchy.ai/sdk/companion-sdk.js",
    "@elevenlabs/client": "https://esm.sh/@elevenlabs/client@1.9.0"
  }
}
</script>
<script type="module">
  import { createCompanion } from '@pouchy_ai/companion-sdk';
  const c = createCompanion({ baseUrl: 'https://pouchy.ai', token: sessionToken });
  await c.connect();
</script>

Production embeds should pin a version instead of riding the floating URL: /sdk/v<version>/companion-sdk.min.js is immutable (1-year cache, never republished), and the import map's integrity field verifies the exact bytes with the sha384 hash below — generated from this release's committed bundle, so the snippet is always in lockstep with what the URL serves. A minified twin of the floating URL also exists at /sdk/companion-sdk.min.js (~10 KB gzipped).

<script type="importmap">
{
  "imports": {
    "@pouchy_ai/companion-sdk": "https://pouchy.ai/sdk/v0.78.1/companion-sdk.min.js"
  },
  "integrity": {
    "https://pouchy.ai/sdk/v0.78.1/companion-sdk.min.js": "sha384-LBguYYXkZ28ea4VFymhzGETbt36ji57lLZZuXV/kdfl0hp9dzNS5bbZgyzrxifWM"
  }
}
</script>

Drop-in widget (zero code)

Fastest possible integration: one <iframe> gives you a fully rendered chat UI — bubbles, composer, streaming replies — authenticated by the same session token your backend already mints. Optional query params: theme=dark flips the palette, accent=%23rrggbb recolors the send button and user bubbles (text ink is picked automatically for contrast).

<!-- zero-code embed: a full chat UI in one tag -->
<iframe
  src="https://pouchy.ai/embed?token=SESSION_TOKEN&theme=dark&accent=%23ff6b81"
  style="width: 380px; height: 560px; border: 0; border-radius: 16px"
  allow="clipboard-write"
></iframe>

Prefer not to put the token in a URL? Drive the widget over postMessage: wait for pouchy:loaded, send pouchy:init with the token, then listen for pouchy:ready / message / sent / tool_call / error and push pouchy:send / pouchy:context at any time.

// drive it from the parent page instead of putting the token in the URL
const frame = document.querySelector('iframe');
window.addEventListener('message', (ev) => {
  if (ev.data?.source !== 'pouchy-widget') return;
  // 'pouchy:loaded' | 'pouchy:ready' | 'pouchy:message' | 'pouchy:sent'
  // | 'pouchy:tool_call' | 'pouchy:error'
  if (ev.data.type === 'pouchy:loaded')
    frame.contentWindow.postMessage({ type: 'pouchy:init', token: SESSION_TOKEN }, 'https://pouchy.ai');
  if (ev.data.type === 'pouchy:message') console.log('companion said:', ev.data.text);
});
// send on the user's behalf / push live context at any time:
frame.contentWindow.postMessage({ type: 'pouchy:send', text: 'hi!' }, 'https://pouchy.ai');
frame.contentWindow.postMessage({ type: 'pouchy:context',
  event: { type: 'app.scene', data: 'user is on the checkout page', retained: true } }, 'https://pouchy.ai');

The widget pins the embedding page’s origin (learned from the referrer or the first parent message) and only exchanges messages with it — sibling frames and popups are ignored. When you outgrow the canned UI, everything the widget does is available headless through the SDK below.

Quickstart

Two steps: your backend exchanges the project <strong>Secret Key</strong> (dashboard → Keys) for a per-user <strong>session token</strong>, then the client connects with it — four calls and you are talking. First-seen <code>external_user_id</code>s are provisioned automatically. <strong>Important: <code>external_user_id</code> must be a stable id from your account system</strong> — a browser-generated / device-local id mints a brand-new instance (fresh memory) every time it changes.

# 1) YOUR BACKEND — exchange the project Secret Key (dashboard → Keys)
#    for a per-user session. First-seen users are auto-provisioned.
POST https://pouchy.ai/v1/sessions
Authorization: Bearer pchy_sk_…
{ "agent": "<agentId>", "external_user_id": "user_4211" }
→ { "session_token": "pchy_…", "expires_in": 3600, "instance": { … } }

// 2) YOUR CLIENT — connect with the session token and talk (four calls):
import { createCompanion } from '@pouchy_ai/companion-sdk';

const companion = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: sessionToken        // from step 1 — never ship the Secret Key
});

await companion.connect();                       // handshake → HelloAck
companion.onMessage((text) => render(text));     // stream the reply
companion.start();                               // open the reply channel (SSE)

await companion.sendText('hey — what should I do next?');

React

Prefer hooks over wiring the client by hand? @pouchy_ai/react is a thin, typed layer over the same SDK: wrap your tree in a <CompanionProvider> and read live state with hooks. The provider owns the handshake, the auto-reconnecting stream and teardown.

# React hooks over the same SDK (peer deps: react + companion-sdk)
npm i @pouchy_ai/react @pouchy_ai/companion-sdk react

react and @pouchy_ai/companion-sdk are peer dependencies. Pass the provider the same short-lived session token as the vanilla client — useCompanion() exposes status, session and grantedScopes; useMessages() accumulates replies and optimistic user turns; useTyping() is a turn-spanning indicator; useCall() drives live WebRTC voice.

import { useState } from 'react';
import {
  CompanionProvider,
  useCompanion,
  useMessages,
  useTyping,
  useCall          // voice: { start, hangup, interrupt, setMuted, sendText, inCall, muted, speaking, transcript, ... }
} from '@pouchy_ai/react';

function Chat() {
  const { status } = useCompanion();     // 'idle' | 'connecting' | 'connected' | 'error'
  const { messages, send } = useMessages(); // replies + optimistic user turns
  const typing = useTyping();            // spans the whole turn (incl. tools)
  const [text, setText] = useState('');

  if (status === 'connecting') return <p>connecting…</p>;
  if (status === 'error') return <p>connection failed</p>;

  return (
    <>
      {messages.map((m, i) => (<p key={i}><b>{m.role}</b> {m.text}</p>))}
      {typing && <p><i>companion is typing…</i></p>}
      {/* send() rejects (and rolls back the optimistic turn) on failure —
          clear the input only once it succeeds, so the text can be retried. */}
      <form onSubmit={(e) => {
        e.preventDefault();
        send(text).then(() => setText('')).catch(() => { /* keep text to retry */ });
      }}>
        <input value={text} onChange={(e) => setText(e.target.value)} />
      </form>
    </>
  );
}

// The provider owns the handshake, the auto-reconnecting stream and teardown.
// Give it the SAME short-lived session token — never the Secret Key.
export default function App() {
  return (
    <CompanionProvider baseUrl="https://pouchy.ai" token={sessionToken} surface="my-web-app">
      <Chat />
    </CompanionProvider>
  );
}

Anything the hooks do not cover — sendWorldState, recall, onToolCall, setModalities — is on the client returned by useCompanion(), so you never lose access to the full surface documented below.

Svelte & Vue adapters

Svelte and Vue ship as subpath exports of the main package (0.31.0) — thin bindings over one shared, fully-tested view controller. The view tracks streamState, the rolling transcript, the streamed draft, typing, and pendingConfirms as immutable snapshots; sendText appends the user turn optimistically, confirmAction keeps the confirm-card bookkeeping (including the retryable re-approve semantic), and restore: N backfills recent history so a reloaded tab isn't blank.

// Svelte — store contract (Svelte 3 → 5), no extra deps
import { companionStore } from '@pouchy_ai/companion-sdk/svelte';
const companion = companionStore(client, { restore: 20 });
// $companion.transcript / $companion.draft / $companion.typing
// companion.sendText('hey') / companion.confirmAction(id, true)

// Vue 3 — shallowRef snapshot, auto-disposed with the effect scope
import { useCompanion } from '@pouchy_ai/companion-sdk/vue';
const { snapshot, sendText } = useCompanion(client, { restore: 20 });
// snapshot.value.transcript / snapshot.value.streamState

// Any other framework — the same tested controller, bind it yourself
import { createCompanionView } from '@pouchy_ai/companion-sdk';
const view = createCompanionView(client);
const unsub = view.subscribe(() => render(view.getSnapshot()));

You own the client lifecycle in every adapter — create, connect(), start(), and close() it yourself; adapters only observe. On any other framework (or none), bind the same controller via the root export createCompanionView(client). React hosts should use the richer @pouchy_ai/react package above instead.

Python SDK (server-side)

pouchy-companion is the official Python client for the same REST/SSE plane — aimed at server-side integrators (game backends, bots, services). Typed with pydantic v2; the protocol vocabulary is generated from the TypeScript source of truth and drift-tested in CI, so the two SDKs cannot silently diverge. Voice/WebRTC and browser surfaces stay with the JS SDK.

pip install pouchy-companion

from pouchy_companion import CompanionClient, MessagePayload

with CompanionClient(base_url="https://pouchy.ai", token=session_token) as client:
    client.connect()

    # Request/response in one call — streams server-side, returns the reply.
    out = client.send_text("what should I do about the boss?", await_reply=True)
    print(out["text"])

    # Or iterate the event stream (proactive messages, confirms, tool calls).
    # events() is ONE ~45s SSE window and ends cleanly — loop around it; the
    # cursor carries across windows, so nothing is lost between them.
    while True:
        for env in client.events():
            if isinstance(env.payload, MessagePayload):
                print("companion:", env.payload.text)

The client covers text turns (plain or streamed await_reply with on_delta, and an optional turn_id for idempotent retries), the SSE event stream with cursor resume, confirms (pending_confirms / confirm_action), app-declared tools (send_tool_result), world state, memory/knowledge, history and the read-only wallet. Errors raise CompanionError with the same .code vocabulary as the JS SDK. Full reference in the package README on PyPI/GitHub.

Dart / Flutter SDK

pouchy_companion on pub.dev is the official Dart client for the same REST/SSE plane. Pure Dart — no package:flutter import anywhere, so it runs in a Flutter app (iOS / Android / web / desktop), a Dart CLI, or a server-side Dart process, and a drift gate pins that property. The protocol vocabulary is generated from the TypeScript source of truth and drift-tested in CI, like the Python and C# SDKs.

# pubspec.yaml
# dependencies:
#   pouchy_companion: ^0.3.0

import 'package:pouchy_companion/pouchy_companion.dart';

final client = CompanionClient(
  baseUrl: 'https://pouchy.ai',
  token: sessionToken, // minted by YOUR backend — never ship an app key
  appContext: {'name': 'My App'},
);

final ack = await client.connect();
final turn = await client.sendText('hello!', awaitReply: true);
print(turn.reply);

// Or iterate the event stream (messages, confirms, tool calls).
// events() is ONE ~45s SSE window and ends cleanly — loop around it; the
// cursor carries across windows, so nothing is lost between them.
while (true) {
  await for (final env in client.events()) {
    if (env.type == 'companion.message') {
      print(MessagePayload.fromJson(env.payloadMap).text);
    }
  }
}

The client covers the session lifecycle, text turns (plain or awaitReply), the SSE event stream with cursor resume across reconnects, confirms, app-declared tools and world state; setToken swaps a re-minted session token on the live client without losing the stream cursor. Voice/WebRTC is deliberately out of scope — mint calls server-side or use the JS SDK in a webview. Mint session tokens in YOUR backend: a mobile binary is not a secret store. Full reference in the package README on pub.dev.

Unity / C# SDK (preview)

ai.pouchy.companion is the official Unity / .NET client for the same REST/SSE plane — for games and .NET integrators. HttpClient + async/await + Newtonsoft; the wire vocabulary is generated from the TypeScript source of truth and drift-tested in CI. The core CompanionClient has no UnityEngine dependency (unit-testable outside the editor); PouchyCompanionBehaviour is the optional MonoBehaviour that marshals every callback onto the Unity main thread. 0.1.0-preview — the protocol is drift-guarded and the whole package compiles clean in CI under Unity 2021.3's profile; what is not yet proven is runtime behaviour inside a real editor or player.

// Packages/manifest.json — UPM reads npm registries natively:
//   "scopedRegistries": [
//     { "name": "Pouchy", "url": "https://registry.npmjs.org", "scopes": ["ai.pouchy"] }
//   ],
//   "dependencies": { "ai.pouchy.companion": "0.1.0-preview.18" }
using Pouchy.Companion;

var client = new CompanionClient("https://pouchy.ai", sessionToken);
await client.ConnectAsync();

// Request/response in one call — streams server-side, returns the reply.
var reply = await client.SendTextAsync("what should I do about the boss?", awaitReply: true);
Debug.Log(reply.Text);

// Or drop the PouchyCompanionBehaviour on a GameObject and wire OnMessage /
// OnConfirmRequest / OnToolCall in the Inspector — all events fire on the
// Unity main thread.

Covers text turns (plain or streamed awaitReply with onDelta), the SSE event stream with cursor resume, confirms, app-declared tools, world state, memory/knowledge, history and the read-only wallet — the same surface as the other SDKs. Errors throw CompanionError with the shared .Code vocabulary. Voice/WebRTC is out of scope. Full reference in the package README on GitHub.

Pouchy World SDK (server-side)

Everything above builds a companion — one character that talks to one user. Pouchy World is the other product: a story, a game world or a business scenario in which several agents play declared roles inside a pinned Story Package, and your backend drives it one beat at a time. The world holds the state, decides which roles answer, commits the result once, and delivers the lines durably. @pouchy_ai/world-sdk is the official server client for that plane — a different API from the companion SDK on this page, not a feature of it.

// npm i @pouchy_ai/world-sdk — Node only: it holds a secret
import { PouchyWorldClient, newTurnId, describeTurn } from '@pouchy_ai/world-sdk';

const world = new PouchyWorldClient({
  projectId: process.env.POUCHY_PROJECT_ID,
  secretKey: process.env.POUCHY_SECRET_KEY,           // pchy_sk_…
  signing: {                                          // POUCHY-SOURCE-V1
    source: process.env.POUCHY_SOURCE,
    keyId: process.env.POUCHY_SOURCE_KID,
    secret: process.env.POUCHY_SOURCE_SECRET
  }
});

// 1. One player enters the experience. The session is bound to YOUR user id.
const session = await world.createWorldSession({
  environment: envId, role: 'heroine', externalUserId: 'player-42'
});
const worldInstanceId = session.world.instance;   // NOT session.instance.id
// session.session_token is the END-USER token — hand it to your frontend,
// which drives the chat with @pouchy_ai/companion-sdk. It carries no
// project credential; the secret key never leaves this process.

// 2. Drive one beat. turnId IS the idempotency key: re-send it to retry,
//    mint a new one for a new beat.
const turnId = newTurnId();
const beat = await world.runTurn({
  environmentId: envId, worldInstanceId, turnId,
  text: 'I take the last ferry.'
});

// 3. Read the answer without guessing what the fields mean.
const read = describeTurn(beat);
if (read.shouldRetrySameTurn) { /* conflict — re-send the SAME turnId */ }
for (const line of beat.roleMessages) console.log(line.roleId, line.message);
for (const next of beat.nextOptions) console.log('branch:', next.condition);

Two lanes, and the credential is the difference. The MACHINE lane drives sessions, turns and events, and needs a project Secret Key (pchy_sk_…) and a POUCHY-SOURCE-V1 signature by the world's bound provider over the exact raw body — either one alone is refused. The OWNER lane authors and reads (story packages, world definitions, state, turn read-back, replay) on a long-lived project admin key (pchy_admin_…). Neither belongs in a browser or a mobile app: this package is Node-only by design, and the session token your backend mints is the only thing a client should ever hold.

The signature's fourth line is an id slot, and it differs per door — world.request_id from the body on /sessions, turnId on the turns door, eventId on /events. A session mint has neither a turn nor an event, and signing an empty or invented id there verifies perfectly on your side and comes back as a bad signature, which reads exactly like a wrong secret. The client handles this for you; it matters when you sign by hand.

Execution and delivery are different questions. executionStatus says whether the world moved; deliveryStatus says whether the audience has heard yet. A pending delivery is a durable outbox retrying, not a failed turn — the intent was written inside the commit. Re-sending the same turnId re-runs nothing; a new id is a new beat, with new model calls and a new commit.

What a world can do

A companion is one character talking to one person. A world is several characters who share one committed state — so the questions are different ones: who speaks this beat, what actually changed, and who is allowed to know it.

How big a world can be. Up to 24 roles (each bound to its own agent), 100 scenes, 200 nodes, and 900,000 bytes for the whole story package. A bigger cast does not make a beat cost more — at most three roles speak, and a role that is not on stage never enters another role’s prompt — but the beat has to choose who is in the room: pass focusRoles. Every limit, and the reason for it: size limits.

Reading a turn result

Four fields answer questions people assume are one question. Getting them confused is the commonest first integration bug, so they are worth reading once before you need them.

The full contract is published as OpenAPI at pouchy.ai/v1/world/openapi, and the package README carries the reference. When a signed door refuses you, the reason is not on the wire — that would make it an oracle — it is in your project's own audit trail, readable at the world's preflight endpoint, which takes a signed-in admin's token rather than the keys your backend holds.

Deeper: the documentation index routes you to the quickstarts (a drama, an NPC, a Next.js app), the error guide, and the production-workflow integration doc.

Authentication & scopes

Every request authenticates with Authorization: Bearer <token>. The standard client credential is the session token your backend mints via POST /v1/sessions (see Quickstart / Platform). A Personal Access Token (PAT), also pchy_-prefixed, is the advanced alternative for personal or single-user integrations — mint it server-side and never ship a long-lived token in client code.

// The client always holds a SHORT-LIVED token: a session token from
// POST /v1/sessions (the standard path), or a server-minted PAT for
// personal/advanced integrations. Never ship a long-lived key in client code.
const companion = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: sessionToken,              // Authorization: Bearer <token> on every call
  modalities: ['text', 'voice'],    // intersected with the token's grant
});

Sensitive capabilities are opt-in per token scope — e.g. wallet.spend, skills.execute, social.message, memory.*, and the representative scopes below. Instant UI rendering rides the non-sensitive ui.render scope. The handshake returns the effective grantedScopes.

Platform: sessions & Admin API

For developers and enterprises building on the dashboard: a project holds agent templates; every end user gets their own instance of an agent (separate memory, relationship progress, wallet). Your end users never handle keys — your backend holds a project Secret Key (pchy_sk_…, or pchy_sk_test_… for the unmetered test environment) and exchanges it per end user for a short-lived session token. First-seen external_user_ids are auto-provisioned. Important: external_user_id must be a stable id from your account system — a browser-generated / device-local id mints a brand-new instance (fresh memory) every time it changes.

# Your BACKEND exchanges the project Secret Key for a per-user session.
# First-seen external_user_ids are auto-provisioned — zero user setup.
POST /v1/sessions
Authorization: Bearer pchy_sk_…
{ "agent": "<agentId>", "external_user_id": "user_4211" }

→ { "session_token": "pchy_…", "expires_in": 3600,
    "instance": { "id": "…", "external_user_id": "user_4211", "created": true } }

// Client: connect the SDK with the session token — same API as a PAT.
const companion = createCompanion({ baseUrl: 'https://pouchy.ai',
  token: sessionToken, modalities: ['text'] });

The returned session_token is an ordinary bearer for the client SDK — pass it as token to createCompanion(). Sessions carry the non-sensitive default scopes and can only be narrowed; agent-level capabilities widen them — setting genui: true on the template adds ui.render (Instant UI) to every session it mints, and a skills list of built-in catalog slugs pre-installs those skills on every instance with skills.execute granted (v1: credential-free public read APIs — dictionary, wikipedia, exchange-rates, …; they auto-run without a confirm round-trip). A social policy ({ pair, crossProject }) opts instances into agent-to-agent pairing + text messaging via the SDK's pairVisitor flow — enforced server-side at pair creation; wallet interactions stay excluded. Templates also carry scripted scenes (story lines with triggers + beats, compiled into a pinned prompt section every turn), per-language voice defaults (voices, ElevenLabs id preferred on calls, OpenAI preset for TTS/fallback) and an initialStage relationship seed applied to new instances only. A wallet flag gives instances a READ-ONLY wallet (balance + own deposit address via wallet.read; end users fund their own instance, paired friends can send to it — instances can never spend). Projects additionally carry a shared knowledge corpus (POST /v1/projects/{id}/knowledge): upload product docs / FAQ / lore once, and every instance recalls the relevant chunks semantically alongside the user's personal memory, cited by document name. Plans cap monthly active users (free tier: 10; a new live user beyond the cap gets 402 limit reached — existing users keep working; test keys are quota-exempt).

Everything the dashboard does is also available programmatically via the Admin API with a project admin key (pchy_admin_…): list/create agents on /v1/admin/agents, read/update a template on /v1/admin/agents/{agentId} (updates bump templateRev; live instances re-apply the persona on their next session). Key types are strictly separated: secret keys can't manage, admin keys can't mint sessions.

# Manage the project from your backend — the dashboard is optional.
# Authorization: Bearer pchy_admin_…   (project implied by the key)
GET/POST/PATCH/DELETE /v1/admin/agents[/{id}]   # templates incl. draft/publish
GET/POST /v1/admin/keys · DELETE /{keyId}       # secret keys (plaintext once)
GET /v1/admin/users · PATCH/DELETE /{iid}       # suspend / GDPR-erase end users
GET/POST /v1/admin/webhooks · DELETE /{whid}    # event push endpoints
GET/POST /v1/admin/channels · PATCH/DELETE /{id}# deploy into Telegram/Slack/…;
                                                #  config.groupMode → one shared
                                                #  brain for a whole GROUP chat
GET /v1/admin/logs · /usage · /billing          # audit, month usage, plan (RO)
GET/PATCH /v1/admin/project                     # rename / archive
GET /v1/admin/openapi                           # OpenAPI 3.1 spec (public)
GET /v1/projects/openapi                        # console-plane spec (public)
# Typed client:  npm i @pouchy_ai/admin-sdk  →  createAdminClient({ adminKey })

Industry templates. Creating a project with { "template": "romance" | "game" | "hardware" | "ecommerce" | "life" } seeds 1–2 polished agent presets so the first session works out of the box — pick one when creating your project in the dashboard, too. Agents also carry a publish lifecycle — status: "draft" refuses live-key mints while test keys keep working, so you can tune a persona safely before going live.

Agent Data. Projects can declare a capability data world — Views (authoritative reads, curated to declared fields), Actions (user-confirmed external mutations, idempotent at your backend by a runtime-minted actionId, with honest unknown outcomes) and Events (deduped "the world changed" signals your backend POSTs to wake the agent) — in the dashboard's Capabilities page or via POST /v1/projects/{id}/capabilities, then grant it per agent with the template's data flags. Every call to your backend carries a signed actor assertion (X-Pouchy-Actor), so you always know who is asking without handing identity to the model. Full integration contract: the API reference's Data capabilities section.

Core concepts

Session & surface

Each surface (e.g. "game", "support-widget") is one resumable session. Reconnecting resumes from a cursor, so replies are never lost.

World-state

A stream of CloudEvents-shaped context (retained state + transient events) that grounds the companion in what is happening right now.

Modalities

Text and voice. Requested modalities are intersected with what the token grants.

Tools

Actions your app declares and performs on the companion's request — the bridge from "talk" to "do".

Capabilities

The companion exposes six capabilities over the SDK. Each is gated by a token scope, and each produces (or consumes) a platform-neutral payload — so the web app is just one host: a native iOS/Android app, a game engine, or a CLI can use the same capability by implementing the matching renderer/executor once.

The full scope vocabulary ships typed in the SDK — import COMPANION_SCOPES, SENSITIVE_SCOPES, DEFAULT_SCOPES and check grants with hasScope(ack.grantedScopes, 'skills.execute') instead of copying strings from this page. hasScope is an exact-grant check; to gate whether an API call will actually succeed, prefer grantsScope(granted, required), which also honours scope subsumption — a wallet.spend-only key can read the wallet, so grantsScope(g, 'wallet.read') stays true where hasScope would wrongly return false.

createCompanion(options)

Returns a CompanionClient. Options:

Client methods

connect(): Promise<HelloAck> / sessionId: string | null

Opens the session and performs the handshake. Returns the granted scopes, negotiated modalities, a resume cursor, (for representative sessions) the visitor-pairing state, and pendingToolCalls (0.37.0) — the outstanding tool calls of a turn that paused before this handshake (empty array when none), so an embed reloaded mid-pause can complete the turn (see the replayPendingToolCalls option). Since 0.77.0 the ack also carries droppedTools: [{ index, reason }] for any tools entry the handshake refused (reason missing_name, duplicate_name or over_cap, more than 32), so a mistyped tool is named instead of silently never firing. After it resolves, the read-only sessionId accessor holds that session’s id (null before the first connect). endSession(), close() and setToken() never clear it, so use it to correlate client logs with server records, not to check whether the session is still live.

start(): void / stop(): void

Open or close the inbound reply channel. SSE by default; stream: "websocket" receives the same frames over one socket when a WebSocket gateway serves the session (a self-hosted worker — pouchy.ai itself serves no socket, so against it the client falls back to SSE, observable as streamState "degraded_sse"). Call start() once after connect().

streamState: CompanionStreamState / onStreamStateChange(handler: (state, prev) => void): () => void

Receive-stream lifecycle (0.30.0): idle | connecting | connected | reconnecting | degraded_sse | stopped. The handler fires on change only, with (state, prev) — drive a connection indicator without hand-rolling one. degraded_sse means the WebSocket transport errored and delivery continues on the SSE fallback; stopped covers stop()/close() and a permanent stream_unauthorized failure. A terminal stream 404 / 410 / 400 (stream_rejected) also ends on stopped.

onMessage(handler: (text, envelope) => void): () => void

Subscribe to streamed companion replies. Returns an unsubscribe function.

sendText(text: string, opts?): Promise<{ seq: number | null; queued?: boolean; turnId?: string }> / sendText(text, { awaitReply: true }): Promise<SendTextReply>

Send a user message. The reply arrives on the stream as a companion.message — subscribe via onMessage(). Or pass { awaitReply: true } and the promise itself resolves with the completed turn's reply { seq, text, envelope } — request/response hosts (a CLI, an HTTP handler, a test) skip the onMessage wiring entirely; works without start(). Since 0.63.0, opts.instructions carries developer-authored rules for THIS turn — joined to the prompt and NEVER scored as user speech by inbound moderation (needs the chat.instructions scope; 32,000 chars max; not persisted, so send it again next turn if it still applies). And a turn the moderation gate refused now says so: blocked, blockedBy (classifier | term), blockedCategories and blockedWindows ride the reply, absent — never false — on an ordinary turn. The chat.instructions scope cannot be requested at mint time — naming it in the session-mint scopes is silently ignored. Enable it on the agent template instead (Dashboard → Agents, or PATCH the agent with instructions: true); newly minted session tokens for that agent then carry it.

sendText(text, { turnId }): Promise<{ seq; turnId; duplicate?: true }>

Retrying a turn safely (0.72.0): every result carries the turnId it went out under, and sendText(text, { turnId }) sends your own (1–256 characters; anything else rejects with code invalid_option before sending). After an ambiguous failure — reply_timeout, request_timeout, a dropped stream, a 5xx — retry under the SAME turnId: the server replays the completed turn with duplicate: true instead of running and billing it again. A retry without it runs a second turn.

onDelta(handler: (chunk, { reset? }) => void): () => void

Token streaming: registering makes sendText stream the reply as it is generated — the handler fires per text chunk (reset = clear the partial render), and onMessage still fires exactly once with the authoritative final text. Force per call with sendText(text, { stream: true | false }).

sendWorldState(input: WorldStateInput | WorldStateInput[]): Promise<{ accepted, dropped, injected, reacted, duplicates }>

Push live context — { type, data, retained? }. Retained values persist for the session; transient ones are one-off signals.

connectCall(opts?): Promise<CompanionCall> / startCall(opts?): Promise<CallCredentials>

Start a live, low-latency voice session over WebRTC. Returns a call handle with .close(); declared tools + host control actions stay available in-call. Pass { bargeIn: true } to keep the mic open while the companion speaks so the user can interrupt mid-utterance (default is half-duplex, which protects speakerphone use from self-echo interruptions). On the ElevenLabs path the mint also returns voiceToolToken (0.46.0), the credential for the server-side tool that recalls memory MID-CALL; connectCall passes it for you, while a self-plumbed startCall host must send it under BOTH dynamic variable names (0.47.0): dynamicVariables: { [VOICE_TOOL_VARIABLE]: creds.voiceToolToken, [VOICE_TOOL_VARIABLE_PLAIN]: creds.voiceToolToken }. ElevenLabs ignores a client-supplied secret__ variable, so the plain twin is the one that reaches the tool header — omitting it raises no error, it just leaves the companion unable to recall anything.

getCallRecording(callGen): Promise<CallRecordingResult> / deleteCallRecording(callGen): Promise<{ deleted }>

Whole-call recording (0.65.0). connectCall({ record: true }) records both voices on the voice provider's side, so a native app that cannot capture call audio on the device gets the same file as a browser. Recording is turned on per agent in the dashboard (the agent's recording switch) and runs on ElevenLabs only; call.recording says whether this call is recorded, and call.recordingUnavailable why not. After the call, getCallRecording(call.callGen) returns { state: 'ready', audio } (MP3), 'pending' with retryAfterSec, or 'none' | 'expired' | 'deleted'. Recordings are kept for 7 days — copy them to your own storage. deleteCallRecording(callGen) removes the provider's copy.

connectCall({ avatar: { faceId } }) → call.video?: CallVideo, call.videoUnavailable?: CallVideoUnavailable

Video avatar (0.75.0). connectCall({ avatar: { faceId } }) puts a talking face on the call: the server opens a Simli session bound to a face registered for your project and returns it as call.video ({ provider, faceId, sessionToken, iceServers, maxSessionLength, model }), or call.videoUnavailable when there is none (not_enabled, not_configured, face_not_allowed, face_not_ready, mint_failed); the call itself connects either way. You draw the face with simli-client and feed it the companion's remote audio track. Needs the call.video scope, which your project gets once a face is registered for it — withhold it per reader with withhold_scopes to keep the voice and drop the face. Billed per minute on top of the voice. avatar.model (0.76.0) picks Simli's animation model — fasttalk (default) or artalk — and call.video.model says which one ran; same rate either way. You build faces yourself with your admin key (admin-sdk createVideoFace, or POST /v1/admin/video-faces): building is off until Pouchy enables it for your project (403 video_face_build_not_enabled until then), and a scheduled sweep marks a finished face ready within about 10 minutes, so there is nothing to poll.

connectCall({ brief?: string }) → call.worldBrief?: 'included' | 'no_story' | 'unavailable', call.privateStory?: 'tool' | 'none', call.trimmed?: { story?, context? }

Tell the call what it is about (0.67.0). connectCall({ brief }) is your application's note on this call ("she just got the acceptance letter"): the companion opens on it and keeps to it, and never reads it out. Up to 1,000 characters; it needs the chat.instructions scope (the agent's per-turn instructions switch), so send it from a token your backend minted. A call made with a Pouchy World role's session token also carries the story's public state from the server (scene, story clock, flags, the facts already revealed, its own last beats), never the role's secrets or unrevealed facts, because a call's instructions reach the reader's device; call.worldBrief says 'included', 'no_story' or 'unavailable'. The call does not change the story: fold it back in as a beat with trigger.context and proposedPatches.

call.setMuted(muted): void / call.isMuted(): boolean / call.canMute / call.sendText(text): boolean

The user's own mute and typed input on a call (0.69.0). call.setMuted(true) mutes the microphone until setMuted(false), and the mute holds across the companion's lines: the half-duplex re-open never undoes it. connectCall({ muted: true }) opens the call muted. Show a mute control only when call.canMute is true. call.sendText(text) sends a line the user typed as their turn: the companion answers it as something they said, it reaches onTranscript as role: 'user', and it is kept in the memory consolidated at hang-up. It cuts an utterance in progress, like speaking would, and the provider sees nothing before you call it. Returns false (nothing sent) for blank text, after close(), or on a closed channel. injectEvent stays a context line and is never reported as the user's words.

call.interrupt(): void

Silent cut (0.36.0): stop the utterance currently being spoken WITHOUT generating new speech — for real-time commentary going stale at a beat boundary. Idempotent (no-op when idle). Tiers by call.provider: openai-realtime is guaranteed (response.cancel + output-buffer clear); elevenlabs-convai is best-effort (user_activity signal + guarded internal buffer cut — no public EL stop API; injectEvent(text, true) remains the guaranteed-but-speaking preemption).

onToolCall(handler): () => void / sendToolResult(callId, result)

Receive companion.tool_call for the tools you declared; argsJson is the args pre-parsed as an object (undefined on empty/malformed args; the raw JSON string still rides in args). Reply with sendToolResult(callId, { ok, result }); the turn resumes once every call is reported (the result apply is idempotent per callId). replayed: true (0.37.0) marks the re-delivery of a still-outstanding call after a mid-pause reconnect — treat the call id as an idempotency key for side-effecting tools.

onRender(handler): () => void

Instant UI — receive companion.ui_action. payload.interface is the platform-neutral genui schema; draw it with your own renderer (web / native iOS+Android / CLI). Requires the ui.render scope.

onInterfaceUpdate(handler): () => void

Receive companion.ui_update — a live { panelId?, updates:[{key,value}] } write into an already-rendered panel. Apply it to the panel state; no rebuild.

onSocialMessage(handler): () => void

Receive companion.social_message — an inbound A2A message from a paired friend, delivered cross-app to any embed holding social.message.

onConfirmRequest(handler): () => void

Subscribe to companion.confirm_request — a sensitive op awaiting approval ({ confirmId, scope, summary, summaryKey?, summaryVars?, stepUp? }). Render summaryKey with summaryVars to show the approval in the user's language, falling back to the English summary. Platform session tokens resolve it with confirmAction; first-party user tokens are observe-only (approval is authed as the Pouchy user, where the biometric/passkey gate lives).

confirmAction(confirmId, approve): Promise<{ status, outcome?, retryable?, outcomeClass? }>

Resolve a pending confirmation (platform session tokens only). Show the event's summary, collect an explicit tap, pass the decision; on approve the recorded action runs server-side and its result returns as `outcome` in the response (render it where the user tapped) and as a normal companion.message on the stream. Single-use: re-resolving a settled confirmId fails with 409 — EXCEPT an approved idempotent action (a wallet payment) that flaked transiently returns `status:'exec_failed'` + `retryable:true`, which you may re-call with the same confirmId to re-run (deduped). `outcome` is a short natural-language summary for display — never the upstream/MCP payload, and not a parseable shape. Switch on `outcomeClass` (0.55.0) — `success` / `error` / `unknown` / `rejected` / `denied` / `unrecognized` — to tell an action that ran from one that failed or was refused; `status:'approved'` alone only says the approval was accepted.

getMandate(confirmId): Promise<MandateDocument | null>

The full user-signed approval document behind a confirm’s mandate receipt (0.59.0): { confirmId, status, intent, intentDigest, mandate, credential: { credentialId, publicKey }, verify: { challenge, rpID, origin }, verified }. A counterparty verifies it offline against the keys the handle publishes at /a2a/{handle}/mandate-keys. Resolves null on mandate_absent (a deny, an approval that needed no step-up, a row from before the feature). credential and verified are both null when the passkey has since been removed from the account (0.59.1).

pendingConfirms(): Promise<PendingConfirm[]>

The session's still-pending confirmations (display-safe: confirmId, scope, summary, summaryKey, summaryVars, createdAt, plus status, execUnsettled and stepUp since 0.61.0) — rebuild your confirm card after a reload, since confirm_request events are not replayed. status 'exec_failed' is one the user ALREADY approved whose run flaked: approving again retries the same request rather than starting a new one, and execUnsettled says that flake was the deadline so it may still complete on its own. stepUp means the approval will demand a passkey / Face ID. Session tokens only.

onAudio / onExpression / onUsage(handler): () => void

Typed subscriptions for companion.audio (a TTS clip of the reply — { url }, a capability URL of an mp3), companion.expression (an avatar cue — { expression }, a VRM 1.0 preset read from the reply’s tone) and control.usage (per-turn token metering). Audio and expression fire after a text turn only when the agent opted in (dashboard → reply cues, the agent’s replyCues field); audio also needs the session to have negotiated voice and the token to hold the voice scope. Neither fires on A2A or channel surfaces.

recall(opts?: { query?: string; limit?: number }): Promise<RecalledMemory[]>

Read back the memories relevant to this session — content, kind, importance and namespace. Pass query to run semantic search over them server-side; omit it for the plain importance/recency ranking.

remember(fact) / ingestKnowledge(doc) / ingestFile(file)

Write memory this token is authorized for: remember stores one short fact in the app namespace; ingestKnowledge distils an already-extracted document (PDF body, transcript, notes) into a summary + embedded recallable chunks; ingestFile hands Pouchy a raw data: URL (PDF / audio / image) to understand server-side first. The two ingest calls surface in "My materials" and need the memory.write:core scope.

forget(id: string): Promise<{ forgotten: boolean; alreadyForgotten: boolean }>

Forget ONE fact by the id a recall row carries — the user's authoritative "stop telling me this": it stops recalling and stays forgotten. Needs the namespace's write scope; anything the token cannot see answers the same 404; idempotent.

history(opts?: { limit?: number }): Promise<CompanionTurn[]>

Fetch this session's recent conversation turns ({ user, assistant, ts }, oldest→newest) so a reconnecting embed can restore its transcript. Distinct from recall (durable memory/facts) — this is the raw exchange log. Own session only; limit default 20, max 50.

setModalities(modalities: string[]): Promise<{ modalities }> / ping()

setModalities(modalities): change the live session's I/O modalities mid-session (e.g. toggle voice); intersected with the token's grant server-side, returns the effective set. ping(): keepalive that bumps the session's last-seen time so a long-idle embed stays live within the TTL.

setToken(token: string): void

Swap the bearer token used by every subsequent request and stream reconnect — session tokens expire (1h by default), so call this when your backend re-mints one, or use the onAuthError option to refresh on demand.

getAvatar(): Promise<CompanionAvatar> / brandIconUrl(size?)

Fetch the live avatar (3D VRM URL + 2D portrait, archetype, display name) to render your own front-end, and the Pouchy brand icon URL.

getWallet(): Promise<CompanionWallet>

Read the instance's own wallet — balances + total USD. Read-only and receive-only; needs wallet.read (or wallet.spend, which subsumes it — gate the affordance with grantsScope, not hasScope).

pendingOutbox() / onOutboxChange(cb) / flushOutbox(): Promise<{ sent, remaining }>

Offline send queue (0.34.0, opt-in via queueOffline): inspect the durable outbox with pendingOutbox(), subscribe to changes with onOutboxChange(), and force a flush attempt with flushOutbox().

pairVisitor(visitorToken: string): Promise<PairVisitorResult> / getPairRequest(id): Promise<PairVisitorResult>

Representative mode only: pair a visitor who is also a Pouchy user to unlock agent-to-agent (A2A) context. Requires the represent:pair scope. Since 0.74.0 a new pairing returns a request: give the visitor its approveUrl to approve on pouchy.ai, then poll getPairRequest(id) until paired.

endSession(opts?: { transcript?, callGen? }): Promise<EndSessionResult | null> / lastEndSessionError: EndSessionError | null

Cleanly end the session and optionally fold a transcript into long-term memory. Returns the server diagnostic { ok, facts?, extracted?, writeError?, skipped? } (or null) — skipped tells you why nothing was written. Also closes the server's call-active window — generation-fenced since 0.41.0: it sends the callGen of the last call this client minted (null if none), so a text client's teardown never clears a live call's window; self-plumbed integrations pass startCall's creds.callGen explicitly. extracted (0.48.0) is how many facts the extraction PROPOSED vs facts stored — extracted > 0 with facts: 0 means the store refused the writes, and writeError says why (e.g. a missing memory-write scope). Since 0.56.0 the diagnostic also carries filtered (proposed facts the identifier filter refused, so extracted − facts is not read as a refused write) and skipped: 'no_visitor_scope' (a representative session whose token lacks represent:remember — nothing was consolidated anywhere). Since 0.62.0 it also carries merged — how many of facts folded into a memory the store ALREADY held instead of minting a new one (write-time dedup), so facts − merged is the count of NEW memories. That is the ordinary path, not an edge one: a repeated endSession() re-reads the same recent turns, so a second call re-extracts what it already stored and reported it as growth. revived is the subset of merged whose target had been archived by memory decay and was brought back.

close(): Promise<EndSessionResult | null>

One-call teardown for a text session: stop the stream and run endSession() once (idempotent). Returns the same consolidation diagnostic.

onError(handler): () => void

Subscribe to transport / protocol errors ({ code, message }).

Events

Everything the companion does flows back as typed events on the stream you opened with start(). Subscribe with on(type, fn) (or '*'), or use the typed convenience helpers below. Each returns an unsubscribe function. Unknown event types are ignored, so the protocol is forward-compatible.

companion.onMessage((text) => render(text));                  // streamed reply
companion.onRender(({ interface: ui }) => myRenderer.draw(ui)); // Instant UI panel
companion.onInterfaceUpdate(({ update }) => myRenderer.apply(update));
companion.onSocialMessage(({ fromName, content }) => notify(fromName, content));
companion.onConfirmRequest((req) => showApproval(req));         // req.stepUp ⇒ gate with Face ID
companion.onAudio(({ url }) => play(url));                      // TTS clip of the reply (0.58.0; opt-in per agent)
companion.onToolCall(async ({ id, name, args }) => { /* … */ });
companion.start();                                              // open the event stream

World-state

Stream live context with sendWorldState({ type, data, retained? }). Retained values represent current state; transient ones are one-off signals. This is what lets the companion say "boss incoming — heal up" at the right moment.

// Retained state — the latest value persists for the session:
companion.sendWorldState({ type: 'game.player', data: { hp: 12, level: 7 }, retained: true });

// Transient event — a one-off signal:
companion.sendWorldState({ type: 'game.event', data: 'boss_appeared' });

// The companion reasons over this live context (and, in voice, reacts in real time).

Tools & actions

Declare the actions your surface can perform; the companion calls them and you return a result. The same handler serves both text and voice sessions.

const companion = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: PAT,
  tools: [
    {
      name: 'highlight_product',
      description: 'Visually highlight a product card in the host UI',
      parameters: { type: 'object', properties: { id: { type: 'string' } }, required: ['id'] }
    }
  ]
});

// The companion CALLS your tool; you perform it locally and report the result.
// argsJson is the pre-parsed args object (undefined if the args were invalid);
// reply with sendToolResult(callId, …).
companion.onToolCall(async ({ id, name, argsJson }) => {
  if (name === 'highlight_product') {
    const { id: productId } = (argsJson ?? {}) as { id?: string };
    highlight(productId);
    await companion.sendToolResult(id, { ok: true });
  }
});

// Also available out of the box on connectCall() voice sessions (text turns
// expose only the tools[] you declared — declare these there yourself if you
// want them outside calls; see docs/companion-host-control.md):
// - HOST_CONTROL_TOOLS — universal verbs (invoke_action, set_feature,
//   set_value, navigate, highlight) offered automatically on calls once you
//   declare tools/handles, so the companion can drive generic host actions.
// - AVATAR_VISUAL_TOOLS — play_gesture / play_expression for embeds that
//   render the VRM avatar; DECLARE them to receive the calls (undeclared
//   they no-op silently).

Instant UI

The companion can render an interactive panel on your surface, not just speak text — a form to collect a few values, a summary card with a progress bar, a chart, a set of choices. It arrives as companion.ui_action; you draw payload.interface with your own renderer.

The payload is platform-neutral JSON — a tree of ~20 typed atoms (Text, Slider, Select, Chart, Table, Group…) plus a state bag — so the same panel renders on web, native iOS (SwiftUI), Android (Jetpack Compose), or a terminal. The renderer is the only platform-specific part; the contract never changes. Input atoms write to state; set reportChanges and the user's edits stream back as a turn (the live-form loop), and companion.ui_update applies live value changes without a rebuild.

// The companion renders an interactive panel; YOU draw the platform-neutral
// schema with your own renderer (web component, SwiftUI, Jetpack Compose, CLI).
let panel: ReturnType<typeof renderInterface>;
companion.onRender(({ interface: ui }) => {
  panel = renderInterface(ui, mountEl, {
    // If the panel set reportChanges, stream the user's edits back as a turn:
    onReportChanges: (state) => companion.sendText('I adjusted: ' + JSON.stringify(state))
  });
});

// Live, in-place updates — the companion changes a value without a rebuild:
companion.onInterfaceUpdate(({ update }) => panel.applyUpdate(update));

Reference renderers (vanilla web, SwiftUI, Jetpack Compose) and the full atom contract live in the Instant UI renderer guide. Gated by the ui.render scope.

Already have a renderer built against Google's A2UI catalog? Since SDK 0.42.0 the projection ships in the package: toA2UI(payload.interface) emits the panel as an A2UI v0.9 surface (standard component/field names; Pouchy-only concepts ride a _pouchy extension A2UI renderers ignore), and fromA2UI ingests one back. INSTANT_UI_NODE_TYPES lists the atom catalog as data.

Social (agent-to-agent)

The companion can message the user's paired friends on their behalf (send_friend_message / message_friends, confirmation-gated), read a friend thread (read_friend_messages), and — the cross-app half — surface an inbound friend message to your embed in real time via companion.social_message. So a companion in App A can message a friend whose companion runs in App B, and App B receives it. Requires the social.message scope.

// Inbound A2A: a message from one of the user's PAIRED friends, delivered
// cross-app to any embed whose token holds social.message (not just Pouchy).
companion.onSocialMessage(({ fromUid, fromName, content, createdAt }) => {
  showFriendMessage(fromName, content);
});

// The companion can also SEND on the user's behalf (confirm-gated):
await companion.sendText('tell Mom I will be late');  // → message_friends / send_friend_message

Beyond paired friends, every companion with a claimed @handle is reachable by external A2A agents over JSON-RPC 2.0 at POST /a2a/{handle}/rpc (discovery: /a2a/{handle}/.well-known/agent-card.json). message/send runs one turn and answers a Message, or a Task when the turn parked an owner confirm or the caller sent configuration.blocking: false; tasks/get and tasks/cancel read and settle it; message/stream and tasks/resubscribe answer a one-shot SSE response of status and artifact events; tasks/pushNotificationConfig/set|get|list|delete register an https webhook per task. The Agent Card says streaming: true and pushNotifications: true for exactly these. No SDK helper wraps this yet — call it as JSON-RPC.

Confirmations & biometric step-up

Sensitive actions — a payment, running a skill, messaging a friend — are never executed inline. The companion emits companion.confirm_request and waits. Who approves depends on the token: a platform session token (an end-user instance minted via /v1/sessions) resolves it directly with confirmAction — your end user is that account's only human, and this is how confirm-gated custom skills (POST / credentialed) run. A first-party user token is observe-only: approval happens on the user's own Pouchy surface (their app or the hosted confirm page, authed with their own login), never in a third-party DOM.

The request carries an advisory stepUp flag (set for irreversible money ops). When true, the first-party approval surface requires a stronger gesture — a passkey / Face ID / Touch ID — before approving, verified server-side via WebAuthn. Users without a passkey, or with step-up disabled, fall back to a tap; the custodial flow is unchanged.

Since 0.57.0 an approval that passed the passkey step-up returns a mandate — { intentDigest, credentialId, signedAt }: the passkey signed a challenge bound to the confirm's canonical intent, so the receipt says what was approved, not merely that someone was present. The signature stays on the server.

Driving the API with plain REST (no SDK)? The confirm card is session state, not part of the /input response: after a turn, GET /api/companion/session/{sid}/confirm returns the pending cards to render, and POST with { confirmId, approve } resolves one — the executed action's result comes back as outcome. Integrating through a channel connector instead? The channel confirm relay needs zero confirm code: the user approves by replying 确认 / confirm in the chat.

// Sensitive ops (pay, run a skill, message a friend) are NOT executed inline:
// the companion emits companion.confirm_request and waits.
companion.onConfirmRequest(async (req) => {
  // req = { confirmId, scope, summary, stepUp? }

  // PLATFORM SESSION TOKENS (your end users, minted via /v1/sessions):
  // show your own confirm card and resolve it — this is how confirm-gated
  // custom skills (POST / credentialed) run.
  const approved = await showConfirmCard(req.summary);   // your UI
  await companion.confirmAction(req.confirmId, approved);

  // FIRST-PARTY USER TOKENS (Login with Pouchy): observe-only — approval is
  // authed as the Pouchy user (their app / hosted confirm page), where the
  // stepUp === true money ops get a passkey / Face ID gate.
});

// After a reload, rebuild the card from the still-pending list. Each row says
// which kind it is, so the rebuilt card can be as honest as the live event:
const pending = await companion.pendingConfirms();
// → [{ confirmId, scope, summary, createdAt, status, stepUp, execUnsettled? }]
//   status 'pending'      — never answered; the ordinary approve card.
//   status 'exec_failed'  — ALREADY approved and the run flaked. Approving
//                           again retries the SAME request (keyed on
//                           confirmId), it does not start a new one.
//   execUnsettled: true   — that flake was the execution DEADLINE, so it may
//                           still complete on its own. Say so before retrying.
//   stepUp: true          — the approval will demand a passkey / Face ID.

Voice

connectCall() opens a live, low-latency voice session over WebRTC. Transcripts fold back into the companion’s memory on close(). The platform’s primary voice route is ElevenLabs Convai, which requires the optional @elevenlabs/client peer dependency (that is the npm package name) — without it the call rejects with call_dependency_missing. Only the OpenAI Realtime fallback route needs nothing extra.

connectCall({ audioTags: true }) (0.64.0) lets the companion perform short voice cues on an ElevenLabs call — it laughs, sighs, whispers and speaks softly instead of only saying words (Eleven v4). The transcript then contains the cues, so connectCall strips them from onTranscript and from memory; if you open the ElevenLabs session yourself, pass each line through stripAudioTags(). Without the flag nothing changes.

To see what was performed (0.70.0): on a call where audioTags was granted, every companion line in onTranscript carries audioTags, the cues it performed in order (['sighs', 'laughs']), or [] when it performed none. The field is absent on user lines and on calls without the grant. Self-plumbed sessions get the same list from extractAudioTags().

Why a call ended (0.71.0): pass onEnd and it fires once per call, on every path, with reason — 'user' (your close()), 'agent' (the companion hung up with ElevenLabs’ end_call tool), 'remote' (the provider closed the call cleanly without saying why) or 'error' (a failure; onError received it first). With onEnd, a clean hang-up from the far side no longer reaches onError; without it nothing changes. The call is still sent to /end on every end.

const call = await companion.connectCall({
  voice: 'default',
  locale: 'en',
  bargeIn: true,      // optional: let the user talk over the companion
  onTranscript: (line) => console.log(line.role, line.text)
});

// world-state still flows during the call — the companion reacts live:
companion.sendWorldState({ type: 'game.event', data: 'low_health' });

await call.close();  // ends the voice session; transcript folds into memory

Representative mode (on-behalf-of)

Pass a visitor and the session flips from owner-facing to representative: the companion answers that visitor on the owner's behalf (customer-service style) using only screened owner context — never the owner's private memory, system prompt, or PII.

const c = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: OWNER_PAT,                       // must hold the `represent` scope
  surface: 'support-widget',
  appContext: { name: 'AcmeShop', description: 'Order support' },
  visitor: { id: stableVisitorId, displayName: 'Sam' }   // stable per end-user
});

const ack = await c.connect();            // → { representative: true, … }
await c.sendText('do you ship to Canada?');

REST & OpenAPI

The SDK is a thin wrapper over a documented HTTP API. The full, machine-readable contract is published as OpenAPI 3.1 at /api/companion/openapi.json — generate clients in any language.

# Every endpoint is Bearer-authenticated with a PAT.
curl https://pouchy.ai/api/companion/openapi.json        # machine-readable spec (OpenAPI 3.1)

# 1) open a session…
curl -X POST https://pouchy.ai/api/companion/session \
  -H "Authorization: Bearer $POUCHY_PAT" \
  -H "Content-Type: application/json" \
  -d '{ "surface": "default" }'
# → { "session": "sess_…", … }

# 2) …then send a turn into it
curl -X POST https://pouchy.ai/api/companion/session/$SESSION_ID/input \
  -H "Authorization: Bearer $POUCHY_PAT" \
  -H "Content-Type: application/json" \
  -d '{ "text": "hello" }'

Errors

Failed calls throw a CompanionError with a stable code and message; transport/protocol errors also surface via onError(). Local precondition mistakes throw the same type with status: 0 and a code: not_connected (called before connect()), missing_option (no baseUrl/token), not_representative, reply_timeout (awaitReply's deadline expired), needs_event_stream (an awaitReply turn paused on your declared tools with no start() — the post-resume reply can only arrive on the event stream), aborted (a caller-supplied AbortSignal cancelled the request — every network method accepts an optional signal, SDK 0.29.0), queued_offline (the offline queue accepted the send — thrown only by an awaitReply: true call, which has no reply to await; a plain send returns { queued: true } instead of throwing, SDK 0.34.0), request_timeout (the requestTimeoutMs deadline expired before response headers arrived, SDK 0.35.0). Common HTTP codes: 401 (invalid token), 403 (missing scope — e.g. supplying a visitor without represent), 429 (rate limited). On a persisted server fault (a real 5xx) the error also carries errorId (0.56.0) — the server's err_… reference to quote to support; absent on 4xx and transport failures. A 429 that also carries degraded: 'limiter_unreachable' comes from the limiter-outage half budget, not your quota: the conversation is being served at a temporary reduced rate, so back off for retryAfter and expect the full budget back. Every SDK surfaces it as CompanionError.degraded.

Server-side failures carry a machine-readable code you can switch on: missing_token, invalid_token, missing_scope, invalid_request, session_not_found, turn_pending, no_pending_tools, unknown_call, payload_too_large, rate_limited, forbidden, unavailable, and — API 1.1 — the confirm/step-up codes confirm_not_found, confirm_resolved, step_up_required, step_up_failed. The vocabulary is append-only; fall back to the HTTP status when code is absent. On rate_limited (429 — turn burst ceiling or demo daily budget) CompanionError.retryAfter carries the server's Retry-After in seconds, so back off exactly that long before retrying (SDK 0.27.0). Since 0.28.0 CompanionError.code is typed as the exported CompanionErrorCodeValue union. A 402 quota_exhausted (SDK 0.45.0) means the account's monthly allowance is spent — credits, or realtime voice minutes on startCall. Unlike rate_limited it is NOT retryable: nothing changes until the plan is upgraded or the allowance resets on the 1st (UTC), so branch on it instead of backing off. A 404 not_found (SDK 0.53.0) means a resource addressed by id does not exist — or is one this token cannot see, deliberately the same answer so that refusals cannot map the space. First user: DELETE /api/companion/memory/{factId}. A 404 mandate_absent means the approval carries no signed mandate (a deny, an approval that needed no step-up, or a row from before the feature); getMandate() returns null on it.

The event stream carries its own control.error vocabulary (also append-only), surfaced via onError(): agent_error — a server-side turn failed after the input was accepted (no reply was produced; safe to re-send); call_mint_failed — start_call couldn't mint the voice-provider credential (the HTTP response errors too; retry later or fall back to text); and stream_unauthorized — synthesized by the SDK when the stream hits a permanent 401/403 and reconnects are exhausted (refresh the token via setToken/onAuthError, then start() again). Also SDK-synthesized: stream_rejected — the stream got a terminal 404 / 410 / 400 (e.g. session_not_found after the session was wiped) and stopped instead of reconnecting forever (connect() again for a fresh session, then start()).

Voice connect-step failures (SDK 0.26.0) reject connectCall/startCall with the same CompanionError type and client-synthesized codes: call_unsupported — no WebRTC/microphone in this environment; call_connect_failed — the mic-permission request timed out or the SDP exchange failed (carries the HTTP status when the exchange answered non-2xx); call_dependency_missing — the optional @elevenlabs/client peer dependency isn't installed. Browser-native getUserMedia rejections (e.g. NotAllowedError) propagate untouched. Since 0.40.1 a connect-phase failure after a successful credential mint first tears the call down like an instant hang-up (/end closes the server's call-active window), then rethrows the original error. A failure DURING a live call reaches the call's onError as a CompanionError too (status 0): call_dropped — the connection went away (offer a reconnect); call_provider_error — the voice provider reported a failure. The message text is unchanged.

Support

Questions or an integration in progress? Request access and the team will help you plan capabilities, personas and wiring — or email support@pouchy.ai.