Building a Custom Agent

A custom agent lets you bring an agent you already have — your own LLM pipeline, RAG system, or business logic — and give it a face and a voice. Mirako handles the avatar, speech recognition, text-to-speech, and lip-synced video. Your endpoint only has to do one thing: receive the visitor's message and stream back a reply.

Choose a custom agent when you need your own model, prompts, or private retrieval and tool execution. You keep those operations on your endpoint; Mirako processes the replies and visual content you return. A managed agent requires less integration work because Mirako runs the language model for you.

How It Fits Together

Visitor speaks
      │
      ▼
Mirako Interactive ── POST {session_id, messages} ──▶ Your endpoint
      │                                                    │
      │◀── AI SDK UI message stream (SSE) ─────────────────┘
      ▼
Avatar speaks the streamed reply, with optional visual side content

For every visitor turn, Mirako sends your endpoint only the new user message plus a session_id identifying the conversation. Your endpoint owns everything else:

• System prompt and conversation history (key your stored history by session_id if you want multi-turn memory)
• Model selection and generation settings
• Tool calling, retrieval, and MCP servers
• Any authentication or rate limiting beyond the shared bearer token below

Mirako does not inject a prompt, a model, or tools into requests to a custom agent.

The Endpoint Contract

Your endpoint must:

1. Accept POST requests with a JSON body (below) over HTTPS.
2. Respond with a Vercel AI SDK UI message stream over server-sent events.
3. Include the response header x-vercel-ai-ui-message-stream: v1.

Request

{
  "session_id": "sticky-conversation-id",
  "messages": [
    {
      "id": "msg_123",
      "role": "user",
      "parts": [{ "type": "text", "text": "What are your opening hours?" }]
    }
  ]
}

If you configured a bearer token on the agent, every request carries it:

Authorization: Bearer <your-configured-token>

Check it before doing any work, and reject requests without it — Mirako is not the only thing that can find a public URL.

Response

Stream AI SDK UI message parts as SSE data: lines, terminated by data: [DONE]:

HTTP/1.1 200 OK
Content-Type: text/event-stream
x-vercel-ai-ui-message-stream: v1

data: {"type":"start","messageId":"msg_456"}

data: {"type":"text-start","id":"text_1"}

data: {"type":"text-delta","id":"text_1","delta":"We are open "}

data: {"type":"text-delta","id":"text_1","delta":"from 9 to 6."}

data: {"type":"text-end","id":"text_1"}

data: {"type":"finish"}

data: [DONE]

Mirako starts speaking as soon as the first complete sentence arrives, so stream text incrementally instead of buffering the whole reply.

A Minimal Endpoint (TypeScript)

With the Vercel AI SDK, toUIMessageStreamResponse() produces exactly the stream Mirako expects, including the required header:

import { convertToModelMessages, streamText, type UIMessage } from "ai";
import { yourModel } from "./model"; // any AI SDK language model

export async function POST(req: Request) {
  const expected = process.env.MIRAKO_AGENT_TOKEN;
  if (expected && req.headers.get("authorization") !== `Bearer ${expected}`) {
    return new Response("Unauthorized", { status: 401 });
  }

  const { session_id, messages }: { session_id: string; messages: UIMessage[] } =
    await req.json();

  const result = streamText({
    model: yourModel,
    system: `You are answering inside conversation ${session_id}. Keep replies short and spoken-friendly.`,
    messages: await convertToModelMessages(messages),
  });

  return result.toUIMessageStreamResponse();
}

You don't have to use TypeScript or the AI SDK — any stack works as long as it emits the same SSE format with the x-vercel-ai-ui-message-stream: v1 header. But if you use the AI SDK, you get the protocol, tool calling, and streaming handled for you.

Note: The example above is stateless per turn because Mirako sends only the new user message. To keep conversation memory, store messages keyed by session_id on your side and prepend them on each request.

Show Images, Videos, and Web Apps

Your agent can put visual content next to the avatar while it talks — a chart, a product video, even a full web app. Emit persistent AI SDK data parts (or built-in file parts) in the same stream:

writer.write({
  type: "data-video",
  id: "product-demo",
  data: {
    url: "https://cdn.example.com/videos/demo.mp4",
    displayMode: "pip",
  },
});

Rules for visual parts:

• url must be an absolute http/https URL reachable by the visitor's browser. Do not send binary data, base64, data: URLs, or local paths.
• displayMode is "pip" (visual takes the main area, avatar shrinks to an overlay) or "sbs" (side-by-side). Omit it to default to "pip".
• Do not set transient: true — transient parts are ignored.
• Add an id when you want to identify a visual; each new accepted visual replaces the one currently shown.
• For data-webapp, the embedded page must allow framing: check its X-Frame-Options and CSP frame-ancestors headers. CORS does not control framing.
• Avoid secrets in visual URLs. If you use presigned URLs, make sure they stay valid long enough to be displayed and replayed.

Wire It Up

Once your endpoint is deployed, register it as a custom agent:

mirako agent create \
    --name "My Existing Agent" \
    --avatar <MY_AVATAR_ID> \
    --voice <MY_VOICE_ID> \
    --runtime-kind custom_agent \
    --custom-agent-url "https://example.com/api/chat" \
    --custom-agent-bearer-token-file ./custom-agent-token.txt

Then create a route and open the link to test. See Create an Agent for the API equivalent and field reference.

Note: instruction and tools do not apply to custom agents — they are cleared when runtime_kind is custom_agent. Your endpoint is the single source of truth for prompt and tools.

Going Next

• Share & Embed: Send visitors to your agent once it is wired up

Dive Deeper

• Create an Agent
• Agent Routes
• MCP Tools (for managed agents)
• Vercel AI SDK — UI Message Stream Protocol