Mirako

MCP Tools in Mirako Interactive

Mirako Interactive can connect to Model Context Protocol (MCP) servers to give your AI avatar superpowers! Think of MCP tools as special abilities that your avatar can use during conversations - like showing videos, searching databases, or performing complex computations.

Quick Start

Enable MCP tools in your session with the --tools flag. The flag itself is a string, but the value must be a valid JSON array string. Here's a simple example using the official MCP demo server:

sh
mirako interactive start \
    --avatar <YOUR_AVATAR_ID> \
    --voice <YOUR_VOICE_ID> \
    --llm-model "gemini-2.5-flash" \
    --instruction "You are a friendly AI that can show videos to users." \
    --tools '[{"url":"https://mcp-demo-video-server.YOUR_ACCOUNT.workers.dev/mcp"}]'

For a single MCP server, pass a one-item array with the server url. If your server requires authentication, include authToken in the same object.

Note: --tools must contain a JSON array string. A bare URL like --tools "https://mcp-demo-video-server.YOUR_ACCOUNT.workers.dev/mcp" will fail because Mirako validates tools as a JSON list.

When the user asks the avatar to show a video, the interactive service will connect to your MCP server, execute the tool, and return the result - all in real-time!

Set up interactive profile in config file

Use JSON when passing tools directly on the command line:

sh
mirako interactive start \
    --avatar <YOUR_AVATAR_ID> \
    --voice <YOUR_VOICE_ID> \
    --llm-model "gemini-2.5-flash" \
    --instruction "You are a friendly AI that can show videos to users." \
    --tools '[{"url":"https://mcp-demo-video-server.YOUR_ACCOUNT.workers.dev/mcp"}]'

If you prefer using ~/.mirako/config.yml, use a normal YAML array instead:

yaml
interactive_profiles:
  default:
    avatar_id: <YOUR_AVATAR_ID>
    model: metis-2.5
    llm_model: gemini-2.5-flash
    voice_profile_id: <YOUR_VOICE_ID>
    instruction: You are a friendly AI that can show videos to users.
    tools:
      - url: https://mcp-demo-video-server.YOUR_ACCOUNT.workers.dev/mcp

How It Works

Understanding the mechanics behind MCP tools integration:

┌─────────────┐     Session Start      ┌──────────────┐
│  User Chat  │ ───────────────────→  │  Mirako      │
└─────────────┘                       │  Interactive │
                                      │  Service     │
                                      └──────┬───────┘
                                             │
                                             │ Pass tools config
                                             ↓
                                       ┌──────────────┐
                                       │  Session     │
                                       │  Started     │
                                       └──────┬───────┘
                                              │
                    ┌──────────────────────────┼──────────────────────────┐
                    │                          │                          │
                    ↓                          ↓                          ↓
            ┌──────────────┐           ┌──────────────┐           ┌──────────────┐
            │  User asks   │           │  Avatar      │           │  MCP Server  │
            │  "Show a     │           │  generates   │           │  Executes    │
            │  video"      │           │  response    │           │  tool call   │
            └──────┬───────┘           └──────┬───────┘           └──────┬───────┘
                   │                          │                          │
                   └──────────────┬───────────┘                          │
                                  │                                     │
                                  │ Tool call                             │
                                  ↓                                     ↓
                              ┌──────────────┐                     ┌──────────────┐
                              │  MCP Client  │                     │  Returns     │
                              │  Connects    │                     │  video URL   │
                              └──────┬───────┘                     └──────┬───────┘
                                     │                                     │
                                     └──────────────┬─────────────────────┘
                                                    │
                                                    │ Display to user
                                                    ↓
                                          ┌──────────────┐
                                          │  Video       │
                                          │  Playing!    │
                                          └──────────────┘

Key Points:

  • Server-side execution: The MCP server runs on your infrastructure, completely separate from the interactive session
  • No client code: Users don't need to install anything or write code - it all happens server-side
  • Real-time: Tool execution and results are delivered instantly during the conversation
  • Context-aware: The LLM decides when and how to use tools based on the conversation context

MCP Server Requirements

To integrate your own MCP server with Mirako Interactive:

  • Must be a valid MCP server: Follow the MCP specification
  • HTTP-based: The server must be accessible via HTTP/S (like Cloudflare Workers, Vercel, etc.)
  • Tool definitions: Your MCP server should expose tools that describe what they do and what parameters they accept
  • Response format: Tools should return results that can be displayed to users (like videos, text, images)

Tool Response Format

When an MCP tool is called, it should return a response that the Interactive client can display. The response format depends on what you want to show users:

Video Example

json
{
  "content": [
    {
      "type": "text",
      "text": JSON.stringify({
        "operation": "show_video",
        "video_url": "https://your-server.com/video.mp4"
      })
    }
  ]
}

Text Example

json
{
  "content": [
    {
      "type": "text",
      "text": JSON.stringify({
        "operation": "display_text",
        "message": "Here's the information you requested!"
      })
    }
  ]
}

Multiple Content Types

json
{
  "content": [
    {
      "type": "text",
      "text": JSON.stringify({
        "operation": "show_video",
        "video_url": "https://your-server.com/video.mp4"
      })
    },
    {
      "type": "text",
      "text": JSON.stringify({
        "operation": "display_text",
        "message": "Watch the video above for more details."
      })
    }
  ]
}

The Interactive client will parse these JSON responses and render them appropriately based on the operation type.

Example: Video Playback

Let's walk through the complete flow with the official MCP demo server. This server demonstrates a single tool called get_video that fetches and plays a video.

Setup the Demo Server

  1. Clone the demo server repository:
sh
cd ..
git clone https://github.com/mirako-ai/mcp-demo-video-server
cd mcp-demo-video-server
npm install
wrangler deploy
  1. You'll get a URL like: https://mcp-demo-video-server.YOUR_ACCOUNT.workers.dev/mcp

Start an Interactive Session

sh
mirako interactive start \
    --avatar <YOUR_AVATAR_ID> \
    --voice <YOUR_VOICE_ID> \
    --llm-model "gemini-2.5-flash" \
    --instruction "You are a friendly AI that can show videos to users when they ask." \
    --tools '[{"url":"https://mcp-demo-video-server.YOUR_ACCOUNT.workers.dev/mcp"}]'

How It Works in Practice

When you start chatting:

You: "Can you show me a fun video?"

Avatar: "I'd love to! Let me play a video for you. ✨

(Mirako Interactive connects to the MCP server and executes the get_video tool)

(Video starts playing in the session!)

Avatar: "That's from our demo video library! Do you enjoy it?"

What's Happening Behind the Scenes

  1. Session Start: You provide one or more MCP server definitions in the --tools JSON array
  2. Tool Discovery: Mirako Interactive connects to your MCP server and discovers available tools
  3. LLM Context: The LLM knows about the available tools and their descriptions
  4. Tool Call: When you ask to "show a video", the LLM decides to call the get_video tool
  5. Execution: Mirako Interactive sends the tool call to your MCP server
  6. Result: Your server returns the video URL in the proper format
  7. Display: The Interactive client parses the response and plays the video
  8. User Experience: Everything happens in real-time during your conversation

Advanced Scenarios

Multiple MCP Servers

Because --tools accepts a JSON array, you can register more than one MCP server in the same session:

sh
mirako interactive start \
    --avatar <YOUR_AVATAR_ID> \
    --voice <YOUR_VOICE_ID> \
    --llm-model "gemini-2.5-flash" \
    --instruction "You are a helpful AI assistant with access to external tools." \
    --tools '[
      {"url":"https://news.example.com/mcp"},
      {"url":"https://weather.example.com/mcp","authToken":"YOUR_AUTH_TOKEN"}
    ]'

Mirako Interactive will discover tools from each server and make them available to the LLM during the session.

Multiple Tools from One Server

You can expose multiple tools from a single MCP server. For example, your server could have:

  • get_news: Fetches current news headlines
  • get_weather: Returns weather information for a location
  • search_database: Searches your custom database

The LLM will intelligently choose which tool to use based on user requests.

Error Handling

If your MCP server returns an error or times out:

  • The avatar will explain the error to the user
  • You'll receive information about what went wrong
  • Users can try again or ask follow-up questions

Server-Side Security

Remember that:

  • MCP servers run on your infrastructure
  • User data is not exposed to Mirako Interactive
  • You have full control over what tools are exposed and how they work
  • Tool execution is completely isolated from the LLM's context

Going Next

Now that you understand the basics of MCP tools, you can:

  • Build your own MCP server: Create custom tools for your specific use case
  • Connect multiple servers: Integrate various MCP tools for comprehensive functionality
  • Create domain-specific experiences: Build tools that enhance your avatar's abilities in specialized areas

The possibilities are endless - from video playback to database queries, from real-time data fetching to complex computations, your avatar can be equipped with exactly the tools it needs to provide amazing user experiences.

Dive Deeper

Previous
Supported LLMs
Next
Overview