RADKit LLM Integration

The radkit_llm package provides a starting point for building AI agents on top of radkit_client to automate troubleshooting and information retrieval of networks.

It can be used as a library, but also provides several tools:

  • An eval-based MCP server for use by LLM agents (works especially well with GitHub Copilot and Claude Sonnet)

  • An eval-based built-in CLI agent using Langchain and Claude

  • A simple MCP server with specific tools

The “eval-based” components expose a Python sandbox to the LLM. This is very powerful but requires the best language models, and the Bubblewrap sandbox only works on Linux. The simple MCP server does not use a Python sandbox but exposes a specific set of tools to the LLM. This is less powerful but does not require the most advanced language models.

Warning

The radkit_llm package is not a hosted service and is not a production-ready solution. It is intended as a starting point for developing custom AI agents on top of RADKit.

Cisco does not provide the LLM models, subscriptions, or related services, nor does it provide specific guidance on their use.

Use this software at your own risk.

Warning

The radkit_llm package is not included in the RADKit installers.

Installation

To use RADKit LLM integration, install the cisco_radkit_llm package using the pip installation method. After installation, a radkit-llm executable will be available in your $PATH.

Using as an MCP Server with GitHub Copilot

To run the MCP server, generate and export a token, then start it:

export RADKIT_LLM_MCP_SERVER_AUTHORIZATION_TOKEN="$(python -c 'import secrets; print(secrets.token_urlsafe(32))')"
radkit-llm mcp-server

Then configure your AI agent. For VS Code with GitHub Copilot, create a .vscode/mcp.json file with the following content, replacing the placeholder with the generated token:

{
    "servers": {
        "radkit": {
            "type": "http",
            "url": "http://localhost:8080/mcp/",
            "headers": {
                "Authorization": "Bearer replace-this-with-the-token-configured-on-the-server"
            }
        }
    }
}

After this, use Copilot in “Agent” mode, go to “tools”, and ensure only radkit_eval is selected among the available tools.

Example interactions:

  • Connect to a RADKit service with serial <serial>

  • List the content of the inventory of the service

  • Execute <command> on all devices of type Linux

By default, this MCP server can connect to any RADKit service. You can also pass a service config path via the settings system with connection information. When provided, the MCP server will automatically connect to that service and will not allow connections to other services.

MCP Server Authorization

The MCP server requires bearer token authentication. Clients must supply a valid Authorization: Bearer <token> header with every request. The server refuses to start unless a valid token is configured (at least 32 characters) — there is no way to run it unauthenticated.

Set the token using one of these methods:

Environment variable:

export RADKIT_LLM_MCP_SERVER_AUTHORIZATION_TOKEN="$(python -c 'import secrets; print(secrets.token_urlsafe(32))')"
radkit-llm mcp-server

Warning

Storing a password in an environment variable is inherently insecure and is not recommended in any way. Use this authentication method at your own risk, preferably in a controlled and secure environment.

TOML configuration (in ~/.radkit/llm/settings.toml):

[llm.mcp_server]
authorization_token = "replace-this-with-a-generated-token-of-at-least-32-characters"

When authorization is enabled, update your VS Code .vscode/mcp.json to include the same token:

{
    "servers": {
        "radkit": {
            "type": "http",
            "url": "http://localhost:8080/mcp/",
            "headers": {
                "Authorization": "Bearer replace-this-with-the-token-configured-on-the-server"
            }
        }
    }
}

Leaving the token empty (the default), or setting one shorter than 32 characters, makes the server exit at startup with an error explaining how to generate a strong one.

Choosing a token

The token is opaque to the MCP server — it is compared as a literal string. It must use the Bearer token character set: ASCII letters, digits, -, ., _, ~, +, or /, with optional trailing = characters.

Use a token of at least 32 characters with high entropy. Shorter or malformed tokens are rejected at startup.

Using as a Standalone CLI Agent

The CLI agent can use either AWS Bedrock or OpenAI.

Using OpenAI:

Set the required environment variables:

export OPENAI_API_KEY="..."
export OPENAI_API_BASE="https://cxai-playground.cisco.com"
radkit-llm -s llm.chat.default_api OPENAI chat

Using AWS Bedrock:

Ensure you have a ~/.aws/credentials file containing a [claude] section, then:

radkit-llm -s llm.chat.default_api AWS_BEDROCK chat

All input/output happens on the CLI. The eval tool will also ask for confirmation on the CLI.

A service connection config can be passed via the settings system: --setting llm.chat.service_config_path <path>.

Using the Simple MCP Server

The Simple MCP server does not expose a Python interpreter but a fixed set of RADKit-specific tools. It requires a service config file with connection information.

Example config file for direct RPC connection:

{
  "type": "direct",
  "username": "superadmin",
  "password_base64": "...",
  "host": "localhost",
  "rpc_port": 8181
}

Run the server (a bearer token is required, exactly like the eval-based MCP server — see MCP Server Authorization):

export RADKIT_LLM_SIMPLE_MCP_SERVER_AUTHORIZATION_TOKEN="$(python -c 'import secrets; print(secrets.token_urlsafe(32))')"
radkit-llm -s llm.simple_mcp_server.service_config_path config.json simple-mcp-server

Architecture

The eval-based MCP server has only one tool: radkit_eval. The server hosts a single Python sandbox with radkit_client installed, which is created on startup and reused for the lifetime of the server process. The agent can submit any Python code which will be evaluated in this sandbox.

Note

Session isolation. All MCP client sessions handled by a single radkit-llm mcp-server process currently share the same Python sandbox. State created by one session — variables, imported modules, open service connections, cached data — is visible to every other session connected to the same server.

If you need separation between users or workloads, run multiple radkit-llm mcp-server instances (each on its own port) and, ideally, configure each one with a distinct bearer token (see MCP Server Authorization). Route each client to its dedicated server instance.

Security Design

The sandbox architecture ensures security:

  • The sandbox has no direct network access to RADKit services

  • RADKit credentials are never available within the sandbox

  • All RADKit operations go through an RPC proxy that can intercept and request confirmation

  • Code runs in isolation (especially with Bubblewrap or Firecracker)

  • Elicits are generated when the sandbox needs to interact with external services

This approach moves manual verification from tool invocation to the elicit that’s generated when going out of the sandbox.

Sandbox Implementations

Currently supported sandbox implementations:

Subprocess

A simple implementation that creates a Python subprocess with the radkit_client library installed. Evaluation of Python code is done by sending it over a local Unix socket to this subprocess.

Bubblewrap (Linux only)

Similar to Subprocess but wraps it in Bubblewrap to isolate it from the network and parts of the filesystem.

To use Bubblewrap:

  1. Install bubblewrap: sudo apt install bubblewrap

  2. Disable AppArmor restrictions (needed on some Ubuntu installations):

    sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

  3. Configure the sandbox via the settings system. The setting name differs per command:

    For radkit-llm mcp-server:

    radkit-llm -s llm.mcp_server.sandbox bubblewrap mcp-server

    For radkit-llm chat:

    radkit-llm -s llm.chat.sandbox bubblewrap chat