RADKit LLM Integration

The radkit_llm package provides a starting point for building AI agents on top of radkit_client to automate troubleshooting and information retrieval of networks.

It can be used as a library, but also provides several tools:

An eval-based MCP server for use by LLM agents (works especially well with GitHub Copilot and Claude Sonnet)
An eval-based built-in CLI agent using Langchain and Claude
A simple MCP server with specific tools

The “eval-based” components expose a Python sandbox to the LLM. This is very powerful but requires the best language models, and the Bubblewrap sandbox only works on Linux. The simple MCP server does not use a Python sandbox but exposes a specific set of tools to the LLM. This is less powerful but does not require the most advanced language models.

Warning

The radkit_llm package is not included in the RADKit installers.

Installation 

To use RADKit LLM integration, install the cisco_radkit_llm package using the pip installation method. After installation, a radkit-llm executable will be available in your $PATH.

Using as an MCP Server with GitHub Copilot 

To run the MCP server:

radkit-llm mcp-server

Then configure your AI agent. For VS Code with GitHub Copilot, create a .vscode/mcp.json file with the following content:

{
    "servers": {
        "radkit": {
            "type": "http",
            "url": "http://localhost:8080/mcp/"
        }
    }
}

After this, use Copilot in “Agent” mode, go to “tools”, and ensure only radkit_eval is selected among the available tools.

Example interactions:

Connect to a RADKit service with serial <serial>
List the content of the inventory of the service
Execute <command> on all devices of type Linux

By default, this MCP server can connect to any RADKit service. You can also pass a service config path via the settings system with connection information. When provided, the MCP server will automatically connect to that service and will not allow connections to other services.

MCP Server Authorization 

The MCP server supports bearer token authentication. When configured, clients must supply a valid Authorization: Bearer <token> header with every request. It is strongly recommended to enable authorization when the server is accessible over a network.

To enable authorization, set the token using one of these methods:

Environment variable:

export RADKIT_LLM_MCP_SERVER_AUTHORIZATION_TOKEN="your-secret-token"
radkit-llm mcp-server

Warning

Storing a password in an environment variable is inherently insecure and is not recommended in any way. Use this authentication method at your own risk, preferably in a controlled and secure environment.

TOML configuration (in ~/.radkit/llm/settings.toml):

[llm.mcp_server]
authorization_token = "your-secret-token"

When authorization is enabled, update your VS Code .vscode/mcp.json to include the token:

{
    "servers": {
        "radkit": {
            "type": "http",
            "url": "http://localhost:8080/mcp/",
            "headers": {
                "Authorization": "Bearer your-secret-token"
            }
        }
    }
}

Leaving the token empty (the default) disables authentication entirely — the server will emit a prominent warning at startup when no token is configured.

Choosing a token 

The token is opaque to the MCP server — it is compared as a literal string.

Use a token of at least 32 characters with high entropy. Shorter tokens trigger a warning at startup but are not rejected.

Using as a Standalone CLI Agent 

The CLI agent can use either AWS Bedrock or OpenAI.

Using OpenAI:

Set the required environment variables:

export OPENAI_API_KEY="..."
export OPENAI_API_BASE="https://cxai-playground.cisco.com"
radkit-llm -s llm.chat.default_api OPENAI chat

Using AWS Bedrock:

Ensure you have a ~/.aws/credentials file containing a [claude] section, then:

radkit-llm -s llm.chat.default_api AWS_BEDROCK chat

All input/output happens on the CLI. The eval tool will also ask for confirmation on the CLI.

A service connection config can be passed via the settings system: --setting llm.chat.service_config_path <path>.

Using the Simple MCP Server 

The Simple MCP server does not expose a Python interpreter but a fixed set of RADKit-specific tools. It requires a service config file with connection information.

Example config file for direct RPC connection:

{
  "type": "direct",
  "username": "superadmin",
  "password_base64": "...",
  "host": "localhost",
  "rpc_port": 8181
}

Run the server:

radkit-llm -s llm.simple_mcp_server.service_config_path config.json simple-mcp-server

Architecture 

The eval-based MCP server has only one tool: radkit_eval. The server hosts a single Python sandbox with radkit_client installed, which is created on startup and reused for the lifetime of the server process. The agent can submit any Python code which will be evaluated in this sandbox.

Note

Session isolation. All MCP client sessions handled by a single radkit-llm mcp-server process currently share the same Python sandbox. State created by one session — variables, imported modules, open service connections, cached data — is visible to every other session connected to the same server.

If you need separation between users or workloads, run multiple radkit-llm mcp-server instances (each on its own port) and, ideally, configure each one with a distinct bearer token (see MCP Server Authorization). Route each client to its dedicated server instance.

Security Design 

The sandbox architecture ensures security:

The sandbox has no direct network access to RADKit services
RADKit credentials are never available within the sandbox
All RADKit operations go through an RPC proxy that can intercept and request confirmation
Code runs in isolation (especially with Bubblewrap or Firecracker)
Elicits are generated when the sandbox needs to interact with external services

This approach moves manual verification from tool invocation to the elicit that’s generated when going out of the sandbox.

Sandbox Implementations 

Currently supported sandbox implementations:

Subprocess: A simple implementation that creates a Python subprocess with the radkit_client library installed. Evaluation of Python code is done by sending it over a local Unix socket to this subprocess.
Bubblewrap (Linux only): Similar to Subprocess but wraps it in Bubblewrap to isolate it from the network and parts of the filesystem.

To use Bubblewrap:

Install bubblewrap: sudo apt install bubblewrap
Disable AppArmor restrictions (needed on some Ubuntu installations):
```
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
```
Configure the sandbox via the settings system. The setting name differs per command:

For radkit-llm mcp-server:
```
radkit-llm -s llm.mcp_server.sandbox bubblewrap mcp-server
```
For radkit-llm chat:
```
radkit-llm -s llm.chat.sandbox bubblewrap chat
```