Parsing outputs with Genie

The radkit_genie module integrates pyATS and Genie parsers into RADKit to convert unstructured CLI data to structured data.

We currently provide the following basic functionality:

parse() to parse device command output into structured data;

learn() to collect full feature state (e.g. BGP) from one ore more devices;

fingerprint() to dynamically learn a device’s OS (required for the Genie parsers to work);

diff() and diff_snapshots() to compare parse/learn results, for example to find out the differences before and after a change;

api() to invoke a raw Genie API and execute device/OS specific tasks.

The radkit_genie module is not loaded by default in the radkit-client REPL. You need to explicitly import it:

import radkit_genie

Warning

The radkit_genie package must be installed separately and is currently not available in the Windows and macOS installers. You must use a pip based method to install it; the package is named cisco_radkit_genie and can be installed exactly the same way as cisco_radkit_client or cisco_radkit_service.

Deriving the Genie OS 

The parse(), learn() and api() functions need information about the operating system (i.e. iosxe, iosxr, nxos, etc.) of the device(s).

A list of supported platforms can be found at this location.

There are three ways RADKit genie can derive the OS information (in the order of preference):

by the user adding it as a keyword argument to parse(), learn() and api() (e.g. os="iosxe")

by running fingerprint() on the devices prior to parsing/learning

by radkit_genie leveraging a RADkit device type to genie OS mapping (e.g. IOS_XE to iosxe)

Genie `parse`

The parse() function processes the output of the RADKit Device.exec() call and invokes Genie parsers to parse the output of the executed commands, returning the information as structured data objects. The list of available Genie parsers can be found at this location.

>>> single_response = service.inventory["router1"].exec("show ip route").wait()
>>> parsed_result = radkit_genie.parse(single_response)
>>> parsed_result["router1"]["show ip route"].data
{
    "router1": {
        "show ip route": {
            "vrf": {
                "default": {
                    "address_family": {
                        "ipv4": {
                            "routes": {
                                "0.0.0.0/0": {
                                    "route": "0.0.0.0/0",
[...]

>>> devices = service.inventory.filter("name", "PE*")
>>> multi_response = devices.exec(["show ip route", "show interfaces"]).wait()
>>> parsed_result = radkit_genie.parse(multi_response, os="iosxe")

Note

If you want to invoke a Genie parser on device output collected via different means than RADKit Device.exec() call, please use parse_text()
You can also define and install custom local Genie parsers, for example to create a new parser for a specific command, or fix a bug in an existing parser. Please refer to Adding Custom Genie Parsers for more information.

Genie `learn`

The learn() function collects information about feature models (i.e. bgp, platform, routing) across different device types/OSes. This information is especially useful for collecting device state and comparing this state before and after network changes using diff(). The list of available Genie models can be found at this location.

>>> result = radkit_genie.learn(service.inventory["router2"], "bgp")
>>> result["router2"]["bgp"].data
{
    "router2": {
        "bgp": {
            "context_manager": {},
            "attributes": None,
            "commands": None,
            "connections": None,
            "raw_data": False,
[...]

`radkit_genie` API reference 

radkit_genie.api( devices: Device | DeviceDict, api: str, os: str | None = None, exec_timeout: int | None = None, skip_unknown_os: bool = False, num_threads: int | None = None, ) → Callable[[...], GenieApiResult]

This method exposes Genie APIs to execute device/OS specific tasks on one or more RADKit devices. Please check https://pubhub.devnetcloud.com/media/genie-feature-browser/docs/#/apis for supported APIs.

Parameters:

devices – a single RADKit Device object or a DeviceDict object containing multiple devices
api – the API to execute
os – the genie device OS. If this option is omitted, the OS found by radkit_genie.fingerprint() is used; else the RADKit Device Type is used. If none of the previous result in a valid genie device OS, this parameter is mandatory)
skip_unknown_os – skip parsing output from devices whose OS is not known instead of raising an exception (default: False)
exec_timeout – timeout waiting for connection/result (in seconds, default: 60)
num_threads – number of threads (default: 5)

Returns:

callable object taking the API parameters as arguments and returning a dict of Genie API return values (key: device name, value: str or list[str], depending on API)

Raises:

RADKitGenieMissingOS if a device OS is missing

Examples:

The method can execute an API on a single device, or multiple devices.

result = radkit_genie.api(
    service.inventory['router1'], 'get_interface_admin_status', os='iosxe'
)('Vlan1022')
status = result['router1'].data

result = radkit_genie.api(
    service.inventory['router1'], 'get_interface_names'
)()
names = result['router1'].data

device = service.inventory['router1']
radkit_genie.fingerprint(device)
result = radkit_genie.api(
    service.inventory['router1'], 'get_interface_admin_status'
)('Vlan1022')
status = result['router1'].data

multiple_devices = service.inventory.filter('name', 'Edge')
radkit_genie.fingerprint(multiple_devices)
result = radkit_genie.api(
    service.inventory['router1'], 'get_interface_admin_status'
)('GigabitEthernet0/1/2')
status = result['router1'].data

radkit_genie.parse( radkitrequest: ExecResponseBase[Any], parser: str | None = None, os: str | None = None, skip_unknown_os: bool = False, ) → GenieResult

This function uses Genie parsers to parse command output returned by RADKit Client’s Device.exec() call into structured data.

Please check https://pubhub.devnetcloud.com/media/genie-feature-browser/docs/#/parsers for supported parsers. Genie tries to search for the relevant parser based on the command executed (using fuzzy search); if the search fails, you can provide the parser and the OS manually.

parse(...).to_dict() converts the result into a special dictionary of type QDict which can also be parsed using Genie’s dq method.

Parameters:

radkitrequest – return value of RADKit Client’s Device.exec() call
parser – parser to choose (if omitted, the parser is derived from the command issued)
os – the genie device OS. If this option is omitted, the OS found by radkit_genie.fingerprint() is used; else the RADKit Device Type is used. If none of the previous result in a valid genie device OS, this parameter is mandatory)
skip_unknown_os – skip parsing output from devices whose OS is not known instead of raising an exception (default: False)

Returns:

GenieResult structure (dict of dict), use result[device][cmd].data to access the parsed data

Raises:

RADKitGenieMissingOS if a device OS is missing

Examples:

# Parse the output from a single device and a single command, specifying the OS explicitly
single_response = service.inventory['devicename'].exec('show ip route').wait()
result = radkit_genie.parse(single_response, os='iosxe')
parsed_data = result['devicename']['show ip route'].data

# Parse the output from multiple devices and multiple commands, leveraging RADkit device type
# to genie OS mapping
multi_response = service.inventory.filter('name', 'Edge').exec(['show ip route', 'show version']).wait()
result = radkit_genie.parse(multi_response)
for device in result.keys():
    parsed_routes = result[device]['show ip route'].data
    parsed_version = result[device]['show version'].data

# turn result into a Genie QDict for easy dict parsing
response = service.inventory.filter('name', 'Edge').exec('show ip route').wait()
qdict = radkit_genie.parse(response).to_dict()
paths = qdict.q.contains('1.1.1.1)

radkit_genie.parse_text( text: str, parser: str, os: str, ) → QDict

While radkit_genie’s parse() function is most commonly invoked when dealing with parsing the output of the RADKit Device.exec() call, we also provide a convenience function to invoke Genie’s parsers on raw text output, for example collected as part of RADKit’s exec-sequence.

This method expects the output of a single command, the parser to be used (i.e. the command executed) and the device’s operating system (os).

Please check https://pubhub.devnetcloud.com/media/genie-feature-browser/docs/#/parsers for supported parsers.

Parameters:

text – the text output of a single command
parser – parser to choose (typically the command executed)
os – the genie device OS (mandatory)

Returns:

dict structure as returned by Genie’s parse method

Examples:

# Parse the output from a device output
parsed_result = radkit_genie.parse_text(output, "show version", "iosxe")
version = parsed_result["version"]["xe_version"]
serial = parsed_result["version"]["chassis_sn"]

This function uses Genie’s learn method to parse device features, so-called models, into structured data. Genie will execute the required commands on the device(s) as it sees fit. Please check https://pubhub.devnetcloud.com/media/genie-feature-browser/docs/#/models for a list of supported models.

learn() returns a dict of dicts, use result[device][model].data to access the result.

learn().to_dict() can convert the result to a special dictionary of type QDict which can also be parsed using Genie’s dq method.

Parameters:

devices – a single RADKit Device object or a DeviceDict object containing multiple devices
models – one or more model(s) to learn, as a str or list[str]
os – the genie device OS. If this option is omitted, the OS found by radkit_genie.fingerprint() is used; else the RADKit Device Type is used. If none of the previous result in a valid genie device OS, this parameter is mandatory)
skip_unknown_os – skip parsing output from devices whose OS is not known instead of raising an exception (default: False)
exec_timeout – timeout waiting for connection/result (in seconds, default: 60)
num_threads – number of threads (default: 5)

Returns:

GenieResult structure (dict of dict), use result[device][model].data to access the learnt data

Raises:

RADKitGenieMissingOS if a device OS is missing

Examples:

# Learn from a single device, single model, specifying the OS explicitly
learn_result = radkit_genie.learn(service.inventory['router1'], 'bgp', os='iosxe')
parsed_result = learn_result['router1']['bgp'].data

# Learn from a single device, single/multiple models, using OS fingerprinting
device = service.inventory['router1']
radkit_genie.fingerprint(device)
single_result = radkit_genie.learn(device, 'bgp').to_dict()
multi_result = radkit_genie.learn(device, ['bgp', 'platform']).to_dict()

# Learn from multiple devices, single model, levering RADkit device type
multiple_devices = service.inventory.filter('name', 'Edge')
learn_result = radkit_genie.parse(multiple_devices, 'routing').to_dict()

Genie needs to know the operating system (OS) of the devices it interacts with, i.e. iosxe, iosxr, linux, etc. Please refer to https://pubhub.devnetcloud.com/media/unicon/docs/user_guide/supported_platforms.html#supported-platforms for the list of supported OSes.

The parse() and learn() functions accept the OS as an optional argument, but we also expose a method which attempts to learn the device’s OS and stores it in the Client’s local device metadata (this ephemeral data is only kept during the course of the session/script). Once the OS fingerprinting results are available, those will be leveraged automatically.

Parameters:

devices – a single RADKit Device object or a DeviceDict object containing multiple devices
update_device_attributes – update OS/platform in local RADKit device inventory (default: True)
exec_timeout – timeout waiting for connection/result (in seconds, default: 60)
num_threads – number of threads (default: 5)

Returns:

a dict with the result of the fingerprinting attempt for each device (the value is set to None if the OS cannot be determined), e.g. {"C9k-u-4": {"os": "iosxe", "platform": "cat9k"}}

Examples:

# Fingerprint a single device
radkit_genie.fingerprint(service.inventory['router1'])
# Fingerprint the entire inventory
radkit_genie.fingerprint(service.inventory)

This function compares the results of parse() or learn() across multiple devices and outputs the differences between the parsed command output or the learned model output.

Unlike diff_snapshots(), which performs a before/after comparison of successive snapshots taken from the same device(s), diff() compares the same output between two or more devices, for example to compare software versions.

If you pass a single result object to diff(), it expects command/model output from a set of devices, and compares the output in all combinations. This allows us, for example, to compare the routes collected across several devices, or a specific feature/command state.

If you pass two result objects, it assumes you collected the same command(s)/model(s) from two different devices and compares them. In this case, each of the result objects can only include output from a single device.

By default, Genie’s parse() and learn() models define a number of attributes which should be excluded from comparison, for example running counters, neighbor uptime, etc. which typically change between invocations and are not necessarily interesting. The list of those attributes can be manipulated using the exclude or exclude_add parameters as described below.

Parameters:

result_a – the result of a previous parse() or learn() call
result_b – the result of a previous parse() or learn() call (if this parameter is set to None or is not specified at all, diff() assumes that result_a contains the command/model results from multiple devices and compares those)
exclude – override the list of excluded attributes (see above)
exclude_add – add to Genie’s default list of excluded attributes (see above)

Returns:

Genie DiffResult object, can be converted to a unix-style diff output using str() (evaluates to False if no diffs are found)

Examples:

Compare the same command/model from two calls to two devices:
cmds = ['show ip route', 'show ip cef']
r1_out = radkit_genie.parse(service.inventory['router1'].exec(cmds).wait(), os='iosxe')
r2_out = radkit_genie.parse(service.inventory['router1'].exec(cmds).wait(), os='iosxe')
diff = radkit_genie.diff(r1_out, r2_out)
Compare the same command across more than two devices, ignoring IPv4 addresses which are expected to be different. Here we only pass a single argument and diff compares the result between each pair of routers:
lo0_state = radkit_genie.parse(service.inventory.filter('name', 'Edge').exec('show interfaces Loopback0')
diff = radkit_genie.diff(lo0_state, exclude_add=['ipv4'])
print(str(diff) or 'No change detected')

radkit_genie.diff_snapshots( result_a: GenieResult | QDict, result_b: GenieResult | QDict, exclude: list[str] | None = None, exclude_add: list[str] | None = None, ) → DiffResult

This function compares two sets of results from parse() or learn() collected on the same set of devices and for the same commands/models, and outputs the differences between the parsed command output or the learned model output.

Unlike diff(), which compares command or model data across multiple devices, diff_snapshots() is meant to perform a before/after comparison of successive snapshots.

It can compare single commands from a single device, single commands from multiple devices or multiple commands from multiple devices. The only constraint is that both snapshots need to be collected from exactly the same set of devices and for the same commands/models.

By default, Genie’s parse() and learn() models define a number of attributes which should be excluded from comparison, for example running counters, neighbor uptime, etc. which typically change between invocations and are not necessarily interesting. The list of those attributes can be manipulated using the exclude or exclude_add parameters as described below.

Parameters:

result_a – the result of a previous parse() or learn() call
result_b – the result of a previous parse() or learn() call, taken from the same set of devices and including the same set of commands/models as result_a
exclude – override the list of excluded attributes (see above)
exclude_add – add to Genie’s default list of excluded attributes (see above)

Returns:

Genie DiffResult object, can be converted to a unix-style diff output using str() (evaluates to False if no diffs are found)

Examples:

Compare routing state of the same device before and after an event:

before = radkit_genie.learn(service.inventory['router1'], 'routing', os='iosxe')
# [...]
after =  radkit_genie.learn(service.inventory['router1'], 'routing', os='iosxe')
diff = radkit_genie.diff_snapshots(before, after)
print(str(diff) or 'No change detected')

Compare multiple commands from the same device:

cmds = ['show ip route', 'show ip cef']
before = radkit_genie.parse(service.inventory['router1'].exec(cmds).wait(), os='iosxe')
# [...]
after =  radkit_genie.parse(service.inventory['router1'].exec(cmds).wait(), os='iosxe')
diff = radkit_genie.diff_snapshots(before, after)
print(str(diff) or 'No change detected')

radkit_genie.diff_dicts( dict_a: dict[Any, Any], dict_b: dict[Any, Any], exclude: list[str] | None = None, exclude_add: list[str] | None = None, verbose: bool | None = None, list_order: bool | None = None, ) → Diff: Compares two discrete genie parse/learn results, i.e. the actual genie result dictionaries. Return value is a genie Diff object, use str() to convert to a traditional diff output.

class radkit_genie.Device

Bases: Device

RADKitGenie Device object

connect(

*args: Any,

**kwargs: Any,

) → None

disconnect(

*args: Any,

**kwargs: Any,

) → None

execute(

command: str,

**kwargs: Any,

) → str

parse( parser: str, output: str | None = None, ) → QDict

exception radkit_genie.RADKitGenieException: Bases: Exception

exception radkit_genie.RADKitGenieMissingOS: Bases: RADKitGenieException

class radkit_genie.GenieResultStatus

Bases: Enum

FAILURE = 'FAILURE'

SUCCESS = 'SUCCESS'

Parsing outputs with Genie

Deriving the Genie OS

Genie parse

Genie learn

radkit_genie API reference

Deriving the Genie OS 

Genie `parse`

Genie `learn`

`radkit_genie` API reference 