Advanced configuration for network performance monitoring

If you want to explore all the options you can use when configuring the monitoring of your network performance, see the following sections.

SNMP-base YAML sample file

Here's an example of the various configuration options available in the snmp-base.yaml file used by the ktranslate docker image to poll for SNMP and flow data devices. You can also see a heavily-commented sample in the ktranslate repository on GitHub.

devices:
  # Sample of SNMP v2c device
  ups_snmpv2c__10.10.0.201:
    device_name: ups_snmpv2c
    device_ip: 10.10.0.201
    snmp_comm: public
    oid: .1.3.6.1.4.1.318.1.3.27
    description: "APC Web/SNMP Management Card (MB:v4.1.0 PF:v6.2.1 PN:apc_hw05_aos_621.bin AF1:v6.2.1 AN1:apc_hw05_sumx_621.bin MN:AP9537SUM HR:05 SN: ABC123DEF456 MD:05/21/2016) (Embedded PowerNet SNMP Agent SW v2.2 compatible)"
    last_checked: 2021-11-09T18:14:59.907821489Z
    mib_profile: apc_ups.yml
    provider: kentik-ups
    poll_time_sec: 300
    retries: 1
    timeout_ms: 5000
    user_tags:
      owning_team: dc_ops
    discovered_mibs:
    - PowerNet-MIB_UPS
    - TCP-MIB
    - UDP-MIB
  # Sample of SNMP v3 device
  router_snmpv3__10.10.0.202:
    device_name: router_snmpv3
    device_ip: 10.10.0.202
    snmp_v3:
      user_name: userNamev3
      authentication_protocol: MD5
      authentication_passphrase: authPassPrivacy
      privacy_protocol: AES256
      privacy_passphrase: passPrivacy
    oid: .1.3.6.1.4.1.9.1.544
    description: "Cisco IOS Software, 3800 Software (C3845-ADVENTERPRISEK9-M), Version
      15.1(3)T4, RELEASE SOFTWARE (fc1)\r\nTechnical Support: http://www.cisco.com/techsupport\r\nCopyright
      (c) 1986-2012 by Cisco Systems, Inc.\r\nCompiled Thu 24-May-12 04:27 by prod_rel_team"
    last_checked: 2021-11-09T18:14:59.907821489Z
    mib_profile: cisco-asr.yml
    provider: kentik-router
    user_tags:
      owning_team: core-networking
    discovered_mibs:
    - BGP4-MIB
    - CISCO-MEMORY-POOL-MIB
    - CISCO-PROCESS-MIB
    - IF-MIB
    - OSPF-MIB
    engine_id: "80:00:01:01:0a:14:1e:28"
    match_attributes:
      if_interface_name: "^Ten.*|^Gig.*"
      "!if_Alias": "[Uu]plink"
  # Sample of SNMP v1 device
  netbotz_snmpv1__10.10.0.203:
    device_name: netbotz_snmpv1
    device_ip: 10.10.0.201
    snmp_comm: public
    use_snmp_v1: true
    oid: .1.3.6.1.4.1.5528.100.20.10.2013
    description: "Linux netbotz930A7A 2.6.12 #307 Wed Dec 29 15:25:32 EST 2010 ppc"
    last_checked: 2021-11-09T18:14:59.907821489Z
    mib_profile: apc-netbotz.yml
    provider: kentik-netbotz
    user_tags:
      owning_team: sys_ops
    discovered_mibs:
    - IF-MIB
    - IP-MIB
    - TCP-MIB
    - UDP-MIB
    no_use_bulkwalkall: true
  # Sample of "flow only" device
  flow_only__10.10.0.210:
    device_name: flow_only
    device_ip: 10.10.0.210
    user_tags:
      owning_team: net_eng
    flow_only: true
  # Sample of "ping only" device
  ping_only__10.10.0.220:
    device_name: ping_only
    device_ip: 10.10.0.220
    user_tags:
      owning_team: load_balancing
    ping_only: true
trap:
  listen: 127.0.0.1:1620
  community: public
  version: ""
  transport: ""
  v3_config: null
discovery:
  cidrs:
  - 10.0.0.0/24
  - 10.0.0.202/32
  debug: false
  ports:
  - 161
  - 1161
  default_communities:
  - public
  - public123
  - Publ!cABC
  use_snmp_v1: false
  default_v3: null
  add_mibs: true
  threads: 4
  add_devices: true
  replace_devices: true
  no_dedup_engine_id: false
global:
  poll_time_sec: 60
  drop_if_outside_poll: false
  mib_profile_dir: /etc/ktranslate/profiles
  mibs_db: /etc/ktranslate/mibs.db
  mibs_enabled:
  - BGP4-MIB
  - CISCO-MEMORY-POOL-MIB
  - CISCO-PROCESS-MIB
  - IF-MIB
  - IP-MIB
  - OSPF-MIB
  - PowerNet-MIB_UPS
  - TCP-MIB
  - UDP-MIB
  timeout_ms: 3000
  retries: 0
  global_v3: null
  response_time: false
  user_tags:
    environment: production
  match_attributes:
    if_Description: ".*WAN.*"

Key name	Required	Description
device_name	✓	Name of the device. This is the unique identifier for the device in New Relic One.
device_ip	✓	Target IP of the device.
snmp_comm	✓ (Required for SNMPv1/2c)	`SNMPv1/2c` community string to use.
use_snmp_v1	✓ (Required for SNMPv1)	Indicates whether to use SNMPv1. By default, it's set to `false`.
snmp_v3	✓ (Required for SNMPv3)	SNMP v3 config
debug		Indicates whether to enable debug level logging during SNMP polling. By default, it's set to `false`.
port		Port to send SNMP queries to. By default, it's set to port `161`.
oid	✓ (Required for SNMP polling)	The discovered `systemObjectID \| sysObjectID \| sysOID` for the device. This is used to match the device to a known SNMP profile and set the `provider` attribute. If no match is found, this sets the `provider` as a kentik-default device.
description		The discovered `sysDescr` of the device. This field is informational.
last_checked		Timestamp when this device was last discovered by the `ktranslate` docker image. This field is informational.
mib_profile	✓ (Required for SNMP polling)	SNMP Profile file that was associated with this device during the discovery run based on its `sysOID`. If this starts with a bang (!) token, it will override the automatic matching from the `sysOID` and use a manual override. Ex: `"!cisco-asa.yml"` (quotes are required).
provider	✓ (Required for New Relic One)	Value used during entity synthesis for New Relic One. This is automatically created based on the matched `mib_profile`.
poll_time_sec		Indicates the SNMP polling frequency in seconds. This setting is used to override the `global.poll_time_sec` attribute.
retries		Indicates the number of attempts to retry polling SNMP OIDs. This setting is used to override the `global.retries` attribute.
timeout_ms		Indicates the SNMP polling timeout in milliseconds. This setting is used to override the `global.timeout_ms` attribute.
user_tags		`key:value` pair attributes to give more context to the device. Tags at this level will be appended to any tags applied in the `global.user_tags` attribute.
discovered_mibs		List of MIBs pulled from matched `mib_profile` that this device can respond to. This field is informational.
engine_id		The unique engine ID discovered for this device's SNMP agent. Generally found during SNMP v3 discovery. This field is informational.
match_attributes		`attribute:regex` pairs to add metrics to allowlist. Pairs at this level will be appended to any pairs applied in the `global.match_attributes` attribute. Uses the RE2 syntax and has a default `OR` operator. Prefix key with `!` to force to `AND` operators.
monitor_admin_shut		Indicates whether to monitor interfaces in `Administratively Shutdown` status. By default, it's set to `false`.
no_use_bulkwalkall		Disables the SNMP `GETBULK` request action when `true`. By default, it's set to `false`.
ping_only		Disables all SNMP polling and enables response time polling when `true`. This setting is used to override the `global.response_time` attribute. By default, it's set to `false`.
flow_only		Disables all SNMP polling when `true`. By default, it's set to `false`.

Key name	Required	Description
listen	✓	Listening IP port for receiving SNMP traps. By default it's set to `127.0.0.1:1620`, using the SNMP Trap default of `162` requires running Docker as root.
community		SNMPv1/v2c community string for receiving SNMP traps.
version		SNMP version to use. Options are `v1`, `v2c`, and `v3`. By default, it's set to `v2c`.
transport		SNMP transport protocol to use. Options are `TCP` and `UDP`. By default, it's set to `UDP`
v3_config		SNMP v3 config to use. Only used if `version: v3`.

Key name	Required	Description
cidrs	✓	Array of target IP ranges in CIDR notation.
debug		Indicates whether to enable debug level logging during discovery. By default, it's set to `false`
ports	✓	Array of target ports to scan during SNMP polling.
default_communities	✓ (Required for SNMPv1/2c)	Array of SNMPv1/v2c community strings to scan during SNMP polling. This array is evaluated in order and discovery accepts the first passing community.
use_snmp_v1	✓ (Required for SNMPv1)	Indicates whether to use SNMPv1 during discovery. By default, it's set to `false`
default_v3	✓ (Required for SNMPv3)	SNMPv3 configuration to scan during SNMP polling.
add_devices	✓	Indicates whether to add discovered devices to the `devices` section of the `snmp-base.yaml` file. By default, it's set to `true`.
add_mibs	✓	Indicates whether to add discovered MIBs to the `global.mibs_enabled` section of the `snmp-base.yaml` file. By default, it's set to `true`.
threads	✓	Integer limit of threads to use during discovery. It should be less than the number of cores available to the container. By default it's set to `4`.
replace_devices	✓	Indicates whether to replace discovered devices if they already exist in the `devices` section of the `snmp-base.yaml` file. By default, it's set to `false`.
no_dedup_engine_id		When set to `true`, disables deduplication of discovered devices if it appears that they are the same device, based on their reported SNMP engine ID. By default, it's set to `false`

Key name	Required	Description
poll_time_sec	✓	Time in seconds to poll devices. This can be overridden per device using the `devices.<deviceName>.poll_time_sec` attribute. By default, it's set to `60`.
drop_if_outside_poll		Indicates whether to drop all values from this cycle if polling takes longer than the value set in `poll_time_sec`. By default, it's set to `false`
mib_profile_dir		Directory to find curated MIB profiles. These are pulled into the `ktranslate` image automatically from Kentik's snmp-profiles repository and can be overridden at Docker runtime by creating a volume mount of your own local directory of profiles.
mibs_db
mibs_enabled	✓	Array of all active MIBs the `ktranslate` docker image will poll. This list is automatically generated during discovery if the `discovery_add_mibs` attribute is `true`. MIBs not listed here will not be polled on any device in the configuration file. You can specify a SNMP table directly in a MIB file using `MIB-NAME.tableName` syntax. Ex: `HOST-RESOURCES-MIB.hrProcessorTable`.
timeout_ms	✓	Time in milliseconds SNMP queries timeout. This can be overridden per device using the `devices.<deviceName>.timeout_ms` attribute. By default, it's set to `5000`
retries	✓	Number of attempts to retry failed SNMP polls. This can be overridden per device using the `devices.<deviceName>.retries` attribute. By default, it's set to `0`
user_tags		`key:value` pair attributes to give more context to the device. Tags at this level will be applied to all devices in the configuration file.
match_attributes		`attribute:regex` pairs to add metrics to allowlist. Pairs at this level will matched against all devices in the configuration file. Uses the RE2 syntax and has a default `OR` operator. Prefix key with `!` to force to `AND` operators.
response_time		Indicates whether response time polling is enabled for all devices in the configuration file. By default, it's set to `false`.

Key name	Required	Description
user_name	✓	User name for SNMPv3 authentication
authentication_protocol	✓	SNMPv3 authentication protocol. The possible values are `NoAuth`, `MD5`, or `SHA`
authentication_passphrase		SNMPv3 authentication passphrase
privacy_protocol	✓	SNMPv3 privacy protocol. The possible values are `AuthNoPriv`, `DES`, `AES`, `AES192`, `AES256`, `AES192C`, or `AES256C`
privacy_passphrase		SNMPv3 privacy passphrase
context_engine_id		SNMPv3 context engine ID
context_name		SNMPv3 context name

Tip

You can use AWS Secrets Manager natively in your SNMP v3 config using the aws.sm.$SECRET_NAME syntax, replacing $SECRET_NAME as necessary to have ktranslate pull in your credentials during Docker runtime.

Optional external config files

To support a wide variety of configuration and automation needs, you can use external files that you volume mount into your Docker container to decouple certain elements of the standard configuration file.

The syntax for these files is "@fileName.extension", including the double quotes.

Discovery CIDRs

Example:

discovery:
  cidrs: "@cidrs.yaml"

The CIDRs file should use a YAML list syntax like this:

- 10.10.0.0/24
- 10.20.0.0/24
- 192.168.0.21/32

Devices

Example:

devices:
  - "@neteng-devices.yaml"
  - "@dc-ops.yaml"

The device files should use the same syntax as the standard devices section of the main config file, omitting the optional fields that are generated during discovery:

devices:
  # Sample of SNMP v2c device
  ups_snmpv2c__10.10.0.201:
    device_name: ups_snmpv2c
    device_ip: 10.10.0.201
    snmp_comm: public
    oid: .1.3.6.1.4.1.318.1.3.27
    mib_profile: apc_ups.yml
    provider: kentik-ups
    poll_time_sec: 300
    retries: 1
    timeout_ms: 5000
    user_tags:
      owning_team: dc_ops

SNMPv3 - AWS Secrets

ktranslate has built-in support for retrieving keys from AWS Secrets Manager to use in your SNMPv3 configuration.

To use this feature, you will need to set the following 3 environmental variables and provide them to Docker at runtime:

Name	Description
AWS_ACCESS_KEY_ID	Specifies the AWS access key used as part of the credentials to authenticate the user.
AWS_SECRET_ACCESS_KEY	Specifies the AWS secret key used as part of the credentials to authenticate the user.
AWS_REGION	Specifies the AWS Region to send requests to for commands requested using this profile.

Example for Docker runtime:

bash

$docker run -d --name ktranslate-snmp --restart unless-stopped --net=host \
>  -v `pwd`/snmp-base.yaml:/snmp-base.yaml \
>  -e NEW_RELIC_API_KEY=$YOUR_NR_LICENSE_KEY \
>  -e AWS_ACCESS_KEY_ID=$YOUR_AWS_ACCESS_KEY_ID \
>  -e AWS_SECRET_ACCESS_KEY=$YOUR_AWS_SECRET_ACCESS_KEY \
>  -e AWS_REGION=$YOUR_AWS_REGION \
>  kentik/ktranslate:v2 \
>    -snmp /snmp-base.yaml \
>    -nr_account_id=$YOUR_NR_ACCOUNT_ID \
>    -log_level=info \
>    -metrics=jchf \
>    -tee_logs=true \
>    nr1.snmp

In your associated configuration file (snmp-base.yaml); you would update your SNMPv3 config snippet as follows, prefixing the secret name with aws.sm.:

default_v3:
    user_name: aws.sm.SECRET_NAME_1
    authentication_protocol: MD5
    authentication_passphrase: aws.sm.SECRET_NAME_2
    privacy_protocol: AES256
    privacy_passphrase: aws.sm.SECRET_NAME_3

Tip

You need to create dedicated secrets per value. Note in the example above there are three distinct secret names in use.

The match_attributes attribute

To support filtering of data that does not create value for your observability needs, you can set the global.match_attributes.{} and/or devices.<deviceName>.match_attributes.{} attribute map.

This will provide filtering at the ktranslate level, before shipping data to New Relic, giving you granular control over monitoring of things like interfaces.

The default behavior of this map is an OR condition, but you can override this and force an AND operator by prefixing your key name with !. This is also useful to return only matched items and omit all null and "" (empty) results.

Match when if_Alias begins with Uplink OR when if_interface_name begins with Gig, keep all null and "" values:

devices:
  deviceName:
    ...
    match_attributes:
      if_Alias: "^Uplink.*"
      if_interface_name: "^Gig.*"

Match when if_Alias begins with Uplink AND when if_interface_name begins with Gig, drop all null and "" values:

devices:
  deviceName:
    ...
    match_attributes:
      if_Alias: "^Uplink.*"
      "!if_interface_name": "^Gig.*"

The flow_only attribute

To support monitoring of devices where performance statistics are nor accessible, available, or desired, you can set the devices.<deviceName>.flow_only attribute to true.

This will generate a Flow Device entity in New Relic One which will only have telemetry in the KFlow event namespace. Alternatively, collecting flow telemetry from a device that is in your configuration file as an SNMP device will add decoration of the KFlow data to the pre-existing entity, such as a Router or Firewall.

In New Relic One, you can see the results of this polling by investigating the following events:

FROM KFlow SELECT count(*) FACET device_name WHERE instrumentation.name = 'netflow-events' TIMESERIES

The response_time and ping_only attributes

To support monitoring of devices where performance statistics are not accessible or available, or in simple cases where basic round-trip time (RTT) monitoring is required, you can either set the global.response_time or devices.<deviceName>.ping_only attributes to true.

This feature uses the go-ping package to send unprivileged UDP packets to devices in order to collect the average, min, max, and stddev round-trip time (RTT). This package also shows packet loss percentage for the endpoint based on sending one packet/sec from ktranslate to the device IP address. You can enable the use of privileged ICMP packets by setting the KENTIK_PING_PRIV=true environment variable during Docker runtime.

Setting the global.response_time attribute to true will add RTT monitoring on top of existing SNMP polling. To monitor devices with only the UDP|ICMP packets for RTT and no SNMP polling, use devices.<deviceName>.ping_only: true.

In New Relic One, you can see the results of this polling by investigating the following metrics:

FROM Metric SELECT average(kentik.ping.AvgRttMs) AS 'Average', max(kentik.ping.MaxRttMs) AS 'Max', min(kentik.ping.MinRttMs) AS 'Min' FACET device_name TIMESERIES

Tip

You can use the ping_only attribute in replacement of the flow_only attribute if you would like to collect RTT metrics from a flow device. If both ping_only and flow_only are true, the device will be treated as a flow_only device.

Flow data application mapping

By default, flow telemetry is mapped to known applications based on evaluation of the layer 4 port in use on a specific flow conversation. If needed, you can override the default mapping by providing a YAML file during Docker runtime to the -application_map flag. This will allow you to specify application names based on ports you identify.

Example syntax:

applications:
  - ports: [9092, 9093]
    name: kafka
  - ports: [80, 8080]
    name: http
  - ports: [443, 8443]
    name: https

Flow data input filtering

By default, flow data containers will collect and process every flow packet they receive. If needed, you can add an inclusion filter to the -nf.source flag that will ignore all traffic not matching the filter you provide.

Syntax: --filters $TYPE,$FIELD,$FUNCTION,$MATCH

Argument Name	Required	Description
$TYPE	✓	The type of filter to apply. Possible values are `string`, `int`, and `addr`.
$FIELD	✓	The name of the field to evaluate the match pattern against.
$FUNCTION	✓	The type of function to use during evaluation. Possible values are `Equal: ==`, `NotEqual: !=`, `LessThan: <`, `GreaterThan: >`, `Contains: %`
$MATCH	✓	The value to be used as a match pattern.

Example Filters

Only collect flow data from source addresses in the 10.0.0.0/24 CIDR range

-nf.source sflow --filters addr,src_addr,%,10.10.0.0/24

Only collect flow data where the destination port is not equal to 8531

-nf.source netflow5 --filters int,l4_dst_port,!=,8531

You can also add multiple filters together with an inherited AND operator

Only collect flow data from source addresses in the 10.0.0.0/24 CIDR range AND where the destination port is not equal to 8531

--filters addr,src_addr,%,10.0.0.0/24 --filters int,l4_dst_port,!=,8531

Advanced configuration for network performance monitoring

SNMP-base YAML sample file

Devices section

Trap section

Discovery section

Global section

Optional SNMPv3 configuration

Tip