Install the Kubernetes integration using Helm

Helm is a package manager on top of Kubernetes. It facilitates installation, upgrades, or revision tracking, and it manages dependencies for the services that you install in Kubernetes.

To install the integration using Helm, we recommend our Kubernetes automated installer, which will prompt for some configuration options and autopopulate secrets and values for you. Additionally, our automated installer also allows installing our integration as plain manifests rather than a Helm release. See Kubernetes integration: install and configure for more details about how to use our automated installer.

Start the installer

This page describes in more depth how to install and configure the New Relic integration without using the automated installer.

Compatibility and requirements

Make sure Helm is installed on your machine. Version 3 of the Kubernetes Integration requires Helm version 3. If you are still using Helm 2, you can still install the legacy version of the integration.

To install the Kubernetes integration using Helm, you will need your New Relic account license key and your Kubernetes cluster's name:

Find and copy your New Relic license key.
Choose a display name for your cluster. For example, you could use the output of:

bash

$kubectl config current-context

Note these values somewhere safe, as you will need them later during the installation process.

Install Kubernetes integration with Helm

New Relic has several charts for the different components which offer different features for the platform:

newrelic-infrastructure-v3: Contains the main Kubernetes integration and the infrastructure agent. This is the core component for the New Relic Kubernetes experience, responsible for reporting most of the data that is surfaced in the Kubernetes Dashboard and the Kubernetes Cluster Explorer.
newrelic-logging: Provides a DaemonSet with New Relic's Fluent Bit output plugin to easily forward your logs to New Relic.
nri-kube-events: Collects and reports cluster events (such as kubectl get events) to New Relic.
nri-prometheus: New Relic's Prometheus OpenMetrics Integration, automatically scrapes Prometheus endpoints present in the cluster and reports metrics to New Relic.
nri-metadata-injection: Sets up a minimal MutatingAdmissionWebhook that injects a couple of environment variables in the containers. These contain metadata about the cluster and New Relic installation and will be later picked up by applications instrumented using APM, allowing to correlate APM and infrastructure data.
nri-statsd: New Relic StatsD integration.

For convenience, New Relic provides the nri-bundle chart, which pulls a selectable set of the charts mentioned above. nri-bundle can also install Kube State Metrics and Pixie for you if needed.

While it is possible to install those charts separately, we strongly recommend using the nri-bundle chart for Kubernetes deployments, as it ensures that values across all the charts are consistent and provides full control over which components are installed, as well as the possibility to configure all of them as Helm dependencies. This is the same chart that is used and referenced by our automated installer.

Installing and configuring `nri-bundle` with Helm

Ensure you are using the appropriate context in the machine where you will run Helm and kubectl:

You can check the available contexts with:

bash

$kubectl config get-contexts

And switch to the desired context using:

bash

$kubectl config use-context _CONTEXT_NAME_

Add the New Relic Helm charts repo:

bash

$helm repo add newrelic https://helm-charts.newrelic.com

Create a file named values-newrelic.yaml, which will be used to define your configuration:

global:
  licenseKey: _YOUR_NEW_RELIC_LICENSE_KEY_
  cluster: _K8S_CLUSTER_NAME_

prometheus:
  # Automatically scrape prometheus metrics for annotated services in the cluster
  # Collecting prometheus metrics for large clusters might impact data usage significantly
  enabled: true
  lowDataMode: true  # Skip ingesting cluster-level metrics
webhook:
  # Deploy our webhook to link APM and Kubernetes entities
  enabled: true
kubeEvents:
  # Report Kubernetes events
  enabled: true
logging:
  # Report logs for containers running in the cluster
  enabled: true
ksm:
  # Deploy kube-state-metrics in the cluster.
  # Set this to true unless it is already deployed.
  enabled: true

Make sure everything is configured properly in the chart by running the following command. Notice that we are specifying --dry-run and --debug, so nothing will be installed in this step:

bash

$helm upgrade --install newrelic newrelic/nri-bundle \
>--namespace newrelic --create-namespace \
>-f values-newrelic.yaml \
>--devel \
>--dry-run \
>--debug

Tip

By specifying --devel, you will be installing the version 3 of our solution, currently in Beta and scheduled to be generally available during Spring 2022. We strongly encourage you to try it out as it includes significant improvements over the v2. See what's changed.

Please notice and adjust the following flags:

global.licenseKey=YOUR_NEW_RELIC_LICENSE_KEY: Must be set to a valid License Key for your account.
global.cluster=K8S_CLUSTER_NAME: Is used to identify the cluster in the New Relic UI, so should be a descriptive value not used by any other Kubernetes cluster configured in your New Relic account.
ksm.enabled=true: Setting this to true will automatically install Kube State Metrics (KSM) for you, which is required for our integration to run. You can set this to false if KSM is already present in your cluster, even if it is on a different namespace.
prometheus.enabled=true: Will deploy our Prometheus OpenMetrics integration, which automatically collects data from Prometheus endpoints present in the cluster.
webhook.enabled=true: Will install our minimal webhook, which adds environment variables that, in turn, allows linking applications instrumented with New Relic APM to Kubernetes.

Our Kubernetes charts have a comprehensive set of flags and tunables that can be edited to better fit your particular needs. Please check the Configure the integration section below to see what can be changed.

Install the Kubernetes integration by running the command without --debug and --dry-run:

bash

$helm upgrade --install newrelic newrelic/nri-bundle \
>--namespace newrelic --create-namespace \
>-f values-newrelic.yaml \
>--devel

Check that pods are being deployed and reach a stable state:

bash

$kubectl -n newrelic get pods -w

You should see:

One newrelic-nrk8s-ksm pod.
One newrelic-nrk8s-kubelet pod for each node in your cluster.
One newrelic-nrk8s-control-plane pod for each master node in your cluster, if any.
One newrelic-kube-state-metrics pod, if you included KSM with our installation.
One newrelic-nri-kube-events pod, if you enabled Kubernetes events reporting.
One newrelic-nri-prometheus pod, if you enabled the Prometheus integration.
One newrelic-newrelic-logging pod for each node in your cluster, if you enabled the Logging integration.

Install with Helm 2 and nri-bundle (legacy)

Important

The last version the nri-bundle chart supporting Helm 2 is 3.2.11. Please upgrade to Helm 3 to install the latest version. We will be removing support for Helm 2 installations soon.

To install using Helm 2:

Add the New Relic Helm charts repo:

bash

$helm repo add newrelic https://helm-charts.newrelic.com

Create a namespace for newrelic:
bash
```
$kubectl create namespace newrelic
```

Make sure everything is configured properly in the chart by running the following command. This step uses the --dry-run and --debug switches and therefore the agent is not installed.

bash

$helm upgrade --install newrelic newrelic/nri-bundle \
>--version 3.2.11 \
>--dry-run \
>--debug \
>--namespace newrelic \
>--set global.licenseKey=_YOUR_NEW_RELIC_LICENSE_KEY_ \
>--set global.cluster=_K8S_CLUSTER_NAME_ \
>--set ksm.enabled=true \
>--set newrelic-infrastructure.privileged=true \
>--set prometheus.enabled=true \
>--set webhook.enabled=true \
>--set kubeEvents.enabled=true \
>--set logging.enabled=true

Install the New Relic Kubernetes integration by running the same command without --dry-run and --debug

bash

$helm upgrade --install newrelic newrelic/nri-bundle \
>--version 3.2.11 \
>--namespace newrelic \
>--set global.licenseKey=_YOUR_NEW_RELIC_LICENSE_KEY_ \
>--set global.cluster=_K8S_CLUSTER_NAME_ \
>--set ksm.enabled=true \
>--set newrelic-infrastructure.privileged=true \
>--set prometheus.enabled=true \
>--set webhook.enabled=true \
>--set kubeEvents.enabled=true \
>--set logging.enabled=true

Wait a few seconds, then check that the DaemonSet and pods have been created:
Check that pods are being deployed and reach a stable state:
bash
```
$kubectl -n newrelic get pods -w
```
Make sure you see a DaemonSet, and one pod per node.
Helm configuration options
When you install or upgrade the Kubernetes integration with Helm using the command line, you can pass your configuration variables with the --set flag.
bash
```
$helm install newrelic/newrelic-infrastructure \
>--set licenseKey=_YOUR_NEW_RELIC_LICENSE_KEY_ \
>--set cluster=_YOUR_CLUSTER_NAME_
```
A full list of the configuration parameters can be found in the newrelic-infrastructure chart README

Configure the integration

Our nri-bundle chart. whose installation instructions can be found above, acts as a wrapper or a meta-package for a couple of other charts, which are the ones containing the components for our solution. By offering such a wrapper we can provide a controlled set of our components with versions that we know are compatible with each other, while keeping the component's charts relatively simple.

To configure the individual integration components, you must use Helm's dependency system, which simply means that configuration for the children charts must be put under a section with the name of said chart. For example, to configure the newrelic-infrastructure chart, you would add the following to the values-newrelic.yaml:

global:
  licenseKey: _YOUR_NEW_RELIC_LICENSE_KEY_
  cluster: _K8S_CLUSTER_NAME_

# ... Other settings as shown above

# Configuration for newrelic-infrastructure
newrelic-infrastructure:
  verboseLog: true  # Enable debug logs
  privileged: false  # Install with minimal privileges
  # Other options from https://github.com/newrelic/helm-charts/tree/master/charts/newrelic-infrastructure-v3

The full list of flags that can be tweaked can be found in our chart's repository:

newrelic-infrastructure
Configure debug logs, privilege mode, control plane monitoring, etc.
nri-kube-events
nri-metadata-injection
Configure how the webhook for APM linkage is deployed.
nri-prometheus
Configure which Prometheus endpoints are scraped.
newrelic-logging
Configure which logs are sent to New Relic.

Tip

Remember that when specifying options for these charts, you must put them under the chart name in your values-newrelic.yaml.

Change the scrape interval

The Kubernetes Integration v3 and above allows changing the interval at which metrics are gathered from the cluster. This allows choosing a tradeoff between data resolution and usage. We recommend choosing an interval between 15 and 30 seconds for optimal experience.

In order to change the scrape interval, add the following to your values-newrelic.yaml, under the newrelic-infratructure section:

common:
  config:
    interval: 25s

So it ends up looking like:

global:
  licenseKey: _YOUR_NEW_RELIC_LICENSE_KEY_
  cluster: _K8S_CLUSTER_NAME_

# ... Other settings as shown above

# Configuration for newrelic-infrastructure
newrelic-infrastructure:
  # ... Other settings as shown above
  common:
    config:
      interval: 25s

Important

Setting interval to values larger than 40s is not allowed.

A full list of the settings that can be modified can be found at the chart's README.

Upgrade using Helm

To update your Kubernetes integration installed via Helm:

Update the local chart repository:
bash
```
$helm repo update
```

Update the release by running again the appropriate helm upgrade --install ... command in the section above

bash

$helm upgrade --install newrelic newrelic/nri-bundle \
>--namespace newrelic --create-namespace \
>-f values-newrelic.yaml \
>--devel

Monitor services running on Kubernetes

After having installed our Kubernetes integration, you can start instrumenting the services that run in your cluster. To learn more about how to do this, please check our Monitor services running on Kubernetes page.

Use your Kubernetes data

To learn more about how to use your Kubernetes data, please head to our detailed Find and use your Kubernetes data pages.

Reduce data ingest

Our charts support setting an option to reduce the amount of data ingested at the cost of dropping detailed information. To enable it, set global.lowDataMode to true in the nri-bundle chart.

lowDataMode affects three specific components of the nri-bundle chart outlined below.

New Relic Infrastructure

If lowDataMode is enabled, the default scrape interval changes from 15s to 30s. You can also specify a custom value for it using config.interval, which will take preference over lowDataMode.

Prometheus OpenMetrics Integration

If lowDataMode is enabled, the following metrics are excluded by default as they are already collected and used by the New Relic Kubernetes Integration.

- kube_
- container_
- machine_
- cadvisor_

New Relic Logging

If lowDataMode is enabled, Labels and Annotations are set to Off in the Filter section of the fluent-bit.conf file. This means that this detail will be dropped from the container log files which reduces the overall data ingest into New Relic.

The following fields are retained:

Allowlist_key  container_name
Allowlist_key  namespace_name
Allowlist_key  pod_name
Allowlist_key  stream
Allowlist_key  log

Low Data Mode Log Example

Complete Log Record

[
  {
    "cluster_name": "api-test",
    "kubernetes": {
      "annotations": {
        "kubernetes.io/psp": "eks.privileged"
      },
      "container_hash": "fryckbos/test@sha256:5b098eaf3c7d5b3585eb10cebee63665b6208bea31ef31a3f0856c5ffdda644b",
      "container_image": "fryckbos/test:latest",
      "container_name": "newrelic-logging",
      "docker_id": "134e1daf63761baa15e035b08b7aea04518a0f0e50af4215131a50c6a379a072",
      "host": "ip-192-168-17-123.ec2.internal",
      "labels": {
        "app": "newrelic-logging",
        "app.kubernetes.io/name": "newrelic-logging",
        "controller-revision-hash": "84db95db86",
        "pod-template-generation": "1",
        "release": "nri-bundle"
      },
      "namespace_name": "nrlogs",
      "pod_id": "54556e3e-719c-46b5-af69-020b75d69bf1",
      "pod_name": "nri-bundle-newrelic-logging-jxnbj"
    },
    "message": "[2021/09/14 12:30:49] [ info] [engine] started (pid=1)\n",
    "plugin": {
      "source": "kubernetes",
      "type": "fluent-bit",
      "version": "1.8.1"
    },
    "stream": "stderr",
    "time": "2021-09-14T12:30:49.138824971Z",
    "timestamp": 1631622649138
  }
]

Log Record after enabling lowDataMode.

[
  {
    "cluster_name": "api-test",
    "container_name": "newrelic-logging",
    "namespace_name": "nrlogs",
    "pod_name": "nri-bundle-newrelic-logging-jxnbj",
    "message": "[2021/09/14 12:30:49] [ info] [engine] started (pid=1)\n",
    "stream": "stderr",
    "timestamp": 1631622649138
  }
]

New Relic Pixie Integration

If lowDataMode is enabled, the newrelic-pixie integration performs heavier sampling on Pixie spans and reduces the collection interval from 10 seconds to 15 seconds.

lowDataMode settings:

HTTP_SPAN_LIMIT: 750
DB_SPAN_LIMIT: 250
COLLECT_INTERVAL_SEC: 15

The default settings for these parameters and others can be found in the newrelic-pixie-integration Github repo.

Uninstall Kubernetes integration

To uninstall the Kubernetes integration using Helm, run the following command:

bash

$helm uninstall newrelic -n newrelic

Install the Kubernetes integration using Helm

Compatibility and requirements

Install Kubernetes integration with Helm

Installing and configuring nri-bundle with Helm

Tip

Install with Helm 2 and nri-bundle (legacy)

Configure the integration

Tip

Change the scrape interval

Important

Upgrade using Helm

Monitor services running on Kubernetes

Use your Kubernetes data

Reduce data ingest

New Relic Infrastructure

Prometheus OpenMetrics Integration

New Relic Logging

New Relic Pixie Integration

Uninstall Kubernetes integration

Installing and configuring `nri-bundle` with Helm