PagerDuty¶

PagerDuty is a popular incident response tool.

Robusta is a popular Kubernetes monitoring solution, based on Prometheus. Robusta can send three types of data to the PagerDuty API:

Change Events - for example, when Deployments are updated
Enriched Prometheus alerts - Robusta receives the alert from Prometheus, attaches context like Pod logs, and forward to PagerDuty
Standalone alerts - if you don’t use Prometheus, Robusta can still send alerts to PagerDuty for errors like CrashLoopBackOff

Prerequisites¶

You need an integration key for a PagerDuty service. Here is how to generate it.

Login to your PagerDuty dashboard
Navigate to Service > New Service (Top right)
Input Name and Description and Tap on the Next button
Choose Generate a new Escalation Policy or Select an existing Escalation Policy and Tap on next
On the next screen input the configuration details
On the Integrations screen, search for and select the integration "Robusta.dev" and tap on the Create Service button in the bottom on the screen
Copy the Integration Key. A sample key would look like: f6c6e02a5a1a490ee02e90cde19ee388

Configuring the PagerDuty sink¶

Sending Alerts to PagerDuty

To send alerts from Robusta to PagerDuty, add the following code to your generated_values.yaml file. This will send all alerts Robusta receives - whether they originate in Prometheus or in Robusta itself.

sinksConfig:
    - pagerduty_sink:
        name: main_pagerduty_sink
        api_key: <api key> # e.g. f653634653463678fadas43534506

Save the file and run

 helm upgrade robusta robusta/robusta --values=generated_values.yaml

Example Output:

To view the Alerts, on PagerDuty's dashboard navigate to Incidents > Alerts

Sending Kubernetes Changes to PagerDuty

To send Kubernetes changes from Robusta to PagerDuty, add the following code to your generated_values.yaml file. This will send all changes to PagerDuty, in addition to the alerts mentioned above.

sinksConfig:
    - pagerduty_sink:
        name: main_pagerduty_sink
        api_key: <api key> # e.g. f653634653463678fadas43534506
customPlaybooks:
   - actions:
     - resource_babysitter: {}
     sinks:
     - main_pagerduty_sink
     triggers:
     - on_deployment_all_changes: {}
     - on_daemonset_all_changes: {}
     - on_statefulset_all_changes: {}

Save the file and run

 helm upgrade robusta robusta/robusta --values=generated_values.yaml

Example Output:

To view the Changed Events, on PagerDuty's dashboard navigate to Incidents > Recent Changes

Support¶

If you need help with the PagerDuty integration, get in touch with the Robusta team:

Slack - link only works on desktop
GitHub Issues
Email - support@robusta.dev