Creating Notifications¶

Playbooks can generate notifications to let a human know about something in your cluster.

A Finding is Robusta's term for a notification.

This guide explains how to create and modify Findings in customPlaybooks.

Creating Notifications¶

There are two generic actions for working with notifications:

• create_finding

• customise_finding

create_finding will generate a new notification.

customise_finding will modify an existing notification, already created by other actions.

For example, we can use create_finding to notify about Pods that were killed because they used up memory.

customPlaybooks:
- triggers:
  - on_pod_oom_killed: {} # (1)
  actions:
  - create_finding:
      title: "Pod $name in namespace $namespace OOMKilled results"
      aggregation_key: "Pod OOMKill"

Enriching Notifications¶

Robusta's true power is the ability to build detailed and context-aware notifications.

Lets use these capabilities to improve the above example. When a Pod is OOMKilled, we usually investigate by:

1. Viewing a graph of Pod memory usage

2. Running kubectl describe and investigating

We can automate this with Robusta:

customPlaybooks:
- triggers:
  - on_pod_oom_killed: {} # (1)
  actions:
  - create_finding:
      title: "Pod $name in namespace $namespace OOMKilled results"
      aggregation_key: "Pod OOMKill"
  - pod_graph_enricher: # (2)
      resource_type: Memory
      display_limits: true
  - pod_oom_killer_enricher: {} # (3)

Here is the result in Slack:

Automatic Findings¶

What happens if you call an enrichment action like pod_graph_enricher but you never call create_finding first?

No worries. In cases like this, a default Finding (notification) is created when the enricher runs. The Finding's title is set automatically based on the event that triggered the action.

This means that the order of actions within a playbook is important! Put create_finding before other actions, so that subsequent actions already have a Finding to work with. If you put pod_graph_enricher before create_finding, you'll end up with two notification messages - one created implicitly by calling pod_graph_enricher and one created explicitly by create_finding.