Advanced Playbook TechniquesΒΆ

This guide assumes you already know playbook basics and how to create notifications. It explains implementation details and common techniques.

Using Filters to Restrict TriggersΒΆ

Many triggers have parameters that restrict when they fire:

- triggers:
  - on_pod_crash_loop:
      restart_reason: "CrashLoopBackOff"
      name_prefix: fluentbit
      namespace_prefix: kube-system

Most Kubernetes-related triggers support at least name and namespace. Refer to Triggers Reference for details.

Running Multiple Playbooks on the Same EventΒΆ

If multiple triggers match an incoming event, all relevant playbooks execute in the order they were defined. For example:

# first playbook
- triggers:
  - on_deployment_create: {}
  actions:
  - my_first_action: {}

# second playbook
- triggers:
  - on_deployment_create: {}
  actions:
  - my_second_action: {}

In the example above, my_first_action runs before my_second_action.

Multiple Playbook InstancesΒΆ

Likewise, you can enable identical playbooks multiple times with different parameters:

customPlaybooks:
- triggers:
  - on_deployment_update:
      name_prefix: MyApp
  actions:
  - add_deployment_lines_to_grafana:
      grafana_api_key: grafana_key_goes_here
      grafana_dashboard_uid: id_for_dashboard1
      grafana_url: http://grafana.namespace.svc
  sinks:
  - "main_slack_sink"

- triggers:
  - on_deployment_update:
      name_prefix: OtherApp
  actions:
  - add_deployment_lines_to_grafana:
      grafana_api_key: grafana_key_goes_here
      grafana_dashboard_uid: id_for_dashboard2
      grafana_url: http://grafana.namespace.svc
  sinks:
  - "main_slack_sink"

If the triggers in multiple playbooks match the same incoming event, all relevant playbooks will run.

Global Configuration for Playbook ParametersΒΆ

In the previous example, grafana_api_key and grafana_url were defined multiple times with the same value.

To avoid repeating yourself, you can define parameters globally for all playbooks. These parameters will be applied to any action or trigger which expects a parameter with the same name.

globalConfig:
  cluster_name: "my-staging-cluster"
  grafana_api_key: "grafana_key_goes_here"
  grafana_url: http://grafana.namespace.svc

 customPlaybooks:
 - triggers:
   - on_deployment_update:
       name_prefix: MyApp
   actions:
   - add_deployment_lines_to_grafana:
       grafana_dashboard_uid: id_for_dashboard1
   sinks:
   - "main_slack_sink"

 - triggers:
   - on_deployment_update:
       name_prefix: OtherApp
   actions:
   - add_deployment_lines_to_grafana:
       grafana_dashboard_uid: id_for_dashboard2
   sinks:
   - "main_slack_sink"

Stopping Playbook ExecutionΒΆ

An action can stop the processing flow if needed, preventing subsequent actions from being run.

This is useful for silencing actions like node_restart_silencer. These actions need to stop alerts from being propogated to other playbooks.

Only actions following the current action will be stopped. Therefore, silencers must be defined before other playbooks.