Smart Triggers

These high-level triggers identify interesting events in your cluster.

Under the hood, smart triggers are implemented on top of lower-level triggers.
For example, the on_pod_crash_loop trigger internally listens to other Kubernetes (API Server) triggers and applies logic to fire only on crashing pods.

Example triggers

Pod Crash Loop

  • on_pod_crash_loop

This trigger will fire when a Pod is crash looping.

customPlaybooks:
- triggers:
  - on_pod_crash_loop:
      restart_reason: "CrashLoopBackOff"
  actions:
  - report_crash_loop: {}

Trigger parameters:

  • restart_reason: Limit restart loops for this specific reason. If omitted, all restart reasons will be included.

  • restart_count: Fire only after the specified number of restarts

  • rate_limit: Limit firing to once every rate_limit seconds

Pod OOMKilled

  • on_pod_oom_killed

This trigger will fire when a Pod is OOMKilled.

customPlaybooks:
- triggers:
  - on_pod_oom_killed:
      rate_limit: 900
      exclude:
        - name: "oomkilled-pod"
          namespace: "default"
  actions:
  - pod_graph_enricher:
      resource_type: Memory
      display_limits: true

Trigger parameters:

  • rate_limit: Limit firing to once every rate_limit seconds

  • exclude: A list of pod name prefixes and/or namespaces that this trigger will ignore.

    • All pods that start with name in namespace namespace will be ignored for this trigger.

    • If A name is defined without a namespace than all pods with that name prefix will be ignored for this trigger.

    • If A namespace is defined without a name than all pods in that namespace will be ignored for this trigger.

Container OOMKilled

  • on_container_oom_killed

This trigger will fire when a Container is OOMKilled.

customPlaybooks:
- triggers:
  - on_container_oom_killed:
      rate_limit: 900
      exclude:
        - name: "oomkilled-container"
          namespace: "default"
  actions:
  - oomkilled_container_graph_enricher:
      resource_type: Memory

Trigger parameters:

  • rate_limit: Limit firing to once every rate_limit seconds

  • exclude: A list of container name prefixes and/or namespaces that this trigger will ignore.

    • All containers that start with name in namespace namespace will be ignored for this trigger.

    • If A name is defined without a namespace than all containers with that name prefix will be ignored for this trigger.

    • If A namespace is defined without a name than all containers in that namespace will be ignored for this trigger.

Job Failure

  • on_job_failure

This trigger will fire when the Job status is updated to "failed".

customPlaybooks:
- triggers:
  - on_job_failure:
      namespace_prefix: robusta
  actions:
  - create_finding:
      title: "Job $name on namespace $namespace failed"
      aggregation_key: "Job Failure"
  - job_events_enricher: { }

Note

Have an idea for another smart trigger? Please open a github issue