Track Kubernetes Changes¶

Robusta lets you get notifications when Kubernetes resources are updated. Users can set up personalized notifications for any Deployment, ReplicaSet, or other resource, ensuring you get notified when new versions are rolled out or other engineers change something important in the cluster. This feature is especially useful for various roles:

DevOps and Platform Teams can track all changes to Ingresses and other sensitive cluster resources.
Developers can receive notifications each time their application is deployed to production.
Security and DevSecOps professionals can track changes to ClusterRoles or ServiceAccounts.

How to Track Changes in Kubernetes Resources¶

Create Custom Playbook: Start by defining a personalized template that specifies when you should be notified and what data you'd like to see. This is your "custom playbook."
Select Kubernetes Object: In your custom playbook, specify which Kubernetes resource you want to monitor, such as Deployment or ReplicaSet.
Filter YAML Fields: To avoid unnecessary notifications, select specific YAML field. For example, when tracking an autoscaled Deployment, you can filter out notifications related to Deployment.spec.replicas, as this field is automatically updated by the Horizontal Pod Autoscaler (HPA) regularly.
Set Up Change Detection: Configure your playbook to send a 'diff' that shows exactly what changed in the selected Kubernetes object.
Route Alerts (Optional): If needed, direct these change notifications to specific destinations, also known as 'Sinks', by adding this information to your custom playbook.

Kubernetes Change Tracking Use Cases¶

Let's explore practical use cases for Kubernetes change tracking.

Use Case 1: Notification on Deployment Image Change¶

Scenario: You want to be notified when a Deployment strategy or container details are changed.

Implementation:

Add the following YAML to the customPlaybooks Helm value:

customPlaybooks:
- triggers:
    - on_deployment_update:
        change_filters:
          ignore: # These are ignored by default
          - status
          - metadata.generation
          - metadata.resourceVersion
          - metadata.managedFields
          - spec.replicas
          include:
            - spec.template.spec.containers[0]
            - spec.strategy
  actions:
    - resource_babysitter: {}
    - customise_finding:
        severity: MEDIUM
        title: "New changes in $kind/$namespace/$name"
  sinks:
    - some_sink_name # Optional

How does it work?

Initialize Custom Playbook: Create a custom playbook where you'll outline the rules for when and how you'll be notified.
Set Up the Deployment Trigger: In your custom playbook, add the on_deployment_update trigger. This ensures you'll receive notifications for deployment changes.
Specify Fields to Monitor: Add change_filters to your on_deployment_update trigger to filter which changes you will be notified for.
Route Notifications (Optional): Optionally, specify in your playbook where these notifications should be sent by defining 'sinks'.

Then perform a Helm Upgrade.

Note: You can also use the Sink Matchers to route notifications instead of explicitly specifying a sink in the playbook.

Testing:

Modify the image of a deployment in your cluster.

Run the following YAML files to simulate a deployment image change

kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/deployment_image_change/before_image_change.yaml
kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/deployment_image_change/after_image_change.yaml

A Robusta notification will arrive in your configured sinks, showing exactly what changed in the deployment.

Sample Alert:

Use Case 2: Notification on Ingress Rules Change¶

Scenario: You want to be notified when an Ingress rules or tls details are changed.

Implementation:

Add the following YAML to the customPlaybooks Helm value:

customPlaybooks:
- triggers:
    - on_ingress_all_changes:
        change_filters:
          ignore:
            - status
            - metadata.generation
            - metadata.resourceVersion
            - metadata.managedFields
            - spec.replicas
          include:
            - spec.rules
            - spec.tls
  actions:
    - resource_babysitter: {}
  sinks:
    - some_sink_name # Optional

How does it work?

Initialize Custom Playbook: Create a custom playbook where you'll outline the rules for when and how you'll be notified.
Set Up the Ingress Trigger: In your custom playbook, add the on_ingress_all_changes trigger. This ensures you'll receive notifications for all ingress changes.
Specify Fields to Monitor: Add change_filters to your on_ingress_all_changes trigger to filter which changes you will be notified for.
Route Notifications (Optional): Optionally, specify in your playbook where these notifications should be sent by defining 'sinks'.

Then perform a Helm Upgrade.

Note: You can also use the Sink Matchers to route notifications instead of explicitly specifying a sink in the playbook.

Testing:

Create, modify, or delete an ingress in your cluster.

Run the following commands to simulate ingress changes:

kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/ingress_port_path_change/before_port_path_change.yaml
kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/ingress_port_path_change/after_port_path_change.yaml

A Robusta notification will arrive in your configured sinks, showing exactly what changed in the ingress.

Sample Alert:

Cleanup¶

Remove the playbook you added based on your specific use case from the customPlaybooks in your generated_values.yaml file. Then, perform a Helm Upgrade.