Track Kubernetes ChangesΒΆ
Robusta lets you get notifications when Kubernetes resources are updated. Users can set up personalized notifications for any Deployment, ReplicaSet, or other resource, ensuring you get notified when new versions are rolled out or other engineers change something important in the cluster. This feature is especially useful for various roles:
DevOps and Platform Teams can track all changes to Ingresses and other sensitive cluster resources.
Developers can receive notifications each time their application is deployed to production.
Security and DevSecOps professionals can track changes to ClusterRoles or ServiceAccounts.
How to Track Changes in Kubernetes ResourcesΒΆ
Create Custom Playbook: Start by defining a personalized template that specifies when you should be notified and what data you'd like to see. This is your "custom playbook."
Select Kubernetes Object: In your custom playbook, specify which Kubernetes resource you want to monitor, such as Deployment or ReplicaSet.
Filter YAML Fields: To avoid unnecessary notifications, select specific YAML field. For example, when tracking an autoscaled Deployment, you can filter out notifications related to Deployment.spec.replicas, as this field is automatically updated by the Horizontal Pod Autoscaler (HPA) regularly.
Set Up Change Detection: Configure your playbook to send a 'diff' that shows exactly what changed in the selected Kubernetes object.
Route Alerts (Optional): If needed, direct these change notifications to specific destinations, also known as 'Sinks', by adding this information to your custom playbook.
Kubernetes Change Tracking Use CasesΒΆ
Let's explore practical use cases for Kubernetes change tracking.
Use Case 1: Notification on Deployment Image ChangeΒΆ
Scenario: You want to be notified when a Deployment strategy or container details are changed.
Implementation:
Add the following YAML to the customPlaybooks
Helm value:
customPlaybooks:
- triggers:
- on_deployment_update:
change_filters:
ignore: # These are ignored by default
- status
- metadata.generation
- metadata.resourceVersion
- metadata.managedFields
- spec.replicas
include:
- spec.template.spec.containers[0]
- spec.strategy
actions:
- resource_babysitter: {}
- customise_finding:
severity: MEDIUM
title: "New changes in $kind/$namespace/$name"
sinks:
- some_sink_name # Optional
How does it work?
Initialize Custom Playbook: Create a custom playbook where you'll outline the rules for when and how you'll be notified.
Set Up the Deployment Trigger: In your custom playbook, add the
on_deployment_update
trigger. This ensures you'll receive notifications for deployment changes.Specify Fields to Monitor: Add change_filters to your
on_deployment_update
trigger to filter which changes you will be notified for.Route Notifications (Optional): Optionally, specify in your playbook where these notifications should be sent by defining 'sinks'.
Then perform a Helm Upgrade.
Note: You can also use the Sink Matchers to route notifications instead of explicitly specifying a sink in the playbook.
Testing:
Modify the image of a deployment in your cluster.
Run the following YAML files to simulate a deployment image change
kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/deployment_image_change/before_image_change.yaml
kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/deployment_image_change/after_image_change.yaml
A Robusta notification will arrive in your configured sinks, showing exactly what changed in the deployment.
Sample Alert:
Use Case 2: Notification on Ingress Rules ChangeΒΆ
Scenario: You want to be notified when an Ingress rules or tls details are changed.
Implementation:
Add the following YAML to the customPlaybooks
Helm value:
customPlaybooks:
- triggers:
- on_ingress_all_changes:
change_filters:
ignore:
- status
- metadata.generation
- metadata.resourceVersion
- metadata.managedFields
- spec.replicas
include:
- spec.rules
- spec.tls
actions:
- resource_babysitter: {}
sinks:
- some_sink_name # Optional
How does it work?
Initialize Custom Playbook: Create a custom playbook where you'll outline the rules for when and how you'll be notified.
Set Up the Ingress Trigger: In your custom playbook, add the
on_ingress_all_changes
trigger. This ensures you'll receive notifications for all ingress changes.Specify Fields to Monitor: Add change_filters to your
on_ingress_all_changes
trigger to filter which changes you will be notified for.Route Notifications (Optional): Optionally, specify in your playbook where these notifications should be sent by defining 'sinks'.
Then perform a Helm Upgrade.
Note: You can also use the Sink Matchers to route notifications instead of explicitly specifying a sink in the playbook.
Testing:
Create, modify, or delete an ingress in your cluster.
Run the following commands to simulate ingress changes:
kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/ingress_port_path_change/before_port_path_change.yaml
kubectl apply -f https://raw.githubusercontent.com/robusta-dev/kubernetes-demos/main/ingress_port_path_change/after_port_path_change.yaml
A Robusta notification will arrive in your configured sinks, showing exactly what changed in the ingress.
Sample Alert:
CleanupΒΆ
Remove the playbook you added based on your specific use case from the customPlaybooks
in your generated_values.yaml
file. Then, perform a Helm Upgrade.