Monitor Kubernetes from Scratch

Estimated time: 5 minutes

Setup Kubernetes monitoring from scratch. Install Robusta, Prometheus, and Grafana on Kubernetes using Helm. This is the recommended way to monitor your cluster, with an all-in-one package.

Prerequisites

Generate a Config

Robusta needs settings to work. For example, if you use Slack then Robusta needs a Slack API key. These settings are configured as Helm values.

Choose a configuration method below:

Create the configuration by signing up for a free Robusta UI account ↗

Note

With the UI, you can easily configure integrations like Slack and Teams, while also unlocking exclusive features such as timeline of changes and alerts, built-in AI root cause analysis, and more.

Use the robusta cli tool to generate the Helm values.

Requirements and Troubleshooting 
pip install -U robusta-cli --no-cache
robusta gen-config --enable-prometheus-stack

Use the robusta cli tool to generate the Helm values. Run it via the pre-built docker container.

Requirements and Troubleshooting

A Docker daemon and bash are required.

On Windows, use bash inside WSL.

curl -fsSL -o robusta https://docs.robusta.dev/master/_static/robusta
chmod +x robusta
./robusta gen-config --enable-prometheus-stack

You should now have a generated_values.yaml file with a Robusta config. Save this file! You'll need it to install Robusta on new clusters.

This file contains sensitive values. Refer to Managing Secrets for tips on protecting them.

Install with Helm

Copy the below commands, replacing the <YOUR_CLUSTER_NAME> placeholder.

On some clusters this can take a while, so don't panic if it appears stuck:

helm repo add robusta https://robusta-charts.storage.googleapis.com && helm repo update
helm install robusta robusta/robusta -f ./generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

To use all Robusta features, ensure storage is enabled on your cluster. If necessary, refer to the EKS documentation and install the EBS CSI add-on

How do I know if my cluster has storage enabled?

Try installing Robusta. If storage is not configured, you'll receive an error:

PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition

Running kubectl get pvc -A will also show PersistentVolumeClaims in Pending state.

In this case, follow the instructions above and enable storage for your cluster.

helm repo add robusta https://robusta-charts.storage.googleapis.com && helm repo update
helm install robusta robusta/robusta -f ./generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

Due to Autopilot restrictions, some components are disabled for Robusta's bundled Prometheus. Don't worry, everything will still work.

helm repo add robusta https://robusta-charts.storage.googleapis.com && helm repo update
helm install robusta robusta/robusta -f ./generated_values.yaml \
    --set clusterName=<YOUR_CLUSTER_NAME> \
    --set kube-prometheus-stack.coreDns.enabled=false \
    --set kube-prometheus-stack.kubeControllerManager.enabled=false \
    --set kube-prometheus-stack.kubeDns.enabled=false \
    --set kube-prometheus-stack.kubeEtcd.enabled=false \
    --set kube-prometheus-stack.kubeProxy.enabled=false \
    --set kube-prometheus-stack.kubeScheduler.enabled=false \
    --set kube-prometheus-stack.nodeExporter.enabled=false \
    --set kube-prometheus-stack.prometheusOperator.kubeletService.enabled=false

First modify the Helm values to enable OpenShift support.

Then install Robusta as usual with Helm:

helm repo add robusta https://robusta-charts.storage.googleapis.com && helm repo update
helm install robusta robusta/robusta -f ./generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

Test clusters tend to have fewer resources. To lower Robusta's resource requests, set isSmallCluster=true.

helm repo add robusta https://robusta-charts.storage.googleapis.com && helm repo update
helm upgrade robusta robusta/robusta -f ./generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME> --set isSmallCluster=true \
    --set kube-prometheus-stack.prometheus.prometheusSpec.retentionSize=9GB \
    --set kube-prometheus-stack.prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.resources.requests.storage=10Gi \
    --set kube-prometheus-stack.prometheus.prometheusSpec.resources.requests.memory=512Mi

Verifying Installation

Confirm that two Robusta pods are running with no errors in the logs:

kubectl get pods -A | grep robusta
robusta logs

See Robusta in action

Deploy a crashing pod:

kubectl apply -f https://gist.githubusercontent.com/robusta-lab/283609047306dc1f05cf59806ade30b6/raw

Verify the pod is crashing:

$ kubectl get pods -A | grep crashpod
NAME                            READY   STATUS             RESTARTS   AGE
crashpod-64d8fbfd-s2dvn         0/1     CrashLoopBackOff   1          7s

Once the pod restarts twice, you'll get notified in your configured sink.

Example Slack Message

Now open the Robusta UI and look for the same message there.

Finally, clean up the crashing pod:

kubectl delete deployment crashpod

Next Steps

See how Robusta improves Prometheus.