External Prometheus

Configure Robusta to use a Prometheus instance running outside your Kubernetes cluster, including centralized solutions like Thanos or Mimir.

Quick Start

Add the following to your generated_values.yaml:

globalConfig:
    prometheus_url: "https://prometheus.example.com:9090"
    alertmanager_url: "https://alertmanager.example.com:9093"

Then update Robusta.

Configuration Examples

Basic External Prometheus:

globalConfig:
    prometheus_url: "https://prometheus.company.com:9090"
    alertmanager_url: "https://alertmanager.company.com:9093"

Thanos Query:

globalConfig:
    prometheus_url: "https://thanos-query.monitoring.company.com:9090"
    alertmanager_url: "https://alertmanager.monitoring.company.com:9093"

Cortex/Mimir:

globalConfig:
    prometheus_url: "https://mimir.company.com/prometheus"
    alertmanager_url: "https://mimir.company.com/alertmanager"

Authentication

Bearer Token:

globalConfig:
    prometheus_url: "https://prometheus.example.com:9090"
    prometheus_auth: "Bearer YOUR_TOKEN_HERE"
    alertmanager_auth: "Bearer YOUR_TOKEN_HERE"

Basic Authentication:

globalConfig:
    prometheus_url: "https://prometheus.example.com:9090"
    # Base64 encode: echo -n "username:password" | base64
    prometheus_auth: "Basic dXNlcm5hbWU6cGFzc3dvcmQ="
    alertmanager_auth: "Basic dXNlcm5hbWU6cGFzc3dvcmQ="

Multi-cluster Setup

When using a centralized Prometheus for multiple clusters:

Option 1: Filter with labels

globalConfig:
    prometheus_url: "https://central-prometheus.company.com:9090"

    # Add cluster label to all queries
    prometheus_additional_labels:
        cluster: 'production-us-east'

Option 2: Filter with query string

globalConfig:
    prometheus_url: "https://central-prometheus.company.com:9090"

    # Append query parameters to all requests
    prometheus_url_query_string: "cluster=production-us-east&region=us-east-1"

Note

When using external Prometheus with multiple clusters, ensure all alerts contain a label named cluster_name or cluster, matching the cluster_name defined in Robusta's configuration. This is necessary to identify which robusta-runner should receive alerts.

SSL/TLS Configuration

Enable SSL verification (recommended for production):

globalConfig:
    prometheus_url: "https://prometheus.example.com:9090"

runner:
    additional_env_vars:
    - name: PROMETHEUS_SSL_ENABLED
      value: "true"

Custom CA Certificate:

globalConfig:
    prometheus_url: "https://prometheus.internal.company.com:9090"

runner:
    additional_env_vars:
    - name: PROMETHEUS_SSL_ENABLED
      value: "true"

    # Base64 encode your CA certificate
    certificate: |
        LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFVENDQWZtZ0F3SUJBZ0lVT...

Network Connectivity

Ensure Robusta can reach your external Prometheus:

  1. Allow egress traffic from Robusta's namespace to your Prometheus URL

  2. Configure firewall rules if Prometheus is behind a corporate firewall

  3. Set up VPN or private endpoints if needed

Test connectivity:

# From within the cluster
kubectl run test-curl --image=curlimages/curl --rm -it -- \
    curl -v https://prometheus.example.com:9090/-/healthy

Advanced Configuration

Custom Headers:

globalConfig:
    prometheus_additional_headers:
        X-Custom-Header: "custom-value"
        X-Scope-OrgID: "tenant-123"

Separate Prometheus Instances per Cluster:

If you prefer separate Prometheus URLs per cluster instead of filtering:

# In production cluster's values
globalConfig:
    prometheus_url: "https://prometheus-prod.company.com:9090"

# In staging cluster's values
globalConfig:
    prometheus_url: "https://prometheus-staging.company.com:9090"

Next Steps