OpenSearch logs¶

By enabling this toolset, HolmesGPT will fetch pod logs from OpenSearch.

You should enable this toolset to replace the default kubernetes/logs toolset if all your kubernetes pod logs are consolidated inside OpenSearch/Elastic. It will make it easier for HolmesGPT to fetch incident logs, including the ability to precisely consult past logs.

HolmesGPT provides several out-of-the-box alternatives for log access. You can select from these options:

kubernetes/logs: Access logs directly through Kubernetes. This is the default toolset.
coralogix/logs: Access logs through Coralogix.
grafana/loki: Access Loki logs by proxying through a Grafana instance.
opensearch/logs: Access logs through OpenSearch.

Configuration¶

Robusta Helm Chart Holmes CLI

holmes:
  toolsets:
    opensearch/logs:
      enabled: true
      config:
        opensearch_url: https://skdjasid.europe-west1.gcp.cloud.es.io:443 # The URL to your opensearch cluster.
        index_pattern: fluentd-* # The pattern matching the indexes containing the logs. Supports wildcards
        opensearch_auth_header: "ApiKey b0ZlwQWEsdwAkv047bafirkallDFWJIWDWdwlQQ==" # An optional header value set to the `Authorization` header for every request to opensearch.
        labels: # set the labels according to how values are mapped in your opensearch cluster
          pod: "kubernetes.pod_name"
          namespace: "kubernetes.namespace_name"
          timestamp: "@timestamp"
          message: "message"

    kubernetes/logs:
      enabled: false # HolmesGPT's default logging mechanism MUST be disabled

Update your Helm values (generated_values.yaml) with the above configuration and run a Helm upgrade:

helm upgrade robusta robusta/robusta --values=generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

Add the following to ~/.holmes/config.yaml, creating the file if it doesn't exist:

toolsets:
  opensearch/logs:
    enabled: true
    config:
      opensearch_url: <your opensearch/elastic URL>
      index_pattern: <name of the index to use> # The pattern matching the indexes containing the logs. Supports wildcards. For example `fluentd-*`
      opensearch_auth_header: "ApiKey <...>" # An optional header value set to the `Authorization` header for every request to opensearch
      labels: # set the labels according to how values are mapped in your opensearch cluster
        pod: "kubernetes.pod_name"
        namespace: "kubernetes.namespace_name"
        timestamp: "@timestamp"
        message: "message"

  kubernetes/logs:
    enabled: false # HolmesGPT's default logging mechanism MUST be disabled

Configuring index_pattern and labels¶

You can tweak the labels used by the toolset to identify kubernetes resources. This is optional and only needed if your logs settings differ from the defaults in the example below.

toolsets:
  opensearch/logs:
    enabled: true
    config:
      index_pattern: fluentd-*
      labels:
        pod: "kubernetes.pod_name"
        namespace: "kubernetes.namespace_name"
        timestamp: "@timestamp"
        message: "message"

Below is a screenshot of a query that was done using Elastic dev tools to find out what should be the values for the labels.

In the image above, the following values and labels are identified by a yellow rectangle:

Configuration field	Value	Description
index_pattern	fluentd-*	This defines what opensearch indexes should be used to fetch logs
pod	kubernetes.pod_name	The kubernetes pod name
namespace	kubernetes.namespace_name	The kubernetes namespace
timestamp	@timestamp	This timestamp is used to search logs by time range.
message	message	This is the content of the log message

Disabling the Default Logging Toolset¶

The default HolmesGPT logging tool must be disabled if you use a different datasource for logs. HolmesGPT may still use kubectl to fetch logs and never call your datasource if kubernetes/logs is not disabled. To disable the default logging toolset, add the following to your holmes configuration:

Robusta Helm Chart Holmes CLI

holmes:
  toolsets:
    kubernetes/logs:
      enabled: false

Update your Helm values (generated_values.yaml) with the above configuration and run a Helm upgrade:

helm upgrade robusta robusta/robusta --values=generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

Add the following to ~/.holmes/config.yaml, creating the file if it doesn't exist:

toolsets:
  kubernetes/logs:
    enabled: false

Capabilities¶

The table below describes the specific capabilities provided by this toolset. HolmesGPT can decide to invoke any of these capabilities when answering questions or investigating issues.

Tool Name	Description
fetch_pod_logs	Retrieve logs using opensearch