Elasticsearch Integration

Elasticsearch / ELK Stack Integration

Overview

The Elasticsearch integration connects your Elasticsearch Watcher alerting system with Overwatch through webhook actions. When a Watcher condition is met, Elasticsearch sends execution data to Overwatch, which parses the watch results, analyzes search output, and creates incidents with full query context.

Overwatch enriches Watcher alerts by fetching additional data from the Elasticsearch API, including the watch definition, recent execution history, and cluster health statistics. This gives the AI layer visibility into both the alert condition and the state of the underlying cluster.

Prerequisites

• An Elasticsearch cluster with Watcher enabled (requires a Gold, Platinum, or Enterprise license, or Elastic Cloud)
• An Overwatch organization with an active subscription
• The integration ID from your Overwatch Settings page
• (Optional) Elasticsearch API credentials for enrichment queries
• (Optional) A webhook secret for signature validation

Setup Steps

1. In Overwatch, navigate to Settings > Integrations > New Integration and select Elasticsearch.
2. Copy the generated Integration ID and the full webhook URL:

   https://your-overwatch-instance/api/v1/webhooks/elasticsearch/{integration_id}

3. (Optional) Enter your Elasticsearch URL and API credentials in the Overwatch integration settings to enable enrichment.
4. In Kibana, navigate to Stack Management > Watcher (or use the Watcher API directly).
5. Create or edit a watch and add a webhook action:

   {
     "actions": {
       "overwatch_alert": {
         "webhook": {
           "method": "POST",
           "url": "https://your-overwatch-instance/api/v1/webhooks/elasticsearch/{integration_id}",
           "headers": {
             "Content-Type": "application/json"
           },
           "body": "{{#toJson}}ctx{{/toJson}}"
         }
       }
     }
   }

6. Save the watch.
7. In Overwatch, click Test Connection to verify the integration.

Tip: Use {{#toJson}}ctx{{/toJson}} in the webhook body to send the full execution context, which gives Overwatch access to the watch ID, execution time, condition result, and input payload.

Alert Format

Overwatch extracts the following fields from Elasticsearch Watcher webhook payloads:

Field	Source	Description
watch_id	watch_id	Watcher identifier (required)
execution_time	execution_time	When the watch executed (required)
condition_met	result.condition.met	Whether the watch condition was satisfied
condition_type	result.condition.type	Condition type (compare, script, array_compare)
input_type	result.input.type	Input type (search, http, simple)
total_hits	result.input.payload.hits.total	Number of matching documents
aggregations	result.input.payload.aggregations	Aggregation results from the search
execution_duration_ms	result.execution_duration_ms	Watch execution duration

Search result analysis: When the input type is search, Overwatch analyzes the search payload to extract:

• Total hit count
• Top 5 matching documents
• Aggregation summaries (bucket counts, metric values)
• Shard execution details

Generated tags: Overwatch automatically generates tags from the watch execution:

• condition:{type} — the condition type used
• input:{type} — the input type used
• volume:high / volume:medium / volume:low — based on hit count (>1000 = high, >100 = medium)
• condition:met or condition:not_met

API enrichment: When Elasticsearch credentials are configured, Overwatch fetches:

• Watch definition: Full watch configuration including trigger, input, condition, and actions
• Execution history: Last 10 executions within the past 24 hours
• Cluster health: Node count, shard status, and cluster state

Chrome Extension Detection

The Overwatch Chrome extension does not have automatic detection for Elasticsearch/Kibana domains at this time. You can access incident context through the Overwatch dashboard or the extension’s search panel.

Troubleshooting

Webhook not receiving alerts

• Verify the webhook URL in the Watcher action is correct, including the integration ID.
• Confirm the watch is active and the condition is being evaluated. Check the Watcher execution history in Kibana.
• Ensure the Elasticsearch cluster can reach the Overwatch endpoint (check network and firewall rules).

Missing watch_id or execution_time

• These fields are required. If using a custom webhook body, ensure watch_id and execution_time are included at the top level of the JSON payload.
• The recommended approach is to use {{#toJson}}ctx{{/toJson}} as the body, which includes all required fields.

No enrichment data

• Confirm the Elasticsearch URL and credentials are entered in the Overwatch integration settings.
• Verify the API credentials have read access to the .watches and .watcher-history-* indices.
• For Elastic Cloud deployments, use an API key with the appropriate cluster privileges.

Incidents created for non-matching conditions

• By default, Overwatch creates incidents whenever a Watcher sends a webhook, regardless of whether the condition was met. To only create incidents when the condition is met, add a condition check in your Watcher action configuration.

Signature validation failures

• Ensure the webhook secret configured in Overwatch matches the value used to calculate the X-ES-Signature header. The signature uses HMAC-SHA256 over the JSON payload.