SigNoz Integration

SigNoz Integration

Overview

The SigNoz integration connects your SigNoz observability platform with Overwatch through alert channel webhooks. SigNoz sends alerts in a Prometheus-compatible format, and Overwatch enriches them with OpenTelemetry telemetry data including traces, error logs, service metrics, and dependency maps.

This deep enrichment is the distinguishing feature of the SigNoz integration. When an alert fires, Overwatch queries the SigNoz API to pull recent traces, error logs, service health metrics, and upstream/downstream dependencies for the affected service, giving the AI layer a comprehensive picture for root cause analysis.

Both SigNoz Cloud and self-hosted deployments are supported.

Prerequisites

• A SigNoz instance (Cloud or self-hosted) with alert channel configuration permissions
• An Overwatch organization with an active subscription
• The integration ID from your Overwatch Settings page
• A SigNoz API key for telemetry enrichment
• (Optional) A webhook secret for X-SigNoz-Signature validation

Setup Steps

1. In Overwatch, navigate to Settings > Integrations > New Integration and select SigNoz.
2. Enter your SigNoz base URL and API key to enable telemetry enrichment.
3. Copy the generated Integration ID and the full webhook URL:

   https://your-overwatch-instance/api/v1/webhooks/signoz/{integration_id}

4. In SigNoz, navigate to Settings > Alert Channels.
5. Click New Channel and configure:
   • Name: Overwatch
   • Type: Webhook
   • Webhook URL: Paste the webhook URL from step 3
6. Save the channel.
7. Assign the channel to one or more alert rules in Alerts > Alert Rules.
8. In Overwatch, click Test Connection to verify.

Note: For self-hosted SigNoz deployments, ensure the Overwatch instance can reach your SigNoz API endpoint for enrichment queries.

Alert Format

Overwatch extracts the following fields from SigNoz webhook payloads:

Field	Source	Description
alertname	labels.alertname	Alert rule name
service_name	labels.service_name	Affected OpenTelemetry service
fingerprint	alert.fingerprint	Unique alert identifier for deduplication
severity	labels.severity	Alert severity level
status	alert.status	Alert state (firing, resolved)
annotations	alert.annotations	Summary, description, value, threshold

Severity mapping: Overwatch maps SigNoz severity labels to incident priority levels:

SigNoz Label	Overwatch Priority
critical	critical
high	high
warning	medium
info	low

OpenTelemetry Enrichment

When a SigNoz API key is configured, Overwatch fetches the following telemetry data for the affected service:

Data Type	Details
Recent traces	Up to 50 traces from the past hour, with trace aggregates grouped by status code
Error logs	Up to 20 error-level logs from the past hour for the affected service
Service metrics	Error rate, P99 latency, throughput, and error count
Service dependencies	Upstream and downstream service dependencies from the service map
Related metrics	PromQL-compatible metric queries auto-generated based on the alert type (error rate, latency, or throughput)

Tip: The enrichment queries use a 1-hour lookback window from the time the alert is received. For intermittent issues, consider configuring longer alert evaluation windows in SigNoz.

Chrome Extension Detection

The Overwatch Chrome extension does not have automatic detection for SigNoz domains at this time. You can access incident context through the Overwatch dashboard or the extension’s search panel.

Troubleshooting

Webhook not receiving alerts

• Verify the alert channel is configured as type Webhook in SigNoz.
• Confirm the alert rule is assigned to the Overwatch channel.
• Check the SigNoz alert history for delivery status.

No enrichment data

• Ensure the SigNoz API key is entered in the Overwatch integration settings.
• Verify the API key has read access to traces, logs, and metrics.
• For self-hosted deployments, confirm the SigNoz API is reachable from the Overwatch instance.

Missing service_name

• Overwatch uses the service_name label to query telemetry data. If this label is not present on the alert, enrichment is skipped.
• Add service_name as a label in your SigNoz alert rule configuration.

Signature validation failures

• The X-SigNoz-Signature header is validated against the webhook secret configured in Overwatch. Ensure the secrets match.
• Signature validation uses HMAC-SHA256 over the sorted JSON payload.

Incomplete trace or log data

• Enrichment queries use a 1-hour lookback window. If the service has low traffic, there may be few or no traces/logs in that window.
• Verify that your OpenTelemetry instrumentation is sending data to SigNoz for the affected service.