Troubleshooting

The pipeline stage by stage — what each failure looks like from outside, and which of the four places to look answers which question.

Almost every “it isn’t working” resolves to one of four stages. Establishing which one you are in takes about a minute and eliminates most of the list.

1. The sender is not sending

   No requests are leaving your service or collector.

2. The request is being refused

   Requests leave and come back with a 4xx.

3. The request is accepted and the data is not where you are looking

   202, but nothing in the product.

4. The data is there and something downstream is not using it

   Records are queryable; a chart, alert or Kubernetes page is still empty.

1 — Nothing is being sent

Signature: no exporter errors, no ingest activity, everything looks healthy.

An OpenTelemetry SDK that cannot export usually says so once at startup and then stays quiet. Turn on its diagnostic logging before assuming the network:

export OTEL_LOG_LEVEL=debug

The three usual causes:

• The instrumentation never loaded. In Node.js, importing the register hook from inside your entry file instead of --requireing it means the modules it patches are already resolved. The app starts and emits nothing.
• OTEL_EXPORTER_OTLP_ENDPOINT is unset, so the SDK is exporting to its default local address and failing silently against nothing.
• The process exits before the batcher flushes. Short-lived jobs and tests need an explicit shutdown; without it the last batch is dropped, which is reliably the one you were watching for.

2 — Requests are being refused

Signature: exporter logs carry a status code.

Each code means one thing, documented in full at Endpoints and errors:

Code	Meaning	First thing to check
401	Key missing, malformed or revoked	That the header is X-License-Key and the value starts oiq_
403	Key valid, signal not enabled for the plan	Which signals your plan includes
404	Wrong path	Whether the endpoint already ends in /v1 — the SDK appends it
413	Batch over the size cap	Lower send_batch_size in the collector
429	Rate limited	Whether many services share one key through a NAT gateway

A 404 that looks like a bad endpoint is usually a doubled path. OTEL_EXPORTER_OTLP_ENDPOINT is a base URL — the SDK appends /v1/traces itself. Setting it to a URL that already ends in /v1/traces produces requests to /v1/traces/v1/traces. The error is a 404, and the natural reading of a 404 is that the endpoint is wrong, which sends people to change the one part that was correct.

Two more that produce a 401 for reasons that are not about the key:

• A revoked key still in a Secret. Revocation is immediate and not reversible; mint a new key rather than trying to restore one.
• Whitespace from a shell heredoc or a Kubernetes Secret created with --from-file. A trailing newline is part of the value. --from-literal does not add one.

3 — Accepted, but nothing shows up

Signature: 202 Accepted, empty screens.

Work through Verify your data arrived, which covers this case in full. The short list, in order of how often each turns out to be the answer:

1. The time range. Narrow to the last 15 minutes before concluding anything. A dashboard on “last 24 hours” reads rolled-up tables that fill on their own schedule.
2. A hardcoded timeUnixNano copied from an example. The record landed where that timestamp points — possibly outside retention, in which case it is accepted and never queryable.
3. No service.name. The record is stored and appears under no service.
4. The wrong workspace. A key belongs to one workspace; if you have several, check that the key and the dashboard you are looking at agree.

4 — The data is there but something is not using it

Kubernetes pages are empty while application telemetry flows

Those pages are built from k8s.* metrics, which come from the cluster collector, not from your services. Enable the clusterMetrics preset — see Send data from Kubernetes.

An alert never fires

Check in this order: the rule is enabled; its window is long enough that a sparse signal produces at least one datapoint in it; and a notification channel is attached and passes its own test send. A rule with no channel evaluates correctly and tells nobody.

Then check the comparator. Only GT, LT and EQ are implemented — see Alerting.

Synthetics show no results

Checks run from probe locations, and a fresh install has none registered until a probe is running. The checks are configured correctly and are not being executed by anything. See Synthetic monitoring.

Logs are searchable but a field filter matches nothing

Field names come from the attributes you sent, and OTLP attribute keys are case-sensitive. service.name and Service.Name are different fields. Searching logs has the full field list.

When the collector is in the path

The collector’s own metrics separate “my app reached the collector” from “the collector reached aiAxonIQ”:

curl -s localhost:8888/metrics | grep -E 'otelcol_(receiver_accepted|exporter_sent|exporter_send_failed)'

Accepted rising while sent stays flat means the collector is receiving and failing to export — its logs carry the status code, and the table above applies. Nothing rising at all means your application is not reaching the collector, and the collector configuration is not the problem to look at yet.

Still stuck

Have these ready, because they are the first things anyone will ask for:

• The base URL you are exporting to, exactly as configured.
• The prefix of your license key — the first 12 characters, oiq_…. Never the whole key.
• The status code and body of one failing request.
• The service name and a UTC timestamp of something you sent and cannot find.

Contact support

With the four items above, most cases resolve on the first reply.