Skip to main content

OpenTelemetry Integration

Run your agent yourself, send its traces to Genum, and turn real conversations into regression tests, with no Genum-specific code in your agent loop.

The Native API runs a prompt for you: Genum calls the model and records the run. When your application runs its own agent loop, with its own model calls and its own tools, use OpenTelemetry instead. Your agent exports traces that follow the OpenTelemetry GenAI semantic conventions, and Genum:

  • shows each conversation in the prompt's Logs as a session, turn by turn, including its tool calls;
  • lets you create a test case from a session and replay it against your prompt.

Ingested traces are free: they are not billed and not added to your Genum usage totals.

Endpoint​

POST https://api.genum.ai/api/public/otel/v1/traces
  • Protocol: OTLP over HTTP with JSON (http/json). Protobuf is not accepted.
  • Authentication: a project API key, sent as Authorization: Bearer YOUR_API_KEY. Keys are managed under Settings → Project → API Keys.
  • Self-hosted: replace https://api.genum.ai with the address of your own Genum API.

The prompt's API tab has an OpenTelemetry section with the endpoint and configuration, with your prompt ID already filled in.

Configure the Exporter​

Every OpenTelemetry SDK reads these environment variables:

OTEL_EXPORTER_OTLP_ENDPOINT=https://api.genum.ai/api/public/otel
OTEL_EXPORTER_OTLP_PROTOCOL=http/json
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer YOUR_API_KEY"
warning
  • Set OTEL_EXPORTER_OTLP_ENDPOINT without /v1/traces. The exporter appends that path itself, and .../v1/traces/v1/traces returns 404. If you use OTEL_EXPORTER_OTLP_TRACES_ENDPOINT instead, give the full URL including /v1/traces.
  • Set the protocol to http/json explicitly. Many SDKs send protobuf by default, and such a batch is refused.

Many GenAI instrumentations do not record message content by default. Turn content capture on in your instrumentation. For example, the OpenTelemetry Python GenAI instrumentations use OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true. Without it, Genum receives the turns but not what was said in them.

Set genum.prompt.id on every span:

span.setAttribute("genum.prompt.id", 123);

// Group a multi-turn conversation into one session:
span.setAttribute("gen_ai.conversation.id", conversationId);

genum.prompt.id is the only thing that ties a span to a prompt. It is never guessed from the API key or the service name, and a span without it is rejected. Set it on each span rather than once per request, because a collector can merge spans from several services into one batch.

Attributes Genum Reads​

AttributeSet onPurpose
genum.prompt.idevery spanRequired. The ID of the prompt the span belongs to.
gen_ai.operation.nameevery spanchat for a model call, execute_tool for a tool call.
gen_ai.conversation.idevery spanGroups turns into one session. Without it, every trace is a separate session of one turn.
gen_ai.input.messageschat spansThe messages sent to the model.
gen_ai.output.messageschat spansThe model's answer. Without it, the turn is stored with an empty answer.
gen_ai.tool.nameexecute_tool spansThe tool that was called.
gen_ai.tool.call.argumentsexecute_tool spansThe call's arguments, as JSON.
gen_ai.tool.call.resultexecute_tool spansWhat the tool returned.
gen_ai.request.model, gen_ai.systemchat spansModel and provider, shown in Logs.
gen_ai.usage.input_tokens, gen_ai.usage.output_tokenschat spansToken usage, shown per step.

For Faithful Replays​

A test case made from a session replays it against your prompt. To replay a turn under the same conditions your user had, also record what the turn ran with:

AttributeValueIf it is missing
genum.prompt.placeholdersA JSON object of placeholder key to value name, for example {"tone":"friendly"}. See Placeholders.The replay uses the prompt's default placeholder values.
gen_ai.tool.definitions or genum.tools.offeredThe tools offered to the model: the tool definitions as the GenAI conventions describe them, or genum.tools.offered as a list of tool names.The replay offers every tool the prompt defines.
genum.prompt.versionThe commitHash of the prompt version you used.Genum cannot tell which version of the prompt the turn was run with.

The simplest way to get these right is to fetch the instruction from the Render endpoint. It returns the exact text Genum would send the model, with placeholders filled in, together with the commitHash to record.

How Traces Become Sessions​

  • One trace is one turn. Spans within a trace are ordered by their start time.
  • gen_ai.conversation.id groups turns into a session.
  • The session opens with the first question found in the input messages. From the second turn on, the last user message in gen_ai.input.messages is shown as the user's reply that started that turn.
  • A tool call is a span of its own, with gen_ai.operation.name set to execute_tool. That is what makes it a step a test case can check.
  • Re-sending a batch is safe. A span received twice is shown once.
  • In the prompt's Logs, a session is listed as an OpenTelemetry trace.

Example​

A turn in which the agent calls a weather tool and then answers:

{
  "resourceSpans": [{
    "scopeSpans": [{
      "spans": [
        {
          "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
          "spanId": "00f067aa0ba902b7",
          "name": "execute_tool get_weather",
          "startTimeUnixNano": "1757500000000000000",
          "endTimeUnixNano": "1757500000300000000",
          "attributes": [
            { "key": "genum.prompt.id",            "value": { "intValue": "123" } },
            { "key": "gen_ai.conversation.id",     "value": { "stringValue": "conv-1" } },
            { "key": "gen_ai.operation.name",      "value": { "stringValue": "execute_tool" } },
            { "key": "gen_ai.tool.name",           "value": { "stringValue": "get_weather" } },
            { "key": "gen_ai.tool.call.arguments", "value": { "stringValue": "{\"city\":\"Kyiv\"}" } },
            { "key": "gen_ai.tool.call.result",    "value": { "stringValue": "{\"celsius\":21}" } }
          ]
        },
        {
          "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
          "spanId": "00f067aa0ba902b8",
          "name": "chat gpt-4o",
          "startTimeUnixNano": "1757500000400000000",
          "endTimeUnixNano": "1757500001200000000",
          "attributes": [
            { "key": "genum.prompt.id",           "value": { "intValue": "123" } },
            { "key": "gen_ai.conversation.id",    "value": { "stringValue": "conv-1" } },
            { "key": "gen_ai.operation.name",     "value": { "stringValue": "chat" } },
            { "key": "gen_ai.system",             "value": { "stringValue": "openai" } },
            { "key": "gen_ai.request.model",      "value": { "stringValue": "gpt-4o" } },
            { "key": "gen_ai.usage.input_tokens", "value": { "intValue": "120" } },
            { "key": "gen_ai.usage.output_tokens","value": { "intValue": "34" } },
            { "key": "gen_ai.input.messages",     "value": { "stringValue": "[{\"role\":\"user\",\"parts\":[{\"type\":\"text\",\"content\":\"What is the weather in Kyiv?\"}]}]" } },
            { "key": "gen_ai.output.messages",    "value": { "stringValue": "[{\"role\":\"assistant\",\"parts\":[{\"type\":\"text\",\"content\":\"It is 21°C and clear in Kyiv.\"}]}]" } }
          ]
        }
      ]
    }]
  }]
}

To send it without an SDK, for example to test your setup:

curl -X POST https://api.genum.ai/api/public/otel/v1/traces \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d @trace.json

Responses​

StatusBodyMeaning
200{ "partialSuccess": {} }Every span was stored.
200{ "partialSuccess": { "rejectedSpans": "1", "errorMessage": "..." } }Some spans could not be stored and errorMessage says why. The rest were stored, so do not resend the batch.
400An error naming the problemNo span in the batch could be stored, for example because none of them has a valid genum.prompt.id.
401Invalid API key, or a missing Authorization headerCheck the API key and the header.
503Could not store the spans. Retry the batch.A temporary failure on Genum's side. Retrying is safe and never creates duplicates.

OpenTelemetry exporters retry failed batches automatically.

From a Session to a Test Case​

  1. Open the prompt's Logs and find the session.
  2. Open it, click Add testcase and select the steps to check: the tool calls the model made, with their arguments, and its answers. If the turns recorded genum.prompt.placeholders and the offered tools, the test case keeps them.
  3. Run the test case. Genum replays the conversation against the prompt and reports every checked step as passed, failed, ignored or not run.

During a replay, tool results come from the recording, so Genum never calls your tools. Replays run against the prompt's current draft, which lets you check an edit before you commit it.

See more on testing

Self-Hosted Instances​

OpenTelemetry ingest needs no extra configuration on a self-hosted Genum. If a reverse proxy in front of the API only forwards selected paths, add /api/public/otel. Requests to it count toward the general rate limit.