TT Lab
Get started
Learn Learning paths Courses

OTCA — OpenTelemetry Certified Associate

Every span arrived as unknown_service

Continue in TT Lab

Goal

Use the real SDK to see the order of precedence by which a resource is determined from defaults, environment variables, and code. Then follow, in code, how a resource is grouped in OTLP and the stable HTTP semantic conventions, and safely merge resources whose schemas differ.

Why it matters

Telemetry says "where did this come from" through the resource. If service.name arrives as unknown_service, the dashboard's service list collapses into a single entry, and if an environment variable and code give different values, you cannot find the cause until you know which one won. Semantic conventions are an agreement between backends and dashboards on what attribute names and statuses mean, so if you mark a 404 as an error or put the raw path in a span name, the error rate and the cardinality both go wrong.

Prepared environment

/opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py init places the starter files resource.py, grouping.py, server.py, and merge.py in /root/otca-resource/. /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py probe [env파일] (the placeholder is the env file) starts a new process with all OTEL_ environment variables cleared, puts in only the values from the env file, and calls Resource.create({}). /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py show N shows the experiment result for step N. It uses the OpenTelemetry Python SDK 1.44.0 (/opt/otel-lab) from the lab-dev image and uses no network. The grader reruns your files with the same SDK and compares the behavior with the values you wrote.

Steps

  1. Create the materials with /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py init, then run /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py probe. It shows the result of Resource.create({}) in a new process with all OTEL_ environment variables cleared. In /root/otca-resource/01-default.txt, write service_name=, sdk_language= (telemetry.sdk.language), and has_instance_id= (true/false for whether service.instance.id exists).
  2. In /root/otca-resource/02-otel.env, write two lines: OTEL_RESOURCE_ATTRIBUTES=service.name=cart,deployment.environment.name=staging and OTEL_SERVICE_NAME=checkout. Look at the result of /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py probe /root/otca-resource/02-otel.env and, in /root/otca-resource/02-env.txt, write service_name=, environment=, and winner= (the name of the environment variable that determined service.name).
  3. Fix make_resource() in /root/otca-resource/resource.py so that it uses Resource.create to set service.name to checkout-api and service.version to 2.4.1 and returns it. The grader calls this function in a process given OTEL_SERVICE_NAME=checkout and OTEL_RESOURCE_ATTRIBUTES=service.version=0.0.1,deployment.environment.name=prod. Write the result of /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py show 3 into /root/otca-resource/03-code.txt as service_name=, service_version=, and environment=.
  4. In /root/otca-resource/04-otel.env, write one OTEL_RESOURCE_ATTRIBUTES line. The value of team.owner is payments,risk, and include cloud.region=ap-northeast-2 as well. Percent-encode the comma (%2C). /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py show 4 shows the encoded value together with the same value when it is not encoded. In /root/otca-resource/04-encoding.txt, write team_owner= and unencoded_team_owner=.
  5. Fix emit(provider_a, provider_b) in /root/otca-resource/grouping.py so that it creates 2 spans with provider_a's tracer and 1 span with provider_b, then finishes. /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py show 5 shows the result of encoding the spans of the two providers (service.name frontend and orders-db) as OTLP. In /root/otca-resource/05-grouping.txt, write resource_spans=, spans=, and service_name_on_spans= (true/false for whether service.name is in the span attributes).
  6. Fix handle(tracer, request) in /root/otca-resource/server.py so that it creates the server span for a single request according to the stable HTTP semantic conventions. The grader checks with three requests, 201, 404, and 503 — kind SERVER; name {메서드} {경로 템플릿} (the placeholders are the method and the route template); attributes http.request.method, url.path, http.route, and http.response.status_code (an integer); none of the old names (http.method, http.target, http.status_code); for 5xx, status ERROR and error.type (the status code string); otherwise leave the status unset. You can see the spans you made with {RS} show 6.
  7. /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py show 7 shows the result of merge-ing the configured resource (schema 1.26.0, service.name checkout) into the detected resource (schema 1.21.0, service.name unknown_service). In /root/otca-resource/07-merge.txt, write naive_service_name=, and fix combine(detected, configured) in /root/otca-resource/merge.py so that it returns a new resource in which the configured values win, the detected host.name is kept, and the schema_url follows the configured resource. Do not modify the input resources.

Notes

The name of a service with nothing configured

Create the materials with /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py init, then run /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py probe. It shows the result of Resource.create({}) in a new process with all OTEL_ environment variables cleared. In /root/otca-resource/01-default.txt, write service_name=, sdk_language= (telemetry.sdk.language), and has_instance_id= (true/false for whether service.instance.id exists).

service.name is a required attribute, so the SDK fills in a default if it is empty. Some attributes are created by the SDK itself even if you do not set them.

When two environment variables name the same thing

In /root/otca-resource/02-otel.env, write two lines: OTEL_RESOURCE_ATTRIBUTES=service.name=cart,deployment.environment.name=staging and OTEL_SERVICE_NAME=checkout. Look at the result of /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py probe /root/otca-resource/02-otel.env and, in /root/otca-resource/02-env.txt, write service_name=, environment=, and winner= (the name of the environment variable that determined service.name).

Both variables create a resource, but a precedence is defined. Decide by looking at the service.name left in the result.

Values set in code versus environment variables

Fix make_resource() in /root/otca-resource/resource.py so that it uses Resource.create to set service.name to checkout-api and service.version to 2.4.1 and returns it. The grader calls this function in a process given OTEL_SERVICE_NAME=checkout and OTEL_RESOURCE_ATTRIBUTES=service.version=0.0.1,deployment.environment.name=prod. Write the result of /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py show 3 into /root/otca-resource/03-code.txt as service_name=, service_version=, and environment=.

Resource.create overwrites the resource built from detectors and environment variables with the attributes passed as arguments. Keys not in the arguments keep the environment variable values. The Resource(...) constructor does not read environment variables.

A value containing a comma

In /root/otca-resource/04-otel.env, write one OTEL_RESOURCE_ATTRIBUTES line. The value of team.owner is payments,risk, and include cloud.region=ap-northeast-2 as well. Percent-encode the comma (%2C). /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py show 4 shows the encoded value together with the same value when it is not encoded. In /root/otca-resource/04-encoding.txt, write team_owner= and unencoded_team_owner=.

In this variable, the comma is the attribute separator. Commas and equals signs inside a value must be encoded. Also read the SDK warning for the side that was not encoded.

Where the resource attaches in OTLP

Fix emit(provider_a, provider_b) in /root/otca-resource/grouping.py so that it creates 2 spans with provider_a's tracer and 1 span with provider_b, then finishes. /opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py show 5 shows the result of encoding the spans of the two providers (service.name frontend and orders-db) as OTLP. In /root/otca-resource/05-grouping.txt, write resource_spans=, spans=, and service_name_on_spans= (true/false for whether service.name is in the span attributes).

OTLP groups spans that share the same resource and writes the resource only once. No individual span carries a resource.

A 404 is not a server error

Fix handle(tracer, request) in /root/otca-resource/server.py so that it creates the server span for a single request according to the stable HTTP semantic conventions. The grader checks with three requests, 201, 404, and 503 — kind SERVER; name {메서드} {경로 템플릿} (the placeholders are the method and the route template); attributes http.request.method, url.path, http.route, and http.response.status_code (an integer); none of the old names (http.method, http.target, http.status_code); for 5xx, status ERROR and error.type (the status code string); otherwise leave the status unset. You can see the spans you made with {RS} show 6.

From the server's point of view, a 4xx is a problem on the request side, so it is not marked with an error status. If you use the raw path in the span name, the name differs for every request.

Merging failed silently because the schema URLs differ

/opt/otel-lab/bin/python /opt/fixtures/otca_resource_lab.py show 7 shows the result of merge-ing the configured resource (schema 1.26.0, service.name checkout) into the detected resource (schema 1.21.0, service.name unknown_service). In /root/otca-resource/07-merge.txt, write naive_service_name=, and fix combine(detected, configured) in /root/otca-resource/merge.py so that it returns a new resource in which the configured values win, the detected host.name is kept, and the schema_url follows the configured resource. Do not modify the input resources.

Merging two resources with different schema URLs is an error according to the specification, and this SDK records the error and then returns the original resource unchanged. Resources are immutable, so build a new resource to merge.