OpenTelemetry Logging Instrumentation

The OpenTelemetry logging instrumentation automatically instruments Python logging with a handler to convert Python log messages into OpenTelemetry logs and export them. You can disable this by setting OTEL_PYTHON_LOG_AUTO_INSTRUMENTATION to false.

Warning

This package provides a logging handler to replace the deprecated one in opentelemetry-sdk. Therefore if you have opentelemetry-instrumentation-logging installed, you don’t need to set the OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED environment variable to true. By default, this instrumentation does not add code namespace attributes as the SDK’s logger does, but adding them can be enabled by using the OTEL_PYTHON_LOG_CODE_ATTRIBUTES environment variable.

Enable trace context injection

The OpenTelemetry logging integration can also be configured to inject tracing context into log statements.

The integration registers a custom log record factory with the the standard library logging module that automatically inject tracing context into log record objects. Optionally, the integration can also call logging.basicConfig() to set a logging format with placeholders for span ID, trace ID and service name.

The following keys are injected into log record objects by the factory:

  • otelSpanID

  • otelTraceID

  • otelServiceName

  • otelTraceSampled

The integration uses the following logging format by default:

%(asctime)s %(levelname)s [%(name)s] [%(filename)s:%(lineno)d] [trace_id=%(otelTraceID)s span_id=%(otelSpanID)s resource.service.name=%(otelServiceName)s trace_sampled=%(otelTraceSampled)s] - %(message)s

The integration is opt-in and must be enabled explicitly by setting the environment variable OTEL_PYTHON_LOG_CORRELATION to true. Setting OTEL_PYTHON_LOG_CORRELATION to true calls logging.basicConfig() to set a logging format that actually makes use of the injected variables.

Environment variables

OTEL_PYTHON_LOG_AUTO_INSTRUMENTATION

Set this env var to false to skip installing the logging handler provided by this package.

The default value is true.

OTEL_PYTHON_CODE_ATTRIBUTES

Set this env var to true to add code attributes (code.file.path, code.function.name, code.line.number) to OpenTelemetry logs, referencing the Python source location that emitted each log message.

The default value is false.

OTEL_PYTHON_LOG_CORRELATION

This env var must be set to true in order to enable trace context injection into logs by calling logging.basicConfig() and setting a logging format that makes use of the injected tracing variables.

Alternatively, set_logging_format argument can be set to True when initializing the LoggingInstrumentor class to achieve the same effect.

LoggingInstrumentor(set_logging_format=True)

The default value is false.

OTEL_PYTHON_LOG_FORMAT

This env var can be used to instruct the instrumentation to use a custom logging format.

Alternatively, a custom logging format can be passed to the LoggingInstrumentor as the logging_format argument. For example:

LoggingInstrumentor(logging_format='%(msg)s [span_id=%(span_id)s]')

The default value is:

%(asctime)s %(levelname)s [%(name)s] [%(filename)s:%(lineno)d] [trace_id=%(otelTraceID)s span_id=%(otelSpanID)s resource.service.name=%(otelServiceName)s trace_sampled=%(otelTraceSampled)s] - %(message)s
OTEL_PYTHON_LOG_HANDLER_LEVEL

Set this env var to filter which log records are exported by OpenTelemetry``LoggingHandler`` instrumentation. Accepts case-insensitive level names: notset, debug, info, warning, error. Only records at or above this level will be exported. For example, setting this to warning means DEBUG and INFO logs are still handled by your normal logging setup, but they are not exported as OTel logs. Unrecognized values fall back to notset.

Alternatively, the level can be set via the log_handler_level argument:

LoggingInstrumentor(log_handler_level=logging.WARNING)

The default value is notset.

OTEL_PYTHON_LOG_LEVEL

This env var can be used to set a custom logging level.

Alternatively, log level can be passed to the LoggingInstrumentor during initialization. For example:

LoggingInstrumentor(log_level=logging.DEBUG)

The default value is info.

Options are:

  • info

  • error

  • debug

  • warning

Manually calling logging.basicConfig

logging.basicConfig() can be called to set a global logging level and format. Only the first ever call has any effect on the global logger. Any subsequent calls have no effect and do not override a previously configured global logger. This integration calls logging.basicConfig() for you when OTEL_PYTHON_LOG_CORRELATION is set to true. It uses the format and level specified by OTEL_PYTHON_LOG_FORMAT and OTEL_PYTHON_LOG_LEVEL environment variables respectively.

If you code or some other library/framework you are using calls logging.basicConfig before this integration is enabled, then this integration’s logging format will not be used and log statements will not contain tracing context. For this reason, you’ll need to make sure this integration is enabled as early as possible in the service lifecycle or your framework is configured to use a logging format with placeholders for tracing context. This can be achieved by adding the following placeholders to your logging format:

%(otelSpanID)s %(otelTraceID)s %(otelServiceName)s %(otelTraceSampled)s

API

from opentelemetry.instrumentation.logging import LoggingInstrumentor

LoggingInstrumentor().instrument(set_logging_format=True)

Note

If you do not set OTEL_PYTHON_LOG_CORRELATION to true but instead set the logging format manually or through your framework, you must ensure that this integration is enabled before you set the logging format. This is important because unless the integration is enabled, the tracing context variables are not injected into the log record objects. This means any attempted log statements made after setting the logging format and before enabling this integration will result in KeyError exceptions. Such exceptions are automatically swallowed by the logging module and do not result in crashes but you may still lose out on important log messages.

class opentelemetry.instrumentation.logging.LoggingInstrumentor(*args, **kwargs)[source]

Bases: BaseInstrumentor

An instrumentor for stdlib logging module.

This instrumentor optionally injects tracing context into logging records and sets the global logging format to the following:

%(asctime)s %(levelname)s [%(name)s] [%(filename)s:%(lineno)d] [trace_id=%(otelTraceID)s span_id=%(otelSpanID)s resource.service.name=%(otelServiceName)s trace_sampled=%(otelTraceSampled)s] - %(message)s

def log_hook(span: Span, record: LogRecord):
    if span and span.is_recording():
        record.custom_user_attribute_from_log_hook = "some-value"
        span_ctx = span.get_span_context()
        record.from_sampled_span = span_ctx.trace_flags.sampled
Parameters:

  • tracer_provider – Tracer provider instance that can be used to fetch a tracer.

  • set_logging_format – When set to True, it calls logging.basicConfig() and sets a logging format.

  • logging_format – Accepts a string and sets it as the logging format when set_logging_format is set to True.

  • log_level – Accepts one of the following values and sets the logging level to it. logging.INFO logging.DEBUG logging.WARN logging.ERROR logging.FATAL

  • log_hook – execute custom logic when record is created

See BaseInstrumentor

instrumentation_dependencies()[source]

Return a list of python packages with versions that the will be instrumented.

The format should be the same as used in requirements.txt or pyproject.toml.

For example, if an instrumentation instruments requests 1.x, this method should look like: :rtype: Collection[str]

def instrumentation_dependencies(self) -> Collection[str]:

return [‘requests ~= 1.0’]

This will ensure that the instrumentation will only be used when the specified library is present in the environment.