OpenTelemetry system metrics Instrumentation

Instrument to report system (CPU, memory, network) and process (CPU, memory, garbage collection) metrics. By default, the following metrics are configured:

{
    "system.cpu.time": ["idle", "user", "system", "irq"],
    "system.cpu.utilization": ["idle", "user", "system", "irq"],
    "system.memory.usage": ["used", "free", "cached"],
    "system.memory.utilization": ["used", "free", "cached"],
    "system.swap.usage": ["used", "free"],
    "system.swap.utilization": ["used", "free"],
    "system.disk.io": ["read", "write"],
    "system.disk.operations": ["read", "write"],
    "system.disk.time": ["read", "write"],
    "system.network.dropped.packets": ["transmit", "receive"],
    "system.network.packets": ["transmit", "receive"],
    "system.network.errors": ["transmit", "receive"],
    "system.network.io": ["transmit", "receive"],
    "system.network.connections": ["family", "type"],
    "system.thread_count": None
    "process.context_switches": ["involuntary", "voluntary"],
    "process.cpu.time": ["user", "system"],
    "process.cpu.utilization": None,
    "process.memory.usage": None,
    "process.memory.virtual": None,
    "process.disk.io": ["read", "write"],
    "process.open_file_descriptor.count": None,
    "process.thread.count": None,
    "process.runtime.memory": ["rss", "vms"],
    "process.runtime.cpu.time": ["user", "system"],
    "process.runtime.gc_count": None,
    "cpython.gc.collections": None,
    "cpython.gc.collected_objects": None,
    "cpython.gc.uncollectable_objects": None,
    "process.runtime.thread_count": None,
    "process.runtime.cpu.utilization": None,
    "process.runtime.context_switches": ["involuntary", "voluntary"],
}

Usage

from opentelemetry.metrics import set_meter_provider
from opentelemetry.instrumentation.system_metrics import SystemMetricsInstrumentor
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import ConsoleMetricExporter, PeriodicExportingMetricReader

exporter = ConsoleMetricExporter()

set_meter_provider(MeterProvider([PeriodicExportingMetricReader(exporter)]))
SystemMetricsInstrumentor().instrument()

# metrics are collected asynchronously
input("...")

# to configure custom metrics
configuration = {
    "system.memory.usage": ["used", "free", "cached"],
    "system.cpu.time": ["idle", "user", "system", "irq"],
    "system.network.io": ["transmit", "receive"],
    "process.memory.usage": None,
    "process.memory.virtual": None,
    "process.cpu.time": ["user", "system"],
    "process.context_switches": ["involuntary", "voluntary"],
}
SystemMetricsInstrumentor(config=configuration).instrument()

Out-of-spec process.runtime prefixed metrics are deprecated and will be removed in future versions, users are encouraged to move to the process metrics.

API

class opentelemetry.instrumentation.system_metrics.SystemMetricsInstrumentor(labels=None, config=None)[source]

Bases: BaseInstrumentor

instrumentation_dependencies()[source]

Return a list of python packages with versions that the will be instrumented.

The format should be the same as used in requirements.txt or pyproject.toml.

For example, if an instrumentation instruments requests 1.x, this method should look like: :rtype: Collection[str]

def instrumentation_dependencies(self) -> Collection[str]:
return [‘requests ~= 1.0’]

This will ensure that the instrumentation will only be used when the specified library is present in the environment.

opentelemetry.instrumentation.system_metrics.environment_variables.OTEL_PYTHON_SYSTEM_METRICS_EXCLUDED_METRICS = 'OTEL_PYTHON_SYSTEM_METRICS_EXCLUDED_METRICS'

OTEL_PYTHON_SYSTEM_METRICS_EXCLUDED_METRICS

Specifies which system and process metrics should be excluded from collection when using the default configuration. The value should be provided as a comma separated list of glob patterns that match metric names to exclude.

Example Usage:

To exclude all CPU related metrics and specific process metrics:

export OTEL_PYTHON_SYSTEM_METRICS_EXCLUDED_METRICS="system.cpu.*,process.memory.*"

To exclude a specific metric:

export OTEL_PYTHON_SYSTEM_METRICS_EXCLUDED_METRICS="system.network.io"

Supported Glob Patterns:

The environment variable supports standard glob patterns for metric filtering:

* - Matches any sequence of characters within a metric name

Example Patterns:

system.* - Exclude all system metrics
process.cpu.* - Exclude all process CPU related metrics
*.utilization - Exclude all utilization metrics
system.memory.usage - Exclude the system memory usage metric