Add support for OpenTelemetry logging

[codeboten]

Currently there's an experimental specification for the logging signal:

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/data-model.md#log-and-event-record-definition.

The work for this has started in #1882

[srikanthccv]

Here is how I plan to send PRs

1. Add overall structures and classes (Add initial overall structure and classes for logs sdk #1894).
2. Add global provider (Add global log emitter provider #1895)[closed due base branch delete, will reopen].
3. Add OTLPHandler for logging module.
4. Add multilog processors and Simple, BatchLogProcessor.
5. Add Logs suport for OTLP Exporter.
6. Complete documentation & examples.

[adriangb]

Hi, thank you for working on this, this is very exciting!

I have 2 questions:

1. Would it make sense to have some sort of logging.Filterer to inject OpenTracing data into logs? The use case would be to also include this data in logs to the console or other destinations
2. For the Handlers, it looks like the Handler itself transmits the record. I thought LogRecord == Event (conceptually, not referring to any actual classes), so shouldn't OTLPHandler .emit just map to add_event in the current span? (sorry if this is a stupid question, I'm new to OT)

And I left a followup question in https://github.com/open-telemetry/opentelemetry-python/pull/1903/files#r662742003

[srikanthccv]

For the Handlers, it looks like the Handler itself transmits the record. I thought LogRecord == Event (conceptually, not referring to any actual classes), so shouldn't OTLPHandler .emit just map to add_event in the current span? (sorry if this is a stupid question, I'm new to OT)

No the LogRecord data model is different and also span events are limited in what they accept as attribute values.

Would it make sense to have some sort of logging.Filterer to inject OpenTracing data into logs? The use case would be to also include this data in logs to the console or other destinations

This https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-logging?. And to give you more context you can configure the processor to export to any place you want file or n/w dest or anything really. It is designed in such way that you can plug in your own exporter if there is a need.

[adriangb]

LogRecord data model is different and also span events are limited in what they accept as attribute values

Hmm, I guess I'm a bit confused now because the spec explicitly says that logs and events are the same thing. Is there any reading material you could point me to that might help me resolve my confusion? I want to minimize how much I hijack this thread with naïve questions.

I guess my main concern was having to configure multiple things / having multiple things writing OpenTelemetry data to the network. Would I be able to configure this handler to share a single batcher/exporter with manually emitted traces, or an instrumented framework?

This (link)

Yeah, I looked at that before. It seems like it's modifying the global log record factory. I would have made it a filter so that it could be applied selectively, but I guess the point is zero configuration, which that achieves.

[srikanthccv]

The intent is to reduce the notion of Event being seen as an entirely new concept OpenTelemetry introduced. The span Events are structured logs. It doesn't have severity, body, and doesn't allow any nested attributes. Please look at the logdata model https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/api.md#add-events and initial logging sdk spec for more info https://github.com/open-telemetry/oteps/blob/main/text/logs/0150-logging-library-sdk.md

[srikanthccv]

Initial set of tasks are completed. Anyone interested in trying this out can use pip editable install and use the collector exporter.