Skip to content

[exporter/kafka] connection initialization failed to Azure event hub #42795

@rafaelrodrigues3092

Description

@rafaelrodrigues3092

Component(s)

exporter/kafka

What happened?

Description

After v0.135.0, the kafka exporter fails to connect to Azure Event Hub using Franz-Go.
I tested the functionality against Azure Event Hub Standard and Premium, both have the same behavior/error.
The same config works with Sarama when disabling the feature gate exporter.kafkaexporter.UseFranzGo with the 0.135.0 agent version.

When running from Windows, I see the message: wsarecv: An existing connection was forcibly closed by the remote host.
From the container image there's an additional message, a little more descriptive of the issue: re-updating metadata due to err: broker closed the connection immediately after a dial, which happens if the client is using TLS when the broker is not expecting it: is TLS misconfigured on the client or the broker?

I have tried few steps such as playing around with the values of tls as well as the protocol version, but I don't seem to be able to find a combination of settings that will make it work with Franz-Go.

A secondary concern is that these connection errors are being logged with the INFO log level. Do you think it would be more beneficial to log connection errors with WARN ?

Any insight is greatly appreciated.
Thank you in advance.

Steps to Reproduce

In a Windows Server machine, push any signal to Azure Event Hub via Kafka Exporter.

Expected Result

Successful connection

Actual Result

Error:

  • wsarecv: An existing connection was forcibly closed by the remote host.
  • re-updating metadata due to err: broker closed the connection immediately after a dial, which happens if the client is using TLS when the broker is not expecting it: is TLS misconfigured on the client or the broker?

Collector version

v0.135.0

Environment information

Environment

OS: Windows Server 2019/2022

OpenTelemetry Collector configuration

exporters:
  debug:
    verbosity: basic
  kafka:
    auth:
      sasl:
        mechanism: PLAIN
        password: Endpoint=sb://<EVH_NAMESPACE>.servicebus.windows.net/;SharedAccessKeyName=<KEY_NAME>;SharedAccessKey=<KEY>
        username: $$ConnectionString
    brokers: [<EVH_NAMESPACE>.servicebus.windows.net:9093]
    client_id: otel-collector
    logs:
      encoding: otlp_proto
      topic: vm-logs
    metadata:
      full: true
      retry:
        backoff: 250ms
        max: 3
    metrics:
      encoding: otlp_proto
      topic: vm-metrics
    partition_logs_by_resource_attributes: false
    partition_metrics_by_resource_attributes: false
    partition_traces_by_id: true
    producer:
      allow_auto_topic_creation: false
      compression: gzip
      compression_params:
        level: 2
      flush_max_messages: 0
      max_message_bytes: 1000000
      required_acks: 1
    protocol_version: 2.1.0
    resolve_canonical_bootstrap_servers_only: false
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_elapsed_time: 5m
      max_interval: 30s
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 100000
    timeout: 30s
    tls:
      insecure: false

receivers:
  hostmetrics:
    collection_interval: 10s
    scrapers:
      cpu:
        metrics:
          system.cpu.frequency:
            enabled: false
          system.cpu.logical.count:
            enabled: true
          system.cpu.physical.count:
            enabled: false
          system.cpu.time:
            enabled: true
          system.cpu.utilization:
            enabled: true
service:
  pipelines:
    metrics:
      exporters:
      - debug
      - kafka
      receivers:
      - hostmetrics
  telemetry:
    logs:
      level: debug

Log output

2025-09-19T12:32:46.314-0400    debug   builders/builders.go:24 Alpha component. May change in the future.      {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics"}
2025-09-19T12:32:46.314-0400    debug   builders/builders.go:24 Beta component. May change in the future.       {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics"}
2025-09-19T12:32:46.319-0400    debug   builders/builders.go:24 Beta component. May change in the future.       {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "hostmetrics", "otelcol.component.kind": "receiver", "otelcol.signal": "metrics"}
2025-09-19T12:32:46.319-0400    info    [email protected]/service.go:211 Starting otelcol-contrib...     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "Version": "0.135.0", "NumCPU": 2}
2025-09-19T12:32:46.319-0400    info    extensions/extensions.go:41     Starting extensions...  {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}}
2025-09-19T12:32:46.324-0400    info    [email protected]/service.go:234 Everything is ready. Begin running and processing data. {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}}
2025-09-19T12:32:47.326-0400    info    Metrics {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "resource metrics": 1, "metrics": 2, "data points": 9}
2025-09-19T12:32:47.327-0400    info    franz   [email protected]/kzap.go:114 immediate metadata update triggered     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "why": "forced load because we are producing to a topic for the first time"}
2025-09-19T12:32:47.328-0400    info    franz   [email protected]/kzap.go:114 producing to a new topic for the first time, fetching metadata to learn its partitions  {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "topic": "vm-metrics"}
2025-09-19T12:32:47.328-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:47.415-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:47.415-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:47.418-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0, "err": null}
2025-09-19T12:32:47.420-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.0005274, "time_to_read": 0.0005683, "err": "read tcp 10.26.82.69:57342->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:47.422-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57342->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:47.422-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:47.439-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:47.442-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:47.451-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0, "err": null}
2025-09-19T12:32:47.451-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.0005111, "time_to_read": 0, "err": "read tcp 10.26.82.69:57343->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:47.455-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57343->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:47.455-0400    debug   franz   [email protected]/kzap.go:110 retrying request        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "request": "Metadata", "tries": 1, "backoff": 0.222367253, "time_since_start": 0.1268127, "request_error": "read tcp 10.26.82.69:57343->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host.", "response_error": null}
2025-09-19T12:32:47.681-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:47.695-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:47.695-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:47.696-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0.0005062, "err": null}
2025-09-19T12:32:47.697-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.0001381, "time_to_read": 0.0005081, "err": "read tcp 10.26.82.69:57344->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:47.698-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57344->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:47.699-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:47.725-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:47.726-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:47.728-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0, "err": null}
2025-09-19T12:32:47.731-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.0005797, "time_to_read": 0.0005064, "err": "read tcp 10.26.82.69:57345->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:47.731-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57345->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:47.732-0400    debug   franz   [email protected]/kzap.go:110 retrying request        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "request": "Metadata", "tries": 2, "backoff": 0.423770826, "time_since_start": 0.4038521, "request_error": "read tcp 10.26.82.69:57345->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host.", "response_error": null}
2025-09-19T12:32:48.158-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:48.174-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:48.175-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:48.175-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0, "err": null}
2025-09-19T12:32:48.177-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.0007926, "time_to_read": 0.0005071, "err": "read tcp 10.26.82.69:57346->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:48.179-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57346->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:48.179-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:48.191-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:48.197-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:48.197-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0, "err": null}
2025-09-19T12:32:48.200-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.0016535, "time_to_read": 0.0005124, "err": "read tcp 10.26.82.69:57347->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:48.202-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57347->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:48.204-0400    debug   franz   [email protected]/kzap.go:110 retrying request        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "request": "Metadata", "tries": 3, "backoff": 0.94738365, "time_since_start": 0.8757673, "request_error": "read tcp 10.26.82.69:57347->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host.", "response_error": null}
2025-09-19T12:32:49.157-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:49.171-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:49.172-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:49.173-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0, "err": null}
2025-09-19T12:32:49.174-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.0005408, "time_to_read": 0.0005076, "err": "read tcp 10.26.82.69:57348->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:49.175-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57348->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:49.176-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:49.189-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:49.189-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:49.190-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0, "err": null}
2025-09-19T12:32:49.191-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.000523, "time_to_read": 0.0005059, "err": "read tcp 10.26.82.69:57349->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:49.192-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57349->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:49.194-0400    info    franz   [email protected]/kzap.go:114 new topic metadata wait failed, retrying wait  {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "topic": "vm-metrics", "err": "read tcp 10.26.82.69:57349->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:49.465-0400    info    franz   [email protected]/kzap.go:114 metadata update triggered       {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "why": "re-updating metadata due to err: read tcp 10.26.82.69:57349->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:54.196-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:54.216-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:54.216-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:54.216-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0, "err": null}
2025-09-19T12:32:54.217-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.0005234, "time_to_read": 0.0005068, "err": "read tcp 10.26.82.69:57355->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:54.219-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57355->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:54.220-0400    debug   franz   [email protected]/kzap.go:110 opening connection to broker    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:54.246-0400    debug   franz   [email protected]/kzap.go:110 connection opened to broker     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0"}
2025-09-19T12:32:54.247-0400    debug   franz   [email protected]/kzap.go:110 issuing api versions request    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "version": 4}
2025-09-19T12:32:54.248-0400    debug   franz   [email protected]/kzap.go:110 wrote ApiVersions v4    {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_written": 42, "write_wait": 0, "time_to_write": 0, "err": null}
2025-09-19T12:32:54.249-0400    debug   franz   [email protected]/kzap.go:110 read ApiVersions v4     {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "broker": "seed_0", "bytes_read": 0, "read_wait": 0.0005698, "time_to_read": 0.0005067, "err": "read tcp 10.26.82.69:57356->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:54.251-0400    debug   franz   [email protected]/kzap.go:110 connection initialization failed        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "addr": "<EVHNS_URL>.windows.net:9093", "broker": "seed_0", "err": "read tcp 10.26.82.69:57356->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:54.251-0400    debug   franz   [email protected]/kzap.go:110 retrying request        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "request": "Metadata", "tries": 1, "backoff": 0.285097554, "time_since_start": 0.0551232, "request_error": "read tcp 10.26.82.69:57356->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host.", "response_error": null}
2025-09-19T12:32:54.576-0400    debug   franz   [email protected]/kzap.go:110 retrying request        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "request": "Metadata", "tries": 2, "backoff": 0.451905812, "time_since_start": 0.3793211, "request_error": "read tcp 10.26.82.69:57358->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host.", "response_error": null}
2025-09-19T12:32:55.069-0400    debug   franz   [email protected]/kzap.go:110 retrying request        {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "request": "Metadata", "tries": 3, "backoff": 0.944401109, "time_since_start": 0.8728143, "request_error": "read tcp 10.26.82.69:57361->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host.", "response_error": null}
2025-09-19T12:32:56.051-0400    info    franz   [email protected]/kzap.go:114 new topic metadata wait failed, retrying wait  {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "topic": "vm-metrics", "err": "read tcp 10.26.82.69:57363->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}
2025-09-19T12:32:56.503-0400    info    franz   [email protected]/kzap.go:114 metadata update triggered       {"resource": {"service.instance.id": "ee10d3de-e909-468b-a018-95e9a3b17c8f", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "why": "re-updating metadata due to err: read tcp 10.26.82.69:57363->10.36.108.31:9093: wsarecv: An existing connection was forcibly closed by the remote host."}

Additional context

Disabling the feature gate exporter.kafkaexporter.UseFranzGo fixes the problem temporarily by reverting back to sarama.

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions