kafka to prometheus metrics

[root-aza]

Hi!
Tell me why the exporter metrics are not displayed in the endpoint 0.0.0.0:9598/metrics?
Below I attach the configuration and the collected event.
Thanks!

Config:

[sources.client_verification_kafka]
type = "kafka"
group_id = "mdm_pos_client_verification_event_router"
topics = [ "mdm.client_verification" ]
bootstrap_servers = "${VECTOR_KAFKA_SERVER}"
sasl.enabled = true
sasl.mechanism = "PLAIN"
sasl.username = "${VECTOR_KAFKA_USERNAME}"
sasl.password = "${VECTOR_KAFKA_USER_PASSWORD}"
decoding.codec = "json"


[transforms.passport_verifyi_event]
type = "filter"
inputs = [ "client_verification_kafka" ]
condition = '.type == "PassportVerificationResultEvent"'

[transforms.passport_verifyi_builder_event]
type = "remap"
inputs = [ "passport_verifyi_event" ]
source = """
. = {
   "metric": {
        "counter": {
         "value": 1
       },
       "kind": "incremental",
       "name": "logins",
       "namespace": "passport",
       "timestamp": now(),
       "tags": {
         "status": .status,
       }
   }
}
"""

[sinks.debug_log_remap]
type = "console"
inputs = [ "passport_verifyi_builder_event" ]
encoding.codec = "json"


[sinks.prometheus_summary_passport_verifyi]
type = "prometheus_exporter"
inputs = [ "passport_verifyi_builder_event" ]
address = "0.0.0.0:9598"

Builded event:

{
  "metric": {
    "counter": {
      "value": 1
    },
    "kind": "incremental",
    "name": "logins",
    "namespace": "passport",
    "tags": {
      "status": "UNKNOWN"
    },
    "timestamp": "2025-01-02T16:09:09.464480881Z"
  }
}

[jszwedko]

Hi @root-aza ,

The issue here is that the passport_verifyi_builder_event transform will be emitting logs and not metrics. Rather than using remap which, when fed logs will emit logs, I think you want to use log_to_metric to emit metrics based on the incoming log events. Something like:

transforms:
  my_transform_id:
    type: log_to_metric
    inputs:
      - my-source-or-transform-id
    metrics:
      field: message
      type: counter
      name: logins
      namespace: passport
      tags:
        status: "{{status}}"

[root-aza]

@jszwedko Thanks you!