Skip to content

Cannot send custom statsd metrics in a plugin #69172

Description

@Jeoffreybauvin

Under which category would you file this issue?

Airflow Core

Apache Airflow version

3.2.1

What happened and how to reproduce it?

A while back, I was using a plugin to send custom statsd metrics after a fail or success dag run :

from airflow.stats import Stats
metric_success = f"brs_dag_last_5_runs_states.success.{dag_id}"
metric_failed = f"brs_dag_last_5_runs_states.failed.{dag_id}"
Stats.gauge(metric_success, success_count)
Stats.gauge(metric_failed, failed_count)

And it was working fine. But, it's not working anymore : no metric is sent to my statsd_exporter.

Here is my configuration :

      AIRFLOW__METRICS__STATSD_HOST: 10.2.218.29
      AIRFLOW__METRICS__STATSD_INFLUXDB_ENABLED: 'True'
      AIRFLOW__METRICS__STATSD_ON: 'True'
      AIRFLOW__METRICS__STATSD_PORT: '9125'
      AIRFLOW__METRICS__STATSD_PREFIX: prod

Please note : I don't have any issue with the "native" statsd metrics from Airflow : my stats_exporter is collecting them.

After some debug of the statsd object inside my plugin file :

  log.info("Stats object: %r", Stats)

  for attr in ("_instance", "_logger", "factory"):
      log.info("Stats.%s = %r", attr, getattr(Stats, attr, None))
  2026-06-30T09:03:49.079501Z [info     ] Stats object: <class 'airflow.sdk._shared.observability.metrics.stats.Stats'> [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:90
  2026-06-30T09:03:49.079666Z [info     ] Stats._instance = None         [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:92
  2026-06-30T09:03:49.079812Z [info     ] Stats._logger = None           [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:92
  2026-06-30T09:03:49.079946Z [info     ] Stats.factory = <class 'airflow.sdk._shared.observability.metrics.base_stats_logger.NoStatsLogger'> [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:92
  2026-06-30T09:03:49.080074Z [info     ] Stats.gauge returned: None     [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:96

What you think should happen instead?

The metrics inside my plugin should be sent to statsd.

Operating System

Ubuntu 24.04 + Docker

Deployment

Docker-Compose

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==9.6.1
apache-airflow-providers-apache-cassandra==3.9.4
apache-airflow-providers-apache-hdfs==4.11.5
apache-airflow-providers-apache-kafka==1.13.3
apache-airflow-providers-apache-spark==6.0.1
apache-airflow-providers-celery==3.18.0
apache-airflow-providers-cncf-kubernetes==10.16.1
apache-airflow-providers-common-compat==1.14.3
apache-airflow-providers-common-io==1.7.2
apache-airflow-providers-common-messaging==2.0.3
apache-airflow-providers-common-sql==1.35.0
apache-airflow-providers-docker==4.5.5
apache-airflow-providers-elasticsearch==6.5.3
apache-airflow-providers-fab==3.6.2
apache-airflow-providers-ftp==3.14.3
apache-airflow-providers-git==0.3.1
apache-airflow-providers-google==21.2.0
apache-airflow-providers-grpc==3.9.4
apache-airflow-providers-hashicorp==4.6.0
apache-airflow-providers-http==6.0.2
apache-airflow-providers-microsoft-azure==13.1.2
apache-airflow-providers-microsoft-mssql==4.5.2
apache-airflow-providers-mysql==6.5.2
apache-airflow-providers-odbc==4.12.2
apache-airflow-providers-openlineage==2.15.0
apache-airflow-providers-oracle==4.5.3
apache-airflow-providers-postgres==6.6.3
apache-airflow-providers-redis==4.4.4
apache-airflow-providers-sendgrid==4.2.3
apache-airflow-providers-sftp==5.7.4
apache-airflow-providers-slack==9.10.0
apache-airflow-providers-smtp==3.0.0
apache-airflow-providers-snowflake==6.12.2
apache-airflow-providers-ssh==5.0.1
apache-airflow-providers-standard==1.12.3

Official Helm Chart version

Not Applicable

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

No response

Anything else?

For the time being, I'm sending "manually" the metrics :

from functools import lru_cache

 from airflow.configuration import conf
 from statsd import StatsClient


 @lru_cache
 def get_statsd_client() -> StatsClient:
     return StatsClient(
         host=conf.get("metrics", "statsd_host"),
         port=conf.getint("metrics", "statsd_port"),
         prefix=conf.get("metrics", "statsd_prefix", fallback=None),
     )


 def statsd_gauge(metric_name: str, value: int) -> None:
     get_statsd_client().gauge(metric_name, value)

 statsd_gauge(metric_success, success_count)
 statsd_gauge(metric_failed, failed_count)

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions