Monitoring ClickPipes

In addition to in-console monitoring, ClickPipes exposes metrics to a Prometheus-compatible endpoint for scraping. These metrics are published with other ClickHouse Cloud service metrics, and allow you to integrate ClickPipes monitoring with your existing observability stack (e.g., Grafana, Datadog).

Metric labels

ClickPipes metrics use the standard service-level labels (clickhouse_org, clickhouse_service, clickhouse_service_name) plus the following ClickPipes-specific labels:

Label	Description
`clickpipe_id`	The unique identifier of the ClickPipe
`clickpipe_name`	The name of the ClickPipe
`clickpipe_source`	The source type (e.g., `kafka`, `postgres`, `s3`)
`clickpipe_state`	The state of the ClickPipe (e.g., `Stopped`, `Provisioning`, `Running`)

Sample response

The following shows what ClickPipes metrics look like in a Prometheus scrape response:

# HELP ClickPipes_Info Always equal to 1. Label "clickpipe_state" contains the current state of the pipe: Stopped/Provisioning/Running/Paused/Failed
# TYPE ClickPipes_Info gauge
ClickPipes_Info{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent",clickpipe_state="Running"} 1

# HELP ClickPipes_FetchedEvents_Total Total number of records fetched from the source.
# TYPE ClickPipes_FetchedEvents_Total counter
ClickPipes_FetchedEvents_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 5535376

# HELP ClickPipes_SentEvents_Total Total number of records sent to ClickHouse
# TYPE ClickPipes_SentEvents_Total counter
ClickPipes_SentEvents_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 5534250

# HELP ClickPipes_FetchedBytes_Total Total uncompressed bytes fetched from the source.
# TYPE ClickPipes_FetchedBytes_Total counter
ClickPipes_FetchedBytes_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 873286202

# HELP ClickPipes_SentBytes_Total Total uncompressed bytes sent to ClickHouse.
# TYPE ClickPipes_SentBytes_Total counter
ClickPipes_SentBytes_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 477187967

# HELP ClickPipes_Errors_Total Total errors ingesting data.
# TYPE ClickPipes_Errors_Total counter
ClickPipes_Errors_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 0

# HELP ClickPipes_Latency Time in milliseconds between when the record was produced to when it was written to ClickHouse.
# TYPE ClickPipes_Latency gauge
ClickPipes_Latency{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 340

Available metrics

Data transfer

Metric	Type	Description
`ClickPipes_FetchedBytes_Total`	Counter	Total uncompressed bytes fetched from the source.
`ClickPipes_FetchedBytesCompressed_Total`	Counter	Total compressed bytes fetched from the source. If the source data is uncompressed, equivalent to `ClickPipes_FetchedBytes_Total`.
`ClickPipes_FetchedBytesInitialLoad_Total`	Counter	CDC ClickPipes Total uncompressed bytes fetched during the initial load phase.
`ClickPipes_FetchedBytesResync_Total`	Counter	CDC ClickPipes Total uncompressed bytes fetched during a resync operation.
`ClickPipes_SentBytes_Total`	Counter	Total uncompressed bytes sent to ClickHouse.
`ClickPipes_SentBytesCompressed_Total`	Counter	Total compressed bytes sent to ClickHouse.

Events and records

Metric	Type	Description
`ClickPipes_FetchedEvents_Total`	Counter	Total number of records fetched from the source.
`ClickPipes_SentEvents_Total`	Counter	Total number of records sent to ClickHouse.
`ClickPipes_FetchedEvents_PerTable`	Gauge	CDC ClickPipes Number of records fetched per destination table and event type. Uses additional `destination_table` and `event_type` labels.

Errors

Metric	Type	Description
`ClickPipes_Errors_Total`	Counter	Total errors encountered during ingestion.

Latency

Metric	Type	Description
`ClickPipes_Latency`	Gauge (avg)	Time in milliseconds between when a record was produced at the source and when it was written to ClickHouse.
`ClickPipes_SourceReplicationLatency_MiB`	Gauge (avg)	CDC ClickPipes Replication lag at the source in MiB. For Postgres ClickPipes, this reflects the replication slot lag.

Resource usage

Metric	Type	Description
`ClickPipes_Replica_CPUUsage`	Gauge (avg)	Average CPU usage of the ingestion replicas, in millicores.
`ClickPipes_Replica_CPULimit`	Gauge (last)	Configured CPU limit of the ingestion replicas, in millicores.
`ClickPipes_Replica_MemoryUsage`	Gauge (avg)	Average memory usage of the ingestion replicas, in bytes.
`ClickPipes_Replica_MemoryLimit`	Gauge (last)	Configured memory limit of the ingestion replicas, in bytes.

State and progress

Metric	Type	Description
`ClickPipes_Info`	Gauge	Always equal to `1`. Use the `clickpipe_state` label to filter or alert on pipe state.
`ClickPipes_LastFetchedBatchId`	Gauge	ID of the last batch fetched from the source.
`ClickPipes_LastSentBatchId`	Gauge	ID of the last batch sent to ClickHouse.

Prometheus endpoint — Endpoint reference, authentication, and setup for external tools (e.g., Grafana, Datadog)
Error reporting — Where record and system errors are stored

Metric labels​

Sample response​

Available metrics​

Data transfer​

Events and records​

Errors​

Latency​

Resource usage​

State and progress​

Related pages​