Push vs pull metrics

This came up in your Civica interview. Know it cold.

• Pull model (Prometheus): The metrics server scrapes targets on a schedule. You configure Prometheus with a list of targets to scrape (scrape_configs). Each target exposes a /metrics endpoint. Prometheus pulls data from each one at the scrape interval (typically 15-60s).
• Push model (Datadog, StatsD, CloudWatch custom metrics): Each application or agent sends metrics to a central server. The server is passive — it receives what's pushed to it.

When to use which:

• Pull is good for service discovery (Prometheus can scrape anything that exposes /metrics), and for environments where the metrics server can reach the targets. Works well with Kubernetes.
• Push is good for short-lived workloads (batch jobs, Lambda functions) where there's nothing to scrape after the job finishes. Push is also better when targets can't be reached from the metrics server (e.g., behind NAT, in a different network).

The crisp answer: "Prometheus pulls — it scrapes targets that expose a /metrics endpoint. Datadog and similar push-based systems use an agent that sends metrics to the server. Pull works well in Kubernetes because it integrates with service discovery. Push works better for ephemeral workloads or when targets can't be reached from the server."