Grafana Alloy

Grafana Alloy is a vendor-neutral OpenTelemetry Collector distribution with built-in Prometheus pipelines. It is designed to collect, process, and forward metrics, logs, traces, and profiles.

🐳 Docker Collection Pattern

When running in a Docker environment, Alloy can automatically discover containers and stream their logs to Loki using the discovery.docker and loki.source.docker components.

Example config.alloy

// Discover containers via the Docker socket
discovery.docker "linux" {
  host = "unix:///var/run/docker.sock"
}
 
// Collect logs from discovered targets
loki.source.docker "default" {
  host       = "unix:///var/run/docker.sock"
  targets    = discovery.docker.linux.targets
  labels     = {"env" = "production", "platform" = "docker"}
  forward_to = [loki.write.local.receiver]
}
 
// Forward logs to a Loki endpoint
loki.write "local" {
  endpoint {
    url = "http://loki:3100/loki/api/v1/push"
  }
}

☸️ Kubernetes Production Patterns

Based on production research, the following topologies are recommended:

Topology	Best For	Rationale
StatefulSet	Prometheus Metrics	Provides stable Pod identifiers for consistent Write-Ahead Log (WAL) volume mapping.
DaemonSet	Node Metrics & Logs	Required for node_exporter hardware metrics and system logs (journald).
Sidecar	Specialized Apps	Minimizes “noisy neighbor” effects; ideal for short-lived or isolated telemetry.

📏 Sizing & Scaling Guidelines

• Memory Prediction: Allocate approximately 10KB of RAM per active series.
• Horizontal Scaling: Plan for horizontal scaling once you cross the 1 million active series threshold.
• Config Management: Use GitOps (Flux/ArgoCD) to manage ConfigMaps and prevent configuration drift across large clusters.

🛠️ Key Components

• discovery.docker: Connects to the Docker daemon to find running containers.
• loki.source.docker: Reads logs from the Docker daemon for specific targets.
• prometheus.exporter.unix: Collects host-level metrics (CPU, Memory, Disk).
• prometheus.exporter.cadvisor: Collects container-level resource metrics.

---

Source: Sourced via Tavily Research Skill (2026-04-22)