Rakesh's Brain

Day 32/40 - Kubernetes Networking Explained | Container Network Interface (CNI) With @kubesimplify

4 min read

Day 32/40 — Kubernetes Networking Explained | Container Network Interface (CNI)

Overview

Video 32 of the Certified Kubernetes Administrator (CKA) 2024 series. A collaboration with Saiyam Pathak (@kubesimplify) explaining how Kubernetes networking works under the hood — from Pod-to-Pod communication to the role of the Container Network Interface (CNI) and popular plugins.

Why Kubernetes Networking Is Different

Docker uses a single-host bridge network by default. Kubernetes must solve multi-node networking: every Pod gets a unique IP across the entire cluster, and any Pod must be able to reach any other Pod directly — regardless of which node they run on.

The CNI Specification

CNI (Container Network Interface) is a standardized specification (not a specific tool) that defines how container runtimes — like containerd or CRI-O — should configure network interfaces for Pods. Key responsibilities:

  • IP Address Management (IPAM): Allocate unique Pod IPs within the cluster CIDR
  • Network Interface Creation: Create virtual interfaces (veth pairs) connecting the Pod namespace to the node
  • Routing / Overlay Setup: Ensure cross-node Pod-to-Pod traffic can flow (via overlay tunnels, BGP, or direct routing)

How CNI Works in Practice

  1. kubelet calls the CNI plugin (a binary in /opt/cni/bin) every time a Pod is created.
  2. The CNI plugin reads its configuration from /etc/cni/net.d.
  3. It allocates an IP address, creates the veth pair, and sets up routes.
  4. For cross-node traffic, CNI plugins use one of:
    • Overlay networks (VXLAN, IPIP) — encapsulate packets for tunneling across nodes
    • BGP / direct routing — advertise Pod routes to the underlying network (Calico)
    • eBPF — kernel-level packet forwarding (Cilium)
PluginModelPolicy SupportBest For
FlannelVXLAN overlay❌ NoSimple clusters, learning, no policy needs
CalicoBGP + optional VXLAN✅ Yes (L3/L4)Production, default on GKE, policy-rich environments
CiliumeBPF-based✅ Yes (L3/L4/L7 + DNS-aware)Advanced security, observability, service mesh
Weave NetOverlay⚠️ DeprecatedLegacy only; unreliable on K8s 1.30+
AWS VPC CNIAWS native ENIs✅ Yes (with add-on)EKS workloads needing native VPC IPs
Azure CNIAzure VNet✅ YesAKS with Azure Network Policy

CNI and Network Policies

NetworkPolicies are not enforced by Kubernetes itself — they are enforced by the CNI plugin. If your CNI does not support policies (e.g., Flannel, kindnet), NetworkPolicy objects exist but have zero effect. Calico and Cilium are the go-to choices for policy-enabled clusters.

CNI on kubeadm Clusters

After kubeadm init, the cluster has no functional Pod network until a CNI is installed. Steps:

  1. Run kubeadm init with --pod-network-cidr matching the CNI’s default pool.
  2. Install the CNI (e.g., Calico manifests or Helm chart).
  3. Wait for CNI Pods to become Ready.
  4. Join worker nodes.

Trap: If --pod-network-cidr mismatches the CNI’s default IP pool, nodes stay NotReady and CoreDNS hangs in ContainerCreating.

Key Takeaways

  1. CNI is a specification, not a product — plugins implement it.
  2. kubelet calls the CNI plugin for every Pod creation/deletion.
  3. CNI plugins handle IP allocation, veth creation, and cross-node routing.
  4. Flannel is simple but lacks policies; Calico is the production default; Cilium is the future with eBPF.
  5. NetworkPolicies require a policy-capable CNI — always verify plugin support before relying on them for security.

See Also

Wiki Concepts

Creator / Entity

Graph View

Latest Blog Posts

  • Loading posts...
Read more on Medium →

Backlinks