Kubernetes CNI

Kubernetes CNI (Container Network Interface)

CNI is the standardized specification that enables Kubernetes to plug in different network implementations. It answers the question: “How does every Pod get a unique IP, and how does traffic flow between Pods on different nodes?” Source: CKA Day 32

The Problem CNI Solves

By default, Kubernetes does not provide a Pod network. After kubeadm init, nodes show NotReady until a CNI plugin is installed. The challenge:

• Every Pod needs a unique IP address across the entire cluster, not just the node.
• Pods on Node A must reach Pods on Node B directly — no NAT, no port mapping.
• The solution must work across cloud providers, bare metal, and virtual networks.

CNI decouples Kubernetes from any specific network implementation, letting operators choose the right plugin for their environment.

CNI Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   kubelet   │────▶│  CNI Plugin │────▶│   Pod Net   │
│  (runtime)  │     │  (binary)   │     │  (veth + IP)│
└─────────────┘     └─────────────┘     └─────────────┘
                           │
                           ▼
                    ┌─────────────┐
                    │  IPAM Driver│
                    │  (IP alloc) │
                    └─────────────┘

kubelet ↔ CNI Plugin Flow

1. Pod creation: kubelet calls the CNI plugin binary with an ADD command.
2. IP allocation: The plugin (or a dedicated IPAM plugin) assigns an IP from the cluster CIDR.
3. Interface creation: A veth (virtual Ethernet) pair is created — one end in the Pod’s network namespace, the other attached to the node’s bridge or tunnel.
4. Route programming: The plugin sets up routes so the Pod can reach other Pods, Services, and external destinations.
5. Pod deletion: kubelet calls DEL, releasing the IP and removing interfaces.

Cross-Node Traffic Models

Model	How It Works	Plugin Examples
Overlay (VXLAN/IPIP)	Encapsulates Pod-to-Pod packets inside node-to-node tunnels. Works on any L3 network without infrastructure changes.	Flannel (VXLAN), Calico (optional), Weave
BGP / Direct Routing	Each node advertises Pod routes to the physical network via BGP. No encapsulation overhead; requires routable Pod CIDRs.	Calico (default), Cilium (optional)
eBPF	Kernel-level packet processing bypasses iptables and the kernel network stack entirely. Highest performance and deepest observability.	Cilium
Cloud Native (ENI/VNet)	Assigns real VPC/VNet IPs directly to Pods. No overlay; Pod IPs are first-class network citizens.	AWS VPC CNI, Azure CNI

Popular CNI Plugins

Flannel

• Model: VXLAN overlay
• Policies: ❌ Not supported
• Best for: Learning, small clusters, environments without policy requirements
• Note: The default Kind CNI (kindnet) is based on Flannel concepts and also lacks policy support.

Calico

• Model: BGP by default; VXLAN overlay optional
• Policies: ✅ L3/L4 NetworkPolicies
• Best for: Production clusters, GKE default, policy-rich environments
• Deployment: calico-node DaemonSet + calico-kube-controllers Deployment
• Exam tip: Most common CNI for CKA labs and production installations.

Cilium

• Model: eBPF-based kernel packet processing
• Policies: ✅ L3/L4/L7 + DNS-aware rules + HTTP visibility
• Best for: Advanced security, observability (Hubble), service mesh replacement
• Note: Cilium can enforce policies at the application layer (e.g., “allow GET but not POST to /admin”).

Weave Net

• Status: ⚠️ Deprecated — last significant update ~2 years ago; unreliable on Kubernetes 1.30+

AWS VPC CNI

• Model: Allocates AWS Elastic Network Interfaces (ENIs) to Pods
• Policies: ✅ Yes (requires Network Policy Agent add-on on EKS)
• Best for: EKS workloads needing native VPC integration

Azure CNI

• Model: Assigns Azure VNet IPs to Pods
• Policies: ✅ Yes (Azure Network Policy or Calico)
• Best for: AKS with deep Azure networking integration

CNI and NetworkPolicies

Critical rule: NetworkPolicies are enforced by the CNI plugin, not by Kubernetes itself. If your CNI does not implement the policy controller, NetworkPolicy YAML objects will be accepted by the API server but will have zero effect.

CNI	Policy Enforcement
Flannel	❌ None
kindnet	❌ None
Weave Net	⚠️ Deprecated / unreliable
Calico	✅ Full L3/L4
Cilium	✅ Full L3/L4/L7 + DNS
AWS VPC CNI	✅ With Network Policy Agent
Azure CNI	✅ With Azure Network Policy

Source: Kubernetes Network Policies

CNI Configuration on Nodes

Standard Paths

Path	Purpose
/opt/cni/bin/	CNI plugin binaries (e.g., bridge, host-local, calico, flannel)
/etc/cni/net.d/	CNI configuration JSON files; kubelet reads the first .conf or .conflist
/etc/kubernetes/manifests/	Static Pod manifests for control plane (unrelated to CNI, but on the same node)

Example CNI Config (/etc/cni/net.d/10-calico.conflist)

{
  "name": "k8s-pod-network",
  "cniVersion": "0.3.1",
  "plugins": [
    {
      "type": "calico",
      "log_level": "info",
      "datastore_type": "kubernetes",
      "nodename": "node-1",
      "ipam": { "type": "calico-ipam" },
      "policy": { "type": "k8s" }
    },
    {
      "type": "portmap",
      "snat": true,
      "capabilities": { "portMappings": true }
    }
  ]
}

CNI Installation on kubeadm

# 1. Initialize control plane with matching Pod CIDR
sudo kubeadm init --pod-network-cidr=192.168.0.0/16
 
# 2. Install Calico
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/tigera-operator.yaml
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/custom-resources.yaml
 
# 3. Wait
kubectl get pods -n calico-system -w
kubectl get nodes

Pod CIDR Alignment Trap: Calico’s default pool is 192.168.0.0/16. Passing --pod-network-cidr=10.244.0.0/16 to kubeadm init causes a mismatch: Calico cannot reconcile and CoreDNS hangs in ContainerCreating. The fix is kubeadm reset and re-initialization. Source: CKA Day 27

Troubleshooting CNI

Symptom	Root Cause	Fix
Node stays NotReady	CNI not installed or CNI pods crashlooping	kubectl get pods -n kube-system or kubectl get pods -n calico-system; check CNI logs
CoreDNS ContainerCreating	CNI IP pool mismatch with --pod-network-cidr	kubeadm reset → re-run kubeadm init with correct CIDR
NetworkPolicy has no effect	CNI does not support policies	Migrate from Flannel/kindnet to Calico/Cilium
Pod has no IP (kubectl get pod -o wide)	CNI plugin failed to allocate	Check /var/log/containers and CNI plugin logs on the node
Cross-node Pod traffic blocked	Firewall / security group blocks VXLAN (UDP 4789) or BGP (TCP 179)	Open required ports in cloud security groups or on-prem firewalls

CNI vs. Service Networking

CNI solves Pod-to-Pod connectivity. Service-to-Pod load balancing is handled by kube-proxy (iptables or IPVS), which sits on top of CNI-provided Pod IPs:

Client Pod
    │
    ▼
Service ClusterIP (kube-proxy iptables rule)
    │
    ▼
Pod IP (provided by CNI)
    │
    ▼
veth → bridge / VXLAN / BGP → target node → target Pod

Related Concepts

• Kubernetes Network Policies — Policy enforcement depends on CNI capabilities
• Kubeadm Cluster Setup — CNI installation as the final cluster bootstrap step
• Kubernetes DaemonSet — CNI agents (calico-node, cilium-agent) deploy as DaemonSets
• Kubernetes Architecture — kubelet, container runtime, and CNI interaction
• Kubernetes Services — Service load balancing sits atop CNI-provided Pod IPs
• CoreDNS — Cluster DNS starts only after CNI is healthy
• Why Kubernetes? — Pluggable networking (CNI) as a core K8s design principle
• Networking Concepts Index — DNS, service discovery, and routing fundamentals

Source

• 40 — Kubernetes Networking Explained | Container Network Interface (CNI) With @kubesimplify

---

Tags: kubernetes cni networking calico flannel cilium pod-networking cka devops overlay bgp ebpf