How do Network Policies work and how do you debug connectivity issues

Crisp answer: Network Policies are Kubernetes objects that define which pods can talk to which other pods (and external endpoints) using label selectors. By default all pod traffic is allowed. Once you apply a NetworkPolicy to a pod, only explicitly allowed traffic is permitted.

The default-deny pattern:

# Deny all ingress to pods in the production namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: production
spec:
  podSelector: {}   # matches ALL pods in the namespace
  policyTypes:
  - Ingress
  # no ingress rules = deny all ingress

After this, you selectively allow:

# Allow the frontend to reach the API on port 8080
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-api
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: api              # this policy applies to api pods
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend     # only allow traffic from frontend pods
    ports:
    - protocol: TCP
      port: 8080

CNI requirement:

NetworkPolicy objects are inert unless your CNI plugin enforces them. Cilium, Calico, and Weave enforce them. Flannel and the AWS VPC CNI alone do not. On EKS with VPC CNI, you need to add Calico or use Cilium as the CNI to get NetworkPolicy enforcement.

# Check if network policies are being enforced:
kubectl get pods -n kube-system | grep -i "cilium\|calico\|weave"

Debugging connectivity:

# Can pod A reach pod B?
kubectl exec -it <pod-a> -- curl http://<pod-b-ip>:8080
kubectl exec -it <pod-a> -- wget -q --spider http://<service-name>:8080

# Telnet/nc for raw TCP:
kubectl exec -it <pod-a> -- nc -zv <pod-b-ip> 8080

# Is DNS resolving?
kubectl exec -it <pod-a> -- nslookup api.production.svc.cluster.local

# Run a debug pod with networking tools:
kubectl run netdebug --image=nicolaka/netshoot --restart=Never -- sleep infinity
kubectl exec -it netdebug -- bash
# Inside: ping, curl, dig, ss, tcpdump, traceroute all available

Cilium-specific debugging (your homelab):

# Check if Cilium is allowing/denying traffic:
cilium monitor --type drop
# Shows dropped packets with reason

# Check NetworkPolicy enforcement for a pod:
cilium endpoint list
cilium policy get

# Connectivity test:
cilium connectivity test

Common NetworkPolicy mistakes:

# WRONG: means "from pods in namespaceA OR pods with label app=frontend"
ingress:
- from:
  - namespaceSelector:
      matchLabels:
        name: namespaceA
  - podSelector:
      matchLabels:
        app: frontend

# CORRECT: means "from pods with label app=frontend IN namespaceA"
ingress:
- from:
  - namespaceSelector:
      matchLabels:
        name: namespaceA
    podSelector:              # same list item = AND condition
      matchLabels:
        app: frontend

What to say in the interview:

"Network Policies define allowed traffic using label selectors. The key behaviour: pods with no NetworkPolicy applied have unrestricted traffic. Once a NetworkPolicy selects a pod, everything not explicitly allowed is denied. The policies are only enforced if the CNI supports it — Cilium and Calico do, the AWS VPC CNI alone doesn't. For debugging I run a netshoot pod as a network debug toolbox — it has curl, tcpdump, dig, and netcat. For Cilium specifically, cilium monitor --type drop shows dropped packets in real time with the reason, which makes it very easy to see exactly what NetworkPolicy is blocking what."