Containers transformed how software is packaged, but running hundreds of container instances across a distributed cluster of bare-metal or cloud servers introduced immense operational complexity: scheduling, self-healing, horizontal autoscaling, rolling upgrades, and network ingress.

Kubernetes (K8s), originally engineered by Google based on their internal Borg system, is the undisputed operating system of the modern cloud. In this architectural guide, we dissect the inner machinery of Kubernetes from a software developer's perspective, exploring Control Plane components, Pod lifecycles, Services, and Ingress routing.

1. The Control Plane vs Worker Node Topology

A Kubernetes cluster is divided into two distinct functional layers:

The Control Plane (The Brain)

Worker Nodes (The Muscle)

2. Pods, Deployments, and ReplicaSets

In Kubernetes, you never deploy containers directly. The smallest deployable unit is a Pod: a wrapper containing one or more tightly coupled containers sharing the same network namespace (IP address) and storage volumes.

YAML (Production Kubernetes Deployment Spec) 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: devinsights-api-deployment
  labels:
    app: devinsights-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: devinsights-api
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1        # Allow 1 extra pod during rolling deploy
      maxUnavailable: 0  # Zero downtime guarantee!
  template:
    metadata:
      labels:
        app: devinsights-api
    spec:
      containers:
      - name: api-container
        image: devinsights/api:v2.4.0
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: "250m"      # 0.25 Core guaranteed
            memory: "256Mi"  # 256MB RAM guaranteed
          limits:
            cpu: "1000m"     # 1 Core maximum
            memory: "512Mi"  # Hard kill (OOMKilled) if exceeded
        readinessProbe:
          httpGet:
            path: /healthz/ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10
        livenessProbe:
          httpGet:
            path: /healthz/live
            port: 8080
          initialDelaySeconds: 15
          periodSeconds: 20

3. Service Discovery: ClusterIP, NodePort, and LoadBalancer

Pods are ephemeral: when a node crashes or autoscaling scales down, Pods are destroyed and recreated with completely new IP addresses. To provide a permanent, stable network address, Kubernetes uses Services:

YAML (Kubernetes ClusterIP Service Definition) 
apiVersion: v1
kind: Service
metadata:
  name: devinsights-api-service
spec:
  type: ClusterIP
  selector:
    app: devinsights-api # Matches Pod template labels!
  ports:
  - port: 80        # Port exposed by the Service
    targetPort: 8080 # Port listening inside the Container

4. Ingress: Layer 7 HTTP Routing and SSL Termination

Creating a separate cloud LoadBalancer for each microservice is prohibitively expensive. An Ingress Controller (e.g., NGINX Ingress, Traefik) acts as a unified Layer 7 reverse proxy, routing traffic based on hostnames and URI paths while terminating SSL certificates.

YAML (Kubernetes Ingress Rule with Path Routing) 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: devinsights-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - codingtutorials.site
    secretName: codingtutorials-tls-cert
  rules:
  - host: codingtutorials.site
    http:
      paths:
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: devinsights-api-service
            port:
              number: 80

Frequently Asked Questions (FAQ)

Q: What is the difference between Readiness and Liveness probes?

Liveness Probe: Checks if the container has crashed or deadlocked. If it fails, kubelet kills and restarts the container. Readiness Probe: Checks if the container is ready to accept incoming traffic (e.g., database connections established). If it fails, Kubernetes stops routing traffic to the Pod without restarting it.

Q: What happens when a container exceeds its resource memory limit?

If a container exceeds its CPU limit, Kubernetes throttles its CPU cycles (slowing execution). But if it exceeds its memory limit (limits.memory), the Linux kernel triggers an Out-Of-Memory kill (OOMKilled - Exit Code 137), terminating the pod immediately!

Conclusion

Kubernetes provides a declarative, resilient infrastructure platform for modern microservices. By defining explicit resource boundaries, configuring liveness and readiness health probes, and orchestrating ingress routing, developers ensure applications scale seamlessly across the global cloud.

💡 Engineering Key Takeaway

Always configure explicit CPU/memory requests and limits, and separate liveness from readiness probes to achieve seamless rolling deployments.

Practical Ingress Routing Manifest with Automated TLS

Configure automated HTTPS termination and host-based routing across multiple microservices:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: main-web-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - codingtutorials.site
    secretName: codingtutorials-tls-cert
  rules:
  - host: codingtutorials.site
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-frontend-service
            port:
              number: 80

ConfigMaps & Secrets: Decoupling Configuration from Code

Never hardcode API keys, database credentials, or environment flags into container images. Kubernetes provides two native resources for configuration management:

Horizontal Pod Autoscaling (HPA) Based on Metrics

Under heavy traffic spikes, Kubernetes automatically scales the number of running pod replicas based on observed CPU utilization or custom application metrics:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-autoscaler
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-service-deployment
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
SK

Written by Sajid Khan

Principal Software Engineer & Author

Sajid is a full-stack engineer and tech writer passionate about web performance, resilient backend architectures, and developer mentorship. He authors in-depth tutorials on modern JavaScript, React, and systems engineering.

Recommended Follow-Up Tutorials