Kubernetes — A Complete Guide
Kubernetes is the de facto standard for deploying containerized applications on the cloud. It acts as a robust orchestrator for your containers, managing tasks like container resurrection, load balancing, and much more. For foundational knowledge, refer to my previous blogs titled, Decoding DevOps, for better understanding.
Understanding Kubernetes Architecture
For an in-depth understanding of the Kubernetes architecture, explore the official documentation: Kubernetes Architecture.
One of the key functionalities of Kubernetes is its ability to provide stable endpoints for accessing pods, even though pods themselves are ephemeral. Kubernetes services bridge this gap effectively.
Enable kubernetes in Docker Desktop, give it some time to come alive.
To check which cluster kubectl
is interacting with, use:
kubectl config current-context
Why Pods?
Pods are a fundamental Kubernetes object that allow you to harness the full potential of Kubernetes. A pod encapsulates one or more containers and provides a unified environment for them.
Sample Pod Definition
apiVersion: v1
kind: Pod
metadata:
name: geeky
labels:
app.kubernetes.io/name: geeky # Standard, the broadest label
app.kubernetes.io/component: frontend # Specifies the role
app.kubernetes.io/instance: frontend # Granular level control
spec:
containers:
- name: geeky
image: <Image>
env:
- name: SERVICE_HOST
value: expected-url # Enables frontend-backend communication via cluster IP service
resources:
requests: # Minimum resource allocation
memory: "128Mi" # Memory in Mebibytes
cpu: "200m" # CPU in millicores
limits: # Resource caps
memory: "128Mi"
ports:
- containerPort: 3000
Key kubectl
Commands for Pods
- Apply pod configuration:
kubectl apply -f pod_config_name.yaml
- List pods:
kubectl get pods
- Describe pod details:
kubectl describe pod pod_name
- View logs:
kubectl logs pod_name
- Stream logs:
kubectl logs -f pod_name
- Port-forwarding for local access:
kubectl port-forward pod_name 8080:3000
# Accessible at: localhost:8080
- Delete pods by label or name:
kubectl delete pod -l "app.kubernetes.io/name=geeky" kubectl delete pod geeky
- Delete all pods:
kubectl delete pods --all
Network Isolation and Communication
When you have multiple pods, Kubernetes ensures network-level isolation using Network Namespaces, which logically separate resources. To enable communication between pods, Kubernetes leverages services. This allows seamless interaction while maintaining strict isolation where needed.
Services: Durable Endpoints
Kubernetes services provide durable endpoints for applications. They address the challenge posed by ephemeral pods by offering stable virtual IP addresses. There are two primary types of services: NodePort and ClusterIP.
NodePort Services
A NodePort service exposes the application on a specific port of each node’s IP address in the cluster. While useful for prototyping, it’s not ideal for production as exposing machine ports can be a security risk. NodePort ranges from 30000 to 32767.
Sample NodePort Service Definition:
apiVersion: v1
kind: Service
metadata:
name: geeky
spec:
type: NodePort
selector:
app.kubernetes.io/instance: frontend
ports:
- port: 3000
targetPort: 3000
nodePort: 32000
- Apply service configuration:
kubectl apply -f service-definition.yaml
- This exposes the application locally on
localhost:32000
, forwarding requests to the pod labeledfrontend
.
ClusterIP Services
A ClusterIP service provides a stable internal IP address and port for applications within the cluster. This is ideal for internal communication between pods.
Sample ClusterIP Service Definition:
apiVersion: v1
kind: Service
metadata:
name: geeky
spec:
selector:
app.kubernetes.io/component: frontend
ports:
- port: 3000
targetPort: 3000
- This service forwards all traffic to any pod matching the label
frontend
. When traffic reaches the pod, it ensures the target port within the pod is accessible and responsive. - List services:
kubectl get services
Namespaces: Organizing Resources
Namespaces in Kubernetes allow you to logically separate and group resources, making it easier to manage and apply Role-Based Access Control (RBAC).
Key kubectl
Commands for Namespaces
- List namespaces:
kubectl get namespaces
- List pods in a namespace:
kubectl get pods -n default
- Delete all pods in a namespace:
kubectl delete pods --all -n default
- Create a namespace:
kubectl create namespace geeky-app
- Apply resources to a namespace:
kubectl apply -f . -n geeky-app
- List resources in a namespace:
kubectl get pods,services -n geeky-app
Defining Namespace in Metadata
Defining a namespace directly in the metadata section of your resource configurations ensures the resource is created in the specified namespace. This is often more convenient and prevents manual namespace specification during kubectl
commands.
Kubernetes Resilience: Behind the Scenes
Kubernetes ensures resilience and self-healing through its controllers:
- restartPolicy: By default,
restartPolicy
is set toAlways
, which ensures pods are restarted automatically if they fail. - Out of Memory (OOM): When a pod exceeds its memory limits, it is marked as
OOMKilled
. Kubernetes restarts such pods to maintain resilience.
Kubernetes Controllers
- Desired State: Controllers compare the current state of a pod with its desired state, taking actions to reconcile any differences.
- Continuous Monitoring: Kubernetes continuously monitors the health of containers and takes corrective actions when needed.
Deployments and Pod Replicas
Deployments in Kubernetes provide a declarative way to manage application updates and scalability. When you create a Deployment, it automatically creates a ReplicaSet, which ensures a specific number of pod replicas are running at all times.
Sample Deployment Definition
apiVersion: apps/v1
kind: Deployment
metadata:
name: geeky
namespace: geeky-ns
spec:
replicas: 3
selector:
matchLabels:
app.kubernetes.io/instance: geeky-api
template:
metadata:
labels:
app.kubernetes.io/name: geeky
app.kubernetes.io/component: backend
app.kubernetes.io/instance: backend
spec:
containers:
- name: geeky
image: <Image>
resources:
requests:
memory: "128Mi"
cpu: "200m"
limits:
memory: "128Mi"
ports:
- containerPort: 3000
Key Points About Deployments
- Manage your pods through Deployments, not standalone Pod primitives.
- List deployments in a namespace:
kubectl get deployments -n geeky-ns
- Deployments provide load balancing out of the box, using a round-robin approach.
Behind the Scenes
- The Deployment Controller creates a ReplicaSet based on the Deployment specification.
- The ReplicaSet ensures the desired number of pod replicas are running.
- The ReplicaSet Controller monitors the state of pods and recreates them if necessary.
- The Deployment Controller manages the overall lifecycle of the Deployment, including updates and rollbacks.
By declaring the desired state in the Deployment object, Kubernetes handles the rest, ensuring your application remains running and scalable as specified.
Rolling Updates and Rollbacks
Kubernetes Deployments enable you to update your applications seamlessly with minimum downtime, using a rolling update strategy. The Deployment Controller gradually replaces old pods with new ones, ensuring continuous availability during the update process.
Rolling Update Strategy
You can control the update process using the strategy
field in the Deployment specification. For example:
spec:
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1 # Maximum number of pods that can be unavailable during the update
maxSurge: 1 # Maximum number of extra pods to be created during the update
Rolling Back Changes
If a deployment update causes issues, you can easily roll back to the previous stable state.
- Rollback to a previous revision:
kubectl rollout undo deployment/<name> -n <namespace>
Probes: Ensuring Application Health
Kubernetes provides Liveness and Readiness Probes to monitor container health and readiness. These probes ensure your application remains resilient and responsive.
Liveness Probe
The Liveness Probe checks if a container is operational. If the probe fails, Kubernetes restarts the container.
Readiness Probe
The Readiness Probe determines if a container is ready to handle traffic. If the probe fails, the container is excluded from service endpoints.
Example Probe Configuration
livenessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 25
periodSeconds: 5
readinessProbe:
httpGet:
path: /ready
port: 8080
periodSeconds: 5
- Key Takeaways:
- Use Liveness Probes to detect and restart unhealthy containers.
- Use Readiness Probes to determine when a container is ready to start accepting traffic.
- Together, these probes help maintain application health and availability.
Storage Orchestration in Kubernetes
Kubernetes simplifies storage management for stateful applications using Persistent Volumes (PVs) and Persistent Volume Claims (PVCs).
StatefulSet with Storage Example
StatefulSets are ideal for managing stateful applications, as they provide stable identifiers and storage for each pod.
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: database
namespace: database-ns
spec:
serviceName: db
replicas: 2
selector:
matchLabels:
app.kubernetes.io/instance: db
template:
metadata:
labels:
app.kubernetes.io/name: geeky
app.kubernetes.io/component: db
app.kubernetes.io/instance: db
spec:
containers:
- name: database
image: registry.k8s.io/nginx-slim:0.8
volumeMounts:
- name: db-persistent-storage
mountPath: /data/db
volumeClaimTemplates:
- metadata:
name: db-persistent-storage
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
Key Components
- Persistent Volume (PV): Represents the actual storage resource.
- Persistent Volume Claim (PVC): A request for storage by a pod.
- Volume Claim Templates: Automatically generate PVCs for each StatefulSet pod.
Benefits of Storage Orchestration
- Automatic provisioning and attachment of storage volumes.
- Seamless data persistence across pod restarts or rescheduling.
- Built-in mechanisms for maintaining pod identity and state.
- List PVCs in a namespace:
kubectl get pvc -n database-ns
Configuration Management
ConfigMaps and Secrets
ConfigMaps: Store non-confidential plaintext data.
Sample ConfigMap Definition:
apiVersion: v1
kind: ConfigMap
metadata:
name: geeky-config
namespace: geeky-ns
data:
player_initial_lives: "3"
ui_properties_file_name: "user-interface.properties"
Secrets: Store sensitive data securely, encoded in Base64.
Sample Secret Definition:
apiVersion: v1
kind: Secret
metadata:
name: geeky-secret
namespace: geeky-ns
type: Opaque
data:
username: YWRtaW4= # base64 encoded 'admin'
password: dDBwLVMzY3IzdA== # base64 encoded 't0p-S3cr3t'
Horizontal Pod Autoscaling
Kubernetes supports automatic scaling of pods based on observed metrics such as CPU or memory usage.
Key Concepts
- Metrics Server: Kubernetes collects resource usage metrics via the metrics-server.
kubectl top pods -n geeky-ns
- HorizontalPodAutoscaler Definition:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: geeky
namespace: geeky-ns
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: geeky
minReplicas: 1
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 50
- Get HPA status:
kubectl get hpa -n geeky-ns
Points to remember
- The Horizontal Pod Autoscaler automatically scales the number of pods in a deployment based on observed CPU utilization (most commonly).
- Autoscaling Behavior: The HPA will increase or decrease the number of replicas to maintain the target CPU utilization.
- Scaling Range: The number of pods will be adjusted between 1 and 10 based on the CPU utilization.
- Resource Metrics: While this example uses CPU, HPAs can also use memory or custom metrics.
- Target Utilization: 50% target utilization is a common starting point, but this can be adjusted based on application needs.
- Namespace Scoping: The HPA is namespace-specific, allowing for isolated scaling policies across different parts of your application.
- Scaling Algorithm: Kubernetes uses a control loop to periodically adjust the number of replicas based on the observed metrics.
Ingress Controller in Kubernetes
The Ingress Controller in Kubernetes functions as a reverse proxy, managing external HTTP(S) traffic and routing it to the correct internal services. Unlike NodePort, which exposes services on every node, the Ingress Controller centralizes the routing process, making it more efficient and scalable.
Setting Up the Ingress Controller
To use Ingress, download and install the Ingress Controller (NGINX or others) in your Kubernetes cluster.
Basic Ingress Configuration Example:
- The YAML below defines a basic Ingress resource that specifies how incoming traffic is routed to internal services.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: geeky
labels:
name: geeky
namespace: geeky-ns
spec:
ingressClassName: nginx
rules:
- host: url
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: frontend
port:
number: 3000
- This setup routes HTTP requests aimed at
url
to the "frontend" service on port 3000.
Viewing Available Ingress Classes:
- To see the available ingress classes, run:
kubectl get ingressclass
Restricting Traffic in Production
In development environments, the configuration is often permissive, but in production, you’ll want to implement more restrictive rules.
For example, after purchasing a domain, you could set rules to limit traffic to a specific host:
spec:
rules:
- host: geeky.example.com
http:
paths:
- path: "/"
pathType: Prefix
backend:
service:
name: geeky
port:
number: 3000
This setup ensures that only traffic from geeky.example.com
can reach the service, providing additional security by rejecting requests from other hosts.
Ingress Controller and Load Balancer in AWS
When deploying an Ingress Controller (e.g., NGINX) to your AWS cluster, a load balancer is automatically provisioned to handle external traffic.
Example NGINX Ingress Controller setup:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/cloud/deploy.yaml
kubectl get services -n nginx
In a Production Environment:
After setting up a load balancer, you can create more restrictive Ingress rules by specifying a host.
Example Ingress configuration:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: geeky-ingress
namespace: geeky-ns
spec:
ingressClassName: nginx
rules:
- host: geeky.example.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: geeky
port:
number: 3000
This restricts access to the service to only geeky.example.com
.
Helm in Kubernetes
Helm is a powerful package manager for Kubernetes that simplifies application deployment, management, and scaling. By using Helm Charts, you can manage Kubernetes resources as a single unit, easing deployment, upgrades, and rollbacks.
Helm Basics
- Helm Chart: A Helm Chart is a collection of files that define a related set of Kubernetes resources.
- Release: A release is a deployment of a Helm chart. You can install or uninstall a release using Helm commands.
A typical Helm Chart structure:
mychart/
├── Chart.yaml # Contains chart information
├── values.yaml # Default configuration values
└── templates/ # Directory for template files
├── deployment.yaml
├── service.yaml
└── ingress.yaml
values.yaml
: Stores values for templates (likedeployment.yaml
orservice.yaml
).Chart.yaml
: Contains metadata about the chart.
Installing and Managing Helm Charts
- Install a Helm Chart:
helm install geeky geeky-1.0.0.tgz
2. Uninstall a Helm Release:
helm uninstall geeky
3. Helm Upgrade:
- To upgrade a release, use:
helm upgrade geeky geeky-1.0.2.tgz
- Or, to upgrade from the current directory:
helm upgrade geeky . -n geeky-ns
4. Rollback a Helm Release:
- If needed, you can rollback a release:
helm rollback geeky 2 -n geeky-ns
5. Helm Package:
- To package a Helm chart:
helm package .
6. List All Helm Releases:
helm list -A
Using External Helm Repositories
Helm supports external repositories like Bitnami, which provides popular charts for services like Elasticsearch and MySQL.
- Add a Helm Repo:
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
2. Install a Chart with Custom Values:
helm install geeky bitnami/mongodb --version 1x.x.1x -f values.yaml -n geeky-ns
This command installs MongoDB with custom configurations.
3. Remove a Helm Repo:
helm repo remove bitnami
Operators in Kubernetes
Kubernetes Operators extend Kubernetes’ functionality by managing complex applications using custom resources. These resources aren’t part of Kubernetes by default but are defined and controlled by the Operator.
Core Concepts
- Custom Resources (CRs): Operators introduce new resources that extend the Kubernetes API, tailored to specific applications.
- Custom Controllers: Operators implement controllers that watch custom resources and manage corresponding Kubernetes resources (e.g., Pods, Services, ConfigMaps).
How Operators Work
- Create a Custom Resource: When you create a custom resource (e.g., a PostgreSQL cluster), the operator’s controller detects it and creates the necessary Kubernetes resources.
- Monitor the Resources: The operator continually monitors and adjusts resources to maintain the desired state.
Example Custom Resource for a PostgreSQL Cluster:
apiVersion: database.example.com/v1
kind: PostgresCluster
metadata:
name: my-db
spec:
version: "13"
instances: 3
storage:
size: 1Gi
Apply the Custom Resource:
kubectl apply -f postgres-cluster.yaml
To monitor the custom and native resources:
kubectl get postgresclusters
kubectl get pods
Common Operators and Use Cases
Operators are available for managing various software systems, including:
- Databases: PostgreSQL, MongoDB, MySQL, Elasticsearch
- Message Queues: Apache Kafka, RabbitMQ
- Monitoring: Prometheus
- Service Meshes: Istio
Deploying to AWS with eksctl
Deploying Kubernetes to AWS with eksctl
offers the flexibility to manage clusters with multiple worker nodes.
- Install eksctl:
choco install -y eksctl eksctl version
2. Create a Cluster with eksctl:
Example command to create a cluster:
eksctl create cluster \
--name my-cluster \
--region us-west-2 \
--node-type t3.medium \
--nodes 3
This command creates a cluster with 3 worker nodes and updates the kubectl
context for easy interaction.
Final Cleanup
After finishing with your Kubernetes cluster and services, it’s important to perform proper cleanup to release resources and ensure everything is cleaned up.
- Delete the Kubernetes Cluster:
- If you’re using AWS and
eksctl
to manage your cluster, you can delete the entire cluster with the following command:
eksctl delete cluster --name my-cluster
- This will remove all resources associated with the cluster, including EC2 instances, VPCs, and other managed services.
2. Docker Cleanup:
- After working with Docker, it’s a good practice to remove unused resources (images, containers, networks, etc.) to free up space.
- To remove all unused Docker objects, including images, containers, volumes, and networks that are not associated with a container:
docker system prune -a
- This command will ask for confirmation before deleting these resources, so make sure to double-check that you don’t need any of them.
3. Disable Kubernetes on Your Machine:
- If you’re using
kubectl
locally to interact with Kubernetes clusters, you may want to disable it when you're done with the cluster. - You can simply remove the Kubernetes configuration file or reset it using:
kubectl config unset current-context
4. Delete Images and Containers from Docker:
To manually remove images and containers from Docker, you can use the following commands:
- Remove all stopped containers:
docker container prune
- Remove all unused images:
docker image prune -a
By following these cleanup steps, you can ensure that no unnecessary resources are left behind, optimizing both your local machine and cloud environment.