Kubernetes on macOS: 3-Node Ubuntu 24.04 ARM Cluster

Prerequisites

3 Ubuntu VMs (18.04/20.04+) VMs running on macOS (using UTM, Parallels, or VMware)
Each VM should have:
- Minimum: 2 CPUs
- Minimum: 2GB RAM (4GB recommended for master)
- 20GB+ SSD or NVMe
Static IP addresses or predictable DHCP
Internet access/NAT

Network Configuration

Assumed IP addresses (adjust to your network):

kubemaster: 192.168.2.12
kubeworker1: 192.168.2.13
kubeworker2: 192.168.2.14

PART 1: PREPARE ALL NODES (Run on ALL 3 machines)

Step 1: Update system and set hostnames

On Kubemaster:

bash
sudo hostnamectl set-hostname kubemaster

On Kubeworker1:

bash
sudo hostnamectl set-hostname kubeworker1

On Kubeworker2:

bash
sudo hostnamectl set-hostname kubeworker2

Step 2: Configure /etc/hosts (on ALL nodes)

bash
sudo nano /etc/hosts

Add these lines (adjust IPs to match your setup):

192.168.2.12   kubemaster kubemaster
192.168.2.13   kubeworker1 kubeworker1
192.168.2.14   kubeworker2 kubeworker2

Step 3: Disable swap (on ALL nodes)

bash
# Disable swap immediately
sudo swapoff -a

# Disable swap permanently
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

# Verify swap is off
free -h

Step 4: Load required kernel modules (on ALL nodes)

bash
# Load modules
sudo modprobe overlay
sudo modprobe br_netfilter

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

Step 5: Configure sysctl parameters (on ALL nodes)

bash
# Add parameters
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF

# Apply settings
sudo sysctl --system

# Verify
sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward

Step 6: Install containerd (on ALL nodes)

bash
# Update packages
sudo apt-get update

# Install containerd
sudo apt-get install -y ca-certificates curl gnupg lsb-release apt-transport-https

# Add Docker's GPG key
sudo mkdir -p /etc/containerd
sudo mkdir -p /etc/apt/keyrings

# Configure containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml

# Set SystemdCgroup = true (IMPORTANT!)
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml

# Restart containerd
sudo systemctl restart containerd
sudo systemctl enable containerd
sudo systemctl status containerd

Specifying which Cgroup driver run in our System using: ps -p 1 (If Systemd)

Step 7: Install Kubernetes components (on ALL nodes)

bash
# Install prerequisites
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg

sudo mkdir -p -m 755 /etc/apt/keyrings

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

Step 8: Configure crictl (on ALL nodes)

bash
cat <<EOF | sudo tee /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOF

Verify installations:

bash
kubeadm version
kubectl version --client
kubelet --version

PART 2: INITIALIZE THE MASTER NODE (Run on Kubemaster ONLY)

Step 9: Initialize the cluster

bash
# On Kubemaster only
sudo kubeadm init \
  --pod-network-cidr=10.244.0.0/16 \
  --apiserver-advertise-address=192.168.2.12 \
  --cri-socket=unix:///run/containerd/containerd.sock \
  --upload-certs

Wait for completion (5-10 minutes). You should see: “Your Kubernetes control-plane has initialized successfully!”

Step 10: Save the join command (IMPORTANT!)

The output will include a join command like:

bash
kubeadm join kubemaster:6443 --token xxxx.xxxxxxxxxx \
  --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxx

SAVE THIS COMMAND - you'll need it for the worker nodes.

Step 11: Configure kubectl for your user (on Kubemaster)

bash
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Verify:

bash
kubectl cluster-info
kubectl get nodes

Step 12: Install Flannel CNI (on Kubemaster)

bash
# Install Flannel (network)
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

# Wait for flannel pods
kubectl get pods -n kube-flannel
kubectl get pods -n kube-system

Step 13: Verify master node is ready

bash
# Check node status (should show Ready after 1-2 minutes)
kubectl get nodes

# Check all pods are running
kubectl get pods --all-namespaces

PART 3: JOIN WORKER NODES (Run on Kubeworker1 and Kubeworker2)

Step 14: Join worker nodes to the cluster

On Kubeworker1:

bash
# Use the exact join command from Step 10 with sudo
sudo kubeadm join kubemaster:6443 --token xxxx.xxxxxxxxxx \
  --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxx

On Kubeworker2:

bash
# Use the same join command (with sudo)
sudo kubeadm join kubemaster:6443 --token xxxx.xxxxxxxxxx \
  --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxx

Step 15: If token expired, generate new one (on Kubemaster)

bash
# If token expired (tokens expire after 24 hours)
# Generate new token on master
kubeadm token create --print-join-command

PART 4: VERIFY CLUSTER (Run on Kubemaster)

Step 16: Check all nodes

bash
kubectl get nodes -o wide

Expected output:

NAME          STATUS   ROLES           AGE   VERSION   INTERNAL-IP
kubemaster    Ready    control-plane   10m   v1.31.0   192.168.0.4
kubeworker1   Ready    <none>          5m    v1.31.0   192.168.0.5
kubeworker2   Ready    <none>          5m    v1.31.0   192.168.0.6

Step 17: Verify all pods are running

bash
# Check all system pods
kubectl get pods --all-namespaces

# Check component status
kubectl get cs

# Check cluster info
kubectl cluster-info

Step 18: Label worker nodes (optional)

bash
kubectl label node kubeworker1 node-role.kubernetes.io/worker=worker
kubectl label node kubeworker2 node-role.kubernetes.io/worker=worker

PART 5: DEPLOY A TEST APPLICATION

Step 19: Deploy nginx test

bash
# Create a test deployment
kubectl create deployment nginx-test --image=nginx --replicas=3

# Check deployment
kubectl get deployments

# Check pods
kubectl get pods -o wide

Finally ! Your 3-node Kubernetes cluster is now fully operational!