Skip to content

Part 2: Kubernetes v1.33.13 Three-Node Cluster Offline Installation Guide (Internal Network Environment Edition) ​

Kubernetes three-node cluster offline installation diagram

Applicable scenarios and prerequisites ​

ProjectDescription
Applicable scenariosIn an intranet or completely offline environment, use the offline package created in the first article to deploy a Kubernetes cluster.
target cluster3 nodes (1 control plane + 2 worker nodes)
Operating systemUbuntu 22.04 (all nodes)
Kubernetes versionv1.33.13
Container runtimecontainerd(>= 1.7.25)
Network pluginCalico v3.26.0
Prerequisite dependenciesCompleted the first article, obtained k8s-offline-v1.33.13.tar.gz

Part Two Objectives ​

On three intranet Ubuntu 22.04 servers, use offline packages to accomplish the following goals:

  • ✅ Basic environment configuration for all nodes (IP, hostname, system parameters)

  • ✅ Offline installation of Kubernetes components and containerd

  • ✅ Import all container images

  • ✅ Initialize the control plane (kubeadm init)

  • ✅ Install Calico network plugin

  • ✅ Join two worker nodes

  • ✅ Verify the cluster is fully available

Simulate an enterprise intranet or fully disconnected environment (all nodes) ​

Why simulate an intranet environment? ​

In actual enterprise private deployment scenarios, Kubernetes clusters often run on Strictly isolated intranet environment In [that context], nodes may be prohibited from accessing the external network. In order to [do so] in a laboratory or test environment Verify in advance With this deployment process, we need to proactively simulate this limitation.

But here's an easy pitfall to fall into:You cannot simply and crudely delete the default gateway., otherwise it will cause communication failures between internal cluster components (such as Calico and CoreDNS), resulting in deployment failure.

Correct approach: keep the gateway + use iptables to control traffic ​

Deleting the default gateway will break routing between nodes; even if all destination IPs are in the same subnet, it may still cause TCP connection anomalies. The correct approach is:

  • Retain default gateway—— Ensure normal internal communication within the cluster

  • Use iptables for precise control— Only allow necessary internal network traffic and deny external network access.

How can you verify the current system's DNS configuration? ​

In Ubuntu 22.04,/etc/resolv.conf usually points to systemd-resolved symlink. To view the DNS configuration actually in effect, you can use the following command:

bash
# View the current system's DNS configuration status
resolvectl status

# View DNS resolution statistics
resolvectl statistics

If you find that the system is configured with external DNS (such as 8.8.8.8, 114.114.114.114), and you want to completely disable external DNS resolution in the internal network, you can restrict it using iptables rules.

iptables rule configuration example ​

The following rules demonstrate how Only allow intranet and Kubernetes-related network segments, and deny all other traffic.:

bash
# 1. Allow local loopback first (always allow)
sudo iptables -A OUTPUT -o lo -j ACCEPT

# 2. Allow the internal network (the target network for SSH connections)
sudo iptables -A OUTPUT -d 192.168.0.0/16 -j ACCEPT

# 3. Allow Kubernetes-related network segments
sudo iptables -A OUTPUT -d 10.0.0.0/8 -j ACCEPT
sudo iptables -A OUTPUT -d 10.96.0.0/12 -j ACCEPT
sudo iptables -A OUTPUT -d 10.244.0.0/16 -j ACCEPT

# 4. Only at the end, deny all other traffic (block internet access)
sudo iptables -A OUTPUT -d 0.0.0.0/0 -j DROP

Special attention: of the above rules Order is crucial., you must first allow (ACCEPT) the required traffic, and only then deny (DROP) all other traffic. If the order is reversed, all traffic will be dropped directly, causing the SSH connection to disconnect.

Verify whether the rule takes effect ​

bash
# View the currently active iptables rules
sudo iptables -L -v -n

# Test whether internal network communication is working properly
ping -c 2 192.168.0.226

# Test whether the external network is blocked (expected to time out or be unresponsive)
ping -c 2 8.8.8.8

# Test whether DNS resolution is blocked (expected to fail)
nslookup bing.com
# Or use dig
dig bing.com

1. Server Planning and Basic Configuration ​

1.1 Cluster Node Planning ​

RolehostnameIP addressConfiguration recommendations
control planenode1192.168.0.2252C/4G+
worker nodenode2192.168.0.2262C/4G+
worker nodenode3192.168.0.2272C/4G+

Please modify the above IP address according to your actual network environment.

1.2 Configure the hostname (all nodes) ​

bash
# Set the hostname (execute based on role)
sudo hostnamectl set-hostname node1   # Execute on node1
sudo hostnamectl set-hostname node2   # Execute on node2
sudo hostnamectl set-hostname node3   # Execute on node3

# Verification
hostname

1.3 Configure Static IP (All Nodes) ​

  • View network adapter name:
bash
ip a
  • Edit the Netplan configuration file(Taking node1 as an example; for other nodes, just change the IP):
bash
sudo vim /etc/netplan/00-installer-config.yaml

The content configuration is as follows (please replace ens160 to your actual network adapter name):

yaml
network:
  version: 2
  ethernets:
    ens160:
      addresses:
        - 192.168.0.225/24
      routes:
        - to: default
          via: 192.168.0.1
      nameservers:
        addresses:
          - 114.114.114.114
          - 8.8.8.8
  • Application Configuration:
bash
sudo netplan apply

1.4 Configure hostname resolution (all nodes) ​

Edit /etc/hosts file, add the IP and hostname mappings for all nodes to enable mutual access within the cluster.

bash
sudo tee -a /etc/hosts <<EOF
192.168.0.225 node1
192.168.0.226 node2
192.168.0.227 node3
EOF

# Verify interoperability
ping node1 -c 2
ping node2 -c 2
ping node3 -c 2

1.5 Basic System Configuration (All Nodes) ​

bash
# 1. Disable the firewall
sudo systemctl disable --now ufw

# 2. Disable swap partition (permanently disable)
sudo sed -ri 's/.*swap.*/#&/' /etc/fstab
sudo swapoff -a  # Takes effect immediately, no restart required.

# 3. Load the necessary kernel modules
sudo tee /etc/modules-load.d/k8s.conf <<EOF
overlay
br_netfilter
EOF

# Load immediately (takes effect for the current session)
sudo modprobe overlay
sudo modprobe br_netfilter

# Verification
lsmod | grep -E "overlay|br_netfilter"

# 4. Configure kernel parameters (allow bridged firewall and IP forwarding)
sudo tee /etc/sysctl.d/99-kubernetes.conf <<EOF
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl --system

# 5. Confirm that cgroups v2 is enabled (expected output is cgroup2)
mount | grep cgroup

cgroups v2 is the recommended configuration for Kubernetes v1.33.13. If your system is still using v1, please refer to the relevant documentation to switch.

1.6 Configure time synchronization (all nodes) ​

Ensuring that all nodes have consistent time is crucial for cluster certificates and security mechanisms.

bash
# Configure the time zone as Asia/Shanghai
sudo timedatectl set-timezone Asia/Shanghai

# Enable NTP synchronization (if it is an ESXi virtual machine, it can also be configured in vCenter)
sudo timedatectl set-ntp true

# Verify time synchronization status
timedatectl

II. Distribute and install the offline package ​

2.1 Transfer the offline package to all nodes ​

Use scp The command distributes the first prepared offline package to all target servers.

bash
# Transfer from the build machine to each node (please replace user and IP)
scp k8s-offline-v1.33.13.tar.gz [email protected]:/tmp/
scp k8s-offline-v1.33.13.tar.gz [email protected]:/tmp/
scp k8s-offline-v1.33.13.tar.gz [email protected]:/tmp/

2.2 Extract the offline package (all nodes) ​

bash
# Create installation directory
sudo mkdir -p /opt/k8s-install

# Extract to the installation directory
sudo tar -xzvf /tmp/k8s-offline-v1.33.13.tar.gz -C /opt/k8s-install

# Enter the extracted directory
cd /opt/k8s-install/k8s-offline-v1.33.13

# Check the content to ensure the file is complete.
ls -l

2.3 Run the installation script (all nodes) ​

Run install.sh Install all deb Package and import the image.

bash
# Execute the offline installation script
sudo ./install.sh

The script will automatically complete the following tasks:

  • Install Kubernetes components (kubeadm、kubelet,kubectl)

  • Install containerd runtime

  • Load k8s-images.tar,calico-images.tar and other image files

  • Lock component versions to prevent automatic updates.

III. Configure containerd ​

3.1 Generate the default configuration file ​

bash
# Generate default configuration
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml

3.2 Correct the sandbox image version (key step) ​

Must ensure sandbox_image Consistent with the cluster version, otherwise kubelet It may not start properly.

bash
# 1. Check the current pause image version in containerd
sudo ctr -n=k8s.io images ls | grep pause

# 2. Edit the containerd configuration file
sudo vim /etc/containerd/config.toml

Find sandbox_image field, change it to:

toml
sandbox_image = "registry.k8s.io/pause:3.10"
bash
# 3. Restart containerd to make the configuration take effect
sudo systemctl restart containerd

3.3 Configure crictl ​

Create /etc/crictl.yaml document, in order to crictl The command can be used with containerd Communication.

bash
sudo tee /etc/crictl.yaml <<EOF
runtime-endpoint: unix:///var/run/containerd/containerd.sock
image-endpoint: unix:///var/run/containerd/containerd.sock
timeout: 10
debug: false
EOF

3.4 Verify containerd status ​

bash
# Check service status
sudo systemctl status containerd

# List the imported images and confirm that all offline images are included.
sudo ctr -n=k8s.io images ls -q

4. Initialize the control plane (only execute on node1) ​

4.1 Execute cluster initialization ​

bash
# Set the API Server address to the IP of node1
NODE1_IP=192.168.0.225

# Initialize the cluster using a CIDR that does not conflict with node IPs (RFC 1918 private address ranges are recommended)
sudo kubeadm init \
  --kubernetes-version v1.33.13 \
  --pod-network-cidr=10.244.0.0/16 \
  --apiserver-advertise-address=$NODE1_IP \
  --control-plane-endpoint=$NODE1_IP \
  --cri-socket=unix:///var/run/containerd/containerd.sock

After successful initialization,Please be sure to record it. output's kubeadm join command that will be used later to add worker nodes to the cluster.

4.2 Configure kubectl ​

bash
# Configure kubectl for the current regular user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# Verify whether kubectl can access the cluster
kubectl get nodes

At this point, you should be able to see that the status of node1 is NotReady, this is normal because the network plugin has not been installed yet.

4.3 Install the Calico network plugin ​

4.3.1 Deploy Calico ​

bash
# Install using the Calico configuration file in the offline package
kubectl apply -f /opt/k8s-install/k8s-offline-v1.33.13/manifests/calico.yaml

# Observe the Calico Pod startup process (press Ctrl+C to exit)
kubectl get pods -n kube-system -w

Wait for all calico-* Pod entry Running Status.

4.3.2 Verify Calico status ​

bash
# Check the status of Calico-related Pods
kubectl get pods -n kube-system | grep calico

# Expected output
# calico-kube-controllers-xxx   1/1     Running   0   2m
# calico-node-xxx               1/1     Running   0   2m

4.3.3 Check node status ​

bash
# Check the node status; it should become Ready
kubectl get nodes

⚠️ Common Issue: CoreDNS CrashLoopBackOff (Important Pitfall in Offline Environments!) ​

In an offline environment, after Calico installation is complete, you may encounter The CoreDNS Pod has been in CrashLoopBackOff state. problem.

Phenomenon:

bash
kubectl get pods -n kube-system

# Output example:
# NAME                                       READY   STATUS             RESTARTS   AGE
# calico-kube-controllers-xxx                1/1     Running            0          5m
# calico-node-xxx                            1/1     Running            0          5m
# coredns-674b8bbfcf-7zlhn                   0/1     CrashLoopBackOff   6          5m
# coredns-674b8bbfcf-hzqw6                   0/1     CrashLoopBackOff   6          5m
# etcd-localhost                             1/1     Running            0          8m
# kube-apiserver-localhost                   1/1     Running            0          8m
# kube-controller-manager-localhost          1/1     Running            0          8m
# kube-proxy-92ftp                           1/1     Running            0          8m
# kube-scheduler-localhost                   1/1     Running            0          8m

Troubleshooting steps:

bash
# 1. Check CoreDNS logs (confirm the cause of the error)
kubectl logs -n kube-system coredns-674b8bbfcf-7zlhn -p

# Error log output:
# plugin/forward: no nameservers found

Error cause analysis:

CoreDNS is configured by default with forward . /etc/resolv.conf, in an offline environment:

  • /etc/resolv.conf point to systemd-resolved Stub parser (127.0.0.53)

  • systemd-resolved No available upstream DNS server is configured.

  • CoreDNS detected no available upstream DNS at startup, causing CrashLoopBackOff

Root cause: CoreDNS's forward The plugin attempts to forward DNS requests to the upstream server in an offline environment, but the upstream DNS is unreachable, causing startup to fail.

Solution: Delete or comment out forward Configuration line

bash
# Edit CoreDNS ConfigMap
kubectl edit configmap coredns -n kube-system

Find and delete or comment out forward The relevant configuration line.

Before modification (default configuration):

yaml
data:
  Corefile: |
    .:53 {
        errors
        health {
            lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
            pods insecure
            fallthrough in-addr.arpa ip6.arpa
            ttl 30
        }
        prometheus :9153
        forward . /etc/resolv.conf   # ← This line needs to be deleted
        cache 30
        loop
        reload
        loadbalance
    }

Revised version (applicable to offline environments):

yaml
data:
  Corefile: |
    .:53 {
        errors
        health {
            lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
            pods insecure
            fallthrough in-addr.arpa ip6.arpa
            ttl 30
        }
        prometheus :9153
        # The forward line has been deleted; no external DNS requests are forwarded.
        cache 30
        loop
        reload
        loadbalance
    }

Restart CoreDNS:

bash
# Delete the CoreDNS Pod so that the Deployment automatically recreates it and loads the new configuration.
kubectl delete pods -n kube-system -l k8s-app=kube-dns

# Monitor startup process
kubectl get pods -n kube-system -w | grep coredns

Verify that CoreDNS has returned to normal:

bash
# View CoreDNS Pod status
kubectl get pods -n kube-system | grep coredns

# Expected output:
# coredns-674b8bbfcf-46mtw   1/1     Running   0   39s
# coredns-674b8bbfcf-m8k5f   1/1     Running   0   39s

# Check CoreDNS logs (to confirm there are no errors)
kubectl logs -n kube-system -l k8s-app=kube-dns --tail=10

Verify cluster DNS resolution:

bash
# Test internal DNS resolution within the cluster.
kubectl run test-dns --image=busybox:1.36.1 --rm -it --restart=Never -- nslookup kubernetes.default.svc.cluster.local

# Expected output:
# Server:    10.96.0.10
# Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
#
# Name:      kubernetes.default.svc.cluster.local
# Address 1: 10.96.0.1

4.3.4 Why does CoreDNS still work normally after deleting forward? ​

Configuration itemeffectIs an offline environment required?
kubernetes pluginResolve the Service inside the cluster (*.cluster.local)✅ Required
forward PluginForward external domain name resolution requests (such as google.com)❌ Not needed

In an offline environment, Pods only need to resolve Services inside the cluster and do not need to resolve external domain names. Delete forward After the plugin, CoreDNS focuses on through kubernetes The plugin handles DNS queries within the cluster, so it is fully self-contained and requires no upstream DNS.

4.3.5 Host DNS Configuration (Optional Notes) ​

In Ubuntu 22.04,/etc/resolv.conf Default target systemd-resolved stub parser (127.0.0.53):

bash
# View DNS configuration status
resolvectl status

# Test external domain name resolution (expected to fail)
nslookup bing.com
# Output: ;; Got SERVFAIL reply from 127.0.0.53

The host machine's inability to resolve external domain names will not affect the normal operation of the Kubernetes cluster. Because:

  • Host DNS resolution is managed by systemd-resolved

  • DNS resolution within the cluster is handled by CoreDNS's kubernetes Plugin processing

  • The two are independent of each other.

4.3.6 Confirm that the cluster is fully ready ​

bash
# View the status of all system Pods
kubectl get pods -n kube-system

# All Pods are expected to be in Running state.
# NAME                                       READY   STATUS    RESTARTS   AGE
# calico-kube-controllers-69c889dc5f-pr8q8   1/1     Running   0          19m
# calico-node-smchk                          1/1     Running   0          19m
# coredns-674b8bbfcf-46mtw                   1/1     Running   0          39s
# coredns-674b8bbfcf-m8k5f                   1/1     Running   0          39s
# etcd-localhost                             1/1     Running   0          21m
# kube-apiserver-localhost                   1/1     Running   0          21m
# kube-controller-manager-localhost          1/1     Running   0          21m
# kube-proxy-92ftp                           1/1     Running   0          21m
# kube-scheduler-localhost                   1/1     Running   0          21m

# View node status
kubectl get nodes
# Expected output:
# NAME      STATUS   ROLES           AGE   VERSION
# node1     Ready    control-plane   21m   v1.33.13

4.4 Confirm cluster status ​

bash
# Check the node status again; it should change to Ready.
kubectl get nodes

V. Add worker nodes (execute on node2 and node3) ​

At each worker node above, the output when initializing with node1 kubeadm join Command.

bash
# Example command (please use your own token and hash)
sudo kubeadm join 192.168.0.225:6443 --token abcdef.0123456789abcdef \
  --discovery-token-ca-cert-hash sha256:7e5a3f9c2b8d1a6e4f7c3b9a2d5e8f1c4a7b2d9e6f3c8a1b5e7d9f2c4a6b8e0f

Return to node1 to verify:

bash
kubectl get nodes

Expected output:

NAME    STATUS   ROLES           AGE    VERSION
node1   Ready    control-plane   159m   v1.33.13
node2   Ready    <none>          11m    v1.33.13
node3   Ready    <none>          11m    v1.33.13

VI. Verify the Cluster ​

6.1 Node and Pod Status Check ​

View detailed information for all nodes ​

bash
kubectl get nodes -o wide

Expected output:

NAME    STATUS   ROLES           AGE    VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION       CONTAINER-RUNTIME
node1   Ready    control-plane   160m   v1.33.13   192.168.0.225   <none>        Ubuntu 22.04.5 LTS   5.15.0-187-generic   containerd://2.3.3
node2   Ready    <none>          12m    v1.33.13   192.168.0.226   <none>        Ubuntu 22.04.5 LTS   5.15.0-187-generic   containerd://2.3.3
node3   Ready    <none>          12m    v1.33.13   192.168.0.227   <none>        Ubuntu 22.04.5 LTS   5.15.0-187-generic   containerd://2.3.3

Check core component Pod status ​

bash
kubectl get pods -n kube-system

Expected output:

bash
NAME                                       READY   STATUS    RESTARTS      AGE
calico-kube-controllers-69c889dc5f-ndqsx   1/1     Running   1 (71m ago)   161m
calico-node-8xk2q                          1/1     Running   0             14m
calico-node-m7hqf                          1/1     Running   1 (71m ago)   161m
calico-node-zmphq                          1/1     Running   0             14m
coredns-674b8bbfcf-5rq2q                   1/1     Running   1 (71m ago)   162m
coredns-674b8bbfcf-jc5wd                   1/1     Running   1 (71m ago)   162m
etcd-node1                                 1/1     Running   1 (71m ago)   162m
kube-apiserver-node1                       1/1     Running   1 (71m ago)   162m
kube-controller-manager-node1              1/1     Running   1 (71m ago)   162m
kube-proxy-46jmp                           1/1     Running   1 (71m ago)   162m
kube-proxy-9jhw7                           1/1     Running   0             14m
kube-proxy-z4s7k                           1/1     Running   0             14m
kube-scheduler-node1                       1/1     Running   1 (71m ago)   162m

6.2 Test DNS Resolution ​

bash
# Use busybox to test whether CoreDNS can resolve a Service
kubectl run -it --rm --restart=Never busybox \
  --image=busybox:1.36.1 \
  -- nslookup kubernetes.default

The expected output should include:

txt
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes.default
Address 1: 10.96.0.1

6.3 Verify Pod network communication ​

bash
# Create and expose an Nginx Deployment
kubectl create deployment nginx --image=nginx:1.31.3
kubectl expose deployment nginx --port=80 --type=ClusterIP

# Get Service IP
kubectl get svc nginx

# Clean up test resources
kubectl delete deployment nginx
kubectl delete svc nginx

6.4 Verify Dashboard (Optional) ​

If the Dashboard image and configuration files are already included in the offline package, you can deploy them by following these steps:

bash
# Deploy Kubernetes Dashboard
kubectl apply -f /opt/k8s-install/k8s-offline-v1.33.13/manifests/kubernetes-dashboard.yaml

# Waiting for the Pod to start
kubectl get pods -n kubernetes-dashboard -w

# Create administrator user
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard
EOF

# Get login Token
kubectl -n kubernetes-dashboard create token admin-user

附录:常见问题排查 ​

Q1: The node remains in the NotReady state. ​

Reason: Network plugin is not ready.

Solve:

bash
# Check whether the Calico Pod is running properly
kubectl get pods -n kube-system | grep calico

# Check kubelet logs
journalctl -u kubelet -f --lines=50

Q2: kubelet cannot start ​

Reason: containerd sandbox image versions are inconsistent.

Solve: Check /etc/containerd/config.toml in sandbox_image Whether it is registry.k8s.io/pause:3.10(see Section 3.2).

Q3: crictl error ​

Reason: Not configured correctly /etc/crictl.yaml.

Solve: Refer to Section 3.3 to create a configuration file.

Q4: Failed to join worker node ​

reason: token has expired or CA certificate hash does not match.

Solve: Regenerate the join command on node1:

bash
kubeadm token create --print-join-command

Q5: DNS resolution failed ​

Reason: CoreDNS Pod is not running properly.

Solve:

bash
# Check CoreDNS Pod status
kubectl get pods -n kube-system | grep coredns

# View CoreDNS logs
kubectl logs -n kube-system <coredns-pod-name>
ResourcesDescription
Offline package downloadk8s-offline-v1.33.13-20260823-1241.tar.gz
SHA256 checksumk8s-offline-v1.33.13-20260823-1241.tar.gz.sha256

📌This concludes Part Two., All commands have been verified in the Ubuntu 22.04 environment. Please execute them in order to ensure successful cluster deployment. If you have any questions, please refer to the FAQ section in the documentation.

Last updated on11/07/2026, 02:16 PM
@fishfinal | @fishfinals | [email protected]