Update Kubernetes to 1.11 on Ubuntu
Goal
Kubeadm is coming with an upgrade option. The goal of this study is to leverage the option.
Initial set up
Kubernetes 1.10.4 is installed
cat /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
sudo dpkg -l kubeadm
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-======================================-========================-========================-=================================================================================
ii kubeadm 1.10.4-00 amd64 Kubernetes Cluster Bootstrapping Tool
sudo dpkg -l kubelet
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-======================================-========================-========================-=================================================================================
ii kubelet 1.10.4-00 amd64 Kubernetes Node Agent
sudo dpkg -l kubectl
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-======================================-========================-========================-=================================================================================
ii kubectl 1.10.4-00 amd64 Kubernetes Command Line Tool
kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:13:03Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:00:59Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Update kubelet and kubectl
Update aptitude
sudo apt-get update
Hit:2 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu xenial InRelease
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [107 kB]
Get:4 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [511 kB]
Get:5 http://security.ubuntu.com/ubuntu xenial-security/main i386 Packages [452 kB]
Get:6 http://security.ubuntu.com/ubuntu xenial-security/main amd64 DEP-11 Metadata [67.7 kB]
Get:7 http://security.ubuntu.com/ubuntu xenial-security/main DEP-11 64x64 Icons [68.0 kB]
Get:8 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [355 kB]
Get:9 http://security.ubuntu.com/ubuntu xenial-security/universe i386 Packages [303 kB]
Get:10 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [133 kB]
Get:11 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 DEP-11 Metadata [107 kB]
Get:12 http://security.ubuntu.com/ubuntu xenial-security/universe DEP-11 64x64 Icons [147 kB]
Hit:13 http://us.archive.ubuntu.com/ubuntu xenial InRelease
Get:14 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]
Get:15 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
Get:16 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [796 kB]
Get:17 http://us.archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages [728 kB]
Get:18 http://us.archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [329 kB]
Get:19 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 DEP-11 Metadata [318 kB]
Get:20 http://us.archive.ubuntu.com/ubuntu xenial-updates/main DEP-11 64x64 Icons [228 kB]
Get:21 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [640 kB]
Get:1 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]
Get:22 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe i386 Packages [585 kB]
Get:23 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 Packages [17.9 kB]
Get:24 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [257 kB]
Get:25 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 DEP-11 Metadata [246 kB]
Get:26 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe DEP-11 64x64 Icons [331 kB]
Get:27 http://us.archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 DEP-11 Metadata [5,964 B]
Get:28 http://us.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [6,740 B]
Get:29 http://us.archive.ubuntu.com/ubuntu xenial-backports/main i386 Packages [6,732 B]
Get:30 http://us.archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [4,180 B]
Get:31 http://us.archive.ubuntu.com/ubuntu xenial-backports/main amd64 DEP-11 Metadata [3,328 B]
Get:32 http://us.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [7,412 B]
Get:33 http://us.archive.ubuntu.com/ubuntu xenial-backports/universe i386 Packages [7,104 B]
Get:34 http://us.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 DEP-11 Metadata [5,100 B]
Get:35 http://us.archive.ubuntu.com/ubuntu xenial-backports/universe DEP-11 64x64 Icons [1,789 B]
Fetched 7,000 kB in 9s (757 kB/s)
Run the apt-get upgrade command. Notice how the kubeadm is “kept back”
sudo apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
linux-headers-4.13.0-26 linux-headers-4.13.0-26-generic linux-image-4.13.0-26-generic linux-image-extra-4.13.0-26-generic
Use 'sudo apt autoremove' to remove them.
The following packages have been kept back:
kubeadm
The following packages will be upgraded:
amd64-microcode console-setup console-setup-linux cpp-5 desktop-file-utils ebtables g++-5 gcc-5 gcc-5-base gir1.2-javascriptcoregtk-4.0 gir1.2-webkit2-4.0 gnome-software
gnome-software-common keyboard-configuration kubectl kubelet libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgcrypt20 libgomp1 libitm1 libjasper1
libjavascriptcoregtk-4.0-18 libldap-2.4-2 liblsan0 libmpx0 libplymouth4 libquadmath0 libssl-dev libssl1.0.0 libstdc++-5-dev libstdc++6 libtsan0 libubsan0
libwebkit2gtk-4.0-37 libwebkit2gtk-4.0-37-gtk2 linux-firmware openssl plymouth plymouth-label plymouth-theme-ubuntu-logo plymouth-theme-ubuntu-text rfkill snapd
ubuntu-core-launcher ubuntu-software update-notifier update-notifier-common wireless-regdb
52 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
sudo apt-get upgrade
sudo apt-get autoremove
sudo reboot
sudo dpkg -l kubectl
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-======================================-========================-========================-=================================================================================
ii kubectl 1.11.0-00 amd64 Kubernetes Command Line Tool
sudo dpkg -l kubeadm
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-======================================-========================-========================-=================================================================================
ii kubeadm 1.10.4-00 amd64 Kubernetes Cluster Bootstrapping Tool
sudo dpkg -l kubelet
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-======================================-========================-========================-=================================================================================
ii kubelet 1.11.0-00 amd64 Kubernetes Node Agent
kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:17:28Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:00:59Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Upgrading kubeadm
Install new kubeadm and associated new cri-tools
sudo apt-get install kubeadm
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
cri-tools
The following NEW packages will be installed:
cri-tools
The following packages will be upgraded:
kubeadm
1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 14.7 MB of archives.
After this operation, 70.9 MB disk space will be freed.
Do you want to continue? [Y/n] y
Get:1 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 cri-tools amd64 1.11.0-00 [5,309 kB]
Get:2 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubeadm amd64 1.11.0-00 [9,422 kB]
Fetched 14.7 MB in 5s (2,780 kB/s)
Selecting previously unselected package cri-tools.
(Reading database ... 225649 files and directories currently installed.)
Preparing to unpack .../cri-tools_1.11.0-00_amd64.deb ...
Unpacking cri-tools (1.11.0-00) ...
Preparing to unpack .../kubeadm_1.11.0-00_amd64.deb ...
Unpacking kubeadm (1.11.0-00) over (1.10.4-00) ...
Setting up cri-tools (1.11.0-00) ...
Setting up kubeadm (1.11.0-00) ...
Installing new version of config file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf ...
kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:14:41Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
Upgrading your Kubernetes cluster
Cluster does not seems be in such a great shape
NAME READY STATUS RESTARTS AGE
pod/calico-etcd-j8zqp 1/1 Running 22 11d
pod/calico-kube-controllers-679568f47c-wgz7n 1/1 Running 27 11d
pod/calico-node-knstj 2/2 Running 39 11d
pod/etcd-xxxxxxx 0/1 Pending 0 1h
pod/kube-apiserver-xxxxxxx 0/1 Pending 0 1h
pod/kube-controller-manager-xxxxxxx 0/1 Pending 0 1h
pod/kube-dns-86f4d74b45-fs7bq 3/3 Running 66 11d
pod/kube-proxy-mpbf7 1/1 Running 22 11d
pod/kube-scheduler-xxxxxxx 0/1 Pending 0 1h
pod/tiller-deploy-5c688d5f9b-6cl5n 1/1 Running 23 11d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/calico-etcd ClusterIP 10.96.232.136 <none> 6666/TCP 11d
service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP 11d
service/tiller-deploy ClusterIP 10.97.44.240 <none> 44134/TCP 11d
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/calico-etcd 1 1 0 1 0 node-role.kubernetes.io/master= 11d
daemonset.apps/calico-node 1 1 0 1 0 <none> 11d
daemonset.apps/kube-proxy 1 1 0 1 0 <none> 11d
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/calico-kube-controllers 1 1 1 0 11d
deployment.apps/kube-dns 1 1 1 0 11d
deployment.apps/tiller-deploy 1 1 1 0 11d
NAME DESIRED CURRENT READY AGE
replicaset.apps/calico-kube-controllers-679568f47c 1 1 0 11d
replicaset.apps/kube-dns-86f4d74b45 1 1 0 11d
replicaset.apps/tiller-deploy-5c688d5f9b 1 1 0 11d
sudo kubeadm upgrade apply --dry-run 1.11.0 > /tmp/traces
[upgrade/health] FATAL: [preflight] Some fatal errors occurred:
[ERROR MasterNodesReady]: there are NotReady masters in the cluster: [xxxxxxx]
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
Usefull Links
TBD