{"id":130,"date":"2017-03-30T15:58:05","date_gmt":"2017-03-30T07:58:05","guid":{"rendered":"http:\/\/www.jsjs.org\/?p=130"},"modified":"2017-03-30T15:58:05","modified_gmt":"2017-03-30T07:58:05","slug":"a","status":"publish","type":"post","link":"https:\/\/blog.jsjs.org\/?p=130","title":{"rendered":"\u4f7f\u7528Kubeadm\u5b89\u88c5Kubernetes"},"content":{"rendered":"

\u5728\u300a\u5f53Docker\u9047\u5230systemd<\/a>\u300b\u4e00\u6587\u4e2d\uff0c\u6211\u63d0\u5230\u8fc7\u8fd9\u4e24\u5929\u513f\u4e00\u76f4\u5728\u505a\u7684\u4e00\u4e2atask\uff1a\u4f7f\u7528kubeadm<\/a>\u5728Ubuntu 16.04<\/a>\u4e0a\u5b89\u88c5\u90e8\u7f72Kubernetes<\/a>\u7684\u6700\u65b0\u53d1\u5e03\u7248\u672c-k8s 1.5.1<\/a>\u3002<\/p>\n

\u5e74\u4e2d\uff0cDocker\u5ba3\u5e03\u5728Docker engine\u4e2d\u96c6\u6210swarmkit\u5de5\u5177\u5305\uff0c\u8fd9\u4e00announcement\u5728\u8f7b\u91cf\u7ea7\u5bb9\u5668\u754c\u5f15\u53d1\u8f69\u7136\u5927\u6ce2\u3002\u6bd5\u7adf\u5f00\u53d1\u8005\u662f\u61d2\u60f0\u7684^0^\uff0c\u6709\u4e86docker swarmkit\uff0c\u9a71\u52a8developer\u53bb\u5b89\u88c5\u5176\u4ed6\u5bb9\u5668\u7f16\u6392\u5de5\u5177\u7684\u52a8\u529b\u5728\u54ea\u91cc\u5462\uff1f\u5373\u4fbfdocker engine\u8fd8\u4e0d\u662f\u5f53\u5e74\u90a3\u4e2a\u88ab\u4eba\u4eec\u9ad8\u9891\u4f7f\u7528\u7684IE\u6d4f\u89c8\u5668\u3002\u4f5c\u4e3a\u9488\u5bf9Docker\u516c\u53f8\u8fd9\u4e00\u5e02\u573a\u884c\u4e3a\u7684\u56de\u5e94\uff0c\u5bb9\u5668\u96c6\u7fa4\u7ba1\u7406\u548c\u670d\u52a1\u7f16\u6392\u9886\u5148\u8005Kubernetes\u5728\u4e09\u4e2a\u6708\u540e\u53d1\u5e03\u4e86Kubernetes1.4.0\u7248\u672c<\/a>\u3002\u5728\u8fd9\u4e2a\u7248\u672c\u4e2dK8s\u65b0\u589e\u4e86kubeadm\u5de5\u5177\u3002kubeadm\u7684\u4f7f\u7528\u65b9\u5f0f\u6709\u70b9\u50cf\u96c6\u6210\u5728docker engine<\/a>\u4e2d\u7684swarm kit\u5de5\u5177<\/a>\uff0c\u65e8\u5728\u6539\u5584\u5f00\u53d1\u8005\u5728\u5b89\u88c5\u3001\u8c03\u8bd5\u548c\u4f7f\u7528k8s\u65f6\u7684\u4f53\u9a8c\uff0c\u964d\u4f4e\u5b89\u88c5\u548c\u4f7f\u7528\u95e8\u69db\u3002\u7406\u8bba\u4e0a\u901a\u8fc7\u4e24\u4e2a\u547d\u4ee4\uff1ainit\u548cjoin\u5373\u53ef\u642d\u5efa\u51fa\u4e00\u5957\u5b8c\u6574\u7684Kubernetes cluster\u3002<\/p>\n

\u4e0d\u8fc7\uff0c\u548c\u521d\u5165docker\u5f15\u64ce\u7684swarmkit\u4e00\u6837\uff0ckubeadm\u76ee\u524d\u4e5f\u5728active development\u4e2d\uff0c\u4e5f\u4e0d\u662f\u90a3\u4e48stable\uff0c\u56e0\u6b64\u5373\u4fbf\u5728\u5f53\u524d\u6700\u65b0\u7684k8s 1.5.1\u7248\u672c\u4e2d\uff0c\u5b83\u4ecd\u7136\u5904\u4e8eAlpha\u72b6\u6001\uff0c\u5b98\u65b9\u4e0d\u5efa\u8bae\u5728Production\u73af\u5883\u4e0b\u4f7f\u7528\u3002\u6bcf\u6b21\u6267\u884ckubeadm init\u65f6\uff0c\u5b83\u90fd\u4f1a\u6253\u5370\u5982\u4e0b\u63d0\u9192\u65e5\u5fd7\uff1a<\/p>\n

[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.<\/p>\n

 <\/p>\n

\u4e0d\u8fc7\u7531\u4e8e\u4e4b\u524d\u90e8\u7f72\u7684k8s 1.3.7\u96c6\u7fa4<\/a>\u8fd0\u884c\u826f\u597d\uff0c\u8fd9\u7ed9\u4e86\u6211\u4eec\u5728k8s\u8fd9\u6761\u8def\u4e0a\u7ee7\u7eed\u8d70\u4e0b\u53bb\u5e76\u8d70\u597d\u7684\u4fe1\u5fc3\u3002\u4f46k8s\u5728\u90e8\u7f72\u548c\u7ba1\u7406\u65b9\u9762\u7684\u4f53\u9a8c\u7684\u786e\u662f\u592a\u7e41\u7410\u4e86\uff0c\u4e8e\u662f\u6211\u4eec\u51c6\u5907\u8bd5\u9a8c\u4e00\u4e0bkubeadm\u662f\u5426\u80fd\u5e26\u7ed9\u6211\u4eec\u8d85\u51fa\u9884\u671f\u7684\u4f53\u9a8c\u3002\u4e4b\u524d\u5728aliyun ubuntu 14.04\u4e0a\u5b89\u88c5kubernetes 1.3.7\u7684\u7ecf\u9a8c\u548c\u6559\u8bad\uff0c\u8ba9\u6211\u7565\u5fae\u6709\u90a3\u4e48\u4e00\u4e22\u4e22\u5e95\u6c14\uff0c\u4f46\u5b9e\u9645\u5b89\u88c5\u8fc7\u7a0b\u4f9d\u65e7\u662f\u4e00\u6ce2\u4e09\u6298\u3002\u8fd9\u65e2\u4e0ekubeadm\u7684unstable\u6709\u5173\uff0c\u540c\u6837\u4e5f\u4e0ecni<\/a>\u3001\u7b2c\u4e09\u65b9\u7f51\u7edcadd-ons\u7684\u8d28\u91cf\u6709\u5173\u3002\u65e0\u8bba\u54ea\u4e00\u65b9\u51fa\u73b0\u95ee\u9898\u90fd\u4f1a\u8ba9\u4f60\u7684install\u8fc7\u7a0b\u5f02\u5e38\u574e\u5777\u66f2\u6298\u3002<\/p>\n

\u4e00\u3001\u73af\u5883\u4e0e\u7ea6\u675f<\/strong><\/p>\n

\u5728kubeadm\u652f\u6301\u7684Ubuntu 16.04+, CentOS 7 or HypriotOS v1.0.1+\u4e09\u79cd\u64cd\u4f5c\u7cfb\u7edf\u4e2d\uff0c\u6211\u4eec\u9009\u62e9\u4e86Ubuntu 16.04\u3002\u7531\u4e8e\u963f\u91cc\u4e91\u5c1a\u65e0\u5b98\u65b916.04 Image\u53ef\u7528\uff0c\u6211\u4eec\u65b0\u5f00\u4e86\u4e24\u4e2aUbuntu 14.04ECS\u5b9e\u4f8b\uff0c\u5e76\u901a\u8fc7apt-get\u547d\u4ee4\u624b\u5de5\u5c06\u5176\u5347\u7ea7\u5230Ubuntu 16.04.1\uff0c\u8be6\u7ec6\u7248\u672c\u662f\uff1aUbuntu 16.04.1 LTS (GNU\/Linux 4.4.0-58-generic x86_64)\u3002<\/p>\n

Ubuntu 16.04\u4f7f\u7528\u4e86systemd<\/a>\u4f5c\u4e3ainit system\uff0c\u5728\u5b89\u88c5\u548c\u914d\u7f6eDocker\u65f6\uff0c\u53ef\u4ee5\u53c2\u8003\u6211\u7684\u8fd9\u7bc7\u300a\u5f53Docker\u9047\u5230system<\/a>\u300b\u3002Docker\u7248\u672c\u6211\u9009\u62e9\u4e86\u76ee\u524d\u53ef\u4ee5\u5f97\u5230\u7684lastest stable release: 1.12.5\u3002<\/p>\n

# docker version<\/p>\n

Client:<\/p>\n

Version:\u00a0\u00a0\u00a0\u00a0\u00a0 1.12.5<\/p>\n

API version:\u00a0 1.24<\/p>\n

Go version:\u00a0\u00a0 go1.6.4<\/p>\n

Git commit:\u00a0\u00a0 7392c3b<\/p>\n

Built:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Fri Dec 16 02:42:17 2016<\/p>\n

OS\/Arch:\u00a0\u00a0\u00a0\u00a0\u00a0 linux\/amd64<\/p>\n

 <\/p>\n

Server:<\/p>\n

Version:\u00a0\u00a0\u00a0\u00a0\u00a0 1.12.5<\/p>\n

API version:\u00a0 1.24<\/p>\n

Go version:\u00a0\u00a0 go1.6.4<\/p>\n

Git commit:\u00a0\u00a0 7392c3b<\/p>\n

Built:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Fri Dec 16 02:42:17 2016<\/p>\n

OS\/Arch:\u00a0\u00a0\u00a0\u00a0\u00a0 linux\/amd64<\/p>\n

 <\/p>\n

\u81f3\u4e8eKubernetes\u7248\u672c\uff0c\u524d\u9762\u5df2\u7ecf\u63d0\u5230\u8fc7\u4e86\uff0c\u6211\u4eec\u5c31\u4f7f\u7528\u6700\u65b0\u53d1\u5e03\u7684Kubernetes 1.5.1\u7248\u672c\u30021.5.1\u662f1.5.0<\/a>\u7684\u4e00\u4e2a\u7d27\u6025fix\u7248\u672c<\/a>\uff0c\u4e3b\u8981\u201dto address default flag values which in isolation were not problematic, but in concert could result in an insecure cluster\u201d\u3002\u5b98\u65b9\u5efa\u8baeskip 1.5.0\uff0c\u76f4\u63a5\u75281.5.1\u3002<\/p>\n

\u8fd9\u91cc\u518d\u91cd\u7533\u4e00\u4e0b\uff1aKubernetes\u7684\u5b89\u88c5\u3001\u914d\u7f6e\u548c\u8c03\u901a\u662f\u5f88\u96be\u7684\uff0c\u5728\u963f\u91cc\u4e91\u4e0a\u8c03\u901a\u5c31\u66f4\u96be\u4e86\uff0c\u6709\u65f6\u8fd8\u9700\u8981\u4e9b\u8fd0\u6c14\u3002Kubernetes\u3001Docker\u3001cni\u4ee5\u53ca\u5404\u79cd\u7f51\u7edcAdd-ons\u90fd\u5728active development\u4e2d\uff0c\u4e5f\u8bb8\u4eca\u5929\u8fd8\u597d\u7528\u7684step\u3001tip\u548ctrick\uff0c\u660e\u5929\u5c31out-dated\uff0c\u56e0\u6b64\u5728\u501f\u9274\u672c\u6587\u7684\u64cd\u4f5c\u6b65\u9aa4\u65f6\uff0c\u8bf7\u8c28\u8bb0\u8fd9\u4e9b^0^\u3002<\/p>\n

\u4e8c\u3001\u5b89\u88c5\u5305\u51c6\u5907<\/strong><\/p>\n

\u6211\u4eec\u8fd9\u6b21\u65b0\u5f00\u4e86\u4e24\u4e2aECS\u5b9e\u4f8b\uff0c\u4e00\u4e2a\u4f5c\u4e3amaster node\uff0c\u4e00\u4e2a\u4f5c\u4e3aminion node\u3002Kubeadm\u9ed8\u8ba4\u5b89\u88c5\u65f6\uff0cmaster node\u5c06\u4e0d\u4f1a\u53c2\u4e0ePod\u8c03\u5ea6\uff0c\u4e0d\u4f1a\u627f\u8f7dwork load\uff0c\u5373\u4e0d\u4f1a\u6709\u975e\u6838\u5fc3\u7ec4\u4ef6\u7684Pod\u5728Master node\u4e0a\u88ab\u521b\u5efa\u51fa\u6765\u3002\u5f53\u7136\u901a\u8fc7kubectl taint\u547d\u4ee4\u53ef\u4ee5\u89e3\u9664\u8fd9\u4e00\u9650\u5236\uff0c\u4e0d\u8fc7\u8fd9\u662f\u540e\u8bdd\u4e86\u3002<\/p>\n

\u96c6\u7fa4\u62d3\u6251\uff1a<\/p>\n

master node\uff1a10.47.217.91\uff0c\u4e3b\u673a\u540d\uff1aiZ25beglnhtZ<\/p>\n

minion node\uff1a10.28.61.30\uff0c\u4e3b\u673a\u540d\uff1aiZ2ze39jeyizepdxhwqci6Z<\/p>\n

\u672c\u6b21\u5b89\u88c5\u7684\u4e3b\u53c2\u8003\u6587\u6863\u5c31\u662fKubernetes\u5b98\u65b9\u7684\u90a3\u7bc7\u300aInstalling Kubernetes on Linux with kubeadm<\/a>\u300b\u3002<\/p>\n

\u672c\u5c0f\u8282\uff0c\u6211\u4eec\u5c06\u8fdb\u884c\u5b89\u88c5\u5305\u51c6\u5907\uff0c\u5373\u5c06kubeadm\u4ee5\u53ca\u6b64\u6b21\u5b89\u88c5\u6240\u9700\u8981\u7684k8s\u6838\u5fc3\u7ec4\u4ef6\u7edf\u7edf\u4e0b\u8f7d\u5230\u4e0a\u8ff0\u4e24\u4e2aNode\u4e0a\u3002\u6ce8\u610f\uff1a\u5982\u679c\u4f60\u6709\u52a0\u901f\u5668\uff0c\u90a3\u4e48\u672c\u8282\u4e0b\u9762\u7684\u5b89\u88c5\u8fc7\u7a0b\u5c06\u5c24\u4e3a\u987a\u5229\uff0c\u53cd\u4e4b\uff0c\u2026\u00a0\u00a0\u3002\u4ee5\u4e0b\u547d\u4ee4\uff0c\u5728\u4e24\u4e2aNode\u4e0a\u5747\u8981\u6267\u884c\u3002<\/p>\n

1<\/strong>\u3001\u6dfb\u52a0<\/strong>apt-key<\/strong><\/p>\n

# curl -s https:\/\/packages.cloud.google.com\/apt\/doc\/apt-key.gpg | apt-key add –<\/p>\n

OK<\/p>\n

2<\/strong>\u3001\u6dfb\u52a0<\/strong>Kubernetes<\/strong>\u6e90\u5e76\u66f4\u65b0\u5305\u4fe1\u606f<\/strong><\/p>\n

\u6dfb\u52a0Kubernetes\u6e90\u5230sources.list.d\u76ee\u5f55\u4e0b\uff1a<\/p>\n

# cat <<EOF > \/etc\/apt\/sources.list.d\/kubernetes.list<\/p>\n

deb http:\/\/apt.kubernetes.io\/ kubernetes-xenial main<\/p>\n

EOF<\/p>\n

 <\/p>\n

# cat \/etc\/apt\/sources.list.d\/kubernetes.list<\/p>\n

deb http:\/\/apt.kubernetes.io\/ kubernetes-xenial main<\/p>\n

\u66f4\u65b0\u5305\u4fe1\u606f\uff1a<\/p>\n

# apt-get update<\/p>\n

… …<\/p>\n

Hit:2 http:\/\/mirrors.aliyun.com\/ubuntu xenial InRelease<\/p>\n

Hit:3 https:\/\/apt.dockerproject.org\/repo ubuntu-xenial InRelease<\/p>\n

Get:4 http:\/\/mirrors.aliyun.com\/ubuntu xenial-security InRelease [102 kB]<\/p>\n

Get:1 https:\/\/packages.cloud.google.com\/apt kubernetes-xenial InRelease [6,299 B]<\/p>\n

Get:5 https:\/\/packages.cloud.google.com\/apt kubernetes-xenial\/main amd64 Packages [1,739 B]<\/p>\n

Get:6 http:\/\/mirrors.aliyun.com\/ubuntu xenial-updates InRelease [102 kB]<\/p>\n

Get:7 http:\/\/mirrors.aliyun.com\/ubuntu xenial-proposed InRelease [253 kB]<\/p>\n

Get:8 http:\/\/mirrors.aliyun.com\/ubuntu xenial-backports InRelease [102 kB]<\/p>\n

Fetched 568 kB in 19s (28.4 kB\/s)<\/p>\n

Reading package lists… Done<\/p>\n

3<\/strong>\u3001\u4e0b\u8f7d<\/strong>Kubernetes<\/strong>\u6838\u5fc3\u7ec4\u4ef6<\/strong><\/p>\n

\u5728\u6b64\u6b21\u5b89\u88c5\u4e2d\uff0c\u6211\u4eec\u901a\u8fc7apt-get\u5c31\u53ef\u4ee5\u4e0b\u8f7dKubernetes\u7684\u6838\u5fc3\u7ec4\u4ef6\uff0c\u5305\u62eckubelet\u3001kubeadm\u3001kubectl\u548ckubernetes-cni\u7b49\u3002<\/p>\n

# apt-get install -y kubelet kubeadm kubectl kubernetes-cni<\/p>\n

Reading package lists… Done<\/p>\n

Building dependency tree<\/p>\n

Reading state information… Done<\/p>\n

The following package was automatically installed and is no longer required:<\/p>\n

libtimedate-perl<\/p>\n

Use ‘apt autoremove’ to remove it.<\/p>\n

The following additional packages will be installed:<\/p>\n

ebtables ethtool socat<\/p>\n

The following NEW packages will be installed:<\/p>\n

ebtables ethtool kubeadm kubectl kubelet kubernetes-cni socat<\/p>\n

0 upgraded, 7 newly installed, 0 to remove and 0 not upgraded.<\/p>\n

Need to get 37.6 MB of archives.<\/p>\n

After this operation, 261 MB of additional disk space will be used.<\/p>\n

Get:2 http:\/\/mirrors.aliyun.com\/ubuntu xenial\/main amd64 ebtables amd64 2.0.10.4-3.4ubuntu1 [79.6 kB]<\/p>\n

Get:6 http:\/\/mirrors.aliyun.com\/ubuntu xenial\/main amd64 ethtool amd64 1:4.5-1 [97.5 kB]<\/p>\n

Get:7 http:\/\/mirrors.aliyun.com\/ubuntu xenial\/universe amd64 socat amd64 1.7.3.1-1 [321 kB]<\/p>\n

Get:1 https:\/\/packages.cloud.google.com\/apt kubernetes-xenial\/main amd64 kubernetes-cni amd64 0.3.0.1-07a8a2-00 [6,877 kB]<\/p>\n

Get:3 https:\/\/packages.cloud.google.com\/apt kubernetes-xenial\/main amd64 kubelet amd64 1.5.1-00 [15.1 MB]<\/p>\n

Get:4 https:\/\/packages.cloud.google.com\/apt kubernetes-xenial\/main amd64 kubectl amd64 1.5.1-00 [7,954 kB]<\/p>\n

Get:5 https:\/\/packages.cloud.google.com\/apt kubernetes-xenial\/main amd64 kubeadm amd64 1.6.0-alpha.0-2074-a092d8e0f95f52-00 [7,120 kB]<\/p>\n

Fetched 37.6 MB in 36s (1,026 kB\/s)<\/p>\n

… …<\/p>\n

Unpacking kubeadm (1.6.0-alpha.0-2074-a092d8e0f95f52-00) …<\/p>\n

Processing triggers for systemd (229-4ubuntu13) …<\/p>\n

Processing triggers for ureadahead (0.100.0-19) …<\/p>\n

Processing triggers for man-db (2.7.5-1) …<\/p>\n

Setting up ebtables (2.0.10.4-3.4ubuntu1) …<\/p>\n

update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults<\/p>\n

Setting up ethtool (1:4.5-1) …<\/p>\n

Setting up kubernetes-cni (0.3.0.1-07a8a2-00) …<\/p>\n

Setting up socat (1.7.3.1-1) …<\/p>\n

Setting up kubelet (1.5.1-00) …<\/p>\n

Setting up kubectl (1.5.1-00) …<\/p>\n

Setting up kubeadm (1.6.0-alpha.0-2074-a092d8e0f95f52-00) …<\/p>\n

Processing triggers for systemd (229-4ubuntu13) …<\/p>\n

Processing triggers for ureadahead (0.100.0-19) …<\/p>\n

… …<\/p>\n

\u4e0b\u8f7d\u540e\u7684kube\u7ec4\u4ef6\u5e76\u672a\u81ea\u52a8\u8fd0\u884c\u8d77\u6765\u3002\u5728 \/lib\/systemd\/system\u4e0b\u9762\u6211\u4eec\u80fd\u770b\u5230kubelet.service\uff1a<\/p>\n

# ls \/lib\/systemd\/system|grep kube<\/p>\n

kubelet.service<\/p>\n

 <\/p>\n

\/\/kubelet.service<\/p>\n

[Unit]<\/p>\n

Description=kubelet: The Kubernetes Node Agent<\/p>\n

Documentation=http:\/\/kubernetes.io\/docs\/<\/p>\n

 <\/p>\n

[Service]<\/p>\n

ExecStart=\/usr\/bin\/kubelet<\/p>\n

Restart=always<\/p>\n

StartLimitInterval=0<\/p>\n

RestartSec=10<\/p>\n

 <\/p>\n

[Install]<\/p>\n

WantedBy=multi-user.target<\/p>\n

kubelet\u7684\u7248\u672c\uff1a<\/p>\n

# kubelet –version<\/p>\n

Kubernetes v1.5.1<\/p>\n

k8s\u7684\u6838\u5fc3\u7ec4\u4ef6\u90fd\u6709\u4e86\uff0c\u63a5\u4e0b\u6765\u6211\u4eec\u5c31\u8981boostrap kubernetes cluster\u4e86\u3002\u540c\u65f6\uff0c\u95ee\u9898\u4e5f\u5c31\u968f\u4e4b\u800c\u6765\u4e86\uff0c\u800c\u8fd9\u4e9b\u95ee\u9898\u4ee5\u53ca\u95ee\u9898\u7684\u89e3\u51b3\u624d\u662f\u672c\u7bc7\u8981\u8bf4\u660e\u7684\u91cd\u70b9\u3002<\/p>\n

\u4e09\u3001\u521d\u59cb\u5316\u96c6\u7fa4<\/strong><\/p>\n

\u524d\u9762\u8bf4\u8fc7\uff0c\u7406\u8bba\u4e0a\u901a\u8fc7kubeadm\u4f7f\u7528init\u548cjoin\u547d\u4ee4\u5373\u53ef\u5efa\u7acb\u4e00\u4e2a\u96c6\u7fa4\uff0c\u8fd9init\u5c31\u662f\u5728master\u8282\u70b9\u5bf9\u96c6\u7fa4\u8fdb\u884c\u521d\u59cb\u5316\u3002\u548ck8s 1.4\u4e4b\u524d\u7684\u90e8\u7f72\u65b9\u5f0f\u4e0d\u540c\u7684\u662f\uff0ckubeadm\u5b89\u88c5\u7684k8s\u6838\u5fc3\u7ec4\u4ef6\u90fd\u662f\u4ee5\u5bb9\u5668\u7684\u5f62\u5f0f\u8fd0\u884c\u4e8emaster node\u4e0a\u7684\u3002\u56e0\u6b64\u5728kubeadm init\u4e4b\u524d\uff0c\u6700\u597d\u7ed9master node\u4e0a\u7684docker engine\u6302\u4e0a\u52a0\u901f\u5668\u4ee3\u7406\uff0c\u56e0\u4e3akubeadm\u8981\u4ecegcr.io\/google_containers repository\u4e2dpull\u8bb8\u591a\u6838\u5fc3\u7ec4\u4ef6\u7684images\uff0c\u5927\u7ea6\u6709\u5982\u4e0b\u4e00\u4e9b\uff1a<\/p>\n

gcr.io\/google_containers\/kube-controller-manager-amd64\u00a0\u00a0 v1.5.1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 cd5684031720\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2 weeks ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 102.4 MB<\/p>\n

gcr.io\/google_containers\/kube-apiserver-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 v1.5.1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 8c12509df629\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2 weeks ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 124.1 MB<\/p>\n

gcr.io\/google_containers\/kube-proxy-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 v1.5.1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 71d2b27b03f6\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2 weeks ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 175.6 MB<\/p>\n

gcr.io\/google_containers\/kube-scheduler-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 v1.5.1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 6506e7b74dac\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2 weeks ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 53.97 MB<\/p>\n

gcr.io\/google_containers\/etcd-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 3.0.14-kubeadm\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 856e39ac7be3\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 5 weeks ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 174.9 MB<\/p>\n

gcr.io\/google_containers\/kubedns-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1.9\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 26cf1ed9b144\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 5 weeks ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a047 MB<\/p>\n

gcr.io\/google_containers\/dnsmasq-metrics-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1.0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 5271aabced07\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 7 weeks ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 14 MB<\/p>\n

gcr.io\/google_containers\/kube-dnsmasq-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1.4\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 3ec65756a89b\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 3 months ago\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a05.13 MB<\/p>\n

gcr.io\/google_containers\/kube-discovery-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1.0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 c5e0c9a457fc\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 3 months ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 134.2 MB<\/p>\n

gcr.io\/google_containers\/exechealthz-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1.2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 93a43bfb39bf\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 3 months ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 8.375 MB<\/p>\n

gcr.io\/google_containers\/pause-amd64\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 3.0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 99e59f495ffa\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 7 months ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 746.9 kB<\/p>\n

\u5728Kubeadm\u7684\u6587\u6863\u4e2d\uff0cPod Network\u7684\u5b89\u88c5\u662f\u4f5c\u4e3a\u4e00\u4e2a\u5355\u72ec\u7684\u6b65\u9aa4\u7684\u3002kubeadm init\u5e76\u6ca1\u6709\u4e3a\u4f60\u9009\u62e9\u4e00\u4e2a\u9ed8\u8ba4\u7684Pod network\u8fdb\u884c\u5b89\u88c5\u3002\u6211\u4eec\u5c06\u9996\u9009Flannel<\/a>\u00a0\u4f5c\u4e3a\u6211\u4eec\u7684Pod network\uff0c\u8fd9\u4e0d\u4ec5\u662f\u56e0\u4e3a\u6211\u4eec\u7684\u4e0a\u4e00\u4e2a\u96c6\u7fa4\u7528\u7684\u5c31\u662fflannel\uff0c\u800c\u4e14\u8868\u73b0\u7a33\u5b9a\u3002\u66f4\u662f\u7531\u4e8eFlannel\u5c31\u662fcoreos\u4e3ak8s\u6253\u9020\u7684\u4e13\u5c5eoverlay network add-ons\u3002\u751a\u81f3\u4e8eflannel repository\u7684readme.md\u90fd\u8fd9\u6837\u5199\u7740\uff1a\u201cflannel is a network fabric for containers, designed for Kubernetes\u201d\u3002\u5982\u679c\u6211\u4eec\u8981\u4f7f\u7528Flannel\uff0c\u90a3\u4e48\u5728\u6267\u884cinit\u65f6\uff0c\u6309\u7167kubeadm\u6587\u6863\u8981\u6c42\uff0c\u6211\u4eec\u5fc5\u987b\u7ed9init\u547d\u4ee4\u5e26\u4e0aoption\uff1a\u2013pod-network-cidr=10.244.0.0\/16\u3002<\/p>\n

1<\/strong>\u3001\u6267\u884c<\/strong>kubeadm init<\/strong><\/p>\n

\u6267\u884ckubeadm init\u547d\u4ee4\uff1a<\/p>\n

# kubeadm init –pod-network-cidr=10.244.0.0\/16<\/p>\n

[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.<\/p>\n

[preflight] Running pre-flight checks<\/p>\n

[preflight] Starting the kubelet service<\/p>\n

[init] Using Kubernetes version: v1.5.1<\/p>\n

[tokens] Generated token: “2e7da9.7fc5668ff26430c7”<\/p>\n

[certificates] Generated Certificate Authority key and certificate.<\/p>\n

[certificates] Generated API Server key and certificate<\/p>\n

[certificates] Generated Service Account signing keys<\/p>\n

[certificates] Created keys and certificates in “\/etc\/kubernetes\/pki”<\/p>\n

[kubeconfig] Wrote KubeConfig file to disk: “\/etc\/kubernetes\/kubelet.conf”<\/p>\n

[kubeconfig] Wrote KubeConfig file to disk: “\/etc\/kubernetes\/admin.conf”<\/p>\n

[apiclient] Created API client, waiting for the control plane to become ready \/\/\u5982\u679c\u6ca1\u6709\u6302\u52a0\u901f\u5668\uff0c\u53ef\u80fd\u4f1a\u5728\u8fd9\u91cchang\u4f4f\u3002<\/p>\n

[apiclient] All control plane components are healthy after 54.789750 seconds<\/p>\n

[apiclient] Waiting for at least one node to register and become ready<\/p>\n

[apiclient] First node is ready after 1.003053 seconds<\/p>\n

[apiclient] Creating a test deployment<\/p>\n

[apiclient] Test deployment succeeded<\/p>\n

[token-discovery] Created the kube-discovery deployment, waiting for it to become ready<\/p>\n

[token-discovery] kube-discovery is ready after 62.503441 seconds<\/p>\n

[addons] Created essential addon: kube-proxy<\/p>\n

[addons] Created essential addon: kube-dns<\/p>\n

 <\/p>\n

Your Kubernetes master has initialized successfully!<\/p>\n

 <\/p>\n

You should now deploy a pod network to the cluster.<\/p>\n

Run “kubectl apply -f [podnetwork].yaml” with one of the options listed at:<\/p>\n

 <\/p>\n

http:\/\/kubernetes.io\/docs\/admin\/addons\/<\/p>\n

 <\/p>\n

You can now join any number of machines by running the following on each node:<\/p>\n

 <\/p>\n

kubeadm join –token=2e7da9.7fc5668ff26430c7 123.56.200.187<\/p>\n

 <\/p>\n

init\u6210\u529f\u540e\u7684master node\u6709\u5565\u53d8\u5316\uff1fk8s\u7684\u6838\u5fc3\u7ec4\u4ef6\u5747\u6b63\u5e38\u542f\u52a8\uff1a<\/p>\n

# ps -ef|grep kube<\/p>\n

root\u00a0\u00a0\u00a0\u00a0\u00a0 2477\u00a0 2461\u00a0 1 16:36 ?\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 00:00:04 kube-proxy –kubeconfig=\/run\/kubeconfig<\/p>\n

root\u00a0\u00a0\u00a0\u00a0 30860\u00a0\u00a0\u00a0\u00a0 1 12 16:33 ?\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 00:01:09 \/usr\/bin\/kubelet –kubeconfig=\/etc\/kubernetes\/kubelet.conf –require-kubeconfig=true –pod-manifest-path=\/etc\/kubernetes\/manifests –allow-privileged=true –network-plugin=cni –cni-conf-dir=\/etc\/cni\/net.d –cni-bin-dir=\/opt\/cni\/bin –cluster-dns=10.96.0.10 –cluster-domain=cluster.local<\/p>\n

root\u00a0\u00a0\u00a0\u00a0 30952 30933 \u00a00 16:33 ?\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 00:00:01 kube-scheduler –address=127.0.0.1 –leader-elect –master=127.0.0.1:8080<\/p>\n

root\u00a0\u00a0\u00a0\u00a0 31128 31103\u00a0 2 16:33 ?\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 00:00:11 kube-controller-manager –address=127.0.0.1 –leader-elect –master=127.0.0.1:8080 –cluster-name=kubernetes –root-ca-file=\/etc\/kubernetes\/pki\/ca.pem –service-account-private-key-file=\/etc\/kubernetes\/pki\/apiserver-key.pem –cluster-signing-cert-file=\/etc\/kubernetes\/pki\/ca.pem –cluster-signing-key-file=\/etc\/kubernetes\/pki\/ca-key.pem –insecure-experimental-approve-all-kubelet-csrs-for-group=system:kubelet-bootstrap –allocate-node-cidrs=true –cluster-cidr=10.244.0.0\/16<\/p>\n

root\u00a0\u00a0\u00a0\u00a0 31223 31207\u00a0 2 16:34 ?\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 00:00:10 kube-apiserver –insecure-bind-address=127.0.0.1 –admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota –service-cluster-ip-range=10.96.0.0\/12 –service-account-key-file=\/etc\/kubernetes\/pki\/apiserver-key.pem –client-ca-file=\/etc\/kubernetes\/pki\/ca.pem –tls-cert-file=\/etc\/kubernetes\/pki\/apiserver.pem –tls-private-key-file=\/etc\/kubernetes\/pki\/apiserver-key.pem –token-auth-file=\/etc\/kubernetes\/pki\/tokens.csv –secure-port=6443 –allow-privileged –advertise-address=123.56.200.187 –kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname –anonymous-auth=false –etcd-servers=http:\/\/127.0.0.1:2379<\/p>\n

root\u00a0\u00a0\u00a0\u00a0 31491 31475\u00a0 0 16:35 ?\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 00:00:00 \/usr\/local\/bin\/kube-discovery<\/p>\n

\u800c\u4e14\u662f\u591a\u4ee5container\u7684\u5f62\u5f0f\u542f\u52a8\uff1a<\/p>\n

# docker ps<\/p>\n

CONTAINER ID\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 IMAGE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0COMMAND\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 CREATED\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 STATUS\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 PORTS\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 NAMES<\/p>\n

c16c442b7eca\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/kube-proxy-amd64:v1.5.1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 “kube-proxy –kubecon”\u00a0\u00a0 6 minutes ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Up 6 minutes\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_kube-proxy.36dab4e8_kube-proxy-sb4sm_kube-system_43fb1a2c-cb46-11e6-ad8f-00163e1001d7_2ba1648e<\/p>\n

9f73998e01d7\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/kube-discovery-amd64:1.0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 “\/usr\/local\/bin\/kube-”\u00a0\u00a0 8 minutes ago\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Up 8 minutes\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_kube-discovery.7130cb0a_kube-discovery-1769846148-6z5pw_kube-system_1eb97044-cb46-11e6-ad8f-00163e1001d7_fd49c2e3<\/p>\n

dd5412e5e15c\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/kube-apiserver-amd64:v1.5.1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 “kube-apiserver –ins”\u00a0\u00a0 9 minutes ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Up 9 minutes\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_kube-apiserver.1c5a91d9_kube-apiserver-iz25beglnhtz_kube-system_eea8df1717e9fea18d266103f9edfac3_8cae8485<\/p>\n

60017f8819b2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/etcd-amd64:3.0.14-kubeadm\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 “etcd –listen-client”\u00a0\u00a0 9 minutes ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Up 9 minutes\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_etcd.c323986f_etcd-iz25beglnhtz_kube-system_3a26566bb004c61cd05382212e3f978f_06d517eb<\/p>\n

03c2463aba9c\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/kube-controller-manager-amd64:v1.5.1\u00a0\u00a0 “kube-controller-mana”\u00a0\u00a0 9 minutes ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Up 9 minutes\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_kube-controller-manager.d30350e1_kube-controller-manager-iz25beglnhtz_kube-system_9a40791dd1642ea35c8d95c9e610e6c1_3b05cb8a<\/p>\n

fb9a724540a7\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/kube-scheduler-amd64:v1.5.1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 “kube-scheduler –add”\u00a0\u00a0 9 minutes ago\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Up 9 minutes\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_kube-scheduler.ef325714_kube-scheduler-iz25beglnhtz_kube-system_dc58861a0991f940b0834f8a110815cb_9b3ccda2<\/p>\n

…. …<\/p>\n

\u4e0d\u8fc7\u8fd9\u4e9b\u6838\u5fc3\u7ec4\u4ef6\u5e76\u4e0d\u662f\u8dd1\u5728pod network\u4e2d\u7684\uff08\u6ca1\u9519\uff0c\u6b64\u65f6\u7684pod network\u8fd8\u6ca1\u6709\u521b\u5efa\uff09\uff0c\u800c\u662f\u91c7\u7528\u4e86host network\u3002\u4ee5kube-apiserver\u7684pod\u4fe1\u606f\u4e3a\u4f8b\uff1a<\/p>\n

kube-system\u00a0\u00a0 kube-apiserver-iz25beglnhtz\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1h\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 10.47.217.91\u00a0\u00a0 iz25beglnhtz<\/p>\n

kube-apiserver\u7684IP\u662fhost ip\uff0c\u4ece\u800c\u63a8\u65ad\u5bb9\u5668\u4f7f\u7528\u7684\u662fhost\u7f51\u7edc\uff0c\u8fd9\u4ece\u5176\u5bf9\u5e94\u7684pause\u5bb9\u5668\u7684network\u5c5e\u6027\u5c31\u53ef\u4ee5\u770b\u51fa\uff1a<\/p>\n

# docker ps |grep apiserver<\/p>\n

a5a76bc59e38\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/kube-apiserver-amd64:v1.5.1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 “kube-apiserver –ins”\u00a0\u00a0 About an hour ago\u00a0\u00a0 Up About an hour\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_kube-apiserver.2529402_kube-apiserver-iz25beglnhtz_kube-system_25d646be9a0092138dc6088fae6f1656_ec0079fc<\/p>\n

ef4d3bf057a6\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/pause-amd64:3.0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 “\/pause”\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 About an hour ago\u00a0\u00a0 Up About an hour\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_POD.d8dbe16c_kube-apiserver-iz25beglnhtz_kube-system_25d646be9a0092138dc6088fae6f1656_bbfd8a31<\/p>\n

 <\/p>\n

inspect pause\u5bb9\u5668\uff0c\u53ef\u4ee5\u770b\u5230pause container\u7684NetworkMode\u7684\u503c\uff1a<\/p>\n

“NetworkMode”: “host”,<\/p>\n

\u5982\u679ckubeadm init\u6267\u884c\u8fc7\u7a0b\u4e2d\u9014\u51fa\u73b0\u4e86\u4ec0\u4e48\u95ee\u9898\uff0c\u6bd4\u5982\u524d\u671f\u5fd8\u8bb0\u6302\u52a0\u901f\u5668\u5bfc\u81f4init hang\u4f4f\uff0c\u4f60\u53ef\u80fd\u4f1actrl+c\u9000\u51fainit\u6267\u884c\u3002\u91cd\u65b0\u914d\u7f6e\u540e\uff0c\u518d\u6267\u884ckubeadm init\uff0c\u8fd9\u65f6\u4f60\u53ef\u80fd\u4f1a\u9047\u5230\u4e0b\u9762kubeadm\u7684\u8f93\u51fa\uff1a<\/p>\n

# kubeadm init –pod-network-cidr=10.244.0.0\/16<\/p>\n

[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.<\/p>\n

[preflight] Running pre-flight checks<\/p>\n

[preflight] Some fatal errors occurred:<\/p>\n

Port 10250 is in use<\/p>\n

\/etc\/kubernetes\/manifests is not empty<\/p>\n

\/etc\/kubernetes\/pki is not empty<\/p>\n

\/var\/lib\/kubelet is not empty<\/p>\n

\/etc\/kubernetes\/admin.conf already exists<\/p>\n

\/etc\/kubernetes\/kubelet.conf already exists<\/p>\n

[preflight] If you know what you are doing, you can skip pre-flight checks with `–skip-preflight-checks`<\/p>\n

kubeadm\u4f1a\u81ea\u52a8\u68c0\u67e5\u5f53\u524d\u73af\u5883\u662f\u5426\u6709\u4e0a\u6b21\u547d\u4ee4\u6267\u884c\u7684\u201c\u6b8b\u7559\u201d\u3002\u5982\u679c\u6709\uff0c\u5fc5\u987b\u6e05\u7406\u540e\u518d\u884c\u6267\u884cinit\u3002\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u201dkubeadm reset\u201d\u6765\u6e05\u7406\u73af\u5883\uff0c\u4ee5\u5907\u91cd\u6765\u3002<\/p>\n

# kubeadm reset<\/p>\n

[preflight] Running pre-flight checks<\/p>\n

[reset] Draining node: “iz25beglnhtz”<\/p>\n

[reset] Removing node: “iz25beglnhtz”<\/p>\n

[reset] Stopping the kubelet service<\/p>\n

[reset] Unmounting mounted directories in “\/var\/lib\/kubelet”<\/p>\n

[reset] Removing kubernetes-managed containers<\/p>\n

[reset] Deleting contents of stateful directories: [\/var\/lib\/kubelet \/etc\/cni\/net.d \/var\/lib\/etcd]<\/p>\n

[reset] Deleting contents of config directories: [\/etc\/kubernetes\/manifests \/etc\/kubernetes\/pki]<\/p>\n

[reset] Deleting files: [\/etc\/kubernetes\/admin.conf \/etc\/kubernetes\/kubelet.conf]<\/p>\n

 <\/p>\n

2<\/strong>\u3001\u5b89\u88c5<\/strong>flannel pod<\/strong>\u7f51\u7edc<\/strong><\/p>\n

kubeadm init\u4e4b\u540e\uff0c\u5982\u679c\u4f60\u63a2\u7d22\u4e00\u4e0b\u5f53\u524dcluster\u7684\u72b6\u6001\u6216\u8005\u6838\u5fc3\u7ec4\u4ef6\u7684\u65e5\u5fd7\uff0c\u4f60\u4f1a\u53d1\u73b0\u67d0\u4e9b\u201c\u5f02\u5e38\u201d\uff0c\u6bd4\u5982\uff1a\u4ecekubelet\u7684\u65e5\u5fd7\u4e2d\u6211\u4eec\u53ef\u4ee5\u770b\u5230\u4e00\u76f4\u5237\u5c4f\u7684\u9519\u8bef\u4fe1\u606f\uff1a<\/p>\n

Dec 26 16:36:48 iZ25beglnhtZ kubelet[30860]: E1226 16:36:48.365885\u00a0\u00a0 30860 docker_manager.go:2201] Failed to setup network for pod “kube-dns-2924299975-pddz5_kube-system(43fd7264-cb46-11e6-ad8f-00163e1001d7)” using network plugins “cni”: cni config unintialized; Skipping pod<\/p>\n

\u901a\u8fc7\u547d\u4ee4kubectl get pod \u2013all-namespaces -o wide\uff0c\u4f60\u4e5f\u4f1a\u53d1\u73b0kube-dns pod\u5904\u4e8eContainerCreating\u72b6\u6001\u3002<\/p>\n

\u8fd9\u4e9b\u90fd\u4e0d\u6253\u7d27\uff0c\u56e0\u4e3a\u6211\u4eec\u8fd8\u6ca1\u6709\u4e3acluster\u5b89\u88c5Pod network\u5462\u3002\u524d\u9762\u8bf4\u8fc7\uff0c\u6211\u4eec\u8981\u4f7f\u7528Flannel\u7f51\u7edc\uff0c\u56e0\u6b64\u6211\u4eec\u9700\u8981\u6267\u884c\u5982\u4e0b\u5b89\u88c5\u547d\u4ee4\uff1a<\/p>\n

#kubectl apply -f\u00a0 https:\/\/raw.githubusercontent.com\/coreos\/flannel\/master\/Documentation\/kube-flannel.yml<\/p>\n

configmap “kube-flannel-cfg” created<\/p>\n

daemonset “kube-flannel-ds” created<\/p>\n

 <\/p>\n

\u7a0d\u7b49\u7247\u523b\uff0c\u6211\u4eec\u518d\u6765\u770bmaster node\u4e0a\u7684cluster\u4fe1\u606f\uff1a<\/p>\n

# ps -ef|grep kube|grep flannel<\/p>\n

root\u00a0\u00a0\u00a0\u00a0\u00a0 6517\u00a0 6501\u00a0 0 17:20 ?\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 00:00:00 \/opt\/bin\/flanneld –ip-masq –kube-subnet-mgr<\/p>\n

root\u00a0\u00a0\u00a0\u00a0\u00a0 6573\u00a0 6546\u00a0 0 17:20 ?\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 00:00:00 \/bin\/sh -c set -e -x; cp -f \/etc\/kube-flannel\/cni-conf.json \/etc\/cni\/net.d\/10-flannel.conf; while true; do sleep 3600; done<\/p>\n

 <\/p>\n

# kubectl get pods –all-namespaces<\/p>\n

NAMESPACE\u00a0\u00a0\u00a0\u00a0 NAME\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 READY\u00a0\u00a0\u00a0\u00a0 STATUS\u00a0\u00a0\u00a0 RESTARTS\u00a0\u00a0 AGE<\/p>\n

kube-system\u00a0\u00a0 dummy-2088944543-s0c5g\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 50m<\/p>\n

kube-system\u00a0\u00a0 etcd-iz25beglnhtz\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 50m<\/p>\n

kube-system\u00a0\u00a0 kube-apiserver-iz25beglnhtz\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 50m<\/p>\n

kube-system\u00a0\u00a0 kube-controller-manager-iz25beglnhtz\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 50m<\/p>\n

kube-system\u00a0\u00a0 kube-discovery-1769846148-6z5pw\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 50m<\/p>\n

kube-system\u00a0\u00a0 kube-dns-2924299975-pddz5\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 4\/4\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 49m<\/p>\n

kube-system\u00a0\u00a0 kube-flannel-ds-5ww9k\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a02\/2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 4m<\/p>\n

kube-system\u00a0\u00a0 kube-proxy-sb4sm\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 49m<\/p>\n

kube-system\u00a0\u00a0 kube-scheduler-iz25beglnhtz\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 49m<\/p>\n

\u81f3\u5c11\u96c6\u7fa4\u7684\u6838\u5fc3\u7ec4\u4ef6\u5df2\u7ecf\u5168\u90e8run\u8d77\u6765\u4e86\u3002\u770b\u8d77\u6765\u4f3c\u4e4e\u662f\u6210\u529f\u4e86\u3002<\/p>\n

3<\/strong>\u3001<\/strong>minion node<\/strong>\uff1a<\/strong>join the cluster<\/strong><\/p>\n

\u63a5\u4e0b\u6765\uff0c\u5c31\u8be5minion node\u52a0\u5165cluster\u4e86\u3002\u8fd9\u91cc\u6211\u4eec\u7528\u5230\u4e86kubeadm\u7684\u7b2c\u4e8c\u4e2a\u547d\u4ee4\uff1akubeadm join\u3002<\/p>\n

\u5728minion node\u4e0a\u6267\u884c\uff08\u6ce8\u610f\uff1a\u8fd9\u91cc\u8981\u4fdd\u8bc1master node\u76849898\u7aef\u53e3\u5728\u9632\u706b\u5899\u662f\u6253\u5f00\u7684\uff09\uff1a<\/p>\n

# kubeadm join –token=2e7da9.7fc5668ff26430c7 123.56.200.187<\/p>\n

[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.<\/p>\n

[preflight] Running pre-flight checks<\/p>\n

[tokens] Validating provided token<\/p>\n

[discovery] Created cluster info discovery client, requesting info from “http:\/\/123.56.200.187:9898\/cluster-info\/v1\/?token-id=2e7da9”<\/p>\n

[discovery] Cluster info object received, verifying signature using given token<\/p>\n

[discovery] Cluster info signature and contents are valid, will use API endpoints [https:\/\/123.56.200.187:6443]<\/p>\n

[bootstrap] Trying to connect to endpoint https:\/\/123.56.200.187:6443<\/p>\n

[bootstrap] Detected server version: v1.5.1<\/p>\n

[bootstrap] Successfully established connection with endpoint “https:\/\/123.56.200.187:6443”<\/p>\n

[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request<\/p>\n

[csr] Received signed certificate from the API server:<\/p>\n

Issuer: CN=kubernetes | Subject: CN=system:node:iZ2ze39jeyizepdxhwqci6Z | CA: false<\/p>\n

Not before: 2016-12-26 09:31:00 +0000 UTC Not After: 2017-12-26 09:31:00 +0000 UTC<\/p>\n

[csr] Generating kubelet configuration<\/p>\n

[kubeconfig] Wrote KubeConfig file to disk: “\/etc\/kubernetes\/kubelet.conf”<\/p>\n

 <\/p>\n

Node join complete:<\/p>\n

* Certificate signing request sent to master and response<\/p>\n

received.<\/p>\n

* Kubelet informed of new secure connection details.<\/p>\n

 <\/p>\n

Run ‘kubectl get nodes’ on the master to see this machine join.<\/p>\n

\u4e5f\u5f88\u987a\u5229\u3002\u6211\u4eec\u5728minion node\u4e0a\u770b\u5230\u7684k8s\u7ec4\u4ef6\u60c5\u51b5\u5982\u4e0b\uff1a<\/p>\n

d85cf36c18ed\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/kube-proxy-amd64:v1.5.1\u00a0\u00a0\u00a0\u00a0\u00a0 “kube-proxy –kubecon”\u00a0\u00a0 About an hour ago\u00a0\u00a0 Up About an hour\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_kube-proxy.36dab4e8_kube-proxy-lsn0t_kube-system_b8eddf1c-cb4e-11e6-ad8f-00163e1001d7_5826f32b<\/p>\n

a60e373b48b8\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/pause-amd64:3.0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 “\/pause”\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 About an hour ago\u00a0\u00a0 Up About an hour\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_POD.d8dbe16c_kube-proxy-lsn0t_kube-system_b8eddf1c-cb4e-11e6-ad8f-00163e1001d7_46bfcf67<\/p>\n

a665145eb2b5\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 quay.io\/coreos\/flannel-git:v0.6.1-28-g5dde68d-amd64\u00a0\u00a0 “\/bin\/sh -c ‘set -e -”\u00a0\u00a0 About an hour ago\u00a0\u00a0 Up About an hour\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_install-cni.17d8cf2_kube-flannel-ds-tr8zr_kube-system_06eca729-cb72-11e6-ad8f-00163e1001d7_01e12f61<\/p>\n

5b46f2cb0ccf\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gcr.io\/google_containers\/pause-amd64:3.0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 “\/pause”\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 About an hour ago\u00a0\u00a0 Up About an hour\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 k8s_POD.d8dbe16c_kube-flannel-ds-tr8zr_kube-system_06eca729-cb72-11e6-ad8f-00163e1001d7_ac880d20<\/p>\n

\u6211\u4eec\u5728master node\u4e0a\u67e5\u770b\u5f53\u524dcluster\u72b6\u6001\uff1a<\/p>\n

# kubectl get nodes<\/p>\n

NAME\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 STATUS\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 AGE<\/p>\n

iz25beglnhtz\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Ready,master\u00a0\u00a0 1h<\/p>\n

iz2ze39jeyizepdxhwqci6z\u00a0\u00a0 Ready\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 21s<\/p>\n

k8s cluster\u521b\u5efa\u201d\u6210\u529f\u201d\uff01\u771f\u7684\u6210\u529f\u4e86\u5417\uff1f\u201c\u6298\u817e\u201d\u624d\u521a\u521a\u5f00\u59cb:(\uff01<\/p>\n

\u4e09\u3001<\/strong>Flannel Pod Network<\/strong>\u95ee\u9898<\/strong><\/p>\n

Join\u6210\u529f\u6240\u5e26\u6765\u7684\u201c\u4f59\u6e29\u201d\u8fd8\u672a\u6563\u53bb\uff0c\u6211\u5c31\u53d1\u73b0\u4e86Flannel pod network\u7684\u95ee\u9898\uff0ctroubleshooting\u6b63\u5f0f\u5f00\u59cb:(\u3002<\/p>\n

1<\/strong>\u3001<\/strong>minion node<\/strong>\u4e0a\u7684<\/strong>flannel<\/strong>\u65f6\u4e0d\u65f6\u5730\u62a5\u9519<\/strong><\/p>\n

\u521ajoin\u65f6\u8fd8\u597d\u597d\u7684\uff0c\u53ef\u8fc7\u4e86\u6ca1\u4e00\u4f1a\u513f\uff0c\u6211\u4eec\u5c31\u53d1\u73b0\u5728kubectl get pod \u2013all-namespaces\u4e2d\u6709\u9519\u8bef\u51fa\u73b0\uff1a<\/p>\n

kube-system\u00a0\u00a0 kube-flannel-ds-tr8zr\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 CrashLoopBackOff\u00a0\u00a0 189\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 16h<\/p>\n

\u6211\u4eec\u53d1\u73b0\u8fd9\u662fminion node\u4e0a\u7684flannel pod\u4e2d\u7684\u4e00\u4e2acontainer\u51fa\u9519\u5bfc\u81f4\u7684\uff0c\u8ddf\u8e2a\u5230\u7684\u5177\u4f53\u9519\u8bef\u5982\u4e0b\uff1a<\/p>\n

# docker logs bc0058a15969<\/p>\n

E1227 06:17:50.605110\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 main.go:127] Failed to create SubnetManager: error retrieving pod spec for ‘kube-system\/kube-flannel-ds-tr8zr’: Get https:\/\/10.96.0.1:443\/api\/v1\/namespaces\/kube-system\/pods\/kube-flannel-ds-tr8zr: dial tcp 10.96.0.1:443: i\/o timeout<\/p>\n

10.96.0.1\u662fpod network\u4e2dapiserver service\u7684cluster ip\uff0c\u800cminion node\u4e0a\u7684flannel\u7ec4\u4ef6\u5c45\u7136\u65e0\u6cd5\u8bbf\u95ee\u5230\u8fd9\u4e2acluster ip\uff01\u8fd9\u4e2a\u95ee\u9898\u7684\u5947\u602a\u4e4b\u5904\u8fd8\u5728\u4e8e\uff0c\u6709\u4e9b\u65f6\u5019\u8fd9\u4e2aPod\u5728\u88ab\u8c03\u5ea6restart N\u591a\u6b21\u540e\u6216\u8005\u88ab\u5220\u9664\u91cd\u542f\u540e\uff0c\u53c8\u7a81\u7136\u53d8\u4e3arunning\u72b6\u6001\u4e86\uff0c\u884c\u4e3a\u5341\u5206\u602a\u5f02\u3002<\/p>\n

\u5728flannel github.com issues\u4e2d\uff0c\u81f3\u5c11\u6709\u4e24\u4e2aopen issue\u4e0e\u6b64\u95ee\u9898\u6709\u5bc6\u5207\u5173\u7cfb\uff1a<\/p>\n

https:\/\/github.com\/coreos\/flannel\/issues\/545<\/p>\n

https:\/\/github.com\/coreos\/flannel\/issues\/535<\/p>\n

\u8fd9\u4e2a\u95ee\u9898\u6682\u65e0\u660e\u786e\u89e3\u3002\u5f53minion node\u4e0a\u7684flannel pod\u81ea\u6062\u590d\u4e3arunning\u72b6\u6001\u65f6\uff0c\u6211\u4eec\u53c8\u53ef\u4ee5\u7ee7\u7eed\u4e86\u3002<\/p>\n

2<\/strong>\u3001<\/strong>minion node<\/strong>\u4e0a<\/strong>flannel pod<\/strong>\u542f\u52a8\u5931\u8d25\u7684\u4e00\u4e2a\u5e94\u5bf9\u65b9\u6cd5<\/strong><\/p>\n

\u5728\u4e0b\u9762issue\u4e2d\uff0c\u5f88\u591adeveloper\u8ba8\u8bba\u4e86minion node\u4e0aflannel pod\u542f\u52a8\u5931\u8d25\u7684\u4e00\u79cd\u53ef\u80fd\u539f\u56e0\u4ee5\u53ca\u4e34\u65f6\u5e94\u5bf9\u65b9\u6cd5\uff1a<\/p>\n

https:\/\/github.com\/kubernetes\/kubernetes\/issues\/34101<\/p>\n

\u8fd9\u79cd\u8bf4\u6cd5\u5927\u81f4\u5c31\u662fminion node\u4e0a\u7684kube-proxy\u4f7f\u7528\u4e86\u9519\u8bef\u7684interface\uff0c\u901a\u8fc7\u4e0b\u9762\u65b9\u6cd5\u53ef\u4ee5fix\u8fd9\u4e2a\u95ee\u9898\u3002\u5728minion node\u4e0a\u6267\u884c\uff1a<\/p>\n

#\u00a0 kubectl -n kube-system get ds -l ‘component=kube-proxy’ -o json | jq ‘.items[0].spec.template.spec.containers[0].command |= .+ [“–cluster-cidr=10.244.0.0\/16”]’ | kubectl apply -f – && kubectl -n kube-system delete pods -l ‘component=kube-proxy’<\/p>\n

daemonset “kube-proxy” configured<\/p>\n

pod “kube-proxy-lsn0t” deleted<\/p>\n

pod “kube-proxy-sb4sm” deleted<\/p>\n

\u6267\u884c\u540e\uff0cflannel pod\u7684\u72b6\u6001\uff1a<\/p>\n

kube-system\u00a0\u00a0 kube-flannel-ds-qw291\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2\/2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 8\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 17h<\/p>\n

kube-system\u00a0\u00a0 kube-flannel-ds-x818z\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2\/2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 17\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1h<\/p>\n

 <\/p>\n

\u7ecf\u8fc717\u6b21restart\uff0cminion node\u4e0a\u7684flannel pod \u542f\u52a8ok\u4e86\u3002\u5176\u5bf9\u5e94\u7684flannel container\u542f\u52a8\u65e5\u5fd7\u5982\u4e0b\uff1a<\/p>\n

# docker logs 1f64bd9c0386<\/p>\n

I1227 07:43:26.670620\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 main.go:132] Installing signal handlers<\/p>\n

I1227 07:43:26.671006\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 manager.go:133] Determining IP address of default interface<\/p>\n

I1227 07:43:26.670825\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 kube.go:233] starting kube subnet manager<\/p>\n

I1227 07:43:26.671514\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 manager.go:163] Using 59.110.67.15 as external interface<\/p>\n

I1227 07:43:26.671575\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 manager.go:164] Using 59.110.67.15 as external endpoint<\/p>\n

I1227 07:43:26.746811\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 ipmasq.go:47] Adding iptables rule: -s 10.244.0.0\/16 -d 10.244.0.0\/16 -j RETURN<\/p>\n

I1227 07:43:26.749785\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 ipmasq.go:47] Adding iptables rule: -s 10.244.0.0\/16 ! -d 224.0.0.0\/4 -j MASQUERADE<\/p>\n

I1227 07:43:26.752343\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 ipmasq.go:47] Adding iptables rule: ! -s 10.244.0.0\/16 -d 10.244.0.0\/16 -j MASQUERADE<\/p>\n

I1227 07:43:26.755126\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 manager.go:246] Lease acquired: 10.244.1.0\/24<\/p>\n

I1227 07:43:26.755444\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:58] Watching for L3 misses<\/p>\n

I1227 07:43:26.755475\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:66] Watching for new subnet leases<\/p>\n

I1227 07:43:27.755830\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:153] Handling initial subnet events<\/p>\n

I1227 07:43:27.755905\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 device.go:163] calling GetL2List() dev.link.Index: 10<\/p>\n

I1227 07:43:27.756099\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 device.go:168] calling NeighAdd: 123.56.200.187, ca:68:7c:9b:cc:67<\/p>\n

issue\u4e2d\u8bf4\u5230\uff0c\u5728kubeadm init\u65f6\uff0c\u663e\u5f0f\u5730\u6307\u5b9a\u2013advertise-address\u5c06\u4f1a\u907f\u514d\u8fd9\u4e2a\u95ee\u9898\u3002\u4e0d\u8fc7\u76ee\u524d\u4e0d\u8981\u5728\u2013advertise-address\u540e\u9762\u5199\u4e0a\u591a\u4e2aIP\uff0c\u867d\u7136\u6587\u6863\u4e0a\u8bf4\u662f\u652f\u6301\u7684\uff0c\u4f46\u5b9e\u9645\u60c5\u51b5\u662f\uff0c\u5f53\u4f60\u663e\u5f0f\u6307\u5b9a\u2013advertise-address\u7684\u503c\u4e3a\u4e24\u4e2a\u6216\u4e24\u4e2a\u4ee5\u4e0aIP\u65f6\uff0c\u6bd4\u5982\u4e0b\u9762\u8fd9\u6837\uff1a<\/p>\n

#kubeadm init –api-advertise-addresses=10.47.217.91,123.56.200.187 –pod-network-cidr=10.244.0.0\/16<\/p>\n

master\u521d\u59cb\u5316\u6210\u529f\u540e\uff0c\u5f53minion node\u6267\u884cjoin cluster\u547d\u4ee4\u65f6\uff0c\u4f1apanic\u6389\uff1a<\/p>\n

# kubeadm join –token=92e977.f1d4d090906fc06a 10.47.217.91<\/p>\n

[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.<\/p>\n

… …<\/p>\n

[bootstrap] Successfully established connection with endpoint “https:\/\/10.47.217.91:6443”<\/p>\n

[bootstrap] Successfully established connection with endpoint “https:\/\/123.56.200.187:6443”<\/p>\n

E1228 10:14:05.405294\u00a0\u00a0 28378 runtime.go:64] Observed a panic: “close of closed channel” (close of closed channel)<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/runtime\/runtime.go:70<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/runtime\/runtime.go:63<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/runtime\/runtime.go:49<\/p>\n

\/usr\/local\/go\/src\/runtime\/asm_amd64.s:479<\/p>\n

\/usr\/local\/go\/src\/runtime\/panic.go:458<\/p>\n

\/usr\/local\/go\/src\/runtime\/chan.go:311<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/cmd\/kubeadm\/app\/node\/bootstrap.go:85<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/wait\/wait.go:96<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/wait\/wait.go:97<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/wait\/wait.go:52<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/cmd\/kubeadm\/app\/node\/bootstrap.go:93<\/p>\n

\/usr\/local\/go\/src\/runtime\/asm_amd64.s:2086<\/p>\n

[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request<\/p>\n

panic: close of closed channel [recovered]<\/p>\n

panic: close of closed channel<\/p>\n

 <\/p>\n

goroutine 29 [running]:<\/p>\n

panic(0x1342de0, 0xc4203eebf0)<\/p>\n

\/usr\/local\/go\/src\/runtime\/panic.go:500 +0x1a1<\/p>\n

k8s.io\/kubernetes\/pkg\/util\/runtime.HandleCrash(0x0, 0x0, 0x0)<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/runtime\/runtime.go:56 +0x126<\/p>\n

panic(0x1342de0, 0xc4203eebf0)<\/p>\n

\/usr\/local\/go\/src\/runtime\/panic.go:458 +0x243<\/p>\n

k8s.io\/kubernetes\/cmd\/kubeadm\/app\/node.EstablishMasterConnection.func1.1()<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/cmd\/kubeadm\/app\/node\/bootstrap.go:85 +0x29d<\/p>\n

k8s.io\/kubernetes\/pkg\/util\/wait.JitterUntil.func1(0xc420563ee0)<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/wait\/wait.go:96 +0x5e<\/p>\n

k8s.io\/kubernetes\/pkg\/util\/wait.JitterUntil(0xc420563ee0, 0x12a05f200, 0x0, 0xc420022e01, 0xc4202c2060)<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/wait\/wait.go:97 +0xad<\/p>\n

k8s.io\/kubernetes\/pkg\/util\/wait.Until(0xc420563ee0, 0x12a05f200, 0xc4202c2060)<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/pkg\/util\/wait\/wait.go:52 +0x4d<\/p>\n

k8s.io\/kubernetes\/cmd\/kubeadm\/app\/node.EstablishMasterConnection.func1(0xc4203a82f0, 0xc420269b90, 0xc4202c2060, 0xc4202c20c0, 0xc4203d8d80, 0x401, 0x480, 0xc4201e75e0, 0x17, 0xc4201e7560, …)<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/cmd\/kubeadm\/app\/node\/bootstrap.go:93 +0x100<\/p>\n

created by k8s.io\/kubernetes\/cmd\/kubeadm\/app\/node.EstablishMasterConnection<\/p>\n

\/go\/src\/k8s.io\/kubernetes\/_output\/dockerized\/go\/src\/k8s.io\/kubernetes\/cmd\/kubeadm\/app\/node\/bootstrap.go:94 +0x3ed<\/p>\n

 <\/p>\n

\u5173\u4e8ejoin panic\u8fd9\u4e2a\u95ee\u9898\uff0c\u5728\u8fd9\u4e2aissue\u4e2d\u6709\u8be6\u7ec6\u8ba8\u8bba\uff1ahttps:\/\/github.com\/kubernetes\/kubernetes\/issues\/36988<\/p>\n

3<\/strong>\u3001<\/strong>open \/run\/flannel\/subnet.env: no such file or directory<\/strong><\/p>\n

\u524d\u9762\u8bf4\u8fc7\uff0c\u9ed8\u8ba4\u60c5\u51b5\u4e0b\uff0c\u8003\u8651\u5b89\u5168\u539f\u56e0\uff0cmaster node\u662f\u4e0d\u627f\u62c5work load\u7684\uff0c\u4e0d\u53c2\u4e0epod\u8c03\u5ea6\u3002\u6211\u4eec\u8fd9\u91cc\u673a\u5668\u5c11\uff0c\u53ea\u80fd\u8ba9master node\u4e5f\u8f9b\u82e6\u4e00\u4e0b\u3002\u901a\u8fc7\u4e0b\u9762\u8fd9\u4e2a\u547d\u4ee4\u53ef\u4ee5\u8ba9master node\u4e5f\u53c2\u4e0epod\u8c03\u5ea6\uff1a<\/p>\n

# kubectl taint nodes –all dedicated-<\/p>\n

node “iz25beglnhtz” tainted<\/p>\n

\u63a5\u4e0b\u6765\uff0c\u6211\u4eeccreate\u4e00\u4e2adeployment\uff0cmanifest\u63cf\u8ff0\u6587\u4ef6\u5982\u4e0b\uff1a<\/p>\n

\/\/run-my-nginx.yaml<\/p>\n

apiVersion: extensions\/v1beta1<\/p>\n

kind: Deployment<\/p>\n

metadata:<\/p>\n

name: my-nginx<\/p>\n

spec:<\/p>\n

replicas: 2<\/p>\n

template:<\/p>\n

metadata:<\/p>\n

labels:<\/p>\n

run: my-nginx<\/p>\n

spec:<\/p>\n

containers:<\/p>\n

– name: my-nginx<\/p>\n

image: nginx:1.10.1<\/p>\n

ports:<\/p>\n

– containerPort: 80<\/p>\n

create\u540e\uff0c\u6211\u4eec\u53d1\u73b0\u8c03\u5ea6\u5230master\u4e0a\u7684my-nginx pod\u542f\u52a8\u662fok\u7684\uff0c\u4f46minion node\u4e0a\u7684pod\u5219\u4e00\u76f4\u5931\u8d25\uff0c\u67e5\u770b\u5230\u7684\u5931\u8d25\u539f\u56e0\u5982\u4e0b\uff1a<\/p>\n

Events:<\/p>\n

FirstSeen\u00a0\u00a0\u00a0 LastSeen\u00a0\u00a0\u00a0 Count\u00a0\u00a0\u00a0 From\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SubObjectPath\u00a0\u00a0\u00a0 Type\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Reason\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Message<\/p>\n

———\u00a0\u00a0\u00a0 ——–\u00a0\u00a0\u00a0 —–\u00a0\u00a0\u00a0 —-\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ————-\u00a0\u00a0\u00a0 ——–\u00a0\u00a0\u00a0 ——\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ——-<\/p>\n

28s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 28s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\u00a0\u00a0\u00a0 {default-scheduler }\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Normal\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Scheduled\u00a0\u00a0\u00a0 Successfully assigned my-nginx-2560993602-0440x to iz2ze39jeyizepdxhwqci6z<\/p>\n

27s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 26\u00a0\u00a0\u00a0 {kubelet iz2ze39jeyizepdxhwqci6z}\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Warning\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 FailedSync\u00a0\u00a0\u00a0 Error syncing pod, skipping: failed to “SetupNetwork” for “my-nginx-2560993602-0440x_default” with SetupNetworkError: “Failed to setup network for pod \\”my-nginx-2560993602-0440x_default(ba5ce554-cbf1-11e6-8c42-00163e1001d7)\\” using network plugins \\”cni\\”: open \/run\/flannel\/subnet.env: no such file or directory; Skipping pod”<\/p>\n

\u5728minion node\u4e0a\u7684\u786e\u6ca1\u6709\u627e\u5230\/run\/flannel\/subnet.env\u8be5\u6587\u4ef6\u3002\u4f46master node\u4e0a\u6709\u8fd9\u4e2a\u6587\u4ef6\uff1a<\/p>\n

\/\/ \/run\/flannel\/subnet.env<\/p>\n

 <\/p>\n

FLANNEL_NETWORK=10.244.0.0\/16<\/p>\n

FLANNEL_SUBNET=10.244.0.1\/24<\/p>\n

FLANNEL_MTU=1450<\/p>\n

FLANNEL_IPMASQ=true<\/p>\n

\u4e8e\u662f\u624b\u52a8\u5728minion node\u4e0a\u521b\u5efa\u4e00\u4efd\/run\/flannel\/subnet.env\uff0c\u5e76\u590d\u5236master node\u540c\u540d\u6587\u4ef6\u7684\u5185\u5bb9\uff0c\u4fdd\u5b58\u3002\u7a0d\u8bb8\u7247\u523b\uff0cminion node\u4e0a\u7684my-nginx pod\u4eceerror\u53d8\u6210running\u4e86\u3002<\/p>\n

4<\/strong>\u3001<\/strong>no IP addresses available in network: cbr0<\/strong><\/p>\n

\u5c06\u4e4b\u524d\u7684\u4e00\u4e2amy-nginx deployment\u7684replicas\u6539\u4e3a3\uff0c\u5e76\u521b\u5efa\u57fa\u4e8e\u8be5deployment\u4e2dpods\u7684my-nginx service\uff1a<\/p>\n

\/\/my-nginx-svc.yaml<\/p>\n

 <\/p>\n

apiVersion: v1<\/p>\n

kind: Service<\/p>\n

metadata:<\/p>\n

name: my-nginx<\/p>\n

labels:<\/p>\n

run: my-nginx<\/p>\n

spec:<\/p>\n

type: NodePort<\/p>\n

ports:<\/p>\n

– port: 80<\/p>\n

nodePort: 30062<\/p>\n

protocol: TCP<\/p>\n

selector:<\/p>\n

run: my-nginx<\/p>\n

\u4fee\u6539\u540e\uff0c\u901a\u8fc7curl localhost:30062\u6d4b\u8bd5\u670d\u52a1\u8fde\u901a\u6027\u3002\u53d1\u73b0\u901a\u8fc7VIP\u8d1f\u8f7d\u5747\u8861\u5230master node\u4e0a\u7684my-nginx pod\u7684request\u90fd\u6210\u529f\u5f97\u5230\u4e86Response\uff0c\u4f46\u662f\u8d1f\u8f7d\u5747\u8861\u5230minion node\u4e0apod\u7684request\uff0c\u5219\u963b\u585e\u5728\u90a3\u91cc\uff0c\u76f4\u5230timeout\u3002\u67e5\u770bpod\u4fe1\u606f\u624d\u53d1\u73b0\uff0c\u539f\u6765\u65b0\u8c03\u5ea6\u5230minion node\u4e0a\u7684my-nginx pod\u5e76\u6ca1\u6709\u542f\u52a8ok\uff0c\u9519\u8bef\u539f\u56e0\u5982\u4e0b\uff1a<\/p>\n

Events:<\/p>\n

FirstSeen\u00a0\u00a0\u00a0 LastSeen\u00a0\u00a0\u00a0 Count\u00a0\u00a0\u00a0 From\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0SubObjectPath\u00a0\u00a0\u00a0 Type\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Reason\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Message<\/p>\n

———\u00a0\u00a0\u00a0 ——–\u00a0\u00a0\u00a0 —–\u00a0\u00a0\u00a0 —-\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ————-\u00a0\u00a0\u00a0 ——–\u00a0\u00a0\u00a0 ——\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ——-<\/p>\n

2m\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2m\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\u00a0\u00a0\u00a0 {default-scheduler }\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Normal\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Scheduled\u00a0\u00a0\u00a0 Successfully assigned my-nginx-1948696469-ph11m to iz2ze39jeyizepdxhwqci6z<\/p>\n

2m\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 177\u00a0\u00a0\u00a0 {kubelet iz2ze39jeyizepdxhwqci6z}\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Warning\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 FailedSync\u00a0\u00a0\u00a0 Error syncing pod, skipping: failed to “SetupNetwork” for “my-nginx-1948696469-ph11m_default” with SetupNetworkError: “Failed to setup network for pod \\”my-nginx-1948696469-ph11m_default(3700d74a-cc12-11e6-8c42-00163e1001d7)\\” using network plugins \\”cni\\”: no IP addresses available in network: cbr0; Skipping pod”<\/p>\n

 <\/p>\n

\u67e5\u770bminion node\u4e0a\/var\/lib\/cni\/networks\/cbr0\u76ee\u5f55\uff0c\u53d1\u73b0\u8be5\u76ee\u5f55\u4e0b\u6709\u5982\u4e0b\u6587\u4ef6\uff1a<\/p>\n

10.244.1.10\u00a0\u00a0 10.244.1.12\u00a0\u00a0 10.244.1.14\u00a0\u00a0 10.244.1.16\u00a0\u00a0 10.244.1.18\u00a0\u00a0 10.244.1.2\u00a0\u00a0\u00a0 10.244.1.219\u00a0 10.244.1.239\u00a0 10.244.1.3\u00a0\u00a0 10.244.1.5\u00a0\u00a0 10.244.1.7\u00a0\u00a0 10.244.1.9<\/p>\n

10.244.1.100\u00a0 10.244.1.120\u00a0 10.244.1.140\u00a0 10.244.1.160\u00a0 10.244.1.180\u00a0 10.244.1.20\u00a0\u00a0 10.244.1.22\u00a0\u00a0 10.244.1.24\u00a0\u00a0 10.244.1.30\u00a0 10.244.1.50\u00a0 10.244.1.70\u00a0 10.244.1.90<\/p>\n

10.244.1.101\u00a0 10.244.1.121\u00a0 10.244.1.141\u00a0 10.244.1.161\u00a0 10.244.1.187\u00a0 10.244.1.200\u00a0 10.244.1.220\u00a0 10.244.1.240\u00a0 10.244.1.31\u00a0 10.244.1.51\u00a0 10.244.1.71\u00a0 10.244.1.91<\/p>\n

10.244.1.102\u00a0 10.244.1.122\u00a0 10.244.1.142\u00a0 10.244.1.162\u00a0 10.244.1.182\u00a0 10.244.1.201\u00a0 10.244.1.221\u00a0 10.244.1.241\u00a0 10.244.1.32\u00a0 10.244.1.52\u00a0 10.244.1.72\u00a0 10.244.1.92<\/p>\n

10.244.1.103\u00a0 10.244.1.123\u00a0 10.244.1.143\u00a0 10.244.1.163\u00a0 10.244.1.183\u00a0 10.244.1.202\u00a0 10.244.1.222\u00a0 10.244.1.242\u00a0 10.244.1.33\u00a0 10.244.1.53\u00a0 10.244.1.73\u00a0 10.244.1.93<\/p>\n

10.244.1.104\u00a0 10.244.1.124\u00a0 10.244.1.144\u00a0 10.244.1.164\u00a0 10.244.1.184\u00a0 10.244.1.203\u00a0 10.244.1.223\u00a0 10.244.1.243\u00a0 10.244.1.34\u00a0 10.244.1.54\u00a0 10.244.1.74\u00a0 10.244.1.94<\/p>\n

10.244.1.105\u00a0 10.244.1.125\u00a0 10.244.1.145\u00a0 10.244.1.165\u00a0 10.244.1.185\u00a0 10.244.1.204\u00a0 10.244.1.224\u00a0 10.244.1.244\u00a0 10.244.1.35\u00a0 10.244.1.55\u00a0 10.244.1.75\u00a0 10.244.1.95<\/p>\n

10.244.1.106\u00a0 10.244.1.126\u00a0 10.244.1.146\u00a0 10.244.1.166\u00a0 10.244.1.186\u00a0 10.244.1.205\u00a0 10.244.1.225\u00a0 10.244.1.245\u00a0 10.244.1.36\u00a0 10.244.1.56\u00a0 10.244.1.76\u00a0 10.244.1.96<\/p>\n

10.244.1.107\u00a0 10.244.1.127\u00a0 10.244.1.147\u00a0 10.244.1.167\u00a0 10.244.1.187\u00a0 10.244.1.206\u00a0 10.244.1.226\u00a0 10.244.1.246\u00a0 10.244.1.37\u00a0 10.244.1.57\u00a0 10.244.1.77\u00a0 10.244.1.97<\/p>\n

10.244.1.108\u00a0 10.244.1.128\u00a0 10.244.1.148\u00a0 10.244.1.168\u00a0 10.244.1.188\u00a0 10.244.1.207\u00a0 10.244.1.227\u00a0 10.244.1.247\u00a0 10.244.1.38\u00a0 10.244.1.58\u00a0 10.244.1.78\u00a0 10.244.1.98<\/p>\n

10.244.1.109\u00a0 10.244.1.129\u00a0 10.244.1.149\u00a0 10.244.1.169\u00a0 10.244.1.189\u00a0 10.244.1.208\u00a0 10.244.1.228\u00a0 10.244.1.248 \u00a010.244.1.39\u00a0 10.244.1.59\u00a0 10.244.1.79\u00a0 10.244.1.99<\/p>\n

10.244.1.11\u00a0\u00a0 10.244.1.13\u00a0\u00a0 10.244.1.15\u00a0\u00a0 10.244.1.17\u00a0\u00a0 10.244.1.19\u00a0\u00a0 10.244.1.209\u00a0 10.244.1.229\u00a0 10.244.1.249\u00a0 10.244.1.4\u00a0\u00a0 10.244.1.6\u00a0\u00a0 10.244.1.8\u00a0\u00a0 last_reserved_ip<\/p>\n

10.244.1.110\u00a0 10.244.1.130\u00a0 10.244.1.150\u00a0 10.244.1.170\u00a0 10.244.1.190\u00a0 10.244.1.21\u00a0\u00a0 10.244.1.23\u00a0\u00a0 10.244.1.25\u00a0\u00a0 10.244.1.40\u00a0 10.244.1.60\u00a0 10.244.1.80<\/p>\n

10.244.1.111\u00a0 10.244.1.131\u00a0 10.244.1.151\u00a0 10.244.1.171\u00a0 10.244.1.191\u00a0 10.244.1.210\u00a0 10.244.1.230\u00a0 10.244.1.250\u00a0 10.244.1.41\u00a0 10.244.1.61\u00a0 10.244.1.81<\/p>\n

10.244.1.112\u00a0 10.244.1.132\u00a0 10.244.1.152\u00a0 10.244.1.172\u00a0 10.244.1.192\u00a0 10.244.1.211\u00a0 10.244.1.231\u00a0 10.244.1.251\u00a0 10.244.1.42\u00a0 10.244.1.62\u00a0 10.244.1.82<\/p>\n

10.244.1.113\u00a0 10.244.1.133\u00a0 10.244.1.153\u00a0 10.244.1.173\u00a0 10.244.1.193\u00a0 10.244.1.212\u00a0 10.244.1.232\u00a0 10.244.1.252\u00a0 10.244.1.43\u00a0 10.244.1.63\u00a0 10.244.1.83<\/p>\n

10.244.1.114\u00a0 10.244.1.134\u00a0 10.244.1.154\u00a0 10.244.1.174\u00a0 10.244.1.194\u00a0 10.244.1.213\u00a0 10.244.1.233\u00a0 10.244.1.253\u00a0 10.244.1.44\u00a0 10.244.1.64\u00a0 10.244.1.84<\/p>\n

10.244.1.115\u00a0 10.244.1.135\u00a0 10.244.1.155\u00a0 10.244.1.175\u00a0 10.244.1.195\u00a0 10.244.1.214\u00a0 10.244.1.234\u00a0 10.244.1.254\u00a0 10.244.1.45\u00a0 10.244.1.65\u00a0 10.244.1.85<\/p>\n

10.244.1.116\u00a0 10.244.1.136\u00a0 10.244.1.156\u00a0 10.244.1.176\u00a0 10.244.1.196\u00a0 10.244.1.215\u00a0 10.244.1.235\u00a0 10.244.1.26\u00a0\u00a0 10.244.1.46\u00a0 10.244.1.66\u00a0 10.244.1.86<\/p>\n

10.244.1.117\u00a0 10.244.1.137\u00a0 10.244.1.157\u00a0 10.244.1.177\u00a0 10.244.1.197\u00a0 10.244.1.216\u00a0 10.244.1.236\u00a0 10.244.1.27\u00a0\u00a0 10.244.1.47\u00a0 10.244.1.67\u00a0 10.244.1.87<\/p>\n

10.244.1.118\u00a0 10.244.1.138\u00a0 10.244.1.158\u00a0 10.244.1.178\u00a0 10.244.1.198\u00a0 10.244.1.217\u00a0 10.244.1.237\u00a0 10.244.1.28\u00a0\u00a0 10.244.1.48\u00a0 10.244.1.68\u00a0 10.244.1.88<\/p>\n

10.244.1.119\u00a0 10.244.1.139\u00a0 10.244.1.159\u00a0 10.244.1.179\u00a0 10.244.1.199\u00a0 10.244.1.218\u00a0 10.244.1.238\u00a0 10.244.1.29\u00a0\u00a0 10.244.1.49\u00a0 10.244.1.69\u00a0 10.244.1.89<\/p>\n

 <\/p>\n

\u8fd9\u5df2\u7ecf\u5c0610.244.1.x\u6bb5\u7684\u6240\u6709ip\u5360\u6ee1\uff0c\u81ea\u7136\u6ca1\u6709available\u7684IP\u53ef\u4f9b\u65b0pod\u4f7f\u7528\u4e86\u3002\u81f3\u4e8e\u4e3a\u4f55\u5360\u6ee1\uff0c\u8fd9\u4e2a\u539f\u56e0\u5c1a\u4e0d\u660e\u6717\u3002\u4e0b\u9762\u4e24\u4e2aopen issue\u4e0e\u8fd9\u4e2a\u95ee\u9898\u76f8\u5173\uff1a<\/p>\n

https:\/\/github.com\/containernetworking\/cni\/issues\/306<\/p>\n

https:\/\/github.com\/kubernetes\/kubernetes\/issues\/21656<\/p>\n

\u8fdb\u5165\u5230\/var\/lib\/cni\/networks\/cbr0\u76ee\u5f55\u4e0b\uff0c\u6267\u884c\u4e0b\u9762\u547d\u4ee4\u53ef\u4ee5\u91ca\u653e\u90a3\u4e9b\u53ef\u80fd\u662fkubelet leak\u7684IP\u8d44\u6e90\uff1a<\/p>\n

for hash in $(tail -n +1 * | grep ‘^[A-Za-z0-9]*$’ | cut -c 1-8); do if [ -z $(docker ps -a | grep $hash | awk ‘{print $1}’) ]; then grep -irl $hash .\/; fi; done | xargs rm<\/p>\n

\u6267\u884c\u540e\uff0c\u76ee\u5f55\u4e0b\u7684\u6587\u4ef6\u5217\u8868\u53d8\u6210\u4e86\uff1a<\/p>\n

ls -l<\/p>\n

total 32<\/p>\n

drw-r–r– 2 root root 12288 Dec 27 17:11 .\/<\/p>\n

drw-r–r– 3 root root\u00a0 4096 Dec 27 13:52 ..\/<\/p>\n

-rw-r–r– 1 root root\u00a0\u00a0\u00a0 64 Dec 27 17:11 10.244.1.2<\/p>\n

-rw-r–r– 1 root root\u00a0\u00a0\u00a0 64 Dec 27 17:11 10.244.1.3<\/p>\n

-rw-r–r– 1 root root\u00a0\u00a0\u00a0 64 Dec 27 17:11 10.244.1.4<\/p>\n

-rw-r–r– 1 root root\u00a0\u00a0\u00a0 10 Dec 27 17:11 last_reserved_ip<\/p>\n

 <\/p>\n

\u4e0d\u8fc7pod\u4ecd\u7136\u5904\u4e8e\u5931\u8d25\u72b6\u6001\uff0c\u4f46\u8fd9\u6b21\u5931\u8d25\u7684\u539f\u56e0\u53c8\u53d1\u751f\u4e86\u53d8\u5316\uff1a<\/p>\n

Events:<\/p>\n

FirstSeen\u00a0\u00a0\u00a0 LastSeen\u00a0\u00a0\u00a0 Count\u00a0\u00a0\u00a0 From\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 SubObjectPath\u00a0\u00a0\u00a0 Type\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Reason\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Message<\/p>\n

———\u00a0\u00a0\u00a0 ——–\u00a0\u00a0\u00a0 —–\u00a0\u00a0\u00a0 —-\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ————-\u00a0\u00a0\u00a0 ——–\u00a0\u00a0\u00a0 ——\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ——-<\/p>\n

23s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 23s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\u00a0\u00a0\u00a0 {default-scheduler }\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Normal\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Scheduled\u00a0\u00a0\u00a0 Successfully assigned my-nginx-1948696469-7p4nn to iz2ze39jeyizepdxhwqci6z<\/p>\n

22s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 22\u00a0\u00a0\u00a0 {kubelet iz2ze39jeyizepdxhwqci6z}\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Warning\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 FailedSync\u00a0\u00a0\u00a0 Error syncing pod, skipping: failed to “SetupNetwork” for “my-nginx-1948696469-7p4nn_default” with SetupNetworkError: “Failed to setup network for pod \\”my-nginx-1948696469-7p4nn_default(a40fe652-cc14-11e6-8c42-00163e1001d7)\\” using network plugins \\”cni\\”: \\”cni0\\” already has an IP address different from 10.244.1.1\/24; Skipping pod”<\/p>\n

 <\/p>\n

\u800c\/var\/lib\/cni\/networks\/cbr0\u76ee\u5f55\u4e0b\u7684\u6587\u4ef6\u53c8\u5f00\u59cb\u8fc5\u901f\u589e\u52a0\uff01\u95ee\u9898\u9677\u5165\u50f5\u5c40\u3002<\/p>\n

5<\/strong>\u3001<\/strong>flannel vxlan<\/strong>\u4e0d\u901a\uff0c\u540e\u7aef\u6362<\/strong>udp<\/strong>\uff0c\u4ecd\u7136\u4e0d\u901a<\/strong><\/p>\n

\u6298\u817e\u5230\u8fd9\u91cc\uff0c\u57fa\u672c\u7b4b\u75b2\u529b\u5c3d\u4e86\u3002\u4e8e\u662f\u5728\u4e24\u4e2anode\u4e0a\u6267\u884ckubeadm reset\uff0c\u51c6\u5907\u91cd\u65b0\u6765\u8fc7\u3002<\/p>\n

kubeadm reset\u540e\uff0c\u4e4b\u524dflannel\u521b\u5efa\u7684bridge device cni0\u548c\u7f51\u53e3\u8bbe\u5907flannel.1\u4f9d\u7136\u5065\u5728\u3002\u4e3a\u4e86\u4fdd\u8bc1\u73af\u5883\u5f7b\u5e95\u6062\u590d\u5230\u521d\u59cb\u72b6\u6001\uff0c\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u4e0b\u9762\u547d\u4ee4\u5220\u9664\u8fd9\u4e24\u4e2a\u8bbe\u5907\uff1a<\/p>\n

# ifconfig\u00a0 cni0 down<\/p>\n

# brctl delbr cni0<\/p>\n

# ip link delete flannel.1<\/p>\n

\u6709\u4e86\u524d\u9762\u51e0\u4e2a\u95ee\u9898\u7684\u201c\u78e8\u70bc\u201d\u540e\uff0c\u91cd\u65b0init\u548cjoin\u7684k8s cluster\u663e\u5f97\u683c\u5916\u987a\u5229\u3002\u8fd9\u6b21minion node\u6ca1\u6709\u518d\u51fa\u73b0\u4ec0\u4e48\u5f02\u5e38\u3002<\/p>\n

#\u00a0 kubectl get nodes -o wide<\/p>\n

NAME\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0STATUS\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 AGE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 EXTERNAL-IP<\/p>\n

iz25beglnhtz\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Ready,master\u00a0\u00a0 5m\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <none><\/p>\n

iz2ze39jeyizepdxhwqci6z\u00a0\u00a0 Ready\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 51s\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <none><\/p>\n

 <\/p>\n

# kubectl get pod –all-namespaces<\/p>\n

NAMESPACE\u00a0\u00a0\u00a0\u00a0 NAME\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 READY\u00a0\u00a0\u00a0\u00a0 STATUS\u00a0\u00a0\u00a0 RESTARTS\u00a0\u00a0 AGE<\/p>\n

default\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 my-nginx-1948696469-71h1l\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 3m<\/p>\n

default\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 my-nginx-1948696469-zwt5g\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 3m<\/p>\n

default\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 my-ubuntu-2560993602-ftdm6\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a01\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 3m<\/p>\n

kube-system\u00a0\u00a0 dummy-2088944543-lmlbh\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 5m<\/p>\n

kube-system\u00a0\u00a0 etcd-iz25beglnhtz\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 6m<\/p>\n

kube-system\u00a0\u00a0 kube-apiserver-iz25beglnhtz\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a01\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 6m<\/p>\n

kube-system\u00a0\u00a0 kube-controller-manager-iz25beglnhtz\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 6m<\/p>\n

kube-system\u00a0\u00a0 kube-discovery-1769846148-l5lfw\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 5m<\/p>\n

kube-system\u00a0\u00a0 kube-dns-2924299975-mdq5r\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 4\/4\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 5m<\/p>\n

kube-system\u00a0\u00a0 kube-flannel-ds-9zwr1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2\/2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 5m<\/p>\n

kube-system\u00a0\u00a0 kube-flannel-ds-p7xh2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 2\/2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1m<\/p>\n

kube-system\u00a0\u00a0 kube-proxy-dwt5f\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a01\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 5m<\/p>\n

kube-system\u00a0\u00a0 kube-proxy-vm6v2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1m<\/p>\n

kube-system\u00a0\u00a0 kube-scheduler-iz25beglnhtz\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Running\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 6m<\/p>\n

 <\/p>\n

\u63a5\u4e0b\u6765\u6211\u4eec\u521b\u5efamy-nginx deployment\u548cservice\u6765\u6d4b\u8bd5flannel\u7f51\u7edc\u7684\u8fde\u901a\u6027\u3002\u901a\u8fc7curl my-nginx service\u7684nodeport\uff0c\u53d1\u73b0\u53ef\u4ee5reach master\u4e0a\u7684\u4e24\u4e2anginx pod\uff0c\u4f46\u662fminion node\u4e0a\u7684pod\u4f9d\u65e7\u4e0d\u901a\u3002<\/p>\n

\u5728master\u4e0a\u770bflannel docker\u7684\u65e5\u5fd7\uff1a<\/p>\n

I1228 02:52:22.097083\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:225] L3 miss: 10.244.1.2<\/p>\n

I1228 02:52:22.097169\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 device.go:191] calling NeighSet: 10.244.1.2, 46:6c:7a:a6:06:60<\/p>\n

I1228 02:52:22.097335\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:236] AddL3 succeeded<\/p>\n

I1228 02:52:55.169952\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:00.801901\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:03.801923\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:04.801764\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:05.801848\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:06.888269\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:225] L3 miss: 10.244.1.2<\/p>\n

I1228 02:53:06.888340\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 device.go:191] calling NeighSet: 10.244.1.2, 46:6c:7a:a6:06:60<\/p>\n

I1228 02:53:06.888507\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:236] AddL3 succeeded<\/p>\n

I1228 02:53:39.969791\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:45.153770\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:48.154822\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:49.153774\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:50.153734\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:220] Ignoring not a miss: 46:6c:7a:a6:06:60, 10.244.1.2<\/p>\n

I1228 02:53:52.154056\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:225] L3 miss: 10.244.1.2<\/p>\n

I1228 02:53:52.154110\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 device.go:191] calling NeighSet: 10.244.1.2, 46:6c:7a:a6:06:60<\/p>\n

I1228 02:53:52.154256\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 network.go:236] AddL3 succeeded<\/p>\n

\u65e5\u5fd7\u4e2d\u6709\u5927\u91cf\uff1a\u201cIgnoring not a miss\u201d\u5b57\u6837\u7684\u65e5\u5fd7\uff0c\u4f3c\u4e4evxlan\u7f51\u7edc\u6709\u95ee\u9898\u3002\u8fd9\u4e2a\u95ee\u9898\u4e0e\u4e0b\u9762issue\u4e2d\u63cf\u8ff0\u9887\u4e3a\u63a5\u8fd1\uff1a<\/p>\n

https:\/\/github.com\/coreos\/flannel\/issues\/427<\/p>\n

Flannel\u9ed8\u8ba4\u91c7\u7528vxlan\u4f5c\u4e3abackend\uff0c\u4f7f\u7528kernel vxlan\u9ed8\u8ba4\u7684udp 8742\u7aef\u53e3\u3002Flannel\u8fd8\u652f\u6301udp\u7684backend\uff0c\u4f7f\u7528udp 8285\u7aef\u53e3\u3002\u4e8e\u662f\u8bd5\u7740\u66f4\u6362\u4e00\u4e0bflannel\u540e\u7aef\u3002\u66f4\u6362flannel\u540e\u7aef\u7684\u6b65\u9aa4\u5982\u4e0b\uff1a<\/p>\n