A box of chocolates

Life is like a box of chocolates, you never know what you are going to get.

Uncategorized

Centos7 下Kubernetes集群安装部署

By Vinny Wang

操作环境

网络拓扑图
、
操作环境：
 
k8s-master：
OS：
  1. [root@k8s-master ~]# cat /etc/redhat-release
  2. CentOS Linux release 7.4.1708 (Core)
etcd：
  1. [root@k8s-master ~]# etcd –version
  2. etcd Version: 3.2.9
  3. Git SHA: f1d7dd8
  4. Go Version: go1.8.3
  5. Go OS/Arch: linux/amd64

Kubernetes：

  1. [root@k8s-master ~]# kubectl version
  2. Client Version: version.Info{Major:”1″, Minor:”5″, GitVersion:”v1.5.2″, GitCommit:”269f928217957e7126dc87e6adfa82242bfe5b1e”, GitTreeState:”clean”, BuildDate:”2017-07-03T15:31:10Z”, GoVersion:”go1.7.4″, Compiler:”gc”, Platform:”linux/amd64″}
  3. Server Version: version.Info{Major:”1″, Minor:”5″, GitVersion:”v1.5.2″, GitCommit:”269f928217957e7126dc87e6adfa82242bfe5b1e”, GitTreeState:”clean”, BuildDate:”2017-07-03T15:31:10Z”, GoVersion:”go1.7.4″, Compiler:”gc”, Platform:”linux/amd64″}


k8s-node1&k8s-node2：

OS：
  1. [root@k8s-node1 ~]# cat /etc/redhat-release
  2. CentOS Linux release 7.4.1708 (Core)

etcd：

  1. [root@k8s-node1 ~]# etcd –version
  2. etcd Version: 3.2.9
  3. Git SHA: f1d7dd8
  4. Go Version: go1.8.3
  5. Go OS/Arch: linux/amd64

docker：

  1. [root@k8s-node1 ~]# docker version
  2. Client:
  3. Version: 1.12.6
  4. API version: 1.24
  5. Package version: docker-1.12.6-68.gitec8512b.el7.centos.x86_64
  6. Go version: go1.8.3
  7. Git commit: ec8512b/1.12.6
  8. Built: Mon Dec 11 16:08:42 2017
  9. OS/Arch: linux/amd64
  11. Server:
  12. Version: 1.12.6
  13. API version: 1.24
  14. Package version: docker-1.12.6-68.gitec8512b.el7.centos.x86_64
  15. Go version: go1.8.3
  16. Git commit: ec8512b/1.12.6
  17. Built: Mon Dec 11 16:08:42 2017
  18. OS/Arch: linux/amd64

Kubernetes：

  1. [root@k8s-node1 ~]# kubectl version
  2. Client Version: version.Info{Major:”1″, Minor:”5″, GitVersion:”v1.5.2″, GitCommit:”269f928217957e7126dc87e6adfa82242bfe5b1e”, GitTreeState:”clean”, BuildDate:”2017-07-03T15:31:10Z”, GoVersion:”go1.7.4″, Compiler:”gc”, Platform:”linux/amd64″}

安装部署

安装前准备

在安装部署集群前，先将三台服务器的时间通过NTP进行同步，否则，在后面的运行中可能会提示错误
[root@k8s-node1 ~]# ntpdate -u cn.pool.ntp.org

在node节点上安装redhat-ca.crt

[root@k8s-node1 ~]# yum install *rhsm* -y

etcd集群配置

master节点配置

1.安装kubernetes etcd
[root@k8s-master ~]# yum -y install kubernetes-master etcd

2.配置etcd选项

ETCD_DATA_DIR/ETCD_LISTEN_PEER_URLS/ETCD_LISTEN_CLIENT_URLS/ETCD_MAX_SNAPSHOTS/ETCD_NAME/ETCD_INITIAL_ADVERTISE_PEERURLS/ETCD_ADVERTISE_CLIENT_URLS/ETCD_INITIAL_CLUSTER
  1. [root@k8s-master ~]# vi /etc/etcd/etcd.conf
  2. #[Member]
  3. #ETCD_CORS=””
  4. ETCD_DATA_DIR=”/var/lib/etcd/default.etcd”
  5. #ETCD_WAL_DIR=””
  6. ETCD_LISTEN_PEER_URLS=”http://10.10.200.224:2380″
  7. ETCD_LISTEN_CLIENT_URLS=”http://10.10.200.224:2379,http://127.0.0.1:2379″
  8. ETCD_MAX_SNAPSHOTS=”5″
  9. #ETCD_MAX_WALS=”5″
  10. ETCD_NAME=”etcd1″
  11. #ETCD_SNAPSHOT_COUNT=”100000″
  12. #ETCD_HEARTBEAT_INTERVAL=”100″
  13. #ETCD_ELECTION_TIMEOUT=”1000″
  14. #ETCD_QUOTA_BACKEND_BYTES=”0″
  15. #
  16. #[Clustering]
  17. ETCD_INITIAL_ADVERTISE_PEER_URLS=”http://10.10.200.224:2380″
  18. ETCD_ADVERTISE_CLIENT_URLS=”http://10.10.200.224:2379″
  19. #ETCD_DISCOVERY=””
  20. #ETCD_DISCOVERY_FALLBACK=”proxy”
  21. #ETCD_DISCOVERY_PROXY=””
  22. #ETCD_DISCOVERY_SRV=””
  23. ETCD_INITIAL_CLUSTER=”etcd1=http://10.10.200.224:2380,etcd2=http://10.10.200.229:2380,etcd3=http://10.10.200.230:2380″
  24. #ETCD_INITIAL_CLUSTER_TOKEN=”etcd-cluster”
  25. #ETCD_INITIAL_CLUSTER_STATE=”new”
  26. #ETCD_STRICT_RECONFIG_CHECK=”true”
  27. #ETCD_ENABLE_V2=”true”
  28. #
  29. #[Proxy]
  30. #ETCD_PROXY=”off”
  31. #ETCD_PROXY_FAILURE_WAIT=”5000″
  32. #ETCD_PROXY_REFRESH_INTERVAL=”30000″
  33. #ETCD_PROXY_DIAL_TIMEOUT=”1000″
  34. #ETCD_PROXY_WRITE_TIMEOUT=”5000″
  35. #ETCD_PROXY_READ_TIMEOUT=”0″
  36. #
  37. #[Security]
  38. #ETCD_CERT_FILE=””
  39. #ETCD_KEY_FILE=””
  40. #ETCD_CLIENT_CERT_AUTH=”false”
  41. #ETCD_TRUSTED_CA_FILE=””
  42. #ETCD_AUTO_TLS=”false”
  43. #ETCD_PEER_CERT_FILE=””
  44. #ETCD_PEER_KEY_FILE=””
  45. #ETCD_PEER_CLIENT_CERT_AUTH=”false”
  46. #ETCD_PEER_TRUSTED_CA_FILE=””
  47. #ETCD_PEER_AUTO_TLS=”false”
  48. #
  49. #[Logging]
  50. #ETCD_DEBUG=”false”
  51. #ETCD_LOG_PACKAGE_LEVELS=””
  52. #ETCD_LOG_OUTPUT=”default”
  53. #
  54. #[Unsafe]
  55. #ETCD_FORCE_NEW_CLUSTER=”false”
  56. #
  57. #[Version]
  58. #ETCD_VERSION=”false”
  59. #ETCD_AUTO_COMPACTION_RETENTION=”0″
  60. #
  61. #[Profiling]
  62. #ETCD_ENABLE_PPROF=”false”
  63. #ETCD_METRICS=”basic”
  64. #
  65. #[Auth]
  66. #ETCD_AUTH_TOKEN=”simple”

nodes节点配置

1.安装部署kubernetes-node/etcd/flannel/docker
[root@k8s-node1 ~]# yum -y install kubernetes-node etcd flannel docker

2.分别配置etcd，node1与node2的配置方法相同，以node1配置文件为例说明

  1. [root@k8s-node1 ~]# vi /etc/etcd/etcd.conf
  2. #[Member]
  3. #ETCD_CORS=””
  4. ETCD_DATA_DIR=”/var/lib/etcd/default.etcd”
  5. #ETCD_WAL_DIR=””
  6. ETCD_LISTEN_PEER_URLS=”http://10.10.200.229:2380″
  7. ETCD_LISTEN_CLIENT_URLS=”http://10.10.200.229:2379,http://127.0.0.1:2379″
  8. #ETCD_MAX_SNAPSHOTS=”5″
  9. #ETCD_MAX_WALS=”5″
  10. ETCD_NAME=”etcd2″
  11. #ETCD_SNAPSHOT_COUNT=”100000″
  12. #ETCD_HEARTBEAT_INTERVAL=”100″
  13. #ETCD_ELECTION_TIMEOUT=”1000″
  14. #ETCD_QUOTA_BACKEND_BYTES=”0″
  15. #
  16. #[Clustering]
  17. ETCD_INITIAL_ADVERTISE_PEER_URLS=”http://10.10.200.229:2380″
  18. ETCD_ADVERTISE_CLIENT_URLS=”http://10.10.200.229:2379″
  19. #ETCD_DISCOVERY=””
  20. #ETCD_DISCOVERY_FALLBACK=”proxy”
  21. #ETCD_DISCOVERY_PROXY=””
  22. #ETCD_DISCOVERY_SRV=””
  23. #ETCD_INITIAL_CLUSTER=”default=http://localhost:2380″
  24. #ETCD_INITIAL_CLUSTER_TOKEN=”etcd-cluster”
  25. “/etc/etcd/etcd.conf” 66L, 1696C
  26. #[Member]
  27. #ETCD_CORS=””
  28. ETCD_DATA_DIR=”/var/lib/etcd/default.etcd”
  29. #ETCD_WAL_DIR=””
  30. ETCD_LISTEN_PEER_URLS=”http://10.10.200.229:2380″
  31. ETCD_LISTEN_CLIENT_URLS=”http://10.10.200.229:2379,http://127.0.0.1:2379″
  32. #ETCD_MAX_SNAPSHOTS=”5″
  33. #ETCD_MAX_WALS=”5″
  34. ETCD_NAME=”etcd2″
  35. #ETCD_SNAPSHOT_COUNT=”100000″
  36. #ETCD_HEARTBEAT_INTERVAL=”100″
  37. #ETCD_ELECTION_TIMEOUT=”1000″
  38. #ETCD_QUOTA_BACKEND_BYTES=”0″
  39. #
  40. #[Clustering]
  41. ETCD_INITIAL_ADVERTISE_PEER_URLS=”http://10.10.200.229:2380″
  42. ETCD_ADVERTISE_CLIENT_URLS=”http://10.10.200.229:2379″
  43. #ETCD_DISCOVERY=””
  44. #ETCD_DISCOVERY_FALLBACK=”proxy”
  45. #ETCD_DISCOVERY_PROXY=””
  46. #ETCD_DISCOVERY_SRV=””
  47. #ETCD_INITIAL_CLUSTER=”default=http://localhost:2380″
  48. #ETCD_INITIAL_CLUSTER_TOKEN=”etcd-cluster”
  49. #ETCD_INITIAL_CLUSTER_STATE=”new”
  50. #ETCD_STRICT_RECONFIG_CHECK=”true”
  51. #ETCD_ENABLE_V2=”true”
  52. ETCD_INITIAL_CLUSTER=”etcd1=http://10.10.200.224:2380,etcd2=http://10.10.200.229:2380,etcd3=http://10.10.200.230:2380″
  53. #
  54. #[Proxy]
  55. #ETCD_PROXY=”off”
  56. #ETCD_PROXY_FAILURE_WAIT=”5000″
  57. #ETCD_PROXY_REFRESH_INTERVAL=”30000″
  58. #ETCD_PROXY_DIAL_TIMEOUT=”1000″
  59. #ETCD_PROXY_WRITE_TIMEOUT=”5000″
  60. #ETCD_PROXY_READ_TIMEOUT=”0″
  61. #
  62. #[Security]
  63. #ETCD_CERT_FILE=””
  64. #ETCD_KEY_FILE=””
  65. #ETCD_CLIENT_CERT_AUTH=”false”
  66. #ETCD_TRUSTED_CA_FILE=””
  67. #ETCD_AUTO_TLS=”false”
  68. #ETCD_PEER_CERT_FILE=””
  69. #ETCD_PEER_KEY_FILE=””
  70. #ETCD_PEER_CLIENT_CERT_AUTH=”false”
  71. #ETCD_PEER_TRUSTED_CA_FILE=””
  72. #ETCD_PEER_AUTO_TLS=”false”
  73. #
  74. #[Logging]
  75. #ETCD_DEBUG=”false”
  76. #ETCD_LOG_PACKAGE_LEVELS=””
  77. #ETCD_LOG_OUTPUT=”default”
  78. #
  79. #[Unsafe]
  80. #ETCD_FORCE_NEW_CLUSTER=”false”
  81. #
  82. #[Version]
  83. #ETCD_VERSION=”false”
  84. #ETCD_AUTO_COMPACTION_RETENTION=”0″
  85. #
  86. #[Profiling]
  87. #ETCD_ENABLE_PPROF=”false”
  88. #ETCD_METRICS=”basic”
  89. #
  90. #[Auth]
  91. #ETCD_AUTH_TOKEN=”simple”

启动etcd cluster

分别在3台服务器启动etcd
  1. [root@k8s-master ~]# systemctl start etcd.service
  2. [root@k8s-master ~]# systemctl status etcd.service
  3. ?etcd.service – Etcd Server
  4. Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
  5. Active: active (running) since Wed 2018-01-03 09:21:36 CST; 1h 41min ago
  6. Main PID: 11426 (etcd)
  7. CGroup: /system.slice/etcd.service
  8. 忖11426 /usr/bin/etcd –name=etcd1 –data-dir=/var/lib/etcd/default.etcd –listen-client-urls=http://10.10.200.224:2379,http://127.0.0.1:2379
查看etcd集群状态
  1. [root@k8s-master ~]# etcdctl cluster-health
  2. member 359947fae86629a7 is healthy: got healthy result from http://10.10.200.224:2379
  3. member 4be7ddbd3bb99ca0 is healthy: got healthy result from http://10.10.200.229:2379
  4. member 84951a697d1bf6a0 is healthy: got healthy result from http://10.10.200.230:2379
  5. cluster is healthy

Kubernetes集群配置

master节点配置

1.apiserver配置文件修改，注意KUBE_ADMISSION_CONTROL选项的参数配置
  1. [root@k8s-master ~]# vi /etc/kubernetes/apiserver
  2. ###
  3. # kubernetes system config
  4. #
  5. # The following values are used to configure the kube-apiserver
  6. #
  8. # The address on the local server to listen to.
  9. #KUBE_API_ADDRESS=”–insecure-bind-address=127.0.0.1″
  10. KUBE_API_ADDRESS=”–address=0.0.0.0″
  12. # The port on the local server to listen on.
  13. KUBE_API_PORT=”–port=8080″
  15. # Port minions listen on
  16. KUBELET_PORT=”–kubelet-port=10250″
  18. # Comma separated list of nodes in the etcd cluster
  19. KUBE_ETCD_SERVERS=”–etcd-servers=http://10.10.200.224:2379,http://10.10.200.229:2379,http://10.10.200.230:2379″
  21. # Address range to use for services
  22. KUBE_SERVICE_ADDRESSES=”–service-cluster-ip-range=10.254.0.0/16″
  24. # default admission control policies
  25. #KUBE_ADMISSION_CONTROL=”–admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota”
  27. KUBE_ADMISSION_CONTROL=”–admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota”
  29. # Add your own!
  30. KUBE_API_ARGS=””

2.启动服务

  1. [root@k8s-master ~]# systemctl start kube-apiserver
  2. [root@k8s-master ~]# systemctl start kube-controller-manager
  3. [root@k8s-master ~]# systemctl start kube-scheduler
  4. [root@k8s-master ~]# systemctl enable kube-apiserver
  5. [root@k8s-master ~]# systemctl enable kube-controller-manager
  6. [root@k8s-master ~]# systemctl enable kube-scheduler

nodes节点配置

1.配置config配置，node1&node2配置相同，以node1为例说明
  1. [root@k8s-node1 ~]# vi /etc/kubernetes/config
  2. ###
  3. # kubernetes system config
  4. #
  5. # The following values are used to configure various aspects of all
  6. # kubernetes services, including
  7. #
  8. # kube-apiserver.service
  9. # kube-controller-manager.service
  10. # kube-scheduler.service
  11. # kubelet.service
  12. # kube-proxy.service
  13. # logging to stderr means we get it in the systemd journal
  14. KUBE_LOGTOSTDERR=”–logtostderr=true”
  16. # journal message level, 0 is debug
  17. KUBE_LOG_LEVEL=”–v=0″
  19. # Should this cluster be allowed to run privileged docker containers
  20. KUBE_ALLOW_PRIV=”–allow-privileged=false”
  22. # How the controller-manager, scheduler, and proxy find the apiserver
  23. KUBE_MASTER=”–master=http://10.10.200.224:8080″

2.配置kubelet

  1. [root@k8s-node1 ~]# vi /etc/kubernetes/kubelet
  2. ###
  3. # kubernetes kubelet (minion) config
  5. # The address for the info server to serve on (set to 0.0.0.0 or “” for all interfaces)
  6. KUBELET_ADDRESS=”–address=127.0.0.1″
  8. # The port for the info server to serve on
  9. # KUBELET_PORT=”–port=10250″
  11. # You may leave this blank to use the actual hostname
  12. KUBELET_HOSTNAME=”–hostname-override=10.10.200.229″
  14. # location of the api-server
  15. KUBELET_API_SERVER=”–api-servers=http://10.10.200.224:8080″
  17. # pod infrastructure container
  18. KUBELET_POD_INFRA_CONTAINER=”–pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest”
  20. # Add your own!
  21. KUBELET_ARGS=””

网络配置

这里使用flannel进行网络配置，已经在2个节点上安装，下面进行配置。

在节点上进行配置flannel

  1. [root@k8s-node1 ~]# vi /etc/sysconfig/flanneld
  2. # Flanneld configuration options
  4. # etcd url location. Point this to the server where etcd runs
  5. FLANNEL_ETCD_ENDPOINTS=”http://10.10.200.224:2379″
  7. # etcd config key. This is the configuration key that flannel queries
  8. # For address range assignment
  9. FLANNEL_ETCD_PREFIX=”/atomic.io/network”
  11. # Any additional options that you want to pass
  12. #FLANNEL_OPTIONS=””

查看集群状态

  1. [root@k8s-master ~]# kubectl get nodes
  2. NAME STATUS AGE
  3. 10.10.200.229 Ready 1h
  4. 10.10.200.230 Ready 1h
  5. [root@k8s-master ~]# etcdctl member list
  6. 359947fae86629a7: name=etcd1 peerURLs=http://10.10.200.224:2380 clientURLs=http://10.10.200.224:2379 isLeader=true
  7. 4be7ddbd3bb99ca0: name=etcd2 peerURLs=http://10.10.200.229:2380 clientURLs=http://10.10.200.229:2379 isLeader=false
  8. 84951a697d1bf6a0: name=etcd3 peerURLs=http://10.10.200.230:2380 clientURLs=http://10.10.200.230:2379 isLeader=false
  9. [root@k8s-master ~]# etcdctl cluster-health
  10. member 359947fae86629a7 is healthy: got healthy result from http://10.10.200.224:2379
  11. member 4be7ddbd3bb99ca0 is healthy: got healthy result from http://10.10.200.229:2379
  12. member 84951a697d1bf6a0 is healthy: got healthy result from http://10.10.200.230:2379
  13. cluster is healthy

测试使用

在测试使用前，现在2个节点上手动下载pod images，以及测试使用的nginx images
  1. [root@k8s-node1 ~]# docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest
  3. [root@k8s-node1 ~]# docker pull nginx

运行nginx

  1. [root@k8s-master ~]# kubectl run my-nginx –image=nginx –replicas=2 –port=80
  2. deployment “my-nginx” created

查看pods

  1. [root@k8s-master ~]# kubectl get pods
  2. NAME READY STATUS RESTARTS AGE
  3. my-nginx-379829228-55s8n 1/1 Running 0 39s
  4. my-nginx-379829228-th5t1 1/1 Running 0 39s

可以在node节点上查看到对应的containers

  1. [root@k8s-node1 ~]# docker ps
  2. CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
  3. 1c32b101cb69 nginx “nginx -g ‘daemon off” About a minute ago Up About a minute k8s_my-nginx.a65fe6c_my-nginx-379829228-55s8n_default_7d5085b9-f035-11e7-a6c1-000c29183fc3_edf658c9
  4. de9936bea577 registry.access.redhat.com/rhel7/pod-infrastructure:latest “/usr/bin/pod” About a minute ago Up About a minute k8s_POD.a8590b41_my-nginx-379829228-55s8n_default_7d5085b9-f035-11e7-a6c1-000c29183fc3_86d80c42
  7. [root@k8s-node2 ~]# docker ps
  8. CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
  9. c064a3fef6d3 nginx “nginx -g ‘daemon off” 2 minutes ago Up 2 minutes k8s_my-nginx.a65fe6c_my-nginx-379829228-th5t1_default_7d50757d-f035-11e7-a6c1-000c29183fc3_a2431e86
  10. 85e437bc7eb7 registry.access.redhat.com/rhel7/pod-infrastructure:latest “/usr/bin/pod” 2 minutes ago Up 2 minutes k8s_POD.a8590b41_my-nginx-379829228-th5t1_default_7d50757d-f035-11e7-a6c1-000c29183fc3_419a9356

——————— 作者：bobpen 来源：CSDN 原文：https://blog.csdn.net/bobpen/article/details/78958675?utm_source=copy 版权声明：本文为博主原创文章，转载请附上博文链接！

By Vinny Wang

Related Post

Uncategorized

html:table属性以及html简单合并单元格应用

Vinny Wang
Uncategorized

处理nginx启动错误Failed to read PID from file /run/nginx.pid

Vinny Wang
Uncategorized

Systemd 入门教程：命令篇

Vinny Wang

Leave a Reply

Your email address will not be published. Required fields are marked *

You Missed

Uncategorized

html:table属性以及html简单合并单元格应用

Uncategorized

处理nginx启动错误Failed to read PID from file /run/nginx.pid

Uncategorized

Systemd 入门教程：命令篇

Uncategorized

Centos安装supervisor，守护Redis进程