陈日志 发布的文章

dashboard组件为我们提供了一个WebUI,使得K8S集群的管理更加的方便和直观。

篇一篇二分别介绍了如果搭建Kubernetes集群和部署metrics-server指标数据监控组件,本篇介绍如何在Kubernetes集群中部署dashboard组件。

dashboardyaml部署文件位于github仓库 这里这里。同样的,作为K8S插件部署与K8S集群中。

下载yaml部署文件

wget https://raw.githubusercontent.com/kubernetes/kubernetes/master/cluster/addons/dashboard/dashboard.yaml

设置登录过期时间

默认情况下在dashboard界面空闲几分钟没有操作就自动登出了,我们可以通过修改启动参数来修改这个自动登出时长。

修改dashboard.yaml,添加启动参数--token-ttl=86400,即无操作24小时后自动登出

          args:
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
            - --token-ttl=86400

- 阅读剩余部分 -

Kubernetes v1.12之前的版本,资源指标监控主要是heapster来实现,在Kubernetes v1.12之后的版本,heapster逐渐被废弃了,取而代之的是metrics-server

metrics-server作为k8s插件部署在k8s集群内,提供资源指标监控数据。其yaml部署文件在这里

但是这个yaml清单是没办法直接使用的,会报一些错误,下面我记录下来都会遇到哪些问题以及其解决方法。

下载yaml部署清单

mkdir metrics-server
cd metrics-server

for file in auth-delegator.yaml auth-reader.yaml metrics-apiservice.yaml metrics-server-deployment.yaml metrics-server-service.yaml resource-reader.yaml; do
  wget https://raw.githubusercontent.com/kubernetes/kubernetes/master/cluster/addons/metrics-server/$file
done

metrics-server-deployment.yaml文件中使用了两个镜像,镜像地址是k8s.gcr.io,由于国内网络限制,改为阿里云的镜像地址

  • registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6
  • registry.cn-hangzhou.aliyuncs.com/google_containers/addon-resizer:1.8.11

metrics-server-deployment.yaml中metrics-server-nanny镜像的启动参数有几个变量,不能直接使用,需要修改如下:

        command:
          - /pod_nanny
          - --config-dir=/etc/config
          - --cpu=100m
          - --extra-cpu=20m
          - --memory=100Mi
          - --extra-memory=10Mi
          - --threshold=5
          - --deployment=metrics-server-v0.3.6
          - --container=metrics-server
          - --poll-period=300000
          - --estimator=exponential
          # Specifies the smallest cluster (defined in number of nodes)
          # resources will be scaled to.
          - --minClusterSize=2
          # Use kube-apiserver metrics to avoid periodically listing nodes.
          - --use-metrics=true

- 阅读剩余部分 -

环境:

K8S版本:v1.19.3

名称系统IP主机名
MasterCentOS 7.8.2003 x86_64172.16.135.10k8s-master01
Node01CentOS 7.8.2003 x86_64172.16.135.11k8s-node01
Node02CentOS 7.8.2003 x86_64172.16.135.12k8s-node02
Node03CentOS 7.8.2003 x86_64172.16.135.13k8s-node03

备注:此集群仅用于测试使用,不可用户生产环境。生产环境需要考虑Master节点的高可用,且需要做适当的优化,后面可能会写一篇文章单独说

环境配置

时间同步

K8S集群应保持时间同步,避免出现奇怪的问题。

# 在所有节点上执行
yum install chrony
systemctl start chronyd
systemctl enable chronyd

关闭swap

k8s默认不允许主机开启swap,这里将其关闭。如果需要启用swap忽略错误,需要在kubelet启动参数上添加--ignore-preflight-errors=Swap。或者修改/etc/sysconfig/kubeletKUBELET_EXTRA_ARGS="--fail-swap-on=false"

关闭swap

# 在所有节点上执行
# 修改 /etc/fstab,注释swap一行
# 然后执行swapoff命令立即关闭swap
swapoff -a

- 阅读剩余部分 -

文章出处,做了一些修改

在一次例行的安全扫描中有发现安全风险提示:SSH服务配置不建议使用arcfour流密码或无任何密码,RFC 4253 不建议使用arcfour弱加密算法,仅使用CTR模式加密算法,如AES-CTR。所以花了一些时间来简单地查找资源并分析了相关的原因,在此写下笔记。

为了确保信息的传输,SSH 在事务中的各个点采用了许多不同类型的数据操作技术。这些包括对称加密,非对称加密和散列的形式。如果配置为CBC模式的话,OpenSSH没有正确地处理分组密码算法加密的SSH会话中所出现的错误,导致可能泄露密文中任意块最多32位纯文本。在以标准配置使用OpenSSH时,攻击者恢复32位纯文本的成功概率为2^{-18},此外另一种攻击变种恢复14位纯文本的成功概率为2^{-14}。查询本机上的ssh服务所支持并启用了的算法:

sshd -T | grep ciphers | perl -pe 's/,/\n/g' | sort -u
sshd -T | grep "\(ciphers\|macs\|kexalgorithms\)"

SSH的配置文件中加密算法没有指定的话,较低版本的ssh-server默认支持所有加密算法,包括arcfour,arcfour128,arcfour256等这些弱加密算法,这就可能会导致安全风险。sshd -T会将显示全量的配置,ssh -Q会查询这些配置中分类的一些参数方法等:

[-Q cipher | cipher-auth | mac | kex | key]

查询支持的加密算法:

ssh -Q cipher-auth    #启用的
ssh -Q cipher    #所有的

- 阅读剩余部分 -

Kubernetes 1.18.x版本部署metrics-server组件时,采集不到数据

表现为:

  • kubectl top nodeskubectl top nodes报错信息如下

    Error from server (ServiceUnavailable): the server is currently unable to handle the request (get nodes.metrics.k8s.io)
    Error from server (ServiceUnavailable): the server is currently unable to handle the request (get pods.metrics.k8s.io)

查看kube-apiserver的日志信息
kubectl -n kube-system logs -f kube-apiserver-master-1 --tail 10

E0826 04:25:10.111976       1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: Get https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0826 04:25:15.112635       1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: Get https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
W0826 04:25:37.762783       1 handler_proxy.go:102] no RequestInfo found in the context
E0826 04:25:37.762890       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0826 04:25:37.762915       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0826 04:25:41.763211       1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: Get https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0826 04:25:46.764318       1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: Get https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0826 04:26:11.763745       1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: Get https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0826 04:26:16.764339       1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: Get https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0826 04:26:41.763920       1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: Get https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0826 04:26:46.764574       1 available_controller.go:420] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: Get https://10.108.243.54:443/apis/metrics.k8s.io/v1beta1: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

这段日志看不出什么问题。

最终解决:kube-apiserver添加--enable-aggregator-routing=true启动参数,原因不明,因为在我另一个k8s 1.16.x集群中,并没有添加这项参数,工作正常。有知道原因的伙伴请不吝告知,在此谢过。

PS:网上有文章说如果master节点没有运行kube-proxy进程才需要加上这个启动参数,而我的集群中master节点是有运行kube-proxy的。

参考链接1:https://github.com/kubernetes-sigs/metrics-server/issues/448
参考链接2:https://blog.z0ukun.com/?p=1462