K8S 集群组件内存-CPU过高排查方法

1、查看服务进程,并跟踪程序系统调用

pgrep  kubelet

#查看资源占用情况

top -p 95786

strace -cp 95786

#显示时间戳

strace -tt -p 95786

 2、用pprof 性能分析工具排查

#提前安装go 环境

2.1、#kubelet 开启debug
/var/lib/kubelet/config.yaml
enableDebuggingHandlers: true
enableProfilingHandler: true
2.2、启动代理采集
kubectl proxy --port=8001 --address=0.0.0.0
#内存
curl -sK -v http://127.0.0.1:8001/api/v1/nodes/master-tg-1【节点hostname名称】/proxy/debug/pprof/heap > kubelet-mem.out
#cpu
curl -sK -v http://127.0.0.1:8001/api/v1/nodes/master-tg-1【节点hostname名称】/proxy/debug/pprof/profile?seconds=10 > kubelet-cpu.out
#命令行分析
go tool pprof kubelet-cpu.out
#页面分析
go tool pprof -http=0.0.0.0:8989 kubelet-cpu.out
2.3、kube-scheduler
#cpu
curl -sK -v http://localhost:10251/debug/pprof/profile >scheduler-cpu.out
#内存
curl -sK -v http://localhost:10251/debug/pprof/heap >scheduler-mem.out
kube-controller-manager
#cpu
curl -sK -v http://localhost:10252/debug/pprof/profile >controller-manager-cpu.out
#内存
curl -sK -v http://localhost:10252/debug/pprof/heap >controller-manager-mem.out
kube-apiserver
#启动代理采集
kubectl proxy --port=8001 --address=0.0.0.0
#cpu
curl -sK -v http://localhost:8001/debug/pprof/profile >apiserver-cpu.out
#内存
curl -sK -v http://localhost:8001/debug/pprof/heap >apiserver-mem.out