一个伪linux粉丝的blog

  1. 首页
  2. unix/linux
  3. 正文

kubernetes namespace delete stuck in Terminating state

7 12 月, 2020 879点热度 0人点赞 0条评论

第二次遇到客户咨询k8s无法删除namespace命名空间,卡在terminating 状态,这里记录一下吧,原因不讨论了,复现方式和解决方法其实都来自这2个地址。

https://github.com/kubernetes/kubernetes/issues/77086
https://github.com/kubernetes/kubernetes/issues/60807 这个issue是3天前关闭的,推荐看这个。

解决方法

做了几次复现后,总结下来最快的处理方式分3行命令。

1,找到对应api地址

kubectl cluster-info

主要是找到对应api地址形式及端口,如下图,我们知道了api地址为 https://127.0.0.1:11081/api/v1/

2,得到terminating 状态的namespace, 本例子为 delete-me

3, 导出delete-me 的修改后json至临时文件

kubectl get ns delete-me -o json | jq '.spec.finalizers=[]' > tmp.json

4,运行命令修复

curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:18081/api/v1/namespaces/delete-me/finalize

实际上,下面这个命令更优化,更简单,个人推荐。

kubectl replace --raw "https://127.0.0.1:11081/api/v1/namespaces/delete-me/finalize" -f tmp.json

强化版,一行命令解决 :smile:

1
2
3
4
for NS in $(kubectl get ns 2>/dev/null | grep Terminating | cut -f1 -d ' '); do
  kubectl get ns $NS -o json | jq '.spec.finalizers=[]' > /tmp/tmp.json
  kubectl replace --raw "/api/v1/namespaces/$NS/finalize" -f /tmp/$NS.json
done

 

正确的打开方式

重要更新 :eek: :smile: 

以上方式虽然能解决问题,但是根据k8s大佬在上周(2020.12.02和 12.04)的说法,都是旁门左道,正确的方式是找到这个报错的api,然后删除。

类似命令如下

Again, please do not remove the finalizer, it is there for a reason. Try to instead find out which resources in the NS are pending deletion by:

Checking if any apiservice is unavailable and hence doesn't serve its resources:

kubectl get apiservice|grep False
Finding all resources that still exist via

kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get -n $your-ns-to-delete

(Kudos to Jordan for that one)
The solution to this problem is not to short-circuit the cleanup mechanism, its to find out what prevents cleanup from succeeding.

大佬在上周的 cncf 会议上还进行了视频介绍,

: 1) My namespace won't delete, help! Why does Kubernetes sometimes refuse to delete a namespace? How to diagnose and resolve the root cause.

并最终关闭了该issue https://github.com/kubernetes/kubernetes/issues/60807

这里补一张视频截图。

相关文章:

  1. kubectl-cheatsheet
  2. k8s multi cluster management with kubeconfig
  3. delete kernel from boot
  4. SanDisk Corporation introduces line of flash memory-based solid-state drives (SSDs)
标签: kubectl Terminating
最后更新:7 12 月, 2020

wanjie

这个人很懒,什么都没留下

点赞
< 上一篇
下一篇 >

文章评论

razz evil exclaim smile redface biggrin eek confused idea lol mad twisted rolleyes wink cool arrow neutral cry mrgreen drooling persevering
取消回复

This site uses Akismet to reduce spam. Learn how your comment data is processed.

归档
分类
  • network / 332篇
  • Uncategorized / 116篇
  • unix/linux / 121篇
  • 业界资讯 / 38篇
  • 公司杂事 / 11篇
  • 数码影像 / 12篇
  • 美剧 / 3篇
  • 美图共赏 / 21篇
  • 英语学习 / 3篇
标签聚合
Google Google Voice dreamhost squid Nginx 虚拟主机 gitlab kernel k8s docker ldap 浏览器 Ubuntu nexus jira openssl deepseek 网站运营 VPS debian kubectl wget 泰国 google-chrome d90 邮件归档 dreamhost空间 postgres ssh 天翼live

COPYRIGHT © 2008-2025 wanjie.info. ALL RIGHTS RESERVED.

Theme Kratos Made By Seaton Jiang