删除无法终止的namespace namespace terminating

 

https://www.ibm.com/docs/en/cloud-private/3.2.0?topic=console-namespace-is-stuck-in-terminating-state

A namespace is stuck in the Terminating state

Symptom

A namespace is stuck in the Terminating state

Cause

If a Kubernetes API extension is not available, the resources that are managed by the extension cannot be deleted. Failure to delete the API extension causes namespace deletion to fail.

Resolving the problem

Obtain the API descriptions that are not deleted

Complete the following steps to get a description of the APIs that are not deleted:

1. View the namespaces that are stuck in a Terminating state:

   kubectl get namespaces
   

2. Find the resources that are not deleted:

   kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get --show-kind --show-all --ignore-not-found -n <terminating-namespace>
   

3. If the previous command returns the following error message: unable to retrieve the complete list of server APIs: <api-resource>/<version>: the server is currently unable to handle the request, continue to run the following command with the information you received:

   kubectl get APIService <version>.<api-resource>
   

   For example, run the following command for an API service that is named custom.metrics.k8s.io/v1beta1:

   kubectl get APIService v1beta1.custom.metrics.k8s.io
   

4. Get a description of the API service to continue to debug your API service. Run the following command:

   kubectl describe APIService <version>.<api-resource>
   

5. Make sure that the issue is resolved. Run the following command to verify that your namespace can be deleted:

   kubectl get namespace
   

Manually delete a terminating namespace

If the issue is not resolved, you can manually delete your namespace that is stuck in the Terminating state.

1. View the namespaces that are stuck in the Terminating state:

    kubectl get namespaces
   

2. Select a terminating namespace and view the contents of the namespace to find out the finalizer:

    kubectl get namespace <terminating-namespace> -o yaml
   

   Your YAML contents might resemble the following output:

    apiVersion: v1
    kind: Namespace
    metadata:
      creationTimestamp: 2018-11-19T18:48:30Z
      deletionTimestamp: 2018-11-19T18:59:36Z
      name: <terminating-namespace>
      resourceVersion: "1385077"
      selfLink: /api/v1/namespaces/<terminating-namespace>
      uid: b50c9ea4-ec2b-11e8-a0be-fa163eeb47a5
    spec:
      finalizers:
      - kubernetes
    status:
      phase: Terminating
   

3. Create a temporary JSON file:

    kubectl get namespace <terminating-namespace> -o json >tmp.json
   

4. Edit your tmp.json file. Remove the kubernetes value from the finalizers field and save the file.

   Your tmp.json file might resemble the following output:

     {
         "apiVersion": "v1",
         "kind": "Namespace",
         "metadata": {
             "creationTimestamp": "2018-11-19T18:48:30Z",
             "deletionTimestamp": "2018-11-19T18:59:36Z",
             "name": "<terminating-namespace>",
             "resourceVersion": "1385077",
             "selfLink": "/api/v1/namespaces/<terminating-namespace>",
             "uid": "b50c9ea4-ec2b-11e8-a0be-fa163eeb47a5"
         },
         "spec": {
            "finalizers": 
         },
         "status": {
             "phase": "Terminating"
         }
     }
   

5. To set a temporary proxy IP and port, run the following command. Be sure to keep your terminal window open until you delete the stuck namespace:

    kubectl proxy
   

   Your proxy IP and port might resemble the following output:

    Starting to serve on 127.0.0.1:8001
   

6. From a new terminal window, make an API call with your temporary proxy IP and port:

    curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/<terminating-namespace>/finalize
   

   Your output might resemble the following content:

    {
      "kind": "Namespace",
      "apiVersion": "v1",
      "metadata": {
        "name": "<terminating-namespace>",
        "selfLink": "/api/v1/namespaces/<terminating-namespace>/finalize",
        "uid": "b50c9ea4-ec2b-11e8-a0be-fa163eeb47a5",
        "resourceVersion": "1602981",
        "creationTimestamp": "2018-11-19T18:48:30Z",
        "deletionTimestamp": "2018-11-19T18:59:36Z"
      },
      "spec": {
   
      },
      "status": {
        "phase": "Terminating"
      }
   }
   

   Note: The finalizer parameter is removed.

7. Verify that the terminating namespace is removed:

    kubectl get namespaces
   

8. Continue to follow the steps for other namespaces that are stuck in the Terminating state.