A user has reported an application is unreachable due to a failing livenessProbe. Perform the following tasks: Find the broken pod and store its name and namespace to /opt/KDOB00401/broken.txt in…

Question

Exhibit

Explanation

The task requires diagnosing a failing Kubernetes pod's liveness probe, identifying the affected pod, storing its details and error events, and then modifying its manifest to implement a functional liveness probe.

Approach. The problem states that an application is unreachable due to a 'failing livenessProbe'. Since the provided YAML does not show a liveness probe, it implies either a probe was added later and is incorrect, or it's completely missing and causing issues (e.g., if another controller expected one). The most robust approach for an Nginx pod is to add a standard HTTP GET liveness probe.

Correct Interaction Steps:

1. Identify the broken pod: Use kubectl get pods to list all pods. Look for pods in a 'CrashLoopBackOff' state or with increasing restart counts. Based on the provided context (creation of nginx-configmap.yml), the likely broken pod is nginx-configmap in the default namespace. Example command: kubectl get pods.
2. Confirm liveness probe failure and gather details: Execute kubectl describe pod nginx-configmap (assuming nginx-configmap is the pod) to view detailed information, including events. In the 'Events' section, you would expect to see messages like 'Liveness probe failed: HTTP probe failed' or similar indicating the probe's failure.
3. Store pod name and namespace: Extract the pod's full name and namespace from the kubectl get pods or kubectl describe pod output. If it's nginx-configmap in the default namespace, the command is: echo 'default/nginx-configmap' > /opt/KDOB00401/broken.txt.
4. Store error events: Filter the events from the describe output for relevant error messages. The question specifically asks for 'error events' related to the failing livenessProbe. The command would be: kubectl describe pod nginx-configmap | grep -i 'liveness' > /opt/KDOB00401/error.txt (or grep -i 'error' for a broader search).
5. Fix the issue (Add/Correct Liveness Probe):
   • Open the Pod's YAML definition for editing: vim nginx_configmap.yml.
   • Navigate to the containers section, under the nginx-configmap container definition.
   • Add or modify the livenessProbe section. For a standard Nginx container, an HTTP GET probe to the root path (/) on port 80 is a reliable check. Add the following (ensuring correct YAML indentation):

       livenessProbe:
         httpGet:
           path: /
           port: 80
         initialDelaySeconds: 10 # Give the app time to start
         periodSeconds: 5
         timeoutSeconds: 1
         failureThreshold: 3
     

   • Save and exit vim (:wq).
   • Apply the updated Pod definition: kubectl apply -f nginx_configmap.yml. This will update the running pod or restart it with the new configuration.
6. Verify the fix: Run kubectl get pods again to confirm that nginx-configmap is now in a Running state without restarts.

Common mistakes.

• common_mistake. Common mistakes include:

• Incorrectly identifying the broken pod: Failing to use kubectl get pods or kubectl describe to pinpoint the exact pod and its namespace that is experiencing the liveness probe failure. Assuming the name without verification is risky.
• Misinterpreting the liveness probe failure: Trying to fix other potential pod issues (e.g., readiness probe, resource limits, image pull errors) when the problem explicitly states a 'failing livenessProbe'.
• Incorrect liveness probe configuration: Defining a liveness probe with an incorrect path for an httpGet probe (e.g., /health when Nginx only serves at /), a wrong port, or an exec command that doesn't exist or consistently fails within the container. Not giving initialDelaySeconds sufficient time can also cause probes to fail on startup.
• Not applying the changes: Editing the YAML file but forgetting to run kubectl apply -f <filename> afterward, meaning the changes are not propagated to the cluster.
• Incorrect file paths for output: Storing the pod name/namespace or error events in the wrong file path or format (/opt/KDOB00401/broken.txt and /opt/KDOB00401/error.txt with specific content formats are required).

Concept tested. This question primarily tests Kubernetes Pod troubleshooting skills, specifically related to livenessProbes. It covers:

• Kubernetes Pod lifecycle and health checks (livenessProbe vs readinessProbe).
• kubectl commands for inspecting pods (get, describe).
• Understanding and configuring liveness probes (HTTP GET, exec).
• Basic YAML editing and resource application (kubectl apply).
• File manipulation in a Linux terminal (echo, redirection, grep).
• General Kubernetes object management (Pods, ConfigMaps) and their interaction.

Reference. https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

No community discussion yet for this question.