Understanding Kubernetes Pods

Pods

The Pod is the smallest compute unit in Kubernetes. It is a Kubernetes object representing one or more containers. Containers inside a Pod share the same IP address, which is the IP address of the Pod.

The Pods containers can communicate with each other locally on any port, using the 127.0.0.1 IP address and mount volumes declared at the Pod level.

InitContainers are special Pods containers that are used for initialization tasks. They are executed prior to containers being started. If any init container fails, the Pod is considered to have failed and is handled according to its restartPolicy

Pods can also have sidecar containers that run alongside the main container to provide additional fonctionalities like logging, monitoring, data sync… Here is the Pod API reference for all the fields we can use inside Pods manifests.

Pods lifecycle

Scheduling

• Kubernetes Pods are managed by the kubelet component
• When a Pod is created, the first thing Kublet does, is to select a node on which to run the Pod (scheduling)
• The selected node must meet the Pods containers resources requests requirements, node selectors (nodeSelector, nodeName), affinity and anti-affinity and topology spread constraints criteria
• To tell Kubelet not to schedule a created Pod, we can add one or more scheduling gates to the Pod, as shown in this example

Pods statuses

The Pod object has a status field that takes its values from the phase field of the PodStatus object, containing information about the status of a Pod. The available phases, representing the statuses of a Pod (Pending, Running, Succeeded, Failed, Unknown) are described here. In the below command output:

$ kubectl get pods -n kube-system
NAME                                  READY   STATUS    RESTARTS        AGE
coredns-55cb58b774-xrqzp              1/1     Running   0               4d5h
(...)

the STATUS column possible values include values from Pods statuses, plus additional values like:

• CrashLoopBackOff: when the pod is failing to restart repeatedly
• Terminating: when a pod is being deleted

$ kubectl describe pods/coredns-55cb58b774-xrqzp -n kube-system
Name:                 coredns-55cb58b774-xrqzp
Namespace:            kube-system
(...)
Status:               Running
(...)

Pods conditions

The PodStatus object also has an array of PodConditions through which the Pod has or has not passed. For details about conditions managed by Kubelet, have a look at pod-conditions. The conditions are shown in the output of a kubectl describe command on a Pod.

Pods containers states, restart policies and probes

Containers of a Pod have states that are tracked by Kubernetes, in order for the Kubelet to be able for instance to restart containers to handle some kind of faults. Those states are currently Waiting, Running, and Terminated. Have a look at Pods containers states for details. In case of failure, containers are restarted according to their restart policy.

Kubelet also periodically performs some diagnostics on Pods containers, called probes to determine for instance, if they have started (startupProbe), are running properly (livenessProbe) or are ready to receive traffic through Kubernetes services endpoints (readinessProbe). Containers probes are customizable and the probes mechanims used to perform the diagnostics are either code execution (exec) or network requests (httpGet, tcpSocket, grpc).

Here is an example configuration manifest for probes and restart policy on a Pod: pod-using-probes-and-restart-policy.

Pods containers lifecycle hooks

Kubernetes containers lifecycle hooks make containers aware of events in their management lifecycle. Hooks are executed on some containers management events like creation and shutdown. Currently, two hooks are exposed to containers:

• PostStart: executed after container creation, before the container enters the Running state
• PreStop: executed before container termination, before the container enters the Terminated state

Have a look at Container hooks for details. Containers can handle hooks and run code when hooks are executed. Currently available hooks handlers can be found at lifecycle-hooks-handlers. Have a look at hook-handler-implementations for more about hooks handlers.

For details about hook handler executions (how hook handlers are executed according to the hook actions: httpGet, exec, sleep... have a look at
hook-handler-execution.

For details about hooks delivery guarantees (guarantees about Kubernetes sending hooks during container lifecycle management events), have a look at hook-delivery-guarantees.

For an example implementation of lifecycle hooks actions on a Pods container, have a look at pod-lifecycle-hook-manifest.

Pods deletion

kubectl delete command by default will do a graceful kill. Which means that a SIGTERM signal will be sent to the first process (PID 1) inside the container(s) of the targeted pod. That's what happens when pods are deleted inside a kubernetes cluster (with kubectl delete or kubectl scale commands, or by HPA or whatever).

A grace period of 30s by default will then be observed, in order to give time to the targeted process to stop its services (not accept new requests) and finish already accepted requests. When the grace period expires, the targeted process will be violently killed. When using kubectl command, the grace period can be passed as an option to override the default value. You can also set the grace period in your pods declarations:

spec:
  terminationGracePeriodSeconds: 60 # in seconds

Not every apps handle SIGTERM properly, so make sure your enderlying app will gracefully shutdown when receiving SIGTERM signal.

If your app doesn't gracefully shutdown after receiving SIGTERM signal, you can use the preStop containers lifecycle hook to execute a specific graceful shutdown command before sending the SIGTERM signal to the pod container first process.

The preStop command is blocking, which means that it must complete before the call to delete the pod is sent. For instance, nginx processes will exit immediately when receiving SIGTERM signals instead of exit gracefully... and gracefully exit when they receive a QUIT signal. So for your nginx pods to gracefully stop, we can for instance do the following:

(...)
spec:
  containers:
  (...)
    lifecycle:
      preStop:
        exec:
          command: ["/usr/sbin/nginx","-s","quit"]