Tutorials:Persistent volumes on the Kubernetes cluster

Persistent volumes

A persistent volume in Kubernetes is a cluster resource which can be requested by a container. For this, you have to claim a persistent volume (PV) using a persistent volume claim (PVC), which you apply in your namespace. The persistent volume claim can then be mounted to directories within a container. The important point is that the PVC survives the end of the container, i.e. the data in the PV will be permanent until the PVC is released. If the PVC is mounted again to a new container, the data will still be present.

On the cluster, there are two types of persistent volumes currently configured:

• Local persistent volumes
• Host directories

Local persistent volumes should be used to import training data and store results and log files of your training. There are special PVs for monitoring the training using Tensorboard. Host directories are meant for common training data sets stored permanently on the host. They are always read only.

Local persistent volumes

These are persistent volumes which are mapped to special folders of the host filesystem of the node. Each node exposes several persistent volumes which can be claimed. The user can not control exactly which volume is bound to a claim, but can request a minimum size. A persistent volume claim (PVC) for a local PV is configured like this:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  # the name of the PVC, we refer to this in the container configuration
  name: tf-mnist-pvc

spec:
  resources:
    requests:
      # storage resource request. This PVC can only be bound to volumes which
      # have at least 8 GiB of storage available.
      storage: 8Gi

  # the requested storage class, see tutorial.
  storageClassName: local-ssd

  # leave these unchanged, they must match the PV type, otherwise binding will fail
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem

The following storage classes are configured in the cluster:

When the claim is defined to your satisfaction, apply it like this:

> kubectl apply -f pvc.yml

You can check on the status of this (and every other) claim:

> kubectl get pvc
NAME           STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
tf-mnist-pvc   Pending                                      local-ssd      11s

Since the claim has not been used by a container yet, it is not yet bound to a persitent volume (PV).

Host directories

Large training data sets which are required by many different users are stored permanently in the filesystem of several nodes. They can be claimed with a PVC as follows:

> kubectl config use-context me@ccu