
The following documentation is intended to explain the procedure for deploying Dynamic NFS Provisioning in Kubernetes.
Dynamic NFS Provisioning: is allows storage volumes to be created on-demand. The dynamic provisioning feature eliminates the need for cluster administrators to code-provision storage. Instead, it automatically provisions storage when it is requested by users.
Code-Requisites for Dynamic NFS Provisioning in Kubernetes
- Linux Work Station
- K8 Cluster with no other load balancer installed
- Kubernetes-cli or kubectl program
- Kubernetes version v1.15.1 (any version should work)
- Routable IP network with DHCP configured
- Helm Package Manager installed
- Tiller Service Running
For a tutorial on Building a K8s Cluster using Vagrant visit: Building a Kubernetes Cluster using Vagrant
For a tutorial on Installing Helm Package Manager (Tiller) visit: Deploying Helm (Package Manager) in Kubernetes.
Step 1) Installing the NFS Server
In this particular example, we'll allocate a local filesystem from which PersistenceVolume Claims can be made. We'll first create "/srv/nfs/kubedata"
[vagrant@kmaster ~]$ sudo mkdir /srv/nfs/kubedata -p
Change the ownership to "nfsnobody"
[vagrant@kmaster ~]$ sudo chown nfsnobody: /srv/nfs/kubedata/
Next, install the NFS-utils. This example is for centos 7
vagrant@kmaster ~]$ sudo yum install -y nfs-utils
Next, enable and start the userspace NFS server using systemctl.
[vagrant@kmaster ~]$ sudo systemctl enable nfs-server
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
[vagrant@kmaster ~]$ sudo systemctl start nfs-server
[vagrant@kmaster ~]$ sudo systemctl status nfs-server
● nfs-server.service - NFS server and services
Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor codeset: disabled)
Active: active (exited) since Sat 2019-11-02 22:06:49 UTC; 12s ago
Next, we need to edit the exports file to add the file system we created to be exported to remote hosts.
[vagrant@kmaster ~]$ sudo vi /etc/exports
/srv/nfs/kubedata *(rw,sync,no_subtree_check,no_root_squash,no_all_squash,insecure)
Next, run the exportfs command to make the local directory we configured available to remote hosts.
[vagrant@kmaster ~]$ sudo exportfs -rav
exporting *:/srv/nfs/kubedata
If you want to see more details about our export file system, you can run "exportfs -v".
vagrant@kmaster ~]$ sudo exportfs -v
/srv/nfs/kubedata
<world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,insecure,no_root_squash,no_all_squash)
Next, let's test the nfs configurations. Log onto one of the worker nodes and mount the NFS filesystem andverify.
[vagrant@kworker1 ~]$ sudo mount -t nfs 172.42.42.100:/srv/nfs/kubedata /mnt
[vagrant@kworker1 ~]$ mount | grep kubedata
172.42.42.100:/srv/nfs/kubedata on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=262144,wsize=262144,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=172.42.42.101,local_lock=none,addr=172.42.42.100)
After verifying that NFS is configured correctly and working we can unmount the filesystem.
[vagrant@kworker1 ~]$ sudo umount /mnt)
Step 2) Deploying Service Account and Role Bindings
Next, we'll configure a service account and role bindings. We'll use role-based access control to do the configuration. The first step is to download the NFS-provisioning repo and change into the NFS-provisioning directory.
git clone https://exxsyseng@bitbucket.org/exxsyseng/nfs-provisioning.git
cd nfs-provisioning
In this directory, we have 4 files. (class.yaml default-sc.yaml deployment.yaml rbac.yaml) We will use the rbac.yaml file to create the service account for NFS and cluster roles and bindings.
[vagrant@kmaster nfs-provisioning]$ kubectl create -f rbac.yaml
We can verify that the service account, clusterrole, and binding was created.
[vagrant@kmaster nfs-provisioning]$ kubectl get clusterrole,clusterrolebinding,role,rolebinding | grep nfs
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner 20m
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner 20m
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner 20m
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner 20m
Step 3) Deploying Storage Class
Next, let's run the "class.yaml" to set up the storageclass. A storageclass provides a way for administrators to describe the “classes” of storage they offer.
Let's edit the "class.yaml" file and set both the storageclass name and the provisioner name.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: example.com/nfs
parameters:
archiveOnDelete: "false"
Once we've updated the class.yaml file we can execute the file using kubectl create
[vagrant@kmaster nfs-provisioning]$ kubectl create -f class.yaml
storageclass.storage.k8s.io/managed-nfs-storage created
Next, check that the storage class was created.
[vagrant@kmaster nfs-provisioning]$ kubectl get storageclass
NAME PROVISIONER AGE
managed-nfs-storage example.com/nfs 48s
Step 4) Deploying NFS Provisioner
Now let's deploy the NFS provisioner. But first we'll need to edit the deployment.yaml file. In this file, we'll need to specify the IP Address of our NFS Server (kmaster) 172.42.42.100.
kind: Deployment
apiVersion: apps/v1
metadata:
name: nfs-client-provisioner
spec:
selector:
matchLabels:
app: nfs-client-provisioner
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: quay.io/external_storage/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: example.com/nfs
- name: NFS_SERVER
value: 172.42.42.100
- name: NFS_PATH
value: /srv/nfs/kubedata
volumes:
- name: nfs-client-root
nfs:
server: 172.42.42.100
path: /srv/nfs/kubedata
Once we've made the changes, save the file and apply the changes by running "kubectl create".
[vagrant@kmaster nfs-provisioning]$ kubectl create -f deployment.yaml
deployment.apps/nfs-client-provisioner created
After applying the changes, we should see a pod was created for nfs-client-provisioner.
[vagrant@kmaster nfs-provisioning]$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nfs-client-provisioner-5b4f5775c7-9j2dw 1/1 Running 0 4m2s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 2d22h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nfs-client-provisioner 1/1 1 1 4m2s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nfs-client-provisioner-5b4f5775c7 1 1 1 4m2s
We can run "kubectl describe" to see more details about the pod.
[vagrant@kmaster ~]$ kubectl describe pod nfs-client-provisioner-5b4f5775c7-9j2dw
Name: nfs-client-provisioner-5b4f5775c7-9j2dw
Namespace: default
Priority: 0
Node: kworker2.example.com/172.42.42.102
Start Time: Sun, 03 Nov 2019 20:11:51 +0000
Labels: app=nfs-client-provisioner
pod-template-hash=5b4f5775c7
Annotations: cni.projectcalico.org/podIP: 192.168.136.65/32
Status: Running
IP: 192.168.136.65
IPs:
IP: 192.168.136.65
Controlled By: ReplicaSet/nfs-client-provisioner-5b4f5775c7
Containers:
nfs-client-provisioner:
Container ID: docker://95432ef4c256b48746b61f44a0292557b73abaced78342acafeae3c36681343b
Image: quay.io/external_storage/nfs-client-provisioner:latest
Image ID: docker-pullable://quay.io/external_storage/nfs-client-provisioner@sha256:022ea0b0d69834b652a4c53655d78642ae23f0324309097be874fb58d09d2919
Port: <none>
Host Port: <none>
State: Running
Started: Sun, 03 Nov 2019 20:11:56 +0000
Ready: True
Restart Count: 0
Environment:
PROVISIONER_NAME: example.com/nfs
NFS_SERVER: 172.42.42.100
NFS_PATH: /srv/nfs/kubedata
Mounts:
/persistentvolumes from nfs-client-root (rw)
/var/run/secrets/kubernetes.io/serviceaccount from nfs-client-provisioner-token-wgwct (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
nfs-client-root:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 172.42.42.100
Path: /srv/nfs/kubedata
ReadOnly: false
nfs-client-provisioner-token-wgwct:
Type: Secret (a volume populated by a Secret)
SecretName: nfs-client-provisioner-token-wgwct
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>
Step 5) Creating Persistent Volume and Persistent Volume Claims
Persistent Volume Claims are objects that request storage resources from your cluster. They’re similar to a voucher that your deployment can redeem for storage access.
Persistent Volume is a resource that can be used by a pod to store data that will persist beyond the lifetime of the pod. It is a storage volume that in this case is a NFS volume.
If we check our cluster we'll see that there are currently no Persistent Volumes or Persistent Volume Claims.
[vagrant@kmaster ~]$ kubectl get pv,pvc
No resources found in default namespace.
Also, we can look in the directory we allocated for Persistent Volumes and see there nothing there.
[vagrant@kmaster ~]$ ls /srv/nfs/kubedata/
Let's create a PVC. Inside the nfs-provisioning repo, there is a file "4-pvc-nfs.yaml". In this example, we will allocate 500 MegaBytes.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc1
spec:
storageClassName: managed-nfs-storage
accessModes:
- ReadWriteMany
resources:
requests:
storage: 500Mi
We can create the PVC by running "kubectl create" against the 4-pvc-nfs.yaml" file.
[vagrant@kmaster nfs-provisioning]$ kubectl create -f 4-pvc-nfs.yaml
persistentvolumeclaim/pvc1 created
We can now view the PVC and PV that was allocated. As we can see below a PCV was created "persistentvolumeclaim/pvc1" and its bound to a PV "pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf". The PV was created automatically by the nfs-provisoner.
[vagrant@kmaster nfs-provisioning]$ kubectl get pvc,pv
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/pvc1 Bound pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf 500Mi RWX managed-nfs-storage 2m30s
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf 500Mi RWX Delete Bound default/pvc1 managed-nfs-storage 2m30s
[vagrant@kmaster nfs-provisioning]$
Step 6) Creating a Pod to use Persistent Volume Claims
Now that we have our nfs-provisoner working and we have both a PVC and OV that it is bound to. Let's create a pod to use our PVC. If we take a quick look at the existing pods we'll see that only the "nfs-client-provisioner" pod is running.
[vagrant@kmaster nfs-provisioning]$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-5b4f5775c7-9j2dw 1/1 Running 0 4h36m
Next, we'll create a pod using the "4-busybox-pv-nfs.yaml" file. But first let's take a look at the file's contents.
apiVersion: v1
kind: Pod
metadata:
name: busybox
spec:
volumes:
- name: host-volume
persistentVolumeClaim:
claimName: pvc1
containers:
- image: busybox
name: busybox
command: ["/bin/sh"]
args: ["-c", "sleep 600"]
volumeMounts:
- name: host-volume
mountPath: /mydata
We'll execute test-pod-pvc1.yaml using "kubectl create".
[vagrant@kmaster nfs-provisioning]$ kubectl create -f 4-busybox-pv-nfs.yaml
pod/busybox created
We can now see that the pod is up and running.
[vagrant@kmaster nfs-provisioning]$ kubectl get pods
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 0 69s
nfs-client-provisioner-5b4f5775c7-9j2dw 1/1 Running 0 7h33m
We can describe the pod to see more details.
[vagrant@kmaster nfs-provisioning]$ kubectl describe pod busybox
Name: busybox
Namespace: default
Priority: 0
Node: kworker1.example.com/172.42.42.101
Start Time: Mon, 04 Nov 2019 03:44:30 +0000
Labels: <none>
Annotations: cni.projectcalico.org/podIP: 192.168.33.194/32
Status: Running
IP: 192.168.33.194
IPs:
IP: 192.168.33.194
Containers:
busybox:
Container ID: docker://f27b38404abbfd3ab77fe81b23e148e0a15f4779420ddfcb17eebcbe699767f3
Image: busybox
Image ID: docker-pullable://busybox@sha256:1303dbf110c57f3edf68d9f5a16c082ec06c4cf7604831669faf2c712260b5a0
Port: <none>
Host Port: <none>
Command:
/bin/sh
Args:
-c
sleep 600
State: Running
Started: Mon, 04 Nov 2019 03:44:34 +0000
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/mydata from host-volume (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-p2ctq (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
host-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: pvc1
ReadOnly: false
default-token-p2ctq:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-p2ctq
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
We can log into the container to view the mount point and create a file for testing
[vagrant@kmaster nfs-provisioning]$ kubectl exec -it busybox -- ./bin/sh
/ #
/ # ls /mydata/
/ # > /mydata/myfile
/ # ls /mydata/
myfile
Now that we've created a file called myfile, we can log into the mastrer node and verify the file by looking in the PV directory that was allocated for this pod.
[vagrant@kmaster nfs-provisioning]$ ls /srv/nfs/kubedata/default-pvc1-pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf/
myfile
Step 7) Deleting Pods with Persistent Volume Claims
- To delete the pod just use "kubectl delete pod [pod name]"
[vagrant@kmaster nfs-provisioning]$ kubectl delete pod busybox
pod "busybox" deleted
- Deleting the pod will delete the pod but not the PV and PVC. This will have to be done separately.
[vagrant@kmaster nfs-provisioning]$ kubectl get pvc,pv
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/pvc1 Bound pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf 500Mi RWX managed-nfs-storage 3h26m
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf 500Mi RWX Delete Bound default/pvc1 managed-nfs-storage 3h26m
- To delete the PV and PVC use "kubectl delete"
[vagrant@kmaster nfs-provisioning]$ kubectl delete pvc --all
persistentvolumeclaim "pvc1" deleted
- PV and PVC resources are gone
[vagrant@kmaster nfs-provisioning]$ kubectl get pvc,pv
No resources found in default namespace.
Final Thoughts on Dynamic NFS Provisioning in Kubernetes.
Dynamic NFS provisioning allows storage volumes to be created on-demand. Without dynamic NFS provisioning, cluster administrators have to manually create new storage volumes, and then create PersistentVolume
objects to recodesent them in Kubernetes. The dynamic NFS provisioning feature eliminates the need for cluster administrators to code-provision storage. Instead, it automatically provisions storage when it is requested by users.
Related Blogs

Deploying Dynamic NFS Provisioning in Kubernetes
The following documentation is intended to explain the procedure for deploying Dynamic NFS Provisioning in Kubernetes.
Dynamic NFS Provisioning: is allows storage volumes to be created on-demand. The dynamic provisioning feature eliminates the need for cluster administrators to code-provision storage. Instead, it automatically provisions storage when it is requested by users.
Code-Requisites for Dynamic NFS Provisioning in Kubernetes
- Linux Work Station
- K8 Cluster with no other load balancer installed
- Kubernetes-cli or kubectl program
- Kubernetes version v1.15.1 (any version should work)
- Routable IP network with DHCP configured
- Helm Package Manager installed
- Tiller Service Running
For a tutorial on Building a K8s Cluster using Vagrant visit: Building a Kubernetes Cluster using Vagrant
For a tutorial on Installing Helm Package Manager (Tiller) visit: Deploying Helm (Package Manager) in Kubernetes.
Step 1) Installing the NFS Server
In this particular example, we'll allocate a local filesystem from which PersistenceVolume Claims can be made. We'll first create "/srv/nfs/kubedata"
[vagrant@kmaster ~]$ sudo mkdir /srv/nfs/kubedata -p
Change the ownership to "nfsnobody"
[vagrant@kmaster ~]$ sudo chown nfsnobody: /srv/nfs/kubedata/
Next, install the NFS-utils. This example is for centos 7
vagrant@kmaster ~]$ sudo yum install -y nfs-utils
Next, enable and start the userspace NFS server using systemctl.
[vagrant@kmaster ~]$ sudo systemctl enable nfs-server
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
[vagrant@kmaster ~]$ sudo systemctl start nfs-server
[vagrant@kmaster ~]$ sudo systemctl status nfs-server
● nfs-server.service - NFS server and services
Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor codeset: disabled)
Active: active (exited) since Sat 2019-11-02 22:06:49 UTC; 12s ago
Next, we need to edit the exports file to add the file system we created to be exported to remote hosts.
[vagrant@kmaster ~]$ sudo vi /etc/exports
/srv/nfs/kubedata *(rw,sync,no_subtree_check,no_root_squash,no_all_squash,insecure)
Next, run the exportfs command to make the local directory we configured available to remote hosts.
[vagrant@kmaster ~]$ sudo exportfs -rav
exporting *:/srv/nfs/kubedata
If you want to see more details about our export file system, you can run "exportfs -v".
vagrant@kmaster ~]$ sudo exportfs -v
/srv/nfs/kubedata
<world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,insecure,no_root_squash,no_all_squash)
Next, let's test the nfs configurations. Log onto one of the worker nodes and mount the NFS filesystem andverify.
[vagrant@kworker1 ~]$ sudo mount -t nfs 172.42.42.100:/srv/nfs/kubedata /mnt
[vagrant@kworker1 ~]$ mount | grep kubedata
172.42.42.100:/srv/nfs/kubedata on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=262144,wsize=262144,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=172.42.42.101,local_lock=none,addr=172.42.42.100)
After verifying that NFS is configured correctly and working we can unmount the filesystem.
[vagrant@kworker1 ~]$ sudo umount /mnt)
Step 2) Deploying Service Account and Role Bindings
Next, we'll configure a service account and role bindings. We'll use role-based access control to do the configuration. The first step is to download the NFS-provisioning repo and change into the NFS-provisioning directory.
git clone https://exxsyseng@bitbucket.org/exxsyseng/nfs-provisioning.git
cd nfs-provisioning
In this directory, we have 4 files. (class.yaml default-sc.yaml deployment.yaml rbac.yaml) We will use the rbac.yaml file to create the service account for NFS and cluster roles and bindings.
[vagrant@kmaster nfs-provisioning]$ kubectl create -f rbac.yaml
We can verify that the service account, clusterrole, and binding was created.
[vagrant@kmaster nfs-provisioning]$ kubectl get clusterrole,clusterrolebinding,role,rolebinding | grep nfs
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner 20m
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner 20m
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner 20m
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner 20m
Step 3) Deploying Storage Class
Next, let's run the "class.yaml" to set up the storageclass. A storageclass provides a way for administrators to describe the “classes” of storage they offer.
Let's edit the "class.yaml" file and set both the storageclass name and the provisioner name.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: example.com/nfs
parameters:
archiveOnDelete: "false"
Once we've updated the class.yaml file we can execute the file using kubectl create
[vagrant@kmaster nfs-provisioning]$ kubectl create -f class.yaml
storageclass.storage.k8s.io/managed-nfs-storage created
Next, check that the storage class was created.
[vagrant@kmaster nfs-provisioning]$ kubectl get storageclass
NAME PROVISIONER AGE
managed-nfs-storage example.com/nfs 48s
Step 4) Deploying NFS Provisioner
Now let's deploy the NFS provisioner. But first we'll need to edit the deployment.yaml file. In this file, we'll need to specify the IP Address of our NFS Server (kmaster) 172.42.42.100.
kind: Deployment
apiVersion: apps/v1
metadata:
name: nfs-client-provisioner
spec:
selector:
matchLabels:
app: nfs-client-provisioner
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: quay.io/external_storage/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: example.com/nfs
- name: NFS_SERVER
value: 172.42.42.100
- name: NFS_PATH
value: /srv/nfs/kubedata
volumes:
- name: nfs-client-root
nfs:
server: 172.42.42.100
path: /srv/nfs/kubedata
Once we've made the changes, save the file and apply the changes by running "kubectl create".
[vagrant@kmaster nfs-provisioning]$ kubectl create -f deployment.yaml
deployment.apps/nfs-client-provisioner created
After applying the changes, we should see a pod was created for nfs-client-provisioner.
[vagrant@kmaster nfs-provisioning]$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nfs-client-provisioner-5b4f5775c7-9j2dw 1/1 Running 0 4m2s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 2d22h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nfs-client-provisioner 1/1 1 1 4m2s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nfs-client-provisioner-5b4f5775c7 1 1 1 4m2s
We can run "kubectl describe" to see more details about the pod.
[vagrant@kmaster ~]$ kubectl describe pod nfs-client-provisioner-5b4f5775c7-9j2dw
Name: nfs-client-provisioner-5b4f5775c7-9j2dw
Namespace: default
Priority: 0
Node: kworker2.example.com/172.42.42.102
Start Time: Sun, 03 Nov 2019 20:11:51 +0000
Labels: app=nfs-client-provisioner
pod-template-hash=5b4f5775c7
Annotations: cni.projectcalico.org/podIP: 192.168.136.65/32
Status: Running
IP: 192.168.136.65
IPs:
IP: 192.168.136.65
Controlled By: ReplicaSet/nfs-client-provisioner-5b4f5775c7
Containers:
nfs-client-provisioner:
Container ID: docker://95432ef4c256b48746b61f44a0292557b73abaced78342acafeae3c36681343b
Image: quay.io/external_storage/nfs-client-provisioner:latest
Image ID: docker-pullable://quay.io/external_storage/nfs-client-provisioner@sha256:022ea0b0d69834b652a4c53655d78642ae23f0324309097be874fb58d09d2919
Port: <none>
Host Port: <none>
State: Running
Started: Sun, 03 Nov 2019 20:11:56 +0000
Ready: True
Restart Count: 0
Environment:
PROVISIONER_NAME: example.com/nfs
NFS_SERVER: 172.42.42.100
NFS_PATH: /srv/nfs/kubedata
Mounts:
/persistentvolumes from nfs-client-root (rw)
/var/run/secrets/kubernetes.io/serviceaccount from nfs-client-provisioner-token-wgwct (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
nfs-client-root:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 172.42.42.100
Path: /srv/nfs/kubedata
ReadOnly: false
nfs-client-provisioner-token-wgwct:
Type: Secret (a volume populated by a Secret)
SecretName: nfs-client-provisioner-token-wgwct
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>
Step 5) Creating Persistent Volume and Persistent Volume Claims
Persistent Volume Claims are objects that request storage resources from your cluster. They’re similar to a voucher that your deployment can redeem for storage access.
Persistent Volume is a resource that can be used by a pod to store data that will persist beyond the lifetime of the pod. It is a storage volume that in this case is a NFS volume.
If we check our cluster we'll see that there are currently no Persistent Volumes or Persistent Volume Claims.
[vagrant@kmaster ~]$ kubectl get pv,pvc
No resources found in default namespace.
Also, we can look in the directory we allocated for Persistent Volumes and see there nothing there.
[vagrant@kmaster ~]$ ls /srv/nfs/kubedata/
Let's create a PVC. Inside the nfs-provisioning repo, there is a file "4-pvc-nfs.yaml". In this example, we will allocate 500 MegaBytes.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc1
spec:
storageClassName: managed-nfs-storage
accessModes:
- ReadWriteMany
resources:
requests:
storage: 500Mi
We can create the PVC by running "kubectl create" against the 4-pvc-nfs.yaml" file.
[vagrant@kmaster nfs-provisioning]$ kubectl create -f 4-pvc-nfs.yaml
persistentvolumeclaim/pvc1 created
We can now view the PVC and PV that was allocated. As we can see below a PCV was created "persistentvolumeclaim/pvc1" and its bound to a PV "pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf". The PV was created automatically by the nfs-provisoner.
[vagrant@kmaster nfs-provisioning]$ kubectl get pvc,pv
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/pvc1 Bound pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf 500Mi RWX managed-nfs-storage 2m30s
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf 500Mi RWX Delete Bound default/pvc1 managed-nfs-storage 2m30s
[vagrant@kmaster nfs-provisioning]$
Step 6) Creating a Pod to use Persistent Volume Claims
Now that we have our nfs-provisoner working and we have both a PVC and OV that it is bound to. Let's create a pod to use our PVC. If we take a quick look at the existing pods we'll see that only the "nfs-client-provisioner" pod is running.
[vagrant@kmaster nfs-provisioning]$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-5b4f5775c7-9j2dw 1/1 Running 0 4h36m
Next, we'll create a pod using the "4-busybox-pv-nfs.yaml" file. But first let's take a look at the file's contents.
apiVersion: v1
kind: Pod
metadata:
name: busybox
spec:
volumes:
- name: host-volume
persistentVolumeClaim:
claimName: pvc1
containers:
- image: busybox
name: busybox
command: ["/bin/sh"]
args: ["-c", "sleep 600"]
volumeMounts:
- name: host-volume
mountPath: /mydata
We'll execute test-pod-pvc1.yaml using "kubectl create".
[vagrant@kmaster nfs-provisioning]$ kubectl create -f 4-busybox-pv-nfs.yaml
pod/busybox created
We can now see that the pod is up and running.
[vagrant@kmaster nfs-provisioning]$ kubectl get pods
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 0 69s
nfs-client-provisioner-5b4f5775c7-9j2dw 1/1 Running 0 7h33m
We can describe the pod to see more details.
[vagrant@kmaster nfs-provisioning]$ kubectl describe pod busybox
Name: busybox
Namespace: default
Priority: 0
Node: kworker1.example.com/172.42.42.101
Start Time: Mon, 04 Nov 2019 03:44:30 +0000
Labels: <none>
Annotations: cni.projectcalico.org/podIP: 192.168.33.194/32
Status: Running
IP: 192.168.33.194
IPs:
IP: 192.168.33.194
Containers:
busybox:
Container ID: docker://f27b38404abbfd3ab77fe81b23e148e0a15f4779420ddfcb17eebcbe699767f3
Image: busybox
Image ID: docker-pullable://busybox@sha256:1303dbf110c57f3edf68d9f5a16c082ec06c4cf7604831669faf2c712260b5a0
Port: <none>
Host Port: <none>
Command:
/bin/sh
Args:
-c
sleep 600
State: Running
Started: Mon, 04 Nov 2019 03:44:34 +0000
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/mydata from host-volume (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-p2ctq (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
host-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: pvc1
ReadOnly: false
default-token-p2ctq:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-p2ctq
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
We can log into the container to view the mount point and create a file for testing
[vagrant@kmaster nfs-provisioning]$ kubectl exec -it busybox -- ./bin/sh
/ #
/ # ls /mydata/
/ # > /mydata/myfile
/ # ls /mydata/
myfile
Now that we've created a file called myfile, we can log into the mastrer node and verify the file by looking in the PV directory that was allocated for this pod.
[vagrant@kmaster nfs-provisioning]$ ls /srv/nfs/kubedata/default-pvc1-pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf/
myfile
Step 7) Deleting Pods with Persistent Volume Claims
- To delete the pod just use "kubectl delete pod [pod name]"
[vagrant@kmaster nfs-provisioning]$ kubectl delete pod busybox
pod "busybox" deleted
- Deleting the pod will delete the pod but not the PV and PVC. This will have to be done separately.
[vagrant@kmaster nfs-provisioning]$ kubectl get pvc,pv
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/pvc1 Bound pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf 500Mi RWX managed-nfs-storage 3h26m
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-eca295aa-bc2c-420c-b60e-9a6894fc9daf 500Mi RWX Delete Bound default/pvc1 managed-nfs-storage 3h26m
- To delete the PV and PVC use "kubectl delete"
[vagrant@kmaster nfs-provisioning]$ kubectl delete pvc --all
persistentvolumeclaim "pvc1" deleted
- PV and PVC resources are gone
[vagrant@kmaster nfs-provisioning]$ kubectl get pvc,pv
No resources found in default namespace.
Final Thoughts on Dynamic NFS Provisioning in Kubernetes.
Dynamic NFS provisioning allows storage volumes to be created on-demand. Without dynamic NFS provisioning, cluster administrators have to manually create new storage volumes, and then create PersistentVolume
objects to recodesent them in Kubernetes. The dynamic NFS provisioning feature eliminates the need for cluster administrators to code-provision storage. Instead, it automatically provisions storage when it is requested by users.