How to provision a sample application on a CSI based cStor volume on a CSPC?

Prerequisites

1. Kubernetes version 1.14 or higher is installed.
2. Recommended installed version of OpenEBS 1.9.0 or above. If OpenEBS is not installed in your setup, it can be installed using the steps mentioned here.
3. A CSPC cluster. The steps for provisioning CSPC  on a OpenEBS cluster can be found here.
4. iSCSI initiator utils to be installed and active on all the worker nodes.
5. You have access to install RBAC components into kube-system namespace. The OpenEBS cStor CSI driver components are installed in kube-system namespace to allow them to be flagged as system critical components.
6. Supported base OS on worker nodes can be Ubuntu 16.04, Ubuntu 18.04 or CentOS.
7. Install OpenEBS CSI Driver if OpenEBS is not installed via Kubera. The installation steps on the supported base OS can be seen here.
8. (Optional) If Kubernetes version is1.14, then you need to enable the feature gates ExpandCSIVolumes and ExpandInUsePersistentVolumes on kubelet in each worker node. For eg, in case of GKE worker nodes, add the following entries into /etc/default/kubelet.

   KUBELET_EXTRA_ARGS=--feature-gates=CSINodeInfo=true,CSIDriverRegistry=true,ExpandInUsePersistentVolumes=true,ExpandCSIVolumes=true,KubeletPluginsWatcher=trueVolumeSnapshotDataSource=true  --allow-privileged=true

   After the above change, the following steps have to be performed to reflect the above changes.

   sudo service kubelet restart
   sudo service kubelet status

9. (Optional) If Kubernetes version 1.14, then you need to enable the feature      gates ExpandCSIVolumes and ExpandInUsePersistentVolumes on kube-apiserver in the master node. In the case of GKE, this step is not required.

 Overview

1. Get the details of CSPC which is running in the cluster.
2. Create a cStor StorageClass with cStor CSI provisioner.
3. Run your application on cStor volume provisioned via CSI Provisioner.

STEP 1

Get the details of CSPC created in your cluster. The default namespace where OpenEBS CSPC is getting installed is openebs. This can be obtained using the following command:

kubectl get cspc -n openebs

 Sample Output:

NAME              AGE
cstor-disk-pool   4m58s

STEP 2

Create a Storage Class to dynamically provision cStor volumes using cStor CSI provisioner. You can save the following sample StorageClass YAML spec as cstor-csi-sc.yaml.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: openebs-csi-cstor-disk
provisioner: cstor.csi.openebs.io
allowVolumeExpansion: true
parameters:
  cas-type: cstor
  replicaCount: "3"
  cstorPoolCluster: cstor-disk-pool

 In the above StorageClass, the following can be changed based on your setup.

metadata.name : StorageClass name which will be used in corresponding PVC spec or VolumeClaimTemplate.

parameters.replicaCount : This defines the number of cStor volume replica. If you have a 3 node cluster, recommend using replicaCount as 3.

parameters.cstorPoolCluster : Name of the CSPC cluster that you have created using Kubera.

After the required modification, apply the StorageClass. In this example, the following command will install the StorageClass.

kubectl apply -f cstor-csi-sc.yaml

 Get the newly created StorageClass details using the following command:

kubectl get sc

 Example output:

NAME                        PROVISIONER                                                AGE
openebs-csi-cstor-disk      cstor.csi.openebs.io                                       6s
openebs-device              openebs.io/local                                           18m
openebs-hostpath            openebs.io/local                                           18m
openebs-jiva-default        openebs.io/provisioner-iscsi                               18m
openebs-snapshot-promoter   volumesnapshot.external-storage.k8s.io/snapshot-promoter   18m
standard (default)          kubernetes.io/gce-pd                                       28m

STEP 3

Run your application by specifying the above created StorageClass in your PVC spec. Sample application YAML can be downloaded using the following command:

wget https://raw.githubusercontent.com/openebs/cstor-csi/v1.9.x/examples/busybox-csi-cstor-sparse.yaml

 Modify the YAML spec with the required PVC storage size, storageClassName. In this example, storageClassName is updated with openebs-csi-cstor-disk.

The following example launches a busybox app with a cStor Volume provisioned via CSI Provisioner.

kubectl apply -f busybox-csi-cstor-sparse.yaml

Now the application will be running on the volume provisioned via cStor CSI provisioner. Verify the status of the PVC using the following command:

kubectl get pvc

Example output:

NAME              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS             AGE
minio-pv-claim1   Bound    pvc-ba839b4a-6234-4e0f-8578-829acad257ca   20Gi       RWO            openebs-csi-cstor-disk   7s

 Verify the status of the application pod using the following command:

kubectl get pod

Example output:

NAME                                READY   STATUS    RESTARTS   AGE
minio-deployment-75f64fd747-h9k4c   1/1     Running   0          40m

 Verify the status of the cStor volume using the following command:

kubectl get cstorvolume -n openebs

Example output:

NAME                                       STATUS    AGE   CAPACITY
pvc-39f7d3c3-1072-4522-bdaf-685dd4fd1aa5   Healthy   39m   20Gi