Prerequisites
- Kubernetes version 1.14 or higher is installed.
- Recommended installed version of OpenEBS 1.9.0 or above. If OpenEBS is not installed in your setup, it can be installed using the steps mentioned here.
- A CSPC cluster. The steps for provisioning CSPC on a OpenEBS cluster can be found here.
- iSCSI initiator utils to be installed and active on all the worker nodes.
- You have access to install RBAC components into
kube-system
namespace. The OpenEBS cStor CSI driver components are installed inkube-system
namespace to allow them to be flagged as system critical components. - Supported base OS on worker nodes can be Ubuntu 16.04, Ubuntu 18.04 or CentOS.
- Install OpenEBS CSI Driver if OpenEBS is not installed via Kubera. The installation steps on the supported base OS can be seen here.
- (Optional) If Kubernetes version is1.14, then you need to enable the feature gates
ExpandCSIVolumes
andExpandInUsePersistentVolumes
onkubelet
in each worker node. For eg, in case of GKE worker nodes, add the following entries into/etc/default/kubelet
.KUBELET_EXTRA_ARGS=--feature-gates=CSINodeInfo=true,CSIDriverRegistry=true,ExpandInUsePersistentVolumes=true,ExpandCSIVolumes=true,KubeletPluginsWatcher=trueVolumeSnapshotDataSource=true --allow-privileged=true
After the above change, the following steps have to be performed to reflect the above changes.sudo service kubelet restart sudo service kubelet status
- (Optional) If Kubernetes version 1.14, then you need to enable the feature gates
ExpandCSIVolumes
andExpandInUsePersistentVolumes
onkube-apiserver
in the master node. In the case of GKE, this step is not required.
Overview
- Get the details of CSPC which is running in the cluster.
- Create a cStor StorageClass with cStor CSI provisioner.
- Run your application on cStor volume provisioned via CSI Provisioner.
STEP 1
Get the details of CSPC created in your cluster. The default namespace where OpenEBS CSPC is getting installed is openebs
. This can be obtained using the following command:
kubectl get cspc -n openebs
Sample Output:
NAME AGE
cstor-disk-pool 4m58s
STEP 2
Create a Storage Class to dynamically provision cStor volumes using cStor CSI provisioner. You can save the following sample StorageClass YAML spec as cstor-csi-sc.yaml
.
kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: openebs-csi-cstor-disk provisioner: cstor.csi.openebs.io allowVolumeExpansion: true parameters: cas-type: cstor replicaCount: "3" cstorPoolCluster: cstor-disk-pool
In the above StorageClass, the following can be changed based on your setup.
metadata.name : StorageClass name which will be used in corresponding PVC spec or VolumeClaimTemplate.
parameters.replicaCount : This defines the number of cStor volume replica. If you have a 3 node cluster, recommend using replicaCount as 3.
parameters.cstorPoolCluster : Name of the CSPC cluster that you have created using Kubera.
After the required modification, apply the StorageClass. In this example, the following command will install the StorageClass.
kubectl apply -f cstor-csi-sc.yaml
Get the newly created StorageClass details using the following command:
kubectl get sc
Example output:
NAME PROVISIONER AGE
openebs-csi-cstor-disk cstor.csi.openebs.io 6s
openebs-device openebs.io/local 18m
openebs-hostpath openebs.io/local 18m
openebs-jiva-default openebs.io/provisioner-iscsi 18m
openebs-snapshot-promoter volumesnapshot.external-storage.k8s.io/snapshot-promoter 18m
standard (default) kubernetes.io/gce-pd 28m
STEP 3
Run your application by specifying the above created StorageClass in your PVC spec. Sample application YAML can be downloaded using the following command:
wget https://raw.githubusercontent.com/openebs/cstor-csi/v1.9.x/examples/busybox-csi-cstor-sparse.yaml
Modify the YAML spec with the required PVC storage size, storageClassName. In this example, storageClassName
is updated with openebs-csi-cstor-disk
.
The following example launches a busybox app with a cStor Volume provisioned via CSI Provisioner.
kubectl apply -f busybox-csi-cstor-sparse.yaml
Now the application will be running on the volume provisioned via cStor CSI provisioner. Verify the status of the PVC using the following command:
kubectl get pvc
Example output:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
minio-pv-claim1 Bound pvc-ba839b4a-6234-4e0f-8578-829acad257ca 20Gi RWO openebs-csi-cstor-disk 7s
Verify the status of the application pod using the following command:
kubectl get pod
Example output:
NAME READY STATUS RESTARTS AGE
minio-deployment-75f64fd747-h9k4c 1/1 Running 0 40m
Verify the status of the cStor volume using the following command:
kubectl get cstorvolume -n openebs
Example output:
NAME STATUS AGE CAPACITY
pvc-39f7d3c3-1072-4522-bdaf-685dd4fd1aa5 Healthy 39m 20Gi