Mayastor – MayaData Help Center

Topics covered in this article:

Introduction
Prerequisites
Deploying Mayastor
Creating Storage Pool
Creating Storage Class
Deploying a test application

Mayastor is currently under development as a storage engine option for the Open Source CNCF project, OpenEBS. It is actively developed and maintained by MayaData, as part of the Kubera platform for data agility.

Mayastor has been developed keeping in mind the following design goals:

High availability
Durable persistence of data
Readily deployable
Low-overhead abstraction
Easy management

It has been designed from the ground up to leverage the protocol and compute efficiency of NVMe-oF semantics, and the performance capabilities of the latest generation of solid state storage devices, in order to deliver a storage abstraction with performance overhead measured to be within the range of single digit percentages.

Mayastor is beta software. It is considered largely, feature complete and substantially without major known defects. Minor and unknown defects can be expected.

Prerequisites

Each Mayastor Node, that is each cluster worker node which will host an instance of a Mayastor pod, must have these resources free and available for use by Mayastor:

Two x86-64 CPU cores with SSE4.2 instruction support:
- Intel Nehalem processor (march=nehalem) and newer
- AMD Bulldozer processor and newer
4GiB of RAM
HugePage support
- A minimum of 1GiB of 2MiB-sized huge pages
To know more about enabling HugePage support click here.
Instances of the Mayastor pod must be run in privileged mode.
The iSCSI client should be installed and correctly configured as per this guide.
Mayastor node candidates must be labeled with the OpenEBS engine type "mayastor". To add label, execute:
```
kubectl label node node1 openebs.io/engine=mayastor
```

Deploying Mayastor

We will be deploying Mayastor using the manifest files present in the deploy folder of Mayastor GitHub repository. The repository is configured for the GitFlow release pattern, wherein the master branch contains official releases.

1. Create Mayastor Application Resources

Namespace

kubectl create -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/namespace.yaml

RBAC Resources

kubectl create -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/moac-rbac.yaml

Custom Resource Definitions

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/csi/moac/crds/mayastorpool.yaml

2.Deploy Mayastor Dependencies

NATS

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/nats-deployment.yaml

To verify the deployment of the NATS application to the cluster was successful, execute:

  kubectl -n mayastor get pods --selector=app=nats

Sample Output:

NAME                   READY   STATUS    RESTARTS   AGE
nats-b4cbb6c96-nbp75   1/1     Running   0          28s

3.Deploy Mayastor Components

1. CSI Node Plugin

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/csi-daemonset.yaml

To verify that the CSI Node Plugin DaemonSet has been correctly deployed to all worker nodes in the cluster, execute:

kubectl -n mayastor get daemonset mayastor-csi

Sample Output:

NAME           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR              AGE
mayastor-csi   3         3         3       3            3           kubernetes.io/arch=amd64   26m

2. Control Plane

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/moac-deployment.yaml

To verify that the MOAC control plane pod is running, execute:

kubectl get pods -n mayastor --selector=app=moac

Sample Output:

NAME                    READY   STATUS    RESTARTS   AGE
moac-7d487fd5b5-9hj62   3/3     Running   0          8m4s

3. Data Plane

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/mayastor-daemonset.yaml

To verify that the Mayastor DaemonSet has been correctly deployed, execute:

kubectl -n mayastor get daemonset mayastor

Sample Output:

NAME       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                                         AGE
mayastor   3         3         3       3            3           kubernetes.io/arch=amd64,openebs.io/engine=mayastor   108s

For each resulting Mayastor pod instance, a Mayastor Node (MSN) custom resource definition should be created. List these definitions and verify that the count meets the expected number and that all nodes are reporting their State as online

kubectl -n mayastor get msn

Sample Output:

NAME                       STATE    AGE
aks-agentpool-12194210-0   online   8m18s
aks-agentpool-12194210-1   online   8m19s
aks-agentpool-12194210-2   online   8m15s

Creating Storage Pool

We need at least one pool that has to be created and hosted by one of the Mayastor Nodes. While creating pools ensure that the number of pools configured should be no lower than the desired maximum replication factor of the PVs to be created, for example, the minimum viable configuration for a Mayastor deployment which is intended to test 3-way mirrored PVs must have three Mayastor Nodes, each having one Mayastor Pool, with each of those pools having one unique block device allocated to it.

For the current version of Mayastor, a pool may have only a single block device member, which constitutes the entire data persistence layer for that pool and thus determines its maximum capacity.

NOTE:Replace the values for the fields "name", "node" and "disks" as appropriate to your cluster's intended configuration.

cat <<EOF | kubectl create -f -
apiVersion: "openebs.io/v1alpha1"
kind: MayastorPool
metadata:
  name: pool-on-node-1
  namespace: mayastor
spec:
  node: workernode-1-hostname
  disks: ["/dev/sdx"]
EOF

cat <<EOF | kubectl create -f -
apiVersion: "openebs.io/v1alpha1"
kind: MayastorPool
metadata:
  name: nvme-pool-on-node-2
  namespace: mayastor
spec:
  node: workernode-2-hostname
  disks: ["nvme://nqn.2014-08.com.vendor:nvme:nvm-subsystem-sn-d78432"]
EOF

cat <<EOF | kubectl create -f -
apiVersion: "openebs.io/v1alpha1"
kind: MayastorPool
metadata:
  name: iscsi-pool-on-node-3
  namespace: mayastor
spec:
  node: workernode-3-hostname
  disks: ["iscsi://iqn.yyyy-mm.naming-authority:unique_name"]
EOF

To verify the status of the pool(s), execute:

kubectl -n mayastor get msp

Sample Output:

NAME             NODE                       STATE    AGE
pool-on-node-0   aks-agentpool-12194210-0   online   127m
pool-on-node-1   aks-agentpool-12194210-1   online   27s
pool-on-node-2   aks-agentpool-12194210-2   online   4s

Creating Storage Class

Mayastor dynamically provisions Persistent Volumes based on StorageClass definition. In the current version of Mayastor, StorageClass definitions are used to control both, which transport protocol is used to mount the PV to the worker node hosting the consuming application pod (iSCSI, or NVMe-oF TCP) and the level of data protection afforded to it (that is, the number of synchronous data replicas which are maintained, for purposes of redundancy). It is possible to create any number of custom StorageClass definitions to span this range of permutations. Execute the below example to deploy StorageClass

Set the value of protocol and name as per requirement. Permissible values for the field "protocol" are either "iscsi", or "nvmf"

cat <<EOF | kubectl create -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mayastor-iscsi
parameters:
  # Set the number of data replicas ("replication factor")
  repl: '1'
  # Set the export transport protocol
  protocol: 'iscsi'
provisioner: io.openebs.csi-mayastor
EOF

cat <<EOF | kubectl create -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mayastor-nvmf
parameters:
  # Set the number of data replicas ("replication factor")
  repl: '3'
  # Set the export transport protocol
  protocol: 'nvmf'
provisioner: io.openebs.csi-mayastor
EOF

Deploying a test application

Once StoragePool and StorageClass is deployed, you are all set to deploy an application and test Mayastor. To deploy an application, create a persistent volume claim.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ms-volume-claim
spec:
  accessModes:
   - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  storageClassName: mayastor-iscsi

Ensure the created PVC should is in bound state. To verify, execute:

NAME              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     AGE
ms-volume-claim   Bound    pvc-1dca40de-bc62-4da4-a85f-8cf6879849e8   2Gi        RWO            mayastor-iscsi   6d11h

In this example, we are deploying a MySQL application.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql-test
  template:
    metadata:
      labels:
        app: mysql-test
    spec:
      containers:
      - image: mysql:8.0.20
        name: mysql
        ports:
        - containerPort: 3306
        env: 
          - name: MYSQL_ROOT_PASSWORD
            valueFrom:
              secretKeyRef:
                name: mysql-secret
                key: password
        volumeMounts:
        - mountPath: /var/lib/mysql
          name: mysqlvol
      volumes:
        - name: mysqlvol
          persistentVolumeClaim:
            claimName: ms-volume-claim

Next, you need to create a secret for the above MySQL example,

kubectl create secret generic mysql-secret --from-literal=password=abc@123 -n mayastor

To deploy the application execute:

kubectl create -f mysql.yaml  -n mayastor

To verify if the application is successfully deployed, execute:

kubectl get pods -n mayastor

Sample Output:

NAME                          READY   STATUS    RESTARTS   AGE
mayastor-2xm4r                1/1     Running   13         50d
mayastor-bcwp7                1/1     Running   12         50d
mayastor-cr5w4                1/1     Running   13         50d
mayastor-csi-fqpm8            2/2     Running   12         50d
mayastor-csi-l28ws            2/2     Running   12         50d
mayastor-csi-s7455            2/2     Running   12         50d
moac-54d5db-mhtsg             3/3     Running   29         50d
mysql-test-59fd865cdd-5flgt   1/1     Running   0          6d11h
nats-6fdd6dfb4f-67xdc         1/1     Running   6          50d

Related articles

Articles in this section