Blockdevices are not detected in some of the nodes

Problem Statement

One disk is attached per Node in a 3 Node cluster in a VM Environment where CentOS is underlying OS and kubectl get blockdevice -n openebs return only one disk. 

OpenEBS version

1.1.0.

OpenEBS Storage engine

cStor

Symptoms

If the particular node is restarted, from where the disk is detected then the description of the disk attached to that node gets modified.

Troubleshooting

Ensure that the serial number of all blockdevices are different. NDM detect and recognize the blockdevice based on their WWN, Model, Serial and Vendor. If the blockdevices have all the parameters same then NDM cannot differentiate the blockdevice and will create only 1 BlockdDevice CR for each unique parameter. To troubleshoot the same user has to make sure the blockdevices are having at least anyone unique parameter from WWN, Model, Serial and Vendor. Usually, this issue is faced in virtualization environment like vSphere, KVM etc. More details about NDM daemon set functionalities can be read from here.

Solution

There are 2 solutions:

- Either user can create a unique serial number for all disks attached to the Nodes and restart all NDM daemonset pods one by one.

OR

- Download custom blockdevice CR YAML file from here and apply with the details of each block device. In the sample spec, ndm.io/managed: is set to false. So NDM will not manage this blockdevice.

Note: If you are creating a block device CR manually for a custom device path, then you must add the corresponding device path under exclude filter so that NDM will not select the particular device for BD creation. For example, if block device CR is creating for /dev/sdb manually, then you must add /dev/sdb under exclude filter of NDM configuration. See here for customizing the exclude filter in NDM configuration.