Repository navigation
CSTACKEX-262: Support for Compute + Volume migration and Volume migration - #112
sandeeplocharla wants to merge 9 commits into
Conversation
15f8e29 to
ddcebda
Compare
🔴 Test Coverage Grade:
|
| Metric | Value |
|---|---|
| Line coverage | 24.86% |
| Branch coverage | 18.99% |
Grade Scale
| Grade | Line Coverage | Meaning |
|---|---|---|
| 🟢 A | ≥ 80% | Excellent - this code sleeps well at night 😴 |
| 🟡 B | 60-79% | Good - almost there, don't stop now 😉 |
| 🟠 C | 40-59% | Acceptable - your code is wearing a seatbelt, but no airbags 😬 |
| 🔴 D | 20-39% | Marginal - boldly shipping where no test has gone before 🖖 |
| ⛔ F | < 20% | Failing - tests? what tests? 🔥 |
Branch coverage is shown as a secondary signal. Grade is determined by line coverage.
View full Actions run
| public static final String VM_WORK_JOB_HANDLER = VirtualMachineManagerImpl.class.getSimpleName(); | ||
|
|
||
| private static final String VM_SYNC_ALERT_SUBJECT = "VM state sync alert"; | ||
| private static final String ONTAP_SVM_NAME_DETAIL = "svmName"; |
There was a problem hiding this comment.
why are these required in this file ? These are ONTAP specific .
There was a problem hiding this comment.
For migration of a volume with VM (Live/Stopped), executeManagedStorageChecksWhenTargetStoragePoolProvided is executed and it rejects all 'managed' storages, unless the pool is of type 'PowerFlex'. Now, as we only have our own pool type for iSCSI but not for NFS, we would be needing explicit ontap relevant checks here.
But we can definitely discuss on the list of checks that we want to keep here. I've retained the same list as in the OntapDataMotionStrategy class with the intention to fail fast.
| volume.getUuid(), currentPool.getUuid(), targetPool.getUuid())); | ||
| } | ||
|
|
||
| protected boolean isSupportedOntapLiveStorageMigration(StoragePoolVO currentPool, StoragePoolVO targetPool) { |
There was a problem hiding this comment.
this is about validating the inputs if it is landing on the ONTAP plugin for migration, can this validation go insode plugin code and get it checked once control is inside plugin block ?
There was a problem hiding this comment.
As replied to the previous comment, we can definitely discuss on the list of checks that we want to do. But, until we get our own pool type for NFS, we would be needing some ontap specific checks here.
| _volumeDao.update(volumeVO.getId(), volumeVO); | ||
| } | ||
| destVolumeInfo = _volumeDataFactory.getVolume(destVolumeInfo.getId(), destVolumeInfo.getDataStore()); | ||
| hostVO = getHostToMigrateVolume(srcVolumeInfo, destVolumeInfo); |
There was a problem hiding this comment.
this method can return exception, and on that situation, we would be left with stale cloudstack volume created as part of createAsync() above, fix it with rollback
There was a problem hiding this comment.
If any exception is thrown, 'completeVolumeMigrationCallback' in this 'copyAsync' method's finally passes it back and the VolumeServiceImpl would handle the cleanup.
| } else if (ProtocolType.NFS3.name().equalsIgnoreCase(details.get(OntapStorageConstants.PROTOCOL))) { | ||
| // For NFS, no access grant needed - file is accessible via mount | ||
| logger.debug("grantAccess: NFS volume [{}], no igroup mapping required", volumeVO.getUuid()); | ||
| if (shouldConnectHostForMigration(volumeVO)) { |
There was a problem hiding this comment.
Can you elaborate the permutations of this method, I am suspecting some corner cases issues?
There was a problem hiding this comment.
| Workflow | Side | State at grantAccess | Attached | Connect host |
|---|---|---|---|---|
| Live VM with storage | source | Migrating | yes | yes, iSCSI only |
| Live VM with storage | destination | Migrating | no | yes |
| Stopped VM with storage | source | Migrating | yes | yes |
| Stopped VM with storage | destination | Creating | no | yes |
| Volume on a stopped VM | source | Migrating | yes | yes |
| Volume on a stopped VM | destination | Creating | no | yes |
| Detached volume | source | Migrating | no | yes |
| Detached volume | destination | Creating | no | yes |
| Volume on a running VM | — | never reached | — | — |
| Deploy, attach, or start | volume | Ready | yes | no |
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
| } | ||
| Lun.Source source = lun.getCopy() != null ? lun.getCopy().getSource() : null; | ||
| Lun.Progress progress = source != null ? source.getProgress() : null; | ||
| if (progress == null || Integer.valueOf(100).equals(progress.getPercentComplete())) { |
There was a problem hiding this comment.
with this condition, wait would over even if progress is not available.
you should keep it as
if( progress != null && )
There was a problem hiding this comment.
sure, will take this up. Thankyou!
e5c27c5 to
9416f32
Compare
9416f32 to
7bca4ca
Compare
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
| checkAvailableForMigration(srcVolumeInfo.getAttachedVM()); | ||
| checkUnsupportedOntapCrossProtocolMigration(srcVolumeInfo, destVolumeInfo); | ||
|
|
||
| destVolumeInfo.getDataStore().getDriver().createAsync(destVolumeInfo.getDataStore(), destVolumeInfo, null); |
There was a problem hiding this comment.
here, callback on createAsycn is null which not handled on createAsycn method so it will throw NPE when callback.complete will get called
There was a problem hiding this comment.
Done, thankyou!
| destVolumeInfo.processEvent(Event.MigrationCopySucceeded); | ||
| destVolumeInfo.processEvent(Event.MigrationRequested); | ||
|
|
||
| destDataStore.getDriver().createAsync(destDataStore, destVolumeInfo, null); |
There was a problem hiding this comment.
same here, callback on createAsync is null
| } | ||
| Lun.Source source = lun.getCopy() != null ? lun.getCopy().getSource() : null; | ||
| Lun.Progress progress = source != null ? source.getProgress() : null; | ||
| if (progress == null || Integer.valueOf(100).equals(progress.getPercentComplete())) { |
| migrateStorage.put(migrateDiskInfo.getSerialNumber(), migrateDiskInfo); | ||
| } | ||
|
|
||
| srcVolumeInfoToDestVolumeInfo.put(srcVolumeInfo, destVolumeInfo); |
There was a problem hiding this comment.
If createAsync, grantAccess, or connectHostToVolume throws exception, the loop never reaches this put.
There was a problem hiding this comment.
If any of those throw, handleFailedMigration still disconnects, revokes, and expunges that volume.
| migrateCommand.setNewVmCpuShares(newVmCpuShares); | ||
| } | ||
|
|
||
| MigrateAnswer migrateAnswer = (MigrateAnswer)_agentMgr.send(srcHost.getId(), migrateCommand); |
There was a problem hiding this comment.
StorageSystemDataMotionStrategy catches OperationTimedoutException, checks the destination host with CheckVirtualMachineCommand, and keeps the migration if the VM is PowerOn. This strategy does not. A timeout becomes a failure, and handleFailedMigration disconnects and expunges the destination disks the VM may be running on.
There was a problem hiding this comment.
Yes, I've now implemented the CheckVirtualMachineCommand to verify the destination. Now, as in StorageSystemDataMotionStrategy, when the VM is in PowerOn state, migration is deemed complete, anything else, including the check itself failing would result in rollback.
| StoragePoolVO srcStoragePoolVO = _storagePoolDao.findById(srcVolumeInfo.getPoolId()); | ||
| List<HostVO> hosts; | ||
| if (srcStoragePoolVO.getClusterId() != null) { | ||
| PrimaryDataStoreInfo srcStore = (PrimaryDataStoreInfo)_dataStoreMgr.getPrimaryDataStore(srcStoragePoolVO.getId()); |
There was a problem hiding this comment.
why only from source side? why not dest side?
There was a problem hiding this comment.
Usually, the source host initiates and handles the copy. So, the check is done on the src side here.
575b714 to
26664a6
Compare
🔴 Test Coverage Grade:
|
| Metric | Value |
|---|---|
| Line coverage | 25.90% |
| Branch coverage | 20.19% |
Grade Scale
| Grade | Line Coverage | Meaning |
|---|---|---|
| 🟢 A | ≥ 80% | Excellent - this code sleeps well at night 😴 |
| 🟡 B | 60-79% | Good - almost there, don't stop now 😉 |
| 🟠 C | 40-59% | Acceptable - your code is wearing a seatbelt, but no airbags 😬 |
| 🔴 D | 20-39% | Marginal - boldly shipping where no test has gone before 🖖 |
| ⛔ F | < 20% | Failing - tests? what tests? 🔥 |
Branch coverage is shown as a secondary signal. Grade is determined by line coverage.
View full Actions run
…ation # Conflicts: # plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/driver/OntapPrimaryDatastoreDriver.java # plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/service/UnifiedSANStrategy.java # plugins/storage/volume/ontap/src/test/java/org/apache/cloudstack/storage/driver/OntapPrimaryDatastoreDriverTest.java # plugins/storage/volume/ontap/src/test/java/org/apache/cloudstack/storage/service/UnifiedNASStrategyTest.java
…ject cross-protocol migration
26664a6 to
be02af2
Compare
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
| if (currentPool.getId() == targetPool.getId()) { | ||
| return; | ||
| } | ||
| if (isSupportedOntapLiveStorageMigration(currentPool, targetPool)) { |
There was a problem hiding this comment.
we do not need this change at all if we enable ALLOW_MIGRATE_OTHER_POOLS with ONTAP pools. That code is already there below. And for an existing pool, we can take care of those through upgrade.
There was a problem hiding this comment.
We have specific set of validations to allow migration, which isn't the case for other storage vendors. If we use ALLOW_MIGRATE_OTHER_POOLS and rely on the checks in OntapDataMotionStrategy, VM migration would've been started while each volume individually reaches the strategy class and if the checks fail only for one volume out of say 2 disks, migration for that volume would fail while the other volume gets migrated. Cloudstack doesn't support moving that volume back.
Also, ALLOW_MIGRATE_OTHER_POOLS is currently used only by storage vendors whose 'managed' flag is set to 'false'.
Once our own pool types are ready, we must be able to reduce the flags here.
|
|
||
| // OntapDataMotionStrategy also returns HIGHEST for these and a tie keeps the first strategy. | ||
| if (srcVolumeInfo.getState() == Volume.State.Migrating | ||
| && srcVolumeInfo.getDataStore().getRole() == DataStoreRole.Primary |
There was a problem hiding this comment.
is not this condition of matching primary redundant if outside if already compares source and destination both as VolumeInfo ? secondary storage to primary for volume to volume itself is not there.
There was a problem hiding this comment.
Yeah good catch! since we already have primary checks before this check can be removed.
| if (copiedVolume.getLun() != null) { | ||
| Lun copiedLun = copiedVolume.getLun(); | ||
| volumeDetailsDao.addDetail(destVolumeInfo.getId(), OntapStorageConstants.LUN_DOT_UUID, copiedLun.getUuid(), false); | ||
| volumeDetailsDao.addDetail(destVolumeInfo.getId(), OntapStorageConstants.LUN_DOT_NAME, copiedLun.getName(), false); |
There was a problem hiding this comment.
we need to save lun serial number of the new copies lun also here as new lun will have new serial number from previous one
There was a problem hiding this comment.
Following the same as createAsync, this gets set in grantAccess
| copyQosIopsFromSourceWhenDestinationHasNone(srcVolumeInfo, destVolumeInfo); | ||
| applyQosPolicyToVolume(destVolumeInfo); | ||
| copiedVolumeTO = (VolumeObjectTO)destVolumeInfo.getTO(); | ||
| copiedVolumeTO.setPath(copiedPath); |
There was a problem hiding this comment.
why are we not setting iscsi_name for iscsi type volumes ?
There was a problem hiding this comment.
grantAccess call comes after copyAsync completes, which would set that.
| volumeDetailsDao.addDetail(destVolumeInfo.getId(), OntapStorageConstants.LUN_DOT_UUID, copiedLun.getUuid(), false); | ||
| volumeDetailsDao.addDetail(destVolumeInfo.getId(), OntapStorageConstants.LUN_DOT_NAME, copiedLun.getName(), false); | ||
| destVolumeVO.setFolder(copiedLun.getUuid()); | ||
| destVolumeVO.set_iScsiName(null); |
There was a problem hiding this comment.
is null set intentionally here ?
There was a problem hiding this comment.
This would have the value from source VolumeVO at this point, so setting it to null so that it could be set to proper value later in grantAccess
Description
This PR...
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Live VM with volumes (
migrateVirtualMachineWithVolume)migrateVolumewith livemigrate on a running volume is rejectedStopped VM storage migration (
migrateVirtualMachineWithVolume, no destination host)Volume migration (
migrateVolume)