Why Large VM vMotion and Clone Tasks Fail: Device Limits, Config Hygiene, and PowerCLI Prechecks

/ ai, AI (Artificial Intelligence), Artificial Intelligence, DVS, ESXi, Migration Precheck, Operational Runbook, powercli, Storage vMotion, vCenter Server, VCF, VCF (Cloud Foundation), VM Cloning, vMotion, vSphere / By

Done — I removed the extra line breaks/page-break-style spacing. The only preserved line breaks are inside the ASCII workflow and PowerCLI script so they remain usable.

Why Large VM vMotion and Clone Tasks Fail: Device Limits, Config Hygiene, and PowerCLI Prechecks

Large VM migrations usually fail at the worst possible time: late in the change window, after the task has already consumed hours of storage, network, and operator attention. When the error is something like “Invalid configuration for device ‘##’,” the first instinct is to look for a broken virtual NIC, a missing port group, an attached ISO, or some odd device left behind in the VM configuration. Sometimes that instinct is right. But Broadcom KB 344928 (KB344928) calls out a more specific failure pattern: shared-nothing vMotion or cloning for very large virtual machines can fail when the operation takes longer than 24 hours. The symptom still surfaces as an invalid device configuration problem, but the underlying issue is tied to Distributed Virtual Switch port reservation timing. Broadcom documents that vCenter reserves a DVS port for migration, that reservation expires after 24 hours, and the destination VM can be left without the expected port backing when the migration or clone finally tries to complete. For vCF operators, the lesson is bigger than one KB fix. Large VM mobility needs a readiness process. Before a shared-nothing vMotion, cross-vCenter migration, or clone operation, you want a fast way to identify the VMs that are large, messy, snapshot-heavy, multi-datastore, or attached to risky devices. That is where a PowerCLI precheck becomes useful.

Scenario

You are preparing to move or clone a large VM in a vSphere or vCF environment. The VM may be tens of terabytes, backed by multiple VMDKs, attached to a distributed port group, and scheduled for a shared-nothing vMotion, cross-vCenter migration, or clone operation. The risk is not just the size of the VM. The risk is the combination of large provisioned size, DVS-backed virtual NICs, snapshots, connected ISO files, host-backed devices, RDMs, independent disks, multi-datastore layouts, missing target networks, and stale distributed port state. Any one of those may be manageable. Several of them together can turn a routine mobility task into a failed change window.

What KB 344928 Is Actually Saying

KB 344928 is not a generic fix for every “Invalid configuration for device” error. It describes a specific condition where shared-nothing vMotion or cloning fails for very large VMs when the operation takes more than 24 hours. The KB shows related log patterns including vim.fault.InvalidDeviceSpec, Device spec doesn’t match up with dvport/dvpg configuration, and a cross-vCenter vMotion message about failing to get DVS state in the restore phase. The documented cause is that vCenter reserves a DVS port for 24 hours. After that reservation expires, DvsMonitor deletes the port reservation, and the VM arriving on the destination has no port to connect to. Broadcom’s documented workaround is to extend the DVS virtual NIC port backing reservation timeout on the source and destination vCenter Servers, restart vpxd, retry the operation, then revert the configuration after the affected migrations are complete. The KB example uses PortReserveTimeoutInMin set to 7200, which equals five days.

<vpxd>
<dvs>
<vnicportbackings>
<PortReserveTimeoutInMin>7200</PortReserveTimeoutInMin>
</vnicportbackings>
</dvs>
</vpxd>

Treat that as a controlled change, not a casual tuning knob. It requires a vpxd restart and should be reverted after the migration wave according to the KB.

Why the Error Message Is Misleading

The phrase “Invalid configuration for device” is accurate, but it is not always specific enough to tell you what failed. Other Broadcom KBs show similar errors for different causes. A stale Distributed Virtual Port allocation state in the vCenter Server database can create a mismatch with the ESXi host’s local DVS proxy switch, causing an invalid device fault during Storage vMotion. A vMotion or Storage vMotion operation can also fail when the VM’s port group does not exist on the destination host. Clone and power-on failures may appear when the VM configuration differs from vCenter database records, an invalid port group is selected, or temporary clone disks remain in the VMX configuration. That means the right workflow is to confirm whether the VM and operation match KB 344928, check for common device hygiene issues, validate target network availability, estimate whether the task may exceed the DVS reservation window, and only then decide whether the vCenter timeout change is warranted.

Workflow at a Glance

The important point in this workflow is the split between VM hygiene and long-running DVS reservation risk. Do not jump directly to editing vpxd.cfg just because the error contains a device number.

What you should notice is that the KB path is only one branch. Most environments will still need ordinary VM cleanup before the migration or clone attempt.

What the Precheck Should Inventory

A useful precheck should produce a report that answers four operational questions. First, is the VM large enough to create timing risk? That requires provisioned size, total disk size, largest disk, and an estimated copy duration. Second, is the network backing risky? That requires vNIC count, DVS-backed NICs, port groups, and missing target network checks. Third, is the VM configuration clean? That requires checking for connected ISO files, USB devices, serial or parallel ports, PCI passthrough, vGPU-style devices, and other special hardware. Fourth, is the disk layout migration-friendly? That requires identifying RDMs, independent disks, thin or eager disk formats, datastore count, and snapshots. The PowerCLI script below uses standard discovery cmdlets such as Get-VM, Get-HardDisk, Get-NetworkAdapter, Get-Snapshot, Get-CDDrive, and Get-UsbDevice.

PowerCLI Precheck Script

This script is intentionally read-focused. It inventories VMs, classifies risk, and exports a CSV. It does not modify VM settings, migrate workloads, or edit vCenter configuration. Adjust the thresholds for your environment. A 10 TB threshold is a practical starting point, but the real question is whether the operation may run longer than the DVS port reservation window.

<#
.SYNOPSIS
Large VM migration and clone readiness precheck.

.DESCRIPTION
Inventories large VMs, disk layouts, snapshots, network adapters,
removable devices, passthrough-style devices, and target network
availability before shared-nothing vMotion, cross-vCenter migration,
or cloning.

.NOTES
This script is read-focused. It does not change VM configuration.
Use the output to plan remediation before the migration window.
#>

[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$VIServer,

[string]$ClusterName,

[string]$TargetClusterName,

[decimal]$LargeVmThresholdGB = 10240,

# Optional. Use a conservative estimate from prior migration tests.
# Example: 500 means 500 GB per hour.
[decimal]$EstimatedCopyRateGBPerHour = 0,

[string]$OutputPath = “.large-vm-migration-precheck.csv”
)

Import-Module VMware.PowerCLI -ErrorAction Stop

Set-PowerCLIConfiguration -InvalidCertificateAction Warn -Confirm:$false | Out-Null
Connect-VIServer -Server $VIServer | Out-Null

function Add-Risk {
param(
[System.Collections.Generic.List[string]]$List,
[string]$Message
)

if (-not [string]::IsNullOrWhiteSpace($Message)) {
[void]$List.Add($Message)
}
}

function Get-DatastoreNameFromPath {
param([string]$Path)

if ($Path -match ‘^[(?<Datastore>[^]]+)]’) {
return $Matches.Datastore
}

return $null
}

# Optional target network inventory. This catches obvious network-name gaps.
# It does not replace vCenter migration compatibility checks.
$targetNetworks = @{}

if ($TargetClusterName) {
$targetHosts = @(Get-Cluster -Name $TargetClusterName -ErrorAction Stop | Get-VMHost)

foreach ($host in $targetHosts) {
foreach ($pg in @($host | Get-VirtualPortGroup -ErrorAction SilentlyContinue)) {
if ($pg.Name) {
$targetNetworks[$pg.Name] = $true
}
}

foreach ($vdpg in @($host | Get-VDSwitch -ErrorAction SilentlyContinue | Get-VDPortgroup -ErrorAction SilentlyContinue)) {
if ($vdpg.Name) {
$targetNetworks[$vdpg.Name] = $true
}
}
}
}

$vmScope = if ($ClusterName) {
Get-Cluster -Name $ClusterName -ErrorAction Stop | Get-VM
}
else {
Get-VM
}

$report = foreach ($vm in $vmScope) {
$view = Get-View -Id $vm.Id -Property Config.Hardware.Device,Config.Version,Config.Files,Runtime.Host

$devices = @($view.Config.Hardware.Device)
$disks = @(Get-HardDisk -VM $vm -ErrorAction SilentlyContinue)
$nics = @(Get-NetworkAdapter -VM $vm -ErrorAction SilentlyContinue)
$cds = @(Get-CDDrive -VM $vm -ErrorAction SilentlyContinue)
$snaps = @(Get-Snapshot -VM $vm -ErrorAction SilentlyContinue)
$usb = @(Get-UsbDevice -VM $vm -ErrorAction SilentlyContinue)

$serialPorts = @($devices | Where-Object { $_.GetType().Name -eq ‘VirtualSerialPort’ })
$parallelPorts = @($devices | Where-Object { $_.GetType().Name -eq ‘VirtualParallelPort’ })
$pciPassthrough = @($devices | Where-Object { $_.GetType().Name -eq ‘VirtualPCIPassthrough’ })

$rdmDisks = @($disks | Where-Object { $_.DiskType -match ‘Raw’ })
$independentDisks = @($disks | Where-Object { $_.Persistence -match ‘Independent’ })
$thinDisks = @($disks | Where-Object { $_.StorageFormat -eq ‘Thin’ })

$totalDiskGB = ($disks | Measure-Object -Property CapacityGB -Sum).Sum
if (-not $totalDiskGB) { $totalDiskGB = 0 }

$largestDiskGB = 0
if ($disks.Count -gt 0) {
$largestDiskGB = ($disks | Sort-Object -Property CapacityGB -Descending | Select-Object -First 1).CapacityGB
}

$provisionedGB = [math]::Round($vm.ProvisionedSpaceGB, 2)
$totalDiskGB = [math]::Round($totalDiskGB, 2)
$largestDiskGB = [math]::Round($largestDiskGB, 2)

# Use the larger value as the sizing basis for planning.
$sizeBasisGB = [math]::Max($provisionedGB, $totalDiskGB)

$estimatedCopyHours = $null
if ($EstimatedCopyRateGBPerHour -gt 0 -and $sizeBasisGB -gt 0) {
$estimatedCopyHours = [math]::Round(($sizeBasisGB / $EstimatedCopyRateGBPerHour), 1)
}

$snapshotSizeGB = $null
if ($snaps.Count -gt 0 -and ($snaps[0].PSObject.Properties.Name -contains ‘SizeGB’)) {
$snapshotSizeGB = [math]::Round((($snaps | Measure-Object -Property SizeGB -Sum).Sum), 2)
}

$datastores = @(
$disks |
ForEach-Object { Get-DatastoreNameFromPath -Path $_.Filename } |
Where-Object { $_ } |
Sort-Object -Unique
)

$connectedCDs = @($cds | Where-Object { $_.ConnectionState.Connected })
$isoCDs = @($cds | Where-Object { $_.IsoPath })
$hostCDs = @($cds | Where-Object { $_.HostDevice -or $_.RemoteDevice })

$dvsNicCount = 0
$nicBackingDetails = New-Object System.Collections.Generic.List[string]
$missingTargetNetworks = New-Object System.Collections.Generic.List[string]

foreach ($nic in $nics) {
$networkName = $nic.NetworkName
$backing = $nic.ExtensionData.Backing
$backingType = if ($backing) { $backing.GetType().Name } else { “UnknownBacking” }

if ($backingType -match ‘DistributedVirtualPortBackingInfo’) {
$dvsNicCount++

$port = $backing.Port
[void]$nicBackingDetails.Add((
“{0}={1};DVS={2};PGKey={3};PortKey={4}” -f
$nic.Name,
$networkName,
$port.SwitchUuid,
$port.PortgroupKey,
$port.PortKey
))
}
else {
[void]$nicBackingDetails.Add((“{0}={1};Backing={2}” -f $nic.Name, $networkName, $backingType))
}

if ($TargetClusterName -and $networkName -and -not $targetNetworks.ContainsKey($networkName)) {
[void]$missingTargetNetworks.Add($networkName)
}
}

$risks = [System.Collections.Generic.List[string]]::new()

if ($sizeBasisGB -ge $LargeVmThresholdGB) {
Add-Risk $risks “Large VM at or above threshold”
}

if ($estimatedCopyHours -ne $null -and $estimatedCopyHours -ge 20) {
Add-Risk $risks “Estimated copy duration near or above 24-hour DVS reservation window”
}

if ($dvsNicCount -gt 0 -and $estimatedCopyHours -ne $null -and $estimatedCopyHours -ge 20) {
Add-Risk $risks “DVS-backed NICs plus long-running copy: review KB 344928 applicability”
}

if ($snaps.Count -gt 0) {
Add-Risk $risks “Snapshots present”
}

if ($missingTargetNetworks.Count -gt 0) {
Add-Risk $risks “Target cluster missing one or more VM networks”
}

if (($nics | Where-Object { [string]::IsNullOrWhiteSpace($_.NetworkName) }).Count -gt 0) {
Add-Risk $risks “One or more NICs have no network name”
}

if ($connectedCDs.Count -gt 0 -or $isoCDs.Count -gt 0 -or $hostCDs.Count -gt 0) {
Add-Risk $risks “Connected ISO, CD, or host device present”
}

if ($rdmDisks.Count -gt 0) {
Add-Risk $risks “RDM disk present”
}

if ($independentDisks.Count -gt 0) {
Add-Risk $risks “Independent disk present”
}

if ($usb.Count -gt 0) {
Add-Risk $risks “USB device present”
}

if (($serialPorts.Count + $parallelPorts.Count) -gt 0) {
Add-Risk $risks “Serial or parallel port present”
}

if ($pciPassthrough.Count -gt 0) {
Add-Risk $risks “PCI passthrough or vGPU-style device present”
}

if ($datastores.Count -gt 4) {
Add-Risk $risks “VM disks span more than four datastores”
}

$severity = if ($risks | Where-Object {
$_ -match ‘DVS-backed|Target cluster missing|Connected ISO|RDM|USB device|PCI passthrough’
}) {
“Red”
}
elseif ($risks.Count -gt 0) {
“Amber”
}
else {
“Green”
}

[pscustomobject]@{
VMName = $vm.Name
PowerState = $vm.PowerState
VMHost = $vm.VMHost.Name
ClusterOrParent = $vm.VMHost.Parent.Name
HardwareVersion = $view.Config.Version

ProvisionedGB = $provisionedGB
TotalDiskGB = $totalDiskGB
LargestDiskGB = $largestDiskGB
EstimatedCopyHours = $estimatedCopyHours

DiskCount = $disks.Count
ThinDiskCount = $thinDisks.Count
RDMCount = $rdmDisks.Count
IndependentDiskCount = $independentDisks.Count
DatastoreCount = $datastores.Count
Datastores = ($datastores -join ‘;’)

SnapshotCount = $snaps.Count
SnapshotSizeGB = $snapshotSizeGB

NICCount = $nics.Count
DVSNicCount = $dvsNicCount
Networks = (($nics | Select-Object -ExpandProperty NetworkName -Unique) -join ‘;’)
MissingTargetNetworks = (($missingTargetNetworks | Sort-Object -Unique) -join ‘;’)
NICBackingDetails = ($nicBackingDetails -join ‘ | ‘)

ConnectedCDCount = $connectedCDs.Count
USBDeviceCount = $usb.Count
SerialParallelPortCount = ($serialPorts.Count + $parallelPorts.Count)
PciPassthroughCount = $pciPassthrough.Count

RiskCount = $risks.Count
Severity = $severity
RiskNotes = ($risks -join ‘ | ‘)
}
}

$report |
Sort-Object Severity, VMName |
Export-Csv -Path $OutputPath -NoTypeInformation -UseCulture

$report |
Sort-Object Severity, VMName |
Format-Table VMName, Severity, ProvisionedGB, TotalDiskGB, EstimatedCopyHours, RiskNotes -AutoSize

Write-Host “Precheck report written to: $OutputPath”

Example Usage

Run the script against a single vCenter and all VMs:

.Get-LargeVmMigrationPrecheck.ps1 `
-VIServer “vcenter01.lab.local” `
-LargeVmThresholdGB 10240 `
-OutputPath “.large-vm-precheck.csv”

Run it for a source cluster and compare network names against a target cluster:

.Get-LargeVmMigrationPrecheck.ps1 `
-VIServer “vcenter01.lab.local” `
-ClusterName “Compute-Cluster-A” `
-TargetClusterName “Compute-Cluster-B” `
-LargeVmThresholdGB 10240 `
-EstimatedCopyRateGBPerHour 500 `
-OutputPath “.cluster-a-to-cluster-b-precheck.csv”

For cross-vCenter migrations, run the script against the source vCenter first, then run a target-side network and datastore validation separately. The script’s network-name comparison is useful, but it is not a substitute for the actual vCenter compatibility check.

How to Read the Output

The CSV is designed for a change review, not just for an engineer’s terminal. Green means no obvious inventory risk was found and the VM can proceed to normal compatibility checks. Amber means the VM has cleanup or planning items that should be remediated before the change window if possible. Red means the VM has a likely migration blocker or KB 344928-style timing risk and should not proceed without remediation or an approved workaround plan. The most important column is RiskNotes. For example, Large VM at or above threshold | Snapshots present is usually an operational cleanup issue. Remove or consolidate snapshots if your backup and application owners agree. DVS-backed NICs plus long-running copy: review KB 344928 applicability is the pattern that should trigger a KB 344928 review, especially if the operation is a shared-nothing vMotion or clone that may exceed 24 hours. Target cluster missing one or more VM networks is a network mapping issue. Do not assume vMotion or clone will fix it for you. Validate the distributed port group, standard port group, DVS membership, and target host connectivity.

Runbook Stage 1: Confirm the Failure Pattern

Before applying the KB workaround, confirm the operation type and symptom. Use KB 344928 when the facts line up: the operation is a shared-nothing vMotion, cross-vCenter migration, or clone of a very large VM; the runtime may exceed or has exceeded 24 hours; the VM uses DVS-backed virtual NICs; the error includes Invalid configuration for device ‘##’, InvalidDeviceSpec, or DVS/dvport mismatch logs; and the environment falls within the KB’s supported vCenter Server scope. Do not use the KB as a blanket response to every invalid device error. Other Broadcom articles document similar symptoms from stale distributed port state, missing destination port groups, invalid port group selections, VM configuration mismatches, and temporary clone disk entries.

Runbook Stage 2: Clean Up VM Configuration

Before the migration or clone, clean up the obvious blockers. Disconnect connected ISO files or switch them to client device if they are no longer needed. Confirm snapshot handling with backup and application owners, then consolidate or remove snapshots where appropriate. Create or map missing destination port groups before the migration window. If stale DVS backing is suspected and the pattern matches Broadcom guidance, move the VM’s network adapter to another valid port group and then back to the original. Validate RDMs, independent disks, USB devices, serial ports, parallel ports, passthrough devices, and vGPU-style hardware before assuming the workload is migration-ready. For multi-datastore VMs, confirm target datastore placement for every VMDK. This is configuration hygiene. It is not glamorous, but it prevents an avoidable migration failure from being misdiagnosed as a platform issue.

Runbook Stage 3: Decide Whether KB 344928 Applies

If the precheck shows a large DVS-backed VM and your estimated migration or clone time approaches 24 hours, review KB 344928 before the change window. The KB workaround is operationally disruptive because it requires restarting the vmware-vpxd service after updating vpxd.cfg. It also instructs administrators to revert the configuration after the affected VMs are migrated and restart vpxd again. A practical decision matrix looks like this: for small or moderate VMs expected to complete under 24 hours, do not apply KB 344928 just because the error text sounds similar; for large VMs with DVS-backed NICs and shared-nothing migration expected near or above 24 hours, plan the KB 344928 workaround with change approval; for errors that occur quickly during validation, investigate port groups, stale DVS state, ISO attachments, disk issues, or VMX mismatches first; for errors that occur near switchover after a long-running task, KB 344928 becomes more likely; and for cross-vCenter migration, validate both source and destination vCenter behavior and network mappings.

Runbook Stage 4: Validate After Migration or Clone

After the task completes, validate more than power state. Confirm that the VM is registered on the expected vCenter, cluster, and host; all vNICs are connected to the expected port groups; guest network connectivity works after migration; all expected VMDKs are attached; no unexpected temporary disks are present; snapshot state is clean or intentionally retained; VMware Tools status is healthy; the application owner confirms service health; the vpxd.cfg timeout change is reverted if the KB workaround was used; and vpxd service health is confirmed after the revert. The last two checks matter. The KB workaround is not “set and forget.” If you temporarily extend the reservation window, revert it after the migration wave unless Broadcom Support advises otherwise.

Troubleshooting Notes

The device number does not always map cleanly in the UI. The error might reference device ’13’, device ’16’, or another index. That number often points toward a virtual device spec entry, but the UI label is not always obvious. Start with network adapters, CD/DVD devices, and disks. A valid-looking port group can still be wrong for the destination. A network name appearing on the source side is not enough. Validate that the destination host or cluster can actually use the required port group. A clone can also fail because of old temporary disk state. If a previous clone failed, do not ignore the VMX and inventory state. Temporary clone disks or invalid VMX entries can keep causing problems until they are cleaned up. Finally, estimated copy time is only an estimate. The script’s EstimatedCopyRateGBPerHour parameter is intentionally manual. Use values from prior migrations, not theoretical network speed. Change rates, storage contention, snapshot depth, and backend array behavior can all distort the final runtime.

Operational Takeaway

KB 344928 is useful because it explains a frustrating edge case: a large VM migration or clone can fail with an invalid device configuration error because the DVS port reservation expired before the operation completed. But the better operational answer is not to memorize one workaround. The better answer is to build a precheck habit. Before moving large VMs, inventory size, disks, snapshots, network backing, removable devices, passthrough devices, and target network availability. Use the report to clean up configuration issues before the change window. Then, when the VM is large enough and slow enough to approach the 24-hour reservation window, review KB 344928 as a planned change instead of discovering it after a failed migration. Large VM mobility is not just a vMotion task. It is a readiness workflow.

Using vCert Without Guesswork: A vCenter Certificate Recovery Runbook
vCenter certificate failures tend to show up at the worst possible time: during an upgrade precheck, after a maintenance window has already…

The post Why Large VM vMotion and Clone Tasks Fail: Device Limits, Config Hygiene, and PowerCLI Prechecks appeared first on Digital Thought Disruption.