Tags

, , , , ,

It was 2013, a warm sunny day in Colombo and another busy day in my virtual world. We just upgraded all our VMware hosts to new hardware. With new hardware comes new features that can be made available for the virtual machines hosted in them. One of those features that almost always arrives with new hardware is new CPU capabilities. When the hardware is upgraded the CPU Identification Masks should be reset or “refreshed” to match the new host CPU register values so that incompatible values are removed to prevent vMotion issues when the EVC mode is changed to the latest level. As you might already know, upgrading the EVC mode to the newer levels exposes more CPU features to the virtual machines. So you should definitely do it unless there is a very good reason to do otherwise. In my opinion this whole refresh process should be automatic but who am I to shout at VMware. 😉 When we performed the initial random check on our virtual machines’ CPU ID Masks the VMs we found sometimes had the identification masks set and sometimes not. There was no consistency across the virtualization environment. Once we ran our script we noticed that only around 20% of the total number of VMs actually had any CPU ID Mask configuration set and this was never set by us. I’m not sure of the exact reason behind this difference in configuration of the CPU ID Masks of these VMs and why these (or some) ID masks “stick” for the lifetime of a VM. My guess is that these CPU ID Masks are somehow set automatically during some events and the resulting mask depends on the hardware of the host that the VM resides during the time. The task of resetting the CPU ID Mask is simple as clicking a button in the vSphere client but is unimaginably complex when it comes to scripting it. There was no easy way of doing this and not even a clue to reach my goal even remotely with PowerCLI. The one that came to the rescue was one of the VMware Flings – Onyx. VMware Flings available at the VMware Labs has a lot of powerful nifty tools that can ease a vSphere admin’s life.

Onyx is a standalone application that serves as a proxy between the vSphere Client and the vCenter Server. It monitors the network communication between them and translates it into an executable PowerShell code.

So once I used Onyx to monitor and retrieve what the vSphere client actually sends to the vCenter server when we reset the CPU ID Mask of a VM the task was pretty straight forward. Playing with the vCPU’s core was frightening for us as this was our first time doing it and there was no much documentation of it online as well. Since we were performing this task on the production VMs we wanted the task to be more than perfect to avoid any angry customers. So we decided to contact the VMware support. I won’t go into the detail here but the VMware support took a relatively long time to respond to us with a clear answer on how we needed to perform this task and whether we were on the right track in doing it. Basically from their responses it seemed that they were new to this task as us. 🙂 Finally they confirmed that what we did in our script to reset the CPU ID Mask was correct. So we decided to proceed with it. Before running it I decided to add extensive logging so I can revert changes in case something goes horribly wrong. We executed the script from the vCenter servers during a maintenance weekend and it worked like a charm. Time to grab a cold beer!


Update: I have noticed that WordPress messes up the PowerShell syntax in the script. Please use this link to download the script directly.


The code:

# ========================================================================
# name		: reset_cpu_id_mask_to_default.ps1
# author	: Nimantha Wickremasinghe
# version	: 1.0
# purpose	: Reset CPU ID Masks to default values.
# requirements  : PowerCLI
# ========================================================================

# ------- Get_Current_User -------
$script:currentUser = $env:userprofile.split('\')[2]
$script:logPath = "C:\$currentUser\Script_Logs\"

# ------- Get_All_Powered_Off_VMs -------
Function Get-PoweredOffVMs {
	return Get-VM | ? {$_.PowerState -eq 'PoweredOff'}
}

# ------- ReconfigVM_Task -------
Function Reset-vCpuMask($vm) {
	try{
	    $vmConfigSpec = new-object VMware.Vim.VirtualMachineConfigSpec
	    $featureMask = new-object VMware.Vim.VirtualMachineCpuIdInfoSpec
	    $featureMask.Operation = "remove"
		$featureMask.info = new-object VMware.Vim.HostCpuIdInfo
		$featureMask.info.level = 0x80000001
		
	    $featureMaskAMD = new-object VMware.Vim.VirtualMachineCpuIdInfoSpec
	    $featureMaskAMD.Operation = "remove"
	    $featureMaskAMD.info = new-object VMware.Vim.HostCpuIdInfo
		$featureMaskAMD.info.level = 0x80000001
	    $featureMaskAMD.info.vendor = "amd"
		
	    $vmConfigSpec.CpuFeatureMask = $featureMask,$featureMaskAMD

		$view = get-view $vm.id
	    $view.ReconfigVM_Task($vmConfigSpec)
		
		Log-Summary ($vm.Name +": Set CPU ID Masks to default ------- OK")
	}
	catch {
		Log-Failed ($vm.Name + ': ------- FAILED! --- ' + $Error[0])
	}
}

# ------- Backup_OldvCpuMask -------
Function Backup-OldvCpuMask($vm) {
	$bkpLogPath = "$logPath\Backup\"
	$bkpFileName = $vm.Name
	
	try{
		[array]$CpuFeatueMask = (Get-View $vm.id).Config.CpuFeatureMask
		
		if($CpuFeatueMask) {
			if (!(Test-Path $bkpLogPath)) {
				New-Item -ItemType directory -Path $bkpLogPath
			}
			try {
				$CpuFeatueMask >> $bkpLogPath"$bkpFileName.txt"
				Log-Summary ($vm.Name +": Back up old CPU ID Masks ------- OK")
			}
			catch {
				Log-Summary ($vm.Name +": Back up old CPU ID Masks ------- FAILED!")
			}
		}
		else {
			Log-Summary ($vm.Name +": Back up old CPU ID Masks --- (Masks Unset. Nothing to backup.) ------- OK ")
		}
	}
	catch {
		Log-Failed ($vm.Name + ': ' + $Error[0])
	}
}

# ------- Logging -------
Function Log-Failed ($logEntry) {
	$ErrorActionPreference = 'Stop'
	
	if (!(Test-Path $logPath)){
		New-Item -ItemType directory -Path $logPath
	}
	try{
		Add-Content $logPath'Failed.log' "$logEntry"
	}
	catch{
		Write-Host $Error[0] -ForegroundColor Red
	}
	Write-Host "$logEntry" -ForegroundColor Yellow
}

Function Log-Summary ($logEntry) {
	$ErrorActionPreference = 'Stop'
	$timestamp = Get-Date -Format 'yyyy-MM-dd HH:mm:ss'
	
	if (!(Test-Path $logPath)) {
		New-Item -ItemType directory -Path $logPath
	}
	try {
		Add-Content $logPath'Summary.log' "$timestamp # $logEntry"
	}
	catch {
		Write-Host $Error[0] -ForegroundColor Red
	}		
	Write-Host "$timestamp # $logEntry" -ForegroundColor Cyan
}

# ------- Main -------
$choice = Read-Host `n'Start the script? [y/n]'
if ($choice -eq 'y') {	
	do {
		$vc = Read-Host `n'Specify the vCenter Server to connect [name/IP]'
		if ($vc) {
			if (Test-Connection -ComputerName $vc -Count 2 -Quiet -ErrorAction Stop) {
				asnp VMware.VimAutomation.Core -ErrorAction SilentlyContinue
				
				Log-Summary ($vc + ' available')
				Log-Summary ('------- Script Started -------')
				Log-Summary ('Connecting to ' + $vc + '...')
				try {
					Connect-VIServer $vc -ErrorAction Stop
					Log-Summary ('Connect to ' + $vc + ' ------- OK')

					Log-Summary ('Retrieving powered off VMs...')
					[array]$pwrOffVms = Get-PoweredOffVMs
					$pwrOffVmCount = $pwrOffVms.count
					Log-Summary ("Retrieved $pwrOffVmCount VMs ------- OK")
					
					foreach ($vm in $pwrOffVms) {
						if ((Get-VM -Id $vm.id).PowerState -eq 'PoweredOff') {
							Log-Summary ($vm.Name + ': Task execution ------- START')
							Backup-OldvCpuMask ($vm)
							Reset-vCpuMask ($vm)
							Log-Summary ($vm.Name + ': Task execution ------- END')
						}
						else {
							Log-Failed ($vm.Name)
							Log-Summary ($vm.Name + ': ------- FAILED! --- PowerState: ON')
						}
						Log-Summary ('------- ########## -------')
					}
					Disconnect-VIServer $vc -Confirm:$false -ErrorAction SilentlyContinue
					Log-Summary ('Disconnect from ' + $vc + ' ------- OK')
				}
				catch {
					Log-Summary ($Error[0])
				}
				Log-Summary ('------- Script Completed -------')
			}
			else {
				Log-Summary ($vc + ' unavailable!')
			}
		}
		else {
			Write-Host `n'Please enter a valid name/IP for the vCenter Server!' -ForegroundColor Red
		}
		$choice2 = Read-Host `n'Do you want to specify another vCenter Server to connect? [y/n]'
	} while ($choice2 -eq 'y')
	
	Write-Host `n"Logs can be found at $logPath" -ForegroundColor Green
	
	Write-Host `n`n"Press any key to continue ..."
	$x = $host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")
}
else{
	exit
}

Advertisements