Clone-to-test: Part 2

In this post I present the first script. It took me quite a while, as I had to figure out how to do everything. I’m no rocket-scientist, so if I can figure it out, you all have hope too. I really do suggest getting your hands dirty in some code, powershell is pretty easy to pick up. :)

If you haven’t done so already, I would suggest reading Part 1. You may also want to check out the architecture of my isolated test environment, as the design is pretty closely linked with my particular requirements.

This time, I made the code as modular as my skills allowed. I had a bit of trouble passing XML objects between functions, so in the end I just re-read in the XML file. If anyone can give me some feedback on this, I’d appreciate it…. its a little sloppy and I’d like to figure out how to pass (Parse?) XML objects properly.

The basic structure is fairly similar to my other scripts and I’m making extensive use of XML again, as I like having human-readable config being passed between scripts.

I’ve tried to include a reasonable amount of commenting in the code itself, but I’ll do a write up on the functions soon… If anyone has anything specific they’d like clarified or explained, please either comment or contact me directly.

Special thanks to LucD for his post, Maish for his vCenter Migration Script and Alan for his writeup on scripting custom attributes, all were most helpful and I doubt I could’ve figured it all out without their help.

# Name : VI_clone-TestVMs.ps1
#
# Sources:     http://technodrone.blogspot.com/2010/01/vcenter-powercli-migration-script.html
#            http://www.virtu-al.net/2009/05/29/powercli-on-steroids-custom-attributes/
#
# Notes :     Reads a CSV of VM's to transfer, dumps vCenter VM information to XML, Clones VM and removes from 
#            source vCenter.
#
# Author : Doug Youd
#
#-------------------------------------------------------------------------------------------------------------
#
#--- Parameters ----------------------------------------------------------------------------------------------
#Input CSV File
$INPUTVMCSV = "c:/tmp/VMsToClone.csv"
#XML Template
$XMLTEMPLATEFILE = "C:/tmp/VIClone_Template.xml"
#Output XML File
$XMLOUTPUT = "c:/tmp/ClonedMachines.xml"

#Use Shortname?
$USESHORTNAME = $true
#Default Retention(in days)
$DEFAULTRETENTION = "14"
#Instance diffenciator (Appended to name of the clones)
$INSTANCETAG = "_TST-Env"

#Transfer Folder (vCenter)
$TRANSFERFOLDER = "Transfer"
#Transfer DataStore
$TRANSFERDATASTORE = "VI3_Transfer"
#Target ESX Host (Host that will perform the clone task)
$TARGETHOST = "FQDN_of_ESX_Server"

#--- Data -----------------------------------------------------------------------------------------------------
$XMLTemplate = @'
<VMachines version="1.5">
    <VMachine>
        <VMName></VMName>
        <TargetName></TargetName>
        <SourceFolder></SourceFolder>
        <VMDiskTotalGb></VMDiskTotalGb>
        <Note></Note>
        <ServiceLife></ServiceLife>
        <VDisks>
            <VDisk>
                <VDiskLabel></VDiskLabel>
                <DataStore></DataStore>
                <CapacityInGB></CapacityInGB>
            </VDisk>
        </VDisks>
        <VNics>
            <VNic>
                <VNicLabel></VNicLabel>
                <VNicPortGroup></VNicPortGroup>
                <VNicVlan></VNicVlan>
                <MacAddr></MacAddr>
            </VNic>
        </VNics>
        <CustomAttributes>
            <CustomAttribute>
                <Key></Key>
                <Value></Value>
            </CustomAttribute>
        </CustomAttributes>
    </VMachine>
</VMachines>
'@

#--- Functions ------------------------------------------------------------------------------------------------

#Function:    get-InputVMs
#Purpose:    return array of VM objects from CSV file.
#Author:    Doug Youd
#Notes:
function get-InputVMs([string]$InputVMCsv)
{
    $VMs = @()
    $Failed_VMs = @()
    $VMList = Import-Csv $InputVMCsv
    #for each line of the CSV file, get the named VM. Add the name to an error array if it fails.
    foreach($line in $VMList){
        $VM = Get-VM -Name $line.VMName
        if($?){
            $VMs += $VM
        }else {
            $Failed_VMs += $line.VMName
        }
    }
    $Error = $Failed_VMs
    return $VMs
}

#Function:    get-InputVMsView
#Purpose:    return view of VM objects from CSV file.
#Author:    Doug Youd
#Notes:
function get-InputVMsView()
{
    #Get list of VM and ESXHost objects for vCenter
    $VMs = @()
    $Failed_VMs = @()
    $VMList = $input
    #for each line of the CSV file, get the named VM. Add the name to an error array if it fails.
    foreach($line in $VMList){
        $VM = get-view -ViewType VirtualMachine | Where-Object {$_.Name -eq $line.VMName}
        if($?){
            $VMs += $VM
        }else {
            $Failed_VMs += $line.VMName
        }
    }
    $Error += "Failed to get View of " + $Failed_VMs
    return $VMs
}

#Function:    create-XMLTemplate
#Purpose:    Create a template file from text
#Author:    Doug Youd
#Notes:
function create-XMLTemplate([String]$XMLTemplateFile)
{
    $input | Out-File $XMLTemplateFile -encoding UTF8
}

#Function:    get-PortGroups
#Purpose:    Get all Portgroups in a DataCenter
#Author:    Doug Youd
#Notes:        
function get-PortGroups()
{
    $ESXHosts = get-view -ViewType HostSystem 
    #Get all the Portgroups (for the VLAN ID)
    $PortGroups = @()
    foreach($ESXHost in $ESXHosts){
        foreach($ESXHostPortGroup in $ESXHost.config.network.portgroup){
            $PortGroups += $ESXHostPortGroup.spec
        }
    }
    return $PortGroups
}

#Filter:    get-FolderPath
#Purpose:    Get the Object's Folder Path in vCenter 
#Author:    Maish (http://technodrone.blogspot.com/2010/01/vcenter-powercli-migration-script.html)
#Notes:        Takes an array of vcenter objects. Returns
filter Get-FolderPath {
    $_ | % {
        $row = "" | select Name, Path
        $row.Name = $_.Name

        $current = Get-View $_.Parent
        $path = $_.Name
        do {
            $parent = $current
            if($parent.Name -ne "vm"){$path = $parent.Name + "\" + $path}
            $current = Get-View $current.Parent
        } while ($current.Parent -ne $null)
        $row.Path = $path
        $row
    }
}

#Function:    get-VMInfoXML
#Purpose:    Get the VM's vcenter info and create XML
#Author:    Doug Youd
#Notes:        Takes an array from the get-view cmdlet and returns VM Info As XML.
function get-VMInfoXML([String]$XMLTemplateFile, [String]$INSTANCETAG, [Bool]$USESHORTNAME = $TRUE, $DEFAULTRETENTION)
{
    #Load the Template into a new XML object
    $XMLVMachines = New-Object XML
    $XMLVMachines.load($XMLTemplateFile)
    $XMLVMachine = @($XMLVMachines.VMachines.VMachine)[0]
    
    #Get the PortGroup Info
    $PortGroups = get-PortGroups
    
    #Get view of the VMs in the input list
    $VMs = $VMList | get-InputVMsView
    
    #Output XML Details for each of the VM's
    foreach($VM in $VMs)
    {
        $Temp = "" | Select-Object FileName, DataStore, capacityinGB, Match, DevType, vNicPg, VMDiskTotalGb
        #Define a custom VMachine XML object
        $VMachine = $XMLVMachine.Clone()
        #Get Source Name
        $VMachine.VMName = [string]$VM.Name
        #Generate the Target Name
        If($UseShortname){
            $Temp.Match = $vm.Name -match '^[a-zA-Z0-9\-]*'
            If($Temp.Match) {
                $VMachine.TargetName = [String]($matches[0] + $INSTANCETAG)
            }else {
                $VMachine.TargetName = [String]($VM.Name + $INSTANCETAG)
            }
        }else {
            $VMachine.VMName = [String]($VM.Name + $INSTANCETAG)
        }
        #Generate the service life
        $TempObj = $VMList | where {$_.VMName -eq $VM.Name}
        $ServiceLife = $TempObj.RetentionPeriod
        if([int]$ServiceLife -gt [int]0){
            $VMachine.ServiceLife = [string]$ServiceLife
        }else {
            $VMachine.ServiceLife = [string]$DEFAULTRETENTION
        }
        
        #VM Device Info
        foreach($dev in $vm.config.hardware.Device){
            $Temp = "" | Select-Object FileName, DataStore, capacityinGB, Match, DevType, vNicPg, VMDiskTotalGb
            $PortGroup = ""
                #vDisk Info
                   if($dev.GetType().Name -eq "VirtualDisk"){
                    #Create a new VDisk XML Object
                    $VDisk = (@($VMachine.VDisks.VDisk)[0]).Clone()
                    #Populate the data into the XML object from the vCenter object $vm
                    $VDisk.vDiskLabel = [String]$dev.DeviceInfo.Label
                    #Just want the Datastore shortname
                    $Temp.FileName = $dev.Backing.FileName
                    $Temp.Match = $Temp.FileName -match '^\[.*\]'
                    $Temp.DataStore = $matches[0]
                    $VDisk.DataStore = [String]$Temp.DataStore
                    #Calculate the Disk Size in GB
                    $Temp.CapacityInGb = [int]($dev.CapacityInKB / 1048576)
                    $VMachine.VMDiskTotalGb = [String]([int]$VMachine.VMDiskTotalGb + [int]$Temp.CapacityInGb)
                    $VDisk.CapacityInGb = [String]$Temp.CapacityInGb
                    #Append the VDisk to the VDisks
                       $VMachine.VDisks.AppendChild($VDisk) > $null
                   }
                #vNIC Info
                if(($dev.GetType().Name -eq "VirtualPCNet32") -or ($dev.GetType().Name -eq "VirtualVmxnet") -or ($dev.GetType().Name -eq "VirtualVmxnet2") -or ($dev.GetType().Name -eq "VirtualVmxnet3") -or ($dev.GetType().Name -eq "VirtualE1000")){
                    #Create a new VDisk XML Object
                    $VNic = (@($VMachine.VNics.VNic)[0]).Clone()
                    $VNic.VnicLabel = $dev.DeviceInfo.Label
                    #Match the VNic to Portgroup and get the VlanId
                    $PortGroup = @($PortGroups | Where-Object {$_.name -eq $dev.DeviceInfo.summary})[0]
                    $VNic.VnicPortGroup = [String]$dev.DeviceInfo.Summary
                    $VNic.VnicVlan = [String]$PortGroup.VlanId
                    #Get the MAC Address
                    $VNic.MacAddr = [String]$dev.macAddress
                    #Append the VNic to the VNics
                    $VMachine.VNics.AppendChild($VNic) > $null
                }
        }
        #Clear blank devices
        $VMachine.VDisks.VDisk | Where-Object {$_.VDiskLabel -eq ""} | ForEach-Object {[Void]$VMachine.VDisks.RemoveChild($_)}
        $VMachine.VNics.VNic | Where-Object {$_.VNicLabel -eq ""} | ForEach-Object {[Void]$VMachine.VNics.RemoveChild($_)}
        
        #VM Custom Attributes / Notes
        $VMachine.Note = [string]$VM.Config.Annotation
        $CustomAttributes = $VM.CustomValue
        foreach ($Attribute in $CustomAttributes) {
            #Create a new CustomAttribute XML Object
            $CustomAttribute = (@($VMachine.CustomAttributes.CustomAttribute)[0]).Clone()
            $CustomAttribute.Key = [string]$Attribute.Key
               $CustomAttribute.Value = [string]$Attribute.value
              #Append the CustomAttribute to the CustomAttributes
            $VMachine.CustomAttributes.AppendChild($CustomAttribute) > $null
        }
        #Clear blank Attributes
        $VMachine.CustomAttributes.CustomAttribute | Where-Object {$_.Key -eq ""} | ForEach-Object {[Void]$VMachine.CustomAttributes.RemoveChild($_)}
        
        #Get the VM's folder
        $VMPathInfo = $VM | Get-FolderPath
        $VMachine.SourceFolder = [string]$VMPathInfo.path
        
        #Add VM to VMachines array
        $XMLVMachines.VMachines.AppendChild($VMachine) > $null
    }
    #Clear blank VMs
    $VMachines.VMachines.VMachine | Where-Object {$_.VMName -eq ""} | ForEach-Object {[Void]$VMachines.VMachines.RemoveChild($_)}
    
    return $XMLVMachines
}

#Function:    get-TotalSpace
#Purpose:    Aggregate the Total Space of the VMs in the group.
#Author:    Doug Youd
#Notes:        Accepts XML VMachines v1.5
function get-TotalSpace([string]$XMLFile)
{
    $VMachines = New-Object XML
    $VMachines.load($XMLFile)
    [int]$TotalSpace = 0
    $VMachines.VMachines.VMachine | ForEach-Object {
        [int]$TotalSpace = [int]$TotalSpace + [int]$_.VMDiskTotalGb
    }
    return $TotalSpace
}

#Function:    check-SufficientSpace
#Purpose:    Checks a Datastore for sufficient space to complete command.
#Author:    Doug Youd
#Notes:        Takes an integer pipeline input (RequiredGB). Returns a boolean, defaults to false.
function check-SufficientSpace([int]$RequiredSpaceGB, [String]$TargetDataStore)
{
    $SufficientSpace = [bool]$false
    
    $RequiredMB = ($RequiredSpaceGB * [int]1024)
    $DataStore = Get-Datastore | Where-Object {$_.name -eq $TargetDataStore}
    
    $AvailableMB = [int]$DataStore.FreeSpaceMB
    
    if ($AvailableMB -ge $RequiredMB) {
        $SufficientSpace = $true
    }
    
    return $SufficientSpace
}

#Function:    clone-VMs
#Purpose:    Clone a group of VMs
#Author:    Doug Youd
#Notes:        http://communities.vmware.com/thread/262297
#            http://www.lucd.info/2010/02/21/about-async-tasks-the-get-task-cmdlet-and-a-hash-table/
#            Accepts VMachines v1.5 as input.
function Clone-VMs([String]$XMLFile, [String]$TargetDataStore, [String]$TargetFolder, [String]$TargetHost)
{
    #Task List hashtable
    $CloneTasks = @{}
    #Load the XML
    $VMachines = New-Object XML
    $VMachines.load($XMLFile)
    $CloneList = @()
    
    #Form a temp Clone list.
    $VMachines.VMachines.VMachine | ForEach-Object {
        if($_.VMName -ne $null -and $_.TargetName -ne $null -and $_.VMName -ne "" -and $_.TargetName -ne ""){
            Write-Host $_.VMName
            $CloneTasks[(New-VM -VM (Get-VM $_.VMName) -Name $_.TargetName -VMHost (Get-VMHost -Name $TargetHost) `
            -Datastore (Get-Datastore -Name $TargetDataStore) -Location (Get-Folder -Name $TargetFolder) -RunAsync).Id] = $_.TargetName
        }
    }
    
    #After the task has completed, remove the VM from the source vCenter.
    $RunningTasks = $CloneTasks.Count
    while($RunningTasks -gt 0){
        Get-Task | % {
            if($CloneTasks.ContainsKey($_.Id) -and $_.State -eq "Success"){
                Remove-VM -VM (Get-VM -Name $CloneTasks[$_.Id]) -Confirm $false
                Write-Output "Clone ran successfully for: " $CloneTasks[$_.Id]
                $CloneTasks.Remove($_.Id)
                $RunningTasks--
            }
            elseif ($CloneTasks.ContainsKey($_.Id) -and $_.State -eq "Error"){
                write-output "Clone Failed for: " $CloneTasks[$_.Id]
                $CloneTasks.Remove($_.Id)
                $RunningTasks--
            }
        }
        Start-Sleep -Seconds 15
    }
}

#--- Main ------------------------------------------------------------------------------------------------

#Generate XML Template
$XMLTemplate | create-XMLTemplate $XMLTEMPLATEFILE
#Get VMs listed in the CSV file.
$VMList = Import-Csv $InputVMCsv

#Get the required XML data for all VM's in the 
$VmXML = New-Object XML
$VmXML = $VMList | get-VMInfoXML $XMLTEMPLATEFILE $INSTANCETAG $USESHORTNAME $DEFAULTRETENTION

#Export the XML to file.
$VmXML.Save($XMLOutput)

#Check for Sufficient Space on Transfer LUN
$TotalSpace =  get-TotalSpace $XMLOutput
$SufficientSpace = check-SufficientSpace $TotalSpace $TRANSFERDATASTORE

#Clone if there is sufficient space.
if ($SufficientSpace) {
    Write-Output "There is Sufficient space to clone to " $TRANSFERDATASTORE
    Clone-VMs $XMLOUTPUT $TRANSFERDATASTORE $TRANSFERFOLDER $TARGETHOST
} else {
    Write-Output "There is insufficient space to clone to " $TRANSFERDATASTORE
}

#Thats all folks! :P

As always, any feedback would be appreciated.

Download the Script :VI_clone-TestVMs.ps1

Leave a Reply

The opinions expressed on this site are my own and not necessarily those of my employer.

All code, documentation etc is my own work and is licensed under Creative Commons and you are free to use it, at your own risk.

I assume no liability for code posted here, use it at your own risk and always sanity-check it in your environment.