VMware Validated Design – Automated Deployment with Cloud Builder – Part 4: Generating SSL Certificates

This is part 4 of a series of posts on VMware Cloud Builder.

In this post I will cover generating the required SSL certificates for deploying this VMware Validated Design with VMware Cloud Builder.

Friendly warning: This is a long post so maybe get a coffee before reading!

Continue reading “VMware Validated Design – Automated Deployment with Cloud Builder – Part 4: Generating SSL Certificates”

Quick Fix: The trust relationship between this workstation and the primary domain failed…

We’ve all been there…attempt to open an RDP session to a VM you haven’t connected to in a while and you see the message above! Traditionally the fix for this was to log on as a local admin user, remove the VM from the AD domain (add to workgroup), reboot, log in again, add to AD domain, reboot….well here is a quicker way of resolving the issue with PowerShell.

Modify the username, password and domain controller FQDN and save the following as ResetDomainMembership.ps1 and run on the affected VM as a local administrator


$password = "Password123!" | ConvertTo-SecureString -asPlainText -Force
$username = "domain\administrator"
$credential = New-Object System.Management.Automation.PSCredential($username,$password)
Reset-ComputerMachinePassword -Server dc01.domain.local -credential $credential
shutdown -r -t 0

Tip: If you dont want to include the password in the script as this is a security concern you can use a Read-Host command to prompt the user for the password


$password = Read-Host -asSecureString "Please enter the password"

Shutdown/Power up a vSAN cluster with PowerCli

I have been doing a lot of lab testing lately and using  vCloud Director is a great way to be able to run side by side tests (sometimes destructive!) against multiple environments without requiring multiple physical clusters. I wanted to emulate a Dell EMC VxRail appliance so I created a 4 node nested vSAN cluster with a vCenter appliance & external PSC running on the cluster, to use as a vCD template. When creating vCD templates it is preferable to power down the environment before adding to the vCD catalog. When it comes to powering down this environment, because the vCenter & PSC are running on the cluster, there is a chicken and egg scenario as you need to power down in this order:

  1. vCenter
  2. PSC
  3. Put ESXi hosts in maintenance mode (vSAN aware operation)
  4. Power down ESXi hosts

The inverse applies when powering up the environment.  So rather than connect to multiple different interfaces to execute the power down or power up operations I put together the following Menu based PowerCli script with the following Options

These options will call the following functions

Option 1. Shutdown vSAN Cluster & PSC/vCenter

  • ConnectViServer $vCenterFQDN $vCenterUser $vCenterPassword
    • Generic function to connect to a VIServer (ESXi or vCenter). Just pass the hostname, user & password. This instance connects to vCenter as we need to perform a DRS operation.
  • ChangeDRSLevel $PartiallyAutomated
    • The ChangeDRSLevel function takes an argument for the level to set it to. In this case it sets it to partially Automated to stop DRS from moving VMs around.
  • MoveVMs
    • This function will move the VMs defined in $VMList to the host defined in $VMHost. This ensures that we know where the VMs are when we come to power the cluster back up
  • ConnectViServer $VMHost $VMHostUser $VMHostPassword
    • Generic function to connect to a VIServer (ESXi or vCenter). Just pass the hostname, user & password. This instance connects to the host defined in the $VMHost variable
  • ShutdownVM
    • This function will gracefully shutdown the VMs in the order defined in $VMList. In this case it will shutdown the vCenter first & then the PSC. This list can be expanded to include other VMs or could be refactored to use a CSV for a large list of VMs
  • EnterMaintenanceMode
    • This function will put all hosts defined in $VMHosts into maintenance mode. As this is a vSAN cluster it passes the NoAction flag to prevent any data moment or rebuild
  • ShutdownESXiHosts
    • This function will shutdown all hosts in the cluster

Option 2. Startup vSAN Cluster & PSC/vCenter

  • ExitMaintenanceMode
    • This function will exit all hosts from maintenance mode
  • ConnectViServer $VMHost $VMHostUser $VMHostPassword
    • Generic function to connect to a VIServer (ESXi or vCenter). Just pass the hostname, user & password. This instance connects to the host defined in the $VMHost variable
  • StartVMs
    • This function will startup the VMs in the reverse order defined in $VMList. In this case it will startup the PSC first & then the vCenter.
  • PollvCenter
    • This function will Poll vCenter until it is up and available
  • ConnectViServer $vCenterFQDN $vCenterUser $vCenterPassword
    • Generic function to connect to a VIServer (ESXi or vCenter). Just pass the hostname, user & password. This instance connects to vCenter as we need to perform a DRS operation
  • ChangeDRSLevel $FullyAutomated
    • The ChangeDRSLevel function takes an argument for the level to set it to. In this case it sets it to Fully Automated as we are done with the maintenance.

The script is posted to Github here and is also posted below. This was written and tested against vSphere 6. I will update it in the coming weeks for vSphere 6.5 and hopefully leverage some of the new APIs

# Script to shutdown & startup a vSAN cluster when vCenter/PSC are running on the cluster
# Created by Brian O'Connell | Dell EMC
# Provided with zero warranty! Please test before using in anger!
# @LifeOfBrianOC
# https://lifeofbrianoc.com/

## User Variables ##
$vCenterFQDN = "vcs01.domain.local"
$vCenterUser = "VC_Admin@domain.local"
$vCenterPassword = "Password123!"
$Cluster = "MARVIN-Virtual-SAN-Cluster"
$VMList = @("VCS01", "PSC01")
$VMHosts = @("esxi04.domain.local", "esxi05.domain.local", "esxi06.domain.local", "esxi07.domain.local")
$VMHost = "esxi04.domain.local"
$VMHostUser = "root"
$VMHostPassword = "Password123!"

### DO NOT MODIFY ANYTHING BELOW THIS LINE ###

# Add Required PowerCli Modules
Get-Module -ListAvailable VM* | Import-Module

# Function to Connect to VI Host (vCenter or ESXi). Pass host, username & password to the function
Function ConnectVIServer ($VIHost, $User, $Password) {
    Write-Host " "
    Write-Host "Connecting to $VIHost..." -Foregroundcolor yellow
    Connect-VIServer $VIHost -User $User -Password $Password | Out-Null
	Write-Host "Connected to $VIHost..." -Foregroundcolor Green
    Write-Host "------------------------------------------------" -Foregroundcolor Green
}

# Define DRS Levels to stop Vms from moving away from the defined host
$PartiallyAutomated = "PartiallyAutomated"
$FullyAutomated = "FullyAutomated"

# Function to Change DRS Automation level						
Function ChangeDRSLevel ($Level) {						

    Write-Host " "
    Write-Host "Changing cluster DRS Automation Level to Partially Automated" -Foregroundcolor yellow
    Get-Cluster $cluster | Set-Cluster -DrsAutomation $Level -confirm:$false | Out-Null
    Write-Host "------------------------------------------------" -Foregroundcolor yellow
}
						
# Function to Move the Vms to a defined host so they can be easily found when starting back up
Function MoveVMs {

    Foreach ($VM in $VMList) {
        # Power down VM
        Write-Host " "
        Write-Host "Moving $VM to $VMHost" -Foregroundcolor yellow
        Get-VM $VM | Move-VM -Destination $VMHost -Confirm:$false | Out-Null
        Write-Host "------------------------------------------------" -Foregroundcolor yellow
    }   
    Disconnect-VIServer $vCenterFQDN -confirm:$false | Out-Null
}

# Function to Shutdown VMs
Function ShutdownVM  {

    Foreach ($VM in $VMList) {
        # Power down VM
        Write-Host " "
        Write-Host "Shutting down $VM" -Foregroundcolor yellow
        Shutdown-VMGuest $VM -Confirm:$false | Out-Null
        Write-Host "------------------------------------------------" -Foregroundcolor yellow
        Write-Host " "
        Write-Host "Waiting for $VM to be Shutdown" -Foregroundcolor yellow
        # Check VM powerstate and wait until it is powered off before proceeding with the next VM
        do {
            sleep 15
            $powerState = (get-vm $VM).PowerState
        }
        while ($powerState -eq "PoweredOn")
        Write-Host " "
        Write-Host "$VM Shutdown.." -Foregroundcolor green
        Write-Host "------------------------------------------------" -Foregroundcolor yellow	
    }
}

# Function to put all ESXi hosts into maintenance mode with the No Action flag for vSAN data rebuilds
Function EnterMaintenanceMode {

    Foreach ($VMHost in $VMHosts) {
        Connect-VIServer $VMHost -User root -Password $VMHostPassword | Out-Null
        # Put Host into Maintenance Mode
        Write-Host " "
        Write-Host "Putting $VMHost into Maintenance Mode" -Foregroundcolor yellow
        Get-View -ViewType HostSystem -Filter @{"Name" = $VMHost }|?{!$_.Runtime.InMaintenanceMode}|%{$_.EnterMaintenanceMode(0, $false, (new-object VMware.Vim.HostMaintenanceSpec -Property @{vsanMode=(new-object VMware.Vim.VsanHostDecommissionMode -Property @{objectAction=[VMware.Vim.VsanHostDecommissionModeObjectAction]::NoAction})}))}
        Disconnect-VIServer $VMHost -confirm:$false | Out-Null
        Write-Host "------------------------------------------------" -Foregroundcolor yellow
        Write-Host " "
        Write-Host "$VMHost in maintenance mode.." -Foregroundcolor green
        Write-Host "------------------------------------------------" -Foregroundcolor yellow	
    }
}

# Function to Exit hosts from maintenance mode
Function ExitMaintenanceMode {

    Foreach ($VMHost in $VMHosts) {
        Connect-VIServer $VMHost -User root -Password $VMHostPassword | Out-Null
        # Exit Maintenance Mode
        Write-Host " "
        Write-Host "Exiting Maintenance Mode for $VMHost" -Foregroundcolor yellow
        Set-VMHost $VMHost -State "Connected" | Out-Null
        Disconnect-VIServer $VMHost -confirm:$false | Out-Null
        Write-Host "------------------------------------------------" -Foregroundcolor yellow
        Write-Host " "
        Write-Host "$VMHost out of maintenance mode.." -Foregroundcolor green
        Write-Host "------------------------------------------------" -Foregroundcolor yellow	
    }
    Write-Host "Waiting for vSAN Cluster to be Online" -Foregroundcolor yellow	
	Sleep 60							
}

# Function to shutdown hosts
Function ShutdownESXiHosts {

    Foreach ($VMHost in $VMHosts) {
        # Exit Maintenance Mode
        Write-Host " "
        Write-Host "Shutting down ESXi Hosts" -Foregroundcolor yellow
        Connect-VIServer -Server $VMHost -User root -Password $VMHostPassword | %{
            Get-VMHost -Server $_ | %{
                $_.ExtensionData.ShutdownHost_Task($TRUE) | Out-Null
            }
        }
        Write-Host "------------------------------------------------" -Foregroundcolor yellow
        Write-Host " "
        Write-Host "ESXi host $VMHost shutdown.." -Foregroundcolor green
        Write-Host "------------------------------------------------" -Foregroundcolor yellow	
    }
		Write-Host "------------------------------------------------" -Foregroundcolor yellow
        Write-Host " "
        Write-Host "All ESXi Hosts shutdown.." -Foregroundcolor green
        Write-Host "------------------------------------------------" -Foregroundcolor yellow
}

# Function to Start VMs in the reverse order they were powered down									
Function StartVMs {
    # Reverse the VM list to start in reverse order
    [array]::Reverse($VMList)

    Foreach ($VM in $VMList) {
        # Power on VM
        Write-Host " "
        Write-Host "Powering on $VM" -Foregroundcolor yellow
        Start-VM $VM -Confirm:$false | Out-Null
        Write-Host "------------------------------------------------" -Foregroundcolor yellow
        Write-Host " "
        Write-Host "Waiting for $VM to be Powered On" -Foregroundcolor yellow
        # Check VM powerstate and wait until it is powered on before proceeding with the next VM
        do {
            sleep 15
            $powerState = (get-vm $VM).PowerState
        }
        while ($VM -eq "PoweredOff")
        Write-Host " "
        Write-Host "$VM Powered On..proceeding with next VM" -Foregroundcolor green
        Write-Host "------------------------------------------------" -Foregroundcolor yellow	
    }

}

# Function to Poll the status of vCenter after starting up the VM
Function PollvCenter {

    do 
    {
        try 
        {
            Write-Host " "
            Write-Host "Polling vCenter $vCenterFQDN Availability...." -ForegroundColor Yellow
            Write-Host "------------------------------------------------" -Foregroundcolor yellow
            # Create Web Request
            [System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
            $HTTP_Request = [System.Net.WebRequest]::Create("https://$($vCenterFQDN):9443")

            # Get a response
            $HTTP_Response = $HTTP_Request.GetResponse()

            # Get the HTTP code
            $HTTP_Status = [int]$HTTP_Response.StatusCode

            If ($HTTP_Status -eq 200) { 
                Write-Host " "
                Write-Host "vCenter $vCenterFQDN is Available!"  -ForegroundColor Green
                Write-Host "------------------------------------------------" -Foregroundcolor Green
                # Close HTTP request
                $HTTP_Response.Close()
            }
        }
        catch { 
            Write-Host " "
            Write-Host "vCenter $vCenterFQDN Not Available Yet...Retrying Poll..."  -ForegroundColor Cyan
            Write-Host "------------------------------------------------" -Foregroundcolor Cyan
    } }
    While ($HTTP_Status -ne 200)	
}

# Function to display the main menu 
Function Menu 
{
    Clear-Host         
    Do
    {
        Clear-Host                                                                        
        Write-Host -Object 'Please choose an option'
        Write-Host     -Object '**********************'	
        Write-Host -Object 'vCenter & vSAN Maintenance Options' -Foregroundcolor Yellow
        Write-Host     -Object '**********************'
        Write-Host -Object '1.  Shutdown vSAN Cluster & PSC/vCenter '
        Write-Host -Object ''
        Write-Host -Object '2.  Startup vSAN Cluster & PSC/vCenter '
        Write-Host -Object ''
        Write-Host -Object 'Q.  Exit'
        Write-Host -Object $errout
        $Menu = Read-Host -Prompt '(Enter 1 - 2 or Q to quit)'

        switch ($Menu) 
        {
            1 
            {
				ConnectVIServer $vCenterFQDN $vCenterUser $vCenterPassword
                ChangeDRSLevel $PartiallyAutomated
                MoveVMs
                ConnectVIServer $VMHost $VMHostUser $VMHostPassword
                ShutdownVM
                EnterMaintenanceMode
                ShutdownESXiHosts   
            }
            2 
            { 
				ExitMaintenanceMode
                ConnectVIServer $VMHost $VMHostUser $VMHostPassword
                StartVMs
				PollvCenter
                ConnectVIServer $vCenterFQDN $vCenterUser $vCenterPassword
                ChangeDRSLevel $FullyAutomated
            }
            Q 
            {
                Exit
            }	
            default 
            {
                $errout = 'Invalid option please try again........Try 1-2 or Q only'
            }

        }
    }
    until ($Menu -eq 'q')
}   

# Launch The Menu
Menu

Snapshot Management with PowerCli

I’ve been doing some lab work recently, testing out some new automation for deploying Dell EMC Enterprise Hybrid Cloud (EHC) and as part of the testing we needed to be able to quickly and easily create on the fly rollback points at incremental steps in the build to assist with troubleshooting (outside of the standard rollback points created by the EHC Automated Install Tool (AIT) ).

Note: AIT is currently an internal Dell EMC tool used by our professional services organisation to deploy and configure EHC on a customer site.

Disclaimer: As with all scripts and code that you find on the web you should thoroughly test this in a lab environment before considering to use it in production. This script comes with zero warranty as it was something that was created quickly for lab use only!

Snapshots were enough to give us a rollback point, so to achieve this I put together a menu based PowerCli script that will take snapshots of a defined list of VMs with a defined snapshot name, rollback to the last snap taken, or rollback to a defined snapshot.

For more information on how to add a menu to a PowerShell script go here, and for how to add a “Press any key to continue..” to a script go here.

Here is the script for snapshot management. It is broken up into the following PowerShell Functions:

  • CreateVMSnapshot
  • RevertLastVMSnapshot
  • RevertSpecificVMSnapshot
  • anyKey
  • Menu

Before running the script you need to edit the user variables for your environment. The $VMList variable is a comma separated list of VM names as they appear in vCenter. In my example these are the components of a distributed vRA deployment. The $SnapshotName vaiable will be used when creating snapshots or when executing the Revert To Specific Snapshot option

# User Variables
$vCenterFQDN = "vcs01.domain.local"
$vCenterUser = "administrator@vsphere.local"
$vCenterPassword = "Password123!"
$VMList = @("vra01", "vra02", "web01", "web02", "mgr01", "mgr02", "dem01", "dem02", "agt01", "agt02")
$SnapshotName = "Snap01"

To execute the script just browse to the directory you saved the script in a PowerShell or PowerCli console, run ./SnapshotManagement.ps1 and you will be presented with a menu

Select the desired option from the menu. The operations are running Async so are quite quick to complete.

The raw code is below. I’ve also posted the script to GitHub here

# User Variables
$vCenterFQDN = "vcs01.domain.local"
$vCenterUser = "administrator@vsphere.local"
$vCenterPassword = "Password123!"
$VMList = @("vra01", "vra02", "web01", "web02", "mgr01", "mgr02", "dem01", "dem02", "agt01", "agt02")
$SnapshotName = "Snap01"
###############################
# DO NOT EDIT BELOW THIS LINE #
###############################
# Add Required Snappins
Get-Module -ListAvailable VM* | Import-Module

Function CreateVMSnapshot {
# Connect to vCenter
Connect-VIServer $vCenterFQDN -username $vCenterUser -password $vCenterPassword
	Foreach ($VM in $VMList) {
	Write-Host "Creating Snapshot for $VM"
	New-Snapshot -VM $VM -Memory -quiesce -Name $SnapshotName -RunAsync
							 }							 
									}
									
Function RevertLastVMSnapshot {
# Connect to vCenter
Connect-VIServer $vCenterFQDN -username $vCenterUser -password $vCenterPassword
	Foreach ($VM in $VMList) {
	Write-Host "Reverting Snapshot for $VM"
	$snap = Get-Snapshot -VM $VM | Sort-Object -Property Created -Descending | Select -First 1
    Set-VM -VM $vm -SnapShot $snap -Confirm:$false  -RunAsync | Out-Null
							 }							 
									}

Function RevertSpecificVMSnapshot {
# Connect to vCenter
Connect-VIServer $vCenterFQDN -username $vCenterUser -password $vCenterPassword
	Foreach ($VM in $VMList) {
	Write-Host "Reverting Snapshot for $VM"
	#$snap = Get-Snapshot -VM $VM | Sort-Object -Property Created -Descending | Select -First 1
    Set-VM -VM $vm -SnapShot $SnapshotName -Confirm:$false  -RunAsync | Out-Null
							 }							 
									}									

Function anyKey 
{
    Write-Host -NoNewline -Object 'Press any key to return to the main menu...' -ForegroundColor Yellow
    $null = $Host.UI.RawUI.ReadKey('NoEcho,IncludeKeyDown')
    Menu
}
									
Function Menu 
{
    Clear-Host         
    Do
    {
        Clear-Host                                                                        
        Write-Host -Object 'Please choose an option'
        Write-Host     -Object '**********************'	
        Write-Host -Object 'Snapshot VM Options' -ForegroundColor Yellow
        Write-Host     -Object '**********************'
        Write-Host -Object '1.  Snapshot VMs '
		Write-Host -Object ''
        Write-Host -Object '2.  Revert to Last Snapshot '
		Write-Host -Object ''
		Write-Host -Object '3.  Revert To Specific Snapshot '
		Write-Host -Object ''
        Write-Host -Object '4.  Exit'
        Write-Host -Object $errout
        $Menu = Read-Host -Prompt '(0-3)'

        switch ($Menu) 
        {
           1 
            {
                CreateVMSnapshot 			
                anyKey
            }
            2 
            {
                RevertLastVMSnapshot
                anyKey
            }
			3 
            {
                RevertSpecificVMSnapshot
                anyKey
            }
            4 
            {
                Exit
			}	
            default 
            {
                $errout = 'Invalid option please try again........Try 0-4 only'
            }

        }
    }
    until ($Menu -ne '')
}   

# Launch The Menu
Menu

Add “Press any key to continue..” to a PowerShell script

From time to time it is nice to have a “Press any key to continue..” break point in a script to allow the user to review the status of an operation or just to add a user interaction to acknowledge the completion of an operation. This is especially useful when using a menu based script (see here) where the script will revert back to the menu once an operation is complete making it difficult to see the status of an operation when it completes or any Write-Host messages that may have been displayed. To get around this I use the following PowerShell Function to insert a “Press any key to continue..” break point that will wait for the user to…you guessed it…press the any key! 🙂

I use then when using a PowerShell Menu (See more about that here). You can edit the text in the quotes on line 3 to suite your use case. In my case i am calling the Menu function on line 5 so that when a user presses a key it will revert to the script menu. Simples!


Function anyKey
{
Write-Host -NoNewline -Object 'Press any key to return to the main menu...' -ForegroundColor Yellow
$null = $Host.UI.RawUI.ReadKey('NoEcho,IncludeKeyDown')
Menu
}


Distributed vRA validation script

From time to time a distributed vRA deployment can have issues…here is a quick script to verify and validate the important components are up and functioning…without the need to log into multiple components. Here is a diagram of my distributed vRA setup

Distributed vRA v4

And here is the script! It has multiple functions to do the following

  • Basic ping tests to each component
  • Get the status of all vRA component services
  • Test the Web & manager server URLs

# Script to check the status of each component of a distributed vRA deployment
# Modify the hostnames to match your environment
$vRAAppliance01FQDN = "vra01.domain.local"
$vRAAppliance02FQDN = "vra02.domain.local"
$vRAWeb01FQDN = "web01.domain.local"
$vRAWeb02FQDN = "web02.domain.local"
$vRAManager01FQDN = "manager01.domain.local"
$vRAManager02FQDN = "manager02.domain.local"
$vRADEM01FQDN = "demw01.domain.local"
$vRADEM02FQDN = "demw02.domain.local"
$vRAAgent01FQDN = "agent01.domain.local"
$vRAAgent02FQDN = "agent02.domain.local"
$vRAComponentServiceURL = "https://vra-vip.domain.local/component-registry/services/status/current"
$webVIPURL = "https://web-vip.domain.local/WAPI"
$managerVIPURL = "https://manager-vip.domain.local/VMPS2"



### DO NOT MODIFY ANYTHING BELOW THIS LINE ###



Function pingHosts {
@"
===============================================================================
Performing basic ping test to each defined component
===============================================================================
"@
$vms = @($vRAAppliance01FQDN, $vRAAppliance02FQDN, $vRAWeb01FQDN, $vRAWeb02FQDN, $vRAManager01FQDN, $vRAManager02FQDN, $vRADEM01FQDN, $vRADEM02FQDN, $vRAAgent01FQDN, $vRAAgent02FQDN)
$collection = $()
foreach ($vm in $vms)
{
 $status = @{ "ServerName" = $vm; "TimeStamp" = (Get-Date -f s) }
 if (Test-Connection $vm -Count 1 -ea 0 -Quiet)
 { 
 $status["Ping Results"] = "Up"
 } 
 else 
 { 
 $status["Ping Results"] = "Down" 
 }
 New-Object -TypeName PSObject -Property $status -OutVariable serverStatus
 $collection += $serverStatus

}
 }
 
Function testvRAServices {
@"
===============================================================================
Getting Status of all vRA component services
===============================================================================
"@
Write-Host "Checking status of vRA Component Services" -ForegroundColor Yellow
# Request Service Information from $vRAComponentServiceURL
$vRAURL = Invoke-WebRequest $vRAComponentServiceURL
# Convert the Json response
$json = ConvertFrom-Json -InputObject $vRAURL.content
# Get Service name & status
$serviceInfo = $json.content
# Loop through each service
foreach ($service in $serviceInfo) {
# Get the Service Name
$serviceName = $service.serviceName
# Get the Service status
$serviceStatus = $service.serviceStatus.serviceInitializationStatus
# If Service Status is blank report it as BLANK POSSIBLY STOPPED
 if (!$serviceStatus) {
 $serviceStatus = "BLANK - POSSIBLY STOPPED?"
 }
# If Service Status is FAILED print to screen in red 
if ($serviceStatus -eq "FAILED") {
 write-host "$serviceName $serviceStatus" -ForeGroundColor Red
 }
# Otherwise print to screen as normal (Remove this if you only want to report failed services) 
 else {
 Write-Host "$serviceName $serviceStatus"}
 }

}


Function testWebVIP {
@"
===============================================================================
Checking status of vRA Web API URL
===============================================================================
"@
Write-Host "Testing IaaS Web Service VIP URL $webVIPURL" -ForegroundColor Yellow
# Create Web Request
$HTTP_Request = [System.Net.WebRequest]::Create($webVIPURL)

# Get a response
$HTTP_Response = $HTTP_Request.GetResponse()

# Get the HTTP code
$HTTP_Status = [int]$HTTP_Response.StatusCode

If ($HTTP_Status -eq 200) { 
 Write-Host "IaaS Web Service is OK!" -ForegroundColor Green
 # Close HTTP request
$HTTP_Response.Close()
}
Else {
 Write-Host "IaaS Web Service is not responding. Restart IIS on Web01. If that does not resolve then Reboot Web01" -ForegroundColor Red
}
 }
 
Function testMgrVIP {
@"
===============================================================================
Checking status of vRA Manager API URL
===============================================================================
"@
Write-Host "Testing IaaS Manager Service VIP URL $managerVIPURL" -ForegroundColor Yellow
# Create Web Request
$HTTP_Request = [System.Net.WebRequest]::Create($managerVIPURL)

# Get a response
$HTTP_Response = $HTTP_Request.GetResponse()

# Get the HTTP code
$HTTP_Status = [int]$HTTP_Response.StatusCode

If ($HTTP_Status -eq 200) { 
 Write-Host "IaaS Manager Service is OK!" -ForegroundColor Green
 # Close HTTP request
$HTTP_Response.Close()
}
Else {
 Write-Host "IaaS Manager Service is not responding. Ensure all vRA services are running on manager01. If that does not resolve then Reboot manager01" -ForegroundColor Red
}
 } 
 

 pingHosts; testvRAServices; testWebVIP; testMgrVIP

The function pingHosts is a basic ping test to each defined vRA component

The function testvRAServices was an interesting one to write as I’m not overly familiar with working with APIs so it was a learning experience. I wanted to be able to report the status of all vRA services listed on the VAMI administration UI of a vRA appliance (https://vra:5480). The URL that the services are listed on is https://vra-vip.domain.local/component-registry/services/status/current so using the powershell Invoke-WebRequest you get back the page information.

Invoke-WebRequest

Line 56 in the script puts the page contents into a variable we can work with. You can see that the information we want is stored in Content (ServiceStatus) in Json format so you need to take that Json and convert it to  to powershell readable text using the ConvertFrom-Json function (ConvertFrom-Json converts a JSON-formatted string to a custom object (PSCustomObject) that has a property for each field in the JSON string) Line 58 does this

We then put each service into the $serviceinfo variable and loop through them to get the service name and service status.

Side note here: Originally I was querying $json.content.serviceStatus to get the details i wanted but i noticed I wasnt getting a full list of services, i was getting some blank content and also some duplicate service names. This is how i was doing it

$vRAURL = Invoke-WebRequest "https://vra-vip.vlab.local/component-registry/services/status/current" 
$json = ConvertFrom-Json -InputObject $vRAURL.content 
$serviceInfo = $json.content.serviceStatus | Select serviceName,serviceInitializationStatus $serviceInfo 

Here is that that returns. As you can see its not the full list and there is a duplicate entry so its not much use

Duplicate Results

I dug a little into the API and it seems that it does indeed contain inconsistent information. Here is an excerpt with some issues where the actual service name is content-management but the serviceStatus reports the name as advanced-designer-service

Service Name issue

So to get an accurate list i queried the serviceName field to get the name and the serviceStatus.serviceInitializationStatus to get the service status. Unfortunately doing it this way doesnt allow creating a nice formatted table (at least i havent figured out how to do it yet!) but i did get it to print out each service & status on the same line.

Line 68: In my lab i use a vRO appliance so the internal vRO service on the vRA appliance is stopped. The service status comes back blank so i added a check to report blank service status as “BLANK – POSSIBLY STOPPED?”.

Line 72: I also added a check to print any failed services in red so they stand out.

The testWebVIP and testManagerVIP functions use the powershell System.Net.WebRequest to get the HTTP status code for a given URL. If the status code comes back as 200 then everything is ok. If not there is an issue with your IaaS components

So there you have it. A quick way to verify the status of all of the important vRealize Automation components and services. In my example below the iaas-service is in a failed state (The driving reason for creating this script! 🙂 )

Script Results

Use PowerCli to answer VM questions

Recently had a datastore in the lab briefly fill up and some VMs go into a suspended state awaiting a Retry/Cancel question to be answered. Rather than manually answer the question on each affected VM i looked to use PowerCli. With a little digging in the PowerCli documentation I found the Get-VMQuestion & Set-VMQuestion cmdlets. In an ideal world all you would need to do is this

Connect-VIServer -Server VC01
Get-VM | Get-VMQuestion | Set-VMQuestion -Option "Retry" -Confirm:$false
Disconnect-VIServer -Server VC01 -Confirm:$false
As i am using vCD all VMs have a UID appended with a space like this
IaaS01 (c4920308-f901-45be-b99c-0095175099e9)

So when you try to pass the VM name to Get-VMQuestion it does not handle the space in the VM name and the command fails. To get around this i created a simple script to first create an array of VM names thereby avoiding the issue with the space. Running the script below against my vCenter answered about 65 VM questions while i made a coffee!

Connect-VIServer -Server VC01
$vm = @()
$vm = Get-VM
$answerQuestion = $vm | Get-VMQuestion | Set-VMQuestion -Option "Retry" -Confirm:$false
Disconnect-VIServer -Server VC01 -Confirm:$false

Script to create vCenter clusters, switches, portgroups & VLANs

When setting up a new environment it is always good practice to plan out and document everything to avoid errors. From hostnames, cluster names, vDS names, portgroup names & VLANs etc. I was recently asked to help script the creation of the above from a csv to avoid fat fingers and to save time through automation. Enter PowerCli! Here is the script i created. Disclaimer: Its version 1…no error checking and can be made more efficient but it works and may be helpful to others! I’ve also posted it to GitHub with an example csv here https://github.com/LifeOfBrianOC/vmware_scripts


# Script to create vCenter Clusters, Distributed Switches & portgroups from CSV
# Edit the CSV location variable and the vCenter FQDN and run the script
# Tested with PowerCli 6.3.0
# This is version 1.0 There is no error checking in place so if an item
# already exists or cannot be found the script will error but should continue

#
$CSVPath = "C:\Scripts\Example.csv"
$vCenter = "vc01.domain.local"

#####################################
# DO NOT EDIT ANYTHING BELOW THIS LINE
#####################################

# Load VMware PowerCli Snapins
add-psSnapin VMWare* | out-null

# Connect to vCenter
Connect-VIserver $vCenter

# Get vCenter Datacenter Name
$datacenter = Get-Datacenter

@"
====================================
Creating Clusters
====================================
"@

# Import CSV and only read lines that have an entry in clusterName column
$csv = @()
$csv = Import-Csv -Path $CSVPath |
Where-Object -FilterScript {
$_.clusterName
}

# Loop through all _s in the CSV
ForEach ($_ in $csv)
{
New-Cluster -Location $datacenter -Name $_.clusterName -HAEnabled | out-null
}

@"
====================================
Creating Distributed Switches
====================================
"@

# Import CSV and only read lines that have an entry in switchName column
$csv = @()
$csv = Import-Csv -Path $CSVPath |
Where-Object -FilterScript {
$_.switchName
}

# Loop through all _s in the CSV
ForEach ($_ in $csv)
{
Import-Module VMware.VimAutomation.Vds
New-VDSwitch -Location $datacenter -Name $_.switchName -Mtu 1600 | out-null
}

@"
==========================================
Creating Distributed Switch Portgroups & Assigning VLANs
==========================================
"@

# Import CSV and only read lines that have an entry in portgroupName column
$csv = @()
$csv = Import-Csv -Path $CSVPath |
Where-Object -FilterScript {
$_.portgroupName
}

# Loop through all _s in the CSV
ForEach ($_ in $csv)
{
Import-Module VMware.VimAutomation.Vds
New-VDPortgroup -Name $_.portgroupName -VDSwitch $_.addToSwitch -VlanId $_.vlan | out-null
}
@"
==========================================
Setting Trunk Ports
==========================================
"@

# Import CSV and only read lines that have an entry in trunkPortgroup column
$csv = @()
$csv = Import-Csv -Path $CSVPath |
Where-Object -FilterScript {
$_.trunkPortgroup
}

# Loop through all _s in the CSV
ForEach ($_ in $csv)
{
Import-Module VMware.VimAutomation.Vds
Set-VDPortgroup $_.trunkPortgroup -VlanTrunkRange $_.trunkRange | out-null
}

@"
============================================
Disconnecting from vCenter....Done!
============================================
"@
# Disconnect vCenter
Disconnect-VIServer $vCenter

Use PowerCLI to patch multiple hosts

I was chatting to a friend who was looking to patch multiple (120) hosts with the same VIB and we discussed using PowerCLI to automate it. I did a quick google and didn’t find a script to do exactly what we needed. I did however find a script to do most of what we needed and modified it to do the VIB install and some basic logging! The script i used as a baseline is posted here on http://www.virtadmin.com/ Below is my modified version to install a VIB on each host before the reboot. ***As with any script please test extensively before running in a production environment.***

The full script can be found here

##############################################
# Script to patch multiple hosts with a VIB
# Inspired by http://www.virtadmin.com/rolling-reboot-vsphere-cluster-powercli/
# Hosts are listed in a text file
# Requires Powercli
##############################################
# Region User Variables

# Set vCenter Hostname Variable. 
# You will be asked for credentials when executing the script. e.g. "vc01.lab.local"
$vCenterServer = "ChangeME"

# Full Path to the text file with the list of ESXi hosts to be patched. 
# e.g. "C:\Scripts\VIHosts.txt"
$VIHosts = "ChangeME"

# Full path to the VIB to be installed. Use a common shared datastore. 
# Suggest a shared NFS/VMFS datastore. Copy the VIB to this location before starting. 
# e.g. "/vmfs/volumes/NFS_Shared/patch1/metadata.zip"
$vibPath = "ChangeME"

# Full path to where you would like the script to create a log. e.g. "C:\Scripts"
$LogDIR = "ChangeME"

# End Region User Variables

# DO NOT MODIFY BELOW THIS LINE!
##############################################

# Load VMware PowerCLI CmdLets
Add-PSSnapin vmware* | Out-Null

$TargetDIR = "$LogDIR\log"
if(!(Test-Path -Path $TargetDIR )){
    New-Item -ItemType directory -Path $TargetDIR
}

$Logfile = "$TargetDIR\patchESXi-Log.txt"
if(!(Test-Path -Path $Logfile )){
    New-Item -ItemType file -Path $Logfile
}

Function verifyTXTPath {
# Verify TXT Path
 if (!(Test-Path $VIHosts))  {
  Write-Host "ESXi Hosts File Not Found. Please verify the path and retry
						 " -ForegroundColor Red
	Write-Host -NoNewLine 'Press any key to exit...' -ForegroundColor Yellow;
$null = $Host.UI.RawUI.ReadKey('NoEcho,IncludeKeyDown');
Exit
		}
		  }


# Function to write Logfile entries
Function LogWrite
{
   Param ([string]$logstring)

   Add-content $Logfile -value $logstring
   }

   
# Connect to  the vCenter defined at the top of this script
Connect-VIServer -Server $vCenterServer | Out-Null

# Write progress to LogFile
LogWrite "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Connected to vCenter $vCenterServer"
 
# Get ESXi hostname from txt file
$ESXiServers = Get-Content $VIHosts | %{Get-VMHost $_}
 
# Install ESXi Patch Function
Function PatchESXiServer ($CurrentServer) {
    # Get ESXi Server name
    $ServerName = $CurrentServer.Name
 
    # Put server in maintenance mode
    Write-Host "** Patching $ServerName **"
    Write-Host "Entering Maintenance Mode"
    Set-VMhost $CurrentServer -State maintenance -Evacuate | Out-Null
	
# Write progress to LogFile
LogWrite "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Put ESXi host $CurrentServer in Maintenance Mode"
	
	# Install Patch
	Get-VMHost $CurrentServer | Install-VMHostPatch -Hostpath $vibPath

# Write progress to LogFile
LogWrite "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Installing Patch on $CurrentServer"
 
    # Reboot host
    Write-Host "Rebooting $ServerName"
    Restart-VMHost $CurrentServer -confirm:$false | Out-Null

# Write progress to LogFile
LogWrite "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Rebooting $CurrentServer"
	
    # Wait for Server to show as down
    do {
    sleep 15
    $ServerState = (get-vmhost $ServerName).ConnectionState
    }
    while ($ServerState -ne "NotResponding")
    Write-Host "$ServerName is Down"
 
    # Wait for server to reboot
    do {
    sleep 60
    $ServerState = (get-vmhost $ServerName).ConnectionState
    Write-Host "Waiting for $ServerName to Reboot ..."
    }
    while ($ServerState -ne "Maintenance")
    Write-Host "$ServerName is back up"
 
    # Exit maintenance mode
    Write-Host "Exiting Maintenance mode"
    Set-VMhost $CurrentServer -State Connected | Out-Null
    Write-Host "** Reboot Complete **"
    Write-Host ""
	
# Write progress to LogFile
LogWrite "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Exit ESXi host $CurrentServer from Maintenance Mode"
	
}
 
## MAIN
verifyTXTPath
foreach ($ESXiServer in $ESXiServers) {
PatchESXiServer ($ESXiServer)
}
 
# Disconnect from vCenter
Disconnect-VIServer -Server $vCenterServer -Confirm:$False

# Write progress to LogFile
LogWrite "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Disconnect from vCenter $vCenterServer"

Powershell Script – Create AD OUs/Groups/Users

When planning a build of any kind of environment it is good practice to pre plan all user/service/application accounts that you will need. Now you could manually create each user but you’ve probably got better things to do (Beer anyone?!) so a script to do it all helps right?!

Continue reading “Powershell Script – Create AD OUs/Groups/Users”