Create a local Tenant user in vRA7 using the rest API

VMware vRealize Automation 7.x is the latest version of VMware’s cloud automation software. With the new version comes a change in the way tenants (and user roles for same) are setup. In previous versions you would do the following to create a tenant

Log into the default tenant
Create a new tenant
Add an identity store
Add domain users/groups as tenant & IaaS admins
Log into the new tenant as a tenant and IaaS admin and start configuring the tenant

With vRA 7.x the procedure changes (and becomes more cumbersome IMO)

Log into the default tenant
Create a new tenant
Create a local user for the tenant
Add the local user as a tenant & IaaS admin
Log into the new tenant as the local user
Setup identity store directories
Log back into the default tenant
Edit the new tenant
Add domain users/groups as tenant & IaaS admins
Log into the new tenant as a tenant and IaaS admin and start configuring the tenant

So when it comes to trying to automate tenant creation its not as easy as it used to be for 6.x. For 6.x I’ve used cloud client (if you’re not familiar with CloudClient i’d recommend checking it out here https://developercenter.vmware.com/tool/cloudclient/4.1.0 )

The CloudClient commands required for 6.x are as follows:

Firstly login to CloudClient

vra login userpass --server vra-vip.domain.local --user administrator@vsphere.local --password Password123! --tenant vsphere.local

Next create the tenant

vra tenant add --name NewTenant --url newtenant

Next add the identity store for AD authentication

vra tenant identitystore add --tenantname NewTenant --name Domain.local --url ldap://domain.local --groupbasedn 'ou=vRA,DC=domain,DC=local' --domain domain.local --userdn 'cn=adbind_vra,OU=vRA,DC=domain,DC=local' --password Password123! --type AD --userbasedn 'ou=vRA,DC=domain,DC=local'

Next add Tenant administrators to the new tenant

vra tenant admin update --tenantname NewTenant --role TENANT_ADMIN --action ADD --users vRA_Tenant_Admins@domain.local

Next add IaaS Administrators to the new tenant

vra tenant admin update --tenantname NewTenant --role IAAS_ADMIN --action ADD --users vRA_IaaS_Admins@domain.local

So enter vRA 7.x. Using CloudClient the first 2 steps are the same as before

Firstly login to CloudClient

vra login userpass --server vra-vip.domain.local --user administrator@vsphere.local --password Password123! --tenant vsphere.local

Next create the tenant

vra tenant add --name NewTenant --url newtenant

Next we need to create a local user in the tenant. I couldn’t find a function in CloudClient to create the local user. I checked the developer API guide and I also tried the excellent community module PowervRA http://www.jonathanmedd.net/2016/03/introducing_powervra.html but no joy so i enlisted the help of my colleague Sean Leahy @leahy_s to query the REST API. We ran the add user operation manually and monitored the process in FireFox (Press F12 to get access to the browser console debugger.) Manually enter the user details and click Ok and you will see a POST operation in the console. See below

Click on the POST operation and inspect the REST information. In the headers tab you can see the Content-Type is application/json and in the params tab you can see the post data

Using the API guide we found the required REST method to get an authentication token from vRA that will allow us to POST data. In this example i’m using Advanced REST client in Chrome to retrieve the auth token from vRA. So the URL to POST to is https://vra-appliance-FQDN/identity/api/tokens and the credentials payload needs to be JSON formatted. See below

The response back should contain a base64 encoded string that will be used as an authorization token to post the new user.
So again using Advanced REST client in chrome (or your preferred method) we need to do a POST operation to create the user. This time you need to use URL https://vra-appliance-FQDN/identity/api/tenants/NewTenant/principals. In the headers you need Accept set to Application/json and Authorization set to the base64 encoded token string we got in the previous step. IMPORTANT: You must prefix the token with Bearer or it will not work! See below

Here is the full JSON payload as you cant see it all in the screenshot

{"@type": "User",
 "firstName": "vRA",
 "lastName": "Admin",
 "emailAddress": "vraadmin@domain.local",
 "description": "vRA Admin",
 "locked": false,
 "disabled": false,
 "password": "Password123!",
"domain": "vsphere.local",
"userName":"vraadmin",
 "principalId": {"domain": "vsphere.local",
 "name": "vraadmin"}
 }

Ok deep breaths…if you’re still with me you can now proceed to adding the user to the tenant and IaaS administrator groups!
In the interest of trying out different tools I decided to try using PowervRA for this task. (I will revisit this using the REST API directly next week) I wont go into installing PowervRA. There are good instructions here https://github.com/jakkulabs/PowervRA
In PowervRA run the following commands

Add-vRAPrincipalToTenantRole -TenantId NewTenant -PrincipalId vraadmin@vsphere.local -RoleId CSP_TENANT_ADMIN

Add-vRAPrincipalToTenantRole -TenantId NewTenant -PrincipalId vraadmin@vsphere.local -RoleId COM_VMWARE_IAAS_IAAS_ADMINISTRATOR

Not sure why the complete difference in RoleId name format but thats the only way i could get it to work!

So now we have a tenant with a local user that is both tenant admin & IaaS admin but we still dont have an AD directory service to authenticate AD users. So for this we will again leverage the REST API. Using the same auth token as before, this time we need to hit https://vra-appliance-FQDN/identity/api/tenants/NewTenant/directories

Here is the full JSON payload as you cant see it all in the screenshot

{
@type: "IdentityStore",
domain: "domain.local",
name: "Domain.local",
alias: "Domain",
type: "AD",
userNameDn: "cn=adbind_vra,OU=EHC,DC=domain,DC=local",
password: "Password123!",
url: "ldap://domain.local:389",
groupBaseSearchDn: "ou=EHC,DC=domain,DC=local",
userBaseSearchDn: "ou=EHC,DC=domain,DC=local"
}

We can now go and add our domain users as Tenant admins and IaaS admins to allow domain users to log into the new tenant! So its a longer process than 6.x but its still doable (even if parts of it are undocumented in the API guide!). I will be working on scripting this for multiple tenants so hopefully keep an eye out for a follow up post as i fumble my way through using the vRealize Automation REST API! 🙂

Special thanks again to my colleague Sean Leahy for the REST pointers!

Use PowerCli to answer VM questions

Recently had a datastore in the lab briefly fill up and some VMs go into a suspended state awaiting a Retry/Cancel question to be answered. Rather than manually answer the question on each affected VM i looked to use PowerCli. With a little digging in the PowerCli documentation I found the Get-VMQuestion & Set-VMQuestion cmdlets. In an ideal world all you would need to do is this

Connect-VIServer -Server VC01
Get-VM | Get-VMQuestion | Set-VMQuestion -Option "Retry" -Confirm:$false
Disconnect-VIServer -Server VC01 -Confirm:$false

As i am using vCD all VMs have a UID appended with a space like this

IaaS01 (c4920308-f901-45be-b99c-0095175099e9)

So when you try to pass the VM name to Get-VMQuestion it does not handle the space in the VM name and the command fails. To get around this i created a simple script to first create an array of VM names thereby avoiding the issue with the space. Running the script below against my vCenter answered about 65 VM questions while i made a coffee!

Connect-VIServer -Server VC01
$vm = @()
$vm = Get-VM
$answerQuestion = $vm | Get-VMQuestion | Set-VMQuestion -Option "Retry" -Confirm:$false
Disconnect-VIServer -Server VC01 -Confirm:$false

Script to create vCenter clusters, switches, portgroups & VLANs

When setting up a new environment it is always good practice to plan out and document everything to avoid errors. From hostnames, cluster names, vDS names, portgroup names & VLANs etc. I was recently asked to help script the creation of the above from a csv to avoid fat fingers and to save time through automation. Enter PowerCli! Here is the script i created. Disclaimer: Its version 1…no error checking and can be made more efficient but it works and may be helpful to others! I’ve also posted it to GitHub with an example csv here https://github.com/LifeOfBrianOC/vmware_scripts


# Script to create vCenter Clusters, Distributed Switches & portgroups from CSV
# Edit the CSV location variable and the vCenter FQDN and run the script
# Tested with PowerCli 6.3.0
# This is version 1.0 There is no error checking in place so if an item
# already exists or cannot be found the script will error but should continue

#
$CSVPath = "C:\Scripts\Example.csv"
$vCenter = "vc01.domain.local"

#####################################
# DO NOT EDIT ANYTHING BELOW THIS LINE
#####################################

# Load VMware PowerCli Snapins
add-psSnapin VMWare* | out-null

# Connect to vCenter
Connect-VIserver $vCenter

# Get vCenter Datacenter Name
$datacenter = Get-Datacenter

@"
====================================
Creating Clusters
====================================
"@

# Import CSV and only read lines that have an entry in clusterName column
$csv = @()
$csv = Import-Csv -Path $CSVPath |
Where-Object -FilterScript {
$_.clusterName
}

# Loop through all _s in the CSV
ForEach ($_ in $csv)
{
New-Cluster -Location $datacenter -Name $_.clusterName -HAEnabled | out-null
}

@"
====================================
Creating Distributed Switches
====================================
"@

# Import CSV and only read lines that have an entry in switchName column
$csv = @()
$csv = Import-Csv -Path $CSVPath |
Where-Object -FilterScript {
$_.switchName
}

# Loop through all _s in the CSV
ForEach ($_ in $csv)
{
Import-Module VMware.VimAutomation.Vds
New-VDSwitch -Location $datacenter -Name $_.switchName -Mtu 1600 | out-null
}

@"
==========================================
Creating Distributed Switch Portgroups & Assigning VLANs
==========================================
"@

# Import CSV and only read lines that have an entry in portgroupName column
$csv = @()
$csv = Import-Csv -Path $CSVPath |
Where-Object -FilterScript {
$_.portgroupName
}

# Loop through all _s in the CSV
ForEach ($_ in $csv)
{
Import-Module VMware.VimAutomation.Vds
New-VDPortgroup -Name $_.portgroupName -VDSwitch $_.addToSwitch -VlanId $_.vlan | out-null
}
@"
==========================================
Setting Trunk Ports
==========================================
"@

# Import CSV and only read lines that have an entry in trunkPortgroup column
$csv = @()
$csv = Import-Csv -Path $CSVPath |
Where-Object -FilterScript {
$_.trunkPortgroup
}

# Loop through all _s in the CSV
ForEach ($_ in $csv)
{
Import-Module VMware.VimAutomation.Vds
Set-VDPortgroup $_.trunkPortgroup -VlanTrunkRange $_.trunkRange | out-null
}

@"
============================================
Disconnecting from vCenter....Done!
============================================
"@
# Disconnect vCenter
Disconnect-VIServer $vCenter

Reload all invalid virtual machines on a single host

From time to time VMs can show in an invalid state in vCenter. It can be difficult to get them back to a normal state. This is a useful command that i found at the end of this KB https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003742
for a in $(vim-cmd vmsvc/getallvms 2>&1 |grep invalid |awk '{print $4}'|cut -d \' -f2);do vim-cmd vmsvc/reload $a;done

If you get an Unexpected token error when running the above command it is probaby due to wordpress messing up the ‘ characters so either replace them or copy the command from the KB!

Failed vRA IaaS Web Server Install

There are many reasons why a vRealize Automation IaaS Web server install can fail

MSDTC Issues
NTP Issues
DNS Issues
Certificate Issues
Mental Issues…. (Caused by all of the above!)

I hit this error in the lab and the fix was a new one to me so said i’d document it. The IaaS web server was failing to install and the error in the logs is below which pointed to the certs

The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. —> System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. —> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.

I checked and double checked all certificates and SAN attributes of each component, checked DNS resolution forward and reverse, verified NTP etc. Then it occurred to me that the signing CA is from a different domain than the domain i was installing in so i added the root cert to the local trusted root certification store and low and behold the install completed successfully. Had to do the same on all other IaaS VMs. So even though the root and signing certs were in the personal store with the SAN certs it was not sufficient.

Find MoRef ID using powercli

From time to time you need to find the moref of an object in vCenter. This is a quick powercli one liner to get the name and ID of an object type (VM, Datastore etc)

This assumes you are already connected to vCenter using Connect-VIServer

# Datastores

Get-Datastore | Select Name,ID

This command returns output like below. In this example i have 1 datastore

#VMs

Get-VM | Select Name,ID

This command returns output like below

Get-VM

Reset VMware appliance root password

Use the following procedure to change the root password for a VMware appliance. This process uses single user mode so it allow you to change the root password if you have forgotten it!

Open the VM console and reboot the appliance

Select the line that starts with kernel and type e to edit the line

At the end of the line, press the spacebar and type init=/bin/sh

Press Enter to exit edit mode.

On the GRUB screen, type b to boot into single-user mode.
The virtual appliance boots in single-user mode.

To change the root user password, type passwd root and follow the on-screen prompts to enter a new password for the root user

To restart the virtual appliance, type reboot and press Enter.

When the virtual appliance restarts, you can log in using the new password.