VMware Cloud Foundation & VMware Validated Solutions

vClound Director

VCF Health Check, Monitoring and Logs

# VCF health check - sos command
/opt/vmware/sddc-support/sos -h     # list options
/opt/vmware/sddc-support/sos --services-health
/opt/vmware/sddc-support/sos --ntp-health

# VCF Dashboard
Check VCF dashboard for health and monitoring

#*** VCF logs
# System logs
/var/log/vmware/vcf/lcm     # lcm log directory
/var/log/vmware/vcf/lcm/lcm.log
/var/log/vmware/vcf/lcm/lcm-debug.log

# Operations Manager Logs, such as certificate changes, vRealize production
/var/log/vmware/vcf/operationsmanager

# Common svcs - API login issue, SDDC component connectivity
/var/log/vmware/vcf/commonsvcs

VCF Design and Implementation

https://www.vmwarensxcloud.com/2021/10/step-by-step-vmware-cloud-foundation-43.html

https://www.vmwarensxcloud.com/2022/03/Step-by-step-VMware-Cloud-Foundation-4.3-design-and-install-upload-vRNI.html

VMware Validated Design - Superceded by VMware Validated Solutions

https://docs.vmware.com/en/VMware-Validated-Design/index.html

vCloud Builder - Deployment

VMware Cloud Foundation fails to claim vSAN Disks on a host using all flash storage when creating a workload domain (52586) https://kb.vmware.com/s/article/52586

VMware Cloud Foundation (VCF) vSAN with multiple disk groups https://communities.vmware.com/t5/VMware-Cloud-Foundation/VMware-Cloud-Foundation-VCF-vSAN-with-multiple-disk-groups/td-p/2914691

Mark Flash Devices as Capacity Using ESXCLI https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vsan-planning.doc/GUID-42E65085-1DA5-4C0B-A397-3497CBBC600E.html

Upgrade the Management Domain to VMware Cloud Foundation 4.5 https://docs.vmware.com/en/VMware-Cloud-Foundation/4.5/vcf-lifecycle/GUID-E101AFB5-1034-4CF9-B96E-A2E70DCF02F5.html

Deployment Overview of VMware Cloud Foundation https://docs.vmware.com/en/VMware-Cloud-Foundation/4.5/vcf-getting-started/GUID-C000FB7D-5A4C-4AA7-AEAB-D67F335BCB21.html

Configure NSX DHCP https://thewificable.com/2020/10/08/how-to-configure-dhcp-in-nsx-t/

vCloud Builder - vCF Deployment

Part 4 – Cloud Builder Validation Error: SSL Certificate common name doesn’t match ESXi FQDN: ESXi Host Configuration Validation Error

https://vmanalyst.com/cloud-builder-validation-error-ssl-certificate-common-name-doesnt-match-esxi-fqdn/

Issue The cause of the above issue is during the installation of ESXi, the installer generates a self-signed certificate for each ESXi host but the process is performed prior to the ESXi identity being configured. This means all ESXi hosts have a common name in their self-signed certificate of localhost.localdomain.

Generate new ESXi host certificates

This error can be prevented at times by generating an ESX Host certificate however this may or may not work so here is the write-up to cover that.

This is a very straight foward and well known method to replace esx certificates

1. Establish an ssh session to the esx host
2. Set the hostname and fdqn as shown from CLI
    esxcli system hostname set –host=sfo01-m01-esx02
    esxcli system hostname set –fqdn=sfo01-m01-esx02.ash.local

Navigate to /etc/vmware/ssl directory

The two files we are interested are rui.crt and rui.key – The cert and key file

# List rui* files
    ls -ltr /etc/vmware/ssl/rui*
        -r——– 1 root root 1704 Apr 21 17:23 /etc/vmware/ssl/rui.key
        -rw-r–r– 1 root root 1411 Apr 21 17:23 /etc/vmware/ssl/rui.crt

# Create backup of the rui* files
    cp -a rui.* /cert-backup/

# To regenerate new certificate on esx, just run the command as shown
    /sbin/generate-certificates

# Restart hostd and vpxa services by executing the following command:
    /etc/init.d/hostd restart && /etc/init.d/vpxa restart 

# Restart the host
    reboot

# Verify self signed SSL certification common name has been updated with FQDN
    Access esxi via https://<esxi-ip>
    Check the certificate common name

Update ESXi host ntp.conf with required NTP servers

# Verify /etc/ntp.conf
    Ensure there are entry
        server x.x.x.x,y.y.y.y
            Where the ntp server IP addreses, separated by ","
                x.x.x.x
                y.y.y.y

# Access ESXi host via https://<esxi-ip>
a. Navigate to "Manage" -> System -> Time & Date
b. Click "Edit NTP settings"
        Update with required NTP server, separate by ","
c. Start the ntpd service

vSAN Disk Availability Validation (All flash)

During the VCF deployment, fails to deployment vSAN, getting error

esxi host ESXi-fqdn has two different sizes of vSAN eligible disks.

# ssh to all esxi hosts that will be installed in management domain
a. verify disks information
    esxcli storage core device list
        # check "Is SSD: true"

vCloud Builder - deploy VCF logs

Check vCloud Builder logs

# Check VCF bringup log
https://virtualrove.com/2020/05/02/vcloud-foundation-4-0/

tail -f vcf-bringup.log

# Check NTP sync on the cloud builder
cd /opt/vmware/bringup/logs
ntpq -p
        # verify ntpq output

# Steps to manually sync NTP 
    ntpq -p
    systemctl stop ntpd.service
    ntpdate 172.16.31.110
    Wait for a min and again run this
    ntpdate 172.16.31.110
    systemctl start ntpd.service
    systemctl restart ntpd.service
    ntpq -p

# use this command to pause the deployment when required
    systemctl restart vcf-bringup

Getting started with VCF Part 7 – NSX-T Edge

https://cormachogan.com/2020/01/31/getting-started-with-vcf-part-7-nsx-t-edge/

VCF Upgrade

VMware Cloud Foundation Offline Bundle https://www.vjonathan.com/post/vmware-cloud-foundation-offline-bundle/

Upgrade the Management Domain to VMware Cloud Foundation 4.5

https://docs.vmware.com/en/VMware-Cloud-Foundation/4.5/vcf-lifecycle/GUID-E101AFB5-1034-4CF9-B96E-A2E70DCF02F5.html

Step-by-step VMware Cloud Foundation Repository update without Internet https://www.vmwarensxcloud.com/2021/07/how-to-update-vmware-cloud-foundation.html

VCF Administration

Add Host https://www.vmwarensxcloud.com/2022/04/Step-by-step-VMware-Cloud-Foundation-4.3-design-and-install-Add-Host.html

Decommision Host https://www.vmwarensxcloud.com/2022/05/Step-by-step-VMware-Cloud-Foundation-4.3-design-and-install-decommision-Host.html

How to Remove host from VMware Cloud Foundation cluster

1. Login to SDDC manager
2. Navigate to workload domain and choose the domain where we have the cluster which needs to be downsized.
3. Now move to the clusters tab and select the cluster
4. Move to hosts tab.
5. Now choose the host which needs to be decommissioned and press "Remove selected hosts" button.
6. Confirm the removal and sddc manager will start removing host from cluster.
7. Monitor the progress.

How to decommission host from VCF inventory

Navigate to hosts tab and verify the status of the host needs removal. It shouldn't be assigned to any cluster. Host removed from the cluster will be in needs clean up state, hence if you wish to re-purpose this host, please perform clean up.
Navigate to unassigned hosts tab and choose the host which needs to be decommissioned.
Wait for Decommission selected host button to highlight and press it.
Confirm the host decommission request.
Monitor the progress in the tasks pane.

How to install vRSLCM

https://www.vmwarensxcloud.com/2021/12/Step-by-step-VMware-Cloud-Foundation-4.3-design-and-install-vrlcm.html

VMware vRealize Lifecycle Manager comes free with VMware vRealize Suite. It is designed to simplify the deployment and on-going management of the vRealize portfolio products. It delivers fast time-to value, improved user experience and greater operational efficiency to vRealize Suite customers.

1. For deploying vrealize lifecycle manager, please login to SDDC manager and navigate to bundle management tab.
    Validate or download the applicable build number as per BOM for vrlcm.
2. Once download is finished, navigate to vRealize suite tab. You will find that deploy button  is now available.
3. Click on deploy button and you will be presented with installation pre-reqs.  Confirm all and begin
4. On the next screen review the network settings.
5. Now populate appliance settings
    FQDN will be looked up in dns database and accordingly IP will be assigned to the virtual appliance, 
    NSX-T T1 gateway IP is referred to IP you have reserved for load balancer.
6. Now on next screen review provided settings and click finish.
7. Wait for all tasks to finish.
8. After successful deployment you will be able to see vrlcm listed under management workload domain services.

How to install Workspace ONE Access - WSA

https://www.vmwarensxcloud.com/2022/01/Step-by-step-VMware-Cloud-Foundation-4.3-design-and-install-WSA.html

If you go to vRealize tab on SDDC manager before installing vRLCM you will get a clear message as in you need to deploy it before you can deploy other vRealize suite products.

As shown in the image below. Once vRLCM is deployed we start vRealize suite deployment with workspace one access.

VMware workspace one access is a vRealize suite product, which was formally known as VMware identity manager.

VMware Workspace ONE Access delivers multifactor authentication (MFA), conditional access and single sign-on (SSO) for applications delivered by VMware Workspace ONE. By acting as a broker to other identity stores and providers, Workspace ONE Access enables organizations to quickly and more securely implement application and device strategies that deliver consistent, enterprise-wide access to applications and data from any device in any location.

How to deploy VMware Workspace one access?

In order to install WSA we will login to SDDC manager.
Now navigate to workload domain and select management workload domain.
Under services tab and select vrlcm.
Login to vrlcm with vcfadmin@local account.
After successful login, you will be presented with My Services screen. Please click on Locker tile.
Now first thing we do is to generate or import certificate. As I am not using third party certificate I will generate but if you have a third party of CA issued cert then this where you will import it. To generate cert choose Generate option, for importing CA cert use import option and for generating CSR for certificate request use generate CSR option. I am going with Generate option.
Fil l in all details for the certificate, I have only used FQDN however I would suggest using hostname as well as FQDN for server domain/hostname field.
Click generate.
Once certificate is generated or imported, its time we create default passwords. Navigate to password tab on the left hand menu. click on ADD button.
Add password for Global admin and config admin.
Now you should have passwords ready for the deployment.
Now click on vRealize suite lifecycle manager on left hand top corner. Which will take you back to my services screen.
Now select lifecycle Operations.
Once you are in lifecycle operations, navigate to settings and click on binary mappings.
Under binary mappings sync binaries from SDDC manager.
After successful sync, navigate to Create environment tab.
Under create environment option you need to choose to enable Install identity manager radio button.
Now click on select default password.
Select admin password from the list.
Select datacenter and check/uncheck CEIP and click next.
Select product which needs to be deployed, I am going to use standard deployment model, however in a production environment based on design decisions you should deploy cluster model. In standard mode only 1 appliance will be deployed and in cluster mode 3 appliances will be deployed. After choosing deployment type click on next.
Now accept EULA.
Select certificate which we created earlier in the post from the list.
Review infrastructure details and click next.
Review network details and click next.
In products tab, fill in wsa node size and configadmin password, scroll down.
As I am deploying it in standard mode, I have only one appliance which will be deployed, however in cluster mode you need to provide details for all three appliances. VM name is needed for vSphere inventory, where in FQDN is the actual name of the appliance.
Once all information is populated click next and run pre-checks under precheck tab.
Once all pre-checks are complete, click next.
Now submit the task.
Once you sumit the task, you can review progress using requests tab.
After successful completion of all stages, you will see Workspace ONE Access listed under services of management workload domain.

How to install vROPS

https://www.vmwarensxcloud.com/2022/02/Step-by-step-VMware-Cloud-Foundation-4.3-design-and-install-vROPS.html

VCF Troubleshooting

Getting past those annoying issues when administering VMware Cloud Foundation (VCF) https://www.lab2prod.com.au/2021/03/the-unofficial-vcf-troubleshooting-guide.html

Reference sites

VMware Validated Solutions

VMware Validated Solutions for Cloud Provider