VMware vCenter 5.1 Installation: Part 3 (Install SSO Service SSL Certificate)

UPDATE 1/30/2013: Jurgen Van De Perre has updated his SSO certificate script to comply with the new VMware guidance. You can get his script here. His script will save you a lot of time and headache when replacing the SSO certificates. You will still need to follow my Update Trusted Certificate Store procedures, below, after running his script.

UPDATE 1/27/2013: This article was updated with new information from the VMware KB articles and reader feedback. I have a lot more details on using intermediary CAs.

UPDATE 10/26/12: While no changes in my original content are required for 5.1.0A, I've consolidated the updates and slightly tweaked the wording for better clarity. If you want to start at part one of this series you can find it here. You can find the official VMware KB article which covers these steps here.
--

In Part 2 of my series on installing and configuring VMware vCenter 5.1 we created a trusted keypair and related certificate files to replace the self-signed certificates of the VMware vCenter Single Sign On service. The resulting files for the SSO service from Part 2 are shown below with a short explanation:

rui.crt - Minted PKCS#7 x.509 certificate from your trusted CA
rui.csr - Base-64 encoded certificate signing request (CSR)
rui.key - Private RSA 2048-bit key
rui.pfx - Password protected PKCS #12 certificate file, private key, and root CA certs
SSO.cfg - OpenSSL certificate request configuration file for the SSO service
root-trust.jks - Java keystore with root certificates
server-identity.jks- Java keystore with root certificates

Now that you have a installed the SSO service, generated your x.509 SSL certificates, lets proceed to updating the Single Sign On and Lookup Service certificates on your Windows Server 2008 R2 server. My steps are assuming the SSO service is installed in the default directory. If you have changed the path, then you will need to modify the steps.

This process is a bit tedious and error prone, so I would strongly encourage your take a snapshot of your vCenter VM prior to starting these steps so you can easily revert back to a known clean state should you botch the replacement process. For a script which automates this process, you can check out this link. I've used that script, and it works great. It just automates the process below, but as you will see, this is tedious so a script is very welcomed indeed.

1. A directory is required to store all of the SSO certificate files. The directory can be anywhere, but the vCenter service accounts needs full access. I would not use a directory path with spaces. I would suggest using:

c:\ProgramData\VMware\SingleSignOn\SSL\

If you used my pre-staging script, it already has created this directory and copied the required files. If you didn't use my pre-staging script, from the D:\certs\SSO directory copy: rui.crt, rui.key, rui.pfx, root-trust.jks, and server-identity.jks into this directory. From the D:\certs directory copy root64.cer into the same SSL directory.

Your directory should now look like the following screenshot:

2. To reconfigure the SSO SSL certificates we first must list the service records for the Group Check, SSO and Security Token Services. Open an elevated command prompt and type the following commands. 

SET JAVA_HOME=C:\Program Files\VMware\Infrastructure\jre

cd /d C:\Program Files\VMware\Infrastructure\SSOServer\ssolscli

ssolscli.cmd listServices https://Server.FQDN:7444/lookupservice/sdk

At this point you should see a lot of output on the screen and three "Service" sections with a bunch of details. Be sure to use the FQDN of the server, or the command may fail.

3. Use Notepad to create three separate text files, one for each of the services (STS, group check, SSO). I created three text files (sts.properties, gc.properties, admin.properties) in D:\Certs. The content of each file is shown below. Of course change your uri hostname to that of your SSO server, and the ssl variable to the proper path of the root certificate. 

The "protocol" field is extremely critical and NOT the same for all three services. Using "wsTrust" for all three result in a smoking hole and a dead SSO server. 

sts.properties

[service]
friendlyName=STS for Single Sign On
version=1.0
ownerId=
type=urn:sso:sts
description=The Security Token Service of the Single Sign On server.

[endpoint0]
uri=https://SSOServer.Domain:7444/ims/STSService
ssl=c:\ProgramData\VMware\SingleSignOn\SSL\Root64.cer
protocol=wsTrust

gc.properties

[service]
friendlyName=The group check interface of the SSO server
version=1.0
ownerId=
type=urn:sso:groupcheck
description=The group check interface of the SSO server

[endpoint0]
uri=https://SSOServer.Domain:7444/sso-adminserver/sdk
ssl=c:\ProgramData\VMware\SingleSignOn\SSL\Root64.cer
protocol=vmomi

admin.properties

[service]
friendlyName=The administrative interface of the SSO server
version=1.0
ownerId=
type=urn:sso:admin
description=The administrative interface of the SSO server

[endpoint0]
uri=https://SSOServer.Domain:7444/sso-adminserver/sdk
ssl=c:\ProgramData\VMware\SingleSignOn\SSL\Root64.cer
protocol=vmomi


3. Create three more text files (sts_id, gc_id, admin_id) in the D:\Certs directory. This time we need to put the unique serviceID shown in step 2 (which will NOT be the same for your install, so don't blindly use my IDs below). Do not enter any additional information to the contents of the file.

 

 

4. As a check point in our configuration process, you should now have the following files in your Certs directory. I would also strongly suggest taking another snapshot of your SSO VM at this point, in case the certificate replacement process goes south and you need to recover to a known state. Certificate information is also stored in the SSO SQL database, so I would urge that you use SQL Studio to do a SSO database backup as well.

5. Stop the vCenter Single Sign On service via the Windows Server Manger.

6. Navigate to: C:\Program Files\VMware\Infrastructure\SSOServer\security and backup the root-trust.jks and server-identity.jks files. Replace these files with your versions (from D:\Certs\SSO).

7. Open an elevated command prompt and type the following commands. When prompted for the master password, enter the SSO password you entered during the SSO installation process.

SET JAVA_HOME=C:\Program Files\VMware\Infrastructure\jre

cd /d C:\Program Files\VMware\Infrastructure\SSOServer\utils

ssocli configure-riat -a configure-ssl --keystore-file C:\ProgramData\VMware\SingleSignOn\SSL\root-trust.jks --keystore-password testpassword




8. Re-start the vCenter Single Sign On service and wait a couple of minutes.

9. Open a web browser and open:

https://SSOserver.FQDN:7444/sso-adminserver/sdk

View the SSL certificate and validate that it's your trusted certificate, not the VMware self-signed certificate.

10. To bind the trusted SSL certificate to each of the three services issue the commands below in the same command prompt Window as above. Note that "updateService" is CASE SENSITIVE.

cd /d C:\Program Files\VMware\Infrastructure\SSOServer\ssolscli

ssolscli.cmd updateService -d https://SSOServer.Domain:7444/lookupservice/sdk -u admin@System-Domain -p YourPassword -si D:\certs\sts_id -ip D:\certs\sts.properties

If the command is successful you will see the following. Repeat the process for the remaining two services.

 

ssolscli.cmd updateService -d https://SSOServer.Domain:7444/lookupservice/sdk -u admin@System-Domain -p YourPassword -si D:\certs\gc_id -ip D:\certs\gc.properties

ssolscli.cmd updateService -d https://SSOServer.Domain:7444/lookupservice/sdk -u admin@System-Domain -p YourPassword -si D:\certs\admin_id -ip D:\certs\admin.properties

11. If you are lucky at this point you have successful return codes for all three commands. However, you may not be out of the woods. Restart the vCenter Single Sign on service and wait a couple of minutes. Re-run step two to list all three services and verify everything is successful.

If you get errors such as "Return Code is: OperationFailed", then Houston, we have a problem. If you see "unable to connect to server" errors while trying to list the services, wait a few minutes and try again.

Update Trusted Certificate Store

Note: You must pay close attention to the version of OpenSSL you are using, or you could end up with the wrong hashes. OpenSSL 1.x has a different default hashing algorithm than the 0.9.8 tree. I show both command lines below, which in my experience, produce the same hashes. I've tried 0.9.8y, 1.0.1c and 1.0.1e all with the same hashing results, assuming you use the proper switches.

1. In Windows Explorer navigate to C:\ProgramData\VMware and see if you have a SSL directory. If not, create one (all CAPS).

2. Copy your Root64.cer file into C:\ProgramData\VMware\SSL and rename it ca_certificates.crt. If you have intermediary CAs, the ca_certificates.crt file should have all CA certificates appended to it. This should already be the case if you followed Part 2 of my guide, under the Download Root Certificates section. Double check the ca_certificates file ends in "crt" NOT "cer".

3. Compute the hash of your Root64.cer file if you have a single CA hierarchy. If you have intermediary CAs, compute the hash of your Root64-1.cer (root CA) in this step. To do this with OpenSSL 1.0.0 and later use the following command:

c:\OpenSSL-Win32\bin\openssl x509 -subject_hash_old -noout -in D:\certs\Root64.cer

If you are using an OpenSSL version prior to 1.0.0, such as 0.9.8 use this command:

c:\OpenSSL-Win32\bin\openssl x509 -subject_hash -noout -in D:\certs\Root64.cer

4. If you have a single CA (no intermediary) copy your Root64.cer file into the SSL directory and rename it using the hash value from the previous step, and change the file extension to 0 (zero). For example, I would copy D:\certs\root64.cer to the SSL folder in step 2, and rename it to 97527d09.0.

If you have intermediary CAs, then copy the Root64-1.cer (your root cert only) file into the SSL folder and rename it using the same methodology. You also need to compute the hash for any intermediary CAs (e.g. Root64-2.cer) and copy those files into the SSL directory and rename them.

In short, each hash file should have only the contents of a single CA certificate. They cannot be appended like they are in the ca_certificates.crt file. Do not create a hash file for your concatenated Root64.cer file, when you have intermediary CAs.

Next up is part four, Installing the vCenter Inventory Service.