VMware vCenter 5.1 Installation: Part 5 (Install Inventory Service SSL Certificate)

UPDATE 1/27/2013: Updated the post with 5.1.0b information, which seems to have resolved a script error with the un-registration process. Other minor tweaks as well.

UPDATE 10/26/12: vSphere 5.1.0A *still* seems to have a problem with the unregister script and required me to modify the script to make it work. However, unlike the GA release, updating the SSL certificates post-install in 5.1.0A does not cause the vCenter installer to fail. So I can now recommend that you configure the inventory service with trusted SSL certificates. Pre-population is still easier, but the procedure below seems to work now. You can find the official VMware KB article covering these steps here.
--

In Part 4 of my VMware vCenter 5.1 installation series we installed the Inventory Service, which was pretty darn easy. In any enterprise environment you want to only use trusted SSL certificates, so this post will cover the process of updating the Inventory service to use the trusted SSL certificate we generated in Part 2 of this series. If you want to start back at the beginning of the series, you can check out part one here.

Let's get rocking!

1. The first step is to UN-register the Inventory service from the vCenter SSO service. When using the 5.1 GA and 5.1.0a versions I ran into a problem with unregister-sso.bat file. When I ran it (instructions will follow), I got the following Java error. However, 5.1.0b seems to have corrected the problem, as the command now works for me.

Exception in thread "main" java.lang.NoClassDefFoundError: Service\sso\log4j/xml
Caused by: java.lang.ClassNotFoundException: Service\sso\log4j.xml
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: Service\sso\log4j.xml.  Program will exit.

After some hair pulling, I found a workaround to the problem. I'll first provide the "normal" steps to un-register the service, which hopefully will work for most people. If that fails with the error above, then implement my fix, or a variation thereof.

2. Open an elevated command prompt and type the following commands:

cd /d C:\Program Files\VMware\Infrastructure\Inventory Service\scripts

unregister-sso.bat https://YourServer.FQDN:7444/lookupservice/sdk admin@System-Domain YourPassword

If successful you should see output similar to the following screenshot. Keep the command window open.

3. Should that command fail with the error noted in step 1, make sure you are using the 5.1.0b installer. You can also try this workaround to see if it fixes your problem. In the directory C:\Program Files\VMware\Infrastructure\Inventory Service\sso make a copy of the regTool.cmd file. Then open regTool.cmd in a text editor such as Notepad.

Find the line in the script which reads:

set LOG4J_CONF=-Dlog4j.configuration=file:%TOOL_DIR%\log4j.xml

And change it to the full path (line wraps, but it should be on one line) using quotes, as shown below:

set LOG4J_CONF=-Dlog4j.configuration=file:"C:\Program Files\VMware\Infrastructure\Inventory Service\sso\log4j.xml"

Save the file, then re-run the unregister-sso.bat command in step 2, and see if you have success.

4. Stop the "VMware vCenter Inventory Service".

5. Copy the three key certificate files we created back in part two of my series to the following directory: C:\ProgramData\VMware\Infrastructure\Inventory Service\ssl. First, make a backup of the keys in the SSL folder. Copy the Inventory service certificate files (rui.crt, rui.key and rui.pfx) from the D:\Certs\Inventory directory and overwrite the versions in the SSL folder.

6. Start the "VMware vCenter Inventory Service".

7. In the same command window you kept open from step 2, enter the following command:

register-sso.bat https://YourServer.FQDN:7444/lookupservice/sdk admin@System-Domain YourPassword

If successful you should see output similar to the following screenshot.

8. Browse to the inventory service URL (https://YourServer.FQDN:10443) and validate that the trusted SSL certificate is being used. You will see a 400 Bad request error, but that can be safely ignored. Just validate the browser is showing the trusted certificate is being used.

Congratulations! You have now updated your Inventory Service SSL certificates and can proceed to creating the vCenter and VUM databases and DSN in part 6.