Continuing the problem from part 1.
Over the past 8 months this template has been updated with Microsoft security patches (released once a month) and during that time the template OS was activated - while connected to the network.
After the upgrade to 4.1, I noticed that I could no longer deploy a Windows 2008 R2 template with customization. (There were several issues here. In the beginning I could not deploy a template at all - but after a restart of the vCenter server and an SR submitted to VMware on the issue that was solved). But when customizing the the template - either with a manual Customization Spec or with a previous one we had been using for almost a year, the customization would not work. None of it.
Now this can become extremely annoying. Because once you get used to working with deploying a template with a customization spec, to do it manually takes time and can be cumbersome and prone to inconsistencies.The VM name would not be changed, network settings would not work, VM's were not being deployed with their vNIC's connected. There of course is a workaround to the whole problem, to do it manually but as I said before, this is not ideal.
Last week I finally found the reason this was happening, and partly for my own benefit (documentation) and to share the experience, let me explain what was happening.
When a Windows (7 or 2008) OS is activated with a MAK License Key, the OS is automatically set with a ReArm count of 3. This you you can see with a running the following command.
C:\Users\msaidelk>cscript c:\Windows\system32\slmgr.vbs -dlv
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.
Software licensing service version: 6.1.7600.16385
Name: Windows(R) 7, Enterprise edition
Description: Windows Operating System - Windows(R) 7, VOLUME_MAK channel
Activation ID: xxxxx-xxxxx-xxx-ad1e-7fe15931a8dd
Application ID: xxxxx-xxxxx-xxx-983e-d6ec3f16059f
Extended PID: xxxxx-xxxxx-xxx-132882-03-1033-7600.0000-2292010
Installation ID: xxxxx-xxxxx-xxx0249086468222046584484281164682806770
Processor Certificate URL: http://go.microsoft.com/fwlink/?LinkID=88338
Machine Certificate URL: http://go.microsoft.com/fwlink/?LinkID=88339
Use License URL: http://go.microsoft.com/fwlink/?LinkID=88341
Product Key Certificate URL: http://go.microsoft.com/fwlink/?LinkID=88340
Partial Product Key: xxxxx
License Status: Licensed
Remaining Windows rearm count: 3
Trusted time: 04/01/2011 08:56:51
An OS which is licensed with a KMS License will look like this
C:\Windows\system32>cscript slmgr.vbs -dlv
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.
Software licensing service version: 6.1.7601.17105
Name: Windows Server(R), ServerStandard edition
Description: Windows Operating System - Windows Server(R), VOLUME_KMSCLIENT channel
Activation ID: xxxxx-xxxxx-xxx-97be-d11a0f55633f
Application ID: xxxxx-xxxxx-xxx983e-d6ec3f16059f
Extended PID: xxxxx-xxxxx-xxx-03-1037-7600.0000-0132010
Installation ID: xxxxxxxxxxxxx073476703659683995395732413372005
Partial Product Key: xxxHC
License Status: Licensed
Volume activation expiration: 253260 minute(s) (175 day(s))
Evaluation End Date: 01/12/2011 01:59:59
Remaining Windows rearm count: 1
Trusted time: 16/01/2011 23:27:12
Key Management Service client information
Client Machine ID (CMID): a41eaf55-64a2-4509-ac85-2118804023f0
KMS machine name from DNS: ilkms.maishsk.local:1688
KMS machine extended PID: xxxxx-xxxxx-xxx-021634-03-1033-7600.0000-0132010
Activation interval: 120 minutes
Renewal interval: 10080 minutes
KMS host caching is enabled
Once your OS has been activated through a KMS server - your ReArm count will be set to 1. If you try and change it back to a MAK key thereafter - the Rearm count will still be kept as 1.
So what happened? Somewhere along the way, my template was was activated with a KMS Key - which meant I only had 1 ReArm left.
During the deployment process of a VM with a Customization Spec, a number of files files are injected into the VM by vCenter.
Expanding C:\Windows\TEMP\vmw979D.tmp\guestcustutil.exe
Expanding C:\Windows\TEMP\vmw979D.tmp\imgcust-reboot.exe
Expanding C:\Windows\TEMP\vmw979D.tmp\sysprep\guestcustutil.exe
Expanding C:\Windows\TEMP\vmw979D.tmp\sysprep\sysprep.xml
Expanding C:\Windows\TEMP\vmw979D.tmp\sysprepDecrypter.exe
These files are then moved later on to c:\Sysprep
Moving directory 'sysprep' to 'C:'
vCenter takes the information passed from the the Customization Spec, decrypts the info and creates a new sysprep.xml file which is then in turn called from the customization process
Executing command C:\windows\system32\sysprep\sysprep.exe /quiet /generalize /oobe /reboot /unattend:C:\sysprep\sysprep.xml
Now how do I know all of this? A bit of reverse engineering. The deployment logs are located in C:\Windows\Temp\vmware-imc. The Sysprep logs are located here: C:\Windows\System32\sysprep\Panther
Two log files are created during the Sysprep process - setupacct.log and setuperr.log The first is the activity of the Sysprep and the second reports the errors that occurred.
When deploying a VM with an activated KMS license - I was getting these errors from the setuperr.log
2010-01-14 09:42:42, Error [0x0f00a4] SYSPRP WinMain: Unable to parse command-line arguments to sysprep; GLE = 0x36b7[gle=0x000036b7]
2010-01-14 09:42:57, Error [0x0f0043] SYSPRP WinMain:The sysprep dialog box returned FALSE
2010-01-14 09:43:04, Error [0x0f0060] SYSPRP ParseCommands:Found unsupported command line option '/?'[gle=0x000036b7]
2010-01-14 09:43:04, Error [0x0f00a4] SYSPRP WinMain: Unable to parse command-line arguments to sysprep; GLE = 0x36b7[gle=0x000036b7]
2011-01-12 11:18:48, Error [0x0f0082] SYSPRP LaunchDll:Failure occurred while executing 'C:\Windows\System32\slc.dll,SLReArmWindows', returned error code -1073425657
2011-01-12 11:18:48, Error [0x0f0070] SYSPRP RunExternalDlls:An error occurred while running registry sysprep DLLs, halting sysprep execution. dwRet = -1073425657
2011-01-12 11:18:48, Error [0x0f00a8] SYSPRP WinMain:Hit failure while processing sysprep generalize internal providers; hr = 0xc004d307
Even though there was 1 left on my ReArm count this was not working. A quick syprep on the machine attested to that fact.
We will now close this series with how the problem was solved in Part 3.