grokgarble.com

Database, Software, and System Engineering

HTTP Error 500.19 “Failed to decrypt attribute “password””

Background

Important information within IIS is stored in flat files with encrypted sections.  The specific sections encrypted are identity of the application pool along with ACL information.  When you move the configurations to a new server, or when you’ve used a hypervisor to clone a server, keys used to decrypt these values will no longer work.  Or more specifically, they’re not found…I’ll get to that (the fourth solution to this problem explains it all).

500.19Error

The 500 primary error means it is a server error. The .19 secondary error code indicates that “The requested page can not be accessed because the related configuration data is invalid.” IIS, points out that the issue is due to its inability to decrypt an attribute of a configuration file for us. Metabase.xml is the central store where IIS 6.0 stores most of its configuration information.  In IIS 7.0 Microsoft shifted to a new XML based configuration store that is modeled after ASP.NET.  The same model is maintained in IIS 7.5.  IIS no longer centralized into a single file. The hierarchical store starts with the applicationHost.config file and can be distributed among web.config files under your application for a more modular inheritance  exposed through the app.configs (web.config).

In IIS 7 the local accounts and groups that IIS 6.0 used (IUSR_MACHINENAME / IIS_WPG) were also replaced by built-in accounts (IUSR / IIS_IUSRS). The built-in accounts have the same SID across Windows 2008/2008 R2 servers and are not machine specific.  The idea is that, like web applications’ configurations, you should be able to have an IIS configuration that could be machine independent.  Therefore, like web.configs, you should be able to just copy your applicationHost.config from one server to another and IIS will pick up the settings and work.

Problem

This all works perfectly, if you don’t care about the identity your application runs under.  In a distributed environment this isn’t always the case.  If you modify the identity of your application pool servicing your application hosted in IIS, then move the applicationHost.config to a different IIS 7.0 server, or clone it you’re greeted with this error when trying to render a page, web service, or worse a failure of application pools with an error from WAS in your event log looking like this:

Application pool <ApplicationPool> has been disabled. Windows Process Activation Service (WAS) did not create a worker process to serve the application pool because the application pool identity is invalid.

If you try to modify the account, depending on how you installed the application  you may not be able to change the password and “reset” this problem away that’s truly a the mismatch of identity on multiple levels.

This issues results in this error:

There was an error while performing this operation.  Details: Bad Data. (Exception from HRESULT: 0×80090005)

It is rooted in the inability to decrypt the sections of the applicationHost.config that’s encrypted with machine keys.  Any custom username / password tried for the identity of the application pool may result in either of these error conditions.  A work around is to set application pool to one of the built in accounts works or remove the encryption in the configuration file.  Both not that great of solutions if you need some level of identity on a remote system or security by encrypting this sensitive data at rest, as it should be.

The issue is with your machine keys used to decrypt sections of your applicationHost.config.  For example, IIS keeps a local copy of the username and password of a custom identity in the applicationHost.config and and encrypts it.  If you open the file you’ll see this:

<processModel identityType="User" userName="domain\username" password="[enc:IISWASOnlyAesProvider:2Woq1bHFmcDxzSEKLe9q1eZsvlUuBcmb0Luy3DzkZWg=:enc]" />

When the applicationHost.config is used on a server it didn’t originate on,  IIS can no longer decrypt the iisConfiguration and iisWasKey containers in this file since it uses the original machine keys to do they encryption.

Solutions

To get this working you have four options, in order of suggested implementation.  The first requiresworking server with a similar configuration, the second requires THE  actual source server, the third is down right insecure, the fourth I would challenge that anyone should use – but works, and is important to understanding the under pinning of what’s going on.

The first three are supported methods by Microsoft documentation, the fourth isn’t – and with all, take proper precautions.

First one (best/easiest)…

Export the IIS config of a working web server and import using the Shared Configuration of IIS 7.

1. Navigate to the root of you server in IIS admin console
2. Double click shared Configuration

IISRoot

3. Click Export Configuration on the left hand pane

ExportConfig

4. Select a location
5. Select a Password
6. And Select OK

6Password

7. This process creates files to be imported on target machine
8. Move the files to a location on the target server
9. In Shared Configuration manager on the target machine check the Enable Shared Configuration and select Apply

SharedConfig

10. Enter the password when prompted
11. Ok to the follow couple of prompts
12. IISReset
13. Navigate to your resource to confirm it renders (no longer an 500.19 error)

ServiceWSDLIIS

14. In Shared Configuration manager on the target machine un-check the Enable Shared Configuration and select Apply
15. Select yes to overwrite

Confirm

16. Yes when prompted to IISReset
17. Perform IISReset

 

Second (easy)…

Export keys from source server:

aspnet_regiis -px "iisConfigurationKey" "C:\IISKEY\iisConfigurationKey.xml" -pri
aspnet_regiis -px "iisWasKey" "C:\IISKEY\iisWasKey.xml" -pri

Import on your target/problem server:

aspnet_regiis -pi "iisConfigurationKey" "C:\IISKEY\iisConfigurationKey.xml"
aspnet_regiis -pi "iisWasKey" "C:\IISKEY\iisWasKey.xml"

 

Third one (work intensive/insecure)  

Remove the encryption tags in the applicationHost.config.  Depending on how big your problem is this may be extensive work.

Open your applicationHost.config in %windir%\system32\inetsrv\config and search for “[enc:” and put the configurations that are encrypted in the file in clear text.

Example Before:

<processModel identityType="User" userName="domain\username" password="[enc:IISWASOnlyAesProvider:2Woq1bHFmcDxzSEKLe9q1eZsvlUuBcmb0Luy3DzkZWg=:enc]" />

Example After:

<processModel identityType="User" userName="domain\username" password="UserNamesPasswordInClearText" />

 

The Fourth, (absolutely crazy, but explains what’s really going on on the file system)…

Modify your MahineGuid to match your keys needed to decrypt.

This is pretty aggressive and I would suggest this only as an academic exercise in figuring out the inner workings of the RSA suite and usage within the Windows OS.  But explains what’s really going on.  I’m not comfortable suggesting any use this as a policy since I can find no documentation on Microsoft’s site to support its validity; but, it was something I spent a good amount of time on and it worked, I’d hate to not post the findings.  It also gives you the whole story of the problem.

In C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys are all of your private keys.  The appropriate key is called up by an application needing the machine key portion by a MachineGuid registry key.  When xcopy-ing or cloning your flat files no longer match your MachineGuid nested in the registry.

So, your inability to decrypt is actually rooted in your inability to find the key required to do the decryption!

The naming convention of all of the keys stored in this location is <KEYGUID>_<MACHINEGUID>.  If you flip the MachineGuid in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography to match the suffix of the keys stored in C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys IIS will start referencing the previous certificates and IIS will be able to decrypt the password sections again.

RSAKeys

Extensive testing was not done on this solution so again, proceed with caution.  Only a cool trick in my book, no telling what else this could break along the way.  You will notice however, both of the other solutions in this post create *new* keys in this location with the correct MachineGuid, so a good troubleshooting skill and confirmation technique to store away for future use.

 

, ,

9 thoughts on “HTTP Error 500.19 “Failed to decrypt attribute “password””

  • justinschmitt08@gmail.com' Justin says:

    Thanks! What a simple solution to a confusing problem! You expect a cloned server to just work, but apparently that is not always the case.

  • jean.snyman@gmail.com' Jean says:

    I resolved this issue by exporting IIS as a package on the source server and importing it on the destination server.

  • aeciolemos@hotmail.com' Aecio Lemos says:

    Hi,

    My situation is a bit different. My server crashed and I restored a System State backup over a new machine/installation of Windows 8 R2.

    I dont have access to a functional server. The previous server crashed. I do have the harddrives, but I dont know how I would export from them. So I tried the third approach in your article.

    I tried removing the [enc from the applicationHost.config file. Although I successfully did that using regular expression to make it easier, and the file now not having ANY [enc information, the TEMP folder files are still generated with the [enc entries. I have no idea where it gets them from.

    One detail I did notice is that the machine key GUID is the same as what is in the registry and the keys in the MachineKeys folder.

    I am at a loss and trying anything here. I need to keep removing the [enc entries manually every time a site is started so that it works for the users.

    Any tips are appreciated.

    Thanks

  • jcastiblanco@synergytpc.com' Julián Castiblanco says:

    thank you man, you really saved my live

  • phqn@hotmail.com' Duck says:

    Thanks!! this article was importante for me… i was very confuse

  • jfw@laposte.net' jf says:

    Many thanks for this solution !

  • eddieg32@gmail.com' Ed says:

    THANK YOU!!! I just so happen to come across this article and it truly saved my butt with a major production disaster recovery project. Fantastic work on the overall explanation!! Greatly appreciate it!!

  • tchamaz@gmail.com' TJ says:

    Thanks, the first option did the trick for me. kudos!!!!!!

Leave a Reply to Jean Cancel reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>