asp.net - System.Web.MVC not copied into bin folder since MS14-059. How to protect against creating builds with missing DLLs as a result of Windows Updates?

Question

This morning it was reported that our web app on our QA server was completely broken with the following error reported from Web.config:

Could not load file or assembly 'System.Web.Mvc, Version=5.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified

Remembering seeing a Windows Update that mentioned MVC, I did some digging and found lots of people reporting a recent Windows Update breaking MVC.

After much digging through those questions and our server, it seems that what's bitten us does not match what's in those other questions, but it does appear related. Here's what we think know:

• Our app that is broken uses ASP.NET MVC 5.1
• MVC was installed via NuGet
• Our BuildServer and QA servers do NOT have MVC 5.1 installed (therefore, not GAC'd)

What we believe has broken caused the "bad build" to be created:

• A patch for MVC 5.1 was installed on the BuildServer via Windows Update despite not having MVC 5.1 installed in the GAC
• The patch has put the "updated" version of MVC 5.1 in the GAC
• CopyLocal=true is ignored when a DLL is in the GAC; therefore since the patch, this means that builds of our app from the BuildServer no longer have System.Web.MVC in the output folder
• Since System.Web.MVC is not in the GAC on our QA servers (they have not yet been patched), the application now fails, because System.Web.MVC cannot be found

Assuming the behavior described above is correct, this means that any time MS service a NuGet DLL via Windows Update that we do not have in the GAC, our BuildServer will start producing incomplete builds (missing out those DLLs that have been injected into the GAC).

Upgrading to MVC 5.2 solves this issue (likely because it wasn't patched, and was therefore not injected into the GAC); the DLL is now copied to the output folder. There are no changes in the diff that upgraded to 5.2.2 except for version number changes (there's specifically no <Private> node been added/edited).

We do not wish to start GACing everything, nor creating manual build steps to copy all of our DLLs into the bin folder just in case MS patches them.

So, what can we change today to ensure we don't ever end up with out BuildServer silently producing back bad builds if MS patch other DLLs in the future?

Answer 1

A patch for MVC 5.1 was installed on the BuildServer via Windows Update despite not having MVC 5.1 installed in the GAC

Yes, this behavior is actually by design. See http://blogs.msdn.com/b/dotnet/archive/2014/01/22/net-4-5-1-supports-microsoft-security-updates-for-net-nuget-libraries.aspx.

---

The patch has put the "updated" version of MVC 5.1 in the GAC

Yes, that's correct; it's how the patch gets the updated code to run instead of the old code. See https://technet.microsoft.com/en-us/library/security/ms14-059.

---

CopyLocal=true is ignored when a DLL is in the GAC; therefore since the patch, this means that builds of our app from the BuildServer no longer have System.Web.MVC in the output folder

Not exactly. What actually happens is a project that previously was CopyLocal=true gets switched to CopyLocal=false. CopyLocal can be set in one of two ways: 1) If there's an explicit <Private>True</Private> setting in the .csproj file, or 2) By default, if no such setting exists (GAC'd assemblies do not CopyLocal by default; other assemblies do).

So what appears to have happened in this case is that your project file didn't have this setting in the csproj file. As a result, the GUI showed the setting based on the evaluated default value before the patch (CopyLocal = true) but then after the patch was installed, the GUI will now show the new default value for a GAC'd assembly (CopyLocal = false).

---

Since System.Web.MVC is not in the GAC on our QA servers (they have not yet been patched), the application now fails, because System.Web.MVC cannot be found

That's correct.

---

Assuming the behavior described above is correct, this means that any time MS service a NuGet DLL via Windows Update that we do not have in the GAC, our BuildServer will start producing incomplete builds (missing out those DLLs that have been injected into the GAC).

For any .csproj reference without an explicit <Private>True</Private> setting, that is correct. Also, note the using NuGet to update your MVC reference can remove this setting even if it was previously present. See http://nuget.codeplex.com/workitem/4344.

---

Upgrading to MVC 5.2 solves this issue (likely because it wasn't patched, and was therefore not injected into the GAC); the DLL is now copied to the output folder. There are no changes in the diff that upgraded to 5.2.2 except for version number changes (there's specifically no node been added/edited).

That's correct. Since MVC 5.2 is not GAC'd, even without an explicit <Private>True</Private> setting, the default value of this non-GAC'd assembly will be CopyLocal=true.

---

We do not wish to start GACing everything, nor creating manual build steps to copy all of our DLLs into the bin folder just in case MS patches them. So, what can we change today to ensure we don't ever end up with out BuildServer silently producing back bad builds if MS patch other DLLs in the future?

The best you can do today is:

1. Put explicit <Private>True</Private> settings in your .csproj file for all your NuGet package assembly references.
2. Until NuGet bug #4344 is fixed, any time you use NuGet to update a package reference, go back into your .csproj file and re-add the explicit <Private>True</Private> setting.