The Daily Parker

Politics, Weather, Photography, and the Dog

Debugging our first Azure 1.8 deployment

I've just spent three hours debugging something caused by a single missing line in a configuration file.

At 10th Magnitude, we've recently upgraded our framework and reference applications to the latest Windows Azure SDK. Since I'd already done it once, it didn't take too desperately long to create the new versions of our stuff.

However, the fact that something works in an emulator does not mean it will actually work in production. So, last night, our CTO attempted to deploy the first application we built with the new stuff out to Azure. It failed.

First, all we got was a HttpException, which is what ASP.NET MVC throws when something fails on a Razor view. The offending line was this:

@{ 
   ViewBag.Title = Html.Resource("PageTitle");
}

This extension method indirectly calls our custom resource provider, cleverly obfuscated as SqlResourceProvider, which then looks up the string resource in a SQL database.

My first problem was to get to the actual exception being thrown. That required me to RDP into the running Web role, open a view (I chose About.cshtml because it was essentially empty), and replace the code above with this:

@using System.Globalization
@{
  try
  {
    var provider = new SqlResourceProvider("/Views/Home/About.cshtml");
    var title = provider.GetObject("PageTitle", CultureInfo.CurrentUICulture);
    ViewBag.Title = title;
  }
  catch (Exception ex)
  {
    ViewBag.Error = ex + Environment.NewLine + "Base:" + Environment.NewLine + ex.GetBaseException();
  }
}
<pre>@ViewBag.Error</pre>

That got me the real error stack, whose relevant lines were right at the top:

System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.WindowsAzure.ServiceRuntime, Version=1.7.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.WindowsAzure.ServiceRuntime, Version=1.7.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'
at XM.UI.ResourceProviders.ResourceCache.LogDebug(String message)

Flash forward an hour of reading and testing things. I'll spare you. The solution is to add a second binding redirect in web.config:

<dependentAssembly>
  <assemblyIdentity name="Microsoft.WindowsAzure.ServiceRuntime" 
    publicKeyToken="31bf3856ad364e35" culture="neutral" />
  <bindingRedirect oldVersion="0.0.0.0-1.0.0.0" newVersion="1.0.0.0" />
  <bindingRedirect oldVersion="1.1.0.0-1.8.0.0" newVersion="1.8.0.0" />
</dependentAssembly>

Notice the second line? That tells .NET to refer all requests for the service runtime to the 1.8 version.

Also, in the Web application, you have to set the assembly references for Microsoft.WindowsAzure.Configuration and Microsoft.WindowsAzure.Storage to avoid using specific versions. In Solution Explorer, under the References folder for the web app, find the assemblies in question, view Properties, and set Specific Version to false.

I hope I have saved you three hours of your life. I will now go back to my deployment, already in progress...

Update, an hour and a half later: It turns out, there's a difference in behavior between <compilation debug="true"> and <compilation> on Azure Guest OS 3 (Windows Server 2012) that did not exist in previous guest OS versions. When an application is in debug mode on Azure Guest OS 3, it ignores some errors. Specifically, it ignores the FileNotFoundException thrown when Bundle.JavaScript().Add() has the wrong version number for the script it's trying to add. In Release mode, it just barfs up a 500 response. That is maddening—especially when you're trying to debug something else. At least it let our app log the error, eventually.

Comments are closed