The Daily Parker

Stuff sent to Instapaper:

• Chicago's John Hancock Center has sold its antennae for $70m. This makes more sense than it might seem.
• No, Virginia, cars are still not greener than buses.
• Bill McBride, aka Calculated Risk, spent time letting Joe Weisentahl interview him. McBride predicted the housing bubble in 2004.
• Wonder what it's like to have an autism-spectrum disorder? Rod Dreher has an example.
• Engineering explains how dogs smell better than we do.
• Programmer jokes just aren't that funny, usually.
• Researchers have cracked a code used by an 18th-century secret society.

Time to dash out for lunch...then more coding. Gotta finish today.

Over the last two days I've spent almost every working minute redesigning the 10th Magnitude framework and reference application. Not new code, really, just upgrading them to the latest Azure bits and putting them into a NuGet package.

That hasn't left much time for blogging. Or for Words With Friends. And I'm using a lot of Instapaper. Without Instapaper, I'd never get to read Wired editor Mat Honan drawing lessons from his epic hack last summer.

Chicago is enjoying its 14th consecutive month of above-average sunshine, along with some unseasonable warmth leading into Thanksgiving (13°C right now, 17°C on Thursday).

Earlier today the NWS Climate Prediction Center released a new 90-day forecast predicting normal temperatures and precipitation in Chicago through February:

This is all fine by us. Though we do hope for a mild winter...

I've spent a good bit of free time lately working on migrating Weather Now to Azure. Part of this includes rewriting its Gazetteer, or catalog of places that it uses to find weather stations for users. For this version I'm using Entity Framework 5.0, which in turn allows me to use LINQ extensively.

I always try to avoid duplicating code, and I always try to write sufficient unit tests to prevent (and fix) any coding errors I make. (I also use ReSharper and Visual Studio Code Analysis to keep me honest.)

There are two methods in the Gazetteer's PlaceFinder class that search for places by distance. The prototypes are:

public static IEnumerable FindNearby(ILocatable center, Length radius)

and:

public static IEnumerable FindNearby(ILocatable center, Length radius, Expression<Func<Place, bool>> predicate)

But in order for the first method to work, it has to create a predicate of its own to draw a box around the center location. (The ILocatable interface requires Latitude and Longitude. Length is a class in the Inner Drive Extensible Architecture representing a measurable two-dimensional distance.) So in order for the second method to work, it has to chain predicates.

Fortunately, I found Joe and Ben Albahari's library of LINQ extensions. Here's the second method:

public static IEnumerable<PlaceDistance> FindNearby(
	ILocatable center,
	Length radius,
	Expression<Func<Place, bool>> predicate)
{
	var searchPredicate = 
		SearchDistancePredicate(center, radius)
		.And(predicate);

	var places = Find(searchPredicate);

	return SearchDistanceResults(places, center, radius);
}

This allows me to use a single Find method that takes a predicate, engages a retry policy, and returns exactly what I'm looking for. And it allows me to do this, which just blows my mind:

var results = PlaceFinder.FindNearby(TestNode, TestRadius, p => p.Feature.Name == "airport");

Compared with the way Weather Now works under the hood right now, and how much coding the existing code took to achieve the same results, I'm just stunned. And it will make migrating Weather Now a lot easier.

Oh, Azure Storage team, why did you break everything?

I love upgrades. I really do. So when Microsoft released the new version of the Windows Azure SDK (October 2012, v1.8) along with a full upgrade of the Storage Client (to 2.0), I found a little side project to upgrade, and went straight to the NuGet Package Manager for my prize.

I should say that part of my interest came from wanting to use some of the .NET 4.5 features, including the asynchronous helper methods, HTML 5, and native support for SQL 2012 spatial types, in the new version of Weather Now that I hope to complete before year's end. The Azure SDK 1.8 supports .NET 4.5; previous version didn’t. And the Azure SDK 1.8 includes a new version of the Azure Emulator which supports 4.5 as well.

To support the new, Azure-based version (and to support a bunch of other projects that I migrated to Azure), I have a class library of façades supporting Azure. Fortunately, this architecture encapsulated all of my Azure Storage calls. Unfortunately, the upgrade broke every other line of code in the library.

0. Many have the namespaces have changed. But of course, you use ReSharper, which makes the problem go away.

1.The CloudStorageAccount.FromConfigurationSetting() method is gone. Instead, you have to use CloudStorageAccount.Parse(). Here is a the delta from TortoiseHg:

- _cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting(storageSettingName);
+ var setting = CloudConfigurationManager.GetSetting(storageSettingName);
+ _cloudStorageAccount = CloudStorageAccount.Parse(setting);

2. BlobContainer.GetBlobReference() is gone, too. Instead of getting a generic IBlobContainer reference back, you have to specify whether you want a page blob or a block blob. In this app, I only use page blobs, so the delta looks like this:

- var blob = _blobContainer.GetBlobReference(blobName);
+ var blob = _blobContainer.GetBlockBlobReference(blobName);

Note that BlobContainer also has a GetPageBlobReference() method. It also has a nearly-useless GetBlobReferenceFromServer method that throws a 404 error if the blob doesn’t exist, which makes it useless for creating new blobs.

3. Blob.DeleteIfExists() works somewhat differently, too:

- var blob = _blobContainer.GetBlobReference(blobName);
- blob.DeleteIfExists(new BlobRequestOptions 
-	{ DeleteSnapshotsOption = DeleteSnapshotsOption.IncludeSnapshots });
+ var blob = _blobContainer.GetBlockBlobReference(blobName);
+ blob.DeleteIfExists();

4. Remember downloading text directly from a blob using Blob.DownloadText()? Yeah, that’s gone too. Blobs are all about streams now:

- var blob = _blobContainer.GetBlobReference(blobName);
- return blob.DownloadText();
+ using (var stream = new MemoryStream())
+ {
+ 	var blob = _blobContainer.GetBlockBlobReference(blobName);
+ 	blob.DownloadToStream(stream);
+ 	using (var reader = new StreamReader(stream, true))
+ 	{
+ 		stream.Position = 0;
+ 		return reader.ReadToEnd();
+ 	}
+ }

5. Because blobs are all stream-based now, you can’t simply upload files to them. Here’s the correction to the disappearance of Blob.UploadFile():

- var blob = _blobContainer.GetBlobReference(blobName);
- blob.UploadByteArray(value);
+ var blob = _blobContainer.GetBlockBlobReference(blobName);
+ using (var stream = new MemoryStream(value))
+ {
+ 	blob.UploadFromStream(stream);
+ }

6. Microsoft even helpfully corrected a spelling error which, yes, broke my code:

- _blobContainer.CreateIfNotExist();
+ _blobContainer.CreateIfNotExists();

Yes, if not existS. Notice the big red S, which is something I’d like to give the Azure team after this upgrade.*

7. We’re not done, yet. They fixed a "problem" with tables, too:

  var cloudTableClient = _cloudStorageAccount.CreateCloudTableClient();
- cloudTableClient.CreateTableIfNotExist(TableName);
- var context = cloudTableClient.GetDataServiceContext();
+ var table = cloudTableClient.GetTableReference(TableName);
+ table.CreateIfNotExists();
+ var context = cloudTableClient.GetTableServiceContext();

8. Finally, if you have used the CloudStorageAccount.SetConfigurationSettingPublisher() method, that’s gone too, but you don’t need it. Instead, use the CloudConfigurationManager.GetSetting() method directly. Instead of doing this:

if (RoleEnvironment.IsAvailable)
{
	CloudStorageAccount.SetConfigurationSettingPublisher(
		(configName, configSetter) => 
		configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)));
}
else
{
	CloudStorageAccount.SetConfigurationSettingPublisher(
		(configName, configSetter) => 
		configSetter(ConfigurationManager.AppSettings[configName]));
}

You can simply do this:

var someSetting = CloudConfigurationManager.GetSetting(settingKey);

The CloudConfiguration.GetSetting() method first tries to get the setting from Azure, then from the ConfigurationManager (i.e., local settings).

I hope I have just saved you three hours of silently cursing Microsoft’s Azure Storage team.

* Apologies to Bill Cosby.

Texas-based Hostess Brands has shut down in preparation for liquidation:

Hostess said a strike by members of the Bakery, Confectionery, Tobacco Workers and Grain Millers International Union that began last week had crippled its ability to produce and deliver products at several facilities, and it had no choice but to give up its effort to emerge intact from bankruptcy court.

The Irving, Texas-based company said the liquidation would mean that most of its 18,500 employees would lose their jobs.

In the Chicago area, Hostess employs about 300 workers making CupCakes, HoHos and Honey Buns in Schiller Park. Hostess also has a bakery in Hodgkins, where 325 workers make Beefsteak, Butternut, Home Pride, Nature’s Pride and Wonder breads.

Union President Frank Hurt said the company's failure was not the fault of the union but the "result of nearly a decade of financial and operational mismanagement" and that management was trying to make union workers the scapegoats for a plan by Wall Street investors to sell Hostess.

Twinkies, legend has it, were invented right here in Chicago in 1930.

I will now go see if the 7-11 across the street still has any left...

Starting before 8am with an international conference call usually means I'll have a full day. Fortunately there's Instapaper, which lets me shove all the interesting things I find during the day onto my Android pad for tomorrow's bus ride to work.

So far today:

• The Guardian's Glenn Greenwald worries more about surveillance than any other part of the Petreaus scandal.
• Wunderground's Jeff Masters explains all the superlatives around Hurricane Sandy.
• Philip Bump, writing at Gristmill, thinks politicians should pay more attention to climate change.
• Nate Silver, having put the election behind him, turns his attention to the American League MVP race.
• The Chicago Board of Elections apparently exposed my personal information along with that of the other 1.7 million voters in the city. (Since voter registration information is public in Illinois, this is actually not a real story. But oooooh! Information breach! Scary hackers!)
• Chicago restaurant L20, which shares an alley with my apartment building, got its second Michelin star back. I ate there once, last May, and was very impressed. Not as impressed as the check would have suggested, but impressed nonetheless.

And...now back to work.

James Fallows, reacting to the Patreaus debacle, reminds everyone of the obvious:

Here is the secret plan:

Never put anything in an email message, to anyone, that would cause you serious problems if it fell into the wrong hands.

That's the plan™. All of it. Never do this. Ever.

Yep. This is the advice security experts have given for, well, ever.

Because you might learn things neatly summed up in the headline "Petraeus Apparently Most Mentally Balanced Individual in His Own Scandal" "(From the Times …)":

Ms. Kelley, a volunteer with wounded veterans and military families, brought her complaint to a rank-and-file agent she knew from a previous encounter with the F.B.I. office, the official also said. That agent, who had previously pursued a friendship with Ms. Kelley and had earlier sent her shirtless photographs of himself, was “just a conduit” for the complaint, he said. He had no training in cybercrime, was not part of the cyber squad handling the case and was never assigned to the investigation.

But the agent, who was not identified, continued to “nose around” about the case...

Later, the agent became convinced — incorrectly, the official said — that the case had stalled. Because of his “worldview,” as the official put it, he suspected a politically motivated cover-up to protect President Obama. The agent alerted Eric Cantor, the House majority leader, who called the F.B.I. director, Robert S. Mueller III, on Oct. 31 to tell him of the agent’s concerns.

Says Josh Marshall:

So basically this entire scandal both at the outset and in the denouement was driven by Freakshow FBI Agent X who both wanted to bed the victim of the alleged harassment and also decided that the FBI was covering up it’s investigation of the Tampa socialite to protect President Obama. And this because of his “worldview”. Please let us meet this awesome example of American law enforcement.

Now, I understand why David Patraeus lost his security clearance and, thus, his job, because of this affair. I also agree with commentators (including one of my co-workers) who said, "he resigned because of an affair? Really?" I don't think there's anything simple about the DCI conducting an affair using a pseudo-anonymous account on GMail that someone could discover without resorting to Bond-esque espionage.

But I think our nation's top law-enforcement agency, the FBI, deserve props for deciding that an investigation of a top political official that turned up nothing criminal should be announced only after one of the most divisive elections of the last 50 years. Law enforcement should not ever be political. We're a nation of laws, not men, as many have observed. Patraeus acted stupidly, but not criminally, and he's given up his career for it. Let him go quietly into the night.

As for the petty, partisan little man that broke the law to use Patraeus's resignation for political gain, he deserves the undying scrutiny of the U.S. Attorney for whatever district he lives in.

And full disclosure: I believe strongly that the DCI needs to give up certain things upon taking office, including GMail, Facebook, and bits on the side. I have no problem with human failings; but I object in the strongest terms to people in certain offices—Director of Central Intelligence and President of the United States included—engaging in any kind of personal deceit. Professional deceit? No problem with DCI or POTUS; that's part of the job. But Bill Clinton disappointed me deeply, and now, so did David Patraeus. And that's even before we talk about illegal drone strikes.

Talking Points Memo has some smart readers. One of them crystallized the debate about whether the Romney campaign's shock after losing was genuine, or a ploy to inoculate themselves against the wrath of their donors:

There’s an old rule of political research: never ascribe to conspiracy what can be more easily explained by stupidity. Was the Romney campaign brilliant masterminds of a coordinated PR strategy to make themselves look stupid? Or we’re they just stupid?

Occam’s razor says the latter.

For background, read Josh Marshall's angst about how the Romney campaign could have missed the facts so completely, even though the campaign ran on a platform of denying evidence that disagreed with them.