The Daily Parker

Politics, Weather, Photography, and the Dog

As the pipeline builds...

I'm waiting for a build to finish so I can sign off work for the day, so I've queued up a few things to read later:

Looks like the build is done, and all the tests passed. (I love green pipelines.)

Fifth month in a row over 50

This is my 55th post this month, and the fifth month in a row in which I've posted over 50 times. That brings my 12-month total to 581, the third record in a row and the fifth record this year. I guess Covid-19 has been good for something.

Here's what I'm reading today:

I'm excited to add a notch on the Brews and Choos project in a few hours. Check back tomorrow.

Two stark comparisons

First: the difference between how Garmin handled a global outage that lasted 5 days, and how SendGrid managed one that lasted 5 hours. SendGrid handles billions of emails per day, including for Microsoft and other massive companies. So SendGrid going down didn't inconvenience a few athletes and pilots; it crippled Fortune-500 companies' marketing departments. (And it delayed a scheduled release on my own team.)

Within about an hour of their outage, SendGrid created an incident response page to which they posted updates every half-hour. They clearly stated what was going on and how they were trying to fix it, even as they were discovering for themselves what had happened:

Contrast that with Garmin, who still haven't really explained what happened or why it took so long to resolve the outage. (They finally declared their remediation "complete" yesterday, almost 6 days after the outage started.)

Second: the difference between how Germany (and other rich countries) have handled Covid-19 and how we have. Josh Marshall looked into the numbers:

The head of Germany’s equivalent of the CDC told reporters today that he’s “very concerned” about the rising case numbers in the country and accuses Germans of becoming “negligent” in their adherence to mitigation measures. He has good reason to be concerned.

Today Germany reported 638 new cases of COVID. That comes out to .76 cases per 100,000 residents. Let’s round that up to 1 new case per 100,000 for good measure. (The need for round numbers will become clear.)

Today New York State had 3 cases per 100,000. So Germany is concerned by 1/3 the number of cases as we have in New York state, a state which is probably controlling the disease as well as any other state in the country.

Florida today had 43 cases per 100,000. Florida’s outbreak is more than forty-three times the size relative to population.

To state the point baldly, Germany is very concerned about a rise in cases that would still be dramatically better than any other part of the United States. They’re ramping up border restrictions to get things back under control and chiding the population to redouble its collective efforts.

[S]eeing this as, "well, look, everyone’s having problems." Or "we’re not the only ones having new outbreaks" or "we can’t go back to shutdowns…" ... only captures how Americans are having a hard time grappling with just how many universes away we are from what is happening in other countries which are comparably affluent, industrialized and able to mount an effective response.

We’re failing that badly.

Anthony Fauci was on BBC just now, struggling not to call the president out on his criminal negligence. Only 98 days until we can vote the bastard out.

Another outage

Even as Garmin picks up the pieces from what they now admit was a massive ransomware attack, bulk email provider SendGrid has gone down spectacularly.

I use SendGrid, as does my company, for status emails and such.

Here's my problem, though: I have a code update to put out that specifically targets a bug in SendGrid's .NET library that they claim to have fixed. My automated build pipelines won't release new code unless all the unit tests pass. Right now, the SendGrid tests fail sporadically, and at least one fails every time the tests run.

Ah, technology.

How to pass secrets into your custom log target

Today I finally solved a problem that has nagged me for months: Given a .NET Core 3.1 Web API, NLog, and a custom log target for NLog, how could I pass secrets into the log target using Azure Key Vault?

The last bit required me to add Azure Key Vault support to the InnerDrive.Logging NuGet package, which I did back in February. The KeyVaultSecretProvider class allows you to retrieve secrets from a specified Azure Key Vault. (I'm in the process of updating it to handle keys as well.) Before that, you'd have to put your secrets in configuration files, which anyone on your development team or who has access to your Git repository can see. For example, to use our SendGridTarget class, you would have to put this in your nLog.config file:

<extensions>
	<add assembly="NLog.Web.AspNetCore"/>
	<add assembly="InnerDrive.Logging"/>
</extensions>

<targets async="false">
	<target xsi:type="SendGridTarget"
	        name="sendgrid"
		apiKey="{supposedly secret key}"
		applicationName="My Cool App"
	        from="service@contoso.org"
	        to="admin@contoso.org"
	/>
</targets>

That is...suboptimal. Instead, you want to use a class that implements ISecretProvider and inject it into the SendGridTarget class through this constructor:

/// <summary>
/// Creates a new instance of <see cref="SendGridTarget"/> with
/// a specified <see cref="ISendGridSender"/> implementation.
/// </summary>
/// <param name="sender">The <see cref="ISendGridSender"/> to use</param>
/// <param name="secretProvider">The <see cref="IConfiguration"/> to use</param>
/// <param name="apiKey">The API key to use</param>
public SendGridTarget(
	ISendGridSender sender, 
	ISecretProvider secretProvider = null, 
	string apiKey = null)
{
	IncludeEventProperties = true;
	if (null != secretProvider) Configuration = secretProvider;
	if (!string.IsNullOrWhiteSpace(apiKey)) ApiKey = apiKey;
	_sender = sender;
}

And now I'm going to save you about 30 hours of research and frustration. To get NLog to inject the secret provider into the target, I added this method to Startup.cs:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env, IComponentContext container)
{
	// Setup code elided
	ConfigureLogging(container);
}

public static void ConfigureLogging(IComponentContext container)
{
	var defaultConstructor = ConfigurationItemFactory.Default.CreateInstance;
	var secretProvider = container.Resolve<ISecretProvider>();
	ConfigurationItemFactory.Default.CreateInstance = type =>
	{
		if (type == typeof(SendGridTarget))
		{
			var sendGridSender = container.Resolve<ISendGridSender>();
			return new SendGridTarget(sendGridSender, secretProvider);
		}

		return defaultConstructor(type);
	};
	LogManager.Configuration = LogManager.Configuration.Reload();
	NLogBuilder.ConfigureNLog(LogManager.Configuration);
}

(Note that the ISecretProvider and ISendGridSender classes need to be registered with your dependency-injection framework.)

Today was a good day.

Lunchtime reading

It has cooled off slightly from yesterday's scorching 36°C, but the dewpoint hasn't dropped much. So the sauna yesterday has become the sticky summer day today. Fortunately, we invented air conditioning a century or so ago, so I'm not actually melting in my cube.

As I munch on some chicken teriyaki from the take-out place around the corner, I'm also digesting these articles:

Can you believe we're only 99 days from the election? How time flies.

Garmin offline

Four days after I switched from Fitbit to Garmin, all of Garmin's online services have gone offline:

The problems with those services also mean that a range of features can't be used on Garmin's own devices: it is not possible to create new routes to go running or cycling, for instance, or to share those activities on services like Strava once they are completed.

The devices themselves continue to work as normal with the data they do have, however, meaning that any data collected during the outage will be safe.

Garmin wrote on its official Twitter pages that the problems were also affecting its call centres, leaving users unable to get in touch through calls or online messages.

"We are currently experiencing an outage that affects Garmin Connect, and as a result, the Garmin Connect website and mobile app are down at this time," it wrote.

"This outage also affects our call centres, and we are currently unable to receive any calls, emails or online chats. We are working to resolve this issue as quickly as possible and apologise for this inconvenience."

This is not what Garmin customers want to see:

The error message on the Garmin Connect website suggests the problem is with their Cloudflare equipment:

Update: Based on Garmin employee's social-media posts, ZDNet now reports that the company experienced a catastrophic ransomware attack, most likely a new strain of WastedLocker. Fortunately, my Venu can hold 200 hours of data. So as long as they get it back up within a week or so, I shouldn't lose anything—unless the ransomware attack already destroyed my data from this past week.

Mid-morning news round-up

I'll get to the final head-to-head comparison between my Garmin Venu and Fitbit Ionic later today. Meanwhile:

And finally, because our Covid-19 numbers have started creeping up, indoor bar service will halt on Friday.

A bit of news overload today

Happy tax day! And now, we're off to the races:

Finally, Bloomberg takes a backward glance at the rise and fall of the Segway.

Today's lunchtime reading

As I take a minute from banging away on C# code to savor my BBQ pork on rice from the local Chinese takeout, I have these to read:

And today's fortune cookie says: "Hope for the best, but prepare for the worst in bed."