The Daily Parker

Politics, Weather, Photography, and the Dog

Really feeling like spring today

The temperature at Inner Drive Technology WHQ just hit 17.5°C, which it hasn't hit since 5:54pm on November 5th. That's almost 125 days, quite a while to go without wearing a jacket outside.

Unfortunately, spring weather isn't the only thing in the news today:

Finally, Metra is seeking public input on a plan to rename the heavy-rail lines around Chicago. Right now, each line has an historic name and a different color. The favored proposal would be to give each line a letter signifying the direction from downtown, plus a number. For example, the Union Pacific North line that goes by my house would be renamed N1. And all the lines departing from a single downtown station would get the same color (green in the case of the three UP lines). I think this is a good proposal, and would bring Chicago in line with international cities like Berlin and Paris.

Got Brews & Choos down to a science

Yesterday was the 5th anniversary of the Brews & Choos Project's high-water mark before the pandemic. On 7 March 2020, I went farther than I'd ever gone before in search of breweries to add to the list, visiting Penrose and Stockholm's in Geneva, then More and Lunar in Villa Park on the way back. A few days later the world stopped for a while. It would be almost three months before I visited another brewery.

Yesterday, I took a half-day of PTO, braved some crappy early-spring weather, and met up with my Brews & Choos buddy at a relatively new place in the Pilsen neighborhood of Chicago. We managed to visit five South Side breweries, and—here's the science part—consumed no more than 3 pints of beer over 5 hours. It was a marathon, not a sprint, after all.

In any event, I've got a lot of photos to go through and a lot of reviews to write, so look for them to come out over the next few days.

And hey, if you want to see more Brews & Choos reviews, contribute to The Daily Parker! Your $5 contribution keeps the site running for a day—or buys a tasing-size beer.

Another reason to contribute: I've started re-developing The Daily Parker's code from scratch. I changed direction slightly on an existing project to make it a blog on steroids, and I think it'll be super-cool when complete. So how about throwing in another $5 a month to support that, too?

Another day, another OAFPOTUS grift

I want to start with a speech on the floor of the French Senate three days ago, in which Claude Malhuret (LIRT-Allier) had this to say about the OAFPOTUS:

Washington has become the court of Nero, an incendiary emperor, submissive courtiers, and a jester high on ketamine in charge of purging the civil service.

This is a tragedy for the free world, but it is first and foremost a tragedy for the United States. Trump’s message is that there is no point in being his ally since he will not defend you, he will impose higher tariffs on you than on his enemies and will threaten to seize your territories while supporting the dictatorships that invade you.

I have faith in the strength of American democracy, and the country is already protesting. But in one month, Trump has done more harm to America than in four years of his last presidency. We were at war with a dictator, now we are fighting a dictator backed by a traitor.

Malheureusement, il a bien raison. And his speech is worth reading (or hearing, si vous parlez français bien).

But that isn't all that happened in the last day or so. No, every day brings new revelations of stupidity and corruption in the new administration:

And now I will take a half-day of PTO and explore four new breweries in Bridgeport and Pilsen. If only the weather had cooperated.

Support The Daily Parker on Patreon

As threatened promised, I'm starting to beg for money to help support The Daily Parker and Weather Now. You can go to Patreon and sign up to help us, with special member benefits as you contribute more.

The Daily Parker costs about $5 a day to run (though I hope to reduce that significantly this fall), and Weather Now costs another $10. They're not entirely labors of love, as I have used Weather Now as a demo project to land new work. But after more than five years with the same full-time employer, those days might be behind me—even though the weather never stops.

So, hey, buy me a coffee. I'll put your name in lights!

More Weather Now improvements

Weather Now v5.0.9194 just hit the hardware, with a new feature that allows you to browse the Gazetteer by finding all the places near a point. (Registration required.) I also added a couple of admin features that I will propagate to every other app I have in production, and made a few minor bug fixes.

Only one minor hiccup: I forgot to add a spatial index to the Gazetteer, which caused searches around a point to take minutes instead of seconds in production. I added the index to the database definition, and after about an hour it had indexed all 15 million locations in the database. So the Nearby Places feature should work perfectly now.

This is one of those things you don't notice in a dev-test environment. The dev-test database only has about 200,000 records in it, so even without the index it only took a moment to find all the places around a point. Nothing like testing in production to find a huge performance miss!

Reading while the world compiles

One of my work projects has a monthly release these days, so right now I'm watching a DevOps pipeline run through about 400 time-consuming integration tests before I release this month's update. That gives me some time to catch up on all this:

The New York Times has a long explanation of how the Clown Prince of X took over the federal bureaucracy.

All right, the build has finished, so I can now deploy. And for no reason other than I like it, here is a photo of Cassie watching TV with me last night:

Why The Daily Parker costs so much

A longtime Daily Parker reader asked this about yesterday's post:

"The Daily Parker costs $4.87 per day" -- I'm really hoping that's a misprint, because that's almost $150 a month, which is ten times what I pay for my web hosting package which comes with unlimited domains, a full email service (IMAP+SMTP over TLS), click-to-install WordPress and MySQL database creation, SSH access to the back-end Linux machine, and excellent customer support.

Also -- and I *really* hate to say this to a fellow IT professional -- your web site often seems rather slow. So much so that I'd built a mental image of it running on an old PC in a corner of your apartment, and I'd put the slow response times down to the latency of a hard disk spinning up from idle.

So, he's not wrong: The Daily Parker right now is slow and buggy. And expensive*. (Ironically, when it was literally running on a PC in the corner of my apartment prior to 2013, it ran like Jesse Owens.)

Sherman, set the Wayback Machine to October 2015, when I deployed the current version of this blog. From the blog's separation from braverman.org in 2005 until 2015, it ran on DasBlog, a .NET 1.1 blog engine that worked most of the time and had a few features I liked. I dragged it kicking and screaming up to .NET 2.0 and later .NET 4.0, and there it stayed.

After 10 years and dozens of tweaks, I decided to modernize by moving to BlogEngine.NET, which I also forked and modified. This engine runs on .NET 4.8, which I had to shoehorn into an Azure App Service when Cloud Services went away a couple of years ago. BlogEngine.NET had modest performance problems when it had a nice virtual machine all to itself, as Cloud Services weren't too different from on-premises hardware. But Azure App Services don't quite work the same way, such that many of the performance optimizations in the BlogEngine.NET code actually cause performance headaches in App Services. For example, at app start, the engine loads the entire blog history into memory, because in 2007, when the project began, memory was fast and disks were slow. (NB: The Daily Parker has over 9,700 posts spanning 27 years.) Also, the code runs entirely synchronously, so under load it spins up more and more threads until it just collapses from exhaustion.

So here we are: running a very old blog engine on a nearing-end-of-life version of .NET that everyone is tired of.

But, aha! There is a solution, which I've been kicking around for almost as long as I've had a blog, and which I finally have the skills and time to work on. I'll simply build my own. It'll be idiosyncratic, sure, but it'll be fast and it'll be cool.

Or maybe I'll go back to DasBlog, now that someone has rebuilt it in .NET Core.

Nah. I'm going to write my own. Target date: October 15th, ten years after I released this version.

* It's actually now around $3.34 per day after a Microsoft Azure pricing change on February 12th which just showed up in the cost management tool today. The costs break down as follows: App Service type B2, $2.49; storage (media and event log), 52¢; database (serverless type B), 33¢. So, around $100 per month.

Fun Gazetteer facts

I meant to add this earlier today, but I had to do some work for my real job.

Uploading 15.4 million place records into Weather Now revealed some unexpected statistics. As you might expect from a military website, the National Geospatial-Intelligence Agency got a lot of its data from military sources. And the military tends to map things they care about in great detail. So the top 10 countries by place counts turn out to be:

  1. China, 2.1 million
  2. United States, 1.1 million
  3. Russia, 857,000
  4. Iran, 686,000
  5. Indonesia, 543,000
  6. Thailand, 530,000
  7. Finland, 495,000
  8. Mexico, 490,000
  9. Republic of Korea, 346,000
  10. Afghanistan, 343,000

The top 5 states by USGS place count might surprise you, too:

  1. California, 52,700
  2. Texas, 43,300
  3. Oregon, 36,800
  4. Tennessee, 33,400
  5. Arkansas, 29,800

Florida, New York, Pennsylvania, and Illinois, the 3rd through 6th most populous states, come in at 19th, 11th, 8th, and 33rd, respectively. I have no idea why.

And the smallest places? All of these NGIA files had fewer than 8 locations—probably because they contain places that are really part of some other country:

  1. Dramana and Shakhatoe, 1
  2. Kalapani, 1
  3. Minerva Reefs, 1
  4. Conejo Island, 2
  5. Geyser Reef, 2
  6. Hans Island, 2
  7. Siachen, 2

I honestly don't know why NGIA puts those places in their own files. I'll correct them eventually.

If you want to have a bit of fun, try searching for places you know and see what weather is closest.

Almost everywhere in the world

Late Tuesday night, Weather Now finished importing and indexing 15,430,045 places from around the world, ending with Mutirikwi Dam, Masvingo, Zimbabwe at 9:29 pm CST. (I need to re-import about 11,000 records for places that don't belong to any particular country, but that's low-priority.)

When I first built the Weather Now Gazetteer in July 2002, I only imported populated places, because database space was a lot more expensive then. So from 2002 until the v5 upgrade launched 3 years ago, the Gazetteer had about 7.5 million records and lived in a relational SQL database that migrated to the Cloud in 2013.

This time, I imported every scrap of geographic data I could find in the US, which added 8 million more. And starting later this year, I'll automate updates, particularly from sources that change frequently like the National Geospatial-Intelligence Agency and the Federal Aviation Administration. I may import other databases later on, but for now, I think we're good.

Pricing for Cosmos DB works differently than it used to for SQL Server. Weather Now v4 (2013-2022) used an Azure SQL DB size that cost $10 per month. That gave me up to 10 GB of relational storage for everything, which is why all the weather data moved to really inexpensive flat storage (Azure Tables) that cost about 4.5c per GB per month. When Weather Now lived on-premises through 2013, the Gazetteer and weather data took up about 50 GB (25 GB for the Gazetteer and 25 GB for the first 11 years of weather data), but it cost nothing except electricity—$100 a month of electricity, in fact.

With Cosmos DB, I pay for database transactions, called Request Units, plus a small fee of 25c per GB per month for storage. It gets a little complicated, but basically, the biggest expense for the database is the import, which cost about $75. Going forward, the biggest database expense will be the search service, which costs $2.42 per day. Storing 20 years of weather data costs $1.65 per day. Of course, the application service hosting Weather Now runs $3.60 a day, so it does all add up.

Because the daily cost summary takes a full UTC day to update, I don't have the new run rate for the application yet. When that comes out tonight, I may have to look into ways to defray the cost of the app, whether by voluntary donations or *gulp* advertising.

Incidentally, The Daily Parker costs $4.87 per day ($3.60 for the app service, 79c for storage, and 48c for the database. Now that Weather Now is pretty much where I want it, my next project will be to write a new blog engine and port this app to something that will cost about half that. So be on the lookout for a tip jar on this app as well.

Still chugging along

The Weather Now gazetteer import has gotten to the Ps (Pakistan) with 11,445,567 places imported and 10,890,186 indexed. (The indexer runs every three hours.) I'll have a bunch of statistics about the database when the import finishes, probably later tonight or tomorrow morning at the latest. I'm especially pleased with the import software I wrote, and with Azure Cosmos DB. They're churning through batches of about 30 files at a time and importing places at around 10,000 per minute.

Meanwhile, in the rest of the world:

Finally, in February 1852, a man calling himself David Kennison died in Chicago. He had clamed to be 115 years old, participated in the Boston Tea Party, and hobnobbed with the great and good in the early days of the Republic. And in the proud tradition of people giving undue acclaim to total charlatans, the entire city turned out for his funeral—173 years ago yesterday.