The Daily Parker

Politics, Weather, Photography, and the Dog

New documentation of an old feature

The Inner Drive Extensible Architecture™ has had support for the tzinfo database for several years now. Weather Now uses it; so do a few of my clients.

Like the lazy software developer I am, however, I never put up a decent demonstration of the code, which might, you know, make someone want to buy it.

Well, the documentation, she is here. Licensing, you will be shocked to learn, is available for a modest fee.

Brief reminder from Krugman

The U.S. government is an insurance company with an army:

The vast bulk of its spending goes to the big five: Social Security, Medicare, Medicaid, defense, and interest on the debt.

But what about recent deficits? They’re caused mainly by a fall in revenue and a mostly automatic increase in spending on safety-net programs. Oh, and the federal government has been providing aid to state and local governments, largely to limit layoffs of schoolteachers.

And if you want smaller government, either you’re talking about cuts in the big five, or you have no idea what you’re talking about.

I disagree with Krugman here, a bit, because I see a third possibility. You could be advocating smaller government without cuts to the big five because you're trying to mislead people.

Analysis of Shanks' atlases against the tzinfo database

To better understand the facts behind Astrolabe’s stupid trolling quixotic lawsuit against the guys who coordinated the worldwide time-zone database (tzinfo), I bought copies of the Shanks Amercian and International atlases that Astrolabe claims to own. (I went through the secondary market, so I didn’t actually give Astrolabe any money.)

First, an update. According to Thomas Eubanks of the IETF, the Electronic Frontier Foundation has taken over Arthur Olson’s legal defense. Mazel tov. I expect to see a response to the complaint against him in a few weeks that includes a motion to dismiss which, I think, may be granted. (I’m thinking about drafting a response myself, just to exercise my legal muscles properly. Watch this space.)

Now to the main post. The Shanks books, rather than containing maps, contain pages and pages of tabular data showing three things:

  1. Names of first- and second-order administrative districts (e.g., states and counties);
  2. Latitudes and longitudes of cities and other named places within countries or states;
  3. Lists of the dates and times of time-zone changes for those named places.

The atlases dump this data to paper using a monospace typeface at 6-point size in what can be nothing other than 1990s-era printouts to some kind of publishing compositing software. This offers a clue to the “original works” claim that Astrolabe makes about the products.

Shanks certainly put a lot of work into these books, especially considering he first published them in 1978 and 1985. He must have spent hundreds of hours looking up and entering data on the thousands of locations in the tables.

For example, the American Atlas contains rows upon rows of data like this:

Chicago 16      1 41N51'00 87W39'00 5:50:36
Chicago Heights 16
                1 41N30'22 87W38'08 5:50:33
Chicago Lawn 16 1 41N47    87W43    5:50:52

I imagine Shanks looked up this data in reference books, then entered it into a home-grown flat-field database through a Vax terminal or on his Apple ][+. I hope he at least let the computer calculate the last column (the location’s offset from GMT), since it’s derived directly from the location’s longitude (the next-to-last column).

I imagine this because, in the early 1990s, I did something similar to study climate data. (Do you know how long it takes to enter 30 years of daily climate data by hand? No? You’re lucky.)

Back to Astrolabe’s complaint. In Count 4, Astrolabe claims ownership of “certain copyright-protected computer software programs and information contained therein…known as the ‘ACS Atlas,’ consisting of both the ‘ACS International Atlas,’ and the ‘ACS American Atlas,’ in the form of computer software program(s) and/or data bases, and in the form of electronic output and future electronic media from said programs....’ ” I infer from the complaint that the software reproduces the books in computer-searchable form, or perhaps contains the raw data that Shanks himself used to produce the books.

I’ll defer my main argument for a moment to speculate further on what parts of the tzinfo database could have copied the Shanks database.

In the tzinfo database, one of the files (zone.tab) contains latitudes and longitudes of locations in this form:

US	+415100-0873900	America/Chicago	Central Time
US	+375711-0864541	America/Indiana/Tell_City	Central Time - Indiana - Perry County
US	+411745-0863730	America/Indiana/Knox	Central Time - Indiana - Starke County

Nothing else in the tzinfo database comes as close to looking like data in the Shanks atlases. I don’t know where the tzinfo list came from, but I suspect it came from public sources like the Census Bureau.

The other possible copying comes from the lists of dates Shanks put together that look like this:

IL # 1
Before 11/18/1883  LMT
11/18/1883  12:00  CST
 3/31/1918  02:00  CWT
10/27/1918  02:00  CST

Here’s how the tzinfo database shows the same information:

# Rule	NAME	FROM	TO	TYPE	IN	ON	AT	SAVE	LETTER
Rule	Chicago	1920	only	-	Jun	13	2:00	1:00	D
Rule	Chicago	1920	1921	-	Oct	lastSun	2:00	0	S
Rule	Chicago	1921	only	-	Mar	lastSun	2:00	1:00	D
Rule	Chicago	1922	1966	-	Apr	lastSun	2:00	1:00	D
Rule	Chicago	1922	1954	-	Sep	lastSun	2:00	0	S
Rule	Chicago	1955	1966	-	Oct	lastSun	2:00	0	S
# Zone	NAME		GMTOFF	RULES	FORMAT	[UNTIL]
Zone America/Chicago	-5:50:36 -	LMT	1883 Nov 18 12:09:24
			-6:00	US	C%sT	1920
			-6:00	Chicago	C%sT	1936 Mar  1 2:00
			-5:00	-	EST	1936 Nov 15 2:00
			-6:00	Chicago	C%sT	1942
			-6:00	US	C%sT	1946
			-6:00	Chicago	C%sT	1967
			-6:00	US	C%sT

That hot mess establishes the specific rules Chicago used to change its clocks in the 1920s and 1950s where the rules differed from the general U.S. rules, then it sets out the dates and times that Chicago’s wall-clock rule sets changed from the beginning of standard time in 1883 through the last change in 1967. (The current rule set for Chicago are the “US” rules, defined elsewhere in the database.)

Shanks has a list, and the tzinfo database has the rules to create the list. Shanks also has an error that the tzinfo database corrects: the tzinfo database establishes that Chicago switched from local mean time (LMT) to standard time at 12:09:24, because Chicago is 9 minutes and 24 seconds ahead of the standard meridian for the time zone. Shanks puts the time at 12 noon, because his list shows the target time, not the trigger time, for the rule change.

Did the tzinfo project use Shanks to determine the rules for time changes? Yes, explicitly, though for highly-documented locations like Chicago the project participants cross-referenced Shanks with original sources, often correcting his errors. But "use" does not mean "copy;" I can use all the baseball statistics I want out of the newspaper without ever copying the newspaper. Data is not protected by copyright.

The tzinfo didn’t infringe on anyone’s copyright because Shanks created very little to protect. As I’ve previously explained, facts and data do not enjoy copyright protection in the United States. Only the expression of facts does. So if the tzinfo project had photocopied Shanks’ atlases, or republished the ACS software wholesale, then perhaps there would be an infringement. But I think I’ve shown a bit of why the tzinfo project hasn’t done anything actionable.

The Shanks atlases are like meticulously hand-copied illuminated codices from the 16th century, years after Gutenberg made his Bible and made hand-copying obsolete. I’m glad Shanks did the work; I’m sure he felt like he’d accomplished something huge. I really admire the work that went into it, while at the same time shaking my head at the wasted effort. Because since the late 1990s, all that data—latitudes, longitudes, place names—has been available for free from the Census Bureau and the CIA.[1] Before around 1998, you couldn’t just download the data through FTP for free; you had to write a letter to the appropriate agency and pay for it. But being U.S. government data, it was in the public domain, so once you’d paid for it, you could republish it in an easier-to-use form and recoup royalties.

In an era before the Census Bureau started dumping terabytes of data to the Internet, Shanks’ atlases would have been incredibly convenient sources of geographic and time-zone data. Today, they’re curiosities, monuments to exactly the kind of mental effort obviated by fast, cheap computers and the Internet.

Poor Shanks, all those data, thousands of rows of it, standing nakedly, and often erroneously, on page after page of tables in two massive volumes, apparently not knowing that he could have gotten it from the U.S. government—you have to admire that work ethic.

Astrolabe, for its part, has degenerated into exactly the kind of mental deficiency reviled by those of us who actually create software for a living. I eagerly await their much-deserved legal defeat in the next few months.

[1] Yes, the CIA publishes tons of free data, from their World Factbook to entire databases of geospatial information.

Edited at 20:58 UTC: Clarified the difference between "use" and "copying."

Occam's razor shaves climate science

Despite tons of research that support the anthropomorphic climate change theory, some people persist in the belief that the data does not support it. And yet, this week, there's more data:

[A] new study of current data and analysis by Berkeley Earth Surface Temperature...estimates that over the past 50 years the land surface warmed by 0.911°C: a mere 2% less than NOAA’s estimate. That is despite its use of a novel methodology—designed, at least in part, to address the concerns of what its head, Richard Muller, terms “legitimate sceptics”. ... At a time of exaggerated doubts about the instrumental temperature record, this should help promulgate its main conclusion: that the existing mean estimates are in the right ballpark. That means the world is warming fast.

My trouble with climate-change skeptics remains the same, a question about economic incentives. I guess I just don't understand why people persist in irrational beliefs when the evidence weighs so clearly towards an incompatible conclusion. In the case of quotidien religion, I live and let live. But in the case of anthropomorphic climate change, I don't get it. If climate scientists are right, and we cut emissions and energy use to slow climate change, we all win. If climate scientists are wrong, and we cut emissions and energy use to no effect, we're out maybe a billion dollars—about 3 cents per person, worldwide. But turn it around: if climate scientists are right, and we do nothing, say goodbye to Hollywood (Florida). And if climate scientists are wrong, and we do nothing, we'll still have the health and cosmetic effects of all those carbon emissions to deal with—and we'll still likely run out of oil in two centuries, after giving all our wealth to people who hate us.

So somebody, please, explain whence the hostility to the theory comes? Because it seems to me like the hostility farmers had to being jabbed with cowpox pus in the 1780s. Of course it's unpleasant, but wow is it better than the alternative.

Submitted without comment

A British plastic surgeon recently announced his findings after a months-long investigation of a particularly British institution:

It sounds almost like parody – a top consultant plastic surgeon spends three months studying models appearing on Page 3 of a bestselling British red-top newspaper. Later this month he reveals his findings: the mathematical proportions of the perfect breast.

This year [Patrick Mallucci, Consultant Plastic Surgeon at University College London and the Royal Free Hospitals,] conducted a three-month study to pinpoint the exact factors that make a woman’s breasts attractive. Titled Concepts In Aesthetic Breast Dimensions: Analysis Of The Ideal Breast, Mallucci’s study analysed the breasts of 100 topless models.

Thank you, Andrew Sullivan, for bringing this to the fore.

More about Groupon's IPO

Yesterday the Tribune reported on Groupon scaling back their IPO, from which they had hoped to raise the equivalent of Norway's GDP. Today's Economist has more:

Groupon created a new market. This is a boon to consumers, but confers no lasting “first-mover” advantage on Groupon. Its business model is unpatentable and simple to replicate, so there are already more than 20 copycats.

Groupon aspires to be global, but the markets it serves are intensely local. Internet selling is best suited to “experience goods”. These are goods and services the quality of which you cannot judge until you experience them, such as haircuts and Thai meals, so there is no advantage in having a bricks-and-mortar shop for people to browse in. (In North America 83% of Groupon’s deals fall into this category.) The trouble with experience goods is that generally you cannot separate manufacture from delivery: you cannot cook a meal in Guangzhou and eat it in New York.

Groupon was, some may recall, the hottest company in Chicago, so of course I want the company to succeed. I've also had some experience with Internet start-ups, so watching Groupon the past couple of years has felt...familiar. In particular, I've seen what happens to companies that grow by an order of magnitude in only two years.

Another interesting tidbit, possibly related: Groupon CEO Andrew Mason said as recently as June that he wasn't getting married until after the IPO. But the Tribune's business blotter reported Monday that he and Jenny Gillespie have tied the knot.

How much would *you* pay?

The Tribune is reporting that Groupon, one of several thousand companies that strikes deals with vendors, has scaled back its IPO:

The size of the sale, expected to be completed in the next two weeks, could be $500 million to $700 million under plans to be disclosed in advance of the company's roadshow beginning in the next few days, the people said. The size is meant to cut the amount of stock being sold at what may be a knock-down valuation, in hopes that more shares can be sold later at higher prices.

Although valuations of $20 billion to $30 billion were bandied about by outsiders at the time the company filed its plans to go public in May, the current goal of less than half that reflects the reality that the IPO window was closed for nearly two months between mid-August and mid-October because of overall stock-market weakness, and missteps by the company itself.

Amazon, Living Social, Google, and others of Groupon's competitors could not be reached for comment.

Those silly hippies!

I can't think for a moment what those odd folks at #OWS are protesting. It couldn't be crap like this, could it?

Buoyed by one-time gains from accounting changes and the sale of assets, Bank of America reported a $6.23 billion profit for the third-quarter Tuesday, even as weakness on Wall Street hammered underlying results and the firm surrendered its position as the country’s largest bank by assets.

The other major commercial banks that have reported earnings in recent days posted profits of around $4 billion each. Both Citigroup and JPMorgan Chase benefited from a $1.9 billion increase from the accounting change applied to the declining value of their company debt, posting profits of $3.8 billion and $4.3 billion, respectively. Wells Fargo had record earnings of $4.1 billion despite a 6 percent drop in revenue.

I have two thoughts. First, this kind of thing has to change. Second, where's mine? (Hey, I'm from Chicago.)

Policy, not providence

Paul Krugman once again points out the obvious and straight line between policy choices and the economy today:

[T]he financialization of America wasn’t dictated by the invisible hand of the market. What caused the financial industry to grow much faster than the rest of the economy starting around 1980 was a series of deliberate policy choices, in particular a process of deregulation that continued right up to the eve of the 2008 crisis.

Not coincidentally, the era of an ever-growing financial industry was also an era of ever-growing inequality of income and wealth. Wall Street made a large direct contribution to economic polarization, because soaring incomes in finance accounted for a significant fraction of the rising share of the top 1 percent (and the top 0.1 percent, which accounts for most of the top 1 percent’s gains) in the nation’s income. More broadly, the same political forces that promoted financial deregulation fostered overall inequality in a variety of ways, undermining organized labor, doing away with the “outrage constraint” that used to limit executive paychecks, and more.

Other commentators have noted that the Occupy Wall Street demonstrations seem to lack coherence or an intellectual center. Krugman disagrees, pointing out that they all share a common target: plutocrats.

Gnawing away at the middle classes leads, eventually, to the destruction of democracy. Today is the anniversary of just such an occurrence.

Simplified explanation of tzinfo mess

The AP has picked up the story about the tzinfo database moving to ICANN:

The organization in charge of the Internet's address system is taking over a database widely used by computers and websites to keep track of time zones around the world.

The transition to the Internet Corporation for Assigned Names and Numbers, or ICANN, comes a week after the database was abruptly removed from a U.S. government server because of a federal lawsuit claiming copyright infringement.

Without this database and others like it, computers would display Greenwich Mean Time, or the time in London when it isn't on summer time. People would have to manually calculate local time when they schedule meetings or book flights.

Ah, I do love the popular press, trying to explain things. AP writer Anick Jesdanun generally did all right explaining the problem and the move, except the story has no information about the tzinfo community's response to the mess. (I'm just sad they didn't mention The Daily Parker.)