set_trace # timfreund's blog on dev, ops, and other work stuff

Digital Speedometer just like KITT

Posted: Sun, 13 Oct 2013

KITT captured my imagination when I was four years old. If you weren’t a child of the 80’s, you may not know the depths of KITT’s awesomeness. Gaze carefully at his feature list and try to control the swell of emotions that stir deep within you. A “Molecular Bonded Shell”, a “Third Stage Aquatic Synthesizer”, “Passive Laser Restraint System”, a built-in ATM, and an artificially intelligent personality voiced by Mr. Feeny. I wanted a Firebird or Trans Am for years after seeing KITT in action.

Around the same time that KITT was on television, my dad was shopping for a new vehicle to replace his incredibly crappy Buick Skylark. Although he eventually purchased a totally plain and perfectly suitable Dodge Ram pickup truck, he considered a couple of different options before signing on any dotted lines. I’m fairly certain he test drove a Dodge Daytona, and it mesmerized me. It had a bright, digital dashboard reminiscent of KITT’s. The steering apparatus was a totally boring wheel, but I was willing to overlook it. Not every car can have a yoke.

Decades later I rented a car with a digital dashboard. Once the novelty wore off, I was disappointed with the utility of it all. The two digit speed display required reading. I couldn’t keep tabs on my speed with my peripheral vision quite like I could in cars with analog dials.

An analog gauge, whether physically analog or digitally rendered, provides the moment in time value along with the context of expected and potential ranges. For instance, the tachometer below shows a potential range of 0 through 16 thousand RPM, but the area between 14K and 16K are clearly marked to indicate potential trouble from running within that range:

Sidney Dekker talks about this briefly in chapter 14 of The Field Guide to Understanding Human Error. He claims that numeric digital readouts “can hide interesting changes and events, or system anomalies.”

Why is it, then, that I have produced so many monitoring scripts that produce little more than a two, or three, or four digit number as output? The raw numbers are necessary but not sufficient for use in a production support situation. The numbers must feed into a system that will graphically provide context over time.

One last thing. It’s OK if you got a little misty eyed when KITT was dropped into a pit of acid.

Monitoring Nuclear Weapons Tests

Posted: Sun, 14 Jul 2013

But every increase in sensitivity resulted in a large increase in the expected number of false alarms. The designers did not understand that in the monitoring of a test ban treaty, a high false-alarm rate would be far more troublesome politically than a low detection rate.

… This is the paradox: too much verification may be as bad as too little.

— Weapons and Hope, p.175

Scientists and engineers worked on the technical implementation of a nuclear test ban treaty with little regard for politics. These professionals made up the top echelon of thought across their areas of expertise, and they aimed for accuracy. No expense would be spared in the quest for accurate detection of nuclear blasts regardless of their size or location.

The rapid escalation inherent in nuclear conflict raised the cost of inaccurate blast detection to such a level that missing a blast would actually be preferable to accidentally classifying an earthquake or other natural event as a blast. Exponentially so if the detection system was ever wired in to a rapid response system, automated or not.

How rapid is rapid escalation? Most of the people reading this grew up in the United States. Think back to grade school. Do you remember the nuclear shelter area in your school if there was one at all? At Queen of the Holy Rosary grade school it was the 400 square foot art classroom. It was in the basement, and there were no windows, but there was also no seal on the door, and no ventilation system, and no stockpile of food or water. It was a tight fit for art class, and there were only 21 kids in my grade. That fallout shelter checked a box on a long and meticulously designed civil defense form, but it sure as hell wasn’t going to provide any shelter from fallout for the 160 kids and 20 staff members in the building if the need arose.

That ridiculously undersized and understocked basement room wasn’t an oversight or a corrupt attempt to save money. It was a conscious decision driven by the strategies and attitudes of the Cold War.

Shelters are perceived to be futile because the assured destruction strategy demands that they be ineffective. [Capable] Shelters are perceived to be threatening because they suggest an intention to make the operation of assured destruction unilateral.

—Weapons and Hope, Chapter 8, p. 90

“A strange game. The only winning move is not to play.”

Monitoring enterprise IT beats us all down at times, but no one dies in the course of our experimentation. Tweet about how much #monitoringsucks all you want. Get it out of your system and then recall that we have the power and time to make things better without killing most of the human population.

Small changes made frequently makes monitoring suck less, so make some changes. Just don’t kill anyone.