Steak and Lobster

I’ve never cruised before. I don’t get excited about beaches, boats, or the ocean, so cruises cost a lot of money for 24x7 access to food and quiet time to read. I caught a thread on reddit concerning the death of nights and weekends for tech workers. A few tech workers commented that cruises were the only way they found peace on vacation, because most problems can be solved by the crew in the office if it costs $10/minute to talk to the guy vacationing on a boat. Concerning food, “either/or” was not a part of the lexicon, but “and” was:

I’m tempted to catch an off season deal and embark while wearing old man style pants with elastic waistbands just for the sport of it all.

We have our own steak and lobster moments. Python or Ruby? Both. Emacs or vim? Both. Java or C#? Both. Debian or Red Hat? Both. MySQL or Postgres? Postgres.

If there are zero additional constraints or prejudices coloring such a question, the best options are to flip a coin and get busy working or run a small experiment with both to form a stronger foundation for a real decision. We aren’t getting married to any technology, and each choice is both excellent and terrible depending on the problem at hand.

Should I learn Python or Ruby for freelancing?

Both. Or neither. Clients either have a technology in mind already or they will be confused and scared that specific technologies come up in the proposal process at all. The Rails shop that needs an extra hand wants a coder that knows Rails. The dentist that needs an appointment reminder system does not care about programming and programming languages as long as his missed appointment ratio looks better by the end of the quarter.

Most importantly, recall that my opinion in this matter is worth very little.

My opinion about barbell rows is as follows: **** barbell rows. Really. **** them. Stop wasting time worrying about barbell rows and get your deadlift up to 500. By then you’ll have your own opinion and you won’t have to worry about mine.

— Mark Rippetoe

The Office Powerball Pool

My dad has played Powerball twice a week since it entered the state of Kansas. I don’t ask about how much he’s won over the years, because I’m certain that I don’t want to know. The game wouldn’t exist if it payed out more than it brought in, so it just doesn’t make much sense.

But someone’s gotta win, and it might be me.”

I make one exception, and that’s the office Powerball pool that starts up once the jackpot gets fairly high. Even then, I don’t play because I think we’ll win. I play just in case we win as an insurance premium on being left behind. I don’t want to be the guy left behind when half of the technical team is no longer financially obligated to show up every day.

$100 Invested in 100 $1 Lottery Tickets made it to the front page of Hacker News the other day. I read it, thinking the whole time my dad’s experience with the lottery. And then I recalled our latest office pool stats:

  • 29 people
  • $5 each (+ an extra dollar from the pool leader to get us an even number)
  • Two runs at the jackpot at $146 each.
  • $26 in winnings.

Any money won in one drawing is rolled into the next “until we hit the big one.” We’ve developed a system for efficiently turning any sum of money into zero dollars. The more we lose, the more it feels like the insurance premium that it is. I keep paying, and I’ll probably never use it, but I dread facing the on-call rotation while we ramp up staff if the pool ever hits.

If you want to be a great developer

If you want to be a well-paid copywriter, please your client.

If you want to be an award winning copywriter, please yourself.

If you want to be a great copywriter, please your reader.

— Steve Hayden, found in chapter 2, page 21 of Hey Whipple, Squeeze This

The software guys and ad guys don’t often mingle in most companies, but that small piece of copy writing advice fits us well. Don’t build a resume builder. Don’t build a portfolio piece. Build a project that pleases the user and the other trappings of success will follow.

We can dazzle our peers with patterns, No-SQL databases, and the latest JavaScript MVC technology, but our success will ring hollow if the users don’t understand the damn thing. No user has once complimented me on my choice of data access technology, despite the ridiculous number of hours that I’ve spent fighting those components through the years. They just want the data to arrive quickly and correctly. Likewise, our clients aren’t impressed by thorough testing and deployment procedures, but they are impressed by rapid turnaround on bugs and feature requests.

Stepping back for a moment, why is a software guy burning time with a classic book in the advertising industry? I don’t know what got me started, but I’m glad I finished. Something somewhere on the Internet pointed me toward this book years ago, and I just found it while unpacking boxes after a move. Sixty-two notes and excerpts later, I recommend it to any developer looking to expand their education beyond a single threaded focus on technology. Old editions are still available for the cost of shipping, and there’s a new edition that is available for the Kindle.

Here’s one more of my six excerpts from chapter 2:

They’ll quickly find you boring or irrelevant if all you can speak about with authority is nginx configurations and WebSockets. Your grasp of the client’s technology situation has to be as well versed as any project manager’s. There are no shortcuts. Know the client. Know their product. Know their market. It will pay off.

— Chapter 2, Hey Whipple, Squeeze This

Wait, sorry, I copied that wrong. Can you make the following substitutions?

  • s/nginix configurations and WebSockets/Century Italic/
  • s/technology situation/marketing situation/
  • s/project manager’s/account executive’s/

Universal stuff. It just takes a little bit of mental search and replace.

Books and Notes

Take notes in and around the books you read, or you’ll lose most of the interesting bits in a very short period of time. (And please don’t tell me that you “aren’t a reader.” It’ll just make me sad.)

There’s a book called How to Read a Book. It’s thick. The authors had serious intentions of improving the quality of reading. I respect that, but with a single eyebrow raised, I first thought “learn how to read a book so you can learn how to read a book from the book Learn How to Read a Book.” Childish of me. I hope they had some fun during the editing process with a dummy draft:

How to Read a Book You’re doing it now. Keep going! [page break] Excellent, you’ve found us over here. That’s about it: left to right, top to bottom.

In fact, that’s the problem they’re trying to solve. Most of us measure reading skill in speed from start to finish with little attention paid to comprehension or retention.

Someone much smarter convinced me to pick up a copy in 2010 despite my initial skepticism. Of all the tips, techniques, and suggestions, taking notes changed my retention and comprehension the most. Marking notes in the margins felt criminal at first, but it’s now so fundamental to my process that I rarely read anything if I don’t have a pencil.

Amazon’s used marketplace is a great source of deals that keeps my reading habit below the expense reporting radar in our family budget. Through this informal research channel, I know that people fall into two camps with notes and markings: either no marks at all, or everything is underlined haphazardly, often with a yellow highlighter or ink pen.

One interesting fact about the over-enthusiastic underliners: their notes often stop entirely by chapter 3, and that means they missed most of the good stuff. I’ve also learned that some Amazon sellers don’t share my definition of “Used - Like New”, but that’s a different soap box rant.

For the sake of note-free readers and over-highlighting readers alike, here are just a few tips that have served me well.

Use a Pencil

Pencil is erasable. That may come in handy.

Underline judiciously.

Underlining entire paragraphs is a complete waste of time. Quit it. Is the entire paragraph really profound? Will it be worth reading again in its entirety when skimming the book in the future?

Are the underlined words key words that require definition, or are they just words that you happen to recognize and you get swept up in the excitement? I owned a book where the previous owner was probably a “Data Manager” because every occurrence of the words “Data Manager” in the first three chapters was circled. Not helpful.

Concisely underline the interesting and the new.

Use vertical lines to highlight paragraphs

Maybe there’s a passage that only makes sense in the context of one or more full paragraphs, and you want to find it again on your next time through the book. Use vertical lines in the margin to highlight those paragraphs rather than tediously underlining everything. Think about the time it would take to draw a vertical line next to this paragraph rather than underlining everything within the paragraph:

Now, go read a book.

Build Application Packages with FPM

FPM changed the way I think about packaging. I wrote so many lines of custom deployment code just to skip packaging our in-house application in the past, and it was completely justifiable given my previous experience with packaging tools

I mentioned last time that we’re living in the future. Things have changed for the better.

We now use Puppet for provisioning and configuration management, and that extends to our in-house applications running on those servers. We also have internal package repositories. Packaging pain kept us from fully using this infrastructure for our in-house applications, but FPM changes that.

Say we have a custom application to package. This isn’t an official release, it’s just a build triggered by the latest commit, and we have it laid out on disk just like it should be deployed. Easy:

timestamp=`date +%Y%m%d%H%M`
version=`git rev-parse --short HEAD`

fpm -s dir -t rpm -p ~/ourawesomeappliation-0.9.${timestamp}git${version}.rpm \
--name ourawesomeapplication -v 0.9.${timestamp}git${version} \
--prefix /the/place/to/deploy/it --exclude '*/.git/*' --verbose \
--debug .

Bash

The time stamp ensures that the system can compare versions correctly since git version hashes aren’t alphabetically ordered.

And for the Mercurial projects out there:

timestamp=`date +%Y%m%d%H%M`
version=`hg log . | head -1 | sed -e 's/.* //' -e 's/:.*//'`

fpm -s dir -t rpm -p ~/ourawesomeappliation-0.9.${timestamp}hg${version}.rpm \
--name ourawesomeapplication -v 0.9.${timestamp}git${version} \
--prefix /the/place/to/deploy/it --exclude '*/.hg/*' --verbose \
--debug .

Bash

Note that we’re creating RPM packages above. Also note that the -t argument defines the OUTPUT_TYPE or target. Swap in deb for rpm to get a Debian themed party started. Solaris packages are an option, too. That’s right, some people still deploy more than just Oracle products on Solaris machines.

Disable DHCP on a QEMU/libvirt/KVM Network

Libvirt + QEMU + KVM allows easy virtualization on Linux. Virtual machines are placed on a virtual network that comes complete with a DHCP server and DNS forwarder. There’s no reason to give up these default conveniences until your work involves building and configuring DNS and DHCP servers. Running two DHCP servers on the same virtual network is a recipe for frustration.

The configuration files for libvirt are stored in /etc/libvirt by default, but pretend like they aren’t even there. Read any of the files and you’ll find a warning:

<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
virsh net-edit devservers
or other application using the libvirt API.
-->

XML

They aren’t kidding. Don’t change these files. There’s one other trick:
Running the net-edit command alone won’t really do anything either if the network is already running. The following operations will do the trick:

root@host # virsh
virsh # net-destroy yournetwork
virsh # net-edit yournetwork
[remove the <dhcp></dhcp> element, save, exit]
virsh # net-start yournetwork

Bash

Rumor has it that later versions of the virt-manager GUI support these operations, but this info will come in handy if you’re still on a LTS system.

Reconnect any hosts that were on the virtual network, because they aren’t anymore. Configure your own DHCP server or configure all hosts with static interfaces as well. Remember that you’re on your own for DNS resolution in addition to DHCP at this point. An easy solution is to set domain-name-servers in your dhcpd.conf file to your router or access point.

Now that we’ve wrapped up our business here, I’d like to take a minute to tell you how happy all this stuff makes me. I’m running a small colony of machines on my middle of the line desktop computer, and I can practice the exact same work that happens in real data centers. Back when I was a newbie every machine was a real, physical machine, and PXE booting required careful hardware selection.

We’re living in the future. We just need the Hoverboard design team to catch up.

San Diego Comic-Con 2011 - Marty's Back to the Future II hoverboard (Profiles in History booth)

Call in the Night

I learned about Call in the Night earlier today.

A few years ago I thought it’d be cool to practice dream recall and eventually start lucid dreaming. With approximately one third of our lives spent asleep, it seemed like a fun way to pass the time. In the end, I didn’t think it was fun enough to really change my sleep habits.

The Call in the Night project looks a little interesting just because it’d be a way to tap into dream recall without waking up and immediately remembering to journal everything possible, but that’s not really what struck me the most.

If we’re writing software with a root cause of waking people up, we should use that power for good. This is obvious, and it’s probably been done: use Twilio Voice + SMS to send a wake up call.

  1. Sleeper receives phone call.
  2. Sleeper hears inspirational text, important business info, personal performance metrics, whatever.
  3. Sleeper receives a random code number and a question, like “what project will receive most of your time today.”
  4. Sleeper texts code number and answer to the wake up service to disable the snooze feature (repeat calls).

Or I could just get out of bed on the first alarm. Because just doing that has worked so well so far. (I’m a 2-3 snoozer, but that can extend if it’s a weekend and particularly chilly. Rationally I know this is stupid, but ask me again after a few hours of sleep.)

Book - Lead With a Story

This book almost lost me several times. Paul told stories. I wanted tactical advice on crafting great stories. Paul kept telling stories. There were just enough tactical nuggets throughout the book that I kept reading.

I read so many uplifting stories about Procter and Gamble that I momentarily doubted my own career path. Is this a part of a P&G propaganda machine? The Manager Tools and Advanced Selling Podcast crews both have experience with P&G as well. Am I missing out?

Stories are persuasive. Got it.

Paul recommends putting any surprise element at the end of a story to cement the story into the audience’s memory. In the very last chapter he got me. My consternation over writing my own stories melted away as I read:

The richest source of stories you’ll ever have are the stories you hear other people tell. There’s only one of you, but about 7 billion other people in the world. Even if you never create a single personal story of your own, you can have an endless supply of great stories by just paying attention to the stories you hear from others.

All of those stories that were stealing stage time from tactical tips? They really were the meat of the book. I have a cache of effective stories now. I should still write my own stories, but I don’t need to write every story I tell.

Chapter 30 also provided a number of solid tips and lists for getting started. The tips and lists I craved, but I would have never acted upon them had I devoured them on their own.

SQLAlchemy + ORA-014161

Python, SQLAlchemy and “ORA-01461: can bind a LONG value only for insert into a LONG column”

If you’re on this page you either read all of my entries regardless of content (thank you!) or you have the error above.

Look through just about every ORM and you’ll find special case handling for Oracle’s large objects. If there’s no special case handling in the ORM itself, then you will write some special case code yourself.

SQLAlchemy’s special case handling of Oracle’s large objects is to automatically load the large objects when retrieving rows from the database and convert them to strings. This means the database results object isn’t tethered to the database cursor while you work. It’s pretty handy, and if you’re just pulling data from the database it means you may never need to think about how weird Oracle large objects are compared to “regular” data.

I was reading and writing large objects between two schemas when the ORA-014161 surfaced. Since this was a one-time ad-hoc deal, I just hand crafted some SQL, and the script looked a little like this:

import sqlkeyring
 
sql_keyring = sqlkeyring.SQLKeyring()
 
targetdb = sql_keyring.get_engine('targetdb')
sourcedb = sql_keyring.get_engine('sourcedb')
 
interesting_results = sourcedb.execute("select * from the_interesting_table")
for row in interesting_results.fetchall():
column_names = row.keys()
column_names.sort()
 
names = ', '.join(column_names)
values = ', :'.join(column_names)
 
params = {}
for k, v in row.items():
params[k] = v
 
sql = "insert into the_interesting_table(%s) values(:%s)" % (names, value)
targetdb.execute(sql, **params)

Python

And that failed terribly with the following:

"ORA-01461: can bind a LONG value only for insert into a LONG column"

Of course it did, otherwise we wouldn’t be here now. Since the values came out of the source database and were converted to strings, SQLAlchemy then attempted to drop them into the target database as strings.

SQLAlchemy is smart enough to coerce types correctly if we give it some information about our schema, either explicitly or by asking it to autoload table metadata. A few changes to the script above and we were in business without manually hacking together SQL:

import sqlkeyring
 
sql_keyring = sqlkeyring.SQLKeyring()
 
targetdb = sql_keyring.get_engine('targetdb')
targetdb_md = MetaData()
targetdb_md.bind = targetdb
sourcedb = sql_keyring.get_engine('sourcedb')
 
the_interesting_table = Table('the_interesting_table', targetdb_md, autoload=True)
 
interesting_results = sourcedb.execute("select * from the_interesting_table")
for row in interesting_results.fetchall():
params = {}
for k, v in row.items():
params[k] = v
 
sql = the_interesting_table.insert()
targetdb.execute(sql, **params)

Python

About the same number of lines, but less hacky and it actually works by letting SQLAlchemy do more heavy lifting for us. Since SQLAlchemy has metadata information for the target table, it knows that any input for the large object columns will need to be properly converted.

Any tool that successfully abstracts away Oracle large objects is a good tool in my book.

Distractions and Truth In My Shell Prompt

My shell prompt doesn’t lie.

I’m a bash user, so I can set $PS1 to contain a variety of useful information, and my prompt will vary depending on where I’m at. We have enough servers and services at $DAYJOB that the following makes a lot of sense:

\u@\h (\t) $

That prints:

USERNAME@HOSTNAME (TIMESTAMP) $

You can learn all about the various bash shell escape sequences if you’d like.

People tell me that showing the time is silly. I like it because there are times when I run a long running process and walk away while it churns. It’s often helpful to know how long something took, and having timestamps on each line let me know.

It’s helpful to me personally as well. There are many times when I’m drawn away from scheduled work into an urgent support situation, and I’ll leave my scheduled work as-is and work the urgent situation in a new terminal tab or window. That’s exactly what happened today, and I was a little shocked at the story the timestamps had to tell:

tfreund@somehost (10:28:15) $ the_last_command_i_ran_that_you_dont_care_about.sh
tfreund@somehost (10:28:20) $ the_first_command_i_ran_after_getting_back_at_it.sh
tfreund@somehost (14:38:31) $

Over four hours. I’ll admit that wasn’t all urgent support. I ate lunch and checked email, too. It took a look at my important task list to jog my memory back to the work I was doing at 10:28. “Oh, right, that’s what I really need to finish today.”

Working for 8 solid hours is an illusion. It’s not going to happen. Unless work happens in a cabin off the grid away from civilization, interruptions will wreck your day. Almost every day. More important than blocking off 8 solid hours is to put in place systems to continually nudge ourselves toward the important tasks once the storms of unavoidable urgent work have passed.

My favorites include: