The Limits of Melancholy

Of all the programming books you've never read that I've read more than once, The Limits of Software by Robert Britcher is my favorite. I say you've never read it with some confidence because a) few programmers read books and b) this book was doomed to a narrow audience by its very nature.

The first audience-limiting feature of the book is its style. As Robert Glass says in the Foreword, "It's part storytelling, part history, part art, part science, part philosophy, part logic--all entwined around the subject of computing and software." At the hub of it all is Britcher's experience with one of the largest engineering projects in history: the computer systems of the FAA. But that is only the hub, and the spokes are long and not straight or predictable. If you just want a straight story on his experience with the FAA Advanced Automation System, you can read Britcher's more ordinary recitation that appeared in the book Software Runaways. But this is more like sitting down to talk to someone who was in World War II--you don't get facts, figures, or dates like a textbook would give you, but you get stories that make you think, that you can find morals in, if you care to look for them.

The second audience-limiting feature of this book is that you kind of have to be an old programmer to appreciate it. For example, Ada is mostly just a name rather than a saga to a great many programmers who never knew a PC-less world. I remember when some distant cousin, being a military contractor, notified my parents that I should be learning Ada because it would soon be required of all programmers. I worried briefly, noticed that nobody else writing C code was worried, then forgot about it. Later, a small consulting company I was in got the job of helping port an Ada compiler to some Unix platforms. We marveled at the test suites it had to pass and wondered no more where our income tax money went. Much later, I briefly plunged into a real, honest-to-God Ada disaster as a sub-sub-sub-contractor on an anti-submarine aircraft software project for the military. The game, as it was explained to me by my elders on the project, was that everyone knew the project would fail, but we had better make damn sure that we were not the first sub-contractor to miss a milestone and get the blame for the true state of things. As I stared at hundreds of lines of paper Ada (there were not enough terminals for me to actually edit anything) and found that not a single identifier could be understood without first looking through reams of definitions and declarations elsewhere, I decided that this was the game Ada was born to play. I think it's hard to grasp lines like "Ada the language was eclipsed by Ada the cottage industry" if you never brushed up against it in your life.

More Than a Feeling

The thing that's unique to me about Britcher's book is that reading it leaves me with a powerful feeling. I get lots of things from programming books, but usually not an emotion. Oh, I did get a giddy feeling when I read Peopleware that the state of programming management was really going to improve. How embarrassing it is now to confess that I actually thought a book could make a dent in management behavior!

It took me a long time to put my finger on exactly what it was that The Limits of Software made me feel. Then I realized: it was melancholy. Melancholy is a hazard of age I conclude, now that my dotage is upon me. When you see email and instant messaging being invented again for the umpteenth time as though it were something new, or watch seven rounds of layoffs each be followed by hire-backs as management could never quite grasp that laying off N people reduces the head count by N+M (where M is the number of people deserting a sinking ship), melancholy is the natural response.

But the melancholy that Britcher instills is grander than I can convey here. The subject, after all, is about limitations, not abilities and accomplishments. If Isaac Newton saw further by standing on the shoulders of giants, Britcher sees further by standing on the piled corpses of our dreams. Dreams of bug-free code. Dreams that formalism and better processes will prevent software disasters. Dreams of "the right way" to develop software. Dreams of a better programming language that will remove our current dissatisfactions. Dreams that reusability will drastically lower development costs. Dreams that technology is the answer to human problems, and all shall be well, and all shall be well, and all manner of things shall be well.

The Old Old Thing

The latest installment of the sad, long-running serial "Programmer, Meet the Book Business" is supplied by Raymond Chen. He writes The Old New Thing, a blog full of highly useful technical information for those dinosaurs (like me) who still interact directly with the Win32 API (rather than relying solely on one of Microsoft's 15 bloated, incompatible frameworks designed to make programmer's lives easier). The blog post title of Whew, I'm not doing *that* again! sort of gives away the crux of his experience as a book author, but in case it left you with doubts, you can read on to find him say "I make barely any money from the book at all.".

One part of me says that stories like his are doomed to keep repeating at least in part due to the same mentality that causes Cartesian Programming. Notice how, long after agreeing to do the book, writing the book, and waiting around to see it flop, he finally gets around to looking to see what other programming book authors' experiences are. My prejudice is to believe that this is characteristic programmer arrogance, but maybe it's just human. After all, there are people who make a living walking up to people, telling them they want to make them famous models, and then extracting fees from them. They couldn't make a living if most people exercised the commonsense strategy of stopping to do a little research on how professional modeling actually works.

What Went Wrong?

Without knowing the slightest detail of Chen's book publishing experience, I can probably paint a picture that describes what happened here that's in the ballpark. Acquisitions editor has a quota to fill. Publishers aren't like Pixar, they don't work as hard as it takes to make every title a hit. Quite the opposite. They work hard to keep their costs down so they can throw a bunch of titles against the wall every year and see if any of them stick.

Raymond was an attractive target, since he had already cranked out a ton of technical information in his blog. Yeah, that has the disadvantage that the content was already out there for free, but it has the advantage of raising the odds that the author can actually deliver a manuscript in a reasonable amount of time. You can take those risks by keeping costs low enough

Part of keeping costs low is that the author gets to market the book. Even though there is often some mention of this fact to the author, usually publishers fail to convey how much the author's efforts can make the difference between success and failure.

Finally, there's the small matter of making the author any money. Almost all publishers who will offer you a book deal are publishing the way God intended: with 45 middlemen taking a cut before the author gets a dime. After all, you want your book to appear in a real bookstore right along with... the latest teen romance/vampire novel, don't you? In this tried and true (and in recent years, truly failing) publishing model, almost none of the people in the chain know or care the slightest thing about the content (remember that thing we somehow expect people to pay actual cash for?) of your programming book. And, of course, the final act of not caring is when the chain bookstore that never made any effort to sell your book decides they can't sell it, so they tear the covers off and mail them in and mulch the books (the covers prove they aren't just lying about how stinking bad your sales were). It really is a sad soap opera, isn't it?

What Could Have Gone Right?

If I had made it my life's duty to make Raymond's book successful, could I have done any better? Who knows, but I think so. First off, forget about putting this book in every book and grocery store. This book should have been sold direct via the web. Do you really think that even 0.0001% of Borders customers just happened to stumble over this highly targeted, highly technical book and decided to buy it on impulse? Of course not. Just about everybody in the target demographic for this book spends lots of time on the web and is comfortably buying books off the web. Selling direct makes it possible to spend a lot less money on middlemen. You think I jest when talking about 45 middlemen? Take one example. When selling books in a bookstore, suddenly the cover and spine of your book take on majestic importance to people at the publisher who will never, ever, actually read your book. Someone has the job of being obsessed with how your spine will stand out from its neighbors in the bookstore. And, being a first-time author, you have some longstanding dream about putting something idiotic on the cover like your Grandma's favorite dead cat -- it's their job to talk you out of that crap while not pissing you off. That person gets paid a salary. Selling direct removes the need for most of that dithering (well, except for the part about convincing the author to give up their nutty cover demands).

Of course, the real pain of the middleman is much bigger than any of the small potatoes like the cover designer. The distributor wants a big percentage of your price just for the privilege of getting a copy or two into the big chains. Once you've got your book distributed, what do all those various bookstores do with it? Why, they start competing on price. And boy, can they afford to compete when their contract says they can just return/mulch any copies they can't sell. For example, right now Amazon claims that Raymond's book costs $40, but they have discounted it to $26. If you sell direct, nobody else gets to decide how much to discount new copies of your book. As you can see, Raymond's book could have been sold direct at a no-discount price of $36 and still left $10 more profit per book to go around.

Or, suppose I really do want the customer to get the book for $26. If I'm a small publisher, the distributor might only want to pay me $10 for each of Raymond's $40 books. If I sell it direct for $26 (same as Amazon's discounted price), my costs for running the credit card and doing fulfillment would have to be $16/book to make life as bad as using a distributor. I can pay others to do the processing and fulfillment for much less than that.

I have no idea what-all marketing took place for this book, but I suspect it could have been better. Like I keep saying, this book has a pretty narrow audience, and you have to turn that to your advantage by knowing (publishers usually don't) where that exact demographic hangs out. For example, the Association of Shareware Programmers has quite a few hard-core API programmers, and when Raymond's book was mentioned there (not by Raymond or any marketing folk from his publisher), most people hadn't heard of the book or him. For the price of a couple of review copies to selected ASP members, I would have stimulated some early easy sales and good blog reviews.

Publishers (and often authors) don't really have the savvy to know things like the fact that the ASP would have been a good (essentially free) place to market this book. They are operating on thin margins, and can't really afford that kind of cross-domain expertise (if they even knew how to find it).

Will This Change?

Publishing changes pretty slowly, but there's no reason tech publishing can't change faster. Most publishers are focused on the threat/opportunity of electronic publishing. I think they've missed a big boat. Good old paper tech books still have a lot of life, IMHO, but the model has to change so that it can actually afford to pay the writer and (even more important and expensive than in other genres) pay a damn good technical editor.

Where's that money going to come from? I think it can come without any cost increase to the customer by being a direct-sale publisher and ceasing to pay all those people who can't actually read your book but currently get a bigger chunk of the book cover price than the author. But that's just me; I could be completely nuts.

Ignite Videos on YouTube

All the videos from talks at the 4/29/2009 Ignite Seattle! have made it up to YouTube now. Some links:

• Ignite Seattle! home page
• Their YouTube "channel"
• video of my talk

I actually only heard about the videos arriving from a friend of a friend who saw it go by on some RSS feed. By that time, there were already 47,000 views, so apparently it got posted on one or more popular sites. I tried submitting it as a SlashDot story, but rejected again (0 for 4 over the years, even though SlashDot likes me enough to let me turn off their advertising).

Alas, the domain I slapped up for the video to link to was inaccessible, proving I'm an incompetent sysadmin. Actually, I do sysadmin-ing to learn, not professionally. My ISP's DNS servers are slaved to my "stealth" server, so it looked to me like ronburk.com was up and running, but I forgot that I had never told my ISP to add that to the list of domains they slave to me for, so nobody outside my network could see it. Oh well, I'll get that little one-page wonder up today for sure.

I was surprised by the 47,000 views. I don't know if that implies something about the talk itself, or just luck of the draw having the right random people post links to it on the right forum. In any case, I'll definitely make an effort to present at Ignite Seattle! again when the book is done. They seem a little fuzzy on scheduling (whaddya want for a free event?), but this last Ignite was by all accounts pretty successful, so I'm hopeful it will become a more regular occurrence.

The popularity of this video has made me rethink the format. I had vowed that if I ever did another Ignite, I would not do a memorized speech, as it's too nerve-wracking. But it's really hard to get the maximal value out of 5 minutes if you don't choose your words carefully, and that pushes you towards using a script. Maybe it won't be as nerve-wracking if I have more than a few days to prepare.

Or maybe one gets better at memorizing with practice. Few of us can recite something the length of the Iliad from memory, but there used to be rather more people on the planet who could do that sort of thing before cheap paper and pens greatly decreased the value of memorization as a skill.