The latest installment of the sad, long-running serial "Programmer, Meet the Book Business" is supplied by Raymond Chen. He writes The Old New Thing, a blog full of highly useful technical information for those dinosaurs (like me) who still interact directly with the Win32 API (rather than relying solely on one of Microsoft's 15 bloated, incompatible frameworks designed to make programmer's lives easier). The blog post title of Whew, I'm not doing *that* again! sort of gives away the crux of his experience as a book author, but in case it left you with doubts, you can read on to find him say "I make barely any money from the book at all.".
One part of me says that stories like his are doomed to keep repeating at least in part due to the same mentality that causes Cartesian Programming. Notice how, long after agreeing to do the book, writing the book, and waiting around to see it flop, he finally gets around to looking to see what other programming book authors' experiences are. My prejudice is to believe that this is characteristic programmer arrogance, but maybe it's just human. After all, there are people who make a living walking up to people, telling them they want to make them famous models, and then extracting fees from them. They couldn't make a living if most people exercised the commonsense strategy of stopping to do a little research on how professional modeling actually works.
What Went Wrong?
Without knowing the slightest detail of Chen's book publishing experience, I can probably paint a picture that describes what happened here that's in the ballpark. Acquisitions editor has a quota to fill. Publishers aren't like Pixar, they don't work as hard as it takes to make every title a hit. Quite the opposite. They work hard to keep their costs down so they can throw a bunch of titles against the wall every year and see if any of them stick.
Raymond was an attractive target, since he had already cranked out a ton of technical information in his blog. Yeah, that has the disadvantage that the content was already out there for free, but it has the advantage of raising the odds that the author can actually deliver a manuscript in a reasonable amount of time. You can take those risks by keeping costs low enough
Part of keeping costs low is that the author gets to market the book. Even though there is often some mention of this fact to the author, usually publishers fail to convey how much the author's efforts can make the difference between success and failure.
Finally, there's the small matter of making the author any money. Almost all publishers who will offer you a book deal are publishing the way God intended: with 45 middlemen taking a cut before the author gets a dime. After all, you want your book to appear in a real bookstore right along with... the latest teen romance/vampire novel, don't you? In this tried and true (and in recent years, truly failing) publishing model, almost none of the people in the chain know or care the slightest thing about the content (remember that thing we somehow expect people to pay actual cash for?) of your programming book. And, of course, the final act of not caring is when the chain bookstore that never made any effort to sell your book decides they can't sell it, so they tear the covers off and mail them in and mulch the books (the covers prove they aren't just lying about how stinking bad your sales were). It really is a sad soap opera, isn't it?
What Could Have Gone Right?
If I had made it my life's duty to make Raymond's book successful, could I have done any better? Who knows, but I think so. First off, forget about putting this book in every book and grocery store. This book should have been sold direct via the web. Do you really think that even 0.0001% of Borders customers just happened to stumble over this highly targeted, highly technical book and decided to buy it on impulse? Of course not. Just about everybody in the target demographic for this book spends lots of time on the web and is comfortably buying books off the web. Selling direct makes it possible to spend a lot less money on middlemen. You think I jest when talking about 45 middlemen? Take one example. When selling books in a bookstore, suddenly the cover and spine of your book take on majestic importance to people at the publisher who will never, ever, actually read your book. Someone has the job of being obsessed with how your spine will stand out from its neighbors in the bookstore. And, being a first-time author, you have some longstanding dream about putting something idiotic on the cover like your Grandma's favorite dead cat -- it's their job to talk you out of that crap while not pissing you off. That person gets paid a salary. Selling direct removes the need for most of that dithering (well, except for the part about convincing the author to give up their nutty cover demands).
Of course, the real pain of the middleman is much bigger than any of the small potatoes like the cover designer. The distributor wants a big percentage of your price just for the privilege of getting a copy or two into the big chains. Once you've got your book distributed, what do all those various bookstores do with it? Why, they start competing on price. And boy, can they afford to compete when their contract says they can just return/mulch any copies they can't sell. For example, right now Amazon claims that Raymond's book costs $40, but they have discounted it to $26. If you sell direct, nobody else gets to decide how much to discount new copies of your book. As you can see, Raymond's book could have been sold direct at a no-discount price of $36 and still left $10 more profit per book to go around.
Or, suppose I really do want the customer to get the book for $26. If I'm a small publisher, the distributor might only want to pay me $10 for each of Raymond's $40 books. If I sell it direct for $26 (same as Amazon's discounted price), my costs for running the credit card and doing fulfillment would have to be $16/book to make life as bad as using a distributor. I can pay others to do the processing and fulfillment for much less than that.
I have no idea what-all marketing took place for this book, but I suspect it could have been better. Like I keep saying, this book has a pretty narrow audience, and you have to turn that to your advantage by knowing (publishers usually don't) where that exact demographic hangs out. For example, the Association of Shareware Programmers has quite a few hard-core API programmers, and when Raymond's book was mentioned there (not by Raymond or any marketing folk from his publisher), most people hadn't heard of the book or him. For the price of a couple of review copies to selected ASP members, I would have stimulated some early easy sales and good blog reviews.
Publishers (and often authors) don't really have the savvy to know things like the fact that the ASP would have been a good (essentially free) place to market this book. They are operating on thin margins, and can't really afford that kind of cross-domain expertise (if they even knew how to find it).
Will This Change?
Publishing changes pretty slowly, but there's no reason tech publishing can't change faster. Most publishers are focused on the threat/opportunity of electronic publishing. I think they've missed a big boat. Good old paper tech books still have a lot of life, IMHO, but the model has to change so that it can actually afford to pay the writer and (even more important and expensive than in other genres) pay a damn good technical editor.
Where's that money going to come from? I think it can come without any cost increase to the customer by being a direct-sale publisher and ceasing to pay all those people who can't actually read your book but currently get a bigger chunk of the book cover price than the author. But that's just me; I could be completely nuts.