A few years later — in 1982 — my close friend Wayne Holder hired me into his nascent software company, Oasis Systems, in part to help with his existing and planned word processing utilities (The Word Plus, Punctuation + Style), but mostly to develop computer games. And we did, developing Sundog: Frozen Legacy on the Apple II, a game for which I still get e-mails (and which Wayne is even now working on resurrecting for modern platforms). In January 1984, a few months before Sundog shipped, we were invited by Guy Kawasaki to come up to Apple to see a preview of the Mac and to talk about what software we could port to the Mac. Through my connections with computer stores in San Diego, I was able to get a personal loan of a Mac for a few days at home prior to the official announcement in Cupertino later that month, which Wayne and I attended as well. That was my first time seeing Steve Jobs in person, and it remains a memorable highlight of my professional life.

When the Mac shipped a few days later, I went down to the one computer store in San Diego that I knew would be getting machines from Apple. I took $3000 in cash with me and managed to convince the store owner — a friend — to let me have one of the three Macs he had to sell. Through a connection with Phil Lemmons — editor-in-chief at BYTE — I ended up writing the official BYTE review of the 128K Macintosh (August 1984 issue). By the end of 1984, I was writing full-time for BYTE, including on-going coverage of the Macintosh, particularly once my BYTE column started in mid-1985. After a few years of writing for BYTE, I switched to writing for Macworld magazine. Steve was now long-gone from Apple, and Apple was having some of its own problems going forward.

But in late 1987, I was contacted by Addison-Wesley. They were interested in having me write a book about Steve Jobs’ new project at NeXT. Folks at NeXT had apparently suggested me to Addison-Wesley, probably due to my writing at BYTE and Macworld. I leapt at the opportunity, particularly since in coincided with our family moving from Utah to just outside Santa Cruz (where I would be doing technical writing for Borland on a consulting basis). Once there, I found myself invited to visit NeXT HQ on Deer Creek Road, sit in on meetings, and attend the 0.3 NeXTstep Dev Camp. And, yes, that meant getting actual face time with Steve Jobs as well — not a lot, but this was a man whose creations had been impacting my personal and professional life for over a decade at this point.

The writing of the book dragged out as I waited to get my hands on an actual NeXT cube, which finally happened (if I recall correctly) at the end of 1988 or early 1989. I wrote the first several drafts of the book on that NeXT cube itself. The book came out in the fall of 1989; it remains the single most successful book I’ve ever written, due to the intense interest in NeXT itself, more than any particular writing skills or technical insight on my part.

The following year, I found myself working with a world-class typographer (Mike Parker) and graphic designer (Vic Spindler) to create a design-oriented desktop publishing system. I was doing all the software prototyping on my NeXT cube, and we made the decision to make the NeXT our first target platform. For five years — 1990 to 1995 — I served as chief architect and CTO at Pages Software Inc, where we developed Pages by Pages and then WebPages, while spending nearly two years just trying to raise venture funding. We closed on funding at the start of 1992 and shipped our first version of Pages in early 1994. We quickly sold all that we were going to in the all-too-small NeXTstep market. My frustrations at seeing larger firm try to leverage off of NeXT’s incredible innovations led to an op-ed piece in the November 1994 issue of BYTE, “Whither NextStep?” The day that issue came out was the last time that Steve Jobs and I spoke — he called me from the back of a car somewhere to ask me what the hell I was doing writing that. I said, telling the truth. Pages would close its door the next year, unable to secure additional funding to move its technology to Windows.

When Steve engineered his brilliant reverse takeover of Apple — getting Apple to buy NeXT for $400 million, then slowly moving himself into the CEO seat — I was not optimistic. I still had unconditional praise for the NextStep technology, but I was dubious about Steve’s ability to sell technology to markets and to compete with Microsoft.

Boy, was I wrong. I was not only wrong about his abilities at Apple, I was wrong in my BYTE article about NextStep being on a downward slope. NextStep, of course, was the foundation of Mac OS X, and Steve transformed Apple into the most-admired, most-imitated, and most-valuable company in the world. And I was tickled that, when Apple brought out its own word processor, it was named “Pages”. Steve had always liked that name when we were developing (and shipping) our own product years before; glad he was able to use it.

To quote John Perry Barlow over on FB, “The world is suddenly a less interesting place.” ..bruce w..

[1] The first was an HP-67 card-reading programmable calculator.

[Cross-posted from And Still I Persist]

Now, thanks to advances in artificial intelligence, “e-discovery” software can analyze documents in a fraction of the time for a fraction of the cost. In January, for example, Blackstone Discovery of Palo Alto, Calif., helped analyze 1.5 million documents for less than $100,000.

Some programs go beyond just finding documents with relevant terms at computer speeds. They can extract relevant concepts — like documents relevant to social protest in the Middle East — even in the absence of specific terms, and deduce patterns of behavior that would have eluded lawyers examining millions of documents.

“From a legal staffing viewpoint, it means that a lot of people who used to be allocated to conduct document review are no longer able to be billed out,” said Bill Herr, who as a lawyer at a major chemical company used to muster auditoriums of lawyers to read documents for weeks on end. “People get bored, people get headaches. Computers don’t.”

Since I work in litigation support myself and have to search and read documents — I’ve had individual cases with over a million pages of documents — I welcome such tools as these. Still, the overall theme in the article seems to be that humans will lose jobs due to these tools:

Quantifying the employment impact of these new technologies is difficult. Mike Lynch, the founder of Autonomy, is convinced that “legal is a sector that will likely employ fewer, not more, people in the U.S. in the future.” He estimated that the shift from manual document discovery to e-discovery would lead to a manpower reduction in which one lawyer would suffice for work that once required 500 and that the newest generation of software, which can detect duplicates and find clusters of important documents on a particular topic, could cut the head count by another 50 percent.

What the article does not address is the enormous financial burden that e-discovery has imposed on both parties in modern civil litigation (see also here and here).

It used to be that each side would gather physical documents from files and storage boxes, review them for responsiveness, then produce them to the other side. Outside of large corporations, most parties would not have great volumes of such documents.

However, with the advent and pervasiveness of digital technology — computers, e-mail, servers, instant messaging, and so on — even a relatively small firm or organization can find itself with gigabytes, if not terabytes, of potentially relevant production. And, to avoid court sanctions, all of that has to be combed through and reviewed for responsiveness. Most organizations below a certain size just can’t afford to do that the old-fashioned way with “a platoon of lawyers and paralegals who [work] for months at high hourly rates”.

The article also does not address the existing document organization and search tools that are a bit less artificially intelligent — and a lot cheaper — than the tools he covers, but that still allow complex search terms to be set up. I make heavy use of dtSearch in most of the cases I work on (and can recommend it highly). I’ve also used such standard legal document management tools as Summation and Concordance.

After conducting a series of experiments comparing a sample Corsair 64GB SSD with a conventional Hitachi 80GB magnetic hard drive (HDD), the team found a layer cake of data recovery problems caused by the ‘garbage collection’ or purging algorithms used in SSDs to keep them at peak performance.

After examining an SSD for traces of data after it had been quick formatted, the team expected the purging routines to kick in around 30-60 minutes later, a process that must happen on SSDs before new data can be written to those blocks. To their surprise, this happened in only three minutes, after which only 1,064 out of 316,666 evidence files were recoverable from the drive.

Going a stage further, they removed the drive from the PC and connected a ‘write blocker’, a piece of hardware designed to isolate the drive and stop any purging of its contents. Incredibly, after leaving this attached for only 20 minutes, almost 19 percent of its files had been wiped for good, a process the researchers put down the ability of SSDs to initiate certain routines independent of a computer.

For comparison, on the equivalent hard drive all data was recoverable, regardless of the time elapsed, as a forensic examiner would expect

“Even in the absence of computer instructions, a modern solid-state storage device can permanently destroy evidence to a quite remarkable degree, during a short space of time, in a manner that a magnetic hard drive would not,” the team concludes.

The results are concerning on a number of levels, forensic, legal and technical.

Yeah, I’ll say, though I suspect most of the concerns are legal and forensic. Hat tip to Slashdot.  ..bruce..

According to Gleick, the impact of information on human affairs came in three installments: first the history, the thousands of years during which people created and exchanged information without the concept of measuring it; second the theory, first formulated by Shannon; third the flood, in which we now live. The flood began quietly. The event that made the flood plainly visible occurred in 1965, when Gordon Moore stated Moore’s Law. Moore was an electrical engineer, founder of the Intel Corporation, a company that manufactured components for computers and other electronic gadgets. His law said that the price of electronic components would decrease and their numbers would increase by a factor of two every eighteen months. This implied that the price would decrease and the numbers would increase by a factor of a hundred every decade. Moore’s prediction of continued growth has turned out to be astonishingly accurate during the forty-five years since he announced it. In these four and a half decades, the price has decreased and the numbers have increased by a factor of a billion, nine powers of ten. Nine powers of ten are enough to turn a trickle into a flood.

I’ve lived through most of the information age (if we peg it at 1940 or so; I was born in 1953), and I’ve worked in information technology for almost 40 years (since 1974). I remember seeing a second megabyte of core memory that was being added to the IBM 360/65 on campus in the mid-1970s: it was the size of a giant shoebox, and it cost $250,000. Now the cost per megabyte for 1GB RAM SIMMs sold over Amazon is as low as $0.02, and there’s 1,024 megabytes on that little stick. I have something approaching 15 terabytes of hard disk storage on the various computers in our house, and — as per Dyson’s review and Gleick’s book — my biggest challenge is keeping it organized and accessible. There’s a high level of redundancy, with numerous minor (or major) variations; at the same time, I fear that a disk failure will forever deprive me of files that I might someday want or need again, whether that’s tomorrow or ten years from now.

I’ll read the book when it comes out (I have a copy preordered from Amazon); in the meantime, Dyson’s review is well worth reading.

At the same time, classic rotating storage devices — such as platter-based hard disk drives — are slowly being supplemented and in some cases supplanted by solid-state media, that is, storage devices that have no moving parts at all. In particular, solid-state disks (SSDs) look and behave just like classic hard disk drives, but with usually faster response times and much lower chance of hardware failure. In particular, USB drives have become ubiquitous as a mechanism for moving files between unconnected computers, while laptops are starting to offer SSDs as the principal internal storage for both speed and weight improvements.

So far, so good. Except that out of the University of California at San Diego comes research that shows that in many cases, standard file- and disk-erasing techniques leave behind recoverable data when used on SSDs:

At the Non-volatile Systems Laboratory we have designed a procedure to bypass the flash translation layer (FTL) on SSDs and directly access the raw NAND flash chips to audit the success of any given sanitization technique. Our results show that naïvely applying techniques designed for sanitizing hard drives on SSDs, such as overwriting and using built-in secure erase commands is unreliable and sometimes results in all the data remaining intact. Furthermore, our results also show that sanitizing single files on an SSD is much more difficult than on a traditional hard drive. We are working on designing new FTLs that correct these issues and also exploit properties of flash memory to maintain performance while sanitizing the flash drive.

I imaging that word of this will quickly spread to companies that perform computer forensics (including recovery of data for storage devices). In the meantime, many organizations may continue to make us of USB drives and internal SSDs in laptops (and, eventually, desktops), and by so doing may leave themselves open to discovery of data that they thought purged in the course of normal, documented operations.

Earlier today, I received an unsolicited e-mail (with attachments) from a 3rd year law student here in the US who is looking for a job. Since I’m not a law firm, as I think is quite clear from this website, and since the e-mail had a strange salutation (“Esteemed Mr. Webster, Partner:”) and sounded like boilerplate, I wrote this person back as follows:

If you did just a wee bit more research — such as actually look at my website — you’d discover that I’m not a lawyer, nor is my company a law firm. I’m an IT consultant who also does work as as a consulting/testifying expert in lawsuits that involve computer technology. If this is an example of the contacts you’re trying to make at actual law firms, I suspect you won’t have much luck.  You really do need to track down actual live people and speak with them.  Good luck on the job search.  ..bruce..

I was a bit surprised to get the following reply:

Bruce:

Your patronizing and condescending tone is not appreciated.  I’m glad you have your career made and are so far removed from the stress of finding a job as a fledgling attorney.  By your own admission, you state that you are not a lawyer, and have thus never had to look for a job as an attorney; but then you proceed to belittle and criticize my method, and to give advice regarding how to get a job as an attorney.

You’re right: in searching for the “Webster & Associates” that was contained in a list of law firms that was given to me from my Career Services department, I was not informed that the firm does not even have a website.  Thus, my cursory review of your website (I did look at your website), and the subsequent assumption that your firm was theirs, led to an innocent mistake on my part.  Fortunately for me, I had a kind-natured accidental recipient who understood the concept of innocent mistake and politely explained the matter in a nonjudgmental way.

The materials I sent to you were not intended for you and I hereby require you to destroy them and any copies of them as required by federal and state law.  Failure to do so, or the publication or distribution of any part of them, will subject you to liability for violation of those laws, including criminal and civil penalties and damages.  Thank you for your immediate compliance.

I was mostly amused, particularly at how quickly I had gone from “Esteemed Mr. Webster, Partner” to “Bruce” (apparently, contempt breeds familiarity). But because I like lost causes, I took the time to write said Third Year Law Student back:

What I said was well-meant advice. I’ve spent 35 years in another stressful (and, generally speaking, less lucrative and less stable) profession, namely information technology. I’ve done my own job hunting at various times in the past; I’ve also done a lot of interviewing and hiring as I’ve built engineering and consulting teams, so my comments were based on my own experience on both sides of the table. Also, I’ve worked closely with dozens of attorneys (and their law firms) all over the US for the past 10 years, so I probably have a better insight into the people and firms you’re trying to get to hire you than you do.

For starters, your greeting line:

Esteemed Mr. Webster, Partner:

made me wonder right off the bat if this e-mail was spam from India or somewhere else overseas, and second if this was a mass mailing. That’s not how cover letters/e-mails are typically sent to professionals within the US. A simple “Dear Mr. Webster:” would be far more appropriate and effective.

Your subsequent comment that

Webster & Associates is precisely the kind of quality organization where I am confident I can gain excellent experience and contribute to the legal field at my full potential.

told me that you knew nothing significant about my firm. That statement was also apparently made with no knowledge about the real “Webster & Associates”, since as you said, you could find no web site for them. Even if I were a law firm, it shows no actual awareness of the firm itself; it sounds like boilerplate language. Were I in your shoes (and, by the way, I did consider for many years going to law school myself, but decided I was just too old to be a first-year associate), I would try to find something specific to the firm — and preferably to the lawyer to whom I was writing — to put in here: location, practice areas, some major case that the firm and/or the lawyer worked on.

The fact that you would take such umbrage at my relatively mild (if brusque) comments — and then go so far as to write me back a clearly hostile letter, instead of just a simple “Oops, sorry.” or even not replying at all — makes me wonder if you’ve got the thick skin it will take to survive and succeed as a lawyer. As a first year associate, you will be at the bottom of the totem pole; you will be criticized, chastised, and cursed and yelled at for things that are not your fault (as well as those that are); and you’ll be expected do to the impossible on a weekly basis, then harassed when what you do isn’t perfect. (True, you could in turn yell at paralegals and secretaries, but that would be a fatal mistake — they have far more power to damage you and your career than you might realize.)

Oh, and while this aspect is changing, some law firms will still expect you to put in 2000 billable hours that first year; many law firms, like big consulting firms (I was a Director at PricewaterhouseCoopers), still work on the ‘pyramid’ model, and partners depend upon revenue from fully-utilized associates. (The change is not necessarily good news for you — those abandoning the pyramid model for the ‘diamond’ model usually don’t hire many, if any, first-year associates.) So the stress level will be pretty high.

Finally, your demands regarding your e-mail attachments are ill-advised and ill-founded (and frankly just plain impolite). You did clearly send them, unsolicited, to my e-mail address and to my firm (as you understood it to be, having been to my web site), so you can’t claim they weren’t intended for me. There’s no privilege or protective order attached to the documents, so while you can ask me to delete them, you can’t enforce your demand that I do so. (Simple question: under penalty of what? What civil or criminal action could you successfully bring against me for keeping that which you sent to me voluntarily and unsolicited by me in an effort to get me to hire you? The misunderstanding was entirely on your part, not mine.) I occasionally have practicing lawyers — partners, even! — accidentally send me things they didn’t intend to (usually due to e-mail address auto-fill-in); their requests that I delete the e-mail (and any attachments) are always polite and never demanding, since they know they cannot compel me to do so; it was their fault, not mine. And, of course, if you’re threatening me with legal action, I now have every incentive and right to hold onto your e-mails and attachments, since they would be critical evidence in said action.

In short: lighten up. Sheesh.

Finally — and this ties back to the previous few paragraphs – I personally know and correspond with over 100 practicing lawyers, most of whom work for major law firms here in the US (actually, I correspond annually with nearly 200 lawyers, but of those, I’ve probably only actually worked with 100-120). I’m talking about lawyers I know on a first name basis and for whom I’ve done work (sometimes more than once) as a consulting/testifying expert, and who therefore are far more likely to open and read my e-mails than they are to open and read yours.

So now, stop and think: what if, instead of the reply you wrote below, you had said, “Sorry for the misunderstanding — but since you clearly work with lawyers, can you think of any who might be interested in hiring me?” That could have led to a few exchanges between us as to what areas of law interest you the most, and that would have probably led to me either giving you some specific contacts at specific law firms (probably pre-vetted by me) or, better yet, having me forward your e-mail on to those specific contacts. It never, ever pays to burn bridges that you could possibly make use of later, even if it’s not quite clear how you can make use of them.

Best of luck in your job hunt; I know this isn’t the greatest time to be competing for opening slots at law firms, but that merely underscores how important it is that you go about your job hunt in an effective, efficient, and polite manner.  ..bruce..

P.S. Be advised that this e-mail exchange (but not your attachments) may well show up on my website, sans any identification of you, of course — I may be blunt, but I’m not cruel.

And here it is.  ..bruce..

UPDATE 02/04/10: I received a lengthy and humble e-mail from the Third Year Student apologizing for his/her initial response to me and outlining the various stress factors — general and personal — of coming out of law school right now. I very much understand them, and it’s far better that s/he blow up at me now rather than at a prospective employer later.  I’ll also note that the Third Year Student said in that e-mail that having this exchange up on this site “is a good idea…as a warning to others.” Class act, that.

UPDATED: 02/13/10: Due to the increasing amount of spam appearing in the comments, I’m changing the comment system to require registration. If you want to post a comment and have problems doing so, drop me a line.

Roger Sessions has published a white paper, “The IT Complexity Crisis: Danger and Opportunity” (PDF). It’s created a bit of a stir in tech circles, largely because Sessions estimates that “worldwide, we are already losing over USD 500 billion per month on IT failure, and the problem is getting worse” (page 1; emphasis in original). He feels that the consequence is a “coming IT meltdown”, then goes on to offer his own solution, namely designing simpler IT systems.

This naturally intrigued me, since for the last 15 years, I have been writing, consulting, lecturing, and testifying about troubled and failed IT projects. While there are indeed tremendous financial losses due to late and failed IT projects, the figures Sessions gives seem much too large to me, and so I decided to do this critique of his analysis.

Sessions is good enough to provide the basis of his estimates and calculations, including footnotes. But that’s where some of the problems start. For example,  on page 3, Sessions cites (his footnote ’02′) to the US Budget, Fiscal Year 2009, Analytical Perspective (PDF), p. 169, for information on “at-risk” or failed IT projects, specifically:

• “According to the 2009 U.S. Budget [02], the failure rate is increasing at the rate of around 15% per year. If this trend continues, within another five years or so a total IT meltdown may be unavoidable.” (p. 3)
• “According to the 2009 U.S. Budget [02], 66% of all Federal IT dollars are invested in projects that are ‘at risk’. I assume this number is representative of the rest of the world.” (p. 3, in “Calculating the Cost of IT Failure” box)
• A large number of these ['at risk' projects] will eventually fail. I assume the failure of an ‘at risk’ project is between 50% and 80%. For this analysis, I’ll use the average: 65%.”

These three statements run into immediate problems. First, and relatively minor, Sessions gets his page number wrong: he’s citing “page 169″ of the Analytical Perspective document, but there is no discussion whatsoever on page 169 of that document about IT projects. However, page 157 of that document (which happens to be page 169 of the PDF document) does start a section titled “INTEGRATING SERVICES WITH INFORMATION TECHNOLOGY”, so I presume that Sessions made the simple mistake of using the PDF page count rather than the document’s actual page numbering.

Even so, serious problems remain with Sessions’ citations and analysis.

Page 157 of the Analytical Perspective document does not say what Sessions claimed in the two comments above. I have not been able to figure out where Sessions gets his figure for “the failure rate increasing around 15% per year” from the cited US Budget Analytical Perspective document, much less his conclusion that “if this trend continues, within another five years or so a total IT meltdown may be unavoidable.” As far as I can tell, the Analytical Perspective document does not talk about failed IT projects at all, much less the increase in failure rates.

Furthermore, the phrase “the failure rate increasing around 15% per year” is itself ambiguous and may not be that significant. To start with an arbitrary number, assume that 100 projects “fail” in a given year. If “the failure rate [is] increasing around 15% per year”, then that means that 115 projects would fail the next year, and 132 projects would fail the year after that. But unless we know both the actual number of failed IT projects and the total number of IT projects in that same year, Sessions’ figure tells us nothing. If there’s only 150 IT projects total, then the 15% failure rate increase becomes very significant; if there’s 1000 IT projects total, then we’re many years away from Sessions’ threatened “meltdown”.

Sessions also ignores or confuses the failure rate for new projects vs. the systems already deployed. In other words, the failure rate for new systems development says very little about the continued functionality of existing, deployed systems now in use. While there are occasions (most notably Y2k, now a decade behind us) where existing IT systems just won’t function or function properly if they aren’t fixed or replaced, by and large both governments and private concerns have gotten along remarkably well for years or even decades with antiquated systems

As for Sessions’ second statement, there is a table on page 158 that may represent the basis for it:

As can be seen in the FY 2009 column, 66% (535 out of 810) of the FY 2009 “Major IT Investments” are projects that are “Not Well Planned and Managed”. Note that this table does not (as Sessions infers) indicate Federal dollars but rather actual projects; that is, in FY 2009, there are 810 projects listed as “Major IT investments”, of which 535 are designated as “Not Well Planned and Managed”. The previous page appears to indicate that these projects represent $27 billion, which is roughly 38% of the proposed Federal IT budget — not a great figure, but still almost half of the 66% that Sessions claims.

What’s more, supplementary data (PDF) for the FY 2009 Analytical Perspective makes it clear that the US Government’s designation of such projects — which puts them on a “Management Watch List” (WML) — has reduced the risk of such projects during each fiscal year:

Note that in FY 2007 and 2008, the number of IT projects designated as “Not Well Planned and Managed” shrunk significantly during the year (from Q1 to Q4) without a proportional shrinkage of the overall number of major IT projects. In other word, it appears that the government’s efforts to remove such projects from the “Not Well Planned and Managed” category is relatively successful. And the actual US IT budget dollars at risk at the end of each of those fiscal years ($4.2 billion for FY 07, $8.6 billion for FY 08)  is a much smaller percentage (6.5% and 13%, respectively) of the Federal IT budget for each of those years ($64.2 billion for FY 07 (XLS), $66.4 billion for FY 08 (XLS)).

Sessions then states that “I assume this number [66% of all Federal IT dollars being at risk] is representative of the rest of the world.” There are numerous problems with this assumption, starting with the fact that the 66% figure is wrong; in fact, the actual “at risk” (his term, not the US Government’s) percentage of the IT budget at the end of FY 07 and FY 08 were, as noted above, 6.5% and 13%, respectively.

Sessions’ error here is significant, since he goes on in several places (cf. page 4) to cite his use of the % of the total IT budget as being significant, when he’s not talking about the total IT budget at all.

Furthermore, it is unclear whether his phrase “the rest of the world” means all other national governments, or all other entities doing IT project development. It seems to be the latter, though it’s hard to tell from his statements. On the other hand, I have spent years consulting with corporations on troubled projects, and I can tell you that they do not have 66% of their IT budgets devoted to “at risk” projects. In fact, the majority of corporate IT budgets are devoted to maintenance of existing systems, not new and risky projects (cf. here, here, here, and here, as simple examples).

As noted, Sessions then assumes that the failure rate for “at risk” IT projects is 65%, which means that (as he says) “I am calculating that 43% (.65 x .66) of the total IT budget” is devoted to failed projects. At this point, his figures become nonsensical, as they are derived both from misreadings and lack of complete information about the Federal IT budget and projects. To wit:

• The 535 “not well planned and managed” IT projects in the US FY 09 budget only represent 38% of the total IT budget, not 66% as Sessions mistakenly states.
• In the two previous years (FY 07 and FY 08), the number of IT projects labeled as “not well planned and managed” dropped during the course of each year (see the 2nd table above). In FY 07, it dropped from 263 projects in Q1 to just 84 in Q4, which means that 69% were moved off of the “not well planned and managed” list during the year. Likewise, in FY 08, it dropped from 346 projects in Q1 to 134 projects in Q4, a drop of 61%. This directly contradicts Sessions’ assumption of a 65% failure rate for projects in the “not well planned and managed” category.
• The FY ’09 Analytical Perspective says nothing about actual failed projects, as far as I can tell.

Sessions then goes on to make further out-of-his-hat assumptions regarding “direct and indirect costs”. He cites an example of the IRS (an agency long troubled by IT woes) and notes a lost opportunity based on fraudulent tax returns due to the system not being operational. He projects a loss over two years ($1.788 billion), compares it to the cost of the failed modernization ($185 million over a ten-year period), and calculates an indirect costs ratio of 9.6 to 1. He then decides — with no other documentation or analysis whatsoever — that the universal ratio of indirect to direct costs for a failed IT project ranges from 5:1 to 10:1, and uses the “average” of 7.5:1.

There are so many problems here that I scarce know where to start. For starters, the term “average” assumes an even distribution of ratios from 5:1 to 10:1 and does not recognize any ratios lower than 5:1. I’ve seen many failed projects that had much lower ratios of “indirect” to “direct” costs, since the firm simply continued to operate using the existing systems, and the “lost opportunity” for not having the new system in place was relatively small.

More importantly, the IRS gets to collect taxes from the entire US: $2.5 trillion in tax collections each year. Using the IRS as a baseline makes little sense for most other government agencies, and even less sense for most corporations and non-government organizations (NGOs), because most IT systems in most organizations (government or private) do not have the ability to generate such magnitudes of revenue, period.

Indeed, there is a long-standing controversy within IT management circles as to whether a new computer system can be relied upon to provide any significant return on investment (ROI), or whether it exists merely to “keep up with the competition”.

Sessions concludes his section on calculations thusly (p. 5, emphasis his):

Of course, these calculations are estimates. I recommend you don’t get overly focused on the exact amounts. I could be off by ten or twenty percent in either directions. The real point is not the exact numbers, but the magnitude of the numbers and the fact that the numbers are getting worse.

Unfortunately, Sessions is fundamentally wrong in his numerical analysis, and his numbers are off by far more than “ten or twenty percent”. For the Federal Government alone, they are off by almost  a full order of magnitude (10x), due to his critical errors both on the percentage of the Federal IT ’09 budget “at risk” (it’s 38%, not 66%) and the number of “at risk” projects that fail (he says 65%; the US government numbers for FY 07 and 08 show that only 35% of the projects — representing just 6.5% to 13% percent of the Federal IT budget — were still “at risk” at the end of each fiscal year, and it gives no figures that I can find for actual failed IT projects).

Furthermore, his projection of the (erroneous) 66%-of-IT-budget-at-risk figure on the rest of the world is just wrong, especially in corporations and business (which spend vastly more on IT than the US government). In those organizations, maintenance costs dominates, and the percentage of the IT budget devoted to new projects tends to be small (20% or less), with an even smaller fraction of that representing “at risk” projects.

I may comment more on Sessions’ paper, but my conclusion here is that his estimate of $500 billion/month in lost direct and indirect costs due to IT systems failure just does not hold up, in my opinion.  ..bruce..

Hat tip to Joel Miller for the quick update and the copy of the ruling. ..bruce..