Roger Sessions has published a white paper, “The IT Complexity Crisis: Danger and Opportunity” (PDF). It’s created a bit of a stir in tech circles, largely because Sessions estimates that “worldwide, we are already losing over USD 500 billion per month on IT failure, and the problem is getting worse” (page 1; emphasis in original). He feels that the consequence is a “coming IT meltdown”, then goes on to offer his own solution, namely designing simpler IT systems.

This naturally intrigued me, since for the last 15 years, I have been writing, consulting, lecturing, and testifying about troubled and failed IT projects. While there are indeed tremendous financial losses due to late and failed IT projects, the figures Sessions gives seem much too large to me, and so I decided to do this critique of his analysis.

Sessions is good enough to provide the basis of his estimates and calculations, including footnotes. But that’s where some of the problems start. For example,  on page 3, Sessions cites (his footnote ‘02′) to the US Budget, Fiscal Year 2009, Analytical Perspective (PDF), p. 169, for information on “at-risk” or failed IT projects, specifically:

• “According to the 2009 U.S. Budget [02], the failure rate is increasing at the rate of around 15% per year. If this trend continues, within another five years or so a total IT meltdown may be unavoidable.” (p. 3)
• “According to the 2009 U.S. Budget [02], 66% of all Federal IT dollars are invested in projects that are ‘at risk’. I assume this number is representative of the rest of the world.” (p. 3, in “Calculating the Cost of IT Failure” box)
• A large number of these ['at risk' projects] will eventually fail. I assume the failure of an ‘at risk’ project is between 50% and 80%. For this analysis, I’ll use the average: 65%.”

These three statements run into immediate problems. First, and relatively minor, Sessions gets his page number wrong: he’s citing “page 169″ of the Analytical Perspective document, but there is no discussion whatsoever on page 169 of that document about IT projects. However, page 157 of that document (which happens to be page 169 of the PDF document) does start a section titled “INTEGRATING SERVICES WITH INFORMATION TECHNOLOGY”, so I presume that Sessions made the simple mistake of using the PDF page count rather than the document’s actual page numbering.

Even so, serious problems remain with Sessions’ citations and analysis.

Page 157 of the Analytical Perspective document does not say what Sessions claimed in the two comments above. I have not been able to figure out where Sessions gets his figure for “the failure rate increasing around 15% per year” from the cited US Budget Analytical Perspective document, much less his conclusion that “if this trend continues, within another five years or so a total IT meltdown may be unavoidable.” As far as I can tell, the Analytical Perspective document does not talk about failed IT projects at all, much less the increase in failure rates.

Furthermore, the phrase “the failure rate increasing around 15% per year” is itself ambiguous and may not be that significant. To start with an arbitrary number, assume that 100 projects “fail” in a given year. If “the failure rate [is] increasing around 15% per year”, then that means that 115 projects would fail the next year, and 132 projects would fail the year after that. But unless we know both the actual number of failed IT projects and the total number of IT projects in that same year, Sessions’ figure tells us nothing. If there’s only 150 IT projects total, then the 15% failure rate increase becomes very significant; if there’s 1000 IT projects total, then we’re many years away from Sessions’ threatened “meltdown”.

Sessions also ignores or confuses the failure rate for new projects vs. the systems already deployed. In other words, the failure rate for new systems development says very little about the continued functionality of existing, deployed systems now in use. While there are occasions (most notably Y2k, now a decade behind us) where existing IT systems just won’t function or function properly if they aren’t fixed or replaced, by and large both governments and private concerns have gotten along remarkably well for years or even decades with antiquated systems

As for Sessions’ second statement, there is a table on page 158 that may represent the basis for it:

As can be seen in the FY 2009 column, 66% (535 out of 810) of the FY 2009 “Major IT Investments” are projects that are “Not Well Planned and Managed”. Note that this table does not (as Sessions infers) indicate Federal dollars but rather actual projects; that is, in FY 2009, there are 810 projects listed as “Major IT investments”, of which 535 are designated as “Not Well Planned and Managed”. The previous page appears to indicate that these projects represent $27 billion, which is roughly 38% of the proposed Federal IT budget — not a great figure, but still almost half of the 66% that Sessions claims.

What’s more, supplementary data (PDF) for the FY 2009 Analytical Perspective makes it clear that the US Government’s designation of such projects — which puts them on a “Management Watch List” (WML) — has reduced the risk of such projects during each fiscal year:

Note that in FY 2007 and 2008, the number of IT projects designated as “Not Well Planned and Managed” shrunk significantly during the year (from Q1 to Q4) without a proportional shrinkage of the overall number of major IT projects. In other word, it appears that the government’s efforts to remove such projects from the “Not Well Planned and Managed” category is relatively successful. And the actual US IT budget dollars at risk at the end of each of those fiscal years ($4.2 billion for FY 07, $8.6 billion for FY 08)  is a much smaller percentage (6.5% and 13%, respectively) of the Federal IT budget for each of those years ($64.2 billion for FY 07 (XLS), $66.4 billion for FY 08 (XLS)).

Sessions then states that “I assume this number [66% of all Federal IT dollars being at risk] is representative of the rest of the world.” There are numerous problems with this assumption, starting with the fact that the 66% figure is wrong; in fact, the actual “at risk” (his term, not the US Government’s) percentage of the IT budget at the end of FY 07 and FY 08 were, as noted above, 6.5% and 13%, respectively.

Sessions’ error here is significant, since he goes on in several places (cf. page 4) to cite his use of the % of the total IT budget as being significant, when he’s not talking about the total IT budget at all.

Furthermore, it is unclear whether his phrase “the rest of the world” means all other national governments, or all other entities doing IT project development. It seems to be the latter, though it’s hard to tell from his statements. On the other hand, I have spent years consulting with corporations on troubled projects, and I can tell you that they do not have 66% of their IT budgets devoted to “at risk” projects. In fact, the majority of corporate IT budgets are devoted to maintenance of existing systems, not new and risky projects (cf. here, here, here, and here, as simple examples).

As noted, Sessions then assumes that the failure rate for “at risk” IT projects is 65%, which means that (as he says) “I am calculating that 43% (.65 x .66) of the total IT budget” is devoted to failed projects. At this point, his figures become nonsensical, as they are derived both from misreadings and lack of complete information about the Federal IT budget and projects. To wit:

• The 535 “not well planned and managed” IT projects in the US FY 09 budget only represent 38% of the total IT budget, not 66% as Sessions mistakenly states.
• In the two previous years (FY 07 and FY 08), the number of IT projects labeled as “not well planned and managed” dropped during the course of each year (see the 2nd table above). In FY 07, it dropped from 263 projects in Q1 to just 84 in Q4, which means that 69% were moved off of the “not well planned and managed” list during the year. Likewise, in FY 08, it dropped from 346 projects in Q1 to 134 projects in Q4, a drop of 61%. This directly contradicts Sessions’ assumption of a 65% failure rate for projects in the “not well planned and managed” category.
• The FY ‘09 Analytical Perspective says nothing about actual failed projects, as far as I can tell.

Sessions then goes on to make further out-of-his-hat assumptions regarding “direct and indirect costs”. He cites an example of the IRS (an agency long troubled by IT woes) and notes a lost opportunity based on fraudulent tax returns due to the system not being operational. He projects a loss over two years ($1.788 billion), compares it to the cost of the failed modernization ($185 million over a ten-year period), and calculates an indirect costs ratio of 9.6 to 1. He then decides — with no other documentation or analysis whatsoever — that the universal ratio of indirect to direct costs for a failed IT project ranges from 5:1 to 10:1, and uses the “average” of 7.5:1.

There are so many problems here that I scarce know where to start. For starters, the term “average” assumes an even distribution of ratios from 5:1 to 10:1 and does not recognize any ratios lower than 5:1. I’ve seen many failed projects that had much lower ratios of “indirect” to “direct” costs, since the firm simply continued to operate using the existing systems, and the “lost opportunity” for not having the new system in place was relatively small.

More importantly, the IRS gets to collect taxes from the entire US: $2.5 trillion in tax collections each year. Using the IRS as a baseline makes little sense for most other government agencies, and even less sense for most corporations and non-government organizations (NGOs), because most IT systems in most organizations (government or private) do not have the ability to generate such magnitudes of revenue, period.

Indeed, there is a long-standing controversy within IT management circles as to whether a new computer system can be relied upon to provide any significant return on investment (ROI), or whether it exists merely to “keep up with the competition”.

Sessions concludes his section on calculations thusly (p. 5, emphasis his):

Of course, these calculations are estimates. I recommend you don’t get overly focused on the exact amounts. I could be off by ten or twenty percent in either directions. The real point is not the exact numbers, but the magnitude of the numbers and the fact that the numbers are getting worse.

Unfortunately, Sessions is fundamentally wrong in his numerical analysis, and his numbers are off by far more than “ten or twenty percent”. For the Federal Government alone, they are off by almost  a full order of magnitude (10x), due to his critical errors both on the percentage of the Federal IT ‘09 budget “at risk” (it’s 38%, not 66%) and the number of “at risk” projects that fail (he says 65%; the US government numbers for FY 07 and 08 show that only 35% of the projects — representing just 6.5% to 13% percent of the Federal IT budget — were still “at risk” at the end of each fiscal year, and it gives no figures that I can find for actual failed IT projects).

Furthermore, his projection of the (erroneous) 66%-of-IT-budget-at-risk figure on the rest of the world is just wrong, especially in corporations and business (which spend vastly more on IT than the US government). In those organizations, maintenance costs dominates, and the percentage of the IT budget devoted to new projects tends to be small (20% or less), with an even smaller fraction of that representing “at risk” projects.

I may comment more on Sessions’ paper, but my conclusion here is that his estimate of $500 billion/month in lost direct and indirect costs due to IT systems failure just does not hold up, in my opinion.  ..bruce..

Categories: managerial

Various industry studies cite the productivity gap between the best and the worst developers. While there is some controversy over the ranges often cited (such as the famous 26:1 figure), anyone who has managed a diverse group of developers won’t argue with the fact that there can be a dramatic difference in productivity among individuals.

What is less obvious is that given a group of developers who are roughly equal in having a sufficient or even superior level of competence and productivity, not all of them will readily adapt to a new language, technology or development approach. Some developers will take to it quickly and do very well; others will have to work harder, but will eventually come up to speed; yet others may never become skilled even though they may not be aware of this fact.

When this happens, all the developers may be working hard, but the relative quality of their work — measured by adherence to best practices for the language/technology in question — can vary dramatically. This may not be apparent for some time, and when it does appear, it can be difficult to correct, from both a technical and organizational viewpoint.

Symptoms

Product architecture that must constantly be revised or refactored. Subsystems that are less stable than others or that are hard to integrate with the rest of the project. Developers consistently complaining about the quality or utility of a given developer’s code.

Consequences

Failure to achieve the expected benefits of the language/technology. Varying quality within the system under development; friction and even serious rifts between team members; project or product failure.

Detection

It’s a big help to have one or more team members who you know are skilled with the language/technology in question. That can be harder to determine than you think, even (or especially) if you think that you’re skilled with the language/technology as well. Process, design and/or code reviews may help to shine light on the problems.

Extraction

This can be one of the hardest issues for a manager, especially if the developer doesn’t recognize the deficiencies. The best solution is probably one of obvious or subtle mentoring: partnering that developer with someone of proven language/technology skills. This approach has two aims: to get the project back on track and to bring the developer’s skills up to speed.

Prevention

Seek the best developers. That may sound obvious, but a lot of companies are more interested in head count than talent and adaptability. The developer should be a self- starter and keenly interested in the language/technology in question, and should also have professional software engineering skills and the discipline to match.

Before the project even starts, spend time to train the developers, and promote the explicit understanding that their language/technology skills will be evaluated on a regular basis. Provide incentives for honesty and alternative assignments, such as building tools, for those whose skills lie elsewhere.

References

Webster, Pitfalls of Object-Oriented Development (1995).

Categories: managerial

That which gets measured gets accomplished or, at least, evaluated. That’s why various software metrics are used as an indication of progress and accomplishment. The best known and easiest to compute is lines of code (LOC), usually measured as thousands of lines of code (KLOC). Luckily, LOC has lost its appeal of the years because it encourages verbosity and constant additions to the code base, even though simplicity and refinement of existing code are ideal. Function points (FPs) are trendier, but they’re also harder to measure and so don’t get used as often.

The point is that these metrics and many others have little bearing on modern software engineering.

Use of the “wrong” metrics may in some circumstances be better than no metrics at all. In one project I worked on, we tracked the change in LOC on a subproject basis week by week; it gave a rough indication of activity in and stability of a subproject. But the wrong metric may turn out to be a bane if misused or misinterpreted by those above you.

Symptoms

Use of metrics that don’t appear to be meaningful or that, worse yet, are misleading. Misuse of metrics as a management club.

Consequences

Time is spent generating information that is of little value and that may give a misleading or erroneous indication of how the project is progressing. More seriously, developers may be encouraged, consciously or unconsciously, to “produce” in ways that run counter to the goals of the project.

Detection

Find out which metrics are being used (if any). Consider their influence and impact; see whether they help predict time or quality of development. Ask other people how they interpret these metrics.

Extraction

Abandon all irrelevant metrics and adopt appropriate ones. Educate all parties involved about what is being done and why. Establish benefits and rewards for those who take the time to use the metrics and whose development efforts improve as a result.

Prevention

From the outset, educate everyone — developers, technical managers, upper management — about what are the appropriate metrics and what they mean. Develop tools to automate metric generation as much as possible. Use the metrics regularly; distribute the results and discuss their meaning.

Be sure to automate these metrics as much as possible; trying to do these by hand will work once or twice but will not give you the constant feedback you need to make them truly useful. Case in point: Do you think anyone would use or continue to use the old “lines of code” metric if they had to count them by hand?

References

Webster, Bruce F. “Lies, Damned Lies, and Project Metrics” (Parts 1, 2, and 3). Baseline (online edition).

Comments posted by Gerald Weinberg at brucefwebster.com.

Categories: managerial

Self delusion and group delusion are all too common in software development projects. Several factors combine to bring this about. One is the natural optimism prevalent among software engineers, particularly when they are not allowed, encouraged, or required to spend sufficient time specifying and designing what they will develop. Another cause is the presumption that no major roadblocks or difficulties will be encountered along the way, either technical or organizational. A third factor is the persistent (if illogical) attitude on the part of upper management that because a project must be completed by a certain date, therefore it can and will be completed by that date, and information to the contrary is not welcome. Mix these and related factors and you have a major disaster in the making.

New technologies and/or methodologies (“TOMs”) have a way of magnifying these tendencies. Engineers become even more optimistic, technical managers see fewer roadblocks, and upper management, having read all the glowing articles about these particular TOMs in the business and trade publications, expects unrealistic results.

Symptoms

Irritation, anger, and disbelief when someone gives unpopular but honest appraisals of how things are going. Long, earnest explanations of why the standard rules of software development don’t apply in this case. And, of course, serious discrepancies between the planned schedule and actual results.

Consequences

Constant slips in the schedule, as expectations are continually reset to adjust to the new version of reality. Projects that ship very late, if at all. Loss of credibility for managers and developers. In the end, you end up looking dishonest, incompetent, or both.

Detection

Take aside the developers, managers, and others involved in the project, one at a time, and ask each person, “Do you think there’s anything we’re fooling ourselves about?” Make notes of the points each person brings up. Make a second pass through, but this time show them the list you’ve compiled and ask them what they agree with, what they disagree with, and what they’d like to add. The number and significance of items on the list should give you a good idea whether you and others are fooling yourselves about how the project is going.

Extraction

Given a significant list of items, you need to reschedule and plan again based on the information collected. This may not be easy or popular; in some cases, it may not even be possible, depending on how upper management reacts. Your choice may then be to either push ahead as best you can or find another job.

Prevention

Use the process described under Detection from the start of project planning and design. You may sacrifice popularity, but you’ll gain credibility—provided, of course, that they don’t reassign (or fire) you and put someone more amenable in charge.

References

[Professional experience]

Categories: managerial

What are the risks in modern software development? Look at the pitfalls listed in this book to start. Kind of makes you want to take up gardening, doesn’t it? On the other hand, being able to identify those risks and then manage them puts you way ahead in the development game and can make you extremely valuable as a technical manager.

Why do we sometimes fail to see and handle risks? There are various reasons. We don’t want to confront others and ask hard questions. We don’t want to expose our own ignorance. We keep hoping things will correct themselves. We don’t want to disappoint or upset those above us. We don’t want to have to fire anyone. We don’t want to appear not to trust others. We face resistance, forceful or subtle, as we try to point out these risks and deal with them.

Risk management is essential in modern software engineering. It’s essential for all the reasons it would be in any software development project. But it’s particularly essential if you are adopting a new technology or methodology (the “TOM”), because:

• the number of IT engineers that you have who are both skilled and experienced in the TOM is likely to be small;
• non-IT personnel, particularly upper management, are likely to have high expectations and mistaken perceptions regarding the TOM.

In short, adoption of a new TOM is likely to increase the risks of the project.

Symptoms

Closed door, closed eyes, closed mind. Constant trickle (or stream) of bad news from developers. Spending more time putting out fires and explaining problems to upper management than actively managing and completing tasks.

Consequences

Slipped schedules, missed milestones, project failures, lost jobs.

Detection

Ask everyone involved with the project what risks they think the project currently faces, with “risks” meaning anything that could cause the project to ship behind schedule, over budget, lacking specified features, in a form not acceptable to customers. If anything comes up you haven’t considered and which you aren’t handling, then you know you’ve fallen into this particular pit.

Extraction

Make an exhaustive list detailing all the previously unidentified and unhandled risks that you gleaned from the questioning in Detection. Distribute the list through the development group and then ask for any additions or corrections (a risk identified by one person may be handled by another). Come up with an assessment of each risk: how probable it is, how it could affect the project, and how it can be managed. Pass this list to upper management. Reset and reschedule the project, if necessary.

Prevention

Risk identification and management should be part of project planning and scheduling from the beginning. The more actively you work as a team to anticipate and handle risks, the fewer problems you’ll encounter. Be aware that this takes political courage, though; those above you don’t always want to hear what the risks are or why the project may not be completed in the time frame demanded.

References

Webster, from Pitfalls of Object-Oriented Development (1995).

Categories: managerial

This is a classic pitfall in software engineering. Typically, insufficient time is allocated for the problem specification, research, design, architecture, and review that should occur before coding and during each development cycle. Likewise, software quality assurance (SQA) is often given little time, money, or people. The whole focus is on coding, and often that is underestimated by assuming that development will be linear and that nothing will go wrong.

Adopting a new technology or methodology (the “TOM”) tends to compound this effect in several ways, some of which are discussed elsewhere. First, unrealistic expectations can creep in. Second, rapid prototyping and feature development can cause a false sense of progress. Third, coding time may well be genuinely reduced by the TOM.

As a result, an expectation can arise that all aspects of the development cycle, not just the coding portion, will be compressed. The first few times you use the TOM, it may require more time up front for design and architecture, and it’s likely to require as much or more testing time. As your development group gains more experience and expertise in the TOM, the entire cycle may start to compress, but that comes with time.

Symptoms

Non-coding tasks taking longer than the time allotted to them. Slowdowns during development due to a need to rethink design. Alpha and beta testing taking much longer than planned.

Consequences

Slipped schedules and missed deadlines. Rude surprises as design must be repeated or testing takes much longer than expected.

Detection

Apply Brooks’ rule of thumb: the time required for a project should break down into one-third for design and prototyping, one-sixth for implementing, and half for testing. If your proportions are radically different, then you may have misjudged the relative costs or at least, mislabeled them; a lot of design and testing gets buried inside implementation.

If you’re far along in your project, keep a close eye on time required for SQA and particularly for testing, which is usually underestimated. Agile methodologies do a better job of addressing testing up front, but SQA remains the poor cousin for most software projects.

Extraction

First, throw out your current schedule and do a hard reset of expectations, particularly among upper management. This is not easy to do, but honesty is always the best policy. Good luck.

Second, set up a development cycle—specify, design, prototype, review, implement, test—with the time allocated to each step in the cycle based roughly on the proportions given above. Recognize that the actual time for each step in each cycle will vary: An early cycle will tend to have more specification and design and less testing; a later cycle will reverse those proportions.

Third, manage through one complete cycle and see how well your estimated costs match reality. Adjust and repeat.

Prevention

First, get some project management software. It doesn’t have to be fancy; it just has to automate the task of adding the estimated times for each task, noting critical paths, and calculating a final date. This is important; attempting to schedule anything except the smallest and simplest project in your head will lead to unpleasant surprises.

Second, use steps two and three under Extraction to set up your schedule and to estimate relative costs. If possible, try this first with a relatively small project and then work up to larger projects. The goal is to be able to estimate both relative and absolute costs within a certain margin of error (say, 10 percent) on a regular basis.

References

Brooks, Frederick P., Jr. The Mythical Man-Month. Reading, MA: Addison-Wesley, 1995.

Webster, from Pitfalls of Object-Oriented Development (1995).

Categories: managerial

The impulse to constantly add new and incremental features to a software program certainly isn’t unique to modern software develoment, or to a particular technology or methodology. It derives largely from three sources. Upper management and marketing want, and sometimes need, those additional features to win a customer, to better position the product against a competitor, or to better justify the expense of an in-house project. Engineers often find it far more fun, more interesting, and more rewarding to add or extend features than to make existing features work completely and perfectly. And customers supply a constant flow of demand for new and improved functionality.

The problem is that many modern technologies and methodologies intensify all these tendencies. With a decent software design and implementation, based on current technology, it can take literally only a few minutes to add or extend features. Frankly, that’s the payoff that lures many engineers to modern technologies in the first place: They get to do really neat stuff, do it quickly, and impress the boss while they’re at it. The boss, having seen how quickly engineers can extend or create functionality, doubles or trebles the list of must-have features. The technical manager, caught in the middle, wonders how to deal with all this.

Symptoms

Engineers focusing on adding new features instead of getting old ones working completely. Upper management passing down long (or even short) lists of additional features when the engineering schedule will be hard-pressed to accommodate the current ones. Customers failing to distinguish between what they’d like and what they’re willing to live with.

Consequences

Features that don’t work as completely or well as they were intended. Missed milestones and schedule slippage.

Detection

Review all existing features with the engineering team and get an honest assessment of where each feature is and what it will take to complete it. If necessary, do a design and code review for each feature. Compare the documented “essential features” list—and the schedule required to complete them—with the current feature set as well as those proposed by engineers and upper management, and find where the differences are coming in.

Extraction

Nothing fancy here: Get the absolute “drop dead” release date for the software and work backward from there, allocating time to design, implement, debug, and test each feature that is not yet completed. Show everyone (especially customers) the gap between the time and resources allocated those required for just the currently planned and requested features. Use whatever process works in your company for performing feature triage: keeping those that are absolutely essential, dropping those that can wait for the next release, and negotiating on the others.

Prevention

Use the process given in Extraction before a line of code is ever written. As features are proposed, collect them in a list, but do not allow any work on them. At intervals, repeat the process: You might have time (or need) to schedule some of the proposed features, but you may well have to drop previously required ones.

As with a prior pitfall on features, this pitfall is so an obvious and well-known that I’m almost embarrassed to include it here. Again, it is still so common that I must.

References

Webster, from Pitfalls of Object-Oriented Development (1995).

Categories: managerial

Let’s start by freely acknowledging that, with rare exceptions, software of any complexity changes between original specification and actual delivery. This is to be expected, and to a certain extent encouraged, when the changes represent a refinement of our understanding of the problem domain and the required solutions. Hence Brooks’ assertion that we must be prepared to “throw one away” — we just don’t know for sure what we’ll need to build and how to build it until we’ve built one. One of the advantages of modern technologies and methodologies is that such changes and adaptations can often be made readily and at lower cost in time and risk than for earlier development approaches.

Let’s also acknowledge that, given the time required to deliver a significant piece of software, customer requirements, technical expectations, and market conditions may change dramatically between the time a project is specified and when it is delivered. As John Donovan noted many years ago, the distance between the leading edge of technology and the trailing edge appears to be constantly shrinking. You may need to redirect or revamp projects as they go along in order to respond to such shifts.

But that’s not what this pitfall is about. This pitfall is about specifications that aren’t nailed down, or that aren’t firmly specified at all. A given feature or aspect of the project is discussed (or maybe it isn’t). As time goes on, the person(s) implementing it have one idea as to what it means or may even have new and improved ideas for it. Those not implementing it may be assuming something quite different from what is being written. Those not involved—including, and especially, upper management—may have quite a different (and erroneous) idea of what the feature will be. And customer expectations may be something else again. When the time comes for feature completion, code integration, project review, or customer acceptance, all hell breaks loose.

Symptoms

Lack of specification documentation. Engineers showing off “new features.” Mismatched subsystems. Constant delays in completing given features.

Consequences

Schedule slips due to ever-changing specification and integration problems. Lack of management, marketing, or customer acceptance of the delivered project.

Detection

First, see whether you can put your hands on a detailed list of features. If no such list exists or if it is incomplete or vague, then you have problems.

Second, see whether you can find a detailed description for each feature, including clear information about what it will not do. Missing or incomplete information is another danger sign.

Third, compare the specification against what is actually being implemented. Variations are yet another indication of trouble.

Fourth, propose a change in specification and see what is the process for making it. The more informal and undocumented the process, the greater the danger of this pitfall.

Extraction

The Detection exercise should give you a good idea about your current state and how to get out of it: Produce the documents described and then use them as a hard spec for the project.

A second (not necessarily exclusive) approach is to write a complete user’s manual for the project and let that be the final specification. This is what we did many years ago at Pages Software: the user documentation, complete with screen shots, was printed a full nine months before the product was released. The president and CEO of Pages gave a copy to every engineer and said, “Here’s the final specification.” It saved all of us from the temptation of making further changes.

Prevention

Establish a process to create, review, modify, and enforce the specification. This is, in effect, the documentation of the design phase of your methodology. This process won’t be easy or popular, but it will pay off significantly in the long run.

This is such an obvious and well-known pitfall that I’m almost embarrassed to include it here. But it is still so common that I must.

References

Webster, from Pitfalls of Object-Oriented Development (1995).

Categories: managerial

If your organization is adopting some new technology or methodology (the “TOM”), it is likely because of wonderful claims about how it will improve your software engineering efforts: faster development time, higher quality, lower complexity, and so on. Leaving aside the likelihood of achieving such claims, these hypothetical benefits still inspire or tempt managers and developers alike to leap feet first into adopting the TOM, using it to tackle large projects with short deadlines.

The result is like crossing a swamp. The ground starts out a bit damp and the undergrowth a bit thick, but you can make good progress with your new TOM machete. As you push ahead, however, you face mud, vines, snakes, water, and alligators in ever- increasing proportions. You may choose to push ahead or to go back, but in either case the effort and cost are greater than if you had used caution before plunging in. And those above you want to know what happened to all the wonderful benefits of this TOM.

Symptoms

A project that doesn’t seem to be making much progress or one that seems ill-defined.

Consequences

Schedule slips and project failure. If initial adoption of the TOM in question (or some key variant thereof), loss of acceptance of that technology and loss of professional credibility.

Detection

Enumerate all that you hope to accomplish with this project. See how soon in the process the coding and prototyping started. Measure current status against the announced schedule and make a realistic assessment of where you are.

Extraction

This can be really tough. Once you’re in the middle of a project, you have a lot of people counting on you to deliver certain solutions by a particular date. You might be hesitant to stop everything and plan the entire development effort again. On the other hand, there is a good chance — in fact, almost a certainty — that if you have stumbled into this pitfall, your efforts to get out of it may not be fully understood or appreciated by those in authority.

Nevertheless, the best solution is to halt development and take the time to scale down the project and train developers; then you can set realistic deadlines.

Prevention

Look again at the pitfall: too much, too soon, too fast. First, start out small when using the new TOM for the first (or second or third!) time. Use well-defined, small-scale pilot projects. Or whittle down your large project into a small one on which you can build in later development cycles.

Second, take time for education, analysis, and design. Make sure that everyone involved understands the concepts, techniques, and — yes — the pitfalls of this particular TOM. Resist the temptation to start prototyping and coding immediately; go through an explicit, detailed analysis phase and take time to go through a design phase.

Third, plan for your initial TOM-based projects to take at least as long as they would if developed using traditional software engineering approaches. You may be pleasantly surprised; you almost certainly won’t have a rude awakening. After you get several projects under your belt, you’ll become more accurate at projecting development cycles.

References

Webster, from Pitfalls of Object-Oriented Development (1995).