Pitfall: Using the wrong metrics (or none at all)

[From Pitfalls of Modern Software Engineering by Bruce F. Webster (forthcoming)]

Categories: managerial

That which gets measured gets accomplished or, at least, evaluated. That’s why various software metrics are used as an indication of progress and accomplishment. The best known and easiest to compute is lines of code (LOC), usually measured as thousands of lines of code (KLOC). Luckily, LOC has lost its appeal of the years because it encourages verbosity and constant additions to the code base, even though simplicity and refinement of existing code are ideal. Function points (FPs) are trendier, but they’re also harder to measure and so don’t get used as often.

The point is that these metrics and many others have little bearing on modern software engineering.

Use of the “wrong” metrics may in some circumstances be better than no metrics at all. In one project I worked on, we tracked the change in LOC on a subproject basis week by week; it gave a rough indication of activity in and stability of a subproject. But the wrong metric may turn out to be a bane if misused or misinterpreted by those above you.

Symptoms

Use of metrics that don’t appear to be meaningful or that, worse yet, are misleading. Misuse of metrics as a management club.

Consequences

Time is spent generating information that is of little value and that may give a misleading or erroneous indication of how the project is progressing. More seriously, developers may be encouraged, consciously or unconsciously, to “produce” in ways that run counter to the goals of the project.

Detection

Find out which metrics are being used (if any). Consider their influence and impact; see whether they help predict time or quality of development. Ask other people how they interpret these metrics.

Extraction

Abandon all irrelevant metrics and adopt appropriate ones. Educate all parties involved about what is being done and why. Establish benefits and rewards for those who take the time to use the metrics and whose development efforts improve as a result.

Prevention

From the outset, educate everyone — developers, technical managers, upper management — about what are the appropriate metrics and what they mean. Develop tools to automate metric generation as much as possible. Use the metrics regularly; distribute the results and discuss their meaning.

Be sure to automate these metrics as much as possible; trying to do these by hand will work once or twice but will not give you the constant feedback you need to make them truly useful. Case in point: Do you think anyone would use or continue to use the old “lines of code” metric if they had to count them by hand?

References

Webster, Bruce F. “Lies, Damned Lies, and Project Metrics” (Parts 1, 2, and 3). Baseline (online edition).

Comments posted by Gerald Weinberg at brucefwebster.com.

About the Author: bfwebster

Webster is Principal and Founder at at Bruce F. Webster & Associates, as well as an Adjunct Professor for the BYU Computer Science Department. He works with organizations to help them with troubled or failed information technology (IT) projects. He has also worked in several dozen legal cases as a consultant and as a testifying expert, both in the United States and Japan. He can be reached at 303.502.4141 or at bwebster@bfwa.com.