[The second in a series of posts on Readings in Software Engineering. Previous post: The Psychology of Computer Programming, Gerald M. Weinberg (1971/1998)]
The Mythical Man-Month: Essays on Software Engineering, Frederick P. Brooks, Jr. Addison-Wesley Publishing Company, Reading, MA, 1975. Softbound, 195 pages. Personal acquisition date: unknown. Original edition out of print.
The Mythical Man-Month: Essays on Software Engineering (Anniversary Edition), Frederick P. Brooks, Jr. Addison-Wesley, Boston, MA, 1995. Softbound, 322 pages. Personal acquisition date: unknown (I’ve bought and given away several copies over the years). Available in Kindle edition as well.
Seminal work; highest recommendation.
(All quotes and citations are from the 1995 Anniversary edition, which contains the slightly-edited text and original page numbering of Brooks’s original book, followed by four new chapters: two additional essays, and two retrospectives on the original edition.)
The Mythical Man-Month is probably the single best-known work in software engineering, as well it should be: it is a classic, in every sense of the word.
First of all, it is (for a thin volume) a remarkably comprehensive look at the practice of professional software engineering. Brooks’s essays cover a wide ranges of topics, with a particular focus on where things are likely to go wrong and what must be done to make things go right. For years, whenever I thought I had conceived some new issue or principle in software development, I would go back and look through Brooks and discover he had already noted it.
Second, it is highly accessible. Brooks’s style is clear, concise, and very readable. It is a book that anyone in a given organization should be able to read and understand. Furthermore, these are true essays: wonderfully written and downright lyric in places. One of my favorite passages is at the end of his discussion as to the ‘Joys of the Craft [of Programming]‘:
Finally, there is the delight of working in such a tractable medium. The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures. (As we shall see later, this very tractability has its own problems.)
Yet the program construct, unlike the poet’s words, is real in the sense that it moves and works, producing visible outputs separate from the construct itself. It prints results, draws pictures, produces sounds, moves arms. The magic of myth and legend has come true in our time. One types the correct incantation on a keyboard, and a display screen comes to life, showing things that never were nor could be.” (pp. 7-8)
Finally, it remains highly relevant for our day, perhaps sadly so. Much of my professional work deals with troubled or failed IT projects, and my analysis of their struggles — whether in vivo or post mortem — usually reveal the same core problems that Brooks warned us of nearly 40 years ago. It seems that in the software industry, we not only insist on re-inventing the wheel, we insist on re-inventing the flat tire. As I said in my look at The Psychology of Computer Programming, billions of dollars wasted in failed IT projects could have been saved if those involved had read and heeded Brooks and Weinberg.
While Brooks is, in fact, mostly focusing on the development of what he calls programming systems products — in effect, writing commercial operating systems and related utilities — most of what he has to say applies to any large scale software development effort. Here, then, are some selected quotes from Brooks (all emphasis in the original):
The Woes of the [Programming] Craft. . . .
First, one must perform perfectly. The computer resembles the magic of legend in this respect, too. If one character, on pause, of the incantation is not strictly in proper form, the magic doesn’t work. . . .
Second, other people set one’s objectives, provide one’s resources, and furnish one’s information. One rarely controls the circumstances of his work, or even its goal. . . .
The next woe is that designing grand concepts is fun; finding nitty little bugs is just work. . . .
Next, one finds that debugging has a linear convergence, or worse, where one somehow expects a quadratic sort of approach to the end. So testing drags on and on, the last difficult bugs taking more time to find than the first. . . .
The last woe, and sometimes the last straw, is that the product over which one has labored so long appears to be obsolete upon (or before) completion. (pp. 8-9, “The Tar Pit”)
The second fallacious thought mode is expressed in the very unit of effort used in estimating and scheduling: the man-month. Cost does indeed vary as the product of the number of men and the number of months. Progress does not. Hence the man-month as a unit for measuring the size of a job is a dangerous and deceptive myth. It implies that that men and months are interchangeable.
Men and months are interchangeable commodities only when a task can be partitioned among many workers with no communications among them. This is true of reaping wheat or picking cotton; it is not even approximately true of system programming. (p. 16, “The Mythical Man-Month”)
Oversimplifying outrageously, we state Brooks’s Law: Adding manpower to a late project makes it later. (p. 25, “The Mythical Man-Month”)
This, then, is the problem with the small, sharp team concept: it is too slow for really big systems. (p. 31, The Surgical Team)
I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas. . . .
Conceptual integrity in turn dictates that the design must proceed from one mind, or from a very small number of agreeing resonate minds. (pp, 42 & 44, “Aristocracy, Democracy, and System Design”)
An architect’s first work is apt to be spare and clean. He knows he doesn’t know what he’s doing, so he does it carefully and with great restraint.
As he designs the first work, frill after frill and embellishment after embellishment occur to him. These get stored away to be used “next time.” Sooner or later, the first system is finished, and the architect, with firm confidence and a demonstrated mastery of that class of systems, is ready to build a second system.
This second is the most dangerous system a man ever designs. . . . The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one. (p. 55, “The Second-System Effect”)
The manual, or written specification, is a necessary tool, though not a sufficient one. The manual is the external specification of the product. It describes and prescribes every detail of what the user sees. As such, it is the chief product of the architect.
Round and round goes its preparation cycle, as feedback from users and implementers show where the design is awkward to use or build. For the sake of implementers it is important that the changes be quantized — that there be dated versions appearing on the schedule.
The manual must not only describe everything the user does see, including all interfaces; it must also refrain from describing what the user does not see. That is the implementer’s business, and there his design freedom must be unconstrained. The architect must always be prepared to show an implementation for any feature he describes, but he must not attempt to dictate the implementation. (p. 62, “Passing the Word”)
If there are n workers on a project, there are (n2-n)/2 interfaces across which there may be communication, and there are potentially almost 2n teams within which communication much occur. The purpose of organization is to reduce the amount of communication and coordination necessary; hence organization is a radical attack on the communication problems treated above. (pp. 78-79, “Why Did The Tower of Babel Fail?”)
[Charles Portman] found his programming teams missing schedules by about one-half — each job was taking approximately twice as long as estimated. The estimates were very careful, done by experienced teams estimating man-hours for several hundred subtasks on a PERT chart. When the slippage pattern occurred, he asked them to keep careful daily logs of time usage. These showed that the estimating error could be entirely accounted for by the fact that his teams were only realizing 50 percent of the working week as actual programming and debugging time. Machine downtime, higher-priority short unrelated jobs, meetings, paperwork, company business, sickness, personal time, etc., accounted for the rest. In short, the estimates made an unrealistic assumption about the number of technical hours per man-year.” (pp 89-90, “Calling the Shot”)
Beyond craftsmanship lies invention, and it is here that lean, spare, fast programs are born. Almost always these are the result of strategic breakthrough rather than tactical cleverness. Sometimes the strategic breakthrough will be a new algorithm . . . Much more often, strategic breakthrough will come from redoing the representation of your data or table. This is where the heart of a program lies. Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious. (p. 102, “Ten Pounds in a Five-Pound Sack”)
Why Have Formal Documents?
First, the writing the decisions down is essential. Only when one writes do the gaps appear and the inconsistencies protrude. The act of writing turns out to require hundreds of mini-decisions, and it is the existence of these that distinguishes clear, exact policies from fuzzy ones.
Second, the documents will communicate the decisions to others. The manager will be continually amazed that policies he took for common knowledge are totally unknown by some member of his team. Since his fundamental job is to keep everybody going in the same direction, his chief daily task will be communication, not decision-making, and his documents will immensely lighten his load.
Finally, a manager’s documents give him a data base and a checklist. By reviewing them periodically, he sees where he is, and he sees what changes of emphasis or shifts in direction are needed. (p. 111, “The Documentary Hypothesis”)
The management question, therefore, is not whether to build a pilot system and throw it away. You will do that. The only question is whether to plan in advance to build a throwaway, or to promise to deliver the throwaway to customers. Seen this way, the answer is much clearer. Delivering that throwaway to customers buys time, but it does so only at the cost of agony for the user, distraction for the builders while they do the redesign, and a bad reputation for the product that the best redesign will find hard to live down.
Hence, plan to throw one away; you will, anyhow. (p. 116, “Plan to Throw One Away”)
The manager of a project, then, needs to establish a philosophy and set aside resources for the building of common [development] tools. At the same time, he must recognize the need for specialized tools, and not begrudge his working teams their own tool-building. This temptation is insidious. One feels that if all those scattered tool builders were gathered in to augment the common tool team, greater efficiency would result. But it is not so. (p. 128, “Sharp Tools”)
Testing the Specification. Long before any code exists, the specification must be handed to an outside testing group to be scrutinized for completeness and clarity. As [V. A.] Vyssotsky [of Bell Labs] says, the developers themselves cannot do this: “They won’t tell you they don’t understand it; they will happily invent their way through the gaps and obscurities.” (p. 142, “The Whole and the Parts”)
When one hears of disastrous schedule slippage in a project, he imagines that a series of major calamities must have befallen it. Usually, however, the disaster is due to termites, not tornadoes, and the schedule has slipped imperceptibly but inexorably. Indeed, major calamities are easier to handle; one responds with major force, radical reorganization, the invention of new approaches. The whole team rises to the occasion.
But the day-by-day slippage is harder to recognize, harder to prevent, harder to make up. Yesterday, a key man was sick, and a meeting couldn’t be held. Today the machines are all down, because lightning struck the building’s power transformer. Tomorrow the disk routines won’t start testing, because the first disk is a week late from the factory. Snow, jury duty, family problems, emergency meetings with customers, executive audits — the list goes on and on. Each one only postpones some activity by a half-day or a day. And the schedule slips, one day at a time. (p. 154, “Hatching a Catastrophe”)
The flow chart is a most thoroughly oversold piece of program documentation. Many programs don’t need flow charts at all; few programs need more than a one-page flow chart. (p. 167, “The Other Face”)
Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any — no inventions that will do for software productivity, reliability, and simplicity what electronics, transistors, and large-scale integration did for computer hardware. . . .
I believe that the hard part of building software to be the specification, design, and testing of the conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared to the conceptual errors in most systems.
If this is true, building software will always be hard. There is inherently no silver bullet. (pp. 181-182, “No Silver Bullet”)
The biggest mistake in the “Build one to throw away” concept is that it implicitly assumes the classical sequential or waterfall model of software construction.” (p. 265, “The Mythical Man-Month after 20 Years”)
“I dismissed Parnas’s concept [of information hiding and encapsulation] as a “recipe for disaster” in Chapter 7. Parnas was right, and I was wrong. I am now convinced that information hiding, today often embodied in object-oriented programming, is the only way of raising the level of software design.” (p. 272, “The Mythical Man-Month after 20 Years”)
The “outrageously simplified” statement of Brooks’s Law is made more useful by these careful treatments [by other researchers] of the proper qualifications. On balance, I stand by the bald statement as the best zeroth-order approximation of truth, a rule of thumb to warn managers against blindly making the instinctive fix to a late project. (p. 275, “The Mythical Man-Month after 20 Years”)
The tar pit of software engineering will continue to be sticky for a long time to come. One can expect the human race to continue attempting systems just within or just beyond our reach; and software systems are perhaps the most intricate of man’s handiworks. This complex craft will demand our continual development of the discipline, our learning to compose in larger units, our best use of new tools, our best adaptation of proven engineering management methods, liberal application of common sense, and a God-given humility to recognize our fallibility and limitations. (p. 289, “The Mythical Man-Month after 20 Years”)
As with Weinberg, some of Brooks’s discussions are dated technically…or, at least, appear to be. One entire essay, “Ten Pounds in a Five-Pound Sack”, is devoted to dealing with memory management and tradeoffs between memory usage, functionality, and speed. That would seem irrelevant in today’s world of multi-gigabyte RAM, much less having virtual memory management standard in just about every operating system. And yet we have watched the emergence of ‘bloatware’ over the past few decades, where software keeps loading slower, running slower, and using more system resources, even as memory gets cheaper and processors get faster (or divide into multiple cores). What used to be craft is now institutionalized sloppiness.
If you are involved in software engineering and IT project management, there are many, many books you should read and be familiar with. But if you need to start somewhere, start here, and read this book. Read it a few times, in fact. The project — and the job — you save may be your own. ..bruce..
TABLE OF CONTENTS (adapted from the 1995 Anniversary Edition)
Chapter 1: The Tar Pit
Chapter 2: The Mythical Man-Month
Chapter 3: The Surgical Team
Chapter 4: Aristocracy, Democracy, and System Design
Chapter 5: The Second-System Effect
Chapter 6: Passing the Word
Chapter 7: Why Did The Tower of Babel Fail?
Chapter 8: Calling the Shot
Chapter 9: Ten Pounds in a Five-Pound Sack
Chapter 10: The Documentary Hypothesis
Chapter 11: Plan to Throw One Away
Chapter 12: Sharp Tools
Chapter 13: The Whole and the Parts
Chapter 14: Hatching a Catastrophe
Chapter 15: The Other Face
(Added to the Anniversary Edition)
Chapter 16: No Silver Bullet — Essence and Accident
Chapter 17: “No Silver Bullet” Refired
Chapter 18: Propositions of The Mythical Man-Month: True or False?
Chapter 19: The Mythical Man-Month after 20 Years
About the Author: bfwebsterWebster is Principal and Founder at at Bruce F. Webster & Associates LLC. He works with organizations to help them with troubled or failed information technology (IT) projects. He has also worked in several dozen legal cases as a consultant and as a testifying expert, both in the United States and Japan. He can be reached at 720.895.1405 or at firstname.lastname@example.org.
Sites That Link to this Post
- RISE: Peopleware: Productive Projects and Teams (DeMarco & Lister, 1987) : Webster & Associates LLC | August 19, 2013
- The Wetware Crisis: the Dead Sea effect : Bruce F. Webster | September 12, 2013
- Remember Conway’s Law : Bruce F. Webster | October 8, 2013
- Obamacare, IT, and magic thinkiing : And Still I Persist… | October 21, 2013
- Obamacare and the Project of Doom : And Still I Persist… | October 23, 2013