What is it about big software projects that makes them fail?
The answer, I think, is that unlike traditional INDUSTRY, software construction has negative economy of scale.
Competent programmers can dash off 100 [correct, not horribly designed] lines in an hour or two. Yet the total output from a large project is typically about 10 lines per programmer per day.
Some of these negative economies come from the amount of time that project members spend communicating with one another. This communication overhead becomes serious as soon as there are too many people on the project to fit at one lunch table at the same time.
At that point, some kind of formal mechanism becomes necessary to ensure that each project member knows enough about what the others are doing to make sure that all the pieces fit together. As the project grows, these mechanisms occupy more of everyone’s time. (There is more to have to understand, too.)
This overhead is the easiest to see, of course.… Because it’s so visible, many people are looking for ways to reduce it. So far, I haven’t seen any approach that works.…
A plethora of complicated systems to manage communication overhead all too often results in well-organised overhead—it still slows the team down and causes just as many mistakes. Some people revel in the neat boxes and charts, even reversing reality and believing that more meetings and communication tools cause productivity.
But I don’t believe that software engineering can be controlled, managed, and organised the way a factory assembly line can. Besides the obvious labour/management tensions instigated by rigid top-down control, factories are premised on creating a large number of identical things—whereas computer programmers produce a relatively small number of different things.
If the things the programmer is making become too similar, s/he will tell the computer how to do it—and once again go back to making dissimilar things.
For this reason, a good software team works more like a machine shop than a factory. Skilled artisans use tools to make better tools to make (someone’s, maybe their) work efficient. Good programmers strive to hand off repetitive work to the computer as much as possible. After all, computers are good at that, and people bore easily.
