The (Hidden) Dangers of Functional Specialization

Like so many things in modern life, software development has become an exercise in absolute functional specialization. That is a single-minded goal of increasing levels of specialization. Economic specialization, whereby an individual worker has a single function, is at the heart of today's global capitalist system. This phenomenon is described in Adam Smith's seminal book The Wealth of Nations.

Economic specialization encourages each contributor to focus on a single goal or very limited set of goals. The payoff is efficiency. Each of us becomes an expert in a single endeavor, enabling a "group of highly optimized individuals" to team together. The proverbial whole is greater than the sum of its parts. There are, of course, darker aspects to this specialization fixation, many of which affect the individual contributors themselves (monotony, lack of motivation, reduced artistic expression, dependency upon others, etc), but it is difficult to argue with the theoretical gains in efficiency of such a system.

Those same efficiencies and "economies of scale" that are a result of specialization must transfer to software development, right?

It has long been argued, in software development circles, that two of the foundational premises upon which absolute functional specialization is based are bogus. The first is that there is some sort of median productivity against which to measure each individual. The second is that by "adding more bodies" we will always reduce the effort of each individual. Each of these assertions has been largely discredited in works such as the Mythical Man Month. This book claims, instead, that the range of productivity amongst individuals is so vast, depending upon expertise and experience, that it is meaningless to try to establish placeholder estimates. Secondly, through a series of case studies at IBM, it shows that the expected "reduction of effort" by continuing to add resources to a team not only fails to materialize but actually has the opposite effect. In other words, there is an optimal size for a team...and it is not infinity! Although written way back in the 1970s, I believe that this book remains highly relevant in today's Web 2.0 world.

As with any engineering project, multiple disciplines must be mastered to ensure quality. For software development, those are typically defined as:

1. Analysis
2. Architecture/Design
3. Development
4. Testing
5. Maintenance
   

Conventional absolute functional specialization wisdom would have us believe that we should employ sub-teams of experts within each of these disciplines. It would also suggest that we decompose those sub-teams further. Perhaps we would create a "Black Box Testing Team" or a "User Experience Design Team"? Further, we might be tempted to create a "Back-End Development Team" or a "Java Development Team", perhaps even before we have defined the architecture? These tendencies toward absolute functional specialization, although logical in the abstract, reveal a subtle difference between creating software and creating something tangible. There is only one "raw material" used by each and every one of the software development disciplines: "words" (either written or verbal).

Think about the raw materials needed to build a house. Here are just a few examples:

• Lumber
• Brick
• Concrete
• Sheet metal
• Stucco
• Paint
  

Given the array of raw materials needed to build a house, it is little wonder that such a project results in teams of sub-contractors (foundation pourers, framers, carpenters, electricians, plumbers, etc). However, software development, in my opinion, is not nearly so complicated as we have made it...and there is just one raw material! Many of the aforementioned software development disciplines are the result of artificial distinctions. Why would someone who can develop software be unable to test it? Is it unreasonable to expect the business analyst to implement his "words" in today's software languages? Why should there be separate "Test Specifications" or "Test Plans" when each is really only a different lens through which to view the Analyst's requirements?

Since the publication of the Mythical Man Month, other factors have emerged (some simply as a result of the maturation of software development industry). These new factors further discredit the notion of absolute functional specialization:
- Democratization/technology mainstreaming: a much broader section of the population is exposed to and comfortable with technology
- Increased demand for software: including "Internet time" releases
- System fragmentation: open-source and other factors have led to a much wider set of tools, technologies and languages that contribute to each "software system solution"

In conclusion, I believe that there is such a thing as "functional over-specialization" in software development.

A better approach, I argue, is to empower a single team to perform all of the necessary disciplines. It is also important to stay current with the disciplines best-practices, allowing you to limit the toolset up-front. (Although common-sense, the "limited toolset" restriction is oftentimes loosened as a compromise that is felt necessary because a sub-team, for example, "only does testing".) Finally, it is critical to enable efficiencies not by defining further functional specialization but by instead enforcing use of a team-wide content management system.