An alarming number of software project fail. This article offers the tendency towards over-complication as a major reason why.

Over the years, I've seen quite a few projects with large requirements and specification phases. A number of these projects eventually either had a drastic reduction in scope or were simply never released at all. I shudder to think of how much time and money these projects have wasted. Most computer professionals have had similar experiences.

Of course, this dirty little industry secret is not very secret. The Standish Group's CHAOS project reports in one survey that "only 16.2% (of) software projects (were) completed on-time and on-budget." …(A) staggering 31.1% of projects (were) canceled before they (were) ever completed." The Standish report and other IT pundits offer a number of reasons for this distressing fact of life: lack of user involvement, the relative youth of the software industry, lack of executive support, poor project management, etc.

I'll add another reason why projects fail: the attraction to complexity. Complexity is like kudzu; it will only grow unless you persistently fight it. This article offers some reasons why complexity attracts people involved with all software projects.

If it must be coded, it must be spelled out.
Those responsible for actually delivering a system are accountable for everything the system will do in every detail. After all, the computer can only do what the programmers code it to do.

Most computer professionals have instinctively asked their clients to spell out everything in as much detail as possible. Unfortunately, people don't necessarily do a great job articulating what they want, especially when they are trying to describe the details of their future work on a new system. After all, their future work practice and the new system itself are primarily abstractions.

Like any story, what clients say they will do in their future work becomes more elaborate over time. However, the essential, important work is often simple and will continue to be simple. But during the development process, everyone, including clients, loses track of the fundamental job they wanted the new system to support in the first place.

Those in command of the detail command the conversation.
Typically a group of business clients and computer professionals creates the scope, requirements, and specifications of a system. The group works through a series of interviews, meetings, emails, and revisions of documents. As the project moves from scope to specifications, the need for detail grows and grows. Those team members that have the best ability to handle detail frequently command the most influence in this process. To some degree, this is a good thing. But too often, especially in meetings, the commandants of detail lead the group into more and more complexity. The leaders' strength in handling detail becomes the eventual weakness of the group and the deliverables produced.

Many side trips on the road to confusion.
Requirements and specifications meetings often put a lot of time into satisfying a variety of oddball situations. Since programmers have to code all the possibilities, they expect clients to nail down all the task situations exhaustively. Couple this with the natural tendency for a committee to over-complicate things anyway, and you have a perpetual fountain of complexity. In particular, too much attention spent in trying to handle all task possibilities in full detail leads to an overly complex user interface.

Get everything you can while you can get it.
Few, if any, business clients enjoy unlimited access to computer staff. The demand and relative scarcity of computer professionals continues to become worse over time. Therefore, clients want to get everything possible from a system project.

The situation may even be worse for projects using an internal IT department. Unlike outside consultants, the people working on an internal project may be pulled away when top management feels another need arises with more urgency or importance.

Sign-offs are meant to be broken.
Every project has a battle of "scope creep." To combat it, project management asks for client management to sign-off on documents detailing scope, requirements, and specifications. I have never, ever, seen the case where some significant change was not made later. After sign-off, clients identify something they feel they cannot live without, and heated discussions follow.

Most of the fault goes in the first place to asking clients to articulate their future task behavior in detail. Again, clients don't have more than abstractions to work from. This process is difficult for clients to do, and it is prone to inaccuracies, imprecision, missing information, and over-complication.

The weight test.
The complexity mystique is most apparent when consultants deliver the requirements and specifications documents. The project leader will slowly sway the stack of deliverables in her two hands, assessing the heft of it much as someone would a prize pig. If it's heavy enough, the documents have passed the "weight test."

Though I've exaggerated this point a bit, there is little doubt that the sheer volume of detail that computer professionals handle adds to their perceived value. If months of project work had not produced a stack of documents, both clients and computer professionals would feel something is horribly wrong.

Of course, this more-is-better attitude is yet another reason why projects become too complex.

Conclusion.
There are many reasons why projects fail. People don't typically recognize the attraction of complexity as a big factor in project failure, probably because complexity is so endemic to most software development processes. In my next article, I'll discuss how a project can fight to maintain a successful focus.

*Kudzu is a vine that "literally attacks and covers all objects in its path" The Tangled Story of Kudzu, by Kurt E. Kinbacher. A picture of kudzu can be found on the cover of R.E.M.'s Murmur album.

who we are

what we do

success stories

the business value

how we make a difference

articles and links

Grant Consulting, Inc. located in metro St. Louis, MO, USA	1013 Bradington Court Columbia, IL 62236	info@grantconsulting.com (314) 581-0384

Copyright © 2003 by Grant Consulting, Inc. All Rights Reserved.