DOH! My Project is Late!

“The performance of a system depends more on how its parts interact than on how well they work independent of one another.”
–James Martin, cybercorp: the new business revolution

“My project is late.” Does this sound familiar? It has been reported that four out of five technology projects undertaken by U.S. businesses are not satisfactorily completed. Fully 80 percent are either late, over budget, did not meet project objectives or completely failed. Why this is true has been examined by numerous writers and pundits with all of the usual rationale:

• Sufficient staff were not assigned or were not appropriately trained
• Project scope creep occurred without sufficient planning for the additional work
• Technology was not appropriate
• Yada, yada, yada

Computer software development tools have improved significantly in the last few decades. Magazines, newspapers, television and the Web continue to inform us that technology is not only better, it is easier to use. PCs have proliferated so much that banks now routinely report more PCs than employees. Vendors, developers and designers write white papers extolling the virtues of the latest tools, methodologies, environments and other gee gaws while proclaiming that development is now 10 to 100 times more efficient. Wow, development, enhancement projects can now take 1/100 th the amount of time!

Project failure due to technology continues to bedevil the industry. Let’s dig a little deeper and one of the reasons for project failure will become clear. First, we need to go back to the good old days of the glass house computing facility with dumb terminals on users’ desks.

Back in the ‘70s when I was a Systems Engineer (read programmer/analyst) working for Electronic Data Systems (EDS), my peers and I worked on a variety of technical projects for our clients. (I must profess that my picture displayed on the GonzoBanker team page was taken at this time.) The tool set used was very simple, the COBOL programming language. Programs were designed and implemented on Big Blue computers, the venerable IBM 360 family.

Our programs were typically a few hundred to several thousand lines (individual statements) of COBOL. Every aspect of the program was contained in this single program. Our new program resided on a single computer under the command of a single operating system. Communication to a user’s terminal was very simple since everything beyond our COBOL program was hardware.

This technical world was very simple, it consisted of the mainframe computer, its operating system, the COBOL program and any hardware physically connected to the computer. When a problem occurred we were always 99.999 percent assured that the source was the program we had written.

Now, technology has taken many giant leaps forward. PCs are on every desktop, we have come to know and love Bill Gates and his fine products, ease of use is on every technologist’s lips, and unit processing costs continue to follow Moore’s law. Phew, all of this and we still can’t seem to get our technical projects completed.

In the 1970s world all the functions were contained in a single program that operated on a single computer. In today’s world each discrete function may operate on separate computers connected in networks. A function as simple as requesting the balance of a checking account could conceivably pass through dozens of intelligent devices (read computers or computer like devices). 

Here is an example:

When a teller, using a state-of-the-art teller system, requests a customer’s DDA balance, the message may pass through the following intelligent devices:

• Teller is using a PC, an intelligent device with a complex operating system
• Network switch, a computer like device with a small operating system that will switch the message
• Network router, an intelligent device that will route the message to the appropriate address to process the request
• Network router at the end address, functions as above
• Network switch at the end address, functions as above
• Web server, a computer running a complex operating system that is running a complex software product called a
• Web server. Web servers are often configured to be load balanced and fault tolerant. This involves running multiple physical servers with complex software to orchestrate and balance load and to fail over to a running device in case of a failure
• Data request made to a data base server for additional customer information, message traverses the network to the desired address
• Database server, a computer running a complex operating system that is running a complex software product retrieves the requested information and passes it back through the network to the requestor
• Request made of the host computer to obtain the customer balance, message is routed through the network to the host computer
• Host computer receives the message and sends a message to its database server to retrieve the balance information
• Database server, a computer running a complex operating system that is running a complex software product, retrieves the requested balance information and passes it back through the network to the requestor
• Web server collects the various pieces of information and passes it to another complex piece of software that will build the screen to be displayed at the user’s computer, completed screen is passed back thorough the network to the Web server
• Web server takes the completed screen and passes it back through the network to the teller PC

You are probably thinking I picked a very complicated situation to illustrate the case. Actually, this example illustrates just how complex a seemingly simple request can be. Remember your request to simply show all the customer’s relations on a single screen? Take this example and multiply it by 10 or even 20 to understand what could occur.

So, let’s once again go back to the ’70s and consider what happens when an error is detected. Our teller notices a problem, I.T. is contacted and the responsible programmer is notified. Where does the programmer look for the problem? It’s easy, in the one program that is used to provide teller functionality.

Now come back to the present and have the same teller report a problem. First of all, who do you contact? Is it a network problem, maybe database, maybe presentation, perhaps a router made a mistake? Finding a problem is compounded by the number of places a problem can occur. In my simple example there are at least 13 places to look. In most I.T. shops, these 13 places could involve four or five functional units and five or six different third party vendors.

The new technical tools are nice and they provide an efficiency boost for a portion of the environment. Technical progress has taken my single program and broken up the functional parts and dispersed them to multiple systems. Each system and its related tools are powerful and provide the ability to do wondrous things on today’s computers. Unfortunately, it has also created complexity.

It is complexity that has imperiled projects. James Martin observed that “hand-coded applications can be so difficult to change that attempts to make a seemingly trivial modification set off a chain reaction of unexpected problems.”

Perhaps future technology will help manage the interaction of the complex pieces. But for today, we will have to live with the fact that these wonderful tools can be a real pain to get right.
-caf