It is Time to Review Your Plans

The events of September 11, 2001, continue to reverberate through my thoughts. I have created and advised on the creation of numerous business continuation and disaster recovery plans. On a recent flight home from an East Coast business trip, I wondered about the adequacy of those plans. Without a doubt, the plans never contemplated losses at the levels experienced in New York and Washington.

Let’s all agree that business continuation planning is one of those things you did because the regulators required it. The plan typically sits on the shelf, gathers dust, and is seldom looked at again. Let’s further agree that the information technology folks are generally the point group for developing, testing and executing the plan.

Most of the banks I visit do a good to excellent job of protecting the main data asset of the bank, the deposit, loan and financial accounting data used by the core system. Either bank staff or the outsourcing vendor backs up and stores this information at an off-site facility. This core process recovery plan is usually tested at least annually, and the results are filed in the dusty business continuation plan book for the regulators to see and check off their lists.

In addition to the core system backups, most banks are doing a decent job of protecting servers under control of the technology group. I’m reasonably comfortable that one of these servers could be destroyed and a backup could restore it to an earlier point in time. It is doubtful that it could be restored to the point of failure. The business unit would need to locate and re-enter information since the last backup. By and large this is not the end of the world, but it does illustrate an inadequacy of many bank recovery plans.

To review briefly, most banks’ core system data is safe and can be restored to a point in time, probably as of last night. Generally, most server information can be restored to a point in time, perhaps within the last few days. Are you feeling safe yet? Now let’s look at the data you don’t protect and how it may impact your bank.

First of all, I rarely see strong documentation of how various ancillary systems interface with the well-protected core systems. Today, banks operate with a myriad of front-end origination systems, analytic systems and delivery channels interfaced to the host system. A serious business disruption might force a bank to re-create a technical octopus without any documentation.

My favorite site for enjoying unprotected data is in accounting and finance. I am willing to bet a GonzoBanker T-shirt that the majority of spreadsheets, profitability information, etc., are stored on the individual analysts’ and accountants’ local hard drives. In fact, if you identify the individuals involved in preparing your Board package and disable their computers, your next Board meeting will be short most of its reports.

My next favorite site is the Information Technology group itself. While they may have written the plan, they seldom consider the information they use as corporate asset. The loss of this information will likely push back the completion dates of the projects they are working on. IT also has all the latest neat gadgets and interesting technical gee gaws, and they rarely consider them to be at risk.

Hopefully you are feeling a bit queasy about the thoroughness of your business recovery plans. I hope so, for that is my objective. But the answer is not just ensuring all, and I mean ALL corporate information is being protected. What Sept. 11 taught me is that it is not enough! Every plan I built assumed sufficient knowledgeable staff and some type of facility would be available to recreate the operating environment. I now know that may not be true.

In addition, most corporations have process knowledge bottlenecked in a few individuals with little cross-training or written procedures to augment what’s in these valuable individuals’ heads.

Here are my suggestions on how you and your bank can respond.

• First, recognize that your business recovery process needs to be reviewed.
• Second, formally document the myriad of systems that interface in and out of your core systems and review the business continuation plans for them.
• Third, identify all those important files that are sitting on employees’ hard drives and ensure they are protected.
• Fourth, consider that the staff that knows how your recovery process works may not be available to assist in the recovery. Build and document plans that don’t require prior knowledge of the bank or its processes.
• Finally, identify where you have critical process knowledge concentrated in one or a few individuals and add some more cross-training and procedural documentation to mitigate this risk. Banks are always at risk for these individuals leaving with or without a disaster or business disruption.

When you have completed the review, give it a real try and see how you do. It won’t be pretty the first time, but it will show you where it needs work. Good luck. -cf