I’ve noticed that a lot of GonzoBankers have a high tolerance for pain.
I’m not talking about physical pain, rather, the mental anguish that comes from listening to countless fintech vendors’ bogus claims about their technologies’ returns on investment (ROI).
***
My latest experience with this form of agony came from a vendor that was briefing me on its big data analytics platform (the ROI on a meaningless buzzword—double the pain). At one point, the vendor put up a slide that said its platform delivers an ROI of 650%. Here’s a snippet of the ensuing conversation:
Me: How was that ROI calculated?
Vendor: It’s based on the revenue that was generated from the marketing campaign that was executed using our platform. The analytics work we did identified consumers who were the most likely to accept an offer for the product in question. Offers were sent out, and we calculated an ROI based on the response rates.
Me: (Somewhat perplexed, but sensing an opportunity to shred the logic to pieces) OK, I might have a few questions here. For starters, wasn’t it really the development of the propensity model that should get the credit for the ROI on that campaign?
Vendor: Absolutely. But it was our analysts who developed the model.
Me: But your analysts could have used any BI platform to create the model. OK, forget that. Why didn’t you factor in the costs of providing the service or product sold into the equation?
Vendor: We didn’t have access to that information.
Me: (Stunned, but pushing ahead nonetheless) Ooooookaaaaay, but why would you base the whole ROI estimate on a single project? What makes you think other firms would achieve the same results?
Vendor: The ROI we’re showing here is for illustrative purposes.
***
It’s painful just recalling that encounter, but these claims run rampant in the industry. The real question that needs to be addressed is: Why do fintech vendors feel compelled to make bogus claims about their technologies’ ROI?
The answer: Because IT buyers want those ROI estimates to justify IT investments.
***
IT project sponsors know that, in order to get funding for an IT investment, the executive team is going to ask, “What’s the ROI?” So project sponsors either rely on vendors’ estimates (incredulously), or make some assumptions about how that will play out in their own organizations.
I don’t really need to tell you that, even if the vendors’ ROI claims were accurate, one firm’s returns are no indication of, or representative of, the ROI that other firms will generate. Different markets, different capabilities, different execution tactics, etc., all combine to make one firm’s ROI completely irrelevant.
But how accurate could the internally-generated estimates really be? Will the firm really be willing to let people go in order to achieve cost savings? Will deployment , integration and execution costs for the new technology really be in line with the vendor’s estimates? How realistic are assumptions regarding 
conversion rates on marketing offers, or adoption rates on new technologies?
Asking “what’s the ROI?” of a technology investment before the actual deployment begins—that is, before all the hard work about what needs to be done, what’s going to get done, and when it’s going to get done given all the other things going on—is no more of a rational question than stopping Dan Rather in the street and asking, “What’s the frequency, Kenneth?”
***
This begs another question: Why do executives ask project sponsors for the ROI of the project? Anyone who thinks it’s because the ROI is needed to justify any IT investment is wrong. Dead wrong.
At a meeting of chief risk and credit officers from 15 of the 50 largest card issuers in the United States, someone asked the group “How many of you have ‘360-degree-view-of-the-customer’ projects currently under way?”
Every one of the execs raised their hands.
I quickly jumped in with this question: “How many of you did ROI projections on those projects?”
Not one person raised a hand. Not a single bank represented in that room felt compelled to estimate what the actual financial benefit—in terms of reduced costs or increased revenue—of spending millions of dollars to integrate customer data files would be.
They made the investments because they believed it was the right thing to do.
***
So, let’s ask again: Why do execs ask IT project sponsors for ROI projections? The answer might be a tough pill to swallow, but “what’s the ROI?” is code for:
Asking project sponsors for ROI projections is a way for execs to say no. “Sorry Jim, but we have other projects with more lucrative ROI projections than yours that we’re going to fund.
Asking project sponsor for ROI projections is a way for execs to say—without having to actually say it—”maybe if you give me ROI projections on this social media monitoring technology it will help me understand what exactly it’s supposed to do, and how it will benefit us.
***
If ROI projections done during the justification phase belong in the fantasy or science fiction—and not the non-fiction—sections of the library, does this mean banks shouldn’t forecast the ROI of IT projects?
No. It simply means that ROI projections shouldn’t be used as justification for a project moving forward from the concept phase. Business strategy, competitive analysis, and lots of other qualitative reasons should drive that determination.
ROI projections—done at the start of a funded project—should be used to gauge a project’s success and to provide management with goals and targets to determine if a project is on track, or if a completed project met it targets. ROI forecasts done at the justification phase are just not reliable enough to be the guide for tracking or performance analysis.
If done at the start of a funded project, however, project sponsors might just be able to reliably answer the question, “What’s the ROI, Kenneth?”
Personal note: I’m excited to be joining Cornerstone Advisors as Director of Research and contributing to the GonzoBanker site. Cornerstone Advisors Senior Director Sam Kilmer and I will be at the 2015 Financial Brand Forum in Las Vegas April 29 to May 1. We’d love to connect with fellow GonzoBankers there. The first five GonzoBankers to find me at the conference and give me a business card (and tell me how much you loved this post) will get a signed copy of my book Smarter Bank.
One point of clarification: You can’t just give me anybody’s business card–it has to be YOUR business card. And you have to work for a bank, credit union, or fintech vendor.
Other side of the coin is that empire builders love to pitch projects that build their personal fiefdom at the expense of the bank as a whole. They like to use squishy qualitative metrics to justify investments in systems that inflates their ego more than their bank’s bottom line. Real ROI models cut through the B.S. and get resources allocated to their highest, best use. That doesn’t mean anyone should rely on a third party’s assumptions. Building out the capability to dispassionately analyze a prospective investment is a critical skill lacking in most banks I work with today.
Ron – Great article. I have been saying this exact same thing for years. As a provider of CRM software, we have been asked to provide an ROI model for years. I’ve declined as there are too many variables and more importantly I have felt that a model provided by “me” the vendor paints the picture I want you to have. The ROI numbers always seem over inflated and unrealistic. I stand behind our software and would rather spend the time counseling my clients on HOW we together can achieve desired results than plugging numbers into a spreadsheet for cost justification. Thanks for providing great insight here.
David —
Thanks. You’re touching on something that far too many execs don’t get: the non-existence of an ROI on infrastructure.
Infrastructure is something that, in and of itself, does not produce an ROI, but instead, enables other things (that hopefully) have an ROI. The ROI of those “other” things can NOT be attributed to the infrastructure investment.
CRM is (at least in part) infrastructure. The marketing modeules of CRM enable marketing campaigns to be executed. Those campaigns have an ROI — but the deployment of the CRM application itself does not.
Contact center CRM may have a more direct line to CRM than the marketing modules, which is why I make the distinction.
As a technology vendor, you’re between a rock and a hard place when it comes to ROI. “Ah feel your pain.” 🙂
Ron
Ron, great article and I am glad to see you join the Gonzo Banking team.
Any big expense needs an assessment to justify its cost. Having a vendor build me a ROI model is ludacris. Having said that, working with a vendor to build a ROI model or an impact assessment is fair game. They know the costs of their system and have data as to how some of their clients have succeeded with it. I’ve worked with vendors to do this. At times I, or others, at companies I worked at had to bully vendors to stop them from adding BS to the models. In banking, I have had to show how a cost for cool technology would be used to increase revenue above and beyond not having it. I think that is fair game. Infrastructure improvements have to be used and monitored over their lifetime or they are just dumb expenses.
Two of my all time favorite experiences were when two different vendors made the claim that increased overdraft behavior alone would nearly cover the cost to implement their technology/product/service. We demanded they remove that from their model in real time in front of us and guess what? No ROI. Also, no sale was made.
Dan mentioned in his comments that some vendors only care about the sale and not the banks bottom line. I’ve experienced that too. Sometimes vendors just do not understand the business model to which they are selling. Not all banks are the same.
The other thing that I find interesting is that why just one ROI number. What is wrong with a range from X to Y. I worked with one risk officer who used to to ask, if this is a complete disaster, what is the downside. He told me he wanted to know if the bankers sponsoring a project ever considered what the impact would be if the project failed. THe response to his question gave him insight.
I look forward to your upcoming commentary here.
ROI is more often than not a bogus calculation…even after the justification process. The calculation assumes that the treatment (new technology, method, whatever) is being applied in a vacuum, and that any results can be: 1) attributed solely to the treatment; 2) repeatable and predictable; and 3) objectively and quantitatively measured. If you pay 20% less for a new paper shredder to do the same amount of work, that’s one thing. But to pretend that most ROI figures deliver anything more than directionality is disingenuous. The causes and effects are often too many and too poorly linked. To David’s point, the honest way of doing things for vendors is to give examples of outcomes clients have experienced, and help the client model out how those results may translate in a particular case. Best case. Worst case. Features. Benefits. Models assume perfect asymmetry in operational translations, implementation efficacy, customer/membership sensibilities, and environmental factors from FI to FI. They’re useful…but merely estimates. If Ron and I buy the same Marketing Analytics text book, I’ll experience a much higher ROI than he will. He’s already an expert.
Obviously, I meant “symmetry,” not “asymmetry.” The ROI on my high school typing class is very low.
Matt — Completely agree. Couple of points though:
1) I wrote this because I wanted to uncover what I believe is the root of the problem: senior execs on the buyer side who pretend like the ROI estimate is what’s driving the investment decision, and
2) Your point about the textbook highlights the point I was trying to raise with David. A textbook–IN AND OF ITSELF–produces NO ROI. It’s only what we do with it that produces an ROI. That is, the textbook is INFRASTRUCTURE. And so, if I am more of a marketing analytics expert than you, I’m probably MORE likely to generate a higher ROI than you, because I presumably know how to leverage that infrastructure.
I actually see this all the time with marketing automation software. Simply deploying the tools does NOT make a bank or credit union better digital marketers.