Can We Measure the ROI of IT? Should We?

The latest issue of CIO Insight features a lead article called, "Most Companies Struggle to Measure the Value of IT," and headlines:  "No right way has emerged to measure IT value, and the most common measures fare the worst."

To hear them tell it, we are truly in the wilderness on this one, folks.  For starters, one clear finding that emerges from their study is that there is no one "right way" to measure the value of IT—and the companies that use the most popular metrics are also the worst at measuring value.

There's enough blame to go around, starting with executives on the business side:  In "four out of five" companies, they report, different executives want to see different metrics, "forcing IT to provide this potpourri."  But the very existence—and continued tolerance of—a "potpourri" testifies to the deep intellectual confusion surrounding this topic.  Consider that the single most popular way of getting one's arms around the ROI of IT, "time to payback," is used at 49% of firms who believe they "accurately capture the value of our IT investments"—but also at 73% of firms who believe their metrics do not accurately capture value.  Or that another perennial favorite, [(savings + additional revenues) - cost], is used at 38% of firms who believe their numbers are good but at 69% of firms who distrust their numbers.

If there's an ironic silver lining to this, it may be that an amazing four out of ten companies don't try to measure the value of IT at all, including two out of ten with revenues north of $1-billion/year.

This may not last, however, as a strong majority of CIO's report that CEO's and/or CFO's are looking for "new and better" ways of demonstrating IT value, and that the pressure to quantify the dollar value of IT's intangible benefits has increased in  recent years.   

Something fundamental is amiss here, and maybe the only good news is that CIO's and CEO's alike are skeptical of business-value metrics.  Across the board, roughly two-thirds of companies say that "it's difficult to calculate the ROI on IT;" 52% of IT executives believe business executives are skeptical of efforts to measure the business value of it, and among business executives themselves an indistinguishable 50% deem themselves "skeptical." 

But doesn't IT demonstrably improve productivity?  Didn't Our Esteemed Former Fed Chairman Himself testify to that effect in the late 1990's, attributing the economy's unprecedented ability to grow with low inflation and historically low unemployment to the beneficent productivity improvements of the technology revolution?  Or consider this more recent data point:

"In the first quarter of 2006, productivity in the U.S. grew 3.9 percent, according to the Bureau of Labor Statistics, considerably higher than historic levels. According to Federal Reserve Chairman Ben Bernanke, among others, information technology is a major reason."

Doesn't, in fact, common sense just tell us that IT improves productivity?  How did we ever get anything done in the days before the Internet?  Doesn't the drastic shrinkage in the ranks of secretaries give us visible proof, as we walk down the halls of our firms, that lawyers can and are getting more done with less?  Assuming your senses aren't lying, who can rationally question IT's contribution?

I'd like to suggest an entirely different approach to this debate, which appears to be stalemated. 

The brilliant advances in IT during our lifetimes—and shockingly, blindingly brilliant they have been—have, from the perspective of firms and organizations, constituted something of a technology "arms race," where equipping one's workforce with the latest is merely the cost of doing battle with one's competitors, who are simultaneously doing exactly the same thing.  Productivity increases, to be sure, but no competitive advantage is gained, and any claim (outlandish or conservative, it hardly matters) about the magic pixie dust of IT is rejected because it doesn't feel that progress is being made.

Nicholas Carr, the writer and former Executive Editor of The Harvard Businesss Review, famously stated this case in his 2004 book, "Does IT Matter?"   Essentially, the argumetn of that book is this:

So, yes, you are more productive.  But just as the BlackBerry raised client expectations surrounding responsiveness to hitherto unimaginable heights, you may not feel more productive:  You just may have learned to type your own letters when your secretarial support was downsized.

Can we, then, measure the ROI of IT?  Particularly in our world of intangible services, where client satisfaction is the metric of all metrics, I would simply say it's the wrong question.  Because there is no question IT helps us serve our clients.  And if that's true, I for one think the debate is over.