CM ROI

I saw a content management ROI case study a few weeks ago that claimed $500,000/yr savings and 15 month payback for implementing this particular CMS in one organization. In this case, the system was used to help call centre staff find and locate content relevant to their calls, and IA was an explicit step in the implementation process.

So what impact did IA have on the bottom line? (Not a rhetorical question unfortunately; the divine sales guy didn't have that much information in the case study.)

The issue, for me and for most people at this seminar, was not whether professional IA makes a difference. The issue is whether there's enough of a difference between lay IA and professional IA to justify an investment in IA services.

And, well, this is something we can test. My (rather obvious) hypothesis is that a site developed with input from professional IAs will have significantly better ease of use, learnability and error recovery compared to a site developed only by lay IAs. And since these differences are quantifiable, they can be turned into numbers we can use in standard ROI formulas.

So the key challenges would be to:

1. define the kinds of services professional IAs provide (content inventories, site maps, controlled vocabularies, etc.)
2. find redesign projects that are limited to mainly professional IA services
3. perform usability tests on before and after versions of that site, with tasks specifically designed to evaluate the ease of use, learnability and error recovery of each version
4. analyze the differences in the two sets of results and convert them into either increased revenue or reduced costs
5. plug those differences, plus the cost of professional IA services, into some standard ROI formula

Then repeat these steps a whole bunch of times and look at the results. Of course, there are always problems: I think this would work better for internal systems where factors like learnability have a direct impact on the organization. And we ought to wonder if step 3 is really necessary. Shouldn't we be able to detect increased revenue or productivity by inspection? (Maybe, in which case step 3 becomes optional.)

And there's the perennial concern that ROI calculations are just smoke and mirrors anyway (although not many people can tell really good smoke and mirrors from the real thing). But that shouldn't stop us from trying to make a compelling business case for IA.

Related articles:

• Finding the ROI in Content Management
• Content Management Server ROI (MS Word)