Marketing Measurement and ROI Analysis – Connecting Aspirations with Realities

I’ve been pretty heads-down over the past few weeks, analyzing the data and results from my graduate research and also working on my upcoming book.  As I’ve dug into the data, there clearly are some self-evident themes emerging around marketers’ opportunities and challenges with adopting strategic marketing systems and technologies (which I will be covering on this blog in more depth over the coming weeks).  One of the clearest themes is the great chasm that exists between aspiration and reality for marketers when it comes to marketing measurement and the analysis of marketing return on investment (ROI).

My research found that these topics are top of mind for marketers, and many state their organizations are already beginning to engage with analytics software.  When asked about tactical/operational objectives for new technology deployments, measurement and ROI analysis are at the top.  This is consistent with a new Lenskold Group / MarketSphere report, released this week.  “Current economic conditions are putting pressures on marketers to better understand their marketing effectiveness as 8 in 10 marketers (79%) report that the need to measure, analyze and report marketing effectiveness is greater in 2009,” according to the press release for the report.

Yet my research found that the same marketers give their organizations low marks on analyzing performance and overwhelmingly comment that their organizations are ‘not aggressive’ when it comes to marketing technology investments.  Aspirations are high, but the reality of investment in systems and technologies to deliver on the aspiration is low.  This also was echoed by Lenskold/MarketSphere, which further commented in their release, “[B]udget pressures are evident with 6 out of 10 (59%) indicating that this higher demand for measuring marketing effectiveness is not budgeted for … .”

The reality is that marketers cannot get enough of systems and technology to tackle measurement and ROI analysis; they have barely scratched the surface.  Far from solved, this is an issue that has only become more important and yet more complicated over time.  Customer channels are exploding in number, and yet marketers are incapable of delivering measurement and ROI analysis that takes this new reality into consideration.  “Buyers are multichannel beings.  Buying cycles are cross-channel,” comments Akin Arikan in his recent book, Multichannel Marketing.  “Yet online and offline marketers still perform their measurements of success in isolation.”

So what are marketers’ aspirations; where is the disconnect; what are their challenges; and what are potential strategies for overcoming these challenges?

To examine this issue, I’ll present some in-progress findings from my own research and compare/contrast this with recent research findings and advice by others who study the topic.

What are marketers’ aspirations for marketing measurement and ROI analysis, and what is the disconnect with reality?

Marketers’ current challenges with measurement and ROI analysis is a story that really requires some digging, because at a surface level, it appears that many marketing organizations have these bases already covered.

As part of my recent graduate research, I deployed a quantitative survey of marketers with 131 complete response records.  Within this group, 64% were corporate-side marketers, 28% were agency-side marketers and 8% were non-agency contractors – a good mix.

Among the corporate-side marketers, I asked them to rate their current level of deployment with a variety of types of key marketing technology platforms.  Cursory inspection indicates a skew in deployment of ‘analytics software’ (see dashed line, below) toward ‘high’/’very-high’ levels, only surpassed by deployment levels for ‘traditional Internet marketing platforms.’  This would seem to be a strong statement that marketers are at an advanced state of deployment, right?

Source: Adam Needles, Propelling Brands (original); click to enlarge graphic

This also seems to hold true when corporate-side marketers are asked what tactical/operational objectives are driving their current marketing technology deployments.  ‘Measure marketing results’ and ‘provide insight into marketing ROI’ bubbles to the top of the list of top objectives.

Source: Adam Needles, Propelling Brands (original); click to enlarge graphic

These responses, above, are consistent with another survey, conducted by the CMO Council, which noted, “More than half of the executives surveyed (55.1%) identified tracking performance, effectiveness and efficiency across the marketing organization as a primary benefit of [marketing automation] solution implementation … .”

This does not speak, though, to the current levels of marketers’ investments in the underlying platforms and infrastructure to tackle measurement and ROI analysis.  In fact, when marketers are asked to rate the overall ‘aggressiveness’ of their organizations’ marketing technology investments, the true reality starts to manifest itself.  Corporate-side marketers overwhelmingly believe their organizations are not aggressive.

Source: Adam Needles, Propelling Brands (original); click to enlarge graphic

What does this have to do with measurement and ROI analysis?  The first two charts identify platforms and objectives where marketers currently have committed resources; however, this tells nothing about the total level of commitment of these resources.  The third chart points to what is certainly an underwhelming investment overall and, presumably, in measurement and ROI analysis.

I dug deeper in qualitative phone interviews with marketers, and the comments were very telling.  Some of the comments by marketers, explaining the reality of their situation, included:

• “Having a closed loop reporting system on leads that come in from multiple sources would be great, but we don’t have this.”
• “To be able to look at referral sources, and the value of those referral sources based on actual customer spending would be great.”
• “Our ability to track or even approximate marketing ROI is the biggest hole we struggle with today.”

The ultimate goal of measurement and ROI analysis should get to the root of whether and to what extent marketing investments are leading to revenue and profit growth for the company.  Yet this is where the chasm between aspiration and reality is the greatest. 

A 2008 Lenskold Group / MarketingProfs special report, “B-to-B Lead Generation:  Marketing ROI & Performance Evaluation study,” showed that only 27% of general marketers and 26% of B2B marketers calculate profitability metrics, including net present value (NPV), for marketing campaigns and/or investments.

What challenges are contributing to the chasm between marketers’ aspirations and their realities for measurement and ROI analysis?

Neither systems nor technology seem to be the culprits when it comes to the chasm between aspiration and reality – i.e., marketers do not lament that these capabilities are not currently available to them.  The larger issues seem to be budget to invest in systems and technologies, as well as organizational dynamics and the relationship between marketing and other functional areas within companies.

> Budgeting for systems and technologies:  The ultimate bottleneck in delivering measurement and ROI analysis clearly seems to be funding.  Despite being asked to deliver more transparency than ever before, marketers are not being provided with sufficient budget to invest in the systems and technologies to deliver continuous measurement and ROI analysis.  This was a key finding of the Lenskold/MarketSphere report, cited above. 

A related finding from my own research is that marketers struggle with making the case for ROI for the underlying systems and technology that might deliver marketing measurement and marketing ROI analysis.  Ironic, but true.  When asked what prevented marketers’ organizations from being more aggressive with investments in marketing technology, some of the responses included:

• “The biggest barrier is cost.”
• “It’s difficult to show the ROI and payback period for implementing such systems.”
• “Not enough foreseeable benefit or ROI to legitimize purchase to CFO for future marketing automation needs.”

> Making ROI tangible; linking measurement to goals and objectives:  There is no question that budget is a key challenge; however, in many ways budget may be the red herring.  Perhaps one reason for seemingly inadequate funding – which may be linked to the comments about not knowing how to prove ROI on investment in these systems and technologies – is marketers not really knowing where to begin.

David Raab talks about this challenge and also helps to tackle this issue in his recent book, The Marketing Performance Measurement Toolkit, in which he argues for a simplification of ROI analysis.  “Many marketing measurement applications simply produce reports for managers to review.  But others feed directly into ongoing business processes, such as media purchases or offer selection,” observes Raab.  “Knowing exactly how participants will process the data gives a concrete understanding of what data they need.”

What is the real objective of your measurement and ROI analysis efforts?  How will the output of such analytics be used, and how will you subsequently tune your marketing investments?  This should guide the approach.  In fact, an outcomes-based posture may even help narrow the scope … and, thus, the cost … of marketing systems and technology investments related to measurement and ROI analysis.

> Making marketing metrics organizational metrics:  Some of the fight over marketing measurement and ROI analysis is really the age-old struggle of the functional company versus the process/customer-oriented company (with the latter being the more enlightened view, I’ll add).  Let’s be clear, marketing metrics are business metrics; they are not separate; and initiatives for measurement and ROI analysis should be an integral part of efforts to build a balanced scorecard for the company.

“A balanced scorecard and strategy map … provide excellent reference points for measurement projects intended to show that marketing is aligned with company strategy,” points out Raab in his book, cited above. 

It’s also worth pointing out that ROI analysis – especially if it is a complete, NPV calculation – is inherently a holistic measurement.  NPV looks at all of the inputs and outcomes of a given marketing activity and discounts them according to the time horizon, and this involves data and inputs from functional areas outside of just marketing. 

Thus, a major barrier to measurement and ROI analysis, as well as an opportunity, is linked to the functional silos of information and activities inside a company.  Tackling this problem holistically – i.e., breaking down the barriers between functional areas within companies and taking a cross-organizational approach to performance metrics – offers benefits beyond merely improving analytics and, in fact, leads to broader transformation of (and synchronization within) the business.

What are strategies for jump-starting measurement and ROI analysis initiatives?

Here are some initial thoughts to get you going …

> Focusing on increasing revenue and profitability versus merely decreasing cost:  Marketing is the combination of activities that ideally increases the overall opportunity for companies … finding new markets, new customers and new opportunities for income and margin.  Yet too much of measurement and ROI analysis seems to focus on taking the cost out of marketing – treating it as though marketing is the problem, not the answer. 

The 2008 Lenskold/MarketingProfs study, cited earlier, speaks to this reality.  Nearly three-quarters of marketers in the study were not focusing on the profitability/NPV of marketing investments, instead they were focusing on cost containment and/or non-financial metrics. 

All the systems and technology in the universe cannot improve measurement and ROI analysis if your underlying impetus and perspective is off.  If a measurement/ROI analysis exercise inside a company is focused merely on cost containment or on non-financial metrics, without any grounding in resultant NPV, there is a bigger issue to tackle.  And getting alignment around the NPV of marketing investment is a crucial first step.

> Securing agreement across the company on marketing measurement and ROI analysis objectives:  Agreeing across the entire management team, not only on the analytical processes, calculations and data to be used, but also on what constitutes a win versus a loss on marketing investments, is critical to successful measurement and ROI analysis.  What NPV is expected from marketing investments, and how does this compare to other potential activities by a company?

Pat LaPointe talked about this in a post on his Marketing Measurement Today blog back in January.  He had spoken with a number of CMOs from major companies and said one of the top four measurement challenges marketers are facing in 2009 is “Lack of clarity – not having a specific definition of what they’re trying to measure, and getting lost in the ambiguity of the process.”  He echoes that it is important to “… define and prioritize the key questions you’re trying to answer BEFORE you set out to measure them.”  And he also advocates these objectives being a company decision, not merely objectives of the marketing organization.

That is why it is critical, as a first step, to secure agreement across the company – most importantly, across the key functional representatives on the management team – before investing in systems and technology to deliver marketing measurements and ROI analysis.

> Baking measurement and ROI analysis into all stages of the marketing process:  Measurement and ROI should not be one-time activities, and they shouldn’t only occur only at the front or the rear end of a marketing program.  They should be continuous; they should constantly be ‘in-process.’  The whole point is to steer the ship and to tune the outcome of marketing activities and investments.

This is why IBM executive Sandy Carter, who published a marketing book profiled in a Q&A on this blog earlier this year, advocates for  a more holistic approach.  “Build a marketing dashboard,” argues Carter in her book, “not a single ROI calculation.”

Also, this stable of analytics should provide visibility into the hierarchy of marketing activities and should analyze not only the entire marketing investment but also the stages of conversion – e.g., going from number of e-mail opens in a campaign to click-throughs on an offer.  These conversion metrics are a critical part of an overall closed-loop analysis system and enable marketers to pinpoint defects in the process. 

> Dedicating systems, technology, staff and other resources to measurement and ROI analysis:  This final recommendation is in fact more than really a recommendation, it is the next step in jump starting your measurement/ROI analysis programs.  Once you have the right mind set, organizational buy-in and thinking about closed-loop analysis, you’re now ready to take your program to the next stage.  And that requires dedicating systems, technology, staff and other resources. 

If you are dedicated to measurement and ROI analysis, it should be a dedicated line item on your overall list of marketing activities – either as a portion of someone’s role or as a dedicated activity, unto itself.  Research shows that organizations that dedicate resources and staff to measurement and ROI analysis ultimately are more successful.

“Firms with marketing operations [teams/functions] tend to have greater adoption of marketing ROI measurements and strengths in ‘using customer analytics to improve marketing effectiveness,’ ‘having data, facts and insight to better guide marketing spending decisions’ and understanding profit drivers to prioritize current budget,’” observed Lenskold/MarketSphere in the press release on their new study (cited above).

In fact, this final topic probably deserves its own post.  So, for now, we’ll say that this topic is to be continued …