Mystery shopping not in pursuit of an overall customer experience objective may be interesting, it may be successful in motivating certain service behaviors, but ultimately will fail in maximizing return on investment.
Consider the following proposition:
“Every time a customer interacts with a brand, the customer learns something about the brand, and based on what they learn, adjust their behavior in either profitable or unprofitable ways.”
These behavioral adjustments could be profitable: positive word of mouth, complain less, less expensive channel use, increased wallet share, loyalty, or purchase intent, etc.. Or…these adjustments could be unprofitable: negative word of mouth, more complaints, decreased wallet share, purchase intent or loyalty, etc.
There is power in this proposition. Understanding it is the key to managing the customer experience in a profitable way. Unlocking this power gives managers a clear objective for the customer experience in terms of what you want the customer to learn from it and react to it. Ultimately, it becomes a guidepost for all aspects of customer experience management – including customer experience measurement.
In designing customer experience measurement tools, ask yourself:
- What is the overall objective of the customer experience?
- How do you want the customer to feel as a result of the experience?
- How do you want the customer to act as a result of the experience?
- Do you want the customer to have increased purchase intent?
- Do you want the customer to have increased return intent?
- Do you want the customer to have increased loyalty?
The answer to the above series of questions will become the guideposts for designing a customer experience which will achieve your objectives.
The answers to the above questions will serve as a basis for evaluating the customer experience against your objectives. In research terms, the answer to this question or questions will become the dependent variable(s) of your customer experience research – the variables influenced or dependent on the specific attributes of the customer experience.
For example, let’s assume your objective of the customer experience is increased return intent. As part of a mystery shopping program, ask a question designed to capture return intent – a question like, “Had this been an actual visit, how did the experience during this shop influence your intent to return for another transaction?” This is the dependent variable.
The next step is to determine the relationship between every service behavior or attribute and the dependent variable (return intent). The strength of this relationship is a measure of the importance of each behavior or attribute in terms of driving return intent. It provides a basis from which to make informed decisions as to which behaviors or attributes deserve more investment in terms of training, incentives, and rewards.
This is what Kinesis calls Key Driver Analysis, an analysis technique designed to identify service behaviors and attributes which are key drivers of your key objectives of the customer experience. In the end, providing an informed basis for which to make decisions about investments in the customer experience.
Customer experience researchers are constantly looking for ways to make their observations relevant, to turn observations into insight. Observing a behavior or service attribute is one thing, linking observations to insight that will maximize return on customer experience investments is another. One way to link customer experience observations to insights that will drive ROI is to explore the influence of customer experience attributes to key business outcomes such as loyalty and wallet share.
The first step is to gather impressions of a broad array of customer experience attributes, such as: accuracy, cycle time, willingness to help, etc. Make this list as long as you reasonably can without making the survey instrument too long.
For additional thoughts on survey length and research design, see the following blog posts:
The next step is to explore the relationship of these service attributes to loyalty and share of wallet.
Two Questions – Lots of Insight
In our experience, two questions: a “would recommend” and primary provider question, yield valuable insight into the relative importance of specific service attributes. Together, these two questions form the foundation of a two-dimensional analytical framework to determine the relative importance of specific service attributes in driving loyalty and wallet share.
Research has determined the business attribute with the highest correlation to profitability is customer loyalty. Customer loyalty lowers sales and acquisition costs per customer by amortizing these costs across a longer lifetime – leading to some extraordinary financial results.
Measuring customer loyalty in the context of a survey is difficult. Surveys best measure attitudes and perceptions. Loyalty is a behavior not an attitude. Survey researchers therefore need to find a proxy measurement to determine customer loyalty. A researcher might measure customer tenure under the assumption that length of relationship predicts loyalty. However, customer tenure is a poor proxy. A customer with a long tenure may leave, or a new customer may be very satisfied and highly loyal.
Likelihood of referral captures a measurement of the customer’s likelihood to refer a brand to a friend, relative or colleague. It stands to reason, if one is going to refer others to a brand, they will remain loyal as well, because customers who are promoters of a brand are putting their reputational risk on the line. This willingness to put their reputational risk on the line is founded on a feeling of loyalty and trust.
Any likelihood of referral question can be used, depending on the specifics of your objectives. Kinesis has had success with both a “yes/no” question, “Would you refer us to a friend, relative or colleague?” and the Net Promoter methodology. The Net Promoter methodology asks for a rating of the likelihood of referral to a friend, relative or colleague on an 11-point (0-10) scale. Customers with a likelihood of 0-6 are labeled “detractors,” those with ratings of 7 and 8 and identified as “passive referrers,” while those who assign a rating of 9 and 10 are labeled “promoters.”
In our experience asking the “yes/no” question: “Would you refer us to a friend, relative or colleague?” produces starker differences in this two-dimensional analysis making it easier to identify which service attributes have a stronger relationship to both loyalty and engagement.
Similar to loyalty, customer engagement or wallet share can lead to some extraordinary financial results. Wallet share is the percentage of what a customer spends with a given brand over a specific period of time.
Also similar to loyalty, measuring engagement or wallet share in a survey is difficult. There are several ways to measure engagement: one methodology is to use some formula such as the Wallet Allocation Rule which uses customer responses to rank brands in the same product category and employs this rank to estimate wallet share, or to use a simple yes/no primary provider question.
Using these loyalty and engagement measures together, we can now cross tabulate the array of service attribute ratings by these two measures. This cross tabulation groups the responses into four segments: 1) Engaged & Loyal, 2) Disengaged yet Loyal, 3) Engaged yet Disloyal, 4) Disengaged & Disloyal. We can now make comparisons of the responses by these four segments to gain insight into how each of these four segments experience their relationship with the brand.
These four segments represent: the ideal, opportunity, recovery and attrition.
Ideal – Engaged Promoters: This is the ideal customer segment. These customers rely on the brand for the majority of their in category purchases and represent lower attrition risk. In short, they are perfectly positioned to provide the financial benefits of customer loyalty. Comparing attribute ratings for customers in this segment to the others will identify both areas of strength, but at the same time, identify attributes which are less important in terms of driving this ideal state, informing future decisions on investment in these attributes.
Opportunity – Disengaged Promoter: This customer segment represents an opportunity. These customers like the brand and are willing to put their reputation at risk for it. However, there is an opportunity for cross-sell to improve share of wallet. Comparing attribute ratings of the opportunity segment to the ideal will identify service attributes with the highest potential for ROI in terms of driving wallet share.
Recovery – Engaged Detractor: This segment represents significant risk. The combination of above average share of wallet, and low commitment to put their reputational risk on the line is flat out dangerous as it puts profitable share of wallet at risk. Comparing attribute ratings of customers in the recovery segment to both the ideal and the opportunity segments will identify the service attributes with the highest potential for ROI in terms of improving loyalty.
Attrition – Disengaged Detractor: This segment represents the greatest risk of attrition. With no willingness to put reputational risk on the line, and little commitment to placing share of wallet with the brand, retention strategies may be too late for them. Additionally, they most likely are unprofitable. Comparing the service attributes of customers in this segment to the others will identify elements of the customer experience which drive attrition and may warrant increased investment, as well as, elements that do not appear to matter very much in terms driving runoff, and may not warrant investment.
By making comparisons across each of these segments, researchers give managers a basis to make informed decisions about which service attributes have the strongest relationship to loyalty and engagement. Thus identifying which behaviors have the highest potential for ROI in terms of driving customer loyalty and engagement. This two-dimensional analysis is one way to turn customer experience observations into insight.
In two earlier posts we discussed 1) including a loyalty proxy as part of your brand perception research and 2) determining the extent to which your desired brand image is reflected in how customers actually perceive the brand.
Now, we expand the research plan to move beyond loyalty and brand perception, and investigate customer engagement, or the extent to which customers are engaged with the brand through share of wallet.
Comparison to Competitors
The first step in measuring customer engagement is capturing top-of-mind comparisons of your brand to competitors. There are many ways to achieve this research objective, perhaps the simplest is to present the respondent with a list of statements regarding the 4-P’s of marketing (product, promotion, place and price) and asking customers to compare your performance relative to your competitors.
The statements you present to customers should be customized around your industry and business objectives, but they may look something like the following:
- Their products and services are competitive
- They are more customer-centric
- They have lower fees
- They have better service
- They offer better technology
- They are more nimble and flexible
- They are more innovative
Similar to the brand perception statements discussed in the previous post, these competitor comparison statements can be used to determine which of these service attributes have the most potential for ROI in terms of driving loyalty, again, by cross tabulating responses to the customer loyalty proxy.
The next step in researching customer engagement is to determine if the customer considers you or another brand their primary provider. This is easily achieved by presenting the customer with a list of providers, including yourself, and asking them which of these the customer consider their primary provider.
Finally, we can tie industry comparisons to primary provider by asking why they consider their selection as a primary provider. This is best accomplished by using the same list of competitor comparison statements above, and asking which of these statements are the reasons they consider their selection to be the primary provider.
Similar to the brand perception statements discussed in the previous post, these competitor comparison statements can be used to determine which of these service attributes have the most potential for ROI in terms of driving loyalty, by cross-tabulating responses to these statements to the loyalty segments.
These days, post-transaction surveys are ubiquitous. Brands large and small take advantage of internet-based survey technology to evaluate the customer experience at almost every touch point. Similarly, loyalty proxy methodologies such as Net Promoter (NPS) are very much in vogue. However, many NPS surveys are fielded in a post-transaction context (potentially exposing the research to sampling bias as a result of only hearing from customers who have recently conducted a transaction), and are not designed in a manner that will give managers appropriate information upon which to take action on the research.
At their core, loyalty proxies are brand perception research – not transactional. We believe it is a best practice to define the sample frame as the entire customer base, as opposed to customers who have recently interacted with the brand. Ultimately, these surveys are image and perception research of the brand across the entire customer base.
Happily, this perception research offers an excellent opportunity to gather customer perceptions of the brand, compare them to your desired brand image, as well as measure engagement or wallet share. An excellent survey instrument to accomplish this is a survey divided into three parts:
- Loyalty Proxy: Consisting of the NPS rating or some other appropriate measure and 1 or 2 follow up questions to explore why the customer gave the NPS rating they did.
- Image perception: consisting of 3 or 4 questions to determine how customers perceive the brand.
- Engagement/Wallet Share: consisting of 3 or 4 questions to determine if the customer considers the brand their primary provider, and to gauge share of wallet of various financial products & services across the brand and its competitors.
This research plan will not only yield an NPS, but it will provide insight into why the customers assigned the NPS they did, evaluate the extent to which the entire customer base’s impressions of the brand matches your desired brand image, as well as identify how the brand is perceived by promoters and detractors. This plan will also yield valuable insight into share of wallet, and how wallet share differs for promoters and detractors.
Such a survey need not be long, the above objectives can be accomplished with 10 – 12 questions and will probably take less than 5 minutes for the customer to complete.
In a subsequent posts, we will explore each of these 3-parts of the survey in more detail:
Calculating ROI on the customer experience typically takes the blind faith approach, where ROI on customer service is considered a given, and the sophisticated approach, where predictive models explain the links between service attributes, customer satisfaction and profitability. Such models can, in fact, be valuable as a means for understanding the associations among different service and profit factors. They can also provide insight into how service attributes interact with each other to influence customer perceptions. A major drawback, however, is that these models tend to have too many moving parts to function as a practical, day-to-day business tool, give the appearance of being far more precise than they actually are, and may be too sophisticated for some audiences.
Sometime managers need a simpler, more intuitive approach to estimating ROI on customer service.
First let me suggest the proposition that every time a company and service provider interact the customer learns something positive or negative and adjusts their behavior, again positive or negative based on what they learn. This is a behavioral approach to managing the customer experience, that by managing customer behaviors in profitable ways, service providers can maximize return on investment in the customer experience.
Using this behavior approach as a model, it possible to construct a simple intuitive ROI estimate of the customer experience.
List customer behaviors with financial implications
The first step in this methodology is to list all customer behaviors that directly drive revenues or costs. Ask yourself, “What specifically, do we want customers to do more or less of?” Don’t include attitudes, such as satisfaction, or feelings such as delight – only include empirically measureable behaviors such as purchase more, purchase more frequently, call for support less often, use more profitable channels, return merchandise less frequently, etc.
Before moving on to the next step, review this list and eliminate any customer behaviors that cannot be influenced through service interactions.
List service attributes that likely influence customer behaviors
Next, work backwards making a second list of specific , measureable service attributes that likely influence desired customer behaviors. This list should only include attributes for which there is a realistic cause and effect relationship between the service attribute and customer behavior. Ask yourself, “What can we (across all service channels) do more of, less of, or do differently to influence customer behaviors?” If it can’t be measured, if it can’t be trained (or programmed) or if it has no likely effect on measureable customer behaviors that affect profit, it should be removed from the list.
Consider how to influence desired customer behaviors (systems, skills, incentives, measurement & rewards)
Now, consider what specific systems, knowledge and skills are required to provide the service that will influence desired customer behaviors. Consider what employee incentives will be most effective in reinforcing the use of those skills and what measurement tools need to be in place to gather the metrics to trigger appropriate rewards.
Link list of customer behaviors to costs and revenues (incremental change)
Finally, link the first list (customer behaviors) to costs and revenues. To do this, calculate the financial impact of an incremental change in each item. For example, what would be the effect on revenue of increasing the average customer purchase by one dollar? What would be the effect on costs if the volume of complaints to call centers were reduced by five percentage points? It quickly becomes clear that even a small change in some customer behaviors can have a substantial financial impact. It also becomes clear which service changes will have the biggest effect.
Thus far you have identified the customer behaviors you want to change, the general influence of each behavior on revenue or cost, and the dollar value of an incremental change in each behavior.
The major element missing from the formula is magnitude. How much change can the company expect to create? Can complaints be reduced by 1%, 5%, 10%? Will average purchase amounts increase by 50 cents? Ten dollars?
Also missing is the interaction among different variables. For example, aggressive up-selling may lead to a 10% increase in the average transaction amount, but it could also lead to a 2% increase in customer turnover, which might counteract the benefit.
The only way to answer these questions is to experiment.
Finally, this method excludes word of mouth customer behavior. In this ad of social media, increasing word of mouth (positive or negative) is an important customer behavior to manage. It’s been excluded from this tool do difficultly empirically measuring its benefits. See the attached post for a description of word of mouth measurement.
What if I told you that after all your efforts with marketing (product, positioning and price), there is a one-in-ten chance the branch representatives will undermine the sale?
Now more than ever, it is critical for banks to establish themselves as the primary provider of financial services, not only for deposit accounts but across a variety of financial products and services. Increasing the average products per customer will require a strategic approach to both product design and marketing. However, at the end of this strategic marketing process, there is the human element, where prospective customers must interact with bank employees to complete the sales process.
As part of our services to our clients, Kinesis tracks purchase intent as a result of in-branch sales presentations. According to our research, 10% of in-branch sales presentations observed by mystery shoppers, result in negative purchase intent.
What do these 10% failed sales presentations look like?
Here are some quotes describing the experience:
“There was no personal attention. The banker did not seem to care if I was there or not. At the teller line, there was only one teller that seemed to care that there were several people waiting. No one moved with a sense of urgency. There was no communication materials provided.”
Here’s another example…
“It was painfully obvious that the banker was lacking basic knowledge of the accounts.”
“Brian did not give the impression that he wanted my business. He did not stand up and shake my hand when I went over to his desk. He very rarely made eye contact. I felt like he was just going through the motions. He did not ask for my name or address me by my name. He told me about checking account products but failed to inquire about my situation or determine what needs I have or might have in the future. He did not wrap up the recommendation by going over everything nor did he ask for my business. He did not thank me for coming in.”
In contrast, here is what the shops with positive intent look like:
“The appearance of the bank was comfortable and very busy in a good way. The customers were getting tended to and the associates had the customers’ best interests in mind. The response time was amazing and I felt as if the associate was sincere about wanting me as a customer, but he was not pushy or demanding about it.”
Now…after all the effort and expense of a strategic cross-sell strategy, which of the above experiences do you want your customers to encounter?
Would it be acceptable to you as a marketer to at the end of a strategic marketing campaign, have 10% of the sales presentations undermine its success?
These are rhetorical questions.
Time and time again, in study after study, we consistently observe that purchase intent is driven by two dimensions of the customer experience: reliability and empathy. Customers want bankers who care about them and their needs and have the ability to satisfy those needs. Specifically, our research suggests the following behaviors are strongly related to purchase intent:
- Greeting/Stand to Greet/Acknowledge Wait
- Interest in Helping/Offer Assistance
- Discuss Benefits/Solutions
- Promised Services Get Done
- Express Appreciation/Gracious
- Personalized Comment (such as, How are you?)
- Listen Attentively/Undivided Attention
As part of any strategic marketing campaign to both bring in new customers as well as increase wallet share of existing customers, it is incumbent on the institution to install appropriate customer experience training, sales and service monitoring, linked with incentives and rewards structures to motivate sales and service behaviors which drive purchase intent.
Increasingly banks must operate in a multi-channel environment. While the changing role of the branch, combined with automated channels such as online and mobile, are getting a lot of attention, there remains a key role for the contact center in delivering an effective customer experience. Central to this key role is designing an effective customer experience, comprised of the right sales and service behaviors – those which influence customer attitudes and behaviors in a profitable way yielding the most return on investment.
To provide direction with respect to what sales and service behaviors will yield the most return on investment, Kinesis conducted a series of mystery shops to identify which sales and service behaviors have the most influence on purchase intent. In addition to observing specific sales and service behaviors, mystery shoppers were also asked to rate how the call would have influenced their purchase intent if they had been a real customer. This purchase intent rating was then used as means of calculating the strength of the relationship between each behavior and purchase intent.
To determine the relationship between these service attributes and purchase intent, the data for these different studies was cross-tabulated by the purchase intent rating and subjected to significance testing. [i]
When the percentage of calls in which purchase intent significantly increased is tested against the percentage of calls where purchase intent significantly decreased, nearly all the sales and service attributes are statistically significant at or above a 95% confidence level.
|Significantly Increased||Significantly Decreased||Test Statistic|
|Explanations easy to understand||99%||45%||9.0|
|When thanked, respond graciously||98%||42%||8.5|
|Friendly demeanor / pleasant voice||100%||60%||8.4|
|Express appreciation for interest / thank you for business||92%||20%||8.3|
|Ask probing questions||79%||10%||6.4|
|Offer further assistance||85%||25%||6.2|
|Speak clearly and avoid bank jargon||98%||68%||5.8|
|Listen attentively to your needs||80%||25%||5.3|
|Mention other bank product||99%||75%||5.3|
|Invite you to visit branch||64%||10%||4.6|
|Explain the value of banking with bank||57%||5%||4.4|
|Offer to mail material / mention website||66%||20%||4.3|
|Ask your name||68%||25%||3.8|
|Ask for your business / close the sale||57%||21%||2.9|
|If no one available to assist you, offered options||100%||0%||2.2|
The differences between the highest and lowest purchase intent for product knowledge and ease to understanding explanations are the most significant, while a professional greeting is the least significant.
Dividing these behaviors into rough quartiles and comparing them side-by-side, reveals some interesting observations:
Explanations easy to understand
When thanked, respond graciously
Friendly demeanor / pleasant voice
Express appreciation for interest / thank you for business
Ask probing questions
Offer further assistance
Speak clearly and avoid bank jargon
When thanked, employee respond graciously
|Listen attentively to your needs
Mention other bank product
Invite you to visit branch
Explain the value of banking with bank
Offer to mail material / mention website
|Ask your name
Ask for your business / close the sale
If no one available to assist you, offered options
The attributes with the most significant differences between high and low purchase intent ratings appear to be those associated with reliability and empathy. It appears mystery shoppers valued such “core” attributes as product knowledge or interest/enthusiasm for the customer. They seem to be less concerned with more peripheral service attributes, such as asking for names, etc. Influencing purchase intent is not as simple as merely using the customer’s name or answering the phone within a short period of time. Rather it is a much more challenging undertaking of being competent in your job and having the customer’s best interests at heart.
[i] Significance testing determines if any differences observed are the result of actual differences in the populations measured rather than the result of normal variation. Without getting into too much detail, significance testing produces a test statistic to determine the probability that differences observed are statistically significant. A test statistic above 1.96 equates to a 95% confidence level, which means there is a 95% chance any differences observed are the result of actual differences in the populations measured rather than normal variation. For all practical purposes a test statistic over 3.1 means there is 100% chance the differences observed are statistically significant (although in reality the probability never reaches 100%). Finally, in interpreting the following analysis, it is important note that test statistics are not lineal. A test statistic of two is not twice as significant as a test statistic of one. The influence on significance decreases as the test statistic increases. However, the test statistic does give us an opportunity to rank the service attributes by their statistical significance.