Research without call to action may be interesting, but in the end, not very useful.
This is particularly true with customer experience research. It is incumbent on customer experience researchers to give management research tools which will identify clear call to action items –items in which investments will yield the highest return on investment (ROI) in terms of meeting management’s customer experience objectives. This post introduces a simple intuitive mystery shopping analysis technique that identifies the service behaviors with the highest potential for ROI in terms of achieving these objectives.
Mystery shopping gap analysis is a simple three-step analytical technique.
Step 1: Identify the Key Objective of the Customer Experience
The first step is to identify the key objective of the customer experience. Ask yourself, “How do we want the customer to think, feel or act as a result of the customer experience?”
- Do you want the customer to have increased purchase intent?
- Do you want the customer to have increased return intent?
- Do you want the customer to have increased loyalty?
Let’s assume the key objective is increased purchase intent. At the conclusion of the customer experience you want the customer to have increased purchase intent.
Next draft a research question to serve as a dependent variable measuring the customer’s purchase intent. Dependent variables are those which are influenced or dependent on the behaviors measured in the mystery shop.
Step 2: Determine Strength of the Relationship of this Key Customer Experience Objective
After fielding the mystery shop study, and collecting a statistically significant number of shops, the next step is to determine the strength of the relationship between this key customer experience measure (the dependent variable) and each behavior or service attribute measured (independent variable). There are a number of ways to determine the strength of the relationship, perhaps the easiest is a simple cross-tabulation of the results. Cross tabulation groups all the shops with positive purchase intent and all the shops with negative purchase intent together and makes comparisons between the two groups. The greater the difference in the frequency of a given behavior or service attribute between shops with positive purchase intent compared to negative, the stronger the relationship to purchase intent.
The result of this cross-tabulation yields a measure of the importance of each behavior or service attribute. Those with stronger relationships to purchase intent are deemed more important than those with weaker relationships to purchase intent.
Step 3: Plot the Performance of Each Behavior Relative to Its Relationship to the Key Customer Experience Objective
The third and final step in this analysis to plot the importance of each behavior relative to the performance of each behavior together on a 2-dimensional quadrant chart, where one axis is the importance of the behavior and the other is its performance or the frequency with which it is observed.
Interpreting the results of this quadrant analysis is fairly simple. Behaviors with above average importance and below average performance are the “high potential” behaviors. These are the behaviors with the highest potential for return on investment (ROI) in terms of driving purchase intent. These are the behaviors to prioritize investments in training, incentives and rewards. These are the behaviors which will yield the highest ROI.
The rest of the behaviors are prioritized as follows:
Those with the high importance and high performance are the next priority. They are the behaviors to maintain. They are important and employees perform them frequently, so invest to maintain their performance.
Those with low importance are low performance are areas to address if resources are available.
Finally, behaviors or service attributes with low importance yet high performance are in no need of investment. They are performed with a high degree of frequency, but not very important, and will not yield an ROI in terms of driving purchase intent.
Research without call to action may be interesting, but in the end, not very useful.
This simple, intuitive gap analysis technique will provide a clear call to action in terms of identifying service behaviors and attributes which will yield the most ROI in terms of achieving your key objective of the customer experience.
Customer experience researchers are constantly looking for ways to make their observations relevant, to turn observations into insight. Observing a behavior or service attribute is one thing, linking observations to insight that will maximize return on customer experience investments is another. One way to link customer experience observations to insights that will drive ROI is to explore the influence of customer experience attributes to key business outcomes such as loyalty and wallet share.
The first step is to gather impressions of a broad array of customer experience attributes, such as: accuracy, cycle time, willingness to help, etc. Make this list as long as you reasonably can without making the survey instrument too long.
For additional thoughts on survey length and research design, see the following blog posts:
The next step is to explore the relationship of these service attributes to loyalty and share of wallet.
Two Questions – Lots of Insight
In our experience, two questions: a “would recommend” and primary provider question, yield valuable insight into the relative importance of specific service attributes. Together, these two questions form the foundation of a two-dimensional analytical framework to determine the relative importance of specific service attributes in driving loyalty and wallet share.
Research has determined the business attribute with the highest correlation to profitability is customer loyalty. Customer loyalty lowers sales and acquisition costs per customer by amortizing these costs across a longer lifetime – leading to some extraordinary financial results.
Measuring customer loyalty in the context of a survey is difficult. Surveys best measure attitudes and perceptions. Loyalty is a behavior not an attitude. Survey researchers therefore need to find a proxy measurement to determine customer loyalty. A researcher might measure customer tenure under the assumption that length of relationship predicts loyalty. However, customer tenure is a poor proxy. A customer with a long tenure may leave, or a new customer may be very satisfied and highly loyal.
Likelihood of referral captures a measurement of the customer’s likelihood to refer a brand to a friend, relative or colleague. It stands to reason, if one is going to refer others to a brand, they will remain loyal as well, because customers who are promoters of a brand are putting their reputational risk on the line. This willingness to put their reputational risk on the line is founded on a feeling of loyalty and trust.
Any likelihood of referral question can be used, depending on the specifics of your objectives. Kinesis has had success with both a “yes/no” question, “Would you refer us to a friend, relative or colleague?” and the Net Promoter methodology. The Net Promoter methodology asks for a rating of the likelihood of referral to a friend, relative or colleague on an 11-point (0-10) scale. Customers with a likelihood of 0-6 are labeled “detractors,” those with ratings of 7 and 8 and identified as “passive referrers,” while those who assign a rating of 9 and 10 are labeled “promoters.”
In our experience asking the “yes/no” question: “Would you refer us to a friend, relative or colleague?” produces starker differences in this two-dimensional analysis making it easier to identify which service attributes have a stronger relationship to both loyalty and engagement.
Similar to loyalty, customer engagement or wallet share can lead to some extraordinary financial results. Wallet share is the percentage of what a customer spends with a given brand over a specific period of time.
Also similar to loyalty, measuring engagement or wallet share in a survey is difficult. There are several ways to measure engagement: one methodology is to use some formula such as the Wallet Allocation Rule which uses customer responses to rank brands in the same product category and employs this rank to estimate wallet share, or to use a simple yes/no primary provider question.
Using these loyalty and engagement measures together, we can now cross tabulate the array of service attribute ratings by these two measures. This cross tabulation groups the responses into four segments: 1) Engaged & Loyal, 2) Disengaged yet Loyal, 3) Engaged yet Disloyal, 4) Disengaged & Disloyal. We can now make comparisons of the responses by these four segments to gain insight into how each of these four segments experience their relationship with the brand.
These four segments represent: the ideal, opportunity, recovery and attrition.
Ideal – Engaged Promoters: This is the ideal customer segment. These customers rely on the brand for the majority of their in category purchases and represent lower attrition risk. In short, they are perfectly positioned to provide the financial benefits of customer loyalty. Comparing attribute ratings for customers in this segment to the others will identify both areas of strength, but at the same time, identify attributes which are less important in terms of driving this ideal state, informing future decisions on investment in these attributes.
Opportunity – Disengaged Promoter: This customer segment represents an opportunity. These customers like the brand and are willing to put their reputation at risk for it. However, there is an opportunity for cross-sell to improve share of wallet. Comparing attribute ratings of the opportunity segment to the ideal will identify service attributes with the highest potential for ROI in terms of driving wallet share.
Recovery – Engaged Detractor: This segment represents significant risk. The combination of above average share of wallet, and low commitment to put their reputational risk on the line is flat out dangerous as it puts profitable share of wallet at risk. Comparing attribute ratings of customers in the recovery segment to both the ideal and the opportunity segments will identify the service attributes with the highest potential for ROI in terms of improving loyalty.
Attrition – Disengaged Detractor: This segment represents the greatest risk of attrition. With no willingness to put reputational risk on the line, and little commitment to placing share of wallet with the brand, retention strategies may be too late for them. Additionally, they most likely are unprofitable. Comparing the service attributes of customers in this segment to the others will identify elements of the customer experience which drive attrition and may warrant increased investment, as well as, elements that do not appear to matter very much in terms driving runoff, and may not warrant investment.
By making comparisons across each of these segments, researchers give managers a basis to make informed decisions about which service attributes have the strongest relationship to loyalty and engagement. Thus identifying which behaviors have the highest potential for ROI in terms of driving customer loyalty and engagement. This two-dimensional analysis is one way to turn customer experience observations into insight.
These days, post-transaction surveys are ubiquitous. Brands large and small take advantage of internet-based survey technology to evaluate the customer experience at almost every touch point. Similarly, loyalty proxy methodologies such as Net Promoter (NPS) are very much in vogue. However, many NPS surveys are fielded in a post-transaction context (potentially exposing the research to sampling bias as a result of only hearing from customers who have recently conducted a transaction), and are not designed in a manner that will give managers appropriate information upon which to take action on the research.
At their core, loyalty proxies are brand perception research – not transactional. We believe it is a best practice to define the sample frame as the entire customer base, as opposed to customers who have recently interacted with the brand. Ultimately, these surveys are image and perception research of the brand across the entire customer base.
Happily, this perception research offers an excellent opportunity to gather customer perceptions of the brand, compare them to your desired brand image, as well as measure engagement or wallet share. An excellent survey instrument to accomplish this is a survey divided into three parts:
- Loyalty Proxy: Consisting of the NPS rating or some other appropriate measure and 1 or 2 follow up questions to explore why the customer gave the NPS rating they did.
- Image perception: consisting of 3 or 4 questions to determine how customers perceive the brand.
- Engagement/Wallet Share: consisting of 3 or 4 questions to determine if the customer considers the brand their primary provider, and to gauge share of wallet of various financial products & services across the brand and its competitors.
This research plan will not only yield an NPS, but it will provide insight into why the customers assigned the NPS they did, evaluate the extent to which the entire customer base’s impressions of the brand matches your desired brand image, as well as identify how the brand is perceived by promoters and detractors. This plan will also yield valuable insight into share of wallet, and how wallet share differs for promoters and detractors.
Such a survey need not be long, the above objectives can be accomplished with 10 – 12 questions and will probably take less than 5 minutes for the customer to complete.
In a subsequent posts, we will explore each of these 3-parts of the survey in more detail:
In an earlier post we explored how customers experience all aspects of their relationship with a brand through the lens of their emotional state, and observed that all brands must be prepared to meet each customer in their specific emotional state – be they happy, excited, depressed or angry.
Research has determined that, not surprisingly, people are motivated to maintain positive moods, and mitigate negative affective states. When feeling good we tend to make choices that maintain a positive mood. Customers in a positive mood are more loyal, and more likely to interpret information favoring a current brand. Meanwhile, people in negative affective states make choices that have the potential to change or, in particular, improve their moods.
A key to maintaining positive moods is arousal, or more specifically, the management of arousal. Let’s take a look at how arousal management influences consumer choice. Consumers in a positive mood prefer products congruent with their state of arousal. Excited or happy consumers want to stay excited or happy, while relaxed and calm consumers what to stay relaxed and calm. Consumers in a negative mood prefer products with the potential to change their level of arousal.
In considering the role of customer emotions in their relationship to a brand, it is important to understand the implications of customer emotions on design of the customer experience. It is impossible, of course, to plan every customer experience or to ensure that every experience occurs exactly as intended. However, brands can identify and plan for the types of experiences that impart the desired emotional state on the customer. It is useful to group these experiences into three categories of interaction with the customer: Stabilizing, Critical, and Planned.
Stabilizing interactions promote customer retention, particularly in the early stages of the relationship.
New customers are probably in a positive state of valence, with either a high state of arousal (happy/excited) or a negative state of arousal (relaxed/calm). Remember, people are motivated to maintain positive moods, therefore, the objective of these stabilizing interactions is to maintain this positive mood.
The keys to an effective stabilizing strategy and maintaining these positive moods are education, competence and consistency.
New customers are at the highest risk of defection. As customers become more familiar with a brand they adjust their expectations accordingly. It is important that expectations be set appropriately to eliminate conflict with reality. Conflict between expectations and reality early in the customer relationship runs the risk changing the customer’s mood from positive to negative. They are more likely to experience disappointment, and thus more likely to defect.
Education influences expectations, helping customers develop realistic expectations. It goes beyond simply informing customers about the products and services offered by the company. It systematically informs new customers how to use the brand’s services more effectively and efficiently, how to obtain assistance, how to complain, and what to expect as the relationship progresses. In addition to influencing expectations, systematic education leads to greater efficiency in the way customers interact with the company, thus driving down the cost of customer service and support.
Critical interactions are service encounters that lead to memorable customer experiences. While most service is routine, from time to time a situation arises that is out of the ordinary: a complaint, a question, a special request, a chance for an employee to go the extra mile. We call these critical interactions “moments of truth.” The outcomes of moments of truth can be either positive or negative – they are rarely neutral.
Because they are memorable and unusual, moments of truth tend to have a powerful effect on the customer relationship. We often think of moments of truth as instances when the brand has an opportunity to solidify the relationship – earning a loyal customer, or risk the customer’s defection. Positive outcomes lead to positive states of valence (excited, happy, relaxed, calm) with greater wallet share, loyalty, and positive word word-of-mouth endorsements; while negative outcomes generate negative states (anger, frustration, depression); and result in customer defection, diminished share of wallet and unfavorable word-of-mouth.
We are in an era of automated channels. Automated channels are essential for meeting customer expectations and reducing transaction costs, but technical solutions are not, by themselves, able to drive an emotional connection between customers and the brand – particularly in moments of truth. Employees, emotionally intelligence employees, empowered to resolve the issue are critical in driving an emotional connection. In a future post, we will discuss the concept of Emotional Intelligence of frontline employees in handling moments of truth.
An effective customer experience strategy should include systems for recording critical interactions, analyzing trends and patterns, and feeding that information back to the organization. Employees can then be trained to recognize critical opportunities, and empowered to respond to them in such a way that they will lead to positive outcomes and desired customer behaviors.
Planned interactions are intended to increase customer profitability through up-selling and cross-selling. These interactions are frequently triggered by changes in the customers’ purchasing patterns, account usage, financial situation, family profile, etc. CRM analytics combined with Big Data are becoming quite effective at recognizing such opportunities and prompting action from service and sales personnel.
Customer experience managers should have a process to record and analyze the quality of execution of planned interactions with the objective of evaluating the performance of the brand at the customer brand interface – regardless of the channel.
The key to an effective strategy for planned interactions is appropriateness. Triggered requests for additional purchases must be made in the context of the customers’ needs and permission; otherwise the requests will come off as clumsy and annoying. By aligning information about execution quality (cause) and customer impressions (effect), customer experience managers can build a more effective and appropriate approach to planned interactions.
Previously we discussed the concept of “moments of truth” where some experiences in the customer journey have far greater importance than others. These moments of truth represent increased risk and opportunity to leave a lasting emotional impression on the customer; a lasting impression with significant long-term implications for both customer loyalty and wallet share. Perhaps the most common moment of truth is when something has gone wrong, the customer is unhappy or scared and the relationship is at risk. These events could be the result of: service delivery failures (unavailable service, unreasonably slow service, or other core service failures); customer needs and requests (special customer needs or customer preferences); or an adverse outcome (loan denial or loss of investment principal).
Also, in an earlier post we introduced a model to define emotional states with two dimensions:
1) valence (the extent to which the emotional state is positive or negative) and
2) arousal (the extent to which the energy mobilization of the emotional state is experienced on a scale of active to passive or aroused to calm).
Together, valence and arousal can define all human emotions. States of high arousal and positive valence are excited or happy; low arousal and negative valence are bored or depressed; while states of positive valence and low arousal are calm and relaxed, and negative valence and high arousal are angry or frustrated.
Not surprisingly, people are motivated to maintain positive moods, and mitigate negative affective states. People in negative affective states desire choices that have the potential to change or, in particular, improve their moods. For example, researchers have demonstrated a preference for TV shows that held the greatest promise of providing relief from negative affective states. People in a sad mood want to be comforted; anxious people want to feel control and safety.
Beyond solving the problem, the objective in dealing with an upset customer is to help relieve their negative affective state. If they are angry, attempt to calm them; if anxious, provide comfort. Time and time again, our research across many brands reveals that beyond resolving their problem as efficiently as possible, what customers want is empathy and reliability. We want to talk to someone who both understands how we feel and is reliable. They both have a solution to the problem and what they say will get done, gets done.
Strategies in CX Design
Anticipate potential needs for recovery: In designing tools to monitor the customer experience, managers must be aware of potential moments of truth and design tools to monitor these critical points in the customer journey. Some of these tools include: monitoring customer comments from comment cards or online forms to identify instances where the customer is either extremely happy or dissatisfied; monitor social media to identify common causes of moments of truth; survey tracking specifically focusing on the responses from dissatisfied customers; and mystery shopping to test the response to specific problem scenarios.
Decentralize decision making & empower front-line employees: In empowering frontline employees to serve customers, brands should arm them with statements of general principles and values rather than scripted procedures, which undermine empowerment. Reinforce these principles often so in the moment, when they are in a moment of truth with a customer in need, they have an appropriate framework from which to resolve the issue – and bond the customer to the brand.
Train the frontline: Training the frontline to handle problem resolution requires training not just in decision making, but also emotional intelligence. Can emotional intelligence be taught? Yes, but it requires a unique approach of self-discovery. Self-discovery is not a top-down process, however. Managers can foster it through feedback, encouragement to reflect on their own successes and failures, and anecdotes about other employees.
Specifically, tactics frontline employees can employ to handle upset customers include:
• Acknowledging the problem;
• Own the problem;
• Fix the problem;
• Provide assurance; and
• Provide compensation.
Customers experiencing a problem want to change their negative affective state. When dealing with an upset customer it is incumbent on the frontline to help relieve this negative state. Time and time again, in research study after research study, Kinesis finds that the two service attributes that influence customers in a positive way when they encounter a problem are empathy and reliability. Customers want to interact with employees who understand their feelings and are able to resolve the problem.
Calculating ROI on the customer experience typically takes the blind faith approach, where ROI on customer service is considered a given, and the sophisticated approach, where predictive models explain the links between service attributes, customer satisfaction and profitability. Such models can, in fact, be valuable as a means for understanding the associations among different service and profit factors. They can also provide insight into how service attributes interact with each other to influence customer perceptions. A major drawback, however, is that these models tend to have too many moving parts to function as a practical, day-to-day business tool, give the appearance of being far more precise than they actually are, and may be too sophisticated for some audiences.
Sometime managers need a simpler, more intuitive approach to estimating ROI on customer service.
First let me suggest the proposition that every time a company and service provider interact the customer learns something positive or negative and adjusts their behavior, again positive or negative based on what they learn. This is a behavioral approach to managing the customer experience, that by managing customer behaviors in profitable ways, service providers can maximize return on investment in the customer experience.
Using this behavior approach as a model, it possible to construct a simple intuitive ROI estimate of the customer experience.
List customer behaviors with financial implications
The first step in this methodology is to list all customer behaviors that directly drive revenues or costs. Ask yourself, “What specifically, do we want customers to do more or less of?” Don’t include attitudes, such as satisfaction, or feelings such as delight – only include empirically measureable behaviors such as purchase more, purchase more frequently, call for support less often, use more profitable channels, return merchandise less frequently, etc.
Before moving on to the next step, review this list and eliminate any customer behaviors that cannot be influenced through service interactions.
List service attributes that likely influence customer behaviors
Next, work backwards making a second list of specific , measureable service attributes that likely influence desired customer behaviors. This list should only include attributes for which there is a realistic cause and effect relationship between the service attribute and customer behavior. Ask yourself, “What can we (across all service channels) do more of, less of, or do differently to influence customer behaviors?” If it can’t be measured, if it can’t be trained (or programmed) or if it has no likely effect on measureable customer behaviors that affect profit, it should be removed from the list.
Consider how to influence desired customer behaviors (systems, skills, incentives, measurement & rewards)
Now, consider what specific systems, knowledge and skills are required to provide the service that will influence desired customer behaviors. Consider what employee incentives will be most effective in reinforcing the use of those skills and what measurement tools need to be in place to gather the metrics to trigger appropriate rewards.
Link list of customer behaviors to costs and revenues (incremental change)
Finally, link the first list (customer behaviors) to costs and revenues. To do this, calculate the financial impact of an incremental change in each item. For example, what would be the effect on revenue of increasing the average customer purchase by one dollar? What would be the effect on costs if the volume of complaints to call centers were reduced by five percentage points? It quickly becomes clear that even a small change in some customer behaviors can have a substantial financial impact. It also becomes clear which service changes will have the biggest effect.
Thus far you have identified the customer behaviors you want to change, the general influence of each behavior on revenue or cost, and the dollar value of an incremental change in each behavior.
The major element missing from the formula is magnitude. How much change can the company expect to create? Can complaints be reduced by 1%, 5%, 10%? Will average purchase amounts increase by 50 cents? Ten dollars?
Also missing is the interaction among different variables. For example, aggressive up-selling may lead to a 10% increase in the average transaction amount, but it could also lead to a 2% increase in customer turnover, which might counteract the benefit.
The only way to answer these questions is to experiment.
Finally, this method excludes word of mouth customer behavior. In this ad of social media, increasing word of mouth (positive or negative) is an important customer behavior to manage. It’s been excluded from this tool do difficultly empirically measuring its benefits. See the attached post for a description of word of mouth measurement.
Not All Customer Experience Variation is Equal: Use Control Charts to Identify Actual Changes in the Customer Experience
Variability in customer experience scores is common and normal. Be it a survey of customers, mystery shops, social listening or other customer experience measurement, a certain amount of random variation in the data is normal. As a result, managers need a means of interpreting any variation in their customer experience measurement to evaluate if the customer experience is truly changing, or if the variation they are seeing is simply random.
One solution to this need is control charts. Control charts are a statistical tool commonly used in Six Sigma programs to measure variation. They track customer experience measurements within upper and lower quality control limits. When measurements fall outside either limit, the trend indicates an actual change in the customer experience rather than just random variation.
To illustrate this concept, consider the following example of mystery shop results:
In this example the general trend of the mystery shop scores is up, however, from month to month there is a bit of variation. Managers of this customer experience need to know if July was a particularly bad month, conversely, is the improved performance of in October and November something to be excited about. Does it represent a true change in the customer experience?
To answer these questions, there are two more pieces of information we need to know beyond the average mystery shop scores: the sample size or count of shops for each month and the standard deviation in shop scores for each month.
The following table adds these two additional pieces of information into our example:
|Month||Count of Mystery Shops||Average Mystery Shop Scores||Standard Deviation of Mystery Shop Scores|
Now, in order to determine if the variation in shops scores is significant or not, we need to calculate upper and lower quality control limits, where any variation above or below these limits is significant, reflecting an actual change in the customer experience.
The upper and lower quality control limits (UCL and LCL, respectively), at a 95% confidence level, are calculated according to the following formulas:
x = Grand Mean of the score
n = Mean sample size (number of shops)
SD = Mean standard deviation
Applying these equations to the data in the above table, produces the following control chart, where the upper and lower quality control limits are depicted in red.
This control chart tells us that, not only is the general trend of the mystery shop scores positive, and that November’s performance has improved above the upper control limit, but it also reveals that something unusual happened in July, where performance slipped below the lower control limit. Maybe employee turnover caused the decrease, or something external such as a weather event was the cause, but we know with 95% confidence the attributes measured in July were less present relative to the other months. All other variation outside of November or July is not large enough to be considered statistically significant.
So…what this control chart gives managers is a meaningful way to determine if any variation in their customer experience measurement reflects an actual change in the experience as opposed to random variation or chance.
In the next post, we will look to the causes of this variation.