Research without call to action may be interesting, but in the end, not very useful.
This is particularly true with customer experience research. It is incumbent on customer experience researchers to give management research tools which will identify clear call to action items –items in which investments will yield the highest return on investment (ROI) in terms of meeting management’s customer experience objectives. This post introduces a simple intuitive mystery shopping analysis technique that identifies the service behaviors with the highest potential for ROI in terms of achieving these objectives.
Mystery shopping gap analysis is a simple three-step analytical technique.
Step 1: Identify the Key Objective of the Customer Experience
The first step is to identify the key objective of the customer experience. Ask yourself, “How do we want the customer to think, feel or act as a result of the customer experience?”
- Do you want the customer to have increased purchase intent?
- Do you want the customer to have increased return intent?
- Do you want the customer to have increased loyalty?
Let’s assume the key objective is increased purchase intent. At the conclusion of the customer experience you want the customer to have increased purchase intent.
Next draft a research question to serve as a dependent variable measuring the customer’s purchase intent. Dependent variables are those which are influenced or dependent on the behaviors measured in the mystery shop.
Step 2: Determine Strength of the Relationship of this Key Customer Experience Objective
After fielding the mystery shop study, and collecting a statistically significant number of shops, the next step is to determine the strength of the relationship between this key customer experience measure (the dependent variable) and each behavior or service attribute measured (independent variable). There are a number of ways to determine the strength of the relationship, perhaps the easiest is a simple cross-tabulation of the results. Cross tabulation groups all the shops with positive purchase intent and all the shops with negative purchase intent together and makes comparisons between the two groups. The greater the difference in the frequency of a given behavior or service attribute between shops with positive purchase intent compared to negative, the stronger the relationship to purchase intent.
The result of this cross-tabulation yields a measure of the importance of each behavior or service attribute. Those with stronger relationships to purchase intent are deemed more important than those with weaker relationships to purchase intent.
Step 3: Plot the Performance of Each Behavior Relative to Its Relationship to the Key Customer Experience Objective
The third and final step in this analysis to plot the importance of each behavior relative to the performance of each behavior together on a 2-dimensional quadrant chart, where one axis is the importance of the behavior and the other is its performance or the frequency with which it is observed.
Interpreting the results of this quadrant analysis is fairly simple. Behaviors with above average importance and below average performance are the “high potential” behaviors. These are the behaviors with the highest potential for return on investment (ROI) in terms of driving purchase intent. These are the behaviors to prioritize investments in training, incentives and rewards. These are the behaviors which will yield the highest ROI.
The rest of the behaviors are prioritized as follows:
Those with the high importance and high performance are the next priority. They are the behaviors to maintain. They are important and employees perform them frequently, so invest to maintain their performance.
Those with low importance are low performance are areas to address if resources are available.
Finally, behaviors or service attributes with low importance yet high performance are in no need of investment. They are performed with a high degree of frequency, but not very important, and will not yield an ROI in terms of driving purchase intent.
Research without call to action may be interesting, but in the end, not very useful.
This simple, intuitive gap analysis technique will provide a clear call to action in terms of identifying service behaviors and attributes which will yield the most ROI in terms of achieving your key objective of the customer experience.
Best in class mystery shop programs provide managers a means of applying coaching, training, incentives, and other motivational tools directly on the sales and service behaviors that matter most in terms of driving the desired customer experience outcome. One tool to identify which sales and service behaviors are most important is Key Driver Analysis.
Key Driver Analysis determines the relationship between specific behaviors and a desired outcome. For most brands and industries, the desired outcomes are purchase intent or return intent (customer loyalty). This analytical tool helps mangers identify and reinforce sales and service behaviors which drive sales or loyalty – behaviors that matter.
As with all research, it is a best practice to anticipate the analysis when designing a mystery shop program. In anticipating the analytical needs of Key Driver Analysis identify what specific desired outcome you want from the customer as a result of the experience.
- Do you want the customer to purchase something?
- Do you want them return for another purchase?
The answer to these questions will anticipate the analysis and build in mechanisms for Key Driver Analysis to identify which behaviors are more important in driving this desired outcome – which behaviors matter most.
Next, ask shoppers if they had been an actual customer, how the experience influenced their return intent. Group shops by positive and negative return intent to identify how mystery shops with positive return intent differ from those with negative. This yields a ranking of the importance of each behavior by the strength of its relationship to return intent.
Additionally, pair the return intent rating with a follow-up question asking, why the shopper rated their return intent as they did. The responses to this question should be grouped and classified into similar themes, and grouped by the return intent rating described above. The result of this analysis produces a qualitative determination of what sales and service practices drive return intent.
Finally, Key Driver Analysis produces a means to identify which behaviors have the highest potential for return on investment in terms of driving return intent. This is achieved by comparing the importance of each behavior (as defined above) and its performance (the frequency in which it is observed). Mapping this comparison in a quadrant chart, provides a means for identifying behaviors with relatively high importance and low performance – behaviors which will yield the highest potential for return on investment in terms of driving return intent.
Behaviors with the highest potential for return on investment can then be inserted into a feedback loop into the mystery shop scoring methodology by informing decisions with respect to weighting specific mystery shop questions, assigning more weight to behaviors with the highest potential for return on investment.
Employing Key Driver Analysis gives managers a means of focusing training, coaching, incentives, and other motivational tools directly on the sales and service behaviors that will produce the largest return on investment. See the attached post for further discussion of mystery shop scoring.
As bank contact centers transition from service hubs to sales centers. It is instructive to investigate which service attributes and behaviors will yield the most ROI in terms of driving sales through the contact center channel.
Beyond the attributes measured in the previous post, Kinesis also performed mystery shop observations of specific behaviors across six institutions with national scope to determine their relationship to purchase intent.
While the importance of a good first impression is true, across the four greeting behaviors measured, there does not appear to any significant differences between shops with positive purchase intent and shops with negative purchase intent.
|Greeting||Increased Purch Intent||Decreased Purch Intent|
|Greet by identifying the name of the institution||99%||97%|
|Greet by identifying themselves||100%||97%|
|Ask how they could assist||100%||98%|
This is mostly due to the high rate of performance across these greeting behaviors. Greetings are strong across all shops regardless of purchase intent.
With respect to broader service behaviors, the behaviors with the largest gaps between shops with positive purchase intent and those with negative purchase intent are: suggesting additional products and asking for the business (each with about 2 times as many shoppers who experience positive purchase intent observing these behaviors compared to shoppers who experience negative purchase intent). Use name, use of understandable descriptions and clarity of speech round out the next three.
Hold & Transfer Behaviors
When the shopper was placed on hold, five behaviors were measured. Of these, giving an estimate of how long the customer will be on hold, and returning if the hold time exceed the original estimate to advice of the status of the call were the behaviors with the strongest relationship to purchase intent. Again, both with about 2 times as many shoppers who experienced positive purchase intent observing these behaviors compared to shoppers who experienced negative purchase intent.
Additionally, five transfer behaviors were measured. Of these, the behavior with the strongest relationship to purchase intent by far is returning to explain any delay after 60 seconds. With five times as many shoppers who reported positive purchase intent observing this behavior relative to those who reported negative purchase intent.
Finally, at the conclusion of the call, asking how else the agent could be of assistance and thanking the shopper for choosing the institution were the two behaviors with the strongest relationship to purchase intent.
Of the behaviors measured, a couple of common themes tend to be present in the behaviors with strong relationships to purchase intent. The behaviors with strong relationships to purchase intent tend to deal with the themes of personalized service and valuing the customer.
These behaviors with strong relationships to purchase intent group into these two themes as follows:
- If transfer hold time exceeds 60 seconds, explain delay and ask if customer wants to continue (this behavior 5 times more likely in shops with positive purchase intent compared to negative)
- Give estimate of hold time (2.1 times more frequent in shops with positive purchase intent compared to negative)
- If hold exceeds estimate, return with status update (1.8 times more frequent)
- If transfer, stay on line until completed (1.6 times)
- Use of name (1.5 times)
- Ask how else could assist (1.5 times)
- Suggest additional products or services (2.1 times more frequent in shops with positive purchase intent)
- Ask for business (1.9 times)
- Thanks for choosing the institution (1.5 times)
To maximize purchase intent, focus agents on behaviors which personalize the service in an empathetic manner (care for the customer and their needs) and value their business.
Historically, bank contact centers have served primarily as service hubs, serving customers who call for information or are seeking assistance dealing with a problem in need of resolution. As banks continue to transition into an omni-channel model where customers can interact with the institution across a broad spectrum of channels, the contact center is transitioning into a sales hub, where customers who have researched a product online may still want to speak with a person prior to completing the purchase. As a result, contact center agents will require a new set of sales skills.
To help understand some of the new skill sets required of contact center agents as they transition from a service to sales role, Kinesis conducted mystery shops of six institutions with national scope to identify what customer experience attributes will yield the most ROI in supporting this sales role.
Our conclusion is customers want empathy and competence. They want agents who both care about their needs and can satisfy those needs.
Kinesis performed an analysis of purchase intent to identify the attributes with the most potential for ROI in supporting a sales role. We asked shoppers to rate the experience across a spectrum of service attributes on a 5-point scale where 1 is poor and 5 is excellent; as well assigning a purchase intent rating on a similar 5-point scale. We then cross tabulated the results by purchase intent to identify which attributes have the largest gap between shops which reported positive purchase intent and those which reported negative purchase intent.
Confidence in the Agent, valuing as a customer, interest in helping and explain the products in understandable terms are the four attributes with largest gaps between shops with positive purchase intent and negative, followed by professionalism and job knowledge. Friendliness/courtesy was the attribute with the smallest gap. While friendliness is important, when it comes to driving purchase intent, the attributes with the largest gaps are those related to care and competence. Customers want agents who care about their needs, and are capable of delivering on those needs.
Increasingly banks must operate in a multi-channel environment. While the changing role of the branch, combined with automated channels such as online and mobile, are getting a lot of attention, there remains a key role for the contact center in delivering an effective customer experience. Central to this key role is designing an effective customer experience, comprised of the right sales and service behaviors – those which influence customer attitudes and behaviors in a profitable way yielding the most return on investment.
To provide direction with respect to what sales and service behaviors will yield the most return on investment, Kinesis conducted a series of mystery shops to identify which sales and service behaviors have the most influence on purchase intent. In addition to observing specific sales and service behaviors, mystery shoppers were also asked to rate how the call would have influenced their purchase intent if they had been a real customer. This purchase intent rating was then used as means of calculating the strength of the relationship between each behavior and purchase intent.
To determine the relationship between these service attributes and purchase intent, the data for these different studies was cross-tabulated by the purchase intent rating and subjected to significance testing. [i]
When the percentage of calls in which purchase intent significantly increased is tested against the percentage of calls where purchase intent significantly decreased, nearly all the sales and service attributes are statistically significant at or above a 95% confidence level.
|Significantly Increased||Significantly Decreased||Test Statistic|
|Explanations easy to understand||99%||45%||9.0|
|When thanked, respond graciously||98%||42%||8.5|
|Friendly demeanor / pleasant voice||100%||60%||8.4|
|Express appreciation for interest / thank you for business||92%||20%||8.3|
|Ask probing questions||79%||10%||6.4|
|Offer further assistance||85%||25%||6.2|
|Speak clearly and avoid bank jargon||98%||68%||5.8|
|Listen attentively to your needs||80%||25%||5.3|
|Mention other bank product||99%||75%||5.3|
|Invite you to visit branch||64%||10%||4.6|
|Explain the value of banking with bank||57%||5%||4.4|
|Offer to mail material / mention website||66%||20%||4.3|
|Ask your name||68%||25%||3.8|
|Ask for your business / close the sale||57%||21%||2.9|
|If no one available to assist you, offered options||100%||0%||2.2|
The differences between the highest and lowest purchase intent for product knowledge and ease to understanding explanations are the most significant, while a professional greeting is the least significant.
Dividing these behaviors into rough quartiles and comparing them side-by-side, reveals some interesting observations:
Explanations easy to understand
When thanked, respond graciously
Friendly demeanor / pleasant voice
Express appreciation for interest / thank you for business
Ask probing questions
Offer further assistance
Speak clearly and avoid bank jargon
When thanked, employee respond graciously
|Listen attentively to your needs
Mention other bank product
Invite you to visit branch
Explain the value of banking with bank
Offer to mail material / mention website
|Ask your name
Ask for your business / close the sale
If no one available to assist you, offered options
The attributes with the most significant differences between high and low purchase intent ratings appear to be those associated with reliability and empathy. It appears mystery shoppers valued such “core” attributes as product knowledge or interest/enthusiasm for the customer. They seem to be less concerned with more peripheral service attributes, such as asking for names, etc. Influencing purchase intent is not as simple as merely using the customer’s name or answering the phone within a short period of time. Rather it is a much more challenging undertaking of being competent in your job and having the customer’s best interests at heart.
[i] Significance testing determines if any differences observed are the result of actual differences in the populations measured rather than the result of normal variation. Without getting into too much detail, significance testing produces a test statistic to determine the probability that differences observed are statistically significant. A test statistic above 1.96 equates to a 95% confidence level, which means there is a 95% chance any differences observed are the result of actual differences in the populations measured rather than normal variation. For all practical purposes a test statistic over 3.1 means there is 100% chance the differences observed are statistically significant (although in reality the probability never reaches 100%). Finally, in interpreting the following analysis, it is important note that test statistics are not lineal. A test statistic of two is not twice as significant as a test statistic of one. The influence on significance decreases as the test statistic increases. However, the test statistic does give us an opportunity to rank the service attributes by their statistical significance.
What impresses customers positively as a result of a call to your call center?
To answer this question, Kinesis conducted research into the efficacy of the bank contact center sales process by observing a battery of sales and service behaviors through the use of mystery shoppers. The objective of this study was to identify which sales and service behaviors drive purchase intent. (See the insert below for a description of the methodology).
The table at the end of this post shows the relative frequency in which each behavior was observed in shops where the shopper reported positive purchase intent as a result of the call, compared to shops with negative purchase intent.
The seven behaviors with the strongest relationship to purchase intent are:
- Invite to visit a branch
- If on hold, thank for waiting
- Express appreciation for interest/thank for business
- Offer further assistance
- Mention/refer to website
- Listen attentively to your needs
- Offer to send material
Each of these behaviors is at least three times more likely to be present in shops with positive purchase intent compared to those with negative purchase intent.
Two observations jump out from this first group of behaviors:
First, integration of other channels into the sales process appears to drive purchase intent. Inviting the shopper to visit a branch was observed 6.4 times more frequently in shops with positive purchase intent compared to negative. The branch still has a role in the sales process; other research consistently points to the convenience of branch location as a driver of selection of a primary financial institution. If contact centers leverage the branch during the sales process, they have a significantly better chance to advance the sale. Additionally, when the agent incorporated the website into the sales presentation, they also have a better chance of advancing the sale. Mentioning the website was 3.3 times more likely to be present in shops with positive purchase intent compared to negative.
Secondly, the balance of these key behaviors all revolve around personal attention (thank for waiting on hold, offing further assistance, listening attentively, offer to send material) and interest in the customer’s business (express appreciation or thank for business).
Nine more behaviors were at least twice as likely to be present in shops with positive purchase intent:
- Product knowledge
- Ask for name
- Ask for your business/close the sale
- If on hold, check back in 1 minute
- When thanked respond graciously
- Ask probing questions
- Explanations easy to understand
- Explain the value of banking with bank
- Thank for calling
The themes most common in this second group of behaviors that appear to influence purchase intent are competence (product knowledge, easy to understand explanations), personal attention (asking name, checking back on hold, probing of needs) and interest in the customer’s business (ask for business, express value, thank for calling).
So…what drives purchase intent as a result of a call to a contact center? Integrating other channels into the conversation, and sincerely expressing interest in the customer broadly drive purchase intent.
Frequency Behavior Observed in Shops with Increased and Decreased Purchase Intent:
|Invite to visit a branch||64%||10%|
|If on hold, thanked for waiting||97%||20%|
|Express appreciation for interest / thank you for business||92%||20%|
|Offer further assistance||85%||25%|
|Mention/refer to website||66%||20%|
|Listen attentively to your needs||80%||25%|
|Offer to send material||97%||31%|
|Ask for name||68%||25%|
|Ask for your business/close the sale||76%||32%|
|If on hold, check back in 1 minute||94%||40%|
|When thanked respond graciously||98%||42%|
|Ask probing questions||94%||42%|
|Explanations easy to understand||99%||45%|
|Explain the value of banking with bank||88%||43%|
|Thank for calling||99%||50%|
|Friendly demeanor / pleasant voice||100%||60%|
|Avoid bank jargon||98%||68%|
|Mention other bank product||99%||75%|
|Wait for response before placed on hold||100%||80%|
|Demonstrate understanding of question||100%||81%|
|Answered in 3 rings||99%||88%|
To evaluate the state of the in-branch sales process, Kinesis mystery shopped five banks with significant North American footprints. Among the objectives of the study were to:
1) Define the sales process among different institutions.
2) Evaluate the effectiveness of specific sales behaviors.
Shoppers were asked a mixture of closed-ended questions to evaluate the presence or frequency of specific behaviors, and open-ended questions to gather the qualitative impressions of these behaviors on the shoppers – in short the how and why behind how the shopper felt. Finally, to provide a basis to evaluate the effectiveness of each sales behavior, shoppers were asked to rate their purchase intent as a result of the visit. This purchase intent rating was then used as a means of evaluating what behaviors tend to be present when positive purchase intent is reported as opposed to negative purchase intent.
Compliance and service quality are not mutually exclusive. In fact there is a positive relationship between compliance and, not only service quality, but stronger sales.
To understand the relationship between Truth in Savings Act (TISA) compliance behaviors and the customer experience, Kinesis mystery shopped five US banks with the objective of evaluating TISA compliance and the customer experience. A description of the methodology is at the conclusion of this post.
Employees who correctly quoted APY provided a more satisfactory experience. The overall impression of the entire experience for shops that passed the TISA disclosure requirements was 4.1 compared to 3.4 for shops that failed, on a scale of 1 to 5, where 1 is “Extremely Dissatisfied” and 5 is “Extremely Satisfied”.
|Passed Ave. (Net Sat.)||Failed Ave. (Net Sat.)|
|Overall impression of entire experience||4.1 (70%)||3.4 (24%)|
Beyond the overall satisfaction with the experience, significant differences also exist across all the individual service attributes measured. In all of the following seven attributes measured, shops which passed the compliance test consistently had higher attribute satisfaction ratings.
|Passed Ave. (Net Sat.)||Failed Ave. (Net Sat.)|
|Job knowledge||4.0 (64%)||3.2 (15%)|
|Confidence in the agent||4.1 (71%)||3.5 (31%)|
|Use understandable terms||4.0 (62%)||3.4 (24%)|
|Professionalism||4.3 (74%)||3.8 (53%)|
|Interest in helping||4.1 (63%)||3.7 (40%)|
|Valuing customer||4.0 (59%)||3.6 (33%)|
|Friendliness/courtesy||4.4 (82%)||4.1 (69%)|
Not surprisingly, there is a relationship between compliance and job competence. Employees who comply with the TISA are more likely to demonstrate competence in other aspects of their job. Job knowledge, confidence in the employee, use of understandable terms, and professionalism are the four attributes with the largest gaps between shops that passed and shops that failed.
Additionally, employees who pass the compliance test are more skilled at connecting with the potential customer; receiving higher average ratings for interest in helping, valuing the customer and even friendliness and courtesy.
Now, as if compliance risk and customer service were not strong enough cases for TISA compliance, what about the sale? What about sales effectiveness?
To evaluate the sales effectiveness of the presentation, shoppers were asked to rate their purchase intent as a result of the interaction. The following chart illustrates the distribution of the purchase intent rating for shops which passed the compliance test compared to shops which failed:
Shops which complied with TISA requirements had significantly higher purchase intent compared to shops that failed. Shops which passed exhibited a net positive purchase intent of 46% (46% more shops received positive purchase intent ratings compared to negative). On the other hand, shops which failed the TISA evaluation received a net negative purchase intent of minus 9%, (9% more shoppers assigned negative purchase intent than positive).
There is a relationship between TISA compliance and the customer experience. Employees who are skilled at compliance behaviors, exhibit similar superior service and sales skills compared to those with weaker skills.
To investigate the relationship between the overall customer experience and compliance, Kinesis mystery shopped 50 branches and the call centers of five banks with significant North American footprints. Among the objectives of the study were to:
1) Evaluate Truth in Savings act compliance, specifically the presence and timing of Annual Percentage Yield quotes, and
2) Evaluate the effectiveness of specific sales behaviors.
Shoppers were asked a mixture of closed-ended questions to evaluate the presence or frequency of specific behaviors, as well as to rate various service attributes with 5-point scales. Finally, to provide a basis to evaluate the effectiveness of each sales behavior, shoppers were asked to rate their purchase intent as a result of the visit. This purchase intent rating was then used as a means of evaluating what behaviors tend to be present when positive purchase intent is reported as opposed to negative purchase intent.