Mystery shop programs measure human interactions; interactions with other humans and increasingly human interactions with automated machines. Given that humans are on one or both sides of the equation, it is not surprising that variation in the customer experience exists.
When designing a mystery shop program, a central decision is the number of shops to deploy. This decision is dependent on a number of issues including: desired reliability, number of customer interactions, and the budgetary resources available for the program. However, one additional and very important consideration, which frankly doesn’t get much attention, is the amount of variation expected in the customer experience to be measured.
The level of variation in the customer experience is an important consideration. Consistent customer experience processes require less mystery shops than those with a high degree of variation. To illustrate this, consider the following:
Assume a customer experience process is 100% consistent with zero variation from experience to experience. Such a process would require only one shop to accurately describe the experience as a whole. Now, consider a customer experience process with an infinite level of variation in the experience. Such a process would require far more than one shop. In fact, assuming an infinite level of variation, 400 shops would be required to achieve a margin of error of plus or minus five percent.
Obviously, the variation of most customer experience processes reside somewhere between perfect consistency and infinite variation. So how do managers determine the level of variation in their process? The answer to this question will probably be more qualitative than quantitative. Ask yourself:
- Do you have a set of standardized customer experience expectations?
- Are these expectations clearly communicated to employees?
- Other than mystery shopping, do you have any processes in place to monitor the customer experience? If so, are the results of these monitoring tools consistent from month-to-month or quarter-to-quarter?
To make it easy, I always ask new clients to give a qualitative estimate of the level of variation in their customer experience from: high, medium to low. The answer to this question will also be considered along with the level of statistical reliability desired and budgetary resources available for the program in determining the appropriate number of shops.
So – ask yourself; how much variation can we expect in our customer experience?
Self-help resources typically take the form of a webpage housed on the mystery shop provider’s website or on an internal resource page. These resources provide a tutorial in the form of either a PowerPoint or video, reinforcing to stakeholders many of the subjects already discussed: definition of the brand, behavioral service expectations, and a copy of the questionnaire.
These self-help resources are also an excellent opportunity to introduce the mystery shop reports and how to read them (both on an individual shop basis and on an analytical level), and introduce concepts designed to identify the relative importance of specific sales and service behaviors which drive desired outcomes like purchase intent and customer loyalty.
Shop Results E-Mail
Upon distribution of the first shop, it is a best practice in launching a mystery shop program to send an e-mail to the supervisor of the employee shopped advising them of a completed shop, and containing either a PDF shop report or access to the shop via an online reporting tool.
The content of this e-mail should be dependent on the performance of the individuals shopped. If a shop is perfect, the e-mail should congratulate the employees on a perfect shop. If a shop is below expectations, it should inform the employees, in as positive way as possible, that their performance was below expectations and set the stage for coaching. It should remind employees that it is not the performance of this first shop that counts, but subsequent improvement as a result of the shops.
Depending on the timing of shop e-mails, some clients prefer the shop to be sent as soon as it clears the provider’s quality control process, while others prefer shops be held and released in mass at the end of a given shopping period (typically monthly). If the e-mail is sent at the end of a given period, this is an excellent opportunity to identify top performers who received perfect shops as a means of both recognizing superior performance, and motivating other employees to seek similar achievement.
Finally, this e-mail should reinforce superior shop performance by reminding front-line employees and managers of the rewards earned by successful shop performance.
This e-mail should be modified for all subsequent waves of shopping and be used as a cover letter for distribution of all future shops.
Additional e-mails may be sent to notify employees and their managers of specific events, such as: perfect shops, failed shops, shops within a specific score range, or shops which identify a specific behavior of an employee like a cross-sell effort.
Post Shop Call/ Presentation
Similar to the kickoff presentation, after the first wave of shopping, it is a best practice to conduct a post shop presentation, again by conference call or WebEx. The purpose of this presentation is to present the reports available, discuss how to read them, and – most importantly – take action on the results through coaching and interpreting call to action elements built into the program. Call to action elements designed to identify which behaviors are most important in terms of driving purchase intent or loyalty.
There should be no surprises in mystery shopping. A key to keeping all stakeholders informed of the mystery shop process is pre-shop communication.
The first communication tool is the kickoff letter. This letter is most often in the form of an e-mail. Sent prior to shopping, its purpose is to introduce employees to the program, explain its purpose in a positive way, make sure employees are aware of what is expected of them, and link shopping to their best interests, by reinforcing it is designed to make them more successful.
The kickoff e-mail should:
- Define the brand and emphasize that frontline employees are the personification of the brand. They are the physical embodiment of the brand.
- Explain that certain behaviors are expected from them in their role as the physical embodiment of the brand.
- List the specific sales and service behaviors that shoppers are asked to observe. Stress that management wants every representative to score well. Management has no interest in setting employees up for failure. If they perform these behaviors, they will receive a perfect shop score.
- Detail the incentive and reward structures in place as a result of the mystery shop program.
A presentation, conference call, or WebEx is an excellent tool to kick off a mystery shop program. All stakeholders in the process should understand their role and what is expected of them.
As with the kickoff letter or e-mail, the presentation should define the brand, stress that employees are the physical embodiment of the brand, and identify the specific sales and service behaviors expected from employees.
It should identify the internal administrator of the program, communicate the dispute process, discuss incentives and rewards earned through positive mystery shops, as well as introduce the concept of coaching as a result of the shop – making sure that managers and customer-facing personnel understand their role in the coaching process.
Finally, this presentation should introduce employees to self-help resources available for taking positive action as a result of the shop.
Research without a clear call-to-action may be interesting but not very useful. Best-in-class mystery shop programs build call-to-action elements into their design to ensure the results provide a clear direction.
Reading and interpreting research reports is a specialized skill. To maximize the value of the mystery shopping, employees should be given instruction into the basic skills of interpreting a research report. Some interpretation skills are easy, such as reading a data table which compares results across units of hierarchy (such as one store compared to another), or over specific time periods (such as one quarter to another). Others may be a little more complicated. such as a cross-tabulation of purchase intent which is a comparison of shops where the shopper reported positive purchase intent and those where the shopper reported negative purchase intent as a result of the shop. However reading them is the same as the comparisons of stores or quarters. All that is needed is just an understanding of how shops are grouped for the comparisons.
Beyond instructing the frontline how to read reports, they should also be given a primer on how to take action on the results.
The most common way managers of frontline employees take action on the results is coaching. Best-in-class mystery shop programs identify employees in need of coaching as a result of the shop, and managers should be instructed on how to use the results to coach employees.
For each such coaching opportunity, frontline managers should be given guidance on how to coach improvement, as well as online tools to log coaching, making both the employee and the manager accountable for coaching.
Additionally, managers should understand the analytical framework for maximizing the value of the program. Best-in-class mystery shop programs build in call-to-action components designed to identify key sales and service behaviors which correlate to a desired customer experience outcome. We call this Key Driver Analysis.
Key Driver Analysis identifies the relationship between specific sales and service behaviors and a desired outcome identifying which behaviors are key drivers of this desired outcome. For most brands, the desired outcomes are purchase intent or return or loyalty. Building these call-to-action elements into the program helps brands identify and motivate the sales and service behaviors with the most potential for return on investment in terms of driving loyalty or purchase intent.
Best in class mystery shop programs provide a central point of internal administration. A central administrator manages the relationship with the mystery shop provider and coordinates with other stakeholders (such as training and human resources).
This central point of administration requires a strong administrator to keep the brand focused and engaged, and to make sure that recalcitrant field managers are not able to undermine the program before it begins to realize its potential value.
A best practice in launching a mystery shop program is to identify, to all stakeholders, the main contact for internal administration, and how to communicate with them. Along with identifying the internal administrator, in most cases, it is a best practice to also identify the mystery shopping provider – just to keep employees comfortable with the measurement process. However, in some cases, such as instances where there has been a history of employees gaming the system, it may be more appropriate to keep the mystery shop provider anonymous.
Disputed shops are part of the mystery shop process. Mystery shops are just a snap shot in time, and measure complicated service encounters. As a result, there may be extenuating circumstances that need to be addressed, or questions about the quality of the shopper’s performance that require both a fair and firm process to resolve.
The specifics of the dispute process should be aligned with the brand’s values and culture. Broadly, there are two ways to design a dispute process: arbitration and fixed number of challenges.
Arbitration: Most brands have a program manager or group of program managers acting as an arbitrator of disputes and ordering reshops or adjusting points to an individual shop as they see fit. The arbiter of disputes must be both fair and firm, otherwise, employees and other managers will quickly start gaming the system, bogging the process down with frivolous disputes.
Fixed Number of Challenges: Other brands give each business unit (or store) a fixed number of challenges in which they can ask for an additional shop. Managers responsible for that business unit can request a reshop for any reason. However, when the fixed number of disputes is exhausted they lose the ability to request a reshop. This approach is fair (each business unit has the same number of disputes), it reduces the administrative burden on a centralized arbiter, and reduces the potential for massive gaming of the system as there is a limited number of disputes.
Best in class mystery shop programs clearly communicate behavioral expectations to frontline employees. There should be no surprises in mystery shopping.
Brands have personality. Brand personality is a set of characteristics associated with the positioning, products, price and service mix offered by a company. Launch the program by communicating your desired brand personality. While branding is a complicated mix of product, price, positioning and place, it often falls on the frontline employees to make the brand real in the perception of the customers – to animate the brand. It is, therefore, critical that employees’ service behaviors be aligned with the brand personality. Start the mystery shop program launch with a clear description of your desired brand personality.
After communication of the brand personality, the next step is to define what specific sales and service behaviors you expect from employees as ambassadors of the brand. Create a list of behavioral expectations by asking yourself the following questions:
- What specific service behaviors do we expect?
- When greeting a customer, what specific behaviors do we expect from staff?
- When meeting with customers after the greeting, what specific behaviors do we expect?
- If a phone interaction, what specific hold/transfer procedures do we expect (for example asking to be placed on hold, informing customer of the destination of the transfer)?
- Are there specific profiling questions we expect to be asked? – If so, what are they?
- What closing behaviors do we expect? How do we want employees to ask for the business?
- At the conclusion of the interaction, how do we want the employee to conclude the conversation or say goodbye?
- Are there specific follow-up behaviors that we expect, such as getting contact information, suggesting another appointment, or offering to call the customer?
- What other specific behaviors do we expect?
Remember the goal is to ensure employees animate the brand. Each behavior expected should support this end.
Ultimately it is a best practice to give employees a copy of the actual questionnaire and shopper guidelines. Best in class mystery shop questionnaires are composed of a mixture of objective behavioral observations and subjective impressions and comments.
The objective observations of behaviors form the backbone of the program. They measure and motivate the specific sales and service behaviors expected from employees. These observations must be both objective and empirical, answering the question, was a specific behavior observed or not?
Rating scales are the most common means of collecting subjective impressions. Measures of how the shopper felt about the experience. They add both a qualitative and quantitative perspective to the objective behaviors, as well as provide a basis for interpreting their importance.
While empirical behaviors are the backbone of the shop, many of Kinēsis’ clients consider open-ended comments the heart of the shop. Subjective open-ended questions should reveal valuable insight into understanding exactly how the shopper felt about the experience.
There should be no surprises in mystery shopping. Customer-facing employees should understand exactly what behaviors are being measured, and how shoppers are to interpret these behaviors in terms of completing the questionnaire.
There should be no surprises in mystery shopping. When investments in mystery shopping fail to achieve their potential, it is often because those who are accountable for the results, the front-line employees and their direct managers, were not properly introduced to the program.
Improper positioning and introduction of the program risks creating internal resistance. Front-line personnel may interpret mystery shopping as something akin to Orwell’s Big Brother – interpreting it as a distrustful management checking up on their employees. They may see the mystery shop program solely as a means of realizing financial rewards, rather than more intrinsic rewards such as being better at their profession, and as a result game the system by frivolously disputing shops. This internal resistance often manifests itself in the form of excessive disputes, questioning everything, wasting hours of time reviewing security films, and playing a game of indentifying the shopper – almost always phantom shoppers (actual customers who are not mystery shopping them). All this internal resistance creates an unnecessary distraction from realizing the brand’s customer experience goals.
Key to launching a successful mystery shopping program is communication, positive communication of: behavioral expectations of employees, guidance regarding internal program administration, and instruction on how to use the results to improve performance. There should be no surprises in mystery shopping, surprises create resistance and kills buy-in.
Position mystery shopping as a win-win. Position it that mystery shopping is designed to help the employee by making them better at their jobs. Employees want to succeed. They want to be good at their jobs. Leverage this desire to succeed in obtaining buy-in from the frontline.
It is, therefore, critical to ensure employees throughout the organization are fully informed and have bought into the program before it is launched. Pre-launch communication should include:
- definition of the brand
- description of the employees’ role as ambassadors of the brand
- list specific behaviors expected of employees (including a copy of the mystery shop questionnaire)
- answering procedural questions of how to communicate program related issues
- training employees how to read mystery shopping reports
- Finally, how to use the information effectively, including and how to set goals for improvement.
Proper launching of a mystery shop program is critical to its success. Starting on the right foot positions mystery shopping in the minds of customer-facing personnel as a positive tool to help them become better at their jobs – and offers real benefits to them both in terms of rewards as a result of the shop, but also intrinsically as it reinforces sales and service behaviors that will benefit them throughout their career.
Communication is key – again, there should be no surprises in a mystery shop program.