Mystery Shopping Questionnaire Design

Previously we examined the process of defining mystery shopping objectives.

Keep it Simple

Often mystery shopping programs are designed by committee which can result in an overly complicated and cumbersome program. Unrealistic scenarios combined with long, overly complex questionnaires result in frustration for mystery shopper, mystery shop provider and the end client.  In such cases the likelihood of shopper exposure is increased and the accuracy of the observations suffers.  Keep it simple – simpler designs work better and provide more value.

Anticipate the Analysis

Finally, identify what specific desired outcome you want from the customer as a result of the experience.  Do you want the customer to purchase something?  Do you want them return for another purchase?  The answer to this question will anticipate the analysis and build in mechanisms for Key Driver Analysis to identify which behaviors are more important in driving this desired outcome – behaviors that matter.

What, How & Why

A best practice in mystery shop questionnaire design is to include observations of objective behaviors, subjective impressions and comments.  Each of these serves a specific purpose in identifying the service behaviors that matter – behaviors which drive profitability.  Together, these three elements of questionnaire design reveal the “what”, “how” and “why” of the customer experience.

Objective Behaviors:  Observations of objective behaviors form the backbone of best in class mystery shops.  These observations identify what specific sales and service behaviors were observed.  Mystery shopping is primarily an observational form of research, and as such, a best practice in mystery shopping is to focus on observations of specific objective and observable behaviors. These objective observations serve two purposes:  First, they measure and motivate expected sales and service behaviors.  Second, they serve as a foundation for Key Driver Analysis, where the other two subjective elements of the questionnaire are used to determine the relationship between employee behaviors and a desired outcome, such as purchase intent or customer loyalty.

Subjective Impressions:  Subjective impressions are primarily captured through scientifically designed and strategically selected rating scales.  These questions reveal how the shopper felt about the experience.  They add both a quantitative and qualitative perspective to the objective behaviors observed and provide a basis for interpretation of not only individual shops, but also an analytical means to determine the relationship between each service behavior and the desired outcome.  We will explore this in more detail in a discussion of Key Driver Analysis.

Subjective Comments:  Beyond measuring what behaviors were observed and how the shopper felt about the experience, open-ended comments capture why shoppers felt the way they did about the experience.  While objective behaviors are the backbone of the shop, many of Kinēsis’ clients consider these comments the heart of the shop, providing a qualitative texture to understand specifically what the shopper felt about the experience.  They not only serve as a framework for understanding each shop individually, but provide raw material for content analysis to determine key qualitative key drivers of the desired outcome (purchase intent and customer loyalty).

Anticipate the Analysis

A best practice in mystery shopping program design is to anticipate the analysis.  Together, these three design elements provide input into Key Driver Analysis techniques to identify key sales and service drivers of purchase intent and loyalty – behaviors that matter.

Click here for the next installment in this series: Best Practices in Mystery Shop Scoring.