Best Practices in Mystery Shop Scoring

Most mystery shopping programs score shops according to some scoring methodology to distill the mystery shop results down into a single number.  Scoring methodologies vary, but the most common methodology is to assign points earned for each behavior measured and divide the total points earned by the total points possible, yielding a percentage of points earned relative to points possible.

Drive Desired Behaviors

Some behaviors are more important than others.  As a result, best in class mystery shop programs weight behaviors by assigning more points possible to those deemed more important.  Best practices in mystery shop weighting begin by assigning weights according to management standards (behaviors deemed more important, such as certain sales or customer education behaviors), or according to their importance to their relationship to a desired outcome such as purchase intent or loyalty.  Service behaviors with stronger relationships to the desired outcome receive stronger weight.

One tool to identify behavioral relationships to desired outcomes is Key Driver Analysis.  See the attached post for a discussion of Key Driver Analysis.

Don’t Average Averages

It is a best practice in mystery shopping to calculate the score for each business unit independently (employee, store, region, division, corporate), rather than averaging business unit scores together (such as calculating a region’s score by averaging the individual stores or even shop scores for the region).  Averaging averages will only yield a mathematically correct score if all shops have exactly the same points possible, and if all business units have exactly the same number of shops.  However, if the shop has any skip logic, where some questions are only answered if specific conditions exist, different shops will have different points possible, and it is a mistake to average them together.  Averaging them together gives shops with skipped questions disproportionate weight.  Rather, points earned should be divided by points possible for each business unit independently.   Just remember – don’t average averages!

Work Toward a Distribution of Shops

When all is said and done, the product of a best in class mystery shop scoring methodology will produce a distribution of shop scores, particularly on the low end of the distribution.

Mystery shop programs with tight distributions around the average shop score offer little opportunity to identify areas for improvement.  All the shops end up being very similar to each other, making it difficult to identify problem areas and improve employee behaviors.  Distributions with scores skewed to the low end, make it much easier to identify poor shops and offer opportunities for improvement via employee coaching.  If questionnaire design and scoring create scores with tight distributions, consider a redesign.

Most mystery shopping programs score shops according to some scoring methodology.  In designing a mystery shop score methodology best in class programs focus on driving desired behaviors, do not average averages and work toward a distribution of shops.