This post considers two types of variation in the customer experience: common and special cause variation, and their implications for customer researchers.
The concepts of common and special cause variation are derived from the process management discipline Six Sigma.
Common cause variation is normal or random variation within the system. It is statistical noise within the system. Examples of common cause variation in the customer experience are:
- Poorly defined, poorly designed, inappropriate policies or procedures
- Poor design or maintenance of computer systems
- Inappropriate hiring practices
- Insufficient training
- Measurement error
Special cause variation, on the other hand, is not random. It conforms to laws of probability. It is the signal within the system. Examples of special cause variation include:
- High demand/ high traffic
- Poor adjustment of equipment
- Just having a bad day
What are the implications of common and special cause variation for customer experience researchers?
Given the differences between common cause and special cause variation, researchers need a tool to help them distinguish between the two. Researchers need a means of determining if any observed variation in the customer experience is statistical noise or a signal within the system. Control charts are a statistical tool to make a determination if variation is noise or a signal.
Control charts track measurements within upper and lower quality control limits. These quality control limits define statistically significant variation overtime (typically at a 95% confidence), which means there is a 95% probability that the variation is the result of an actual change in the customer experience (special cause variation) not just normal common cause variation. Observed variation within these quality control limits are common cause variation. Variation which migrates outside these quality control limits is special cause variation.
To illustrate this concept, consider the following example of mystery shop results:
This chart depicts a set of mystery shop scores which both vary from month to month and generally appear to trend upward.
Customer experience researchers need to provide managers a means of determining if the month to month variation is statistical noise or some meaningful signal within the system. Turning this chart into a control chart by adding statistically defined upper and lower quality control limits will determine if the monthly variation is common or special cause.
To define quality control limits, the customer experience researcher needs to determine the count of observations for each month, the monthly standard deviation, and the average count of shops across all months.
The following table adds these three additional pieces of information into our example:
|Count of Mystery Shops||Average Mystery Shop Scores||Standard Deviation of Mystery Shop Scores|
To define the upper and lower quality control limits (UCL and LCL, respectively), apply the following formula:
x = Grand Mean of the score
n = Mean sample size (number of shops)
SD = Mean standard deviation
These equations yield quality control limits at 95% confidence, which means there is a 95% probability any variation observed outside these limits is special cause variation, rather than normal common cause variation within the system
Calculating these quality control limits and applying them to the above chart produces the following control chart, with upper and lower quality control limits depicted in red:
This control chart now answers the question, what variation is common cause and what variation is special cause. The general trend upward appears to be statistically significant with the most recent month above the upper quality control limit. Additionally, this control chart identifies a period of special cause variation in July. With 95% confidence we know some special cause drove the scores below the lower control limit. Perhaps this special cause was employee turnover, perhaps a new system rollout, or perhaps a weather event that impacted the customer experience.
Previously, we discussed the implications of inter-channel consistency for researchers, and introduced a process for management to define a set of employee behaviors which will support the organization’s customer experience goals across multiple channels.
This post considers the implications of intra-channel consistency for customer experience researchers.
As with cross-channel consistency, intra-channel consistency, or consistency within individual channels requires the researcher to identify the causes of variation in the customer experience. The causes of intra-channel variation, is more often than not at the local level – the individual stores, branches, employees, etc. For example, a bank branch with large variation in customer traffic is more likely to experience variation in the customer experience.
Regardless of the source, consistency equals quality.
In our own research, Kinēsis conducted a mystery shop study of six national institutions to evaluate the customer experience at the branch level. In this research, we observed a similar relationship between consistency and quality. The branches in the top quartile in terms of consistency delivered customer satisfaction scores 15% higher than branches in the bottom quartile. But customer satisfaction is a means to an end, not an end goal in and of itself. In terms of an end business objective, such as loyalty or purchase intent, branches in the top quartile of consistency delivered purchase intent ratings 20% higher than branches in the bottom quartile.
Purchase intent and satisfaction with the experience were both measured on a 5-point scale.
Again, it is incumbent on customer experience researchers to identify the causes of inconsistency. A search for the root cause of variation in customer journeys must consider processes cause variation.
One tool to measure process cause variation is a Voice of the Customer (VOC) Table. VOC Tables have a two-fold purpose: First, to identify specific business processes which can cause customer experience variations, and second, to identify which business processes will yield the largest ROI in terms of improving the customer experience.
VOC Tables provide a clear road map to identify action steps using a vertical and horizontal grid. On the vertical axis, each customer experience attribute within a given channel is listed. For each of these attributes a judgment is made about the relative importance of each attribute. This importance is expressed as a numeric value. On the horizontal axis is a exhaustive list of business processes the customer is likely to encounter, both directly and indirectly, in the customer journey.
This grid design matches each business process on the horizontal axis to each service attribute on the vertical axis. Each cell created in this grid contains a value which represents the strength of the influence of each business process listed on the horizontal axis to each customer experience attribute.
Finally, a value is calculated at the bottom of each column which sums the values of the strength of influence multiplied by the importance of each customer experience attribute. This yields a value of the cumulative strength of influence of each business process on the customer experience weighted by its relative importance.
Consider the following example in a retail mortgage lending environment.
In this example, the relative importance of each customer experience attributes was determined by correlating these attributes to a “would recommend” question, which served as a loyalty proxy. This yields an estimate of importance based on the attribute’s strength of relationship to customer loyalty, and populates the far left column. Specific business processes for the mortgage process are listed across the top of this table. Within each cell, an informed judgment has been made regarding the relative strength of the business process’s influence on the customer experience attribute. This strength of influence has been assigned a value of 1 – 3. It is multiplied by the importance measure of each customer experience attribute and summed into a weighted strength of influence – weighted by importance, for each business process.
In this example, the business processes which will yield the highest ROI in terms of driving the customer experience are quote of loan terms (weighted strength of influence 23.9), clearance of exemptions (22.0), explanation of loan terms (20.2), loan application (18.9) and document collection (16.3).
It is incumbent on researchers fielding self-administered surveys to maximize response rates. This reduces the potential for response bias, where the survey results may not accurately reflect the opinions of the entire population of targeted respondents. Previously we discussed ways researchers can increase the likelihood of respondents opening an email survey invitation. This post addresses how to get respondents to actually click on the survey link and participate in the survey.
Make the Invite Easy to Read
Don’t bury the lead. The opening sentence must capture the respondent’s attention and make the investment in effort to read the invitation. Keep in mind most people skim emails. Keep text of the invitation short, paying close attention to paragraph length. The email should be easy to skim.
Give a Reward
Offering respondents a reward for participation is an excellent way to motivate participation. Tangible incentives like a drawing, coupon, or gift card, if appropriate and within the budget, are excellent tools to maximize response rates. However, rewards do not necessarily need to be tangible. Intangible rewards can also prove to be excellent motivators. People, particularly customers who they have a relationship with the brand, want to be helpful. Expressing the importance of their option, and communicating how the brand will use the survey to improve its offering to customers like the respondent is an excellent avenue to leverage intangible rewards to motivate participation.
Intangible rewards are often sufficient if the respondent’s cost to participate in the survey is minimal. Perhaps the largest cost to a potential respondent is the time required to complete the survey. Give them an accurate estimate of the time it takes to complete the survey – and keep it short. We recommend no more than 10 minutes, more preferably five to six. If the research objectives require a longer survey instrument, break the survey into two or three shorter surveys and deliver them separately to different targeted respondents. Do not field excessively long surveys or mis-quote the estimated time to complete the survey – it is rude to impose on your respondents not to mention disastrous to your participation rates – and it’s unethical to mis-quote the survey length. As with getting the participants to open the email – creditability plays a critical role in getting them to click on the survey.
One of the best ways to garner credibility with the survey invite is to assure the participant confidentiality. This is particularly important for customer surveys, where the customers interact commonly with employees. For example, a community bank where customers may interact with bank employees not only in the context of banking but broadly in the community, must ensure customers that their survey response will be kept strictly confidential.
Personalizing the survey with appropriate merge fields is also an excellent way to garner credibility.
Make it as easy as possible for the participant to enter the survey. Program a link to the survey, and make sure it is both visible and presented early in the survey. Again, most people skim the contents of emails, so place the link in the top 1/3 of the email and make it clear that it is a link to enter the survey.
In designing survey invitations, remember to write short, concise, easy to read emails that both leverage respondent’s reward centers (tangible or intangible), and credibly estimate the short time required to complete the survey. This approach will help maximize response rates and avoid some of the pitfalls of response bias. Click here for the next post in this series in prompting respondents to complete the survey.
The business attribute with the highest correlation to profitability is loyalty. Loyalty lowers sales and acquisition costs per guest by amortizing these costs across a longer lifetime – leading to some extraordinary financial results. However, the question remains, what service attributes drive guest loyalty?
To answer this question from a behavioral standpoint Kinesis conducted 400 restaurant mystery shops with the purpose of determining which service attributes/behaviors drive guest return intent. Forty-six service attributes were observed across five dimensions of the guest experience: environment, food & beverage quality, greeting, personal attention and timing of food and beverage delivery.
The attributes measured grouped into these five dimensions as follows:
- Table maintained appropriately throughout the meal
- Dining room clean, organized and well maintained
- Exterior building, parking lot, walkways and planters clean
- Silverware, china, glassware and your table clean
- Men’s restroom clean and stocked with supplies
- Lighting fixtures clean and working
- Lobby area clean and organized
- Menus clean and in good condition
- Women’s restroom clean and stocked with supplies
- Bar clean, organized and well maintained
- Room temperature level comfortable
Food & Beverage Quality
- Entrees presented attractively, and tasted good
- Appetizer presented attractively, and tasted good
- Drinks attractively presented, and tasted good
- Dessert presented attractively, and tasted good
- Greeting made feel welcome
- Prompt greeting
- Staff members greet with a friendly smile as being seated
- Thanked and encouraged to visit again
- Ask specific questions about your experience upon leaving
Service: Personal Attention
- Server attentive and prompt throughout the meal
- Server discuss the beverage menu, suggest an item or ask about your preferences
- Server discuss the appetizer menu, suggest an item or ask about your preferences
- Server promote daily specials
- Host carry on a conversation as being seated
- Server discuss the beverage menu or ask about preferences
- Receive appetizer in a timely manner
- Manager engage guests in conversation
- Server smiling and enjoying time with all the guests
- Acknowledged by a server in a timely manner
- Attentive to needs while in the bar area
- Server discuss the dessert menu, suggest an item or ask about preferences
- Server knowledgeable and confident when responding to questions
- Manager present
- Server try and entice you to order their favorite appetizer(s)
- Resolve any service, food or beverage issues
- Food and beverage service timed well
- Receive entrees in a timely manner
- Receive starter soup/ salad in a timely manner
- Receive appetizer in a timely manner
- Manager engage guests in conversation
- Receive drink orders in timely manner
- Receive dessert in a timely manner
- Cashed out in a timely manner
- Acknowledge and get order in a timely manner
- Drinks arrive in a timely manner
In order to determine the relationship of these attributes to return intent, Kinesis asked mystery shoppers if, based on the guest experience, they intended to return to the restaurant. This independent variable was then used as a basis for cross-tabulation to determine the frequency with which the behaviors were observed in shops with positive return intent and negative return intent.
The results of this cross tabulation is as follows:
|Environment||Shops with …|
|Positive Return Intent||Negative Return Intent|
|Table maintained appropriately throughout the meal||96%||73%|
|Dining room clean, organized and well maintained||100%||90%|
|Exterior building, parking lot, walkways and planters clean||100%||94%|
|Silverware, china, glassware and your table clean||98%||94%|
|Men’s restroom clean and stocked with supplies||96%||91%|
|Lighting fixtures clean and working||98%||95%|
|Lobby area clean and organized||100%||98%|
|Menus clean and in good condition||99%||97%|
|Women’s restroom clean and stocked with supplies||93%||92%|
|Bar clean, organized and well maintained||99%||98%|
|Room temperature level comfortable||95%||94%|
|Food & Beverage Quality||Shops with …|
|Positive Return Intent||Negative Return Intent|
|Entrees presented attractively, and tasted good||98%||58%|
|Appetizer presented attractively, and tasted good||97%||88%|
|Drinks attractively presented, and tasted good||97%||88%|
|Dessert presented attractively, and tasted good||97%||97%|
|Greeting||Positive Return Intent||Negative Return Intent|
|Thanked and encouraged to visit again||95%||63%|
|Ask specific questions about your experience upon leaving||35%||8%|
|Greeting made feel welcome||93%||70%|
|Staff members greet with a friendly smile as being seated||60%||44%|
|Service: Personal Attention||Positive Return Intent||Negative Return Intent|
|Server attentive and prompt throughout the meal||93%||45%|
|Server discuss the beverage menu, suggest an item or ask about your preferences||80%||43%|
|Server discuss the appetizer menu, suggest an item or ask about your preferences||68%||33%|
|Server promote daily specials||64%||33%|
|Host carry on a conversation as being seated||70%||41%|
|Server discuss the beverage menu or ask about preferences||63%||35%|
|Manager engage guests in conversation||73%||47%|
|Server smiling and enjoying time with all the guests||97%||73%|
|Acknowledged by a server in a timely manner||96%||73%|
|Attentive to needs while in the bar area||92%||72%|
|Server discuss the dessert menu, suggest an item or ask about preferences||81%||65%|
|Acknowledge and get order in a timely manner||94%||80%|
|Server knowledgeable and confident when responding to questions||98%||86%|
|Server try and entice you to order their favorite appetizer(s)||64%||57%|
|Resolve any service, food or beverage issues||53%||67%|
|Service: Timing||Positive Return Intent||Negative Return Intent|
|Food and beverage service timed well||92%||51%|
|Receive entrees in a timely manner||92%||59%|
|Server promote daily specials||64%||33%|
|Receive starter soup/ salad in a timely manner||91%||60%|
|Receive appetizer in a timely manner||93%||65%|
|Receive drink orders in timely manner||96%||73%|
|Receive dessert in a timely manner||95%||77%|
|Cashed out in a timely manner||97%||81%|
|Acknowledge and get order in a timely manner||94%||80%|
|Drinks arrive in a timely manner||98%||85%|
Putting all this together, the ten attributes with the largest difference between shops with positive and negative return intent are:
|Top 10 Attributes|
|Service: Personal Attention||Server attentive and prompt throughout the meal||48%|
|Service: Timing||Food and beverage service timed well||41%|
|Food||Entrees presented attractively, and tasted good||40%|
|Service: Personal Attention||Server discuss the beverage menu, suggest an item or ask about your preferences||37%|
|Service: Personal Attention||Server discuss the appetizer menu, suggest an item or ask about your preferences||35%|
|Service: Timing||Receive entrees in a timely manner||33%|
|Service: Personal Attention||Server promote daily specials||31%|
|Greeting||Thanked and encouraged to visit again||31%|
|Service: Timing||Receive starter soup/ salad in a timely manner||30%|
|Service: Personal Attention||Host carry on a conversation as being seated||29%|
Of the ten attributed with the strongest relationship to return intent, five belong to the personal attention dimension, three belong to the timing dimension, the food & beverage quality and greeting dimensions round out the top ten with one attribute each.
Directing our attention from specific attributes to broader dimensions, the following chart shows the average difference in shops with positive return intent to shops with negative return intent:
Outside of the timing of food and beverage delivery, the dimensions of the customer experience with the strongest correlation to return intent are the greeting and personal attention, followed by food and beverage quality and the physical environment.
Every time a guest visits a restaurant they learn something as a result of the experience, and depending on what they learn, they adjust their behavior based on the experience. The guest may stay longer or leave early, purchase more items or purchase less, tell others of their experience, or decide to return for a repeat experience. This link between the experience and guest behaviors offers managers a clear path to manage the guest experience in profitable ways.
Kinēsis conducted a survey of 800 respondents asking them to recall their most recent experience at a restaurant and rate the experience across the following 16 attributes grouped into five dimensions.
|Arrival||Greeted promptly and felt welcomed|
|Greeting friendly and cheerful|
|Service||Food delivered in a timely manner|
|Server attentive to needs|
|Prompt attention of server|
|Friendliness of server|
|Food||Order delivered accurately|
|Taste of food|
|Temperature of food|
|Overall presentation of the food|
|Value||Good value for the money|
|Environment||Server’s appearance neat and clean|
|Restaurant clean, comfortable and appealing|
|Table clean, dry and presentable|
Additionally, to serve as a dependent variable for additional analysis, respondents were asked to rate their return intent on a 5-point scale. We then cross tabulated the responses by positive and negative return intent to determine the frequency in which each attribute is positive in surveys with positive return intent compared to those with negative return intent. This cross tabulation yielded the following results:
Positive Return Intent
Negative Return Intent
|Greeted promptly and felt welcomed||
|Greeting friendly and cheerful||
|Food delivered in a timely manner||
|Server attentive to needs||
|Prompt attention of server||
|Friendliness of server||
|Order delivered accurately||
|Taste of food||
|Temperature of food||
|Overall presentation of the food||
|Good value for the money||
|Server’s appearance neat and clean||
|Restaurant clean, comfortable and appealing||
|Table clean, dry and presentable||
Combining these into one net difference between surveys with positive and negative return intent ranks the attributes in terms of the strength of their relationship to return intent:
Difference Between Positive & Negative Return Intent
Grouping these attributes back into the dimensions described below reveals the following average net strength of change in return intent by dimension:
This analysis gives managers an informed view of which attributes in which to invest. Clearly, the experience attributes that will yield the most ROI in terms of return intent are the quality of the food and perceptions of value for the money. Next is the human element, how are guests greeted and made to feel welcomed and how they are served throughout the visit. Finally, the physical environment plays an important but secondary role to the quality of the meal, perceptions of value and service.The quality of the meal and perceptions of value for the money each have the strongest relationship to return intent, followed by the service, greeting on arrival and the environment.