Archive | Guest Experience Research RSS for this section

Implications of CX Consistency for Researchers – Part 3 – Common Cause v Special Cause Variation

Previously, we discussed the implications of intra-channel consistency for researchers.

This post considers two types of variation in the customer experience: common and special cause variation, and their implications for customer researchers.

The concepts of common and special cause variation are derived from the process management discipline Six Sigma.

Common cause variation is normal or random variation within the system.  It is statistical noise within the system.   Examples of common cause variation in the customer experience are:

  • Poorly defined, poorly designed, inappropriate policies or procedures
  • Poor design or maintenance of computer systems
  • Inappropriate hiring practices
  • Insufficient training
  • Measurement error

Special cause variation, on the other hand, is not random.  It conforms to laws of probability.   It is the signal within the system.  Examples of special cause variation include:

  • High demand/ high traffic
  • Poor adjustment of equipment
  • Just having a bad day

What are the implications of common and special cause variation for customer experience researchers?

Given the differences between common cause and special cause variation, researchers need a tool to help them distinguish between the two.  Researchers need a means of determining if any observed variation in the customer experience is statistical noise or a signal within the system.  Control charts are a statistical tool to make a determination if variation is noise or a signal.

Control charts track measurements within upper and lower quality control limits.  These quality control limits define statistically significant variation overtime (typically at a 95% confidence), which means there is a 95% probability that the variation is the result of an actual change in the customer experience (special cause variation) not just normal common cause variation.  Observed variation within these quality control limits are common cause variation.  Variation which migrates outside these quality control limits is special cause variation.

To illustrate this concept, consider the following example of mystery shop results:

Mystery Shop Scores

This chart depicts a set of mystery shop scores which both vary from month to month and generally appear to trend upward.

Customer experience researchers need to provide managers a means of determining if the month to month variation is statistical noise or some meaningful signal within the system.  Turning this chart into a control chart by adding statistically defined upper and lower quality control limits will determine if the monthly variation is common or special cause.

To define quality control limits, the customer experience researcher needs to determine the count of observations for each month, the monthly standard deviation, and the average count of shops across all months.

The following table adds these three additional pieces of information into our example:

 

Month

Count of Mystery Shops Average Mystery Shop Scores Standard Deviation of Mystery Shop Scores

May

510 83% 18%

June

496 84% 18%

July

495 82% 20%

Aug

513 83%

15%

Sept 504 83%

15%

Oct 489 85%

14%

Nov 494 85%

15%

Averages 500 83.6%

16.4%

To define the upper and lower quality control limits (UCL and LCL, respectively), apply the following formula:

Where:

x = Grand Mean of the score

n = Mean sample size (number of shops)

SD = Mean standard deviation

 

These equations yield quality control limits at 95% confidence, which means there is a 95% probability any variation observed outside these limits is special cause variation, rather than normal common cause variation within the system

Calculating these quality control limits and applying them to the above chart produces the following control chart, with upper and lower quality control limits depicted in red:

Control Chart

This control chart now answers the question, what variation is common cause and what variation is special cause.  The general trend upward appears to be statistically significant with the most recent month above the upper quality control limit.  Additionally, this control chart identifies a period of special cause variation in July.  With 95% confidence we know some special cause drove the scores below the lower control limit.  Perhaps this special cause was employee turnover, perhaps a new system rollout, or perhaps a weather event that impacted the customer experience.

 

 

Implications of CX Consistency for Researchers – Part 2 – Intra-Channel Consistency

Previously, we discussed the implications of inter-channel consistency for researchers, and introduced a process for management to define a set of employee behaviors which will support the organization’s customer experience goals across multiple channels.

This post considers the implications of intra-channel consistency for customer experience researchers.

As with cross-channel consistency, intra-channel consistency, or consistency within individual channels requires the researcher to identify the causes of variation in the customer experience.  The causes of intra-channel variation, is more often than not at the local level – the individual stores, branches, employees, etc.  For example, a bank branch with large variation in customer traffic is more likely to experience variation in the customer experience.

Regardless of the source, consistency equals quality.

In our own research, Kinēsis conducted a mystery shop study of six national institutions to evaluate the customer experience at the branch level.  In this research, we observed a similar relationship between consistency and quality.  The branches in the top quartile in terms of consistency delivered customer satisfaction scores 15% higher than branches in the bottom quartile.  But customer satisfaction is a means to an end, not an end goal in and of itself.  In terms of an end business objective, such as loyalty or purchase intent, branches in the top quartile of consistency delivered purchase intent ratings 20% higher than branches in the bottom quartile.

Satisfaction and purchase intent by customer experience consistency

Purchase intent and satisfaction with the experience were both measured on a 5-point scale.

Again, it is incumbent on customer experience researchers to identify the causes of inconsistency.   A search for the root cause of variation in customer journeys must consider processes cause variation.

One tool to measure process cause variation is a Voice of the Customer (VOC) Table. VOC Tables have a two-fold purpose:  First, to identify specific business processes which can cause customer experience variations, and second, to identify which business processes will yield the largest ROI in terms of improving the customer experience.

VOC Tables provide a clear road map to identify action steps using a vertical and horizontal grid.  On the vertical axis, each customer experience attribute within a given channel is listed.  For each of these attributes a judgment is made about the relative importance of each attribute.  This importance is expressed as a numeric value.   On the horizontal axis is a exhaustive list of business processes the customer is likely to encounter, both directly and indirectly, in the customer journey.

This grid design matches each business process on the horizontal axis to each service attribute on the vertical axis.  Each cell created in this grid contains a value which represents the strength of the influence of each business process listed on the horizontal axis to each customer experience attribute.

Finally, a value is calculated at the bottom of each column which sums the values of the strength of influence multiplied by the importance of each customer experience attribute.  This yields a value of the cumulative strength of influence of each business process on the customer experience weighted by its relative importance.

Consider the following example in a retail mortgage lending environment.

VOC Table

In this example, the relative importance of each customer experience attributes was determined by correlating these attributes to a “would recommend” question, which served as a loyalty proxy.  This yields an estimate of importance based on the attribute’s strength of relationship to customer loyalty, and populates the far left column.  Specific business processes for the mortgage process are listed across the top of this table.  Within each cell, an informed judgment has been made regarding the relative strength of the business process’s influence on the customer experience attribute.  This strength of influence has been assigned a value of 1 – 3.  It is multiplied by the importance measure of each customer experience attribute and summed into a weighted strength of influence – weighted by importance, for each business process.

In this example, the business processes which will yield the highest ROI in terms of driving the customer experience are quote of loan terms (weighted strength of influence 23.9), clearance of exemptions (22.0), explanation of loan terms (20.2), loan application (18.9) and document collection (16.3).

Next, we will look into the concepts of common and special cause variation, and another research methodology designed to identify areas for attention. Control charts as just such a tool.

Maximizing Response Rates: Get Respondents to Start the Survey

147953294 Resize

It is incumbent on researchers fielding self-administered surveys to maximize response rates.  This reduces the potential for response bias, where the survey results may not accurately reflect the opinions of the entire population of targeted respondents. Previously we discussed ways researchers can increase the likelihood of respondents opening an email survey invitation.  This post addresses how to get respondents to actually click on the survey link and participate in the survey.

Make the Invite Easy to Read

Don’t bury the lead.   The opening sentence must capture the respondent’s attention and make the investment in effort to read the invitation.   Keep in mind most people skim emails.  Keep text of the invitation short, paying close attention to paragraph length.  The email should be easy to skim.

Give a Reward

Offering respondents a reward for participation is an excellent way to motivate participation.  Tangible incentives like a drawing, coupon, or gift card, if appropriate and within the budget, are excellent tools to maximize response rates.   However, rewards do not necessarily need to be tangible.  Intangible rewards can also prove to be excellent motivators.  People, particularly customers who they have a relationship with the brand, want to be helpful.  Expressing the importance of their option, and communicating how the brand will use the survey to improve its offering to customers like the respondent is an excellent avenue to leverage intangible rewards to motivate participation.

Survey Length

Intangible rewards are often sufficient if the respondent’s cost to participate in the survey is minimal.  Perhaps the largest cost to a potential respondent is the time required to complete the survey.  Give them an accurate estimate of the time it takes to complete the survey – and keep it short.  We recommend no more than 10 minutes, more preferably five to six.   If the research objectives require a longer survey instrument, break the survey into two or three shorter surveys and deliver them separately to different targeted respondents.  Do not field excessively long surveys or mis-quote the estimated time to complete the survey – it is rude to impose on your respondents not to mention disastrous to your participation rates – and it’s unethical to mis-quote the survey length.  As with getting the participants to open the email – creditability plays a critical role in getting them to click on the survey.

Credibility

One of the best ways to garner credibility with the survey invite is to assure the participant confidentiality.  This is particularly important for customer surveys, where the customers interact commonly with employees.  For example, a community bank where customers may interact with bank employees not only in the context of banking but broadly in the community, must ensure customers that their survey response will be kept strictly confidential.

Personalizing the survey with appropriate merge fields is also an excellent way to garner credibility.

Survey Entry

Make it as easy as possible for the participant to enter the survey.  Program a link to the survey, and make sure it is both visible and presented early in the survey.  Again, most people skim the contents of emails, so place the link in the top 1/3 of the email and make it clear that it is a link to enter the survey.

In designing survey invitations, remember to write short, concise, easy to read emails that both leverage respondent’s reward centers (tangible or intangible), and credibly estimate the short time required to complete the survey.  This approach will help maximize response rates and avoid some of the pitfalls of response bias. Click here for the next post in this series in prompting respondents to complete the survey.

Click Here For More Information About Kinesis' Research Services

Guest Return Intent Drivers in the Restaurant Experience

Young couple in restaurant

The business attribute with the highest correlation to profitability is loyalty.  Loyalty lowers sales and acquisition costs per guest by amortizing these costs across a longer lifetime – leading to some extraordinary financial results.  However, the question remains, what service attributes drive guest loyalty?

To answer this question from a behavioral standpoint Kinesis conducted 400 restaurant mystery shops with the purpose of determining which service attributes/behaviors drive guest return intent.  Forty-six service attributes were observed across five dimensions of the guest experience: environment, food & beverage quality, greeting, personal attention and timing of food and beverage delivery.

The attributes measured grouped into these five dimensions as follows:

Environment

  • Table maintained appropriately throughout the meal
  • Dining room clean, organized and well maintained
  • Exterior building, parking lot, walkways and planters clean
  • Silverware, china, glassware and your table clean
  • Men’s restroom clean and stocked with supplies
  • Lighting fixtures clean and working
  • Lobby area clean and organized
  • Menus clean and in good condition
  • Women’s restroom clean and stocked with supplies
  • Bar clean, organized and well maintained
  • Room temperature level comfortable

Food & Beverage Quality

  • Entrees presented attractively, and tasted good
  • Appetizer presented attractively, and tasted good
  • Drinks attractively presented, and tasted good
  • Dessert presented attractively, and tasted good

Greeting

  • Greeting made feel welcome
  • Prompt greeting
  • Staff members greet with a friendly smile as being seated
  • Thanked and encouraged to visit again
  • Ask specific questions about your experience upon leaving

Service: Personal Attention

  • Server attentive and prompt throughout the meal
  • Server discuss the beverage menu, suggest an item or ask about your preferences
  • Server discuss the appetizer menu, suggest an item or ask about your preferences
  • Server promote daily specials
  • Host carry on a conversation as being seated
  • Server discuss the beverage menu or ask about preferences
  • Receive appetizer in a timely manner
  • Manager engage guests in conversation
  • Server smiling and enjoying time with all the guests
  • Acknowledged by a server in a timely manner
  • Attentive to needs while in the bar area
  • Server discuss the dessert menu, suggest an item or ask about preferences
  • Server knowledgeable and confident when responding to questions
  • Manager present
  • Server try and entice you to order their favorite appetizer(s)
  • Resolve any service, food or beverage issues

Service: Timing

  • Food and beverage service timed well
  • Receive entrees in a timely manner
  • Receive starter soup/ salad in a timely manner
  • Receive appetizer in a timely manner
  • Manager engage guests in conversation
  • Receive drink orders in timely manner
  • Receive dessert in a timely manner
  • Cashed out in a timely manner
  • Acknowledge and get order in a timely manner
  • Drinks arrive in a timely manner

 

In order to determine the relationship of these attributes to return intent, Kinesis asked mystery shoppers if, based on the guest experience, they intended to return to the restaurant.  This independent variable was then used as a basis for cross-tabulation to determine the frequency with which the behaviors were observed in shops with positive return intent and negative return intent.

The results of this cross tabulation is as follows:

Environment Shops with …
Positive Return Intent Negative Return Intent
Table maintained appropriately throughout the meal 96% 73%
Dining room clean, organized and well maintained 100% 90%
Exterior building, parking lot, walkways and planters clean 100% 94%
Silverware, china, glassware and your table clean 98% 94%
Men’s restroom clean and stocked with supplies 96% 91%
Lighting fixtures clean and working 98% 95%
Lobby area clean and organized 100% 98%
Menus clean and in good condition 99% 97%
Women’s restroom clean and stocked with supplies 93% 92%
Bar clean, organized and well maintained 99% 98%
Room temperature level comfortable 95% 94%

 

Food & Beverage Quality Shops with …
Positive Return Intent Negative Return Intent
Entrees presented attractively, and tasted good 98% 58%
Appetizer presented attractively, and tasted good 97% 88%
Drinks attractively presented, and tasted good 97% 88%
Dessert presented attractively, and tasted good 97% 97%

 

Greeting Positive Return Intent Negative Return Intent
Thanked and encouraged to visit again 95% 63%
Ask specific questions about your experience upon leaving 35% 8%
Greeting made feel welcome 93% 70%
Prompt greeting 93% 76%
Staff members greet with a friendly smile as being seated 60% 44%

 

Service: Personal Attention Positive Return Intent Negative Return Intent
Server attentive and prompt throughout the meal 93% 45%
Server discuss the beverage menu, suggest an item or ask about your preferences 80% 43%
Server discuss the appetizer menu, suggest an item or ask about your preferences 68% 33%
Server promote daily specials 64% 33%
Host carry on a conversation as being seated 70% 41%
Server discuss the beverage menu or ask about preferences 63% 35%
Manager engage guests in conversation 73% 47%
Server smiling and enjoying time with all the guests 97% 73%
Acknowledged by a server in a timely manner 96% 73%
Attentive to needs while in the bar area 92% 72%
Server discuss the dessert menu, suggest an item or ask about preferences 81% 65%
Acknowledge and get order in a timely manner 94% 80%
Server knowledgeable and confident when responding to questions 98% 86%
Manager present 43% 31%
Server try and entice you to order their favorite appetizer(s) 64% 57%
Resolve any service, food or beverage issues 53% 67%

 

Service: Timing Positive Return Intent Negative Return Intent
Food and beverage service timed well 92% 51%
Receive entrees in a timely manner 92% 59%
Server promote daily specials 64% 33%
Receive starter soup/ salad in a timely manner 91% 60%
Receive appetizer in a timely manner 93% 65%
Receive drink orders in timely manner 96% 73%
Receive dessert in a timely manner 95% 77%
Cashed out in a timely manner 97% 81%
Acknowledge and get order in a timely manner 94% 80%
Drinks arrive in a timely manner 98% 85%

 

Putting all this together, the ten attributes with the largest difference between shops with positive and negative return intent are:

Top 10 Attributes
Dimension Attributes Difference
Service: Personal Attention Server attentive and prompt throughout the meal 48%
Service: Timing Food and beverage service timed well 41%
Food Entrees presented attractively, and tasted good 40%
Service: Personal Attention Server discuss the beverage menu, suggest an item or ask about your preferences 37%
Service: Personal Attention Server discuss the appetizer menu, suggest an item or ask about your preferences 35%
Service: Timing Receive entrees in a timely manner 33%
Service: Personal Attention Server promote daily specials 31%
Greeting Thanked and encouraged to visit again 31%
Service: Timing Receive starter soup/ salad in a timely manner 30%
Service: Personal Attention Host carry on a conversation as being seated 29%

 

Of the ten attributed with the strongest relationship to return intent, five belong to the personal attention dimension, three belong to the timing dimension, the food & beverage quality and greeting dimensions round out the top ten with one attribute each.

Directing our attention from specific attributes to broader dimensions, the following chart shows the average difference in shops with positive return intent to shops with negative return intent:

Return Intent Gaps

Outside of the timing of food and beverage delivery, the dimensions of the customer experience with the strongest correlation to return intent are the greeting and personal attention, followed by food and beverage quality and the physical environment.

Click Here For More Information About Kinesis' Research Services

Taste, Value & Service: Guest Return Intent Drivers

Every time a guest visits a restaurant they learn something as a result of the experience, and depending on what they learn, they adjust their behavior based on the experience.  The guest may stay longer or leave early, purchase more items or purchase less, tell others of their experience, or decide to return for a repeat experience.  This link between the experience and guest behaviors offers managers a clear path to manage the guest experience in profitable ways.

Kinēsis conducted a survey of 800 respondents asking them to recall their most recent experience at a restaurant and rate the experience across the following 16 attributes grouped into five dimensions.

Dimension Attribute
Arrival Greeted promptly and felt welcomed
Greeting friendly and cheerful
Seated quickly
Service Food delivered in a timely manner
Server attentive to needs
Prompt attention of server
Friendliness of server
Food Order delivered accurately
Taste of food
Temperature of food
Overall presentation of the food
Value Good value for the money
Portion appropriate
Environment Server’s appearance neat and clean
Restaurant clean, comfortable and appealing
Table clean, dry and presentable

Additionally, to serve as a dependent variable for additional analysis, respondents were asked to rate their return intent on a 5-point scale.  We then cross tabulated the responses by positive and negative return intent to determine the frequency in which each attribute is positive in surveys with positive return intent compared to those with negative return intent. This cross tabulation yielded the following results:

Attribute

Positive Return Intent

Negative Return Intent

Greeted promptly and felt welcomed

75%

61%

Greeting friendly and cheerful

81%

58%

Seated quickly

74%

65%

Food delivered in a timely manner

79%

58%

Server attentive to needs

80%

58%

Prompt attention of server

74%

66%

Friendliness of server

77%

58%

Order delivered accurately

75%

65%

Taste of food

94%

34%

Temperature of food

76%

61%

Overall presentation of the food

80%

58%

Good value for the money

86%

49%

Portion appropriate

76%

62%

Server’s appearance neat and clean

75%

65%

Restaurant clean, comfortable and appealing

78%

63%

Table clean, dry and presentable

73%

67%

Combining these into one net difference between surveys with positive and negative return intent ranks the attributes in terms of the strength of their relationship to return intent:

Difference Between Positive & Negative Return Intent

Difference Between Positive and Negative Return Intent

Grouping these attributes back into the dimensions described below reveals the following average net strength of change in return intent by dimension:

Dimension's Difference Between Positive & Negative Return Intent

This analysis gives managers an informed view of which attributes in which to invest.  Clearly, the experience attributes that will yield the most ROI in terms of return intent are the quality of the food and perceptions of value for the money.  Next is the human element, how are guests greeted and made to feel welcomed and how they are served throughout the visit.  Finally, the physical environment plays an important but secondary role to the quality of the meal, perceptions of value and service.The quality of the meal and perceptions of value for the money each have the strongest relationship to return intent, followed by the service, greeting on arrival and the environment.

Click Here For More Information About Kinesis'; Guest Experience Research Services