In a previous post we discussed the importance of research objectives in program design. A natural progression of this subject is using research objectives to design a successful questionnaire.
All too often, I find clients who have gone online, found a questionnaire and implemented it into a survey process, in effect, handing research design over to an anonymous author on the Internet who has given no consideration to their specific needs. Inexperience with both the art and science of questionnaire design, conspires to cause them to miss out on building a research tool customized to their specific need.
While questionnaire design is a professional skill fraught with many perils for the inexperienced, the following process will eliminate some common mistakes.
First, define research objectives. Do not skip this step. Defining research objectives prior to making any other decisions about the program is by far the most effective way to make sure your program stays on track, on budget, and produces results that drive business success. See the previous post regarding research objectives. Once a set of objectives has been defined questionnaire design naturally falls out of the process; simply write a survey question for each objective.
For example, consider the following objective set:
1. Determine the level of customer satisfaction and provide a reference point for other satisfaction-based analysis.
2. Identify which service attributes drive satisfaction and which investments yield the greatest improvement in customer satisfaction.
3. Identify moments of truth where the danger of customer attrition is highest.
4. Track changes in customer satisfaction over time.
For each objective write a survey question. For the first objective, (overall satisfaction) write an overall satisfaction question. For objective #2 (attribute satisfaction) develop a list of service attributes and measure satisfaction relative to each. Continue the process for each objective for which a survey question can be written.
Question order is important and the placement of every question should be considered to avoid introducing bias into the survey as a result of question order. Generally, we like to place overall satisfaction questions early in the survey to avoid biasing the results with later attribute questions.
Similarly, question phrasing needs to be carefully considered to avoid biasing the responses. Keep phrasing neutral to avoid biasing the respondents one way or the other. Sure there is a temptation to use overly positive language with your customers, but this really is a bad practice.
Finally, anticipate the analysis. As you write the questionnaire, consider how the results will be reported and analyzed. Anticipating the analysis will make sure the survey instrument captures the data needed for the desired analysis.
Research design is a professional art. If you are not sure what you are doing, seek a professional to help you rather than field poor research with a do-it-yourself tool.
How do you make research actionable?
With the advent of do-it-yourself survey tools, there is a trend away from professional research design processes. One can search on line for a questionnaire, grab it off the internet, and field it on the cheap with a do-it-yourself survey tool with no consideration of the research needs at hand. This, in effect, hands research design over to an anonymous author on the Internet who has given no consideration to your specific needs.
Defining research objectives prior to making any other decisions about the program is by far the most effective way to make sure your program stays on track, on budget, and produces results that drive business success. It sounds very simple, and for the most part it is, however, I’m always surprised when I ask potential clients what their research objectives are how many cannot list anything other than the most general of objectives.
Defining research objectives is a fairly simple process. First, generate a list of everything you want to know as a result of the research.
For example, you may come up with the following list:
- How satisfied are our customers?
- Which key factors drive satisfaction among our customers?
- What are the causes of customer dissatisfaction?
- How can we measure customer satisfaction over time?
- Which business processes can most improve customer satisfaction and increase our financial returns?
- How can we measure the relationships between customer satisfaction, profitability and purchase or retention behavior?
- How can we evaluate our customers’ referral activity?
- How can we measure the value of our customers’ purchasing behavior?
- How can we identify changes in our customers’ purchasing or referral behaviors over time?
Note, these are not survey questions; they are questions to which you want answers. This is what you want to know.
Once you have developed a list of what you want to know as a result of the research, the next step is to map each of your questions to a specific research objective. For each question you should write a clear objective starting with a verb such as: determine, identify, track, link, measure, etc. Starting with verbs is excellent way to make sure you can take action on the results.
So, continuing with the example, the above list of questions may map into the following set of research objectives:
|What do you want to know?||Objectives|
|How satisfied are our customers?||Determine the level of customer satisfaction and provide a reference point for other satisfaction-based analysis.|
|Which key factors drive satisfaction among our customers?||Identify which service attributes drive satisfaction and which investments yield the greatest improvement in customer satisfaction.|
|What are the causes of customer dissatisfaction?||Identify moments of truth where the danger of customer attrition is highest.|
|How can we measure customer satisfaction over time?||Track changes in customer satisfaction over time. Determine if changes in satisfaction are significant.|
|Which business processes can most improve customer satisfaction and increase our financial returns?||Link key service attributes to specific business processes. Identify which processes maximize ROI.|
|How can we measure the relationships between customer satisfaction, profitability and purchase or retention behavior?||Identify the relationship between customer satisfaction and customer behaviors such as retention, purchase behavior, and likelihood of referral, which drive profitability.|
|How can we evaluate our customers’ referral activity?||Conduct loyalty-based customer satisfaction analysis, using net promoters and customer advocacy as a measurement for customer loyalty.|
|How can we measure the value of our customers’ purchasing behavior?||Determine the relationship between customer satisfaction and purchase behavior. Identify the ROI of satisfaction-based management.Make a financial case to all stakeholders (management, employees and shareholders) that the customer experience impacts financial performance.|
|How can we identify changes in our customers’ purchasing or referral behaviors over time?||Continue to track the relationship between satisfaction and purchase behavior. Analyze satisfaction by customer segments and the financial value of each individual segment.|
Once a clear set of research objectives is defined, you now have a road map to inform all subsequent decisions about sample frame, data collection, survey instrument, and analysis plan. Each of these issues deserves more attention than can be addressed in what is intended to be a brief blog post. In future posts, we will look into each of these issues individually.