In fielding surveys researchers must be aware of the concepts of error and bias and how they can creep into a survey, potentially making the survey unreliable in ways that cannot be predicted. For example, one source of error is statistical error, where not enough respondents are surveyed to make the results statistically reliable. Another source of error, or bias, is response bias caused by not having a random sample of the targeted population.
A key concept of survey research is randomness of sample selection, in essence to give each member of the targeted survey population an equal chance of being surveyed. Response rates are important in self administered surveys (such as an email surveys), because it is possible non-responders (people who for some reason choose not to complete the survey) have different opinions than those who choose to participate in the survey. As a result, the survey is not purely random. If non-responders are somehow different than responders, the survey results will reflect that difference – thus biasing the research. It is therefore incumbent on researchers to maximize the survey response rate.
Say for example, a bank wants to survey customers after they have completed an online transaction. If customers who love the bank’s online capabilities are more likely to participate in the survey than those who do not like the bank’s online capabilities, the survey results will be biased in favor of a positive view of the bank’s online offering because it is not a representative sample – it is skewed toward customers with the positive view.
It is, again, incumbent on researchers to maximize the response rate as much as possible in self-administered email surveys.
Pre-Survey Awareness Campaign
One strategy to maximize response rates (particularly in a customer survey context) is a pre-survey awareness campaign to make customers aware of the coming survey and encourage participation. Such a campaign can take many forms, such as:
- Letter on company letterhead, signed by a high profile senior executive.
- Statement or billing inserts
- Email in advance of the survey
Each of these three are excellent ways to introduce the survey to respondents and maximize response rates.
The next steps in maximizing response rates in email surveys is passing SPAM filter tests, and prompting the recipient to open the email. The core concept here is credibility – to make the email appear as credible as possible.
The first step to maintaining credibility is to avoid getting caught in SPAM filter tests, the email content should avoid the following:
- Words common in SPAM, like “win” or “free”
- The use of ALL CAPS
- Excessive punctuation
- Special characters
Additionally, do not spoof emails. Spoofing is the forgery of an email header to make it appear it originated from a source other than the actual source. Send emails from your server. (Sometimes Kinesis has clients who want the email to appear to originate from their server. In such cases, we receive the sample from the client, append a unique identifier and send it back to the client to actually be mailed from their servers.)
Perhaps the best strategy to maintain the credibility of the email invite is to conform to Marketing Research Association (MRA) guidelines. These guidelines include:
- Clearly identify the researcher, including phone number, mailing address, and email
- Post privacy policies online and include a link to these policies
- Include a link to opt out of future emails
From and Subject Lines
Both the FROM and SUBJECT lines are critical in getting the respondent to open the email.
The FROM line has be as credible and recognizable as possible, avoiding vague or generic terms like “feedback”. For surveys of customers, the company name or the name of a recognizable representative of the company should be used.
The SUBJECT line must communicate the subject of the email in a credible way that will make the respondent want to open the email. Keep it brief (50 characters or less), clear, concise and credible.
Not only is the content of the email important, but the timing of delivery plays a role in response rates. In our experience sending the survey invitation in the middle of the week (Tuesday – Thursday) during daytime hours increases the likelihood that the email will be noticed by the respondent.
After an appropriate amount of time (typically for our clients 5 days), reminder emails should be sent, politely reminding the respondent of the previous invitation, and highlighting the importance of their opinion. One, perhaps two, reminder emails are appropriate, but do not send more than two.
To maximize the probability that respondents will receive and open the email focus on sending a credible email mid-week, one which will pass SPAM filter tests, contain accurate credible and compelling SUBJECT and FROM lines, and send polite reminder emails to non-responders.
But opening the email is just the first step. The actual objective is to get the respondents to open and complete the survey. Click here for the next post in this series in prompting respondents to participate in the survey.