This blog post is based on a presentation prepared for the 79th AAPOR Conference looking at how survey researchers can optimize post-incentives in an era where data collection is getting more costly; while, at the same time, response rates are declining. In this post we present evidence of how providing a minimum $10 post-incentive can reduce costs and improve response rates. We also report on how different demographic groups – particularly across race/ethnic groups and income brackets – are more likely to accept different incentive amounts.
Research Objectives
- Assess if there is an optimal post-incentive amount following a pre-incentive: Analyze the response rates and cost per interview for different post-incentive amounts (i.e., $5 vs. $10) to identify the minimum post-incentive needed in a general population survey to maximize response rates while minimizing overall costs.
- Analyze the demographic composition of the sample: Identify any significant differences between participants who accept or refuse a $5 post-incentive versus a $10 post-incentive.
- Explore Incentive Preferences Across Key Demographic Groups: Identify trends in incentive preferences (gift card vs. check) in general and among key demographic groups.
Study Design
- The survey used an address-based sampling (ABS) protocol, with the goal of reaching U.S adults, age 19 and older. The ABS survey was a mail, push to web or call-in design.
- SSRS developed a stratified ABS design aimed at increasing the incidence of Black, Hispanic, and low-income respondents.
- To boost cooperation rates, SSRS included a visible cash pre-incentive in the invitation letters. Additionally, a post-incentive (either by check or electronic gift code) was offered to approximately 80% of the sample, specifically targeting groups expected to have lower-than-average response rates based on prior ABS studies—particularly individuals likely to be Black, Hispanic, or low-income. The recruitment protocol also included a reminder postcard and a reminder letter.
- Due to some last-minute budgetary constraints, we needed to look for an opportunity to bring down outgoing costs. For this, SSRS decided to develop an experimental incentivization protocol for the sample offered a pre- and post-incentive.
- A random half was offered a $5 post-incentive, while the other half was offered a $10 post-incentive. Regardless of whether they were offered a $5 or $10 post-incentive, they could choose if they wanted to receive an instant online gift code or a check that would be mailed to them.
Findings
Why $10 Delivers: Better Response Rates, Higher Yields, and Lower Costs
Looking at response rates and yields for the experiment; we find that the $10 post-incentive condition is associated with a 35% improvement in response rate and yield compared with the $5 post-incentive. Specifically, we obtained a 6.7% raw response rate and a 15:1 sample yield for the $10 post-incentive, compared with 4.9% and 20.3:1 yield for the $5 post-incentive condition.
Additionally, the $10 post-incentive condition cost $8.50 less on average per interview than the $5 post-incentive condition. Thus, the $10 post-incentive not only improved response rates or yields, but it was also more cost effective.
$5 Incentive Leads to Twice the Refusals Compared to $10
The refusal rate for the $5 post-incentive was twice as high as the refusal rate for the $10 post-incentive: (15% vs. 7%).
Incentive Preferences Across Demographics: Who Accepted or Refused
Respondents in every income bracket are about twice as likely to decline the $5 post-incentive as compared to the $10 post-incentive. For households earning under $40K, 8% turned down the offer compared to just 4% in the $10 group. In the $40K to under $60K range, the refusal rate jumps to 14% vs. 8%, and for households earning $60K or more, 22% rejected the incentive, compared to only 10% in the $10 group.
Looking at racial and ethnic groups, a similar pattern emerges, with about twice as many respondents decline the $5 incentive compared to the $10 incentive.
Respondents who completed the survey had the option to receive either an instant gift code or a mailed check. Interestingly, while 48% opted for the instant gift code, a notable 41% still preferred the traditional check. Delving deeper into the two incentive conditions, a higher percentage of respondents offered the $10 post-incentive chose the check over the gift code compared to those offered $5 (44% vs. 37%).
The data show that refusal rates for the $5 incentive are driven primarily by higher-income respondents. There is also more variation among those who chose to receive their incentives by check.
For individuals earning less than $40K, about 4 in 10, across both the $5 and $10 incentive groups, selected the check option. However, among respondents earning $40K or more, the proportion choosing a check drops for the $5 incentive group, while refusal rates increase. This trend is especially pronounced among those earning $60K or more: in the $5 group, 31% opted for a check, while 22% refused the incentive entirely. In contrast, in the $10 group, 44% chose the check and 10% refused the incentive.
Relevance for Survey Research
This research shows that there may be a minimum threshold needed to maximize response rates, minimize costs, and make incentives worthwhile. We find that including a $10 post-incentive (along with a pre-incentive) may be the minimum required to maximize response and minimize costs for many ABS studies.
Survey researchers need to be aware of how different demographic groups respond to financial incentives. When incentives are too low, some respondents may choose to participate without accepting the incentive offered, but others may be less likely to participate. Given that we find differences across racial and ethnic groups and across different income brackets, it is important that researchers consider the implications for sample composition among those completing the survey of offering a given incentive amount.
Finally, a large portion of the population, particularly those with lower incomes, still prefer checks to virtual incentives. Additionally, since lower income, Black, and Hispanic respondents are more likely to prefer a check, it is important to take these demographic profiles into account when designing post-incentives structures.
Next Steps
Based on this study and other research, we believe it is important to continue to rethink theories related to incentivization and consider innovative approaches aimed at enhancing willingness to participate in survey research. Our team is currently exploring other incentivization protocols and looking for opportunities to learn more from qualitative research.
