Research Objectives

  1. Compare the sample composition resulting from a fresh address-based sample (ABS) frame with the sample collected from a state health probability panel of randomly-recruited panel members in the 2023 Minnesota Health Access Survey (MNHA).
  2. Examine differences in responses to a range of healthcare and health outcome measures such as health insurance, health status, and chronic disease prevalence between the two samples.
  3. Assess the positive and negative aspects of using probability panels for state Health research.

Study Design

  • The MNHA is a survey conducted every two years by The State Health Access Data Assistance Center (SHADAC) at the University of Minnesota and the Minnesota Department of Health (MDH) with SSRS as the data collection partner. The goal of this survey is to document health insurance coverage and access to health care for the non-institutionalized population of Minnesota.
  • The 2023 MNHA utilized a combination of an ABS frame and, for the first time, a probability panel recruited sample, with 13,029 completes from the ABS frame and 2,191 from the panel. The general methodology is mail push-to-web.
  • The survey uses a proxy methodology where the respondent answers on behalf of one randomly selected household member of any age, referred to as the “target”. Results represent data related to this target person.
  • The Minnesota Voices on Health Panel is an emergent panel that was created for SHADAC and MDH. The panelists were recruited in two waves, once during a recontact survey consisting of respondents from the 2021 MNHA, and again during the 2023 MNHA.
  • For the 2023 MNHA, the ABS recruitment protocol included three mailings: a letter with either a visible $1 or $2 bill included; a postcard; and a final letter. Sample in high-density Black/African American, low income, and Latino/a/Hispanic areas, as well as a residual stratum, received the third letter in a Fedex envelope. Respondents were also offered a phone number to call in to do the survey.
  • The panel recruitment protocol involved an initial introduction letter and then subsequent email reminders and text messages where permitted. Panelists who do not use online technologies were called to complete the survey. All received a $5 post incentive for participation.
  • The ABS portion of the survey had an AAPOR Response Rate 4 of 16.4%, and the panel portion had a Response Rate of 22.9%.
  • Demographic and response data from the two sample frames are compared here to evaluate the similarities and differences between the two and to contribute to knowledge about how to use probability panels to increase efficiency of health data collection while maintaining data quality.

Principal Findings

Demographic Differences Between Panel and ABS Sample

There are distinct demographic differences between the panel sample and ABS respondents.

  1. The panel sample is older, less racially diverse, more educated, and has a slightly higher income.
  2. Correlations between these characteristics and health-related measures and outcomes contribute to differences in survey results between the sample types.

Health Insurance Status

Looking in aggregate, it appears that the panel sample overall represents a higher percentage of public insurance recipients. Uninsurance is significantly higher among the ABS members, which would be expected with the demographic profile described above.

Examining all panel members and the specific type of public insurance received, there is almost a 20% difference between the two sample sources in rates of Medicare, accounting for the higher public insurance finding for panelists. The higher SES of the panel respondents is reflected in the percent of Medicaid recipients found in the AB sample versus the panel sample, with the ABS having nearly twice as a high of a proportion of Medicaid respondents.

Reporting Of Health Status by Sample Type

Looking only at those younger than 65 to control for the age gap, panel respondents are significantly more likely to characterize their health as excellent or very good than members of the ABS sample, again reflecting expected characteristics of a more affluent, educated group of survey respondents. The basic relationship is the same when the older population is included, but the differences are smaller.

Reporting Of Chronic Condition by Sample Type

Surprisingly considering the SES differences noted and the pattern of health status response, panel respondents report higher rates of having one or more chronic conditions even when looking at only those younger than 65. Overall experience of these conditions is lower among the younger segment of the sample. Further research is needed to understand this finding.

Relevance To Policy Research

Conducting methodologically rigorous statewide health policy research is a challenging and expensive endeavor. One of the goals of such research is often to achieve a representative sample of the geographic and demographic diversity of a state with sufficient statistical power to meet analytic objectives. This requires large sample sizes. Often the populations most in need of effective health policy reform are the most traditionally underrepresented in surveys, requiring oversampling of these groups, generally at great cost. One avenue toward ameliorating these obstacles is creating a panel of randomly sampled survey respondents who agree to be interviewed for future surveys in exchange for small compensation.

As the demographics of our two sample groups demonstrate, the panel is not a panacea for providing access to underrepresented groups. However, completing interviews with more responsive groups through the panel can free up resources for enhanced outreach to populations of interest.

Weighting can correct for certain biases in the sample and combining cross-sectional samples with panel data can strike a balance between representativeness and efficiency. Understanding the relative benefits and skews of these samples will guide researchers and policy makers in determining tradeoffs and optimal balance between panel and new sample.