An Experimental Study Using Opt-in Internet Panel Surveys for Behavioral Health Surveillance

OBJECTIVE: To present the design and preliminary results of a pilot study to investigate the use of opt-in Internet panel surveys for behavioral health surveillance. INTRODUCTION: Today, surveyors in both the private and public sectors are facing considerable challenges with random digit dialed (RDD) landline telephone samples. The population coverage rates for landline telephone surveys are being eroded by wireless-only households, portable telephone numbers, telecommunication barriers (e.g., call forwarding, call blocking and pager connections), technological barriers (call-blocking, busy circuits) and increased refusal rates and privacy concerns. Addressing these issues increasingly drives up the costs associated with dual-frame telephone surveys designed to be representative of the target population as well as hinders their ability to be fully representative of the adult population of each state and territory in the United States. In an effort to continue to meet these challenges head on and assist state and territorial public health professionals in the continued collection of data that are representative of their respective populations, novel approaches to behavioral health surveillance need continued examination. Both private and public sector researchers are evaluating the use of Internet opt-in panels to augment dual-frame RDD survey methods. Compared to dual-frame RDD, opt-in Internet panels offer lower costs, quick data collection and dissemination, and the ability to gather additional data on panelists over time. However, as with dual-frame RDD, this mode has similar challenges with coverage error and non-response. Nevertheless, survey methodologists are moving forward and exploring ways to reduce or eliminate biases between the sample and the target population. METHODS: A collaborative pilot project was designed to assess the feasibility and accuracy of opt-in Internet panel surveys for behavioral health surveillance. This pilot project is a collaboration between the CDC, four state departments of health, opt-in Internet panel providers and the leads of several large surveys and systems such as the Patient-Reported Outcome Measures Information System (PROMIS) and the Cooperative Congressional Election Study (CCES). Pilot projects were conducted in four states (GA, IL, NY, and TX) and four Metropolitan Statistical Areas (Atlanta, Chicago, New York City, and Houston). Data were collected using three different opt-in Internet panels and sampling methods that differ with respect to recruitment strategy, sample selection and sample matching to the adult population of each geography. A question bank consisting of 80 questions was developed to benchmark with other existing surveys used to assess various public health surveillance measures (e.g., the Behavioral Risk Factor Surveillance System, the PROMIS, National Survey on Drug Use and Health, and the CCES). RESULTS: We present comparative analyses that assess the advantages and disadvantages of different opt-in Internet panels sampling methodologies across a range of parameters including cost, geography, timeliness, usability, and ease of use for technology transfer to states and local communities. Recommendations for future efforts in behavioral health surveillance are given based on these results.