Telephone polling has long provided public officials with valuable information. Phone surveys have asked voters about ballot measures for road-maintenance funding; state or city residents about affordable-housing options; neighborhood residents about higher-density development; and business leaders about the importance of promoting international trade. Focus groups and other forms of qualitative research have supported survey questionnaire development and helped to elaborate survey findings.
All of this is changing. The biggest change? Well, what do you do when your phone rings? More and more, people look at the number and if they don't recognize it, they don't answer. Or if they do answer, they get off the line as quickly as possible -- often without waiting to find out what the survey is really about. A growing refusal to participate in surveys is the single biggest development the opinion-research industry is dealing with. The upshot is that many more phone numbers are needed to complete a valid, statistically reliable survey -- so many more that completing a survey with a representative sample of residents is impossible in many communities. There just aren't enough numbers to call.
And when people do answer the phone and agree to participate in a survey, it's more difficult to keep them on the line as long as in the past. Our era of sound bites and 140-character tweets makes it hard to complete the lengthy questionnaires that government officials are used to fielding in their efforts to gather in-depth information.
The rise of the cellphone represents a third cultural shift. More than four in 10 Americans rely on cellphones alone with no residential landline, and the rate is even higher among young adults and some communities of color. This change has made survey research more expensive. Federal regulations require that cellphone numbers be dialed manually, as opposed to using the auto-dialers that reach landline numbers. Interviewers also must screen respondents to ensure they are in a safe place, and catch them when they are available to talk for possibly an extended period about potentially sensitive topics that require privacy.
Partly in response to these challenges, researchers have begun using professionally recruited and maintained panels for regular online surveys. The best of these consist of people of all different demographics and lifestyles, recruited through different means. Participants receive some form of compensation, similar to the honorariums offered to focus-group participants.
Long disdained by academics and telephone-survey purists, these panels nevertheless are becoming increasingly common. And done well -- using demographic quotas and statistical weighting to assure representative samples -- online panels should be accepted as a legitimate sample source for public-sector surveys. In fact, they offer certain advantages over telephone surveys, including the ability to display visuals, such as pictures and maps; to collect verbatim responses to open-ended questions, yielding more valid content analysis; and to use tradeoff techniques -- pressing respondents to choose between key variables -- that are not possible with telephone-surveys. They are also less expensive.
The evolution of new approaches and blending conventional and new methodologies to adapt to and take advantage of social and technological change is good news for government officials. Knowing what the public thinks about what government is doing -- and is thinking about doing -- is as important as ever.
