The ubiquity of smartphones make them a great resource for gathering all sorts of data. UCSF’s Health e-Heart study is built on that idea, aiming to collect health data on a million people through their smartphones – including survey data. And with the release of ResearchKit, we’re seeing many more researchers jump in as well. In fact, one of the initial modules included in ResearchKit is to conduct surveys.
In this study, researchers signed up just over 600 people and randomized them to 32 question surveys delivered in one of four ways: automated text message, human text message, automated voice calls, and human voice calls.
They found that surveys delivered by text message appeared to get more conscientious responses. They assessed that primarily by looking at “rounding” of answers. In other words, how often answers to questions like “how many movies did you see last year” end in a 0 or 5. They also looked at how often people picked the same choice for questions i.e. picking “B” for all of the questions. And texting seemed to do better here as well.
People also seemed to be more honest via text message, reporting higher rates of socially “undesirable” behaviors like drinks per week, number of sexual partners, and more. That also held true when comparing automated voice interviews to human voice interviews. They speculate that it may simply be that people are more comfortable reporting these kinds of things when there’s not a person on the other end.
There are several assumptions here that are worth noting. First, there is no objective measure of the things being assessed in the survey. So we don’t know whether text messaging got more accurate results. Instead, we have surrogates that suggest people put more thought into their answers. And we assume higher rates of reported behavior are more accurate based on the assumption that people would have no incentive to over-report “undesirable” behaviors.
In addition to these findings, the authors include a great discussion about using these different modalities to perform surveys. For example, they highlight that one advantage of automated text message surveys is that people can multitask. The survey’s take longer to finish, but may be less disruptive than a traditional interview.
As the authors appropriately point out, neither these findings nor common sense would suggest that automated text message surveys are the best choice for every situation. But it does highlight some potential strengths worth considering.