Video: Avoiding Bad Survey Design

Categories: SAGE Posts, Tools and Resources

Tags: , ,

“Bad questionnaire design is really the bane of my professional existence. It is immensely frustrating. It is something we see, unfortunately, a lot …” So starts Julia Clark, senior vice president for Ipsos Public Affairs, a for-profit research agency, as she discusses bad survey design. While the flaws she identifies in this SAGE Research Methods video – viewable for free by clicking on the image below — aren’t seen so often in top research companies’ work (like that of Ipsos) they are definitely apparent in places where he enthusiasm is abundant and the oversight lacking.

Her first example is biased question design, which not only creates bad data, but then may be pushed into the public sphere as reliable survey results. Plus, she adds, bias can exist apart just ham-handed questions: “We can’t just think about the question itself when we think about a biased question design. We have to think about the context.”

After setting the scene, Clark then workshops a questionnaire with three other practitioners, who show the technical and ethical issues – and putative solutions – that real-life survey professionals deal with. For example, do you call possible respondents, or do you offer an online interface? The questions and context differ, and so do answers.

“People simply answer questions differently,” Clark explains. “Again, this goes back to the psych principles, to some of the sort of fundamental psychology of not just questionnaire design but of how humans function. You read something on a paper, and you answer it one way. And you talk to someone on the phone, and you’re maybe answering a little bit differently for a huge host of reasons.

SAGE Video offerings are close captioned but also include a searchable transcript that’s clickable to that point in the video, so when O’Leary shows a chart of various reasons for a lit review, you can go directly there in the video by clicking on the text. They also include citation information and the ability to make your own clips from the longer video. You can share via e-mail or social media, save to a playlist, and speed up of slow down play.

 

Julia Clark bad survey design

Julia Clark says the mode of doing a survey — such as by phone or online — will affect the answers. Click image to see the full video.

To view this particular video, for free, click here.

For more about SAGE Video Research Methods Videoclick here.

 

One thought on “Video: Avoiding Bad Survey Design

  1. el vídeo esta interesante me gustaría que mas personas pudieran observarlo , excelente resto del dia

Leave a Reply