Research for Social Good is a MethodSpace focus for October. We are delving into this broad topic with guest posts, interviews, and links to articles or instructional resources.
Participation and voice relate to research for social good. In research terminology we discuss population and sample. Regardless of the research approach or data collection method, someone’s voice is being represented, and others’ voices are not. The researcher must choose, whether we are talking about interviews for qualitative study or selection of Big Data in a quantitative study.
Obviously, no matter how large the study, we can’t collect data from everyone. But I am concerned that some are consistently left out, because they are hard to reach, time-consuming to find and engage, or simply outside the researchers’ comfort zone.
Questions about population and sample also tie into researcher positionality and potential researcher bias. Is an article about problems in a culture different from one’s own told from the perspective of the researcher, or from those living and working within that culture? Is the position of the researcher made clear, and biases acknowledged? Qualitative researchers are more accustomed to discussing these matters, but I would argue that all researchers make choices, and those choices are often influenced by the place and culture we are coming from.
How are today’s scholars are thinking about these issues? I did a search in SAGE journals that focus on Big Data research and qualitative methods. Here are abstracts and links to a collection of open access articles that explore the issue of representation in research and offer strategies for researchers who want to be inclusive. If you have other examples to share, please post them in the comment area.
by Meadows, L. M., Lagendyk, L. E., Thurston, W. E. and Eisener, A. C. (Meadows, Lagendyk, Thurston, & Eisener, 2003)
Including Aboriginal women in qualitative health research expands our understanding of factors that contribute to their health and well-being. As part of the larger WHEALTH study, we gathered qualitative health data on midlife Aboriginal women living both on and off reserves. Despite careful planning and a commitment to methodological congruence and purposiveness we encountered a number of challenges that raised ethical questions. We present how we addressed these issues as we attempted to produce ethical, culturally sensitive, and sound research in a timely fashion. This article provides important considerations for other researchers and funding bodies while illustrating the benefits of working with Aboriginal women as an under researched population.
Finding the Hidden Participant: Solutions for Recruiting Hidden, Hard-to-Reach, and Vulnerable Populations
by Amy Ellard-Gray, Nicole K. Jeffrey, Melisa Choubak (Ellard-Gray, Jeffrey, Choubak, & Crann, 2015)
Certain social groups are often difficult for researchers to access because of their social or physical location, vulnerability, or otherwise hidden nature. This unique review article based on both the small body of relevant literature and our own experiences as researchers is meant as a guide for those seeking to include hard-to-reach, hidden, and vulnerable populations in research. We make recommendations for research process starting from early stages of study design to dissemination of study results. Topics covered include participant mistrust of the research process; social, psychological, and physical risks to participation; participant resource constraints; and challenges inherent in nonprobability sampling, snowball sampling, and derived rapport. This article offers broadly accessible solutions for qualitative researchers across social science disciplines attempting to research a variety of different populations.
by Jackie Sanders, Robyn Munford (Sanders & Munford, 2017)
This article elaborates upon a model used to engage marginalized young people in a longitudinal study of youth transitions. The model PARTH elaborates upon a set of principles that were successfully used to engage marginalized young people in a 6-year project. PARTH principles focus researcher attention on the ways they think about and relate to young people and they support researchers to empowerment and the exercise of personal agency by young people. The model was instrumental in achieving high retention rates that exceeded 89% across the 6-year study.
by Evalina van Wijk and Tracie Harrison (van Wijk & Harrison, 2013)
The purpose of the researcher’s study was to examine the meaning that intimate partners of female rape victims attached to their lived experiences after the rape. The conduct of qualitative research concerning non-offending partners of female rape victims, however, often involves multifaceted ethical and practical challenges, which can be managed through the use of pilot studies. The pilot study described in this report had three objectives. The first was to pretest and refine the proposed method for locating, accessing, and recruiting intimate partners of female rape victims, within the first two weeks after the rape, for participation in a six-month longitudinal study. The second objective was to identify and prevent all possible risk factors in the proposed recruitment and data collection methods that could harm the participants’ safety during the main study. The third objective was to determine the feasibility of the main study, in terms of the limited financial and human resources available. The pilot phase was valuable in identifying ethical and methodological problems during the recruitment of participants and collection of data. It allowed for methodological adjustments prior to the main study and confirmed the feasibility of the overall research design. A pilot, pretesting phase is therefore seen as an essential component of a qualitative study involving a vulnerable population.
by Brooke Foucault Welles (Welles, 2014)
In this essay, I make the case for choosing to examine small subsets of Big Data datasets—making big data small. Big Data allows us to produce summaries of human behavior at a scale never before possible. But in the push to produce these summaries, we risk losing sight of a secondary but equally important advantage of Big Data—the plentiful representation of minorities. Women, minorities and statistical outliers have historically been omitted from the scientific record, with problematic consequences. Big Data affords the opportunity to remedy those omissions. However, to do so, Big Data researchers must choose to examine very small subsets of otherwise large datasets. I encourage researchers to embrace an ethical, empirical and epistemological stance on Big Data that includes minorities and outliers as reference categories, rather than the exceptions to statistical norms.
Bas Hofstra and Niek C. de Schipper (Hofstra & de Schipper, 2018)
Social scientists increasingly use (big) social media data to illuminate long-standing substantive questions in social science research. However, a key challenge of analyzing such data is their lower level of individual detail compared to highly detailed survey data. This limits the scope of substantive questions that can be addressed with these data. In this study, we provide a method to upgrade individual detail in terms of ethnicity in data gathered from social media via the use of register data. Our research aim is twofold: first, we predict the most likely value of ethnicity, given one’s first name, and second, we show how one can test hypotheses with the predicted values for ethnicity as an independent variable while simultaneously accounting for the uncertainty in these predictions. We apply our method to social network data collected from Facebook. We illustrate our approach and provide an example of hypothesis testing using our procedure, i.e., estimating the relation between predicted network ethnic homogeneity on Facebook and trust in institutions. In a comparison of our method with two other methods, we find that our method provides the most conservative tests of hypotheses. We discuss the promise of our approach and pinpoint future research directions.
Recruiting and retaining service agencies and public health providers in longitudinal studies: Implications for community-engaged implementation research
Rogério M Pinto, Susan S Witte, Melanie M Wall, Prema L Filippone (Pinto, Witte, Wall, & Filippone, 2018)
This article addresses a lack of attention in the implementation science literature regarding how to overcome recruitment and retention challenges in longitudinal studies involving large samples of service agencies and health service providers (“providers”). Herein, we provide a case-illustration of procedures that improved recruitment and retention in a longitudinal, mixed-method study—Project Interprofessional Collaboration Implementation—funded by the US National Institute of Mental Health. Project Interprofessional Collaboration Implementation included counselors, program workers, educators, and supervisors. We present a research-engagement model to overcome barriers that included developing a low-burden study, social gatherings to engage stakeholders, protocols to recruit agencies and providers, comprehensive record-keeping, research procedures as incentives to participation, a plan to retain hard-to-reach participants, and strategies for modifying incentives over time. Using our model, we retained 36 agencies over the life of the project. Between baseline (N = 379) and 12-month follow-up (N = 285), we retained 75% of the sample and between the 12- (N = 285) and 24-month follow-ups (N = 256), we retained 90%. For qualitative interviews (between baseline and 12-month follow-up and between 12- and 24-month follow-ups), we retained 100% of the sample (N = 20). We provide a summary of frequency of contacts required to initiate data collection and time required for data collection. The model responded to environmental changes in policy and priorities that would not have been achievable without the expertise of community partners. To recruit and retain large samples longitudinally, researchers must strategically engage community partners. The strategies imbedded in our model can be performed with moderate levels of effort and human resources. Creating opportunities for research partners to participate in all phases of the research cycle is recommended, which can help build research capacity for future research.
By Salaam Semaan (Semaan, 2010)
Time-space sampling (TSS; also referred to as time-location sampling, TLS) and respondent-driven sampling(RDS) are strategies that can be used for sampling hard-to-reach populations, for whom it is difficult to construct a sampling frame of the individual members of the population. With proper planning, execution, weighting, and analysis of relevant sampling-related data, both strategies have the potential to produce samples that are representative of the target populations. TSS is a probability-based strategy for recruiting members of a target population congregating at specific locations and times. RDS is predicated on the recognition that project participants are better able than project staff to locate and refer to the study site other potential participants; peers from the target population with whom they have an established relationship. Capture-recapture analysis can incorporate TSS and RDS data to estimate the size of a hard-to-reach population. TSS and RDS have been used extensively around the world in public health projects with populations at high risk for HIV infection. The collective experience gained from using TSS and RDS in HIV-related projects can be valuable in using these sampling strategies with other hard-to-reach populations in projects related to economics, political science, or sociology. Although TSS and RDS have specific strengths and limitations in terms of their abilities to produce valid results that enhance generalizability of findings, the choice of a particular sampling strategy depends on characteristics of the target population and the goal and resources of the project. Proper planning, monitoring, and evaluation of the sampling strategy and attention to logistical, regulatory, and ethical considerations are important to the successful implementation and effectiveness of the sampling strategy.
Update on respondent-driven sampling: Theory and practical considerations for studies of persons who inject drugs
by Lucie Léon, Don Des Jarlais, Marie Jauffret-Roustide(Léon, Des Jarlais, Jauffret-Roustide, & Le Strat, 2016)
In the last 5 years, more than 600 articles using respondent-driven sampling has been published. This article aims to provide an overview of this sampling technique with an update on the key questions that remain when using respondent-driven sampling, with regard to its application and estimators. Respondent-driven sampling was developed by Heckathorn in 1997 and was based on the principle of individuals recruiting other individuals, who themselves were recruited in previous waves. When there is no sampling frame, respondent-driven sampling has demonstrated its ability to capture individuals belonging to “hidden” or “hard-to-reach” populations in numerous epidemiological surveys. People who use drugs, sex workers, or men who have sex with men are notable examples of specific populations studied using this technique, particularly by public agencies such as the Centers for Disease Control and Prevention in the United States. Respondent-driven sampling, like many others, is based on a set of assumptions that, when respected, can ensure an unbiased estimator. Based on a literature review, we will discuss, among other topics, the effect of violating these assumptions. A special focus is made on surveys of persons who inject drugs. Publications show two major thrusts—methodological and applied researches—for providing practical recommendations in conducting respondent-driven sampling studies. The reasons why respondent-driven sampling did not work for a given population of interest will usually provide important insights for designing health-promoting interventions for that population.
See related article: Sampling Hard-to-Reach Populations with Respondent Driven Sampling Lisa G. Johnston, Keith Sabin (Johnston & Sabin, 2010)
Where the Rubber Meets the Road: Probability and Nonprobability Moments in Experiment, Interview, Archival, Administrative, and Ethnographic Data Collection
by Samuel R. Lucas, (Lucas, 2016)
Sociologists use data from experiments, ethnographies, survey interviews, in-depth interviews, archives, and administrative records. Analysts disagree, however, on whether probability sampling is necessary for each method. To address the issue, the author introduces eight dimensions of data collection, places each method within those dimensions, and uses that resource to assess the necessity and feasibility of probability sampling for each method. The author finds that some methods often seen as unique are not, whereas others’ unique natures are confirmed. More surprisingly, some methods for which probability sampling is rare were found to require it, whereas one for which probability sampling is usually believed to be impossible was found to easily use it. Efforts to salvage nonprobability samples and eight additional general justifications for nonprobability sampling are addressed. Advice for individual analysts and counsel for collective responses to improve research are offered.
Ellard-Gray, A., Jeffrey, N. K., Choubak, M., & Crann, S. E. (2015). Finding the hidden participant: Solutions for recruiting hidden, hard-to-reach, and vulnerable populations. International Journal of Qualitative Methods, 14(5), 1609406915621420. doi:10.1177/1609406915621420
Hofstra, B., & de Schipper, N. C. (2018). Predicting ethnicity with first names in online social media networks. Big Data & Society, 5(1), 2053951718761141. doi:10.1177/2053951718761141
Johnston, L. G., & Sabin, K. (2010). Sampling hard-to-reach populations with respondent driven sampling. Methodological Innovations Online, 5(2), 38-48. doi:10.4256/mio.2010.0017
Léon, L., Des Jarlais, D., Jauffret-Roustide, M., & Le Strat, Y. (2016). Update on respondent-driven sampling: Theory and practical considerations for studies of persons who inject drugs. Methodological Innovations, 9, 2059799116672878. doi:10.1177/2059799116672878
Lucas, S. R. (2016). Where the rubber meets the road: probability and nonprobability moments in experiment, interview, archival, administrative, and ethnographic data collection. Socius, 2, 2378023116634709. doi:10.1177/2378023116634709
Meadows, L. M., Lagendyk, L. E., Thurston, W. E., & Eisener, A. C. (2003). Balancing culture, ethics, and methods in qualitative health research with aboriginal peoples. International Journal of Qualitative Methods, 2(4), 1-14. doi:10.1177/160940690300200401
Pinto, R. M., Witte, S. S., Wall, M. M., & Filippone, P. L. (2018). Recruiting and retaining service agencies and public health providers in longitudinal studies: Implications for community-engaged implementation research. Methodological Innovations, 11(1), 2059799118770996. doi:10.1177/2059799118770996
Sanders, J., & Munford, R. (2017). Hidden in plain view: Finding and enhancing the participation of marginalized young people in research. International Journal of Qualitative Methods, 16(1), 1609406917704765. doi:10.1177/1609406917704765
Semaan, S. (2010). Time-space sampling and respondent-driven sampling with hard-to-reach populations. Methodological Innovations Online, 5(2), 60-75. doi:10.4256/mio.2010.0019
van Wijk, E., & Harrison, T. (2013). Ethical problems in qualitative research involving vulnerable populations, using a pilot study. International Journal of Qualitative Methods, 12(1), 570-586. doi:10.1177/160940691301200130
Welles, B. F. (2014). On minorities and outliers: The case for making Big Data small. Big Data & Society, 1(1), 2053951714540613. doi:10.1177/2053951714540613