21st November 2012 at 1:32 pm #1974aliceParticipant
I am currently rather confused about the design type for my study.
I am looking at a number of measures pre and post intervention in a community sample (there will be no other group) – it is a kind of pilot/demonstration site for a new project. Is the design type quasi-experimental? And would I use paired sample t tests to compare before and after?21st November 2012 at 4:27 pm #1979Katie MetzlerParticipant
Yes, it’s a quasi-experimental design if you have no control group, just one group pre and post intervention. Be careful drawing conclusions with this kind of design though – without a control group to compare to, you can’t be sure that the differences in the pre-test and the post-test are causally related to the intervention.
These tables can help you choose an appropriate statistical test given your design. http://www.ats.ucla.edu/stat/stata/whatstat/default.htm29th November 2012 at 3:06 pm #1978aliceParticipant
thank you! an additional question , if you don’t mind, is it normal to have inclusion/exclusion criteria in designs like these. My supervisor has advised me to be ‘broad’ but i’am uncertain if this will just look too vague in the proposal30th November 2012 at 12:55 pm #1977Katie MetzlerParticipant
Forgive the very long URL, but you might find this helpful, it’s all about writing your proposal and has some good examples throughout of mistakes people make.
I would think that you should specify inclusion/exclusion criteria, but if your supervisor has told you to be broad, he/she might have a reason for that I don’t know about… maybe go back and ask again for clarification?2nd December 2012 at 3:10 pm #1976Ingo RohlfingMember
No, I strongly recommend to see this as a quasi-experimental design. The inventors of quasi-experiments have a specific and good definition of quasi-experiment: a design is a quasi-experiment when it is able to produce the same results as a real experiment (Cook, Thomas D. and Vivian C. Wong (2008): Better Quasi-Experimental Practice. Alasuutari, Pertti, Leonard Bickman and Julia Brannen (eds.): The Sage Handbook of Social Research Methods. Thousand Oaks: Sage: 134-165). Under certain conditions, the only designs meeting this requirement are interrupted time-series designs, differences-in-differences-designs, and regression-discontinuity designs. As I see it, your design does not fall under any of the three. You have a pre-test-post-test design without control. A very good reading on different forms of design can be found in de Vaus, David A. (2001): Research Design in Social Research. London: Sage. I think it is chapter 4.
.27th January 2013 at 9:13 pm #1975Roger GommMember
On inclusion/exclusion. Presumably by community sample, you mean a sample of people drawn from the community. If so there will be an issue of how far the people in your sample are representative of those in the community (or of those in the community who might experience the intervention in which you are interested): ie of how far you can generalise from your sample to the community from which it was drawn..Some kinds of people will exclude themselves by being unavailable at the outset, and some who you study at the outset will have disappeared post intervention. However you select your sample it is important that you know enough about the demographics of the community so that you can say who got included and who got excluded -any generalisations you make will be more likely to be true of the kinds of people who got included, and less likely to be true of those who didn’t. It might be better to think of the sample first and then only to claim that what was true of the sample would likely be true of a community with precisely the same structure as the sample.
The demographic structure of the community you are sampling in relation to your sample size is an important consideration in setting inclusions and exclusions. The smaller your sample the less adquately it will represent demographic and other dimensions of diversity. So, for example if your sample is so small that it contains only a handful of minority ethnic people, what you observe happening to them will almost certainly not be representative of what happens to all minority ethnic people in the community. Under these circumstances you might be advised to exclude minority ethnic subjects, about whom your results will be quesitionable, and conduct some more soundly based research on the majority population only. On the whole it is better to be confident about results drawn from and applicable to a narrow range of subjects, than to find oneself with results drawn from a large and diverse range of subjects but be unsure as to who these people are and how like other people they are. Or alternatively if you are particularly interested in ethnic diversity use inclusion criteria to to create a disproportionate quotas of subjects from different ethnic groups. Generally research investigating mechanisms, giving explanations about what gives rise to what, are more soundly based on homogeneous samples (lots of exlcusions), while research designed to find out the distribution of some characteristic (even the distribution of different responses to an intervention) are more soundly based on heterogeneous samples – (fewer exclusions – but larger samples)
In addition your theoretical and practical interests should determine inclusions and exclusions. Obviously if you are interested in an intervention targetted at the under 25s then you will exclude the over 25s. If you were interested in discovering the take up of some service by poor people, then you would use an income level discriminator, but if you were interested in the penetration of some health education message – then you would probably want as broad a sample as possible.
- You must be logged in to reply to this topic.