23rd August 2009 at 2:32 pm #5673
To one of the question our group member asked, what is the best way to analyse a questionnaire, I have replied:
For questionnaire analysis, all depends on the design. Is it open-ended questionnaire or closed questionnaire? Let me assume it is closed questionnaire with possibility to have numerical data. Quantitative data analysis does not actually start after all the data have been collected, you must think about the analysis during the design of the questionnaire. In the first place, you need to sort and clean your data. This helps you to know the flaws existing in your data set. You should also classify the data into different types of VARIABLES (interval, ordinal, nominal, or dichotomous variables), thus avoiding mixing them up unknowingly during the analysis.
Another thing to take into consideration is the sample size, there are analyses (eg regression) you may not be able to conduct because of your sample size.
After you have classified the variables, you need to analyze them differently depending on what you want:
– univariate analysis (focusing on one variable): approaches include frequency, diagrams, mean, dispersion. etc.)
– bivariate analysis (focusing on 2 variables to look into the possible relationship between them): frequency tables for a combination including a nominal variable, chi-square, Cramer’s V, etc.)
– multivariate analysis (focusing on 3 or more variables): frequency tables, this seems still complex to me to explain, Group members and other research colleagues will help, etc.
There are other approaches to quantitative data analysis, and myself I am still reading and learning to use them.
I hope other knowledgeable colleagues will boost this by taking the discussion to the next level.
Jean Providence23rd August 2009 at 2:54 pm #5675
For data analysis, our colleague can also read: How to Conduct Surveys, A Step-by-Step Guide, Fourth Edition by Arlene Fink, SAGE (2009). You can read Chapter 1 for free, especially p.6, and if interested you can buy the books.
Jean Providence24th August 2009 at 10:11 am #5674Mark FreestoneMember
I think this is a really clear introduction to the issue of survey design. I would add a couple of points.
1) You talk about ‘sample size’. Sample size in survey design is a tricky question because it is not always the case that ‘bigger is better’. The best way to understand it is through reference to the notion of statistical power, which is the way statisticians approach our strange world of surveys!
Statistical power is best explained by the idea of differences between groups. Say I have two groups: Group A are recovered throat cancer sufferers and group B are those going through treatment. If I administer a questionnaire about quality of life to both, I would expect a big difference between the two groups, and for Group A to have a much higher mean score than Group B. Because this ‘effect’ (the effect of throat cancer on life quality) is so big, I would only need a small number of people – perhaps as few as 15 in each group – to detect that difference. Despite the small numbers, the study would still be ‘adequately powered’.
Conversely if I was to administer the same questionnaire to those who were undergoing radiotherapy treatment and those undergoing chemotherapy treatment for the same disorder, I would expect there to be a much more subtle difference, if any, between the groups, and I would need a much bigger sample to exclude the possibility that a small but real difference was going unnoticed (a type II or Beta – β – error), or that I had mistakenly found a significant difference without enough ‘power’ to exclude this being just sampling error (a type I or alpha – α – error).
Statistical power is therefore a calculation between acceptable error (α and/or β), sample size (n) and the expected size of the difference, or the effect (d). ‘Power’ as a statistic is given as (1-β).
2) A lot of textbooks on regression only consider linear regression as an analytical technique: the idea that as one continuous (interval) variable changes, another variable changes proportionally: for example, we know that for large representative population, as age increases, income also increases in a fairly linear fashion.
However as a postgraduate student working with surveys I was never taught about logistic regression, which is a multivariate technique that predicts the probability of a given dichotomous variable given several others that may be interval, ordinal or dichotomous. For example, using hospital records, I can perform a logistic regression with someone’s age (interval) and their gender (dichotomous) as the independent variables to calculate the probability (between 0 and 1) that they will suffer from throat cancer at a given age (dichotomous). I need to include gender in this analysis to establish whether there might be significant differences between men and women (women typically live longer and die from heart attacks less frequently).
I wonder if anyone else has discovered anything in the course of performing survey research that they’ve found very valuable?
- The forum ‘Default Forum’ is closed to new topics and replies.