I conducted a study in which two groups of teachers (pre-service and veteran) all watched four video clips of instances of aggression between students. The clips differed by type of aggression and gender of the students depicted (physical vs. relational aggression; male vs. female). After viewing each clip, the participants answered a series of questions about their perceived severity of the incident, empathy for the victim, empathy for the bully, and likelihood of intervention on a 5-point Likert scale (these constituted the dependent variables). I conducted four separate mixed design ANOVAs in SPSS for each dependent variable, with one between-subject independent variable and two within-subject independent variables:
Teacher status - pre-service vs. veteran (between-subject)
Aggression - physical vs. relational (within-subject)
Gender - male vs. female (within-subject)
Unfortunately, I have unequal samples sizes for the two groups (28 pre-service teachers, 14 veteran). Also, some of the dependent variables are not normally distributed. I did not violate the assumptions of homogeneity of variance or sphericity.
First of all, I am confused about checking the normality of the data vs. the normality of the residuals in SPSS. When you run the normality tests (K-S and S-W) via "Descriptive Statistics" -> "Explore," do you include the DVs under the "Dependent List"?
Secondly, how robust is a mixed/repeated measures ANOVA to the violation of normality? Can I follow the general rules of thumb regarding the degree of skewness and kurtosis?
I am a graduate student in the social sciences, and statistics is by no means my forte. I have spent several weeks attempting to make sense of all of this, but I find that I am just confusing myself further and further. Any straightforward answers would be much appreciated!
At this point, I am hoping to talk about the violation of normality in the discussion section of my paper, noting it as a limitation for this pilot study. Can anyone point me in the direction of references that basically argue that this violation of normality is not the end of the world?
I suspect that you keep getting more and more confused because you may be approaching the analysis in an unproductive way. Having said that, I recognize that there are several ways to statistically analyze data, but some are definitely better than others. Choosing the best analysis depends on what you want to show or compare.
Do you want to show that teacher experience (pre-service vs. veteran) influences perceptions of aggressive acts? If the answer is yes, then you should consider comparing perceptions of each of the 4 video scenarios with separate analyses - one for each video. In this situation you could run a MANOVA with 4 outcome variables (victim empathy, bully empathy, severity, and intervention) and a single between subjects variable which is teacher experience. This analysis will tell whether the teacher categories differ when all outcomes (multivariate) are taken into consideration at the same time, and if this multivariate result is significant, then it will give you the univariate analyses comparing each individual outcome variable between teacher category. The univariate results will tell you which outcomes are driving the significant difference in the multivariate result. You don’t have to worry about sphericity when running MANOVA. Run a MANOVA for each video.
If you want to examine multiple within subject contrasts that detail every possible combination of variables including interactions, and whether they are significant, then the mixed design ANOVA that you’ve planned is ideal. The mixed design will produce a lot of outcome data. ANOVA is fairly robust to violations of normality when the underlying population is known to be normal or, if the population is not normal, when the sample size is large enough to satisfy the CLT. In either case most people just say the ANOVA is robust to violations of normality and leave it at that. You might want to do the same.