24th March 2014 at 4:21 pm #1125
I conducted a driving simulator study in which each participant (32 participants in total) passed each of the four infrastructural conditions in a randomized order. The road segment nearby the infrastructural condition was subdivided in ten sections of 50 meter. For each section, we recorded the mean speed and the mean lateral position for each participant. My dataset has thus 40 columns (4 conditions x 10 road sections).
Based on this research design, I would like to perform a 4 (condition) x 10 (section) within-subjects MANOVA for mean speed and mean lateral position. In SPSS I run a GLM_Repeated measures with two within-subject factors (condition and section) which have 4 and 10 levels respectively. The measure names are speed and LP (from lateral position). Than, I select my columns and drag them to the field “within-subjects variables”.
In my SPSS output I find two tables which attract my attention: first there is the table called “Multivariate Tests” and second a table “Multivariate” under the heading “Tests of within-subjects effects”. Because the study has a full within-subjects design, my question is “Which table do I have to use in my analysis description?”. It is important to note that some of the test statistics differ between both tables and that some cells of the table “Multivariate Tests” are empty because SPSS “Cannot produce multivariate test statistics because of insufficient residual degrees of freedom”.
Can someone explain the difference between the two tables (Multivariate Tests and Tests of within-subjects_Multivariate) in SPSS and which table is preferable to use in my data analysis?24th March 2014 at 7:18 pm #1135
The first table labelled “Multivariate tests” contains the multivariate test results when your two outcome measures (speed and LP) are taken into consideration. In the top half of this table you should see a section for each of your between subjects factors, if you have any (e.g., gender). The sig values tell you whether the levels of your factors differ when both outcome measures are taken into consideration. The lower half of this table contains the multivariate results for your within-subjects variables (condition, section). It tells you whether levels of the repeated measures variables differ when both outcomes are considered. This table is important to your analysis. If there are significant p-values, follow-up by looking at the contrast results to see which measures are significant across levels of your within and between subjects variables.
The Tests of Within Subjects Multivariate table is a multivariate analysis of univariate repeated measures data. Why does SPSS do this? Because repeated measures univariate data can be analyzed as though it were multivariate. This multivariate table usually agrees with the “Univariate Tests of Within-Subjects Effects” table when an univariate repeated measures analysis is run. When should you view the multivariate test of within subjects table? Usually when the sphericity assumption is not met on the univariate repeated measures test. When the sphericity test is not met (i.e., results are significant), researchers can report the multivariate results. This table is less important to your analysis and can probably be ignored.
-Dave24th March 2014 at 8:40 pm #1134
Thank you for your clear and quick answer. Unfortunaltely, I still have some questions related to your reply.
Firstly, you write that “this multivariate table usually agrees with the ‘Univaritate Tests of Within-Subjects Effects’ table when an univariate repeated measures analysis is run”. This is not the case for my data: when I run the MANOVA I get two tables under the heading ‘Tests of within-subjects effects’: the table ‘Multivariate’ on the one hand and the table ‘Univariate tests’ on the other hand. This last table gives me the same results as the table ‘Univariate tests of within-subjects effects’ when I perform an ANOVA for the dependent variables separately. However, these results do not agree with the test statistics in the table ‘Multivariate’.
Secondly, how can I deal with the absence of some test statistics in the table ‘Multivariate tests’ when SPSS gives the comment: “cannot produce multivariate test statistics because of insufficient residual degrees of freedom”?
Finaly, Mauchly’s test of sphericity shows that both depend variables do not meet the sphericity assumption.
Which table do I need to use preferably? The table ‘Multivariate tests’ or the table ‘Multivariate’ under the heading ‘Tests of within-subjects effects’?
Caroline24th March 2014 at 9:50 pm #1133
The decision to reject (sig. values <0.05) or fail to reject (sig. values >0.05) the null hypotheses should be fairly similar between the multivariate and univariate test of the within subjects effects. The actual statistics will differ but the significance values should lead to the same conclusions.
The table for univariate test of within-subjects variables should be giving the same results as running separate ANOVAs for each variable, so that is good.
If your repeated measure outcome variables fail the test of sphericity then just read from the significance value for either of the epsilon multipliers in the univariate within-subjects table: “Greenhouse”, “Huyn”, or “Lower Bound”. I think it is better to report stats for an epsilon multiplier than report multivariate results for repeated within subjects data. Reporting an epsilon multiplier F value etc. makes more sense to readers than reporting multivariate results for repeated measures univariate results. This is especially true after you’ve already reported overall multivariate results for the first table.
Is the absence of test statistics due to insufficient degrees of freedom happening in your multivariate tests table or your multivariate test of within subjects table?25th March 2014 at 8:04 am #1132
The absence of test statistics due to insufficient degrees of freedom is happening in the table “Multivariate tests”. Therefore it is important to know which table I have to use and how I can deal with this absence in case it is preperable to use the table “Multivariate tests”. For you information, I attached a pdf with a sample my SPSS-output.
You will see that the final results (and thus my conclusions) will depend on the table I use. For example, the factor Zone (section in my problem description) is only marginally significant in the table “Multivariate tests” (p = 0.051), but highly significant in the table “Tests of within-subjects effects_Multivariate” (p < 0.0001).
In addition, you will see that the results of the multivariate and univariate tests of the within-subjects tables do not give the same conclusion for the interaction Condition x Zone (multivariate p = 0.047 and univariate mean speed p = 0.169 and lateral position p = 0.386).
Caroline25th March 2014 at 10:08 pm #1131
To begin with, the first multivariate tests table says that 4 Condition levels are significantly different (p=0.001) when the two outcome variables are considered simultaneously. Similarly the 10 zone levels are close to being different when both outcomes are considered (p=0.051). I am not sure why you are not getting the multivariate interaction results for condition*zone. The problem may be that you have 2 within subjects variables which has surpassed the complexity of the repeated measures manova procedure. This is not a problem because the univariate results (see below) tell you what is going on with the interaction.
Now you want to look at the univariate within subjects results to find out what outcome measures are driving the significant multivariate results. Because you have repeated measures, SPSS runs a Mauchly’s test. The test fails for Zone and condition*zone so that means that you need to read the multivariate results for within subjects effects *OR* read the Greenhouse, Huynh, or Lower-Bound (something other than Sphericity Assumed) statistics from the univariate tests. I suggest reading from the univariate Greenhous-Geisser values (ignore the tests of within-subjects multivariate box). You can use the “sphericity assumed” stats for Condition in the univariate box because it passed Mauchly’s test.
It looks like mean_speed differs across levels of Condition (sphericity assumed F=5.406, p = 0.002). It also looks like mean_accdec differs across levels of Condition (sphericity assumed F = 2.838, p =0.042). This makes sense because the first multivariate table showed that Condition levels differed when mean_speed and mean_accdec are considered, so we expect at least one of the univariate results to be significant.
Now look at Zone in the univariate tests results. It looks like mean_speed is not significantly different across levels of Zone (Greenhouse Geisser F=2.322, p = 0.118), however mean_accdec is significantly different across levels of Zone (GG F= 3.936, p = 0.012). So one outcome measure is significantly different at different levels of Zone but the other is not. This explains why the multivariate p-value for Zone was close to significant (p = 0.051).
Now look at the interaction Condition*Zone. Neither mean_speed nor mean_accdec are significant. This means that the multivariate interaction, which is missing, is not significant anyway. There are no significant interactions between Condition and Zone for mean_speed and mean_accdec.
How does that sound?27th March 2014 at 4:03 pm #1130
Thank you for the elaborated response. Your explanation of the univariate statistics sounds familiar for me. However, I do not feel very comfortable with the sentence “[The absence of the multivariate interaction results for condition * zone] is not a problem because the univariate results tell you what is going on with the interaction” because the main purpose of a MANOVA is to first check for effects on a multivariate level. And afterwards – and only when the multivariate tests give significant results – to look at the univariate test statistics. Thus in case the results of the multivariate tests are not significant, it is not allowed to consider the univariate tests.
When I follow my reasoning above, I still face the problem that I miss the multivariate interaction results for condition*zone in the “Multivariate tests” table which is advised by you. Based on the SPSS comment “cannot produce multivariate test statistics because of insufficient residual degrees of freedom” I would think that a solution might be to increase the sample size (althoug my sample size of 32 participant is already good for a driving simulator study) or to lower the number of levels per factor and thus reduce the complexity of my model. As an example, I run the same MANOVA for mean speed and mean accdec but this time I incorporate the within-factor Condition with 4 levels (same as in my first example) and Zone with a decreasing number of levels. I start with 9 level for Zone but I have to lower the number of level of Zone to 6 to end up with a completely filled table of Multivariate tests. For you information, I attach the SPSS output for the MANOVAs with 9, 8, 7 and 6 levels for the factor Zone. What do you think about this approach?
When I come back to my initial problem about which table I have to report (Multivariate tests or Tests of within-subjects effects_Multivariate), I found on internet the following sentence (see document in attachment): “Although I have seen both the fully multivariate and the averaged multivariate tests used, it seems to me that the fully multivariate tests are the better choice, except, perhaps, when sample sizes are very small (the averaged tests have more error df).” What do you think about this? In case it is allowed to use the table Tests of within-subjects effects_Multivariate, I do not have to reduce the complexity of my model.
Thank you for your response!
Caroline27th March 2014 at 5:30 pm #1129
You are right about the differences between the multivariate tables. I did not look closely at the bottom of the test of within subjects multivariate table. That table is a multivariate *test of averages* as you point out, not a multivariate test of univariate repeated measures as I had supposed. The multivariate test of univariate repeated measures I spoke of only comes up when *one* outcome measure is entered. You entered two outcome variables.
It looks like you can report the within subjects multivariate results for averages and justify the decision by relatively low sample size. This will help you avoid the problem of insufficient residual DF for the interaction. Sorry for the confusion, but at least you found the correct answer.7th April 2014 at 7:23 am #1128
Thank you for helping me discussing this issue!
One last question: do you know a reference in which I can find some information to found my decision to use the multivariate test of averages due to relatively low sample size?
Caroline7th April 2014 at 4:34 pm #1127
I am not aware of any references. If you can’t find any, you might just state that you used the test of averages because the interaction effect did not show up in the other multivariate analysis.
If you copied and pasted your data set, thus doubling its size, then re-ran the analysis, I bet there would be no missing interaction results which suggests that the problem is due to low sample size.11th April 2014 at 4:34 pm #1126
Something went wrong with my Methodspace webpage through which I was not able to reply to your message below.
I re-ran the GLM with the double sample size and you are write: SPSS gave me all test statistics in the table “Multivariate tests” 🙂
My conclusion from this very interesting discussion is: preferably use the table “Multivariate tests” when SPSS produces all test statistics, but use the table with the averaged multivariate tests of within-subject effects when the first table is not completely filled due to a low sample size (in combination with a ‘complex’ model).
In case you find a reference related to this specific issue, please let me know!
Thank you once again for the very interesting discussion!
- You must be logged in to reply to this topic.