15th March 2011 at 5:26 pm #3658
I’m conducting a reserach that requires me to collect motivation (independent variable with two levels, High and low) data from about 15-25 lower and middle level employees in about 100-120 organizations by administering a survey. Also, I will be collecting data about information systems (dependent variable with two levels, Good and Bad) by surveying 1-2 members of top management teams in each of these 100-120 organizations. I have a few questions:
a) Since I will be using a 7 point likert scale to collect data on independent and dependent variable, I’m not sure how would I group companies on high and low. For example, i collected data from 15-20 employees in organization by asking 4 items. I’m not sure how can I classify this company as high or low on motivation. I need this because I want to run ANOVA as a part of my reserach method to show that companies with high motivation and low motivation use systems differently.
Are there any grouping or banding techniques that can be used? I just want to group companies into two clusters, i.e., high and low.
b) Can I use SEM for my above reserach? What’s the minimum sample size requirements for SEM? If not, what are the other alternatives for this kind of reserach.
Thank you much for all your help in this regard.
Gaurav Gupta17th March 2011 at 3:33 am #3663Jeremy MilesParticipant
There are a few things to think about here.
First, you say you “want to run anova”. Surely you want to run the most appropriate statistical technique? ANOVA isn’t it, because that requires you discard some information and dichotomize your variables. It’s a bit like saying that you know people’s ages, and so you will classify them as ‘young’ or ‘old’. Then you will say that a 20 year old and a 40 year old are, as far as you are concerned, the same. And that a 41 year old is as different from a 40 year old as an 80 year old is. This is obviously wrong, so don’t do it.
Second, assuming that the employees agree with one another, I don’t think you need 25 in each group.
Third, what you (probably) want to do is multiple regression. You could do multilevel modeling, but I think that’s going to be too much.
Fourth, your sample size (which is going to be about 120) is probably too low for SEM.
Final bit of advice. Work out the question that you want to ask of your data, then work out how to answer it. Don’t start with the statistical technique you want to use. That’s a bit like saying “I need to fix my car and I want to use my hammer to do it”.
If any of this is unclear, feel free to post a reply and I (or someone else) might try to explain more.17th March 2011 at 5:08 am #3662
Hi, Thank you much for your reply. I need a few clarifications, and would highly appreciate if you could answer them.
I wanted to run ANOVA to look for differences between two groups of companies (i.e. High motivated and Low motivated companies) in their usage of information systems (Good versus Bad). Do you think, is there any problem if I dichotomize all companies in two groups (i.e. high and low) using any test score banding procedure or any arbitary procedure? Also, what’s the alternative statistical technique for answering this question?
Since, my reserach involves measuring organizational level motivation, I wanted to survey 15-25 employees per company, but you said 25 are not required. Do you know how many would I need? Are there any papers which clarify on these issues. Remember, i am not measuring perception by just surveying one employee per company. I am trying to measure the actual motivation levels in an organization by taking multiple responses.
Since I will be collecting data, I can surely use multiple regression as you suggested, but should I use logistic as I plan on dichotomizing organizations into two clusters, high and low motivated. Please help me clarify more.
I’m a first year doctoral student, and still trying to figure out some finer points about statistics. I thank you for all your help.
Gaurav18th March 2011 at 1:06 pm #3661Louise JohannessonMember
You want a 2 by 2 contingency table, why use a 7 point scale?
Either you let the employees take a stand on a blunt yes/no question. Or you aggregate ex post, possibly loosing a lot of information as Mr. Miles pointed out.
However, this is not a really a statistics question in my opinion, but a decision based on theory. How do you think people will answer? Are the prone to self-censoring? Do they have self-knowledge about their own motivation? What do you actually need to answer you research question? Do you have a concrete research question?
This variable, information system, what is an information system? Do you think motivation affects information systems?
As I said, I don’t know what it is, but I would guess it is something about communication within the work place?? Then I would suspect that information systems affect motivation (which sounds like a more interesting research question to me).
Is Good/Bad information systems something that is clear for everyone or just a definition that you want to test in this research?
If there is already a clear definition of a Good information systems and this is fixed within a firm, I would also suspect that “Good” information system will attract more motivated employees, and the model will suffer from self-selection bias.
Most of these are not statistics questions per se, but a matter of having a clear underlying theory.
Cheers!20th March 2011 at 9:15 am #3660Kristian KarlsonMember
Perhaps a multilevel model, as suggested by Jeremy, would be the way to go. The reason for this is that you have people nested in organizations. Because your outcome measure is ordered discrete, you could estimate a random effects ordered probit or logit. That would allow you to learn about the company-variation in the motivation measure, and you can also enter company-specific predictor variables (such as the system use you refer to) to see whether these variables “explain” company-variation in the motivation outcome.
There is a user-written Stata command called, reoprob, which estimates random effects (i.e. “random intercept” in multilevel model terminology) ordered probit.
That said, I agree with the other comments: The research question comes before the method. I would suspect that a multilevel model approach would be able to help you answer some of the research questions you posed above.
Regarding SEM: You have a situation with a multilevel SEM with ordered outcomes. I guess that can be solved with MPLUS.
/Kristian20th March 2011 at 11:22 am #3659
Thank you Louise and Kristian for answering my questions. I am going to reserach more on the methodologies you suggested.
Best of luck,
- The forum ‘Default Forum’ is closed to new topics and replies.