Home › Forums › Default Forum › How to satisfy journal reviewers re the ‘rigour’ of qualitative data analysis?
- This topic has 14 replies, 10 voices, and was last updated 10 years, 11 months ago by
Dr. Arup Barman.
-
AuthorPosts
-
5th October 2009 at 11:32 pm #5484
Christopher Wright
ParticipantHi, I’ve just joined this group and would be interested in anyone’s experience in writing up the Methods section of their articles re qualitative data analysis and the use of software like NVivo.
A problem I’ve found is that reviewers tend to want significant detail about how coding concepts were developed and require convincing that these concepts are ‘rigorous’ – this becomes a bit tough when the word limit on an articles is 7,000-8,000 words! I’ve started using tables with examples of parent and child nodes and sections of coded text BUT this all adds to the word count, and some reviewers push for number counts of coding (assuming frequency of coding equates to significance). Any tips would be much appreciated.
Chris
15th October 2009 at 2:41 am #5498Dr. Arup Barman
ParticipantHi Chris,
Why are thinking to satisfy other through the qualitative inquiry. The rigours of qualitative inquiry can be satisfied by coding. In the new ground breaking exploration this pose a serious challenges to researchers. Hence such researches should not articulate for such journals which accepts only 7000-8000 words. While you work for 12000 words, then the same can be expressed in 8000 words. Apply tables and matrices are the foundational tools can be rightly applied to sqeeze the methodological description rigours.Arup Barman
15th October 2009 at 10:21 am #5497June Lennie
MemberHi Chris
I think that rigour can be increased through some quantification of your data. I don’t know what your study was about but if it was an evaluative study you could count the number of instances where an interviewee indicated their strength of feeling about something (ie ‘five interviewees indicated that the program increased their confidence significantly while seven indicated that their confidence increased fairly well’). Another effective way of increasing rigour is triangulation of different sources and types of data. In addition to your coding concepts, derived from the data, you could draw on the literature on the concept when determining meanings. Also, if you’ve written a more detailed account of the coding and analysis process elsewhere (maybe in a thesis or a report) you could just refer to that document. Hope this is helpful.
cheers
June
19th October 2009 at 9:14 pm #5496Merlien
MemberHi Chris,
I may be able to give you some examples of papers on how to report qualitative research findings using software. We have a couple of papers from our CARQ2008 and CAQR2009 workshops that deal with this issue. Drop me a mail and I’ll point you to the right direction.
Kind Regards,
Jasper Lim | events@merlien.org | http://www.merlien.org
Follow us on twitter.com/merlien27th October 2009 at 10:38 pm #5495Katy Gregg
MemberHello Chris, I am finding myself with struggling with some of the same issues. It can be difficult to express your analysis procedures and data findings in such stringent word limitations. One day I hope journal editors will realize the difference in data presentation for Qual and Quant and make the appropriate but separate word requirements! With that said, one of my qual professors actually just put out this article: http://edr.sagepub.com/cgi/reprint/36/1/25 in Educational Researcher which may be helpful when referencing (rather than explaining in your own article) where rigor in qual comes from…
Good Luck!26th November 2009 at 9:53 am #5494Nigel Mellor
MemberDear colleagues
I don’t know if any of you caught my email somewhere else in this system, but anyway, I tried to develop a different approach to the rigour and validity issue. It might not help you exactly, as I was not using any software, however my research did raise, and try to answer, some basic questions about the topic. You might want to look at:
“The Secrets of the Doctorate: An introduction to messy method as a means of creating knowledge. A primer for researchers”The above free book is available on my website http://sites.google.com/site/nigelsbitsandbobs/
You need to hunt around a little bit for the page called “Messy method: the secrets of the doctorate” and you will find the material there as a powerpoint attachment. It is best to print it out in colour for the full effect.
Best wishes
Nigel
26th November 2009 at 8:07 pm #5493Judith L Green
MemberHi,
I have been listening to the discussion and find this an important question. Perhaps the issues is more than rigor–perhaps it is an issue of transparency. The American Education Research Association has identified two sets of Standards for Reporting Empirical Social Science and Humanities oriented Research in AERA journals. For both the issue is transparency, which is an underpinning of all research, not just qualitative. You can find the standards and arguments on the AERA website (http://www.aera.net) and go to publications and download them. AERA journals are published by SAGE and are open to a diverse range of research traditions, including arts-based and history.
On the point of tables and (re)presentations, these have been helpful in publishing in international journals. One area that we are often asked about is how to make visible the multiple questions raised by our ethnographic work in a brief way to show the logic. To address this, our research community has developed a series of representations that document the logic of inquiry we use in our ethnographic approach that makes visible the layers of analysis. The logic of inquiry chart frames multiple levels of analytic scale or multiple questions grounded in and suggested by the overtime inquiry:
Overarching question– The primary question guiding the study
Initiating Question 1 — a beginning point or question that serves as the initial focus
collection and documentation– what is recorded, collected from whom in what ways
analysis — how records were turned into data and then how analyzedInitiating question 2– suggested from the first analysis as another layer or angle of analysis
collection-documentation
analysisetc.
This type of table addresses issues of rigor in collection and analysis. Like others, we also use transcript segments with analysis in a table to illustrate the analysis. We don’t use coding but rather annotation or analysis guided by discourse or content dimensions. This poses a different question. We have found that if we include those as we display the analysis, then we do not get the typical questions of coding.
One final thought, when I edited the Reading Research Quarterly, we asked that all researchers include the coding, not just qualitative researchers. Transparency of the inscribed processes is important to uncovering the link between ontology and epistemology. This is true of quantatively grounded codings as well as qualitatively grounded ones.
I am happy to share published work that has some of these. I would also like to have recommendations from others for published work that shows how they address this issue.
Judith
15th January 2010 at 10:26 am #5492Francisco Vieira
ParticipantHi everybody,
There are two kinds of reviewers: those who think they are gods and those who are sure about that ! I see, at least, two problems: one is that nowadays we need to satisfy reviewers when we are writing and submitting a paper to publish. In certain way, the paper is not only ours. The other is that some qualitative studies do not allow word counts or coding. This is a tough question.
Francisco Vieira
21st January 2010 at 8:37 pm #5491GIUSEPPINA
MemberDear Francisco Vieira,
I think in the same way. When we are writing a paper had to satisfy the reviewers. It is difficult, but it is really.
Good luck!
Best regards, Giuseppina21st January 2010 at 8:41 pm #5490GIUSEPPINA
MemberDear Katy Gregg,
Your answer to Chris is right and I am according with you.
Good Luck!
Giuseppina9th February 2010 at 11:34 am #5489Muhammad Zubir
Memberit’s a matter of transparency in data collection and the right audit trail.
11th February 2010 at 11:42 pm #5488GIUSEPPINA
MemberDear Chris,
I think that the qualitative search requires the satisfaction of the coding. It is necessary to observe the quantity of the words accepted.
Best regards
Giuseppina27th February 2010 at 8:46 pm #5487Muhammad Zubir
Member..and also triangulation which refer to a comprehensive approach to the conduct of research using multiple theories, data and methods
2nd March 2010 at 10:13 pm #5486GIUSEPPINA
MemberDear Muhammad Zubir,
I think that your message is right, I am in accordance with you. The comprensive and global approach of research is the most important.
Best regards
Giuseppina2nd March 2010 at 10:18 pm #5485GIUSEPPINA
MemberDear Chris,
I think in the same way and I have observed the same problem. So the use of a number counts of coding is a good simple method.
Best regards
Giuseppina -
AuthorPosts
- The forum ‘Default Forum’ is closed to new topics and replies.