How to satisfy journal reviewers re the ‘rigour’ of qualitative data analysis?

Home Forums Default Forum How to satisfy journal reviewers re the ‘rigour’ of qualitative data analysis?

Viewing 15 posts - 1 through 15 (of 15 total)
  • Author
    Posts
  • #5484
    Christopher Wright
    Participant

    Hi, I’ve just joined this group and would be interested in anyone’s experience in writing up the Methods section of their articles re qualitative data analysis and the use of software like NVivo.

    A problem I’ve found is that reviewers tend to want significant detail about how coding concepts were developed and require convincing that these concepts are ‘rigorous’ – this becomes a bit tough when the word limit on an articles is 7,000-8,000 words! I’ve started using tables with examples of parent and child nodes and sections of coded text BUT this all adds to the word count, and some reviewers push for number counts of coding (assuming frequency of coding equates to significance). Any tips would be much appreciated.

    Chris

    #5498
    Dr. Arup Barman
    Participant

    Hi Chris,
    Why are thinking to satisfy other through the qualitative inquiry. The rigours of qualitative inquiry can be satisfied by coding. In the new ground breaking exploration this pose a serious challenges to researchers. Hence such researches should not articulate for such journals which accepts only 7000-8000 words. While you work for 12000 words, then the same can be expressed in 8000 words. Apply tables and matrices are the foundational tools can be rightly applied to sqeeze the methodological description rigours.

    Arup Barman

    #5497
    June Lennie
    Member

    Hi Chris

    I think that rigour can be increased through some quantification of your data. I don’t know what your study was about but if it was an evaluative study you could count the number of instances where an interviewee indicated their strength of feeling about something (ie ‘five interviewees indicated that the program increased their confidence significantly while seven indicated that their confidence increased fairly well’). Another effective way of increasing rigour is triangulation of different sources and types of data. In addition to your coding concepts, derived from the data, you could draw on the literature on the concept when determining meanings. Also, if you’ve written a more detailed account of the coding and analysis process elsewhere (maybe in a thesis or a report) you could just refer to that document. Hope this is helpful.

    cheers

    June

    #5496
    Merlien
    Member

    Hi Chris,

    I may be able to give you some examples of papers on how to report qualitative research findings using software. We have a couple of papers from our CARQ2008 and CAQR2009 workshops that deal with this issue. Drop me a mail and I’ll point you to the right direction.

    Kind Regards,
    Jasper Lim | events@merlien.org | http://www.merlien.org
    Follow us on twitter.com/merlien

    #5495
    Katy Gregg
    Member

    Hello Chris, I am finding myself with struggling with some of the same issues. It can be difficult to express your analysis procedures and data findings in such stringent word limitations. One day I hope journal editors will realize the difference in data presentation for Qual and Quant and make the appropriate but separate word requirements! With that said, one of my qual professors actually just put out this article: http://edr.sagepub.com/cgi/reprint/36/1/25 in Educational Researcher which may be helpful when referencing (rather than explaining in your own article) where rigor in qual comes from…
    Good Luck!

    #5494
    Nigel Mellor
    Member

    Dear colleagues

    I don’t know if any of you caught my email somewhere else in this system, but anyway, I tried to develop a different approach to the rigour and validity issue. It might not help you exactly, as I was not using any software, however my research did raise, and try to answer, some basic questions about the topic. You might want to look at:
    “The Secrets of the Doctorate: An introduction to messy method as a means of creating knowledge. A primer for researchers”

    The above free book is available on my website http://sites.google.com/site/nigelsbitsandbobs/

    You need to hunt around a little bit for the page called “Messy method: the secrets of the doctorate” and you will find the material there as a powerpoint attachment. It is best to print it out in colour for the full effect.

    Best wishes

    Nigel

    #5493

    Hi,

    I have been listening to the discussion and find this an important question. Perhaps the issues is more than rigor–perhaps it is an issue of transparency. The American Education Research Association has identified two sets of Standards for Reporting Empirical Social Science and Humanities oriented Research in AERA journals. For both the issue is transparency, which is an underpinning of all research, not just qualitative. You can find the standards and arguments on the AERA website (http://www.aera.net) and go to publications and download them. AERA journals are published by SAGE and are open to a diverse range of research traditions, including arts-based and history.

    On the point of tables and (re)presentations, these have been helpful in publishing in international journals. One area that we are often asked about is how to make visible the multiple questions raised by our ethnographic work in a brief way to show the logic. To address this, our research community has developed a series of representations that document the logic of inquiry we use in our ethnographic approach that makes visible the layers of analysis. The logic of inquiry chart frames multiple levels of analytic scale or multiple questions grounded in and suggested by the overtime inquiry:

    Overarching question– The primary question guiding the study
    Initiating Question 1 — a beginning point or question that serves as the initial focus
    collection and documentation– what is recorded, collected from whom in what ways
    analysis — how records were turned into data and then how analyzed

    Initiating question 2– suggested from the first analysis as another layer or angle of analysis
    collection-documentation
    analysis

    etc.

    This type of table addresses issues of rigor in collection and analysis. Like others, we also use transcript segments with analysis in a table to illustrate the analysis. We don’t use coding but rather annotation or analysis guided by discourse or content dimensions. This poses a different question. We have found that if we include those as we display the analysis, then we do not get the typical questions of coding.

    One final thought, when I edited the Reading Research Quarterly, we asked that all researchers include the coding, not just qualitative researchers. Transparency of the inscribed processes is important to uncovering the link between ontology and epistemology. This is true of quantatively grounded codings as well as qualitatively grounded ones.

    I am happy to share published work that has some of these. I would also like to have recommendations from others for published work that shows how they address this issue.

    Judith

    #5492
    Francisco Vieira
    Participant

    Hi everybody,

    There are two kinds of reviewers: those who think they are gods and those who are sure about that ! I see, at least, two problems: one is that nowadays we need to satisfy reviewers when we are writing and submitting a paper to publish. In certain way, the paper is not only ours. The other is that some qualitative studies do not allow word counts or coding. This is a tough question.

    Francisco Vieira

    #5491
    GIUSEPPINA
    Member

    Dear Francisco Vieira,
    I think in the same way. When we are writing a paper had to satisfy the reviewers. It is difficult, but it is really.
    Good luck!
    Best regards, Giuseppina

    #5490
    GIUSEPPINA
    Member

    Dear Katy Gregg,
    Your answer to Chris is right and I am according with you.
    Good Luck!
    Giuseppina

    #5489

    it’s a matter of transparency in data collection and the right audit trail.

    #5488
    GIUSEPPINA
    Member

    Dear Chris,
    I think that the qualitative search requires the satisfaction of the coding. It is necessary to observe the quantity of the words accepted.
    Best regards
    Giuseppina

    #5487

    ..and also triangulation which refer to a comprehensive approach to the conduct of research using multiple theories, data and methods

    #5486
    GIUSEPPINA
    Member

    Dear Muhammad Zubir,
    I think that your message is right, I am in accordance with you. The comprensive and global approach of research is the most important.
    Best regards
    Giuseppina

    #5485
    GIUSEPPINA
    Member

    Dear Chris,
    I think in the same way and I have observed the same problem. So the use of a number counts of coding is a good simple method.
    Best regards
    Giuseppina

Viewing 15 posts - 1 through 15 (of 15 total)
  • The forum ‘Default Forum’ is closed to new topics and replies.