The January 2020 HSLS Update article, Write Better and Get Published: Health Research Reporting Guidelines, discussed health research reporting guidelines (RGs) which addressed the importance of clear and transparent writing. This article focuses on the importance of critical appraisal (CA) of study execution (also called study conduct) and provides tools to aid in the CA process as well as suggestions for settings to conduct critical appraisal.
Why critical appraisal? Simply put, critical appraisal of studies is key to the incorporation of quality research into evidence-based practice. Not all research is good; critical appraisal offers a reliable method of objectively identifying what is good about a study and what is not. It is a systematic method for reviewing study execution based on a validated checklist of items. Rather than relying on the reader to notice inconsistencies or incorrect analyses (i.e. making notes in the margin), a critical appraisal ensures that the appropriate questions are asked during each phase of the study from rationale to study method choice to discussion.
In the past, critical appraisal has been employed primarily in two areas: the development of clinical practice guidelines and as part of systematic reviews, especially those that include randomized trials. However, some “off-label” uses have been promoted as well:
- Protocol and grant preparation: Check your next protocol or grant application against the appropriate critical appraisal checklist before submission.
- Manuscript preparation: Before sending off your next manuscript, make certain your article describes good study conduct based on the appropriate critical appraisal checklist.
- Peer review of manuscripts: Why allow bad research to be published? Or, why allow an incomplete description of a study to be published? If you serve as a peer reviewer of journal manuscripts, use the appropriate critical appraisal checklist to guide your evaluation in an objective manner.
- Journal club: Prepare for journal club by using a critical appraisal checklist to prepare for and guide the discussion of the article. Are you the faculty lead for a journal club? Consider requiring (or at a minimum) strongly recommending the use of a critical appraisal checklist.
Of course, a protocol or manuscript that is not written in a clear and transparent manner is difficult to appraise. To further improve writing and/or peer review, be sure to incorporate the appropriate health research reporting guideline as well in each of the above settings!
Please see the Health Research Reporting Guidelines, Study Execution Manuals, & Study Execution Assessment guide for more information, or contact us via Ask a Librarian.
~Helena VonVille