Quality of the reviews submitted by attendees of a workshop on peer review

Samir Praharaj, Shahul Ameen

Research output: Contribution to journalReview article

Abstract

Objective: The objective of the study was to study the methodological quality and error detection of the review by the participants of a peer review workshop. Methods: All participants of the workshop were invited to peer review a randomized controlled trial. The manuscript was E-mailed to them after introducing eight deliberate errors to it. Specific instructions and a deadline were provided. All the reviews were analyzed using review quality instrument (RQI). Furthermore, the rate and the type of errors identified were recorded. Results: Of 25 participants, 16 (64%) returned the reviews. The mean total score on RQI was 4.12 (standard deviation 0.70, 95% confidence interval 3.74-4.50); the items which most reviewers did not discuss where the importance of research question and originality of the paper. The number of errors correctly identified varied from 0 to 6 (median 3), the most common being a wrong conclusion (87.5%), randomization procedure (50%), written informed consent (50%), ethics committee approval (42.8%), and masking (31.2%). Only 5 (31.2%) gave an overall recommendation on whether the manuscript should be accepted or not. Conclusions: Major errors were readily identified by the reviewers; however, the need for training was felt in some areas in which the review quality was modest.

Original languageEnglish
Pages (from-to)785-788
Number of pages4
JournalIndian Journal of Psychological Medicine
Volume39
Issue number6
DOIs
Publication statusPublished - 01-11-2017

Fingerprint

Peer Review
Manuscripts
Education
Ethics Committees
Random Allocation
Informed Consent
Randomized Controlled Trials
Confidence Intervals
Research

All Science Journal Classification (ASJC) codes

  • Clinical Psychology

Cite this

@article{b64b0542be334435ba7237d378e05d04,
title = "Quality of the reviews submitted by attendees of a workshop on peer review",
abstract = "Objective: The objective of the study was to study the methodological quality and error detection of the review by the participants of a peer review workshop. Methods: All participants of the workshop were invited to peer review a randomized controlled trial. The manuscript was E-mailed to them after introducing eight deliberate errors to it. Specific instructions and a deadline were provided. All the reviews were analyzed using review quality instrument (RQI). Furthermore, the rate and the type of errors identified were recorded. Results: Of 25 participants, 16 (64{\%}) returned the reviews. The mean total score on RQI was 4.12 (standard deviation 0.70, 95{\%} confidence interval 3.74-4.50); the items which most reviewers did not discuss where the importance of research question and originality of the paper. The number of errors correctly identified varied from 0 to 6 (median 3), the most common being a wrong conclusion (87.5{\%}), randomization procedure (50{\%}), written informed consent (50{\%}), ethics committee approval (42.8{\%}), and masking (31.2{\%}). Only 5 (31.2{\%}) gave an overall recommendation on whether the manuscript should be accepted or not. Conclusions: Major errors were readily identified by the reviewers; however, the need for training was felt in some areas in which the review quality was modest.",
author = "Samir Praharaj and Shahul Ameen",
year = "2017",
month = "11",
day = "1",
doi = "10.4103/IJPSYM.IJPSYM_372_17",
language = "English",
volume = "39",
pages = "785--788",
journal = "Indian Journal of Psychological Medicine",
issn = "0253-7176",
publisher = "Medknow Publications and Media Pvt. Ltd",
number = "6",

}

Quality of the reviews submitted by attendees of a workshop on peer review. / Praharaj, Samir; Ameen, Shahul.

In: Indian Journal of Psychological Medicine, Vol. 39, No. 6, 01.11.2017, p. 785-788.

Research output: Contribution to journalReview article

TY - JOUR

T1 - Quality of the reviews submitted by attendees of a workshop on peer review

AU - Praharaj, Samir

AU - Ameen, Shahul

PY - 2017/11/1

Y1 - 2017/11/1

N2 - Objective: The objective of the study was to study the methodological quality and error detection of the review by the participants of a peer review workshop. Methods: All participants of the workshop were invited to peer review a randomized controlled trial. The manuscript was E-mailed to them after introducing eight deliberate errors to it. Specific instructions and a deadline were provided. All the reviews were analyzed using review quality instrument (RQI). Furthermore, the rate and the type of errors identified were recorded. Results: Of 25 participants, 16 (64%) returned the reviews. The mean total score on RQI was 4.12 (standard deviation 0.70, 95% confidence interval 3.74-4.50); the items which most reviewers did not discuss where the importance of research question and originality of the paper. The number of errors correctly identified varied from 0 to 6 (median 3), the most common being a wrong conclusion (87.5%), randomization procedure (50%), written informed consent (50%), ethics committee approval (42.8%), and masking (31.2%). Only 5 (31.2%) gave an overall recommendation on whether the manuscript should be accepted or not. Conclusions: Major errors were readily identified by the reviewers; however, the need for training was felt in some areas in which the review quality was modest.

AB - Objective: The objective of the study was to study the methodological quality and error detection of the review by the participants of a peer review workshop. Methods: All participants of the workshop were invited to peer review a randomized controlled trial. The manuscript was E-mailed to them after introducing eight deliberate errors to it. Specific instructions and a deadline were provided. All the reviews were analyzed using review quality instrument (RQI). Furthermore, the rate and the type of errors identified were recorded. Results: Of 25 participants, 16 (64%) returned the reviews. The mean total score on RQI was 4.12 (standard deviation 0.70, 95% confidence interval 3.74-4.50); the items which most reviewers did not discuss where the importance of research question and originality of the paper. The number of errors correctly identified varied from 0 to 6 (median 3), the most common being a wrong conclusion (87.5%), randomization procedure (50%), written informed consent (50%), ethics committee approval (42.8%), and masking (31.2%). Only 5 (31.2%) gave an overall recommendation on whether the manuscript should be accepted or not. Conclusions: Major errors were readily identified by the reviewers; however, the need for training was felt in some areas in which the review quality was modest.

UR - http://www.scopus.com/inward/record.url?scp=85038359146&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038359146&partnerID=8YFLogxK

U2 - 10.4103/IJPSYM.IJPSYM_372_17

DO - 10.4103/IJPSYM.IJPSYM_372_17

M3 - Review article

VL - 39

SP - 785

EP - 788

JO - Indian Journal of Psychological Medicine

JF - Indian Journal of Psychological Medicine

SN - 0253-7176

IS - 6

ER -