Standard setting of objective structured practical examination by modified Angoff method

A pilot study

M. Ganesh Kamath, Vinod Pallath, K. Ramnarayan, Asha Kamath, Sharmila Torke, James Gonsalves

Research output: Contribution to journalArticle

Abstract

Background. The undergraduate curriculum at our institution is divided system-wise into four blocks, each block ending with theory and objective structured practical examination (OSPE). The OSPE in Physiology consists of 12 stations, and a conventional minimum score to qualify is 50%. We aimed to incorporate standard setting using the modified Angoff method in OSPE to differentiate the competent from the non-competent student and to explore the possibility of introducing standard setting in Physiology OSPE at our institution. Methods. Experts rated the OSPE using the modified Angoff method to obtain the standard set cut-off in two of the four blocks. We assessed the OSPE marks of 110 first year medical students. Chi-square test was used to compare the number of students who scored less than standard set cut-off and conventional cut-off; correlation coefficient was used to assess the relation between OSPE and theory marks in both blocks. Feedback was obtained from the experts. Results. The standard set was 62% and 67% for blocks II and III, respectively. The use of standard set cut-off resulted in 16.3% (n=18) and 22.7% (n=25) students being declared unsuccessful in blocks II and III, respectively. Comparison between the number, who scored less than standard set and conventional cut-off was statistically significant (p=0.001). The correlation coefficient was 0.65 (p=0.003) and 0.52 (p<0.001) in blocks II and III, respectively. The experts welcomed the idea of standard setting. Conclusion. Standard setting helped in differentiating the competent from the non-competent student, indicating that standard setting enhances the quality of OSPE as an assessment tool.

Original languageEnglish
Pages (from-to)160-162
Number of pages3
JournalNational Medical Journal of India
Volume29
Issue number3
Publication statusPublished - 01-05-2016

Fingerprint

Students
Chi-Square Distribution
Medical Students
Curriculum

All Science Journal Classification (ASJC) codes

  • Medicine(all)

Cite this

@article{6955bde65e9c4d76896a6a5e02b44eea,
title = "Standard setting of objective structured practical examination by modified Angoff method: A pilot study",
abstract = "Background. The undergraduate curriculum at our institution is divided system-wise into four blocks, each block ending with theory and objective structured practical examination (OSPE). The OSPE in Physiology consists of 12 stations, and a conventional minimum score to qualify is 50{\%}. We aimed to incorporate standard setting using the modified Angoff method in OSPE to differentiate the competent from the non-competent student and to explore the possibility of introducing standard setting in Physiology OSPE at our institution. Methods. Experts rated the OSPE using the modified Angoff method to obtain the standard set cut-off in two of the four blocks. We assessed the OSPE marks of 110 first year medical students. Chi-square test was used to compare the number of students who scored less than standard set cut-off and conventional cut-off; correlation coefficient was used to assess the relation between OSPE and theory marks in both blocks. Feedback was obtained from the experts. Results. The standard set was 62{\%} and 67{\%} for blocks II and III, respectively. The use of standard set cut-off resulted in 16.3{\%} (n=18) and 22.7{\%} (n=25) students being declared unsuccessful in blocks II and III, respectively. Comparison between the number, who scored less than standard set and conventional cut-off was statistically significant (p=0.001). The correlation coefficient was 0.65 (p=0.003) and 0.52 (p<0.001) in blocks II and III, respectively. The experts welcomed the idea of standard setting. Conclusion. Standard setting helped in differentiating the competent from the non-competent student, indicating that standard setting enhances the quality of OSPE as an assessment tool.",
author = "{Ganesh Kamath}, M. and Vinod Pallath and K. Ramnarayan and Asha Kamath and Sharmila Torke and James Gonsalves",
year = "2016",
month = "5",
day = "1",
language = "English",
volume = "29",
pages = "160--162",
journal = "National Medical Journal of India",
issn = "0970-258X",
publisher = "All India Institute of Medical Sciences",
number = "3",

}

Standard setting of objective structured practical examination by modified Angoff method : A pilot study. / Ganesh Kamath, M.; Pallath, Vinod; Ramnarayan, K.; Kamath, Asha; Torke, Sharmila; Gonsalves, James.

In: National Medical Journal of India, Vol. 29, No. 3, 01.05.2016, p. 160-162.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Standard setting of objective structured practical examination by modified Angoff method

T2 - A pilot study

AU - Ganesh Kamath, M.

AU - Pallath, Vinod

AU - Ramnarayan, K.

AU - Kamath, Asha

AU - Torke, Sharmila

AU - Gonsalves, James

PY - 2016/5/1

Y1 - 2016/5/1

N2 - Background. The undergraduate curriculum at our institution is divided system-wise into four blocks, each block ending with theory and objective structured practical examination (OSPE). The OSPE in Physiology consists of 12 stations, and a conventional minimum score to qualify is 50%. We aimed to incorporate standard setting using the modified Angoff method in OSPE to differentiate the competent from the non-competent student and to explore the possibility of introducing standard setting in Physiology OSPE at our institution. Methods. Experts rated the OSPE using the modified Angoff method to obtain the standard set cut-off in two of the four blocks. We assessed the OSPE marks of 110 first year medical students. Chi-square test was used to compare the number of students who scored less than standard set cut-off and conventional cut-off; correlation coefficient was used to assess the relation between OSPE and theory marks in both blocks. Feedback was obtained from the experts. Results. The standard set was 62% and 67% for blocks II and III, respectively. The use of standard set cut-off resulted in 16.3% (n=18) and 22.7% (n=25) students being declared unsuccessful in blocks II and III, respectively. Comparison between the number, who scored less than standard set and conventional cut-off was statistically significant (p=0.001). The correlation coefficient was 0.65 (p=0.003) and 0.52 (p<0.001) in blocks II and III, respectively. The experts welcomed the idea of standard setting. Conclusion. Standard setting helped in differentiating the competent from the non-competent student, indicating that standard setting enhances the quality of OSPE as an assessment tool.

AB - Background. The undergraduate curriculum at our institution is divided system-wise into four blocks, each block ending with theory and objective structured practical examination (OSPE). The OSPE in Physiology consists of 12 stations, and a conventional minimum score to qualify is 50%. We aimed to incorporate standard setting using the modified Angoff method in OSPE to differentiate the competent from the non-competent student and to explore the possibility of introducing standard setting in Physiology OSPE at our institution. Methods. Experts rated the OSPE using the modified Angoff method to obtain the standard set cut-off in two of the four blocks. We assessed the OSPE marks of 110 first year medical students. Chi-square test was used to compare the number of students who scored less than standard set cut-off and conventional cut-off; correlation coefficient was used to assess the relation between OSPE and theory marks in both blocks. Feedback was obtained from the experts. Results. The standard set was 62% and 67% for blocks II and III, respectively. The use of standard set cut-off resulted in 16.3% (n=18) and 22.7% (n=25) students being declared unsuccessful in blocks II and III, respectively. Comparison between the number, who scored less than standard set and conventional cut-off was statistically significant (p=0.001). The correlation coefficient was 0.65 (p=0.003) and 0.52 (p<0.001) in blocks II and III, respectively. The experts welcomed the idea of standard setting. Conclusion. Standard setting helped in differentiating the competent from the non-competent student, indicating that standard setting enhances the quality of OSPE as an assessment tool.

UR - http://www.scopus.com/inward/record.url?scp=84994619187&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994619187&partnerID=8YFLogxK

M3 - Article

VL - 29

SP - 160

EP - 162

JO - National Medical Journal of India

JF - National Medical Journal of India

SN - 0970-258X

IS - 3

ER -