ASSESSING INTER-RATER AGREEMENT ABOUT ITEM-WRITING FLAWS IN MULTIPLE-CHOICE QUESTIONS OF CLINICAL ANATOMY

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2013

Autores

Guimaraes, B (Autor) Outra; Pais, J (Autor) Outra; Coelho, E (Autor) Outra; Silva, A (Autor) Outra; Povo, A (Autor) Outra; Lourinho, I (Autor) Outra; Severo M (Autor) FMUP; Ferreira, MA (Autor) Outra

Ata de Conferência Internacional

Título: EDULEARN13: 5TH INTERNATIONAL CONFERENCE ON EDUCATION AND NEW LEARNING TECHNOLOGIES

Páginas: 5921-5924

5th International Conference on Education and New Learning Technologies (EDULEARN)

Barcelona, SPAIN, JUL 01-03, 2013

Indexação

ISI Web of Knowledge - 0 Citações

Outras Informações

ID Authenticus: P-00P-93F

Abstract (EN): Multiple-choice questions (MCQs) are regularly used in exams in order to assess students in health science disciplines. Despite this fact, MCQ items often have item-writing flaws, and few educators have formal instruction in writing MCQs. The major purpose of our study was to estimate the inter-rater agreement about item classification as either standard or flawed. In order to achieve this goal, four judges (2 teacher/2 students), blinded to all item performance data, independently classified each one of 920 test items from 10 examinations as either standard or flawed. If flawed the exact type of item flaw or flaws present in the question stem and respective options were recorded. In this study, the standard item was operationally defined as any item that did not violate one or more of the 31 principles noted in a review article which summarized current educational measurement recommendations concerning item writing. The Fleiss' Kappa was use to evaluate the inter-rater agreement between 4 judges previous the consensus process. In respect to the agreement about item classification as either standard or flawed was fair (kappa=0.3). Despite the agreement was substantial for the more prevalent principles, generally the results showed many disagreements among judges about item classification, previous the consensus process. In a future investigation it is important to assess if presence of flaw or flaws in the MCQ item have impact its quality, namely, if there are interference with difficulty and discrimination indices of the MCQ item.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 4