THE IMPACT ON THE QUALITY AND SATISFACTION OF THE TESTS OF AN UNCUED QUESTION FORMAT VERSUS A SHORT ANSWER QUESTION FORMAT

• Type: Article in International Conference Proceedings Book
• Date: 2016

Authors

Fevereiro, B (Author) Other; Rocha-Gonçalves F (Author) FMUP; Torres Guimaraes, B (Author) Other; Tomaz, DF (Author) Other; Vasconcelos Oliveira, AC (Author) Other; Machado JC (Author) FMUP; Severo M (Author) FMUP

Conference proceedings International

Title: INTED2016 Proceedings

Pages: 6846-6851

Indexing

ISI Web of Knowledge - 0 Citations

Other information

Authenticus ID: P-00P-F6X

Abstract (EN): Objectives: To assess the reliability and quality of an uncued question format (UQF), when compared to a short answer question format (SAF) in the course of Biopathology II at the Faculty of Medicine of Porto University and compare the satisfaction of students with the introduction of a new glossary based on an uncued question format. Methods: Two hundred and eighteen students (218) were tested using SAF and 283 students were assessed using UQF. We resorted to both the Classical Test Theory (CTT) and the Item Response Theory (IRT) to compare both types of questions. To assess the level of satisfaction with the new glossary, we analyzed the answers of 227 students that fulfilled a questionnaire that evaluates 3 dimensions (Satisfaction with new glossary (G1); Satisfaction with new answer sheet (G2); Model preference (G3)) ranged from 1 to 5 and Biopathology I classification (G4)). The Spearman's Correlation test was applied between dimensions. Results: Trough the CTT, we determined that the differences between the Difficulty Index on the first test (0.674 +/- 0.291) and on the second test (0.638 +/- 0.303) were not significant (p=0.367). In addition, the differences found on the Discrimination Index of the first and second test (0.318+/-0.116 and 0.300+/-0.122, respectively) were not statistically different (p=0.267). Using the IRT, the Difficulty Index (SAF: 0.698 +/- 0.312 and UQF: 0.655 +/- 0.325 after) and the Discrimination Index (SAF: 0.685 +/- 0.163 and UQF: 0.639 +/- 0.203) were not statistically different between both tests (p=0.318 and p=0.0698, respectively). Results indicate that both approaches were reliable, with Cronbach's Alpha of 0.9298 on the SAF and 0.9760 on the UQF. The students disagreed with the change in the model of evaluation (Mean=3.84). The analysis of the satisfaction questionnaires showed that students who preferred the UQF also preferred the new answer sheet and/or the new glossary. In general, students were not satisfied with the answer sheet. Similarly, students who were not satisfied with the UQF (answer sheet and/or glossary) preferred the previous model of evaluation (rho GI/GIII=-0.664 and rho GII/GIII=-0.542). We found no correlation between the student's classification and their preference with the new model of evaluation (p=0.428) Conclusions: The change of format did not have an effect in the quality of the test, however the students were not very satisfied with the new format independently of the student's classification. Even though this format did not decrease the quality parameters it was a more efficient tool in terms of time and resources saved by the correctors and faculty, in general.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 6