Automated Assessment in Computer Science Education: A State-of-the-Art Review

• Type: Another Publication in an International Scientific Journal
• Date: 2022

Authors

Paiva, JC (Author) Other; José Paulo Leal (Author) FCUP; Figueira, A (Author) FCUP

Journal

Title: ACM Transactions on Computing Education

Vol. 22

Initial page: 34:1

Publisher: ACM

Indexing

ISI Web of Knowledge - 93 Citations

Scopus - 145 Citations

Other information

Authenticus ID: P-00W-5Q4

DOI: 10.1145/3513140

Abstract (EN): Practical programming competencies are critical to the success in computer science (CS) education and goto-market of fresh graduates. Acquiring the required level of skills is a long journey of discovery, trial and error, and optimization seeking through a broad range of programming activities that learners must perform themselves. It is not reasonable to consider that teachers could evaluate all attempts that the average learner should develop multiplied by the number of students enrolled in a course, much less in a timely, deep, and fair fashion. Unsurprisingly, exploring the formal structure of programs to automate the assessment of certain features has long been a hot topic among CS education practitioners. Assessing a program is considerably more complex than asserting its functional correctness, as the proliferation of tools and techniques in the literature over the past decades indicates. Program efficiency, behavior, and readability, among many other features, assessed either statically or dynamically, are now also relevant for automatic evaluation. The outcome of an evaluation evolved from the primordial Boolean values to information about errors and tips on how to advance, possibly taking into account similar solutions. This work surveys the state of the art in the automated assessment of CS assignments, focusing on the supported types of exercises, security measures adopted, testing techniques used, type of feedback produced, and the information they offer the teacher to understand and optimize learning. A new era of automated assessment, capitalizing on static analysis techniques and containerization, has been identified. Furthermore, this review presents several other findings from the conducted review, discusses the current challenges of the field, and proposes some future research directions.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 40