Prompting Analysis for Programming Exercises in Computer Science Courses

Shiguihara P.; Díaz E.; Ceballos D.; Dioses J.; Morán J. · ICACIT 2025 - Proceedings: 11th International Symposium on Accreditation of Engineering and Computing Education (2025) · DOI: 10.1109/icacit66767.2025.11369586

Resumen

This study explores how different prompting strategies influence student evaluations of programming exercises in computer science education. We analyze six prompting types—Zero-Shot, Few-Shot, Chain-of-Thought, Role, Instruction, and Multi-Turn—comparing student-generated assessments with expert judgments under optimistic, neutral, and pessimistic approaches. Our findings reveal that structured prompts, particularly Chain-of-Thought and Role, lead to higher alignment with expert evaluations, especially under stricter assessment conditions. However, all strategies exhibit performance variability and low-end outliers, suggesting that prompting alone does not guarantee expert-level reasoning. These results underscore the need for adaptive prompt design that accounts for diverse student profiles and cognitive patterns.

Instituciones: Universidad San Ignacio de Loyola