Elicitação de métricas para avaliação de aspectos de aprendizagem em jogos educacionais

Abstract

Educational games can contribute to different aspects of human development, including cognitive, motor, and social dimensions. Understanding how players interact with these games facilitates their evaluation and provides insights for improvements in game design and learning processes. However, much of the literature evaluates this capability through predominantly subjective approaches, usually based on questionnaires, focusing mainly on the game rather than on the player, and often disregarding interaction data during the analyses.

This dissertation proposes an approach to identify indicators of learning based on students' interaction data in educational games, aiming to verify whether the game achieves the objectives for which it was designed. For this purpose, nine metrics were defined, related to cognitive, motor, engagement, and curricular content aspects addressed in the game. These metrics aim to quantify different dimensions of student performance, such as problem-solving ability, adaptation to new situations, and mechanical skills, based on interaction data recorded during gameplay.

The research was conducted using the Design-Based Research (DBR) methodology, combining exploratory qualitative and quantitative analyses in a real educational context. Two experiments were carried out: the first with the game "Robô Euroi", involving five undergraduate students, and the second with the game "My Name", involving fifteen students from the Computer Science Teacher Education program. The metrics were applied to player interaction data to analyze their adaptation to the structure of the games.

The results indicate that the proposed metrics are correlated with the player's journey, providing relevant evidence for learning assessment. In addition, a comparative study was conducted between the metrics and the players' self-assessments. The results revealed significant differences between the participants' subjective perceptions and the values obtained through the metrics, suggesting greater accuracy in assessments based on interaction data.

Finally, the metrics were evaluated through questionnaires by human experts, including data analysts and teachers, as well as by intelligent agents specialized in psychometrics. Overall, the metrics were positively evaluated, highlighting their coherence with the player's journey and their potential to support learning assessment. As recommendations, it was suggested to clarify operational definitions, discuss limitations and potential biases, strengthen evidence of validity and reliability, and include examples and reference criteria to facilitate their application in educational contexts.