Deep Knowledge Tracing On Programming Exercises

Deep Knowledge Tracing On Programming Exercises

Lisa Wang, Angela Sy, Larry Liu, and Chris Piech

Modeling student knowledge while students are acquiring new concepts is a crucial stepping stone towards providing person- alized automated feedback at scale. This task, also referred to as“knowledge tracing”, has been explored extensively on exer- cises where student submissions fall into a finite discrete so- lution space, e.g. a multiple-choice answer. However, knowl- edge tracing on open-ended problems where answers extend beyond binary solutions is difficult for Bayesian models. We believe a rich set of information about a student’s learning is captured within their responses to open-ended problems. This is a challenging task, but with recent advances in machine learning, there are more promising techniques to represent rich, complex entities in ways that are appropriate for machine learning tasks. In our work, we embed student programs into euclidean space. We then feed these vector representations into Recurrent Neural Nets (LSTMs) that predict a student’s performance on subsequent exercises. We demonstrate that this deep learning model is able to make future predictions for students who are solving “Hour of Code” programming exercises on “Code.org”, an online learning platform. In mak- ing predictions, the model learns nuanced patterns in how students’ code evolves over time. Our results are a first step in a line of research which may enable us to provide a temporal model that understands how students learn over time and is generalizable across domains and assignment types.