D.TRUMP: Data-mining Textual Responses to Uncover Misconception Patterns

D.TRUMP: Data-mining Textual Responses to Uncover Misconception Patterns

Joshua Michalenko, Andrew Lan, and Richard Baraniuk

An important, yet largely unstudied, problem in student data analysis is to detect misconceptions from students’ responses to open-response questions. Misconception detection enables instructors to deliver more targeted feedback on the misconceptions exhibited by many students in their class, thus improving the quality of instruction. In this paper, we propose D.TRUMP, a new natural language processing framework to detect the common misconceptions among students’ textual responses to open-response, short-answer questions. We introduce a probabilistic model for students’ textual responses involving misconceptions and experimentally validate it on a real-world student-response dataset. Preliminary experimental results show that D.TRUMP excels at classifying whether a response exhibits one or more misconceptions. More importantly, it can also automatically detect the common misconceptions exhibited across responses from multiple students to multiple questions; this is especially important at large scale, since instructors will no longer need to manually specify all possible misconceptions that students might exhibit.