Humans and Machines Together

Humans and Machines Together: Improving Characterization of Large Scale Online Discussions through Dynamic Interrelated Post and Thread Categorization (DIPTiC)

Yi Cui, Wan Qi Jin and Alyssa Wise

This paper presents a thread characterization method that compares categorization results for thread starters and replies made by a previously-developed natural language model, using human judgment to resolve discrepancies. In an example application using the complete discussion forum data from a MOOC on medical statistics, the method increased the estimation of classification accuracy from .81 to .88 with the addition of a minimal number of human hours.