Automated Short-Answer Grading Using Deep Neural Networks and Item Response Theory

Automated short-answer grading (ASAG) methods using deep neural networks (DNN) have achieved state-of-the-art accuracy. However, further improvement is required for high-stakes and large-scale examinations because even a small scoring error will affect many test-takers. To improve scoring accuracy, we propose a new ASAG method that combines a conventional DNN-ASAG model and an item response theory (IRT) model. Our method uses an IRT model to estimate the test-taker’s ability from his/her true-false responses to objective questions that are offered with a target short-answer question in the same test. Then, the target short-answer score is predicted by jointly using the ability value and a distributed short-answer representation, which is obtained from an intermediate layer of a DNN-ASAG model.