Evaluation of Mathematics Achievement Test: A Comparison Between Classical Test Theory (CTT) and Item Response Theory (IRT)

Abstract

Item bias is critical to the process of evaluating the quality of an educational assessment in terms of reliability and
validity. This study applied the Classical Test Theory and Item Response Theory to evaluate the quality of an assessment
constructed by the researchers to measure National Certificate of Education (NCE) students’ achievement in Mathematics. The
sample for this study consisted of the junior and senior Mathematics and English major teacher-education student from the Abia
State College of Education, Arochukwu. A sample of 80 students was drawn for this study. The Mathematics Achievement Test
(MAT) for College students developed by the authors was used. Data was analyzed in two dimensions. First, the psychometric
properties of the instrument were analyzed using CTT and IRT and the detection of item bias was performed using the method
for Differential Item Functioning (DIF). The results showed that although Classical Test Theory (CTT) and Item Response Theory
(IRT) methods are different in so many ways, outcome of data analysis using the two methods in this study did not say so. Items
which where found to be “bad items” in CTT came out not fitting also in the Rasch Model.