EXSPRT: An Expert Systems Approach to Computer-Based Adaptive Testing
Expert systems can be used to aid decision making. A computerized adaptive test (CAT) is one kind of expert system, although it is not commonly recognized as such. A new approach, termed EXSPRT, was devised that combines expert systems reasoning and sequential probability ratio test stopping rules. EXSPRT-R uses random selection of test items, whereas EXSPRT-I incorporates an intelligent selection procedure based on item utility coefficients. These two new methods are compared to the traditional SPRT and to an adaptive mastery testing (AMT) approach based on item response theory (IRT). Three empirical studies using different tests and examinees were conducted. Study 1 included samples of 25 and 50 current or former graduate students who took the Digital Authoring Language Test; Study 2 included samples of 25, 50, 75, and 100 students in an introductory graduate-level course; and Study 3 included 333 college freshmen and sophomores. Results indicate that the EXSPRT-I is more efficient or as efficient as is the AMT model. When the distribution of examinees was not clustered near the mastery cutoff, all four methods made accurate mastery classifications. Although further research is needed, the EXSPRT initially appears to be a strong alternative to IRT-based adaptive testing when categorical decisions about examinees are desired. The EXSPRT is less complex conceptually and mathematically; and it appears to require many fewer examinees to empirically establish a rule base, when compared to the large numbers required to estimate parameters for item response functions in the IRT model. (TJH)
Frick, T.W. EXSPRT: An Expert Systems Approach to Computer-Based Adaptive Testing.