Controlled experiment replication in evaluation of e-learning system’s educational influence
Computers & Education Volume 53, Number 3, ISSN 0360-1315 Publisher: Elsevier Ltd
We believe that every effectiveness evaluation should be replicated at least in order to verify the original results and to indicate evaluated e-learning system’s advantages or disadvantages. This paper presents the methodology for conducting controlled experiment replication, as well as, results of a controlled experiment and an internal replication that investigated the effectiveness of intelligent authoring shell eXtended Tutor–Expert System (xTEx-Sys). The initial and the replicated experiment were based on our approach that combines classical two-group experimental design and with factoral design. A trait that distinguishes this approach from others is the existence of arbitrary number of checkpoint-tests to determine the effectiveness in intermediate states. We call it a pre-and-post test control group experimental design with checkpoint-tests. The gained results revealed small or even negative effect sizes, which could be explained by the fact that the xTEx-Sys’s domain knowledge presentation is rather novel for students and therefore difficult to grasp and apply in earlier phases of the experiment. In order to develop and improve the xTEx-Sys, further experiments must be conducted.
Grubisic, A., Stankov, S., Rosic, M. & Zitko, B. (2009). Controlled experiment replication in evaluation of e-learning system’s educational influence. Computers & Education, 53(3), 591-602. Elsevier Ltd.