Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments Article
Ryan Baker, Carnegie Mellon University, United States ; Jason Walonoski, The MITRE Corporation, United States ; Neil Heffernan, Worcester Polytechnic Institute, United States ; Ido Roll, Albert Corbett, Kenneth Koedinger, Carnegie Mellon University, United States
Journal of Interactive Learning Research Volume 19, Number 2, ISSN 1093-023X Publisher: Association for the Advancement of Computing in Education (AACE), Waynesville, NC
In recent years there has been increasing interest in the phenomena of “gaming the system,” where a learner attempts to succeed in an educational environment by exploiting properties of the system's help and feedback rather than by attempting to learn the material. Developing environments that respond constructively and effectively to gaming depends upon understanding why students choose to game. In this article, we present three studies, conducted with two different learning environments, which present evidence on which student behaviors, motivations, and emotions are associated with the choice to game the system. We also present a fourth study to determine how teachers' perspectives on gaming behavior are similar to, and different from, researchers' perspectives and the data from our studies. We discuss what motivational and attitudinal patterns are associated with gaming behavior across studies, and what the implications are for the design of interactive learning environment.
Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A. & Koedinger, K. (2008). Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments. Journal of Interactive Learning Research, 19(2), 185-224. Waynesville, NC: Association for the Advancement of Computing in Education (AACE). Retrieved October 18, 2018 from https://www.learntechlib.org/primary/p/24328/.
© 2008 Association for the Advancement of Computing in Education (AACE)
- Aleven, V. (2001, October). Helping students to become better help seekers: Towards supporting metacognition in a cognitive tutor. Paper presented at the German-USA Early Career
- Aleven, V., McLaren, B.M., Roll, I., & Koedinger, K.R. (2004, August) Toward tutoring help seeking: Applying cognitive modeling to meta-cognitive skills. Proceedings of the 7th International Conference on Intelligent Tutoring Systems (ITS2004, pp. 227-239), Maceió, Brazil.
- Aleven, V., McLaren, B.M., Roll, I., & Koedinger, K.R. (2006). Toward meta-cognitive tutoring: A model of help seeking with a cognitive tutor. International Journal of Artificial Intelligence in Education, 16, 101-130.
- American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders: DSM-IV. Washington, DC: American Psychiatric Publishing.
- Anderson, J.R., Corbett, A.T., Koedinger, K.R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207.
- Arbreton, A. (1998). Student goal orientation and help-seeking strategy use. In S.A. Karabenick (Ed.), Strategic help seeking: Implications for learning and teaching (pp. 95-116). Mahwah, NJ: Lawrence Erlbaum.
- Arroyo, I., & Murray, T. (2002, June). Towards measuring and maintaining the zone of proximal development within the AnimalWatch intelligent tutoring system. Proceedings of the 6th International Conference on Intelligent Tutoring Systems (ITS 2002) Biarritz, France. Arroyo, I., & Woolf, B. (2005, July). Inferring learning and attitudes from a Bayesian network of log file data. Proceedings of the 12th International Conference on Artificial Intelligence in Education, (pp. 33-40), Amsterdam, The Netherlands.
- Baker, R.S. (2005). Designing intelligent tutors that adapt to when students game the system. Doctoral dissertation, Carnegie Mellon University, Pittsburgh, PA. Published as CMU Technical Report CMU-HCII-05-104.
- Baker, R.S., Corbett, A.T., & Koedinger, K.R. (2004, August). Detecting student misuse of intelligent tutoring systems. Proceedings of the 7th International Conference on Intelligent Tutoring Systems, (pp. 531-540), Maceió, Brazil.
- Baker, R.S., Corbett, A.T., Koedinger, K.R., & Wagner, A.Z. (2004, April). Off-task behavior in the cognitive tutor classroom: When students "game the system.” Proceedings of ACM CHI 2004: Computer-Human Interaction, (pp. 383-390), Vienna, Austria.
- Beal, C.R., Qu, L., & Lee, H. (2006, July). Classifying learner engagement through integration of multiple data sources. Paper presented at the 21st National Conference on Artificial Intelligence (AAAI-2006), Boston, MA.
- Baker, R.S., Roll, I., Corbett, A.T., & Koedinger, K.R. (2005, August). Do performance goals lead students to game the system? Proceedings of the International Conference on Artificial Intelligence and Education (AIED2005; pp.57-64), Amsterdam, The Netherlands.
- Beck, J. (2005, June). Engagement tracing: Using response times to model student disengagement. Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED 2005, pp. 88-95) Amsterdam, The Netherlands.
- Bickmore, T.W., & Picard, R.W. (2004). Towards caring machines. CHI Extended Abstracts, (pp. 1489-1492).
- Carnegie Learning (2002). Cognitive tutor: Summary of results report. Retrieved February 26, 2008, from http://www.carnegielearning.com/web_docs/summary_of_results.pdf
- Cheng, R., & Vassileva, J. (2005, June). Adaptive reward mechanism for sustainable online learning community. Proceedings of the 12th International Conference on Artificial Intelligence in Education, (pp. 152-159), Amsterdam, The Netherlands.
- Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology, 88(4), 715–730.
- Corbett, A.T., & Anderson, J.R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4, 253-278.
- Cupach, W.R., & Spitzberg, B.H. (1983). Trait versus state: A comparison of dispositional and situational measures of interpersonal communication competence. The Western Journal of Speech Communication, 47, 364-379.
- De Angeli, A., Brahnam, S., Wallis, P., & Dix, A. (2006, April). Proceedings of the CHI2006 workshop on Misuse and Abuse of Interactive Technologies (pp. 1647-1650), Montreal, Canada.
- Dillon, T.W., Garner, M., Kuilboer, J., & Quinn, J.D. (1998). Accounting student acceptance of tax preparation software. Journal of Accounting and Computers, 13, 17-29.
- Dweck, C.S. (2000). Self-theories: Their role in motivation, personality, and development. Philadelphia, PA: Psychology Press.
- Frantom, C.G., Green, K.E., & Hoffman, E.R. (2002). Measure development: The children’s attitudes toward technology scale (CATS). Journal of Educational Computing Research, 26(3), 249-263.
- Harnisch, D.L., Hill, K.T., & Fyans, L.J. (1980, April). Development of a shorter, more reliable, and more valid measure of test motivation. Paper presented at the 1980 Annual Meeting of the National Council on Measurement in Education, Boston, MA. (ERIC Document Reproduction Service No. ED193273)
- Johns, J., & Woolf, B. (2006, July). A dynamic mixture model to detect student motivation and proficiency. Paper presented at the 21st National Conference on Artificial Intelligence (AAAI2006), Boston, MA.
- Kelley, T., & Littman, J. (2001). The art of innovation: Lessons in creativity from IDEO, America’s leading design firm. New York: Doubleday.
- Klawe, M.M. (1998, November). Designing game-based interactive multimedia mathematics learning activities. Proceedings of UCSMP International Conference on Mathematics Education, Chicago,IL.
- Knezek, G., & Christensen, R. (1995). Computer attitudes questionnaire. Denton, TX: Texas Center for Educational Technology.
- Koedinger, K.R., Anderson, J.R., Hadley, W.H., & Mark, M. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43. Lewis, J.R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57-78. Magnussen, R., & Misfeldt, M. (2004, December). Player transformation of educational multiplayer games. Proceedings of Other Players, Copenhagen, Denmark. Retrieved February 26, 2008, from http://www.itu.dk/op/proceedings.htm
- Maris, E. (1995). Psychometric latent response models. Psychometrika, 60(4), 523-547. Martínez-Mirón, E.A., du Boulay, B., & Luckin, R. (2004, August). Goal achievement orientation in the design of an ILE. Proceedings of the ITS2004 Workshop on Social and Emotional Intelligence in Learning Environments, (pp. 72-78), Maceió, Brazil.
- McCrae, R.R., & Costa, P.T. (1990). Personality in adulthood. New York: Guilford University Press. Mitrovic, A., & Martin, B. (2003, July). Scaffolding and fading problem selection in SQL-tutor. Proceedings of the 11th International Conference on Artificial Intelligence in Education, (pp. 479-481), Sydney, Australia.
- Mueller, C.M., & Dweck, C.S. (1998). Praise for intelligence can undermine children’s motivation and performance. Journal of Personality and Social Psychology, 75(1), 33-52.
- Murray, R.C., & VanLehn, K. (2005, June). Effects of dissuading unnecessary help requests while providing proactive help. Proceedings of the 12th International Conference on Artificial Intelligence in Education, (pp. 887-889), Amsterdam, The Netherlands.
- Paas, F., & Van Merrienboer, J. (1994). Variability of worked examples and transfer of geometry problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122-133.
- Parker, G., & Hadzi-Pavlovic, D. (2001). A question of style: Refining the dimensions of personality disorder style. Journal of Personality Disorders, 15(4), 300-318.
- Pettitt, L. (1995, April). Middle school students’ perception of math and science abilities and related careers. Paper presented at the 61st Biennial Meeting of the Society for Research in Child Development, Indianapolis, IN.
- Rhodewalt, F. (1994). Conceptions of ability, achievement goals, and individual differences in self-handicapping behavior: On the application of implicit theories. Journal of Personality, 62, 67-85.
- Sansone, C., Wiebe, D.J., & Morgan, C. (1999). Self-regulating interest: The moderating role of hardiness and conscientiousness. Journal of Personality, 67(4), 701- 733.
- Sarason, S.B. (1978). Anxiety in elementary school children: A report of research. Westport, CT: Greenwood Press.
- Schiefele, U. (1991). Interest, learning, and motivation. Educational Psychologist, 26(3&4), 299-323. Schofield, J.W. (1995). Computers and classroom culture. Cambridge, UK: Cambridge University Press.
- Selwyn, N. (1997). Students’ attitudes towards computers: Validation of a computer attitude scale for 16-19 education. Computers & Education, 28, 35-41.
- Stevens, R., & Soller, A. (2005). Machine learning models of problem space navigation: The influence of gender. Journal of Computer Science and Information Systems, 2(2), 83-98. Stipek, D., & Gralinski, J.H. (1996). Children’s beliefs about intelligence and school performance. Journal of Educational Psychology, 88(3), 397-407.
- Stone, M.H. (1993). Abnormalities of personality: Within and beyond the realm of treatment. New York: W.W. Norton.
- Venkatesh, V., & Davis, F.D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186-204.
- Vygotsky, L. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.
- Walonoski, J.A., & Heffernan, N.T. (2006a,June). Detection and analysis of off-task gaming behavior in intelligent tutoring systems. In Ikeda, Ashley & Chan (Eds.), Proceedings of the 8th International Conference on Intelligent Tutoring Systems, (pp. 382-391), Jhongli, Taiwan. Berlin: Springer-Verlag.
- Walonoski, J.A., & Heffernan, N.T. (2006b, June). Prevention of off-task gaming behavior in intelligent tutoring systems. In Ikeda, Ashley & Chan (Eds.), Proceedings of the 8th International Conference on Intelligent Tutoring Systems, (pp. 722-724), Jhongli, Taiwan. Berlin: Springer-Verlag.
- Wood, H., & Wood, D. (1999). Help seeking, learning, and contingent tutoring. Computers and Education, 33, 153-169.
These references have been extracted automatically and may have some errors. If you see a mistake in the references above, please contact firstname.lastname@example.org.
Tenzin Doleck, McGill University, Canada; Eric Poitras, University of Utah, United States; Laura Naismith, University Health Network, Toronto Western Hospital, Canada; Susanne Lajoie, McGill University, Canada
EdMedia + Innovate Learning 2016 (Jun 28, 2016) pp. 498–503
These links are based on references which have been extracted automatically and may have some errors. If you see a mistake, please contact email@example.com.