You are here:

A Review of Research Methods for Assessing Content of Computer-Mediated Discussion Forums
Article

, University of Missouri, United States

Journal of Interactive Learning Research Volume 17, Number 3, ISSN 1093-023X Publisher: Association for the Advancement of Computing in Education (AACE), Waynesville, NC

Full Text

Abstract

Computer-mediated communication (CMC) technologies are becoming increasingly important components of online educational environments (Romiszowski & Mason, 2004). The online discussion forum that is a significant component of many web-based courses is one type of CMC. Instructors and students rely on these asynchronous forums to engage one another in ways that replace face-to-face communication. The goal of such interactions is to promote critical thinking, meaningful problem solving, and knowledge construction (Kirschner, Strijbos, Kreijns, & Beers, 2004). In spite of the importance of these forums, the most common methods for assessing the content and outcomes of these forums have often been limited to frequency counts and other quantitative measures (Mason, 1992; Romiszowski & Mason, 2004). In order to assess any meaning that results from these discussions, it is necessary to perform some kind of semantic analysis of them. This article provides an overview of current methods of quantitative and qualitative research paradigms for analyzing the content of asynchronous computer-mediated discussion forums. For each analysis method we describe the research method, provide an example of a research study that used that method, describe the types of questions that this method can address, and compare the methods in terms of validity and reliability. Researchers and instructors will be able to use this analysis to become familiar with the choices available and make decisions about appropriate methods for analyzing CMC forums.

Citation

Marra, R. (2006). A Review of Research Methods for Assessing Content of Computer-Mediated Discussion Forums. Journal of Interactive Learning Research, 17(3), 243-267. Waynesville, NC: Association for the Advancement of Computing in Education (AACE). Retrieved August 12, 2022 from .

© 2006 Association for the Advancement of Computing in Education (AACE)

Keywords

References

View References & Citations Map
  1. Asbell-Clarke, J., & Foster, J. (2004). Theoretical frameworks for online learning environments, TERC publications. Retrieved May 10, 2004, from http://www.terc.edu .
  2. Bakeman, R. & Gottman, J. (1997). Observing interaction: An introduction to sequential analysis. Cambridge, MA: Cambridge University Press.
  3. Beaudrie, B. P. (2000). Analysis of group problem-solving tasks in a geometry course for teachers using computer-mediated conferencing. Unpublished Ed.D. Thesis, Montana State University, Bozeman.
  4. Bell, M. C., & Linn, P.(1977). Scientific arguments as learning artifacts: Designing for learning on the web. Paper presented at AERA, Chicago, IL.
  5. Carswell, L., Thomas, P., Petre, M., Price, B., & Richards, M. (2000). Distance education via the Internet: The student experience. British Journal of Educational Technology, 31(1), 29 – 46.
  6. Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6(3), 271 - 315.
  7. Cho, K. L., & Jonassen, D. H. (2002). The effects of argumentation scaffolds on argumentation and problem solving. Educational Technology: Research & Development, 50(3), 5-22.
  8. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(10), 37 – 46.
  9. Condon, S. L, & Cech, C. G. (1996). Discourse management strategies in face-to-face and computer mediated decision making interaction. Electronic Journal of communication/LA Revue Electronique de Communication, 6(3).
  10. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Forth Worth, TX: Harcourt Brace Jovanovich College Publishers.
  11. Cumming, S., & Ono, T. (1997). Discourse and grammar. In T. Van Dijk, (Ed). Discourse as structure and process, pp. 257-291. Thousand Oaks, CA: Sage
  12. Garrison, D. R. (1992). Critical thinking and self-directed learning in adult education: an analysis of responsibility and control issues. Adult Education Quarterly, 42(3), 136-148.
  13. Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105. Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7-23.
  14. Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397 –431.
  15. Guzdial (1995). Software-realized scaffolding to facilitate programming for science learning. Interactive Learning Environments, 3(4): 1-44.
  16. Harasim, L. M. (1989). Online education as a new domain. In R. Mason & A. R. Kaye (Eds.), Mindweave: Communication, computers and distance education (pp. 50 - 62). Oxford: Pergamon Press.
  17. Henri, F. (1992). Computer conferencing and content analysis. In Designing Environments for Constructivist Learning A. R. Kaye (Ed.), Collaborative Learning Through Computer Conferencing (pp. 117-136). Berlin: Springer-Verlag. Marra
  18. Jeong, A. (2003). Sequential analysis of group interaction and critical thinking in online threaded discussions. The American Journal of Distance Education, 17(1), 25-43. Retrieved February 17, 2004 from http://bbproject.tripod.com/SequentialAnalysis_Jeong2003.pdf.
  19. Jonassen, D. H., Mayes, T. & McAleese, R. (1993). A Manifesto for a constructivist approach to technology in higher education. In T. Duffy, J. Lowycky, & D. Jonassen (Eds.). Berlin: SpringerVerlag.
  20. Jonassen, D. H., & Kwon, H. (2001). Communication patterns in computer mediated versus face-to-face group problem solving. Educational Technology Research and Development, 49(1), 1042 – 1629.
  21. Jonassen, D. H., & Remidez, H. (2005). Mapping alternative discourse structures onto computer conferences. International Journal of Knowledge and Learning, 1(1/2), 113-129.
  22. Kiesler, S., Siefel, J., & McGuire, T. (1984). Social psychological aspects of computer-mediated communication. American Psychologist, 39, 1123-1134.
  23. Kirschner, P., Strijbos, J., Kreijns, K., & Beers, P. (2004). Designing electronic collaborative learning environments. Educational Technology Research and Development, 52(3), 47-66. Krippendorf, K. (1980). Content analysis: An introduction to its methodology. Vol. 5. The Sage COMMTEXT Series, Beverly Hills: Sage Publications.
  24. Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159 -174.
  25. Marra, R. M., Moore, J., & Klimczek, A. (2004) A comparative analysis of content analysis protocols for online discussion forums. Educational Technology Research and Development, 52(2), 23 - 40.
  26. Mason, R. (1992). Evaluation methodologies for computer conferencing applications. In A. R. Kaye (Ed.), Collaborative Learning Through Computer Conferencing (pp. 105 - 116). Berlin: Springer-Verlag. Mazur, J.
  27. McCreary, E. K. (1990). Three behavioral models for computer-mediated communication. In
  28. Newman, D. R., Webb, B., & Cochrane, C. (1996). A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Retrieved February 15, 2005, from http://www.qub.ac.uk/mgt/papers/methods/contpap.html.
  29. Newman, D. R., Johnson, C., Webb, B., & Cochrane, C. (1997). Evaluating the quality of learning in computer supported co-operative learning. Journal of the American Society for Information Science, 48(6), 484-495.
  30. Oh, S. (2004). The role of an argumentation scaffolding tool and epistemological beliefs on solving problems in online discussion. Unpublished doctoral dissertation, University of Missouri, Columbia.
  31. Olaniran, B. A., Savage, G. T., & Sorenson, R. L. (1996). Experimental and experiential approaches to teaching face-to-face and computer-mediated group discussion. Communication Education, 45, 244-259.
  32. Pea, R. (1993). Practices of distributed intelligence and designs for education. In G. Salomon (Ed.), Distributed Cognitions. Cambridge, MA: Cambridge University Press. Pullinger, D. J.
  33. Reeves, T., Herrington, J., & Oliver, R. (2004). A development research agenda for online collaborative learning. Educational Technology Research and Development, 52(4), 53 – 65. Romiszowski, A., & Mason, R. (2004). Computer-mediated communication. In D.H. Jonassen
  34. Rourke, L., Anderson, T., Garrison, R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts, International Journal of Artificial Intelligence in Education, 12(10), 8-22.
  35. Scardamalia, M., & Bereiter, C.(1994). Computer support for knowledge-building communities. The Journal of the Learning Sciences, 3(3), 265-283.
  36. Stemler, S. (2001). An overview of content analysis. Practical Assessment, Research & Evaluation, 7(17). Retrieved February 17, 2005 from http://PAREonline.net/getvn.asp?v=7&n=17.
  37. Strauss, A. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Thousand Oaks, CA: Sage Publications.
  38. Strijbos, J. W., Martens, R. L., Prins, F. J., & Jochems, W. M. G. (2006). Content analysis: What are they talking about? Computers & Education 46(1), 29 - 48.
  39. Suthers, D. D., & Hundhausen, C. D. (2001). Learning by constructing collaborative representations: An empirical comparison of three alternatives. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.), European perspectives on computer-supported collaborative learning (pp. 577-584). Universiteit Masstricht.
  40. Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
  41. Warkentin, M. E., Sayeed, L., & Hightower, R. (1997). Virtual teams versus face-to-face teams: An exploratory study of a web-based conference system. Decision Sciences, 28, 975 – 996.
  42. 1The reader is referred to Romiszowski and Mason (2004) for a treatment of these methodologies relative to CMC. 2Note that the original work further defines each phase. For instance, Phase II included further definitions for sub-phases A – C, and for Phase III there are sub-phases A-E.

These references have been extracted automatically and may have some errors. Signed in users can suggest corrections to these mistakes.

Suggest Corrections to References

Cited By

View References & Citations Map

These links are based on references which have been extracted automatically and may have some errors. If you see a mistake, please contact info@learntechlib.org.