Rhetorical Approach to Assessing Online Discussion
Duane Graddy, Middle Tennessee State University, United States
Society for Information Technology & Teacher Education International Conference, in Nashville, Tennessee, USA ISBN 978-1-880094-44-0 Publisher: Association for the Advancement of Computing in Education (AACE), Chesapeake, VA
A key element in successful online learning is full and fruitful participation of learners in group discussions. These collaborative forums provide the basis for devising shared goals, encouraging comments, using the semantics of the subject area, developing critical thinking skills, providing personal examples, asking questions, lending support to others, sharing responsibility for completion of assignments, and promoting feedback. Online discussions provide a means of enhancing cognitive skills as well as a method for assessing learning outcomes. In these respects, online discussions are dynamic processes that develop over the life of a course. Knowledge is constructed in these collaborative forums through the process of social negotiations among the discussions' participants. Therefore, recognizable advancements in social and cognitive skills should be evident in the degree of content sophistication and the level of perspective taking by the learners. Various authors have characterized this collaborative learning process in different ways. For instance, Jarvela and Hakkinen describe web-based discussion as an ontogenetic process.[Jarvelia, 2000] They use the five stages of Selman's model of social cognitive development to portray the evolution of online course discussion.[Selman, 1980] Palloff and Pratt see online involvement as ideally evolving from initial dissonance to content mastery and then to transformative learning.[Palloff, 1999]. The cognitive analysis model of Henri [Henri, 1992] is built around four dimensions of interaction ranging from social interchange to revealing metacognitive skills. Gunawardena, Lowe, and Anderson [Gunawardena, 1997] theorize that the active construction of knowledge in online discussions moves through five phases. Rourke, et al [Rourke, 2001] considers the issue from a model of community of inquiry. In this model, deep and meaningful learning occurs through the interaction of cognitive presence, teaching presence, and social presence. Others [Bonk, 1998] approach the development of online course discussion in the context of Bloom's cognitive taxonomy. With the sophistication of learner questioning evolving from the lower stages to the synthesis and evaluation levels. The purpose of this paper is to assess the fruition of online course discussions by analyzing the transcripts of these interactions for a graduate survey course in economics. The method is unlike other works in that it is lexical in nature. Dictionaries are used to develop quantitative measures of the linguistic characteristics of the online conversations. These rhetorical facts are used to assess the linguistics of general course discussions and specific team discussions. The aim is to use this rhetorical information to shed light on several important questions. 1.Does the character of the discussion change over the life of the course? Does the linguistic nature and tone of the conversation change as discussions develop? Is there linguistic evidence of moving from low-level discussion to deeper collaborative synthesis? 2.Can generalization be made about communality, accomplishment, optimism, and understanding from the rhetorics of the discourse? Are there key indicators of conversational sophistication? 3.Do the linguistics of the messages reveal anything about the learners and the learning process? Can linguistic indicators suggest anything about conversational sophistication of individual learners and about the entire class? This paper uses Diction 5.0 [Hart, 1999] to analyze the content of the online discussions. This lexically based program searches the content of online discussions for five semantic features as well as thirty-five sub-features. Each semantic feature is scored on the basis of the thirty-five sub-feature (dictionaries) according to a series formulas incorporated into the software. Appendix 1 describes the scoring process in more detail. The online discussions are compared to forty standard dictionaries and word lists in segments of 500 word. For discussions exceeding 500 words, the overall scores are the average of the individual 500 word segments. In discussions not evenly divisible into 500 word units, the scores for the uneven segments are weighted proportionally to their word count. [DiLeo, 1999] Custom dictionaries on microeconomics, macroeconomics and noted economists were also included in the database. No terms are duplicated between the micro and macro dictionaries. The noted economists dictionary is a list of surnames of famous individuals in the evolution of economic thought. The custom dictionaries focus on word usage in contrast to calculating a specific score. The sample for this study includes thirteen graduate students in an MBA prerequisite course focusing on the principles of economics. None of the students had taken the principles of economics as undergraduates so this was their first formal exposure to the material. Three discussion forums are analyzes in this paper. The combined scores on participation in the forums counted twenty percentage of the students overall grade. The forums dealt with assigned topics. The first one center on an article by James Fallows entitled "What's an Economy For?" which compared the economic structures of Western and Asian economies. The California energy crisis was the theme of the second forum. The third forum, which was somewhat shorter than the other two, focused the use of the Federal government surplus for tax cuts versus additions to the Social Security trust fund. Forums one and two ran for a week each while forum three lasted approximately three days. The results of the study provide evidence of the usefulness of rhetorical assessment. Bonk, C. and W. Sugar. "Student Role Play in the World Forum:Analysis of an Arctic Learning Apprenticeship." Interactive Learning Environments, 1998, 6(1-2), pp. 1-29. DiLeo, D. "Influences on Realism in Governors' State-of-the-State Speeches." Commonwealth:A Journal of Political Science, 1999. Gunawardena, L., C. Lowe, and T. Anderson. "Interaction Analysis of a Global on-Line Debate and the Development of a Constructivist Interaction Analysis Model for Compuer Conferencing." Journal of Educational Computing Research, 1997, 17(4), pp. 395 429. Henri, F. " Computer Conferncing and Content Analysis," A. Kaye, Collaborative Learning through Computer Conferencing:The Najaden Papers. Berlin: Spinger-Verlag, 1992, 117-36. Jarvelia, Sanna and Paivi Hakkinen. "Levels of Web-Based Discussion: Theory of Perspective-Taking as a Tool for Analyzing Interaction," B. F. a. S. O'Connor-Divelbiss, Fourth International Conference of Learning Sciences. Erlbaum, 2000, 22-26. Mezirow, J. Fostering Critical Reflection in Adulthood: A Guide to Transformative and Emancipatory Learning. San Francisco: Jossey-Bass, 1990. Palloff, Rena and Keith Pratt. Building Learning Communities in Cyberspace. Jossey-Bass Publisher, 1999. Rourke, L., T. Anderson, D. Garrison, and W. Archer. "Assessing Social Presence in Asynchronous Text-Based Computer Conferencing." Journal of Distance Education, 2001, pp. 1-22. Selman, R. L. The Growth of Interpersonal Understanding. New York: Academic Press, 1980. Thomas, M. "The Impacts of Technology on Communication-Mapping the Limits of Online Discussion Forms." Intranet Project, The University of Adelaide, 2000, pp. 1-6.
Graddy, D. (2002). Rhetorical Approach to Assessing Online Discussion. In D. Willis, J. Price & N. Davis (Eds.), Proceedings of SITE 2002--Society for Information Technology & Teacher Education International Conference (pp. 173-174). Nashville, Tennessee, USA: Association for the Advancement of Computing in Education (AACE).