You are here:

Impact of AWE Rubrics and Automated Assessment on EFL Writing Instruction
ARTICLE

, School of Online and Continuing Education, Beijing Foreign Studies University, Beijing, China ; , School of English and International Studies, Beijing Foreign Studies University, Beijing, China

IJCALLT Volume 7, Number 2, ISSN 2155-7098 Publisher: IGI Global

Abstract

This paper addressed a gap in research literature on rubrics by investigating how the formative use of rubrics of an automated writing evaluation (AWE) tool, the Writing Roadmap, along with a novel type of meta-cognitive activity, i.e., automated assessment, assisted EFL writing instruction. A one-year teaching experiment incorporating the use of the tool was undertaken at the tertiary level. A mixed-methods research approach in the form of questionnaires, interviews, and the participants' interim project reports was employed to evaluate the efficacy of the teaching experiment. The research demonstrated that formative use of AWE rubrics along with automated assessment mediated writing instruction via offering timely and objective assessment, aligning teaching and assessment goals, aiding the feedback process, increasing student-teacher interactions, and promoting learner autonomy.

Citation

Tang, J. & Wu, Y. (2017). Impact of AWE Rubrics and Automated Assessment on EFL Writing Instruction. International Journal of Computer-Assisted Language Learning and Teaching, 7(2), 58-74. IGI Global. Retrieved May 18, 2022 from .

Keywords

References

View References & Citations Map
  1. Andrade, H., Wang, X.L., Du, Y., & Akawi, R.L. (2009). Rubric-referenced self-assessment and self-efficacy for writing. The Journal of Educational Research, 102(4), 287–301.
  2. Beeth, M.E., Cross, L., Pearl, C., Pirro, J., Yagnesak, K., & Kennedy, J. (2001). A continuum for assessing science process knowledge in grades K-6. Electronic Journal of Science Education, 5(3). Retrieved July 21, 2017 from http://wolfweb.unr.edu/homepage/crowther/ejse/beethetal.html
  3. Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31.
  4. Brookhart, S. (2014). How to create and use rubrics for formative assessment and grading. ASCD. Retrieved January 10, 2015 from http://www.ascd.org/publications/books/112001/chapters/What-Are-Rubrics-and-WhyAre-They-Important%C2%A2.aspx
  5. Brown, G.T.L., Glasswell, K., & Harland, D. (2004). Accuracy in the scoring of writing: Studies of reliability and validity using a New Zealand writing assessment system. Assessing Writing, 9(2), 105–121.
  6. Brown, S., & Knight, P. (1994). Assessing Learners in Higher Education. Kogan Page Limited.
  7. Brown, S., Race, P., & Smith, B. (1996). An assessment manifesto. 500 Tips on Assessment. Retrieved August 18, 2006 from http://www.city.londonmet.ac.uk/deliberations/assessment/manifest.html
  8. CCCC Executive Committee. (2004). CCCC position statement on teaching, learning, and assessing writing in digital environments. NCTE (National Council of Teachers of English). Position Papers.
  9. Chen, E.E., & Cheng, E. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning& Technology, 12(2), 94–112.
  10. Crusan, D. (2010). Assessment in the Second Language Writing Classroom. Ann Arbor, MI: University of Michigan Press.
  11. Crusan, D. (2015). Editorial: Dance, ten; looks, three: Why rubrics matter. Assessing Writing, 26, 1–4. Doi:10.1016/J.asw.2015.08.002CTB/McGraw-Hill.(2008).WritingRoadmap. Monterey, CA: Assessment Rubrics.
  12. Dempsey, M.S., PytlikZillig, L.M., & Bruning, R.H. (2009). Helping preservice teachers learn to assess writing: Practice and feedback in a Web-based environment. Assessing Writing, 14(1), 38–61. Doi:10.1016/J.Asw.2008.12.003
  13. Elliot, N., & Williamson, D.M. (2013). Assessing Writing special issue: Assessing writing with automated scoring systems. Assessing Writing, 18(1), 1–6.
  14. Ellis, R. (2007). Corrective feedback in theory, research and practice. Keynote speech presented at the Fifth International Symposium of ELT in China, Beijing, China.
  15. Ericsson, P.F. (2006). The meaning of meaning: Is a paragraph more than an equation? In P.F. Ericsson& R. Haswell (Eds.), Machine Scoring of Student Essays: Truth and Consequences (pp. 28–37). Logan, UT: Utah
  16. Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of Automated Writing Evaluation. The Journal of Technology, Learning, and Assessment, 8(6), 4–43.
  17. Heilman, M., & Tetreault, J. (2012). Using automated scoring to analyze student writing. Paper presented at Georgetown University RoundTable on Languages and Linguistics (GURT), Washington DC.
  18. Hounsell, D. (1987). Essay writing and the quality of feedback. In J. Richardson, M.W. Eysenck, & D.W. Piper (Eds.), Student Learning: Research in Education and Cognitive Psychology (pp. 109–119). Milton Keynes:
  19. Jin, Y. (2010). The formative assessment function of standardized language tests. In Proceedings of lecture at Beijing Jiaotong University, Beijing.
  20. Jonsson, A. (2010). The use of transparency in the Interactive Examination for student teachers. Assessment in Education: Principles, Policy& Practice, 17(2), 183–197.
  21. Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130–144. Doi:10.1016/J.edurev.2007.05.002
  22. Klobucar, A., Elliot, N., Deess, P., Rudniy, O., & Joshi, K. (2013). Automated scoring in context: Rapid assessment for placed students. Assessing Writing, 18(1), 62–84. Doi:10.1016/J.asw.2012.10.001
  23. Laurian, S., & Fitzgerald, C.J. (2013). Effects of using rubrics in a university academic level Romanian literature class. Procedia: Social and Behavioral Sciences, 76, 431–440.
  24. Lefrancois, G.R. (2004). Theories of Human Learning: What the Old Man Said (4thedition). Beijing: Foreign Language Teaching and Research Press.
  25. Li, J.R., & Lindsey, P. (2015). Understanding variations between student and teacher application of rubrics. Assessing Writing, 26, 67–79.
  26. Li, J.R., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1–18.
  27. Luft, J. (1998). Rubrics: Design and use in science teacher education. Paper presented at the Annual Meeting of the Association for the Education of Teachers in Science.
  28. Neuendorf, K.A. (2002). The Content Analysis Guidebook. Sage Publications.
  29. Page, E. (2003). Project essay grade: PEG. In M.D. Shermis& J. Burstein (Eds.), Automated Essay Scoring: ACross-disciplinary Perspective (pp. 43–54). Mahwah, NJ: Lawrence Erlbaum Associates.
  30. Panadero, E., Alonso-Tapia, J., & Huertas, J.A. (2012). Rubrics and self-assessment scripts effects on selfregulation, learning and self-efficacy in secondary education. Learning and Individual Differences, 22(6), 806–813.
  31. Panadero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129–144. Doi:10.1016/J.edurev.2013.01.002
  32. Piaget, J. (1970). Structuralism. New York: Harper& Row.
  33. Ramineni, C. (2013). Validating automated essay scoring for online writing placement. Assessing Writing, 18(1), 40–61. Doi:10.1016/J.asw.2012.10.005
  34. Ramineni, C., & Williamson, D.M. (2013). Automated essay scoring: Psychometric guidelines and practices. Assessing Writing, 18(1), 25–39. Doi:10.1016/J.asw.2012.10.004
  35. Rezaei, A.R., & Lovorn, M. (2010). Reliability and validity of rubrics for assessment through writing. Assessing Writing, 15(1), 18–39.
  36. Rich, C., Harrington, H., Kim, J., & West, B. (2008). Automated essay scoring in state formative and summative writing assessment. Paper presented at Annual Conference of American Educational Research Association, New York..
  37. Sauro, S. (2009). Computer-mediated corrective feedback and the development of L2 grammar. Language Learning& Technology, 13(1), 96–120.
  38. Tang, J.L. (2014). How to integrate an automated writing assessment tool in the EFL classroom? Foreign Language Learning Theory and Practice, 1, 117–125.
  39. Tang, J.L., Rich, C.S., & Wang, Y.H. (2012). Technology-enhanced English writing assessment in the classroom. Canadian Journal of Applied Linguistics, 35(4), 385–399.
  40. Tang, J.L., & Wu, Y.A. (2012). Using automated writing assessment in the college EFL classroom. Foreign Languages and Their Teaching, 265(4), 53–59.
  41. Vygotsky, L. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press.
  42. Wang, W. (2014). Students perceptions of rubric-referenced peer feedback on EFL writing: A longitudinal inquiry. Assessing Writing, 19, 80–96.
  43. Wang, Y.J., Shang, H.F., & Briody, P. (2013). Exploring the impact of using automated writing evaluation in English as aforeign language university students writing. Computer Assisted Language Learning, 26(3), 234–257.
  44. Warschauer, M., & Grimes, D. (2008). Automated writing assessment in the classroom. Pedagogies, 3(1), 22–36.
  45. Weigle, S.C. (2007). Teaching writing teachers about assessment. Journal of Second Language Writing, 16(3), 194–209.
  46. White, L., Hixson, N., D’Brot, J., Perdue, J., Foster, S., & Rhudy, V. (2010). Research brief, impact of Writing Roadmap 2.0 on WESTEST 2 online writing assessment scores. Retrieved September 20, 2010 from http://wvde.state.wv.us/oaa/pdf/research/Research%20Brief%20-%20WRM2.0%20Impact%20FINAL%2001.27.10.pdf
  47. Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 2–14.
  48. Wotjas, O. (1998). Feedback? No, just give us the answers. Times Higher Education Supplement.
  49. Wu, Y.A. (2011). Teacher change in the application of modern education technology in teaching English writing [Keynote speech]. Presented at 17th NAFEL Annual Conference, Beijing, November 4-5.
  50. Wu, Y.A., & Tang, J.L. (2012). Impact of integrating an automated assessment tool into English writing on university teachers. Computer-Assisted Foreign Language Education, 146(4), 3–10.

These references have been extracted automatically and may have some errors. Signed in users can suggest corrections to these mistakes.

Suggest Corrections to References