You are here:

On the Relation between Automated Essay Scoring and Modern Views of the Writing Construct
ARTICLE

Assessing Writing Volume 18, Number 1, ISSN 1075-2935

Abstract

This paper examines the construct measured by automated essay scoring (AES) systems. AES systems measure features of the text structure, linguistic structure, and conventional print form of essays; as such, the systems primarily measure text production skills. In the current state-of-the-art, AES provide little direct evidence about such matters as strength of argumentation or rhetorical effectiveness. However, since there is a relationship between ease of text production and ability to mobilize cognitive resources to address rhetorical and conceptual problems, AES systems have strong correlations with overall performance and can effectively distinguish students in a position to apply a broader writing construct from those for whom text production constitutes a significant barrier to achievement. The paper begins by defining writing as a construct and then turns to the e-rater scoring engine as an example of AES state-of-the-art construct measurement. Common criticisms of AES are defined and explicated--fundamental objections to the construct measured, methods used to measure the construct, and technical inadequacies--and a direction for future research is identified through a socio-cognitive approach to AES. (Contains 4 figures.)

Citation

Deane, P. (2013). On the Relation between Automated Essay Scoring and Modern Views of the Writing Construct. Assessing Writing, 18(1), 7-24. Retrieved May 25, 2022 from .

This record was imported from ERIC on April 18, 2013. [Original Record]

ERIC is sponsored by the Institute of Education Sciences (IES) of the U.S. Department of Education.

Copyright for this record is held by the content creator. For more details see ERIC's copyright policy.

Keywords