You are here

Writing Evaluation

Title: Writing Evaluation: Rater and Task Effects on the Reliability of Writing Scores for Children in Grades 3 and 4.
Name(s): Kim, Grace Young-Suk, author
Schatschneider, Christopher, author
Wanzek, Jeanne, author
Gatlin, Brandy, author
Al Otaiba, Stephanie, author
Type of Resource: text
Genre: Journal Article
Date Issued: 2017-06-01
Physical Form: computer
online resource
Extent: 1 online resource
Language(s): English
Abstract/Description: We examined how raters and tasks influence measurement error in writing evaluation and how many raters and tasks are needed to reach a desirable level of .90 and .80 reliabilities for children in Grades 3 and 4. A total of 211 children (102 boys) were administered three tasks in narrative and expository genres, respectively, and their written compositions were evaluated in widely used evaluation methods for developing writers: holistic scoring, productivity, and curriculum-based writing scores. Results showed that 54% and 52% of variance in narrative and expository compositions were attributable to true individual differences in writing. Students' scores varied largely by tasks (30.44% and 28.61% of variance), but not by raters. To reach the reliability of .90, multiple tasks and raters were needed, and for the reliability of .80, a single rater and multiple tasks were needed. These findings offer important implications about reliably evaluating children's writing skills, given that writing is typically evaluated by a single task and a single rater in classrooms and even in state accountability systems.
Identifier: FSU_pmch_29075050 (IID), 10.1007/s11145-017-9724-6 (DOI), PMC5653319 (PMCID), 29075050 (RID), 29075050 (EID)
Keywords: Generalizability theory, Assessment, Rater effect, Task effect, Writing
Grant Number: P50 HD052120
Publication Note: This NIH-funded author manuscript originally appeared in PubMed Central at
Persistent Link to This Record:
Owner Institution: FSU
Is Part Of: Reading and writing.
Issue: iss. 6, vol. 30

Choose the citation style.
Kim, G. Y. -S., Schatschneider, C., Wanzek, J., Gatlin, B., & Al Otaiba, S. (2017). Writing Evaluation: Rater and Task Effects on the Reliability of Writing Scores for Children in Grades 3 and 4. Reading And Writing. Retrieved from