U N I V E R S I T Y  O F  P I T T S B U R G H

TTimes Banner Teaching Times Teaching Times

 A newsletter devoted to the support of teaching and learning at the University of Pittsburgh 

 
 
 
 
 
 
 
 
 
 
 
 
 

Software helps students in large classes to write and revise papers

SWoRD enables students to get feedback on writing, regardless of class size.

Writing, especially in a process that involves feedback and rewriting, is one of the best ways of learning, as evidenced by abundant writing-to-learn literature. Writing also is perceived by many faculty as a more accurate way than testing to assess what students have learned. However, because of the time it takes to read and comment on student papers, writing is generally not assigned in large undergraduate classes.

Photo by Jim Burke, CIDDE
Christian Schunn, Psychology, uses Scaffolded Writing and Rewriting in the Discipline (SWoRD), an anonymous, Web-based peer review software, to enable students in large undergraduate classes to perform the writing and revision that are critical for undergraduate education.

In addition to missed writing-to-learn opportunities, students miss opportunities to learn to write. “Faculty simply don’t have the time to grade that many papers, especially not with the two drafts and written comments that research has shown are necessary for writing to improve. As an unfortunate result, undergraduates have minimal writing experiences and, therefore, they don’t really progress in their ability with this limited practice,” comments Christian Schunn, Psychology.

Schunn, who holds secondary appointments in the Intelligent Systems Program and the School of Education, addressed this problem by creating Scaffolded Writing and Rewriting in the Dicipline (SWoRD), a Web-based software that enables students in large undergraduate classes to perform the writing and revision that are so critical for undergraduate education. SWoRD is an anonymous, customized peer review system. Default grading rubrics with criteria in the areas of flow, argument definition, and sharing of new insights are available and can be tweaked to suit individual instructors. Each student paper is distributed to five to six peers; each student, in turn, downloads the same number of papers and rates them on parameters specified by the instructor. Reviewers provide numerical ratings and written comments to help the author rewrite the paper.

After revising, authors turn in a final draft, and the same reviewers rate the revisions. These peer ratings determine the grade for the paper. In turn, each paper’s reviewers are graded by the author on the helpfulness of their comments, and the system automatically grades reviewerson the accuracy of their ratings. If a particular reviewer is “way off” track, his/her evaluation is automatically discounted by the system. All of these computations take place automatically “in the background,” and the results are provided to the instructor. Students even receive automatic e-mail reminders about paper deadlines.

In student interviews, Schunn has found that undergraduates appreciate the opportunity to write papers, especially if they are allowed to explore topics they are interested in. (Schunn encourages his own students to pick a topic from a research finding presented in class, explain it in depth, and apply it to the real world.) Students strongly perceive11/17/2006rn, and they prefer to write papers rather than take exams.

Faculty who have implemented SWoRD have required little guidance to download and operate it. Help manuals are provided for instructors and all students in a class. In addition to explanations of the default grading rubrics, model papers can be added for any course to help explain criteria to students. Each phase of the assignments takes an estimated 5-plus hours, and faculty are advised to take this into account and leave two weeks for each phase.

Originally funded by the Provost’s Advisory Council on Instructional Excellence (ACIE) and currently funded by a three-year grant from the A.W. Mellon Foundation, SWoRD has been co-developed by Kwangsu Cho, a postdoctoral research associate. The program has undergone several revisions based on formative evaluations. Research on the software implementations has shown, for example, that

  • The average of five students’ ratings is as accurate as any one faculty member’s rating.
  • Student papers improve more when the author gets comments from five peers rather than from one faculty member.
  • The optimal paper length is five to eight pages focused on a main theme or argument. Longer, more complex research papers require that criteria be established for each section, and it is too time consuming for students to provide the requisite feedback on each section.
  • Undergraduates have confidence in the system and in the peer reviews.
  • SWoRD is reliable in smaller classes, as well, and the instructor can be a reviewer of all papers.
  • SWoRD removes various student “status effects,” and students who tend to be passive in class become more involved.Because SWoRD is a Web application, all users (teachers and students) simply access it from any computer using their favorite Web browser. Faculty who are interested in more information about SWoRD may contact Schunn at schunn@pitt.edu.

SWoRD enables students to get feedback on writing, regardless of class size. education. SWoRD is an anonymous, customized peer review system. Default grading rubrics with criteria in the areas of flow, argument definition, and sharing of new insights are available and can be tweaked to suit individual instructors. Each student paper is distributed to five to six peers; each student, in turn, downloads the same number of papers and rates them on parameters specified by the instructor. Reviewers provide numerical ratings and written comments to help the author rewrite the paper. After revising, authors turn in a final draft, and the same reviewers rate the revisions. These peer ratings determine the grade for the paper. In turn, each paper’s reviewers are graded by the author on the helpfulness of their comments, and the system automatically grades reviewersthe

A newsletter devoted to the support of teaching and learning at the University of Pittsburgh

Center for Instructional Development & Distance Education
1996-2005 © University of Pittsburgh, CIDDE. All Rights Reserved.
Editor: Carol DeArment, Production: Joyce Walsh
Questions or comments, please contact CIDDE Webmaster, 05/02/2005