Question:
Anthea's musing about holistic grading being perhapsprescientific causes me to wonder if there are any more scientific examswhich are given by colleges or universities which qualify or disqualify astudent who wants to matriculate in a degree program. I'd like to hear fromanyone who knows about them what the exam is like, what the grading of itis like and whether the results conform at all to the holistic gradingassessment done by the esl writing teachers.

Answer: I have been interested in the comments on this list regarding holistic grading,but I think that there may be a misconception about the meaning of "holistic."

To begin with, when most people refer to "holistic scoring," they have an imageof a reader with a tall stack of papers, reading each one rapidly, jotting anumber from 1 to 9 on it and then moving on. In fact, this is not the onlykind of holistic scoring.

A excellent survey article by Charles Cooper in _Evaluating Writing_ (SUNY atBuffalo, 1977) describes a number of holistic approaches, many of which areprobably in use by subscribers to this list. For example, an "analytic scale,"in which a reader gives a rating to each of a number of important features ofwriting (e.g. organization, content, sentence structure, verb forms, mechanics,etc.) is an example of holistic scoring, in that each specific feature is ratedholistically rather than being given a score based on a sum or errors.Nonetheless, many people don't recognize this as holistic because they assumethat nothing which is analytic in character is really holistic.

The term "holistic" here should properly be contrasted with "mechanical." Ihave met many instructors who determine a paper's grade by simply adding up theerrors and assigning a point value for errors of different types. When I pointout that an essay of 600 words will certainly accumulate more errors on averagethan one of 300 words, the teacher assures me that that isn't a problem, butsomehow I am not reassured as to the validitiy or reliability of this process.

To return to the traditional view of holistic scoring, I have to say that thismethod, for all its apparent arbitrariness, has established itself as one ofthe most *reliable* methods of scoring essays, with two significantlimitations:

1) This kind of scoring *must* be preceded by a training session in which thereaders read essays which have already been scored, in order to "calibrate" thereaders' judgment;

2) This kind of scoring, while very valuable for judging the general level of astudent's writing (and hence for placement in a multi-level program) has littleif any value for pedagogy. I would hesitate to employ a teacher who returnedstudents' compositions with a simple number from 1 to 9 written in it! I guessthat the art of teaching writing consists partly in learning what mysteriousquality (qualities) of writing lead to consistent scores on general impressionholistic scoring, and teaching students to replicate it (them)!