I will second what Karl says. About ten years ago I was, for several years, a grader for the ECT (English Composition Test) and was impressed by the training and monitoring of readers and the efforts to get consistent scoring. I'd say the the essay tests are no less (or more) reliable a measure of general writing ability than the other SATs are at measuring verbal and quantitative ablities.
Dick Veit
-----Original Message-----
From: Assembly for the Teaching of English Grammar on behalf of Karl Hagen
Sent: Sun 5/8/2005 1:08 PM
To: [log in to unmask]
Subject: Re: Writing and assessment
I have a fair bit of experience with this sort of essay grading. It's
widely used in state and national assessments, as well as in colleges
and universities for their internal placement tests into composition
courses.
Personally, I prefer authentic assessment and dislike standardized
testing on principle. But within the framework of a standardized test
(something we are stuck with for many reasons), you are more or less
forced into something very similar to this sort of essay task, and the
grading practices are defensible if the results are used appropriately.
That is a big if, however, as these results are often used
inappropriately. This essay measures students' ability to write a brief,
impromptu essay on an incredibly general topic. In terms of
authenticity, the task is lacking in obvious ways. This writing task is
far removed from the revised composition, and not even much like writing
an exam essay (where at least one has a narrowly defined subject). So if
we use scores from these essays as a competency test (say, to exempt
students from a freshman comp class), I would argue that they are misused.
However if the purpose is to get a relative ranking of students'
abilities, these tests do work. There is research to back up the
assertion that students' relative performance remains consistent if you
change the nature of the essay (for example, by altering the time
constraints), and that there's a positive correlation between these
essays and freshman comp grades.
This, by the way, is a big problem with the uses to which standardized
tests are put generally. There is a large difference in the way you
construct an assessment test like the SAT or GRE (designed to
distinguish student performance across the full range of ability levels)
and a competency test like the NCLEX or a state bar exam (designed to
establish a minimum performance standard). To try to use the results of
a test for a purpose it was not designed for guarantees unfairness.
Since I have trained graders myself for several years, I can definitely
state that graders are not trained to look for any formulaic wording for
transitions. Or at least that's not in any of the training material I've
ever looked at. There tends to be instructions to the effect that no
specific formulaic approach is to be favored or penalized. In other
words, readers are not supposed to favor structures like the
five-paragraph essay over any logical organization.
Karl Hagen
Department of English
Mount St. Mary's College
PAUL E. DONIGER wrote:
> Wow, Jan, aren't you glad NOT to have gone to that training? It's a
> great lesson for students, isn't it? All they need to do is use lots
> of transitions and not worry about accuracy or meaning. What a sad
> state of things we have gotten into. No wonder so many arrive at
> college unready for composition classes.
> Feeling depressed,
>
> Paul
>
> */Jan Kammert <[log in to unmask]>/* wrote:
>
> >
> > Check out this news. It would be funny if it weren't so true and
> so sad!
> Meaningful grades on essays that are read in 2-3 minutes? Bulk is
> valued more
> than content? Made up facts are good? WOW!!!!!
> >
> The assessment test that all students have to take in my state
> includes an
> essay. The people who read the essay are expected to spend about that
> much time on each essay. Although I have never gone to the
> training (only
> one person per school district is allowed to go per year), I have been
> told that the graders look mostly for transition words: "next" "then"
> "finally."
> Jan
>
> To join or leave this LISTSERV list, please visit the list's web
> interface at:
> http://listserv.muohio.edu/archives/ateg.html
> and select "Join or leave the list"
>
> Visit ATEG's web site at http://ateg.org/
>
>
>
> "If this were play'd! upon a stage now, I could condemn it as an
> improbable fiction" (_Twelfth Night_ 3.4.127-128). To join or leave
> this LISTSERV list, please visit the list's web interface at:
> http://listserv.muohio.edu/archives/ateg.html and select "Join or
> leave the list"
>
> Visit ATEG's web site at http://ateg.org/
>
To join or leave this LISTSERV list, please visit the list's web interface at:
http://listserv.muohio.edu/archives/ateg.html
and select "Join or leave the list"
Visit ATEG's web site at http://ateg.org/
To join or leave this LISTSERV list, please visit the list's web interface at:
http://listserv.muohio.edu/archives/ateg.html
and select "Join or leave the list"
Visit ATEG's web site at http://ateg.org/
|
