I'm probably going to sound horribly pompous in the following
observations, but I think this thread is bringing up a common, general
problem in assessment, and it will inevitably apply to any attempt to
set up standardized benchmarks and assessment strategies for grammar
education: some things lend themselves far more easily to operational
definitions than others (at least, in the old logical-positivist sense
of operational definitions), and one has to constantly work to prevent
the naturally-occurring "lines of least resistance" from distorting the
assessment system and the goals to which it is connected.

It is, for example, far, far easier to define operationally what
constitutes an acceptable level of control over subject-verb agreement
than to construct an equivalent definition of what constitutes an
acceptable level of control over word-choice. "Being able to turn an
active sentence into a passive sentence" is a skill that easy to define
operationally; "being able to use passive sentences appropriately" is
less so. In practice, operational definitions of the difficult stuff
only *look* operational, or are a bit circular; e.g. "A piece of writing
shows appropriate use of passives if a representative set of composition
teachers think it shows appropriate use of passives." After all, the
logical-positivists themselves, who pushed the idea of operational
definitions about as far as it would go, had to acknowledge the problems
inherent in them in the end. 

This does not, of course, entail that we should not try to define things
operationally. The basic, common-sense assessment idea that "it's a good
thing to know whether you're doing what you think you're doing" is hard
to argue with, and the vaguer the definition, the harder it is to use it
for anything worthwhile. I've seen numerous examples of definitions that
were seemingly intended almost solely to create a nice
responsibility-free spin zone (and the evils of vague definitions pale
in comparison to the horror that is the average goals statement). But
I've also seen what happens when assessment is driven by what is *easy*
to operationalize, and the results are a disservice to our students and
to the democracy they'll shape. 

The solution may be to adopt the same kind of strategy toward each case
of  operationalization that we do toward most other public-policy
issues: do a cost-benefit analysis. Whenever it's possible to create a
nice, tight, operational definition, do so. If consensus can't be
reached on an operational definition for proficiency in a skill that
everyone views as vital, however, tolerating some fuzziness may be
warranted -- I'm not going to ignore tone as a writing issue, for
example, simply because it's not amenable to exact definition or
measurement. If considerations of assessment language pressure
instructors to drop entire topics from the curriculum, we're in trouble.


Bill Spruiell

Dept. of English
Central Michigan University. 

-----Original Message-----
From: Assembly for the Teaching of English Grammar
[mailto:[log in to unmask]] On Behalf Of DD Farms
Sent: Wednesday, September 19, 2007 9:04 PM
To: [log in to unmask]
Subject: Re: Predictors

At 10:16 PM 9/18/2007, Paul E. Doniger wrote: . . .
>DD responded:"So true. But the schools seem to have no clearly 
>assigned missions.

>I'm not sure what DD means, but every school I've worked in has had 
>a mission statement, and mostly good ones.

DD: I meant an operationally definition. Not to improve the student 
in something not defined, but to improve the students' score on the 
ACT, or their results in some other test. Not feel good, achieve 
excellence, if that is not operationally defined. I don't mind a 
definition that strives for more students passing the National Latin 
Test, nor achieve a lower drop out rate.

>My present school's mission (and I hope to stay here for the rest of 
>my career) is as follows: "The mission of Pomperaug Regional High 
>School, a caring community committed to excellence, is to educate 
>each student to become a productive, responsible, enlightened 
>citizen and a creative life-long learner through high quality, 
>dynamic, innovative learning experiences in collaboration with the 
>Region 15 community."

DD: And how is one supposed to measure that?

>It seems a pretty "clearly assigned mission" to me, and one that 
>drives our curriculum; it's well worth supporting. We are constantly 
>asked to hold our students accountable to a high standard.

DD: What standard?

>We also try to keep the community involved (it's easy for me, 
>running a theatre program that depends on parents and local 
>businesses being involved).
>I often wonder where people get their ideas about public schools and 
>what goes on inside them Certainly, having taught both in the inner 
>cities and in a blue collar town, I have seen some of the failings 
>that have been mentioned in this thread, but mostly, I've seen good 
>teaching and very little "dumbing down." . . .

DD: I have also taught in the public school system. I did my utmost 
to achieve what I thought was desirable, but no clear mission statement
had I.

>  What really causes problems for education are often those things 
> that are out of our control: Budget cuts, large classes (usually a 
> result of budget restraints), unfunded government mandates (like 
> Craig, I am "not a fan" of NCLB -- actually, I hate the thing), 
> standardized testing interrupting the process, consumerism, 
> materialism, technological distractions (TV was just the 
> beginning), and the drive to focus on the goal of creating a work 
> force rather than creating "good citizens" in a Jeffersonian sense.

DD: I note you specify "education." Undefined. No clear mission
statement.

>Add to this that since the 1940s, 12 years of public education has 
>become the minimal requirement for every American; there is a 
>greater percentage of people who are expected to graduate from high 
>school than ever in history -- and the stigma of not graduating is 
>immense, both socially and economically.

DD: So the standards for so doing have been reduced to see it that they
do.

To join or leave this LISTSERV list, please visit the list's web
interface at:
     http://listserv.muohio.edu/archives/ateg.html
and select "Join or leave the list"

Visit ATEG's web site at http://ateg.org/

To join or leave this LISTSERV list, please visit the list's web interface at:
     http://listserv.muohio.edu/archives/ateg.html
and select "Join or leave the list"

Visit ATEG's web site at http://ateg.org/