I like to remind the students to find the best answer given, not the "perfect" answer.
TZ
________________________________________
From: Academy of Legal Studies in Business (ALSB) Talk [[log in to unmask]] On Behalf Of M. Katz [[log in to unmask]]
Sent: Monday, October 18, 2010 3:16 PM
To: [log in to unmask]
Subject: Re: can see "true" but can not see "false"

One of the interesting things after giving a scantron test is to look at the test question sheet returned by the student.  Frequently, I find that students "correct" false questions and then answer their edited question.  In essence, some change a false question to a true question and answer accordingly.  This may account for the discrepancies you have found.



Note:  Upon returning the first test and scantron, I point out that they need to answer my questions rather than theirs.  A look of "I can't believe I did that" occurs on multiple faces and this cuts down on repeat offenders.



I am not a witch!*

Mike



* The new approved business closing in Delaware.



----- Original Message -----
From: "Michael O'Hara" <[log in to unmask]>
To: [log in to unmask]
Sent: Monday, October 18, 2010 3:24:51 PM
Subject: can see "true" but can not see "false"

ALSBTALK:

Sometimes technology lets you see things that otherwise you might not notice.

UNO has switched from producing paper-only to digital-only exam results from computer graded bubble sheet results.  Since I never would re-type paper-only question-by-question results, but I am willing to copy-paste a table from within a PDF into Excel, I now can see something that was hiding in plain sight.  Once in Excel I can sort the test results by a variety of metrics.  One sort just jumps right off the page.

If I sort the questions by percent of correct answers, then there is a clear pattern in the answer key column:  all of the questions with high correct percentages are "true" and all questions with low correct percentages are "false".  I mentioned this as an oddity at the end of last semester when I thought it was a characteristic of just one b-law-1 exam.  However, now I have spotted the same answer key pattern on a pair of b-law-1 exams this semester as well as on a b-law-2 exam, and a Microeconomics exam.  For example, on a multiple choice Microeconomics exam with answer options a, b, c, all, and none; and with as nearly as possible 50% of the a's, the b's, and the c's being false, the 25% of the exam questions with the correct answer of "all of the above" were the top scoring questions as well as the 25% of the exam questions with the correct answer of "none of the above" were the bottom scoring questions.  In contrast, on that same exam, the 50% of the exam question with a correct answer of a, b, or c had roughly randomly distributed frequency of correct answers between the polar extremes of all and none.

On this smattering of data I am going to dare to propose a hypothesis.

H1:  Students today have very low order pattern recognition skills (i.e., to their eyes similar is equivalent to identical).

After 12 years of multiple choice questions of low difficulty (e.g., The first President of the USA was:  a. George Bush;  b. George Foreman; c. George Washington; d. all of the above.) they have been trained to believe guessing is as good as knowing.  After striving mightily they can spot true, but can not see false.

It might generate some very interesting results to simultaneously administer a 100% true exam in the same room as a 100% false exam.  Wanna take bets on which exam gets the higher mean score?  As with all bets, the action is in the spread.  I want "true"; how many points do you need?  Oh, by the way, on the last b-law-1 exam the mean of the top 15 questions high correct percentage was 96% (ranging between 93 and 100) and the mean of the bottom 15 questions of low correct percentage was 50% (ranging between 33 and 59).  How about you give me true and I give you 33 points?

Michael

Professor Michael J. O'Hara, J.D., Ph.D.
Finance, Banking, & Law Department
College of Business Administration
Mammel Hall 228
University of Nebraska at Omaha
6708 Pine Street
Omaha NE 68182-0048
[log in to unmask]
(402) 554 - 2823 voice fax (402) 554 - 2680
http://cba.unomaha.edu/faculty/mohara/web/ohara.htm