Mark, 
Would you like me to send you the test? 
 
Mark, DD, et al., 
 I don't think my students are as smart as their scores indicate, though they are much smarter than a truly random sample of 7th graders.  There is really quite a range, though very few below the 50th percentile (two, in my group last year of 112, as I recall). I expect that students whose reading percentile scores are lower will present a greater difference between the control and experimental conditions.  This seemed to be the case for the previous test, based on raw scores.  If such a difference is more pronounced in a timed situation, that would seem to validate the experimental condition as improving how well students understand text.   (Anecdotally, the overwhelming majority of my students report that it is extremely helpful). 
 
An additional experiment would test the influence of extensive practice in reading graphic syntax on reading skill.  This experiment would compare the results of a second test with changes in reading skill as measured on a standardized test.  My prediction is that after a year of reading difficult material in graphic syntax, as well as difficult material in normal text, reading scores would go up (this would not be a surprising or revealing result) and the difference between experimental and control performance for lower performers whose performance improved would become similar to that of higher performers in the first test (this would be important). I expect that if good readers are good readers partially because they understand syntactic connections better, and consequently are less aided by graphic syntax, then improving student understanding of syntactic connections by extensive practice with graphic syntax should improve their reading
 comprehension.  By showing a strong correlation between standardized reading comprehension scores and the difference between scores on graphic and normal text, I should be able to isolate the factor of graphic syntax from all the other excellent things I do in the classroom and show that reading in graphic syntax format can improve reading comprehension. 
 
Does this make sense? Is it sound?  Is there another way to show that reading skill can be improved by this method?  
 
Thanks, 
Scott Woods
 
--- On Thu, 11/27/08, MC Johnstone <[log in to unmask]> wrote:

From: MC Johnstone <[log in to unmask]>
Subject: Re: Experimental design help
To: [log in to unmask]
Date: Thursday, November 27, 2008, 10:50 AM

Scott,

High IQ and all that... I have done a similar exercise with functionally
emerging-literate Arab students reading English and am willing to try it again,
in the interests of science.

I don't have IQ scores for them but I can provide recent TOEFL data.
Perhaps that can be calibrated to something or other...

Mark


> At 03:12 PM 11/26/2008, Scott Woods wrote:
>> I want to test a technique involving grammar for its effectiveness in
improving reading comprehension.  Please let me know what you think of my design
and if you have any suggestions for related research. . . .
> 
> DD: You called? From out of the wood work comes there now a
Psychometrician (Retired.) It sounds like a sound design, as presented. However
{Ever notice how there is always a caveat?} you are dealing with a rather high
end sample.* High IQ and all that. Do the study. Slap it into a Chi Square
contingency test and see. Make sure the groups are selected truly randomly.
Report the results. I personally think you are doing true science here. A lot
rarer than you would think. The null hypothesis is there will not be a
significant difference. Fifty fifty. I'd go for the more risky a priori
prediction that there will be one in favor of the graphic syntax. That way I get
to use a one tail test and that allows significance at a lower level. It is
risky, though, because if it turns around and bites YOU on the tail and the
normal text group comes out ahead, the experimenter is required to perform a
ritual self immolation. We rarely do, though. Just lie and report we did no a
priori post hoc corrections. Keep me posted. Fascinating to see a well designed
experiment before it is done. Usually the psychometrician just gets a bunch of
results and is asked to make sense out of it. {Usually by an attractive graduate
student. Blonde preferred.} I suspect that timing will indeed change the
results, but that a correlation between speed v extended time will show a high
r. Suggestion for future reading - anything by Ohmer Milton. I remain, your
faithful friend and joyous companion in original research.
> 
> * Ware the regression toward the mean.
> 
> To join or leave this LISTSERV list, please visit the list's web
interface at:
>     http://listserv.muohio.edu/archives/ateg.html
> and select "Join or leave the list"
> 
> Visit ATEG's web site at http://ateg.org/

To join or leave this LISTSERV list, please visit the list's web interface
at:
    http://listserv.muohio.edu/archives/ateg.html
and select "Join or leave the list"

Visit ATEG's web site at http://ateg.org/



      

To join or leave this LISTSERV list, please visit the list's web interface at:
     http://listserv.muohio.edu/archives/ateg.html
and select "Join or leave the list"

Visit ATEG's web site at http://ateg.org/