Sign Up For Our Newsletter

Statistics, Studies and Lies (Robert Teegarden)

It seems striking that a so-called national study comes out debunking the superiority of private schools to union schools within days that the Administration introduces a voucher-type remedy to that same inferiority.  Hmmmm.  Coincidence?  Probably.  Intended?  Maybe.  But flawed?  Absolutely.  Read the cautions by the authors.  Read their warnings.  Were any of those caveats posted or followed in the subsequent reports?  No.  Someone’s hidden agenda?  You betcha.

Besides adjusting results after-the-fact using Hierarchical Linear Modeling–the same kind of clever readjustment that took place several years hence in the SAT re-norming when it was determined that a score of 742 was just as good as last year’s  perfect score of 800 (I wish my bank thought that way!)–besides demonstrating again that private school are not the presupposed private enclaves for the privileged and rich, besides iterating the care that must be taken with statistical analysis…the report does note that not all schools are the same.  Wow!  How do you re-norm scores to balance a 99% graduation rate versus a 62% graduation rate?  What do you say to the one out of three kids who doesn’t even make it to 12th grade for graduation?

The examiners seem to look upon test scores the way many state high school athletic directors look upon sports activities in private schools. When only ten percent of the population (the private schools) win over 60% of the sports trophies in a district or a state, there’s obviously something wrong.  Some districts and states have attempted to exclude private schools from competition all together.  The reason: They’re good!  They win games.  But what they fail to really say is, “It’s not fair.”  The size of the trophy cases in most school lobbies will attest to the value placed on their argument and concern.

But like the educrats in this recent testing study, the rest of the athletic directors haven’t excluded the private school students from competition (yet!).  They’ve only handicapped them.

In athletics, most schools are ranked according to their size. Schools with 1,000 students have one classification while schools with 300 or less would be in an entirely different league.  The reasoning being that the school with 1,000 has a larger pool from which to draw talented athletes than does the school with only 300.  But because the private schools have demonstrated a quality of play that can’t be beat, their numbers are handicapped.  If you’re a private school with 282 students and you win games, each of your students would receive an index ranking of, say, 1.72.  So, instead of having 282 real students, this school now has 485 ranked students. They now have to play in a different (larger) league.  In other words, 1 private school student equals 1.72 government school students.

These national testers did just the same. They added adjusted values to the scores.  If your parents make less than $40,000 a year, your scores might go up by a factor of 1.35.  If you happen to have a skin color of this or that type, your scores will increase by a factor of 1.72.  It’s interesting to note that the conclusion suggests, more or less, no substantial differences.  But they forgot a couple of things.

Like the athletic teams, it’s the raw score that wins the race, regardless of your skin color and relative wealth.  Anything less is the soft underbelly of bigotry.  Just because you’re poor doesn’t mean you’re stupid.  And just because your ancestors may have been raised in an area 10 degrees north of mine doesn’t mean that we also have a legacy of ignorance.  It’s not the spin or adjusted parameters that help students graduate; that only assuages the guilt of the adult bystanders. That only attempts to assuage failure.

The flaws of omission seem glaring. Is there not a selection bias associated with the fact that private school students choose their schools and government school students cannot?  What accounts for the number of years in a union school or a school or choice? Shouldn’t that make a difference?  And then there are the parents.

How do you statistically account for a family background factor versus a school effect?  Is it the background factor of the parents that got the child to this point in her educational career, or is it exactly because the school has an expectation of parents that causes them to act a certain way vis-à-vis their child and, thus, appear statistically “less poor”?  

But we shouldn’t be so worried about where a student is educated–we should be concerned with whether that student is educated.

Robert Teegarden is director of state projects for the Alliance for School Choice.