by Jeanne Allen, Founder and Chief Executive
The white paper “North Carolina Opportunity Scholarships: Countering a Flawed Duke Report,” developed by Brian Jodice of Parents for Educational Freedom in North Carolina (PEFNC), is not only a victory for scholarship programs but for honesty in research which, when it comes to school choice issues, is often negatively skewed in both its construction and execution.
Jodice’s paper delineates the fundamental problems with the Duke report and other studies evaluating school scholarship initiatives like the recently published negative assessment of the DC Opportunity Scholarship Program. The Duke report and others like it are inherently flawed because they inaccurately depict students who participate in school choice programs as performing below expectations just because they have not skyrocketed to the top of the class during their first year in the program.
The new analysis sets the record straight on several fronts. First, the data analysis strategy for the Duke report is poorly designed using both different comparison groups and different tests. For example, scholarship students who tend to be predominantly from lower-income families are matched against a national sampling of students who come from a cross-section of socio-economic backgrounds, creating an apples-to-oranges comparison. The study also integrates metrics such as “national average” without an explanation as to how the term is defined or calculated.
Jodice also points out that Duke’s student test-performance data was only collected for one year which is hardly enough time to make an adequate assessment of the program’s results. Most studies show that it could take at least three years for aggregate test scores to show improvement. Furthermore, the report does not include a comparison of the students’ test scores before and after their enrollment in the scholarship program, making it impossible to know if a student performed better or worse than he or she had previously.
That the Duke study completely ignores the good news from the North Carolina Opportunity Scholarship and other national school choice programs. For example, a recent study has shown that the likelihood of DC Opportunity Scholarship participants graduating from high school has jumped from 70 percent to 91 percent, a statistic that is a harbinger of positive outcomes for similar scholarship programs.
More than half of the 740 private schools in North Carolina participate in the program and more than 90% of first-year enrollees opt to enroll their children for a second year – all of which, arguably, is the most important statistic of all. When parents buy into schools their students are more likely to succeed.
There are three kinds of lies, the saying goes: lies, damned lies, and statistics. And while that adage may overstate the case, it can, at least, stand as a constant caution. First, to average people who read stories based on statistics, heed this generic advice: don’t believe everything you read.
Second, to those who write about studies and statistics. Far too many news stories, opinion pieces, and issue analyses trumpet statistical “findings” that are overblown, misleading – sometimes wildly so – and that often draw conclusions that are, unjustified, unfair, or flat-out wrong. Be less accepting of information that is handed to you, and more discerning in your assessment of it.
And third, the caution should be greatest to those who construct the surveys, create the data sets, and crunch the numbers. Your work may be “accurate” but does it carry a bias, is it fair, is it honest? Are their flaws? What have you missed? What have you omitted? Why?
By calling out Duke, Brian Jodice has done a public service for the people of North Carolina, and for all of us who are advocates for options, opportunities and choice in education.
Jeanne Allen is an entrepreneur, an innovator and a leader. Her entire career has been devoted to education reform, and as a result, Jeanne is the most recognized and respected expert, thought leader, speaker, and writer in the field. She founded the Center for Education Reform (CER) in 1993 and leads the nationwide fight to ensure that the bedrock of U.S. schooling is innovation, freedom and flexibility. Follow Jeanne on Twitter @JeanneAllen