A Stanford University report from the Center for Research on Education Outcomes (CREDO) gained major attention in June 2009 when the New York Times ran its findings that public charter schools do no better or worse than traditional public schools. Unfortunately, these findings continue to be taken as fact today despite the study’s shortcomings. Below are some brief talking points on why the CREDO report is flawed.
You can find more detailed information in “Fact-Checking Charter School Achievement.”
Uncorrelated Variables
CREDO’s analysis does not account for the great variances in charter laws from state to state or how those laws may differ from paper to practice.
- While the report suggests correlations exist between student achievement and charter law components, they admit to not fully understanding the impact of specific laws.
- The report suggests a negative correlation between student achievement and multiple authorizers. In fact, such charter authorizers vary greatly in law and practice, as CER’s 2012 study and scorecard demonstrate. There is clear evidence that charter students succeed in states witha number of meaningful, independent and highly accountable authorizers who compete for chartering. See our Multiple Authorizers Primer for more information on charter authorizers.
- The study was based on student population and not the overal strength of their charter system. Therefore CREDO missed most opportunities to see really strong charters in action.
Virtual Methodology
While there are virtual schools, there is no such thing as “virtual” student achievement.
- The CREDO report acknowledges the creation of new research tools to assess the unknowable. Instead of comparing real students who attend charter schools to real students who attend conventional public schools, CREDO merged demographic data to create “virtual twins.” Randomization, the gold standard of research, is not used.
- By virtually replicating the demographic profile of a charter school student — ethnicity, age, socio-economic status — and averaging that same profile in a conventional public school students, researchers ignore something every parent and teacher knows is an impossible comparison since no two students are alike. The study essentially treats group averages as if they were students themselves. (Respected researcher Caroline Hoxby explains this point in greater detail in Fact-Checking Charter School Achievement)
Contradictory Demographics
The reports’ findings are inconsistent across demographic populations and geographic jurisdictions.
- While CREDO found that charter performance is lower for African-Americans and Hispanics, they find effects of charters greater for students in poverty. Well-documented and recognized survey work which represents more than a third of all charters, finds such demographics to be one and the same.
- The CREDO results are an anomaly in finding some states falling short on student achievement. Yet these same states – Florida, Minnesota, California, DC, and Georgia – have demonstrated enormous success on state and national assessments and have undergone intense scrutiny only to report superior student performance year after year.
The demonstrated success of, demand for, and attendance in charter schools escape broad stroke conclusions. Without student-by-student, school-by-school, and critical, specific data, we remain publicly ignorant of what makes schools work for children.