A recent study by Stanford’s Center for Research on Education Outcomes (CREDO) found that charter school students in New Jersey had learning gains in reading and math that were 0.06 and 0.08 standard deviations greater than the averages of their virtual comparison panel, or students in conventional public schools. Overall, 30 percent of charters had significantly more positive learning gains in reading, and 40 percent had significantly more positive learning gains in math.

However, they found differing effects based on the “urbanicity” of the charter school. Students in urban charter schools performed better than those in suburban schools; students in urban charters scored about 0.12 standard deviations greater than the virtual twin in math, whereas students in suburban charters scored only 0.05 standard deviations greater than their virtual twin. Students in rural charter schools performed significantly worse, about 0.14 and 0.18 standard deviations lower than the virtual twin in math and reading, respectively. Charter schools were also found to be more effective than traditional public schools at closing racial achievement gaps for black and Hispanic students, and students in poverty performed better in charter schools than in public schools. No significant difference in achievement was found for special education students attending charter schools, though this is likely due to the small number of students in question.

This report used the same virtual control record (VCR) or “virtual twin” methodology as the 2009 report Multiple Choice: Charter School Performance in 16 States (New Jersey was not one of the 16 states analyzed). The Center commented on the problems with the methodology in this paper, although there are some differences with this report.

The problem that CER noted in the past about studies overly emphasizing new charter schools does not appear to exist here. The majority of the sample comes from students in schools operating for nine years or longer, and while there are plenty of observations from young charter schools, there is not as much room for systematic bias. On the other hand, when CREDO breaks the sample down by the student’s previous time in a charter school there are significantly more first-year (newly transferred) charter students than second or third year students. It’s hard to say whether this will cause bias because first, second, and third-year students only account for about 6,000/16,000 of the students in the sample, so there are plenty of unreported results that would help determine the amount of bias.

Looking through the appendix, there is not any information about the actual number of schools in each category, but they do break the samples down by number of students in the appendix. The sample has a lot fewer students in rural charter schools (about 700 vs. 8,600 and 7,200 in urban and suburban schools, respectively), but that is due to the nature of the charter school system in New Jersey, where schools are generally concentrated in urban areas. This may cause bias when breaking down the school achievement by urban, suburban, or rural because so few students are attending rural charter schools, each individual’s scores are weighted more heavily.