How we graded the schools

iWatch News and Newsweek analyzed high-school graduation rates posted by the nationally recognized Editorial Projects in Education Research Center and math and English standardized test scores for kindergarten through 12th grade that are maintained by MPR Associates Inc., a well-respected education statistics agency.  Among other things, MPR helps philanthropist Eli Broad’s foundation select urban school districts for its annual “most-improved” charitable awards.

Both agencies are respected throughout the education industry, in part because they use consistent methodology are normalize data to allow comparisons between individual school districts and their states. Using test scores to measure school effectiveness remains controversial inside and outside education circles. But they are used by foundations and government alike, and the philanthropists made such measurement a key part of their platform. MPR’s data only compare years when testing methods were constant. Because state tests vary in rigor, proficiency scores are not comparable across states.

The analysis focused on 10 urban districts that have gotten much of the philanthropists’ attention and $1 billion of their money:  New York, Chicago, Los Angeles, Washington D.C., Austin, Tex., Houston, Dallas, Milwaukee, Denver and Oakland. The goal was to compare their performances against all other districts in their states in years since reform efforts began. For Washington D.C., where the District is not part of a state, test results were compared against scores from independent charter schools in the District. However, Editorial Projects in Education Research Center does not have comparable graduation rates to compare the charter schools to the District’s graduation rates.

Test scores covered 2006-2010. Over that period, iWatch News calculated how many times one of the targeted school districts raised its test scores over the previous year’s scores more points than the average of school districts in the rest of the state. In years when test scores fell, iWatch News also tracked whether the targeted schools fell fewer points than their peers statewide. Similar calculations were used for measuring changes in graduation rates but were based on 2002-2007 data, the most recent years available.

The Center then calculated a simple percentage to see overall how many times the urban districts improved performance faster than the other schools in their respective states. Those percentages were also calculated for comparisons among elementary, middle and high schools in math and reading scores separately.