Will testing irregularities knock the wind out of Chicago’s sails?

11 Mar Will testing irregularities knock the wind out of Chicago’s sails?

By Dale Chu

In 2017, an analysis by Stanford researchers appeared to show students in Chicago Public Schools (CPS) learning faster than almost every other school system in the country. The data was so compelling, it defied conventional wisdom about the relationship between wealth and educational attainment. Not surprisingly, CPS officials have taken great pride in their progress and the tailwind afforded by the positive news and subsequent attention.

To be sure, Chicago is in many ways an exemplar for large urban school systems. However, a recent report provided to the Chicago Board of Education, and their reaction to it, raises red flags that could threaten to undermine the work that has been done to correct the district’s historically lackluster trajectory.

The controversy that’s swirling about revolves around CPS’s use of the NWEA MAP assessment system. Specifically, the district’s outgoing inspector general (OIG) found dramatic differences in the amount of time Chicago students took to complete the untimed test when compared to national averages. Because CPS uses MAP to power its elementary school ratings system (the MAP can make up nearly 50% of a school’s score), these discrepancies are no small matter and could suggest nefarious attempts to game the system.

Local media coverage offers useful analysis and details, but the OIG’s review deck is worth reviewing in its entirety. Notably, CPS has rebutted the report’s suggestion of malfeasance while offering a lukewarm point-for-point response to each of the OIG’s recommendations. The OIG’s report coupled with the Board’s hostility towards it raises a couple of key questions:

1 . Is this an isolated incident or part of a larger pattern?

If you’re CPS, the good news is that the data used in Stanford’s favorable analysis was based on a state standardized test that is no longer given. In this sense, there’s a firewall to the district’s halo. Or is there? Where there’s smoke…

2. Can formative assessments also be high stakes?

This is a huge tension in testing, which I’ve written about previously in these pages. CPS operates under the assumption that an instructionally useful diagnostic assessment system (in this case, MAP) can also be used as part of the school’s accountability system without being at cross-purposes. To their credit, NWEA and others are working to square this circle, but color me skeptical on the potential of better assessment technology overcoming human nature.

Rather than making excuses and discounting the report, let’s hope CPS tackles these troubling issues head on. Anything less risks a real erosion of trust in the integrity of the district’s assessment system.