Indiana Jones and the Dial of (Data) Disarray

06 Dec Indiana Jones and the Dial of (Data) Disarray

By Dale Chu

Indiana Jones is back! For better or worse, the legendary archaeologist will have his hands full again with Nazis and their grotesque ilk, but at least Dr. Jones is not dealing with the disarray that has been the hunt for 2022 statewide test results. Following closely on the heels of the assessment snapshot we published last month, Education Reform Now recently released their landscape analysis and found “a big gap” between states that simply made their data available versus those that did so in an accessible and intelligible way.

Authors Rianna Saslow, Yaz Liow, and Charlie Barone sorted through an unwieldy morass of state data spreadsheets, and what they found wasn’t pretty. The files contained “massive amounts of data that take up a great amount of computing capacity and [were] poorly organized, making it difficult for parents and other stakeholders to make sense of.” In one egregious example, Delaware’s assessment data from the past seven years were all contained within “a single Excel sheet with 989,179 rows and over 10 million data entries. A standard laptop doesn’t even have the capability to work with such a large data set without freezing or crashing at every press of a button.”

But instead of running away from that “boulder,” Saslow, Liow, and Barone put together a helpful digest—including easy-to-understand maps—that highlights the strengths and weaknesses of each state vis-à-vis data transparency. The analysts who assembled our snapshot could immediately empathize, as they spent hours upon hours trying to make heads from tails with some of the spreadsheets they downloaded—if they could even find them. While the dedication and resilience of our team and ERN are admirable, this exercise raises an important issue: the (crazy) need for different groups to sift, sort, and scrutinize their way through state education agency websites in order to get the same information. Why should it be this difficult to do (what should be) a straightforward 50-state comparison? And if it’s this laborious for us, folks who have some background knowledge on the topic, imagine what this means for local parents and other community stakeholders.

Which is to say, getting one’s hands on statewide testing data shouldn’t be akin to finding buried treasure. X should mark the spot. To wit, it took “forever” to find the latest assessment data for some states, while others were no further than a few clicks away (hats off to Alabama, D.C., Colorado, Massachusetts, Tennessee, and Wyoming!). And as of this writing (in December no less), there are still a handful of states that haven’t released any data at all! The delay and disarray are, in a word, problematic. Saslow, Liow, and Barone are right: “Publicly reporting data in a timely and accessible manner is critical to the legitimacy and transparency of the American education system; missing data… is simply unacceptable.”