SAT test score scandal shows the value of verification

Lately I've been reading up on a scandal that's erupted in standardized testing. The company that processes SAT tests, Pearson Educational Measurement, had at least a one percent error rate in its October 2005 test results. The company claims that the scanning errors were the result of humidity, which it says caused the test paper to expand and the scanning devices to incorrectly read the answers.

Overall, of the 495,000 students who took the October test, 4,411 were awarded lower scores than they deserved, and 600 got higher scores. As you might expect, the lawsuits are flying, since many students were adversely impacted by the incorrect test scores and did not apply to schools they preferred.

This morning I spoke with Bob Schaeffer, Public Education Director of the National Center for Fair & Open Testing who filled me in on the details of the scandal. Many people have said that optical scan voting works the same way as standardized tests. It turns out the parallels don't stop there. Like voting equipment, the standardized test process is managed by private companies, and it is difficult to check their work. Mr. Schaeffer informed me that test takers can obtain a copy of their test and the answers, for a small fee. In fact, according to one article I read about this scandal in People magazine, the scoring errors went unnoticed until two students asked that their tests be hand-graded.

The SAT scandal demonstrates why it is necessary to verify the performance of computer software and hardware when important transactions, like college test scores or votes are at stake. When I first heard the explanation by Pearson, the test counters, that the scoring errors were caused by humidity, I thought to myself, I have heard this before. I have heard this explanation given many times, in many parts of the country, for why electronic voting machines failed, or why in-precinct optical scanners didn't operate properly.

Mr. Schaeffer at the National Center for Fair and Open Testing wants to change the testing process so that all takers would get their results back in the mail and have the opportunity to double-check their own scores. I'm not suggesting we do the same thing with ballots, since ballots are secret and our votes are private. But there may be may be other valuable lessons to learn from the SAT scandal, such as the need for transparency in the test-scoring and vote-counting processes, as well as the need for routine and public verification of test results as well as vote counts.

More details about the SAT scoring scandal can be found in this story from CBS News.