Does U.S. News Fabricate Data?
Filed in archive College Rankings by Mark on March 13, 2007

(Source: Sarah Lawrence College)
Yes, according to at least one college president. The head of Sarah Lawrence College called out U.S. News in a Washington Post op-ed, claiming the magazine estimates SAT scores in the absence of real data. Does this cast a shadow on the validity of the rankings?
Consider this from InsideHigherEd:
It's not unusual for college presidents to complain about U.S. News rankings (at least out of the earshot of U.S. News editors). But on Sunday, the president of Sarah Lawrence College publicly charged that the magazine is preparing to publish made up, false data about her institution. Meanwhile, Inside Higher Ed has learned that 10 other liberal arts college presidents are preparing a letter to be sent to hundreds of college presidents proposing a new set of policies that might challenge the role of the rankings. The policy options include complete non-cooperation with U.S. News and refusing to fill out the "reputational" survey - which many educators deride as a "beauty contest" that is particularly lacking in substance, even though it represents 25 percent of the magazine's rankings formula.
In an op-ed in The Washington Post, Michele Tolela Myers, Sarah Lawrence's president, wrote that because her college no longer collects or examines SAT scores, U.S. News officials have said that the magazine will just assume that the average SAT would have been one standard deviation (about 200 points) below the average of Sarah's Lawrence's peers. "In other words, in the absence of real data, they will make up a number," wrote Myers.
In an interview Sunday, the head of the U.S. News college rankings division acknowledged that he had told Myers of the magazine's plan to use the system she described in her article. But Robert Morse said that while he told her that was the plan, he also said that the magazine would "seriously" consider other approaches, which he declined to name. Myers, in an interview Sunday, said flatly that Morse had never said there was any other approach under consideration.
The dispute between Sarah Lawrence and U.S. News highlights more than just the rankings issue. The reason Sarah Lawrence is facing this problem is that the college - possibly alone among the many colleges that are dropping SAT requirements - won't even look at SAT scores any more. As more liberal arts colleges like Sarah Lawrence have dropped SAT requirements, the norm has been to go "SAT optional," meaning that students are still welcome to submit the scores. Because a majority of applicants do so, SAT-optional colleges continue to have average SAT scores to report to U.S. News.
And that's where the dispute starts to point to potential problems with both SAT averages and U.S. News. When applicants learn that a college is SAT-optional, it doesn't take an 800 math score to figure out the statistically wise strategy. If your scores are at or above reported averages, submit them. Otherwise, don't. Not surprisingly then, many colleges that go SAT-optional experience both a surge in applications and an increase in their SAT averages ... and their U.S. News rankings go up.
Read the rest here. And draw your own conclusions about rankings and their value.
Permalink: Does U.S. News Fabricate Data?
Tags:
U.S. News Rankings college news does+news sarah+lawrence fabricate+data
Trackback: http://www.creative-weblogging.com/cgi-bin/mt-tb.pl/57838







