AAMC Reporter: December 2007

Inaugural Year of Computerized MCAT® Earns a Good Score

2007 marked the first year that prospective medical students logged into a computer to take the MCAT.

After what was generally regarded as a successful maiden voyage, the Medical College Admission Test (MCAT) is set to begin its second year as a fully computerized examination.

The computerized MCAT exam provided students with shorter testing days, greater scheduling flexibility, and faster dissemination of test results. Given the complexities involved in this kind of shift, testing experts noted that some difficulties are inevitable—and the MCAT exam was no exception. However, glitches affected only a slim minority of test takers, and MCAT exam officials are confident that a stronger, more efficient product will be ready for the 2008 testing cycle.

"In general, the move went well," said MCAT exam director Karen J. Mitchell, Ph.D. "There were some challenges in this first cycle. But it was a learning year. We learned a lot about student preference, among other things, and moving forward we are better prepared."

Of the issues that occurred, perhaps the most widely publicized was a content error that affected approximately one percent of the more than 70,000 tests distributed. In the verbal reasoning section of this exam, a reading passage about robotic fish was mistakenly paired with a series of questions about songbirds.

"I was totally prepared to take it again, but much to my surprise I had done really well," said Daniel Sonshine, a 2007 Brown University graduate who took the exam containing the error, and who will apply to medical school for the 2008-09 academic year. "I only came out of this with a couple of bumps and scrapes, which is surprising because I thought the world was ending."

Once aware of the mismatch, the AAMC did not score these items. Students also had a chance to void their tests at no charge. According to AAMC data, the average score on the flawed test was no different than the average on the rest of the exams. New quality assurance processes were implemented to prevent a repeat incident, Mitchell said.

Other difficulties also cropped up. The testing registration system at times grew sluggish with unexpectedly heavy use. At some testing sites, there was a shortage of available seating toward the end of the testing cycle. (This was especially pronounced in Canada, where students are more likely to schedule their MCAT examinations in the summer, thus creating a higher-than-expected demand for August and September spaces.) In response, the AAMC added four additional dates with over 4,000 testing appointments.

"Every transition to computer-based testing, no matter what the program is, is going to have start-up problems," said Janet Bowker, an educational testing consultant who collaborated with the AAMC on the project.

"Any time you transition from a test that's given two or three times a year to 22 or 23 times a year, predicting the way the volume will work its way through the system is very, very difficult," she said. "It's important to step back and look at the fact that everyone who wanted to take the test got to take it, and everyone who took it had their scores computed correctly and sent to the right places. In a first year of operation, that's what counts."

Another improvement was the length of the exam day which was 30 percent—or about three hours—shorter than it was for the paper-based exam delivery. In almost all cases, exam results came to students within a month of the test date, or half the time as before.

"Getting your scores back in 30 days is fabulous when you're sweating bullets," said Carol Baffi-Dugan, M.A., M.Ed., program director for health professions advising at Tufts University.

More flexible scheduling gave students extra chances to improve their viability as medical school applicants.

"Basically, the extra testing dates keep students in play a bit longer," said Catherine Solow, M.A., assistant dean of student affairs and curriculum at University of Iowa Roy J. and Lucille A. Carver College of Medicine. "They have more opportunity to retake the test and improve their score."

Emily Torell, an undergraduate at Washington University in St. Louis, was mildly disappointed with her first MCAT exam score, but took advantage of the multiple dates and retested a few months later.

"The fact that it was offered so many times affected my decision to retake the exam," she said, adding that her score improved on the second testing. "I was on the border of whether to take it again."

Many students reported a better exam experience. AAMC data show that complaints about bothersome testing conditions, such as excessive noise or heat, were down 60 percent from last year.

"Now you have your own private cubicle," Baffi-Dugan said. "Before it was a big room, and there were a lot of complaints about the space. I didn't really hear any this time around."

Moving forward, examinees should continue to benefit from these perks without facing the implementation issues associated with the computerized MCAT exam's first year. In 2008, new technologies and staggered registration periods will prevent registrants from overloading the system. Twenty-five testing dates and more sites in large urban centers and rural areas will ease capacity issues.

Beyond these enhancements, special consideration has been given to Canadian examinees as a result of their specific issues. Additional testing dates in August and September will better meet the preferences of Canadian students, and the use of mobile testing units allow for placement of test sites in more convenient locations.

—By Elissa Fuchs

Contact Us    © 1995-2008 AAMC    Terms and Conditions    Privacy Statement