Executive Council Decides to Continue MCAT Flagging

After a thorough review of the Medical College Admission Test's (MCAT) score flagging policy - which requires that some exams administered with certain accommodations be marked with an asterisk - the AAMC's Executive Council has voted in favor of continuing the practice.

AAMC's Executive Council decided that the "flagging" of MCAT scores balances the association's responsibilities to disabled examinees and to our institutions.

During the last Executive Council meeting on September 17, staff from the AAMC's Division of Medical School Services and Studies presented findings of a large-scale research project conducted to determine if "flags" were necessary and justified. The research results showed evidence that scores earned by students taking the MCAT under non-standard conditions are not equivalent to those earned by students taking the test under standard conditions.

"The flag is simply a recommendation to admissions officers that they should rely more heavily on other sources of information when making decisions about applicants with flagged scores," said Ellen Julian, Ph.D., director of the MCAT. "The MCAT is a standardized test and, as such, has a responsibility to attest to the validity and comparability of all of the scores it reports."

The Disability Rights Advocates (DRA), a non-profit legal center dedicated to protecting the civil and human rights of people with disabilities, requested that the MCAT program cease flagging scores, and threatened a lawsuit if the association failed to comply. Other testing publishers have faced legal challenges by disability rights organizations, and some have decided to no longer flag their tests.

As a result, the AAMC conducted a thorough analysis examining whether scores obtained under non-standard conditions have validity equivalent to those earned under standard conditions, and whether examinees with flagged scores have been discriminated against in the admissions process, among other things.

Overall, the average flagged score is higher than the average unflagged score. AAMC research shows that examinees with documented disabilities who took the test under standard conditions and then later retook it with extra-time accommodations increased their scores by, on average, more than four times the score increase for the typical test repeater. The research also shows that unanticipated extra time increased scores for 70 percent of examinees in the Verbal Reasoning (VR) and nearly 80 percent in the Physical Sciences (PS) MCAT sections (Biological Sciences was not included in the study). Average examinees increased their scores by three quarters of a scaled-score point for both VR and PS.

Other AAMC research shows that MCAT scores obtained with extra time tend to over-predict USMLE Step scores. In addition, accommodated examinees encountered academic difficulty of sufficient severity to be reported to the AAMC's Student Record System as delaying graduation at more than twice the rate of standard examinees, even after MCAT and other predictor variables were taken into account. "This was the most telling piece of data," Dr. Julian says. "The flagged scores simply do not predict performance in medical school as well as scores earned under standard testing conditions. The testing standards require that we notify score users when we provide scores that we know will not function as promised."

Those who object to flagging worry that marked scores may stigmatize applicants, and ultimately result in a lower acceptance rate for students with disabilities. The American Medical Student Association (AMSA) has taken a position against flagging. "AMSA does not support flagging; we support the right of individual students to choose whether or not to disclose to admissions committees that they received testing accommodations because of a [documented] disability," said Sophie Jan, AMSA's advocacy action committee chair. "Generally, students receive testing accommodations because they have a diagnosed learning disability, and this is a matter of privacy. There haven't been any studies that show that students with learning disabilities make poor clinicians in the long run, and because of it, there's no basis to place a flag on their score."

AAMC research data do not support the idea that admissions committees are viewing the flags as an indication that applicants will be poorer students or clinicians. There is no evidence that examinees with flagged scores have been discriminated against in the admissions process. While a lower proportion of applicants with flagged scores are accepted into medical school -- 49 percent versus a 52 percent acceptance rate for applicants with unflagged scores -- once MCAT scores and GPAs are taken into account, those with flagged scores have a slightly higher probability of being accepted.

"Our study data show that applicants with flagged and non-flagged scores have about the same average MCAT scores," Dr. Julian said. "However, among those applicants who are accepted, flagged scores tend to be lower than non-flagged scores. This leads us to believe that admissions officers may be using the flagged scores as directed, as a cue to give more weight to non-MCAT criteria when making decisions about applicants with flagged scores."

-Suria Santana

Contact Us    © 1995-2008 AAMC    Terms and Conditions    Privacy Statement