|
|
|
|
|
|
||||||||||
|
Congressional Report: Congress Has Some Unfinished Business as New Term Begins AAMC Efforts Help Roll Back CMS Paperwork Requirements Innovations in Medical Education: Doctors-in-Training Learn How to Tell Stories Healing Deep Wounds: Program at Bellevue / NYU Provides Care for Torture Survivors Nursing Shortage Prompts Creative Solutions Author Q&A: 'Sometimes Wrong, Never in Doubt' 'Flagging' Debate Continues; Data Show Score Fluctuations A Word from the President "A Day in the Life of a Medical Student"
|
|
'Flagging' Debate Continues; Data Show Score FluctuationsFor years there has been an ongoing debate about whether standardized tests such as the Medical College Admission Test (MCAT) should have their scores "flagged" - marked with an asterisk - when administered with accommodations for examinees with disabilities. Federal law requires that test takers with disabilities, including learning disabilities (LD), attention-deficit/hyperactivity disorder (ADHD), and other similar conditions, be granted accommodations, which often take the form of separate rooms or extra time. To protect the integrity of their scores, the MCAT and other testing publishers have chosen to annotate the scores from their standardized tests that have been earned under such non-standard circumstances. Many types of accommodations are not flagged, because they do not fundamentally alter the testing conditions. The only accommodations flagged by the MCAT are extra time or the use of tools not available to the standard test taker, such as a computer or a reader. But after facing legal challenges, some testing publishers have decided to no longer flag their tests. Although the MCAT continues to do so, it is now actively reviewing this policy and has a large-scale research project under way to determine if flags are necessary and justified. The first report of a planned series of three research papers, composed by the AAMC and the University of North Carolina at Chapel Hill Medical School - and partially funded by the ACLD Foundation (which supports the activities of the Learning Disabilities Association of America) - examines the impact of accommodations on test scores and is almost complete. More time, better scoresAnalyzing data from 1994 to 2000, the report compares the scores of MCAT examinees who took the test under standard conditions (standard) to those of examinees who took the test under non-standard (flagged) conditions. The results were consistent each year of the study, and indicate that the examinees taking the MCAT with extended time achieved higher mean scores on all sections of the test. On the total of the three multiple-choice test sections, Physical Sciences, Biological Sciences, and Verbal Reasoning, the 2,401 examinees taking the MCAT with flagged accommodations averaged 26.3, compared with 25.1 for the almost 300,000 examinees taking it under standard conditions (almost one-fifth of a standard deviation difference).
Test takers with ADHD scored higher than those with LD, who scored higher than all other groups, including those taking the examination under standard conditions. The most common accommodations provided between 1994 and 2000 were extended time and separate room. Other accommodations, constituting a very small percentage of total accommodations provided, included an enlarged print test, a reader, the use of sign language, and wheelchair accommodations, though not all of these accommodations were flagged. According to the report, students who took and then retook the MCAT under standard conditions achieved an average increase of 1.5 points for the combined multiple-choice sections of the test. In contrast, students with disabilities who took the test first under standard conditions and then retook it with an accommodation gained an average of 6.5 points. Practical significanceThe practical significance of the higher mean scores achieved by flagged exams when viewed as admission criteria or predictors of success in medical school is unclear, according to the report. "A variety of factors might explain the higher mean scores of examinees granted special accommodations," according to Ellen Julian, Ph.D., director of the MCAT. "One explanation could be that accommodations properly compensated for the test-takers' disabilities, and that the flagged population had slightly more academic ability than the standard examinees. One reason that might happen," Dr. Julian said, "is that students with learning disabilities, having succeeded for years in meeting academic challenges, are more aware of their strengths and weaknesses, and may be more realistic in assessing their educational potential, so fewer of them take the MCAT when unprepared." Another possibility, she added, is that the extended time actually is overcompensation for some of the students with disabilities and artificially inflates their scores. By the same note, considering the fact that the MCAT is a timed test, the question remains if extending the time for disabled test-takers fundamentally alters what the MCAT measures. "One way to tackle this question - and to assure equitable conditions for all examinees - would be to carefully assess the processing speed of each person requesting extended time," says Dr. Julian. "Then disabled test takers could be granted extended time in smaller increments than the time-and-a-half or double time now given, so that both disabled and non-disabled test takers could experience the need for speed to the same degree. Unfortunately, accurate and 'non-fakable' measures of processing speed are not currently available." The upcoming papers analyzing this issue will look at the impact of flagging on admissions probability and the predictive validity of flagged scores. Research is also under way on how extra time would improve the scores of non-disabled MCAT examinees. - Suria Santana, ssantana@aamc.org |
||||||||
|
||||||||||
|
Contact Us © 1995-2008 AAMC Terms and Conditions Privacy Statement |