MCAT is a program of the Association of American Medical Colleges

MCAT AMCAS ERAS NRMP Publications More Resources	Shopping Cart Site Map

About the MCAT Exam

Registration

Examinees with Disabilities

Swanson, D.B.; Case, S.M.; Koenig, J.A.; Killian, C.D. Preliminary Study of the Accuracies of the Old and New Medical College Admission Tests for Predicting Performance on USMLE Step 1. Academic Medicine, 71;S25-S30, 1996.

PURPOSE: To examine the utility of the new MCAT for making admission decisions using scores on Step 1 of the United States Medical Licensing Examination (USMLE); to compare the accuracy of the "new" and "old" MCATs in predicting Step 1 performance; and to investigate school-to-school variability in the accuracy of predictions.

METHODS: Two sample groups were used for which parallel analyses were performed for the new MCAT (N=11,145) and for the old MCAT (N=11,350). The new MCAT sample consisted of examinees who took Step 1 for the first time in June 1994; had taken the new MCAT in 1991 or 1992; entered medical school in 1992; had all undergraduate information required for analysis (GPA, index of the selectivity of the undergraduate institution attended, and product of index and science GPA); and came from one of the 112 schools with at least 30 examinees taking both the June 1993 and June 1994 Step 1 exams. The old MCAT sample consisted of examinees who met the same criteria as the new MCAT group but took Step 1 for the first time in June 1993; had taken the old MCAT in 1989 or 1990; and entered medical school in 1991. The total score on USMLE Step 1 was the dependent variable. The predictors for this analysis were the Biology, Chemistry, Physics, Skills Analysis: Quantitative, and Reading scores on the old MCAT and the Biological Sciences, Physical Sciences, Verbal Reasoning, and Writing Sample scores on the new MCAT, as well as undergraduate information. Descriptive statistics for all measures and correlations between undergraduate information, MCAT scores, and Step 1 total scores were calculated. Using the dependent variable and the predictors, regression analysis was performed for the full group of each cohort ("full-group analyses"). A second set of analyses was performed within-school for each cohort ("within-school analyses") at each of the 112 schools included in the study.

RESULTS: The mean Step 1 total score was approximately one point higher in 1994 than in 1993. GPAs were also higher in 1994 and showed more variability than in 1993. The range of correlations between MCAT section scores and the Step 1 total score showed more spread for the 1991 exam than the earlier test version, ranging from .14 to .52 for the new MCAT, and .40 to .48 for the old MCAT.

For the full-group analyses, regression analysis using old MCAT and undergraduate information accounted for 36% of the variation in Step 1 total test scores; the same value was arrived at for the new MCAT scores. When only MCAT scores were used in the equation, similar values (R²=.32) were observed for the 1993 and 1994 groups. The new MCAT Biological Sciences and Physical Sciences scores contributed most to predictive accuracy. The undergraduate information added little to predictive accuracy.

Median correlations were reported for the within-school analyses. For the 1993 cohort, the median was .49 (range of .16 to .77). For the 1994 cohort, the median was .51 (range of .13 to .86). Comparison of standard errors of prediction revealed that the standard error for the new MCAT was, on average, approximately one point larger (less precise) than for the older test.

CONCLUSIONS: The new MCAT's predictive accuracy was found to be similar to that of the older test version, and both were found to provide a more accurate prediction of USMLE Step 1 performance than undergraduate information alone. The level of accuracy was sufficient to establish that the MCAT has utility for the medical school admissions process. The authors noted that this was an important finding given that the new test is approximately 18% shorter. The authors also noted that the limited range of skills that Step 1 measures and the exclusion of students who may have not taken Step 1 due to not having completed the required basic science courses posed limitations for this study. The authors recommend additional research on students with academic difficulty and exploring reasons for school-to-school variance in predictive accuracy.