Koenig, J.A.; Leger, K.F. A Comparison of Retest Performance and Test-preparation Methods for MCAT Examinees Grouped by Gender and Race-Ethnicity. Academic Medicine, 72; S100-S102, 1997

PURPOSE: To examine performance for repeat MCAT test-takers classified by racial-ethnic status and gender, and to examine the extent to which retest performance is affected by method of preparation, particularly partcipation in commercial review courses.

METHOD: This study used a sample comprised of individuals who tested for the first time in April and repeated the test in August. Two examinee cohorts were included: individuals testing twice in 1993 and those testing twice in 1994 (N=12,080).

For retest performance, multiple regression techniques were used to conduct a series of analyses of covariance (ANCOVAs) in which the initial score (April) was considered the covariate, the retest score (August) was the dependent variable, and racial-ethnic status and gender were the grouping variables. Two sets of analyses were run, one on individuals grouped by racial-ethnic status and another for gender groups. Analyses were run separately for each of the four MCAT test sections, with Writing Sample alphabetic score converted to numbers. For comparing preparation techniques, data were obtained from a questionnaire, administered at the end of the MCAT test day. Preparation methods were categorized as accessible (inexpensive and accessible to most examinees, i.e., enrollment in a commercial review course (CRC)). A series of exploratory analyses were performed to determine if preparation methods affected retest performance differentially for men vs. women and African-Americans vs. Caucasians. Then, a series of regressions were run in which variables were entered hierarchically as follows: initial score and grouping variable; all accessible techniques; and CRC participation.

RESULTS: The average score changes between the initial test and the retest were .62 for Verbal Reasoning (VR), .55 for Biological Sciences (BS), .51 for Physical Sciences (PS), and .28 for the Writing Sample (WS). Comparison of retest score changes across groups revealed that Caucasians and men tended to make larger gains than did African-Americans and women. ANCOVA results showed statistically significant differences (p<.001) between groups for all test sections when the initial score was held constant.

The exploratory analyses revealed that preparation methods did not differentially affect retest performance for men vs. women and African-Americans vs. Caucasians. R² increases associated with the various test preparation methods were very small, no larger than .01. Regression coefficients were also low with some being negative. The regression coefficients associated with CRC participation for the racial-ethnic group analyses were .24 for BS, .13 for PS, .10 for VR, and .02 for WS. For gender groups, coefficients were similar: .27 for BS, .15 for PS, .11 for VR and .01 for WS. Regression coefficients were slightly higher for those individuals who reported that they had not taken a commercial review course prior to the April test. For the racial-ethnic groups, coefficients were .42 for BS, .26 for PS, .16 for VR, and .01 for WS. For gender groups, they were .51 for BS, .30 for PS, .17 for VR, and .03 for WS.

CONCLUSIONS: The authors found that retest gains, on average, were smaller for African-Americans and women than for Caucasians and men. It was also found that the regression coefficients obtained from the analyses of covariance indicated that retest gains were smaller at every initial score level for African-Americans and women. Further, the mean initial scores for Caucasians and males were higher than for African-Americans and for females indicating that Caucasians and males who decide to retest have higher scores than African-Americans and females. The authors suggested that these findings raise many questions regarding the factors that influence retest decisions, such as do African-Americans and women receive different advice than Caucasians and men? And, what role do finances play in influencing decisions?

The initial exploratory analyses on the effects of various preparation techniques did not uncover any differential effects across study groups -- preparation style appeared to be constant across groups. The regression weights for commercial review course enrollment tended to be higher than those for the more accessible preparation methods such as independent study. Commercial review course participation was associated with a .27 improvement in retest performance. The authors stated that several issues need to be considered in putting these findings in perspective. For instance, test preparation data were collected at the end of the MCAT test day, when examinees may have been fatigued. In addition, some response options (i.e., independent study) may have been interpreted quite differently by different individuals. Suggestions for additional research are given.

Contact Us    © 1995-2008 AAMC    Terms and Conditions    Privacy Statement