Blue, A.V., Gilbert, G.E., Elam, C.L., & Basco, W.T., Jr. (2000).
Does Institutional Selectivity Aid in the Prediction of Medical
School Performance? Academic Medicine, 75, S31-S33.
PURPOSE: Institutional validity studies of admission decision-making
data help to determine which characteristics should be accorded
highest importance in applicant selection. Given the reliance upon
institutional selectivity as an important admission characteristic
and the different types of selectivity classifications available
for medical schools to use, the purpose of this study was to examine
how well three measures of institutional selectivity could predict
medical students' performances, specifically their performances
on the USMLE Step 1 and Step 2 and their final medical school GPAs.
METHODS: Admission and medical school performance data were
obtained for the 199201995 matriculants at the study institution,
the Medical University of South Carolina (MUSC). Admission data
for each student consisted of his or her MCAT scores, undergraduate
GPA, undergraduate institution, three institutional selectivity
categorization indices (the 1983 HERI Index, Barron's Admissions
Selector Rating, and the Carnegie Classification), age, gender,
and underrepresented minority (URM) status. Medical school performance
data consisted of USMLE Step 1 and 2 scores and final GPA. Students
admitted under the institution's existing Early Assurance Program
(EAP) were excluded from analysis because an MCAT score was not
required for their admission. Descriptive statistics were performed
for all variables. Stepwise linear regression (adjusted R-square
method) was used to assess which control variables (undergraduate
GPA, gender, URM status, age) contributed significantly to predicting
USMLE Step 1 and 2 scores and final GPA. Age was the only control
variable that did not contribute significantly to predicting any
of the dependent variables. Multiple linear regression was then
performed with each of the institutional selectivity or categorization
indices, controlling for URM status and gender. The powers of the
multiple regression equations ranged from 88.2% to 96.0% for an
alpha of 0.05 and with estimating of small effect sizes.
RESULTS: All models predicted statistically significant
variations in the dependent variables. Uniformly, the worst-fitting
model was that which consisted of only the three control variables
GPA, gender, and URM status. The amounts of explained variation
ranged from 17% to 32%. Addition to the model of any institutional
selectivity index or categorization slightly improved prediction
(as measured by proportion of variance explained) above the prediction
provided with GPA and demographic characteristics alone. When the
MCAT score was added to the model involving the control variables
and the GPA, it improved predictive ability of the equation by 6-13%.
The addition of the institutional selectivity indices or categorization
after the MCAT score was in the model added nothing to the predictive
ability. Control variables plus MCAT score accounted for 38% of
the variation in USMLE Step 1 scores, 38% of the variation in final
GPA, and 28% of the variation in USMLE Step 2 scores.
CONCLUSIONS: Our results show that none of the three institutional
selectivity indices or categorizations and any GPA adjustment that
would follow will improve correlations with performances on USMLE
Step 1 and Step 2 and final GPA if MCAT scores and unadjusted GPA
are used in conjunction. Our findings suggest that using institutional
selectivity indices or categorizations as an admission characteristic
may not be necessary. In addition, use of institutional selectivity
indices or categorizations may discriminate against applicants with
other desirable characteristics who have been granted degrees from
less selective undergraduate institutions. Our results reassure
admissions officers that the performances of students who attend
smaller undergraduate institutions or community colleges are predictable
when using their MCAT scores and undergraduate GPAs. In summary,
our results indicate that the characteristics of the degree-granting
undergraduate institution, as measured by three different types
of institutional selectivity or categorization, do not add to the
ability to predict performances on USMLE Steps 1 and 2 and overall
medical school GPA if the MCAT score and unadjusted undergraduate
GPA are available. The result also further support the predictive
validity of the scores on the MCAT examination for medical school
performance.
|
