Situational Judgment Test Background
The American healthcare system has changed. Medicine in the 21st century is team-based; it requires a collaborative approach to provide health care in a system that meets the access, safety and quality needs of all patients. Physicians need awareness and appreciation of socio-cultural issues that affect interactions with a more diverse set of colleagues and patients. Accordingly, medical students need to demonstrate both academic and pre-professional competencies to be successful in medical school, in residency training, and as physicians. In addition to students who are academically prepared, we need to select those who have strong professional and interpersonal skills necessary to develop clinical skills.
The AAMC developed a Situational Judgment Test (SJT) to enhance the medical school admissions process by providing reliable and accurate information about applicants’ pre-professional competencies in an easy-to-understand and easy-to-use format. We believe that if the AAMC SJT is incorporated into pre-interview screening, it can help to facilitate holistic review earlier in the application screening process, identify applicants with strong pre-professional competence who otherwise may not have been considered, and select a more diverse pool of applicants.
What is a Situational Judgment Test (SJT)?
The AAMC SJT is a standardized test that presents a series of hypothetical scenarios students may encounter in medical school and asks examinees to evaluate the effectiveness of a series of behavioral responses to each scenario. The AAMC has collaborated with subject matter experts in the medical school community, including faculty, admissions officers, and student and diversity affairs officers to develop content and a scoring key for the SJT.
Test takers’ responses provide insight into their knowledge of effective and ineffective behaviors across eight core competencies for entering medical students: Service Orientation, Social Skills, Cultural Competence, Teamwork, Ethical Responsibility to Self and Others, Reliability and Dependability, Resilience and Adaptability, and Capacity for Improvement.
Research on the AAMC SJT
The AAMC is executing a multi-phase research plan to evaluate the SJT in four areas, including psychometrics, predicting medical student performance, fairness, and community reactions.
In 2017, the AAMC Admissions and Selection Research Team launched a longitudinal validity study with eight U.S. medical schools. The purpose of the study is to explore the relationship between SJT scores and medical student performance. The validity study builds upon a 2016 field test study with MCAT examinees, which investigated the relationship between SJT scores and academic metrics (e.g., MCAT scores, GPA), personality factors, and other admissions data, and evaluated group differences on SJT scores. The AAMC has also engaged medical school admissions officers in focus groups to assess reactions to the SJT. Results from the focus groups suggest medical schools continue to face challenges in remediating students who demonstrate professionalism issues, and new assessments that measure pre-professionalism during pre-screening could add value.
Overall results of these research studies have demonstrated the AAMC SJT:
- is reliable and valid,
- predicts medical student performance,
- demonstrates small to no group differences, and
- adds value to the admissions process above and beyond existing application data.
In 2019, we are conducting a field test with MCAT examinees to continue examining the psychometric characteristics of the SJT and finalize the scoring approach. In addition, we will study the effects of coaching and faking on SJT performance and assess operational considerations such as the administration format of the exam and potential cost. To learn more about the AAMC’s SJT research, see the SJT slide deck presented at the 2019 GSA/GEA meetings .
For more information on the upcoming pilot study, please contact Rebecca Fraser and Tom Geiger at SJT@aamc.org.