The following guidance was developed by the Association of American Medical Colleges (AAMC) in response to inquiries about residency interviews for the 2022-23 residency selection cycle. The recommendations outlined below reflect the collective sense of how to proceed, and the AAMC urges each medical school, sponsoring institution, specialty society, and residency program to carefully consider them and commit to working together to create an equitable, transparent, and successful residency selection cycle for all stakeholders. Because students rely on predictable, common practices across schools and programs as they prepare to transition to residency, this guidance will reduce unnecessary confusion, stress, and inequity among students while promoting a more successful residency selection process for all.
This guidance is intended to add to, but not supersede, the independent judgment of a medical school, sponsoring institution, or residency program. The AAMC is proposing these recommendations to help specialty and training programs consider their interviewing and recruiting options and to make evidenced-based decisions.
- Recommendation #1: Programs should conduct virtual interviews for all applicants (including local applicants) for the 2022-23 cycle.
- Recommendation #2: Hybrid interviewing within the same program is strongly discouraged for the 2022-23 cycle.
- Recommendation #3: Programs should share their interviewing plans with applicants clearly and early, preferably when application requirements are released.
- Recommendation #4: Programs should prepare for the interview cycle by reviewing resources on anti-bias practices, best practices in creating and implementing virtual interviews, and creating tools for recruiting in a virtual context.
- Recommendation #5: Organizations should commit to collaborative research to explore key aspects and outcomes of in-person and virtual interviews.
In-person interviewing has been a staple in residency and fellowship selection processes for decades. There is considerable validity evidence to support the use of structured interviews in selection, and learners and program staff have positive reactions to in-person interviews. Prior to the COVID-19 pandemic, little consideration had been given to virtual interviewing to fill residency positions. Faced with the pandemic, the academic medicine community pressed forward to address concerns over education, patient care, and research. Specifically, the openness to explore innovations and new methods in educational curricula, the clinical environment, and the residency selection process should be commended.
After two recruitment cycles using virtual interviews, both learners and program staff have effectively navigated their use and have reported some benefits. Learners reported travel savings and fewer disruptions and time missed from course work and clinical training. Program directors reported lower recruiting expenses as well as time savings and schedule flexibility for program faculty and staff. There is also a growing sense that virtual interviewing leads to increased equity in the process due to substantial reduction in applicants’ travel costs.
As the community moves from full immersion in pandemic-related efforts to consider solutions for a post-COVID-19 era, it is important to consider the changes made in the residency selection process and to evaluate their impact using an evidenced-based approach as the foundation for long-term decision making. More study is needed to evaluate both virtual and in-person interviewing to uncover possible inequities, mitigate those inequities, and establish best practices to address the multiple needs expressed by all stakeholders. These studies are critical to long-term decision making.
While cost savings are important to some applicants, other concerns that deserve consideration have been raised. Some learners believe they can better demonstrate who they are and why they are uniquely suited for a program in-person rather than virtually. Applicants also report that in-person interviews provide a better sense of the program culture as well as offer intangible benefits and insights that are not available in a virtual format. Anecdotally, some students and learners who are underrepresented in medicine and from lesser-known schools have reported that they prefer in-person interviews to increase their exposure to the program and because they believe there is less potential for bias when meeting program directors in person.
In the short term, the AAMC recommends continuing the virtual interview experiment that was necessary for the last two cycles. There are clear cost and time savings to applicants and programs. High-level data (overall Match rates, general satisfaction measures, etc.) suggest the process has been generally successful. And most importantly, there is an opportunity to study the process and make a more informed decision for the longer term. The AAMC is also considering incremental process improvements that could be achieved by more programs sharing information about their training program, application requirements, and deadlines. This may minimize learner stress and anxiety, as well as improve perceptions of fairness and create positive reactions about the training program, specialty, and/or sponsoring institution.
Recommendation #1: Programs should conduct virtual interviews for all applicants (including local applicants) for the 2022-23 cycle.
Virtual interviews improve equity by eliminating a substantial portion of the cost of application and by opening opportunities for applicants who may have otherwise declined an interview invitation due to financial constraints. Furthermore, the relative ease of scheduling virtual interviews and reducing time away from clinical rotations improves their educational experience and may reduce applicant stress and anxiety. Virtual interviewing also reduces expenses and resource constraints for programs. Reduced time commitment from faculty may have the added benefit of more faculty being willing to participate in the interviewing process. Given the number of interviews conducted each cycle, virtual interviewing is also environmentally responsible (Donahue et al, J Grad Med Educ. 2021; https://pubmed.ncbi.nlm.nih.gov/33680306/).
Recommendation #2: Hybrid interviewing within the same program is strongly discouraged for the 2022-23 cycle.
While the recommendation is that all programs should conduct interviews in a virtual-only format, if in-person interviews are considered, it is strongly recommended that organizations use a single interview format (i.e., all virtual or all in-person) during the same stage of the selection process. Research comparing the two formats shows that, on average, interviewee performance ratings are lower for virtual than in-person interviews. (Melchers et al, Frontiers in Psychology, Jan 2021, Vol 11, Article 603632). Another potential risk for applicants who choose a virtual format over in-person, if both are offered, is that they may be perceived by program directors as being less interested in the program. Using a hybrid approach could disadvantage applicants who interview virtually and could further exacerbate inequities if applicants from disadvantaged backgrounds interview virtually at a higher rate due to financial constraints.
Recommendation #3: Programs should share their interviewing plans with applicants clearly and early, preferably when application requirements are released.
Applicants have more positive reactions to selection systems and organizations when they are transparent. Providing information early in the cycle will reduce unnecessary stress for applicants and help them better prepare for interviews. The AAMC will work with specialty societies to share residency application information with medical schools and learners. Individual programs may enter their decision about interview format along with other application requirements in the Electronic Residency Application Service® Account Maintenance System.
Recommendation #4: Programs should prepare for the interview cycle by reviewing resources on anti-bias practices, best practices in creating and implementing virtual interviews, and creating tools for recruiting in a virtual context.
Continuous improvement in virtual interviewing and creating an equitable recruitment process for learners requires planning. Virtual interviewing can amplify biases (e.g., technology inequities, cognitive load, cues in an interviewee’s living space) that can be mitigated with the appropriate strategies and advanced planning (Marbin et al, Acad Med. 2021; 96: 1120-1124). Staff and faculty training should be implemented to mitigate potential bias and variability in approaches.
According to the 2021 National Resident Matching Program Applicant and Program Director Research Brief, more than 50% of applicants in the 2020-21 cycle reported that learning about a program’s culture was moderately or very difficult from web-based materials. Developing virtual resources (e.g., virtual open houses, presentations about the city, etc.) for applicants could improve their perception of prospective training programs.
Recommendation #5: Organizations should commit to collaborative research to explore key aspects and outcomes of in-person and virtual interviews.
In collaboration with the Undergraduate Medical Education-Graduate Medical Education (UME-GME) communities, the AAMC plans to develop a multiorganization research collaboration. The association will work with other organizations to create a research program that includes these key topics and example research questions (more examples in Appendix C):
- Fairness (e.g., mean ratings by group, perceptions of fairness).
- Psychometrics (e.g., rater reliability, mean and standard deviation of ratings).
- Validity (e.g., correlations with other selection data and trainee performance and satisfaction data).
- Applicant reactions (e.g., ease of use, ability to demonstrate “who you are”).
- Program reactions (e.g., value added).
- Cost and resources (e.g., dollars saved, time saved, environmental impact).
- Learner and program resources (e.g., resources used, usefulness of resources).
- Implications for recruitment (e.g., applicant yield).
Multi-organizational projects will be featured on our AAMC Research and Innovations Supporting the Residency Application Process page.
A rigorous research agenda will require well-defined metrics and mechanisms to collect and share data; therefore, specialties, institutions, and programs are encouraged to collaborate with appropriate organizations to design and conduct research to evaluate the interviewing process.
Without further study and an evaluation process, the community is left with perspectives based on innuendo, anecdotal experiences, and collective biases toward one method or the other. Both methods require more study and evidence upon which long-term decisions can be made for the ultimate benefit of both learners and program directors. The AAMC, in partnership with the following organizations and the UME-GME community, are prepared to lead a research agenda that would encourage continuous process improvement to help support future decision making in this realm.
This guidance was developed by:
The Association of American Medical Colleges
- American Medical Association
- Council of Medical Specialty Societies/Organization of Program Director Associations
- Educational Commission for Foreign Medical Graduates
- National Resident Matching Program