The following supports guidance that was developed by the AAMC in response to inquiries about residency interviews for the 2022-23 residency selection cycle.
- Appendix A: Planning for Virtual Interviews — Best Practices
- Appendix B: Diversity Equity and Inclusion Recruitment Resources
- Appendix C: Possible Areas of Research and Example Questions
Appendix A: Planning for Virtual Interviews — Best Practices
Frohna, J. G., Waggoner-Fountain, L. A., Edwards, J., Fussell, J. J., Wueste, B., Gigante, J., Vinci, R. J., Heitkamp, N. M., Neelakantan, M. K., Degnon, L. E., & Blankenburg, R. L. (2021). National Pediatric Experience With Virtual Interviews: Lessons Learned and Future Recommendations. Pediatrics, 148(4). https://doi.org/10.1542/peds.2021-052904
Huppert, L. A., Hsiao, E. C., Cho, K. C., Marquez, C., Chaudhry, R. I., Frank, J., Goglin, S. E., Hsu, G., Kathpalia, P., Khanna, R., Kompala, T., Rao, M. N., Bower, B. A., Trafas, V., Santhosh, L., Schwartz, B. S., Babik, J. M. (2020). Virtual Interviews at Graduate Medical Education Training Programs: Determining Evidence-Based Best Practices. Acad Med. https://doi.org/10.1097/acm.0000000000003868
Melchers, Klaus G, Petrig, Amadeus, Basch, Johannes M., Sauer, Juergen. A Comparison of Conventional and Technology-Mediated Selection Interviews With Regard to Interviewees’ Performance, Perceptions, Strain, and Anxiety. Frontier Psychology, 12 January 2021. https://doi.org/10.3389/fpsyg.2020.603632
National Resident Matching Program. Research Brief: Impact of Virtual Experience on the Transition to Residency – Findings From the 2021 Applicant and Program Director Surveys. July 2021. Research-Brief-Virtual-Experience-2021-FINAL.pdf (kinstacdn.com)
Patel, T. Y., Bedi, H. S., Deitte, L. A., Lewis, P. J., Marx, M. V., Jordan, S. G. (2020). Brave New World: Challenges and Opportunities in the COVID-19 Virtual Interview Season. Acad Radiol, 27(10), 1456-1460. https://doi.org/10.1016/j.acra.2020.07.001
Straus, S. G., Miles, J. A., Levesque, L. L. (2001). The effects of videoconference, telephone, and face-to-face media on interviewer and applicant judgments in employment interviews. Journal of Management, 27(3), 363-381.
Wolff, M., Burrows, H. (2021). Planning for Virtual Interviews: Residency Recruitment During a Pandemic. Acad Pediatr, 21(1), 24-31. https://doi.org/10.1016/j.acap.2020.10.006
Appendix B: Diversity Equity and Inclusion Recruitment Resources
Bendick M, Nunes A. Developing the research basis for controlling bias in hiring. J Soc Issues. 2012:238–262.
Budhu, J. A., Velazquez, A. I., Said, R. R., Jordan, J. T. (2021). Opinion & Special Articles: Maximizing Inclusiveness and Diversity Through Virtual Residency Applications and Interviews. Neurology, 97(13), 647-650. https://n.neurology.org/content/neurology/97/13/647.full.pdf
Chaisson, N. F., Ashton, R. W. (2021). Virtual Interviews and Their Effect on Cognitive Load for Graduate Medical Education Applicants and Programs. ATS Sch, 2(3), 309-316. https://doi.org/10.34197/ats-scholar.2020-0156PS https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8518638/pdf/ats-scholar.2020-0156PS.pdf
Freeman, C. D., Guissé, N. F., Ceasar, D. R., Fakunle, O., Fonseca, C. A., Fraz, F., Gillis, R. P., Harris, N. M., Nichols, A. C., Oboh, O., Henry, T. L. (2022). Reflections From Underrepresented in Medicine Applicants on the 2020 Virtual Interview Season. J Grad Med Educ, 14(2), 155-157.
Gerull, K. M., Enata, N., Welbeck, A. N., Aleem, A. W., Klein, S. E. Striving for Inclusive Excellence in the Recruitment of Diverse Surgical Residents During COVID-19. Academic Medicine: February 2021. Volume 96. Issue 2. P 210-212. https://Doi.org/10.1097/ACM.0000000000003812
Hennigan M., Evans L. Does hiring for “culture fit” perpetuate bias? Does Hiring for 'Culture Fit' Perpetuate Bias? (shrm.org). Published October 31, 2018.
Marbin, Y., Hutchinson, Y-V., and Schaeffer, S. Avoiding the Virtual Pitfalls: Identifying and Mitigating Biases in Graduate Medical Education Videoconference Interviews. Academic Medicine, Vol. 96, No. 8 / August 2021. pp 1120-1124.
Otugo, O., Alvarez, A., Brown, I., Landry, A. (2021). Bias in recruitment: A focus on virtual interviews and holistic review to advance diversity. AEM Educ Train, 5(Suppl 1), S135-s139. https://doi.org/10.1002/aet2.10661
Appendix C: Possible Areas of Research and Example Questions
This appendix lists possible areas of research and example questions that could be used to evaluate virtual interviews. It is intended to provide the reader with an understanding of the research program the AAMC plans to co-create with multiple stakeholders.
Fairness
- Are there differences in interview ratings by applicant type and demographic group? Does the pattern of results differ for virtual and in-person interviews?
- Did access to technology affect applicants?
Validity and psychometrics
- Are ratings lower for virtual or in-person interviews?
- Does rater reliability differ between virtual and in-person interviews?
- Are there different Match rates and/or transfer rates (or positions on the Rank Order List) for applicants who interviewed virtually or in-person?
- How do the correlations between virtual interview ratings and trainee outcomes compare to the correlations between in-person ratings and trainee outcomes?
Applicant preparation, use, and reactions
- Do applicants believe that they were able to adequately show “who they are” in a virtual interview?
- What do applicants view as the pros and cons of virtual and in-person interviews? Which do they prefer?
- Using fellowship learners who interviewed in-person for residency and virtually for fellowship, how did experiences impact interview method preference?
- Two years post COVID-19, are learners still happy with the program where they matched?
Program use and reactions
- Is a broader pool of applicants (e.g., more geographically and demographically diverse, etc.) invited to virtual interviews than in-person?
- Does the applicant “no-show” rate change in virtual versus in-person?
- How do the interview processes compare for virtual and in-person (e.g., schedule, time, faculty involvement, etc.)?
- Two years post-COVID-19, are program directors still happy with their selection decisions?
Cost and resources
- Did virtual interviewing substantially decrease the cost of applying to residency?
- How much educational or clinical time was spent on virtual versus in-person interviews?
- What is the environmental impact of virtual compared to in-person interviews?
Implications for recruitment
- How does the applicant yield compare for virtual compared to in-person interviews?
- Do trainees who interviewed virtually report less satisfaction after entering training than do trainees who interviewed in-person?