aamc.org does not support this web browser.

Best Practices for Creating and Conducting Interviews

What is the role of the interview in the residency selection process?

The residency selection process aims to find the best alignment, commonly known as “fit,” between prospective applicants and the residency program. While medical education literature lacks a standardized definition of fit, the organizational psychology literature highlights two dimensions of fit that can provide valuable insights: person-organization fit, and person-job fit.

Person-organization fit pertains to the compatibility between an applicant’s personality, attitudes, work/learning style preferences, goals, and the residency program’s mission, values, culture, and learning approaches/resources. In the context of residency selection, this can be referred to as “applicant-program fit.”

Person-job fit focuses on the compatibility between an applicant’s competencies, knowledge, skills, abilities, and other attributes and the competencies and characteristics required to succeed in residency. In the residency selection context, this type of fit can be understood as “applicant-specialty fit.”

The interview plays a crucial role in assessing applicant fit for residency programs, especially in evaluating competencies that may not be easily assessed through other components of the selection process.

Setting Up the Interview Process

Overview and intended audience

This section includes guidance regarding:

This content is specifically tailored for program directors, individuals who serve on selection committees, and those responsible for developing interview content and establishing interview processes within residency and/or fellowship programs. Additionally, individuals who serve as interviewers may also find value in certain aspects of this content.

Selecting an interview format

With the widespread adoption of virtual interviews during the COVID-19 pandemic, residency programs now have a range of options to choose from when determining the interview format for each application cycle. The AAMC continues to recommend that residency programs use a virtual interview format. Every residency program has their own unique mission, goals, and context and thus may take different approaches for sound reasons. We recognize that what is best for one institution may not be best for another. Each institution should evaluate the pros and cons of interview formats in their unique context to determine what is best for their program. As you decide what interview format is best for your institution, consider referring to the AAMC’s recommendation and key considerations.

Back to top

Selecting an interview approach

Selecting an appropriate interview approach, such as a structured or unstructured interview process, is a crucial decision when designing the selection process for your residency program. In this section, we will explore the concepts of structured and unstructured interviews and discuss the factors to consider when choosing an approach.

Structured versus unstructured interviews

Interviews are widely utilized as a selection method in both academic and applied settings, and extensive research has been conducted, including within the field of medical education. While some studies have shown limited correlation between interview performance and future success as a medical student or resident, a recent systematic review on interviewing methods for medical school selection by Lin et al. underscores the significance of distinguishing between unstructured and structured interviews.

Unstructured interviews involve flexible content, with no predetermined questions, and lack a standardized evaluation process or specific scoring system.

Structured interviews involve standardized procedures, including predefined questions and established scoring rules, which enhance reliability and validity.

As Lin et al., reported in 2022, in the medical school context, it was discovered that unstructured interviews exhibited lower validity and were susceptible to biases influenced by factors like age, gender, or academic achievement. On the other hand, structured interviews, which incorporated standardized questions, demonstrated stronger validity, and were preferred by applicants. This highlights the importance of incorporating structure into the interview process. Refer to the table below for more information on the advantages of structured interviews and the limitations of unstructured interviews, which includes citations from other scholarly literature summarizing these findings.

Literature supporting use of structured interviews:

Structured Interviews Unstructured Interviews
  • Improve inter-rater reliability, validity, and predictive value.2
  • Can reduce race, age, and gender bias and have been shown to increase workforce diversity.3,22,16,17
  • Have been demonstrated to correlate with markers of clinical and test performance in medical trainees.13,15
  • Benefits may be more pronounced in an online format.17
  • Increased subjectivity.12
  • Prone to interviewer bias.8,24
  • Poor inter-rater reliability.6,7
  • Research has not found an association between faculty (unstructured) interview scores and Accreditation Council for Graduate Medical Education milestone ratings.12
  • Provide opportunities for interviewers to ask inappropriate or illegal questions20, which tend to be disproportionately directed towards certain groups.4,18

Back to top

Components of Structure in a Structured Interview

Structured interviews typically incorporate two components of structure: content-related components and evaluation-related components.

Content-related components of structure aim to increase the standardization of interview content. This can be achieved by asking questions that are directly relevant to the job or role being assessed. Additionally, using the same set of questions for all applicants helps ensure consistency and comparability in evaluating their responses.

Evaluation-related components of structure focus on standardizing the evaluation process itself. This may involve using rating scales or scoring rubrics to assess applicant performance consistently across multiple interviewers. Training interviewers on evaluation procedures helps to minimize variability and enhance reliability in the evaluation process.

Refer to Developing Structured Interviews and Sample Structured Interview Guide for more specific guidance.

The table below provides further detail on the content- and evaluation-related components of structure and the effects of each on the interview’s reliability, validity, fairness, and applicant reactions to the interview. As shown in the table, the effects of each component differ. For example, including questions that are job related increases validity, fairness, and positive applicant reactions, whereas limiting probing questions increases validity and fairness, but may lead to less favorable applicant reactions.

When incorporating structure into the interview process, it is essential for residency programs to consider their specific selection goals and operational constraints. Programs have the flexibility to choose the components that align best with their objectives and resources.

Even implementing minor changes to enhance the level of structure can yield improvements in the reliability and validity of interview results, while maintaining positive applicant perceptions and reactions.

The effects of components of structure on reliability, validity, fairness, and applicant reactions:

Content Reliability Validity Fairness Applicant Reactions
Ask questions that are professionally relevant and/or specialty specific N/A Yes Yes Yes
Ask all applicants questions that cover the same topics Yes Yes Yes N/A
Limit probing questions Yes Yes Yes No
Use behavioral or situational questions Yes Yes Yes N/A
Use a longer interview Yes Yes N/A No
Have no access to applicant information before or during interview Yes N/A Yes No
Have applicants not ask any questions Yes N/A N/A No
Evaluation Reliability Validity Fairness Applicant Reactions
Rate each answer or use multiple rating scales Yes Yes N/A N/A
Use defined rating scales Yes Yes Yes N/A
Take detailed notes Yes Yes Yes N/A
Use multiple interviewers Yes Yes Yes No
Use the same interviewers for all applicants Yes N/A No N/A
Have no discussion between interviews No N/A Yes N/A
Train interviewers Yes Yes Yes Yes
Use formulas to create interview total scores Yes Yes Yes N/A

Notes: “Yes” means overall positive effect, “No” means overall negative effect, and “N/A” mean insufficient research on the effect of the enhancement. Reliability refers the extent to which the evaluation process is consistent and candidate responses are evaluated consistently. Validity refers to the accuracy of inferences made from interview scores.

Source: Adapted from Campion et al. (1997) and Levashina et al. (2014).

Back to top

Gaining buy-in for structured interviews

A concern often raised when advocating for the introduction of greater structure in the interview process is the potential lack of support from faculty. However, an encouraging example comes from a residency program where faculty initially expressed doubts about the use of structured interviews for resident selection, fearing that it would impede natural conversational flow. Nonetheless, their apprehensions were pleasantly dispelled by the positive outcomes achieved (refer to the table below for more information).

In one program in which faculty worried that structured interviews for resident selection would be awkward and would eliminate conversational flow, research demonstrated the following:
All 12 members found the structured interview system easy to use and score.
There was a learning curve in how to introduce questions to applicants and probe for behaviors and outcomes.
Faculty reported that important information was obtained by asking about past behaviors that would not have been determined by an unstructured approach.
Applicants said they did not find structured interviews to be more difficult than unstructured interviews and that the interviews did not interfere with faculty interaction.

To increase the probability of similar success stories, the key is to effectively communicate the advantages and significance of structured interviews. For example, research has found that interview scores were predictive of the first-year residents’ end-of-year performance in the areas of professional competency, patient care, and communication skills. Involving faculty and interviewers in the identification of desired competencies and interview questions can foster their engagement and support in the process.

Back to top

Peer-reviewed articles about structured and unstructured interviews

  1. Basco WT Jr, Lancaster CJ, Gilbert GE, Carey ME, Blue AV. Medical school application interview score has limited predictive validity for performance on a fourth year clinical practice examination. Adv Health Sci Educ Theory Pract. 2008;13(2), 151-162. doi: 10.1007/s10459-006-9031-5.
  2. Campion MA, Palmer DK. A review of structure in the selection interview. Pers Psychol. 1997;50:655-702.
  3. Chapman DS, Rowe PM. The impact of videoconference technology, interview structure, and interviewer gender on interviewer evaluations in the employment interview: a field experiment. J Occup Organ Psychol. 2001;74:279-298.
  4. Costa PC, Gardner AK. Strategies to increase diversity in surgical residency. Curr Surg Rep. 2021;9(5):11. doi:10.7759/cureus.25962.
  5. Harasym PH, Woloschuk W, Mandin H, Brundin-Mather R. Reliability and validity of interviewers’ judgments of medical school candidates. Acad Med. 1996;71(1 Suppl):S40-S42.
  6. Henneman A, Haines S. Implementation of a modified multiple mini-interview method to assess non-cognitive qualities during resident candidate interviews. Curr Pharm Teach Learn. 2020;12:585-589.
  7. Kenny S, McInnes M, Singh V. Associations between residency selection strategies and doctor performance: a meta-analysis. Med Educ. 2013;47:790-800.
  8. Kulasegaram K, Baxan V, Giannone E, Latter D, Hanson M. Adapting the admissions interview during COVID-19: a comparison of in-person and video-based interview validity evidence. Acad Med.2022;97(2), 200-206.
  9. Lin JC, Lokhande A, Margo CE, Greenberg PB. Best practices for interviewing applicants for medical school admissions: a systematic review. Perspect Med Edu. 2022;11(5), 239-246.
  10. Lund S, D’Angelo J, D’Angelo AL, Heller S, Stulak J, Rivera M. New heuristics to stratify applicants: predictors of general surgery residency applicant step 1 scores. J Surg Educ. 2022;79:349-354. doi:10.1016/j.jsurg.2021.10.007. Epub 2021 Nov 11. PMID: 34776371.
  11. Lund S, D’Angelo JD, Mohamed, B, Yeh, VJ-H, Stulak J, Rivera M. Simulation as soothsayer: simulated surgical skills MMIs during residency interviews are associated with first year residency performance. J Surg Edu. 2022;79(6), e235-e241.
  12. Ma C, Harris P, Cole A, Jones P, Shulruf B. Selection into medicine using interviews and other measures: much remains to be learned. Issues in Educational Research. 2016;26(4), 623-634.
  13. Marcus-Blank B, Dahlke JA, Braman JP, et al. Predicting performance of first-year residents: correlations between structured interview, licensure exam, and competency scores in a multi-institutional study. Acad Med. 2019;94:378–387.
  14. McCarthy JM, Van Iddekinge CH, Campion MA. Are highly structured job interviews resistant to demographic similarity effects? Pers Psychol. 2010;63:325-359.
  15. McFarland LA, Ryan AM, Sacco JM, Kriska SD. Examination of structured interview ratings across time: the effects of applicant race, rater race, and panel composition. J Manage. 2004;30:435-452.
  16. Otugo O, Alvarez A, Brown I, Landry A. Bias in recruitment: a focus on virtual interviews and holistic review to advance diversity. AEM Edu Train. 2021;5(S1):S135–9.
  17. Patterson F, Knight A, Dowell J, Nicholson S, Cousans F, Cleland J. How effective are selection methods in medical education? a systematic review. Med Edu. 2016;50(1), 36–60.
  18. Resident Match process policy and guidelines. 2022; Available from: https://sauweb.org/match-program/resident-match-process.aspx.
  19. Roberts C, Clark T, Burgess A, Frommer M, Mossman K, Grant M. The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training. BMC Medical Education. 2014;14(1).
  20. Sacco JM, Scheu CR, Ryan AM, Schmitt N. An investigation of race and sex similarity effects in interviews: a multilevel approach to relational demography. J Appl Psychol. 2003;88:852-865.
  21. Sackett PR, Zhang C, Berry CM, Lievens F. Revisiting meta-analytic estimates of validity in personnel selection: addressing systematic overcorrection for restriction of range. J Appl Psychol. 2022;107(11), 2040–2068.
  22. Schaverien MV. Selection for surgical training: an evidence-based review. J Surg Educ. 2016;73:721–729.
  23. Stephenson-Famy A, Houmard BS, Oberoi S, Manyak A, Chiang S, Kim S. Use of the interview in resident candidate selection: a review of the literature. J Grad Med Edu. 2015;7(4), 539–548.