Best Practices for Conducting Residency Program Interviews

On This Page

• Introduction
• Purpose of This Guide
• Section 1: Structuring Your Interview Process—A Program Director’s Guide
  • Unstructured and Structured Interviews
  • Behavioral and Situational Questions
  • Job-Related Interview Content
  • Evaluating Interview Responses
• Section 2: The Resident Applicant Interview—A Practical Guide for Faculty
  • Stage 1: Preparing for the Interview
  • Stage 2: Starting the Interview
  • Stage 3: Conducting the Interview
  • Stage 4: Closing the Interview
  • Stage 5: Evaluating the Interview
• Interview Do’s and Don’ts
• References

Introduction

The AAMC is focusing on enhancing the transition to residency for all stakeholders, including residency programs, applicants, medical educators, and student advisors. The AAMC survey of program directors conducted as part of this effort indicated that they evaluate a wide variety of information about applicants—including experiences, academic metrics, and other attributes and competencies—when creating rank order lists. The same survey found that characteristics such as professionalism, integrity, interpersonal and communication skills, and reliability and dependability are among the most important ones.

Program directors also reported that the residency interview is one of the most common and important tools used in ranking applicants but that significant variability exists in how applicants are interviewed across programs and specialties. While many resources are available to help applicants prepare for interviews (for example, AAMC Careers in Medicine and the American College of Physicians Guidelines for the Residency Interview Process), fewer resources exist for interviewers. Additionally, though physicians receive considerable training on how to interview patients, interviewing applicants is different in purpose, design, and implementation.

When this gap was identified, the AAMC sought to summarize best practices and assemble this guide to help program directors quickly understand their options for enhancing the interview process. Our hope is that the guide will serve as a helpful and useful introduction to interview practices and processes that can be used to improve consistency and better predict resident performance.

Return to top ↑

Purpose of This Guide

This guide describes interview best practices to assist with implementing valid and fair interview processes. Any enhancements in the interview process, including components of interview structure and interviewer training, will enable programs to make more informed decisions that contribute to the increased likelihood of success and compatibility for both the programs and the applicants.

The information in this guide is divided into two sections:

• Section 1: Structuring Your Interview Process—A Program Director’s Guide provides an overview of the current state of research and best practices in selection interviews.

• Section 2: The Resident Applicant Interview—A Practical Guide for Faculty provides practical information on conducting interviews, including helpful tips and traps to avoid.

While residency programs use many sources of information to evaluate and rank applicants, this guide focuses specifically on interviews. It does not include information about developing and integrating other common assessments (for example, work samples and OSCEs) into the selection process.

On a related note, residency programs use interview days for multiple purposes. In addition to the formal interviews, common interview day activities include recruitment activities such as meals with current residents and faculty, question and answer sessions, and campus or city tours. These more informal activities and interactions with house and program support staff yield valuable information about applicants’ interpersonal and communication skills and play an important role in helping programs—and applicants—determine whether there is a good fit between the applicant and the program. While there are good practices for planning informative, effective, and enjoyable interview days—such as doing advance work to ensure everyone involved shares a common understanding of what the program seeks in applicants and implementing feedback mechanisms that enable everyone involved to share relevant information—this guide focuses specifically on best practices for the formal interview component of the day.

Finally, many institutions provide interviewer resources and guidance that complement the material in this guide. Program directors and interviewers are encouraged to explore those resources. Before modifying the interview process, the designated institutional official, program director, and legal counsel should be consulted about the institution’s and the program’s interview policies and requirements.

Return to top ↑

Section 1: Structuring Your Interview Process—A Program Director’s Guide

This section provides an overview of interview structure and the current state of interview research.

Interviews are a valuable way to assess whether an applicant demonstrates the personal characteristics that are critical for success as a resident in a specific program and fit within the program’s mission and goals. For example, the missions and goals of a residency program or institution may include a desire for residents to focus on research, urban or rural practice, underserved communities, or academic or community leadership.

Selection interviews may vary on a number of dimensions and may:

• be structured or unstructured,
• use behavioral or situational questions,
• assess job-related content, including technical (medical and clinical knowledge and procedural skills) and nontechnical (interpersonal skills, professionalism) topics, and
• use rating scales to evaluate responses.

Unstructured and Structured Interviews

Unstructured interviews are characterized by discretionary content (that is, no preselected questions) and an unstandardized evaluation process (that is, no question-specific scoring system).

Structured interviews are characterized by any enhancement of the interview designed to improve reliability and validity by increasing standardization. Research on employment interviews has identified two categories of components of structure: those that influence interview content and those that influence the evaluation process.

• Content-related components of structure are any enhancements that increase the standardization of the interview content, such as asking questions that are job-related and asking the same questions to all applicants.
• Evaluation-related components of structure are any enhancements that increase standardization of the evaluation process, such as using rating scales to evaluate the interview and training interviewers on evaluation procedures.

Table 1 lists the content- and evaluation-related components of structure and the effects of each on the interview’s reliability, validity, and fairness and the applicant reactions to the interview. As shown in the figure, the effects of each component differ. For example, including questions that are job-related increases validity, fairness, and positive applicant reactions, whereas limiting probing questions increases validity and fairness, but may lead to negative applicant reactions.

When thinking about how to introduce structure into the interview process, programs should select the components of structure that best match its selection goals and operational constraints. Not all components of structure need to be implemented in order to improve interview results. Even making modest increases in structure can have a positive effect on the reliability and validity of interview results while maintaining positive reactions from applicants.

Table 1. The Effects of Components of Structure on Reliability, Validity, Fairness, and Applicant Reactions

Note: “positive” means overall positive effect, “negative” means overall negative effect, and "insufficient" means insufficient research on the effect of the enhancement. "Reliability" refers the extent to which the evaluation process is consistent and candidate responses are evaluated consistently. "Validity" refers to the accuracy of inferences made from interview scores.

Source: Adapted from Campion et al. (1997) and Levashina et al. (2014).

Behavioral and Situational Questions

There are many types of interview questions. Two of them—behavioral and situational questions—have been widely studied and are considered relatively structured.

Behavioral questions are based on the premise that past behavior predicts future behavior. They ask applicants to describe what they did in a previous context (typically, in previous jobs, at school, or in volunteer experiences) that are related to situations they may face in the job for which they are interviewing. Past-behavior questions often ask an applicant to describe a specific situation, the behavior or action they took, and the outcome or consequence of that behavior.

Example: Please describe a time when you observed a member of the medical team that you were working with behave in a manner that was inconsistent with an established protocol. Explain what the situation was, what actions you took, and the outcome.

Situational questions are based on the premise that intentions predict future behavior. They pose hypothetical situations that might occur on the job and ask applicants to describe how they would respond in the situations.

Example: I’d like you to imagine that you are on your morning rounds. The chief resident describes a difficult case that you and another PGY-1 worked on earlier in the week and compliments your handling of a difficult situation. In doing so, she gives you sole credit and fails to mention that your colleague played a major role. What would you do?

Research on the type of interview questions suggests that both behavioral and situational questions have strong psychometric properties; however, they may be measuring slightly different constructs. Behavioral questions may primarily measure experiences and some personality traits, while situational questions may measure job knowledge. Both types of questions are reliable and are valid predictors of future job performance, with behavioral questions having slightly higher validity. Some research suggests that past-behavior questions may be slightly more resistant to faking and have slightly lower group differences than situational questions.

Job-Related Interview Content

Regardless of question type, all interview questions should be job-related. That is, all questions included in the selection interview should be clearly linked to key requirements of the PGY-1 position. To the extent that alignment between an applicant’s learning and work styles and the program’s culture is essential to success in your program, including questions to assess if the fit is appropriate and job-related. Interview questions should require applicants to elaborate on specific examples that address those key requirements and/or fit. If possible, identify the key requirements of the PGY-1 position and the competencies necessary to perform the work and learn new skills. Competencies that are identified as critical for success at entry are good targets for the interview because they are required to perform work on day 1 and are more likely to predict applicants’ future performance in the program.

If you do not have the resources required to conduct a thorough analysis of the key requirements of the PGY-1 position, consider using your program’s PGY-1 performance evaluation tools (for example, milestones) and talking to other faculty about what competencies are critical for success when PGY-1s enter your program (and whose lack would lead to failure). Another option is to use the Accreditation Council on Graduate Medical Education (ACGME) Core Competencies, the AAMC Core Entrustable Professional Activities for Entering Residency, or your specialty organization’s milestones as a starting point for thinking about what content to include in the interview. If the milestones or other models are used as a starting point for your selection interview, remember that they are broad descriptions of residency performance. It is important to consider which aspects are relevant to the program’s PGY-1 position. The selection interview should only target content that trainees should be expected to demonstrate on day 1, not content that they will be expected to learn during training.

As mentioned in the introduction, the interview day may have several components besides the applicant interview. It is important to think about the purpose of each component of the interview day and what information needs to be gathered in each part. Using interviews to recruit applicants can distract from assessing an applicant’s preparedness for your program. If possible, dedicate separate time for recruiting and answering applicants’ questions during the interview day. This may help keep the interview focused on job-related content and prevent ancillary information from influencing interviewer ratings.

Evaluating Interview Responses

A best practice is to use rating scales to evaluate applicants’ responses. Incorporating rating scales into the interview typically enhances reliability, validity, and fairness of interview scores without causing negative applicant reactions. It will also increase interviewers’ ability to compare applicants because they were evaluated on a common scale.

There are five key points to note about rating scales:

• They can be designed to evaluate applicants’ responses to individual questions, a small number of competencies, or overall suitability for the job. A best practice is to develop rating scales for the small number of competencies that the interview was designed to assess.

• They can be developed to work with behavioral and situational questions.

• The number of points on the rating scale often varies from three to seven. Deciding how many points there should be on the rating scale typically depends on the number of proficiency levels (or scale points) you think reflect the range of behaviors observed among PGY-1s.

• Ideally, each point on the rating scale is anchored with behavioral examples that describe each level of proficiency. The behavioral examples on the rating scales should reflect faculty expectations of each level of performance for PGY-1s, providing raters with common definitions for each point on the scale. This will both make the rating task easier for raters and help ensure that applicants are being evaluated in a consistent manner.

• Interviewers should be instructed to use the behavioral examples on the rating scale as a general guide for evaluating applicants’ responses.

Return to top ↑

Section 2: The Resident Applicant Interview—A Practical Guide for Faculty

This section provides general guidance on best practices for the stages of the residency applicant interview process:

1. Preparing for the interview
2. Starting the interview
3. Conducting the interview
4. Closing the interview
5. Evaluating the interview

Prospective faculty and resident interviewers should consider taking the institution’s interviewer training course (if available), participating in institution or national training programs such as a course on unconscious bias, and familiarizing themselves with the job requirements for PGY-1 residents and the mission and the goals of the institution and the residency program.

Stage 1: Preparing for the Interview

Before meeting the applicant, it is important that interviewers familiarize themselves with the interview materials (for example, the interview script, possible or required interview questions, competency definitions, and descriptions of rating scales). The following should be readily available:

• A list of questions or topics to discuss
• A way to take notes during the interview
• The scoring rubric or rating scale(s), if applicable
• The interview schedule

Stage 2: Starting the Interview

Create a comfortable atmosphere. To create an open and relaxed atmosphere that will encourage the applicant to share information:

• Welcome the person in a friendly manner.
• Introduce yourself, giving your name and title.
• Tell the applicant how long the interview will take.
• If you plan on taking notes during the interview, tell the applicant before you begin the interview. You can explain that taking notes helps to ensure that you remember responses accurately.
• If you know that you might be interrupted during the interview by a call or urgent matter, tell the applicant about that possibility before you begin the interview.

Avoid the influence of first impressions or “gut” instinct. No matter what the applicant's personal appearance is or the "chemistry" between you and the applicant, remember that these do not predict how well a person is likely to do as a resident in your program. Relying on a first impression may limit the quality and amount of job-related information you gather during an interview; it is important to keep your focus on listening to the applicant’s answers and to be as thorough as possible with all applicants.

Stage 3. Conducting the Interview

Ask job-relevant questions. It is a best practice for interview questions to be clearly linked to job requirements. If there is a set of predetermined questions or topics from which questions should be asked, it is important to adhere to those questions or topics for consistency across interviews. If interviewers can ask their own questions, make sure they are relevant to the job.

When possible, use situational and behavioral questions. Both behavioral and situational questions improve interview structure and have strong psychometric properties. Behavioral questions ask applicants to describe a specific situation, the behavior or action they took, and the outcome or consequence of that behavior. Situational questions pose hypothetical situations that may occur on the job and ask applicants to describe how they would respond in the situation.

Avoid inappropriate questions and always check with your institution’s legal counsel about inquiries that may be prohibited by law or employer policy. The following topics should be avoided during an interview:

• Demographics: Age, race, religion, socioeconomic status, ethnicity, sexual orientation, gender identity, national origin

• Family: Lineage, ancestry, primary or native language, marital status, maiden name or family surname, relationships or people the applicant lives with, family issues (parental status, age of dependents, plans for children)

• Personal: Height and weight, physical and mental disabilities, physical appearance, personal activities that probe for personal affiliations

• History: Military discharge, arrests, criminal convictions

• Other programs or specialties, and ranking plans: Information about other programs or 

  specialties to which they might be applying and/or how the applicant plans to rank your program. (NRMP policies state that programs cannot require any applicant to disclose information about where they have applied and how they plan to rank programs.)

Ask job-related probing questions. In many cases, applicants do not provide enough information in their initial response, so the interviewer may need to prompt or ask follow-up questions. Asking a follow-up question such as, “Could you be more specific?” or “Could you tell me more about that?” is helpful in gathering as much information as possible without leading the applicant to an answer. If probing questions are used, they should be used consistently with all applicants who provide an incomplete initial response to ensure that everyone has the same opportunity to explain a response. Be aware that asking too many probing questions provides a cue to applicants about the types of answers you are looking for and may increase the likelihood of faking a response.

If using situational or behavioral interview questions, the STAR acronym can help interviewers ensure they gather all important information about each question:

• Situation or Task: Did the applicant describe the context for the event being discussed?

• Action: Did the applicant describe the exact behaviors or actions taken (or what would 

  be done)?

• Result: Did the applicant describe the outcomes or consequences of the behaviors or 

  actions?

Probes for Situational Interview Questions

Situation or Task

• What do you consider the most critical issue in this situation?
• What other issues are of concern?

Action

• What would you say?
• What is the first thing you would do?
• What factors would affect your course of action?
• What other actions could you take?

Results

• How do you think your action would be received?
• What would you do if your action was not received well?
• What do you consider benefits of your action?

Probes for Behavioral Interview Questions

Situation or Task

• What factors led up to the situation or task?
• Could you or anyone else have done something to prevent the situation or task?
• What did you determine as the most critical issue to address in this situation or task?

Action

• How did you respond?
• What was the most important factor you considered in taking action?
• What is the first thing you did?

Results

• What was the outcome?
• Is there anything you would have said and/or done differently?
• Were there any benefits from the situation?

If the program does not use situational or behavioral interview questions, more generic and open-ended probing questions can be used.

Open-ended probing questions: Ways to ask an applicant to elaborate on a response:

• Tell me more about that.
• What happened?
• Why is that?
• How did you react?
• How did that come about?
• Help me understand that better.
• Please go on.
• Explain that to me.
• Could you be more specific?
• How come?
• Give me another example, please.

Take short notes. Notes allow you to base your evaluations on all available information and assist you in making the most objective evaluations possible. Your notes should provide sufficient information about the interview content to justify how you evaluate candidates. Focus on recording relevant key words or phrases in the applicant’s responses rather than trying to capture responses verbatim.

Some important traps to avoid when taking notes include:

• Don't make inferences about what the applicant meant or write down your opinions of 

  what the applicant said. For example, do not write “bad answer” or “great problem solver” as these are evaluative statements that provide no factual evidence of the applicant’s response.

• Don't include any reference to race, color, religion, sex, national origin, age, or other factors not related to the stations or rating scales, even if the applicant mentions these things.

• Don’t allow your note taking to be a signal to applicants about the quality of their responses. For example, do not take notes only when something very positive or very negative has just been said.

Stage 4: Closing the Interview

At the end of the interview, thank the applicant and explain where to go next. Be careful not to give the applicants any feedback on their interview performance or share your notes.

Stage 5: Evaluating the Interview

As soon as possible after the applicant leaves the room, review your notes. Fill in any important details you may have missed. Evaluate the applicant using the approach designed by your program—ideally, before the next interview begins. Ratings of the applicant should be supported by the notes.

Be aware of your unconscious bias. Everyone holds unconscious biases about other people or groups of people based on attitudes, associations, and stereotypes. Interviewers can help mitigate their individual biases through:

• awareness of strong reactions for or against a particular applicant or type of applicant
• basing scores on deliberate thinking and decision-making rather than on first impressions
• perspective taking

Increasing standardization of interview content and evaluation is likely to reduce the impact of unconscious bias on the interview process. For example, the impact of unconscious bias can be mitigated in part through enhancements to the interview process such as clearly defining criteria, using a scoring rubric, having a diverse pool of interviewers, and training interviewers on proper interview technique.

For more information, please see the AAMC’s Unconscious Bias Resources for Health Professionals.

Be aware of common rating errors. Although often unintentional, common rating errors can decrease the validity and fairness of interviews. Here are some of the most common types of rating errors to be mindful of when rating each applicant’s interview responses:

• Halo/Horns effect: Allowing ratings of performance based on one response to influence ratings for another response. For example, allowing a rating on a question assessing teamwork to influence the rating on a question assessing motivation.

• Central tendency: Rating all applicants in the middle of the rating scale (for example, giving all 3s in a 5-point rating scale). Interviewers should feel comfortable using the entire range of the rating scale.

• Leniency/severity: Giving high or low ratings to all applicants, irrespective of their actual responses.

• Contrast effects: Comparing one applicant with the performance of previously interviewed applicants. The order in which the applicants are interviewed can affect the ratings they are given. While making ratings, interviewers should refrain from comparing applicants. Instead, interviewers should focus on evaluating each applicant’s response in relation to the rating scale.

Providing interviewers with regular training and feedback on their scoring relative to program standards and other interviews can help mitigate these errors.

Return to top ↑

Interview Do’s and Don’ts

Conducting Interviews

Do:

• Ask job-related questions.

• Use positive body language such as smiling and nodding occasionally.

• Refocus the applicant if he or she goes off track, by making a brief comment about the applicant’s remarks (such as, “OK”) and then firmly move back to the original question.

• Spend more time listening than talking.

Don't:

• Use negative body language such as raising an eyebrow, frowning, or using a harsh tone of voice.

• Give feedback to the applicant about his or her performance during the interview (such as, “Good” or “Great”).

• Ask judgmental, why, leading, or yes/no questions.

Evaluating Interviews

Do:

• Stay objective―focus on facts, not opinions.

• Focus on the applicant’s responses to interview questions.

• Focus on one question or dimension at a time.

• Focus on comparing applicants’ responses with scale anchors (if your program uses a rating scale).

Don't:

• “Fill in” parts of the answer based on your own interpretations of the applicant’s response.

• Judge an applicant based on anything outside the scoring rubric (for example, personal appearance or your “chemistry”).

• Compare responses of one applicant with those of other applicants during the interview.

Return to top ↑

References

Banaji MR, Greenwald .G. Blindspot: The Hidden Biases of Good People. New York: Delacourte Press, 2013.

Campion MA, Palmer DK, Campion JE. A review of structure in the selection interview. Pers Psychol. 1997;50:655-702.

Easdown LJ, Castro PL, Shinkle EP, Small L, Algren J. The behavioral interview, a method to evaluate ACGME competencies in resident selection: a pilot project. J Educ Perioper Med. 2005;7(1):E032.

Hartwell CJ, Campion MA. Getting on the same page: The effect of normative feedback interventions on structured interview ratings. J Appl Psychol. 2016;101(6):757-778.

Huffcutt AI, Roth PL. Racial group differences in employment interview evaluations. J Appl Psychol. 1998;83(2):179-189.

Kristof-Brown AL, Zimmerman RD, Johnson EC. Consequences of individuals’ fit at work: A meta-analysis of person-job, person-organization, person-group, and person-supervisor fit. Pers Psychol. 2005;58(2):281-342.

Levashina J, Campion MA. Measuring faking in the employment interview: development and validation of an interview faking behavior scale. J Appl Psychol. 2007;92(6):1638-1656.

Levashina J, Hartwell CJ, Morgeson FP, Campion MA. The structured employment interview: Narrative and quantitative review of the research literature. Pers Psychol. 2014;67(1):241-293.

McDaniel MA, Whetzel DL, Schmidt FL, Maurer SD. The validity of employment interviews: A comprehensive review and meta-analysis. J Appl Psychol. 1994;79 599-616.

Patrick LE, Altmaier EM, Kuperman S, Ugolini K. A structured interview for medical school admission, Phase 1: Initial procedures and results. Acad Med. 2001;76(1):66-71.

Roth PL, Van Iddekinge CH, Huffcutt AI, Eidson Jr CE, Bobko P. Corrections for range restriction in structured interview ethnic group differences: The values may be larger than researchers thought. J Appl Psychol. 2002;87(2):369-376.

Santen SA, Davis KR, Brady DW, Hemphill RR. Potentially discriminatory questions during residency interviews: frequency and effects on residents' ranking of programs in the national resident matching program. J Grad Med Educ. 2010;2(3):336-340.

Swider BW, Barrick, MR, Harris TB. Initial impressions: what they are, what they are not, and how they influence structured interview outcomes. J Appl Psychol. 2016;101(5):625-638.

Return to top ↑