Practice Overview
| Practice Name | Three Rivers Curriculum (3RC) Check-Ins |
| Description | Voluntary feedback sessions for staff involved with curriculum transformation process including mechanisms for escalation to leadership. |
| Participants | Medical education staff (~ 20 staff). Medical education leaders. |
| Frequency | Every other week. |
| Modality | In-person. |
| Resources | Staff time, no financial resources required. |
Introduction
In August 2023, University of Pittsburgh School of Medicine implemented the new Three Rivers Curriculum (3RC), which prompted a change in the staffing structure for the Office of Medical Education (OMED). To support the new team structure during the curriculum transition and ensure effective implementation, Katie Maietta, MPPM, executive director of OMED, instituted a robust feedback and evaluation process. This included biweekly check-in meetings led by Greg Null, MA, director of program evaluation, CQI, and accreditation, and Allison Serra, MD, MPH, faculty director of program evaluation, which are still being implemented today.
These meetings use the After Action Review (AAR) framework reported on by Fletcher, et al. to gather real-time feedback, foster a collaborative problem-solving environment, and create a supportive community among staff.
Goals and Objectives
The goals of the AAR check-ins include:
- Collect real-time feedback from staff related to curricular operations.
- Create a safe space for staff to provide honest reactions.
- Build and fortify community among staff members.
- Create time and space for information-sharing and collaborative problem-solving.
- Develop opportunities for staff to invest in each other's success and in the success of the new curriculum.
Promising Practice Overview
To support the successful implementation of 3RC, the OMED initiated biweekly, hour-long check-ins. Sessions are open to all OMED staff and are promoted through the weekly OMED newsletter. Unlike traditional all-hands-on-deck meetings involving faculty, staff, and leadership, the check-ins are limited to staff — with the exception of a single, nonsupervising faculty member attendee — in order to create a safe space to discuss operational issues that may involve faculty. At each check-in, the facilitator poses the same four questions in reference to the previous two weeks to keep the conversation current and focused:
- What was expected to happen?
- What actually happened?
- What went well and why?
- What can be improved and how?
Although attendance is voluntary, almost all OMED staff (approximately 20 people) regularly attend. The small group size and frequent meetings allow for more open dialogue where staff voice their concerns and questions more effectively. For example, when staff experienced delays in receiving course materials from faculty and difficulties navigating the new learning management system, the check-ins provided a crucial platform for staff to escalate these issues in a safe and collaborative setting. Holding regular meetings every other week also prompts staff to proactively anticipate emerging questions or concerns and to address them more quickly.
The OMED leads compile, de-identify, and share the meeting notes with staff for them to review, add, and clarify what came up during the sessions. Then, the medical education leaders receive the notes to enhance their awareness of the challenges affecting implementation and the opportunities to act. As a result, leaders respond to the staff, sharing what actions have been taken or explanations about why action has not been taken. Finally, a summary of the leaders’ responses is shared at the beginning of the following AAR meeting.
Results and Impact
The AAR structure creates a safe and productive environment for the staff members to reflect and react, as well as to identify actionable solutions for any issues or problems. Staff appreciate the opportunity to provide feedback and learn from each other and feel the program empowers them to share honest reactions and to take on new roles and responsibilities related to process improvement. Many staff members who don’t usually speak up in bigger meetings expressed the AARs provide a space to voice their thoughts. Thus, this practice successfully engages medical school staff in a supportive way that promotes psychological safety and establishes effective problem-solving.
“Staff were definitely sort of a neglected population ... but having the [meeting series] really has been powerful in terms of building a better work environment,” said Serra.
At the end of the academic year in 2024, the AAR process was evaluated using the AAR format to identify strengths and areas for improvement. Staff unanimously agreed to continue the check-ins, showing this style of check-ins brings value to the staff experience during a major organizational transformation.
During the first year of AAR implementation, there was no attrition among OMED staff. This retention is notable as both the curriculum reform and “great resignation” could have proved a challenging time for a newly formed team. As an added benefit, the AAR staff leads noticed the process facilitates the onboarding experience of new team members and supports existing staff with new responsibilities by helping them learn what everyone does.
Resources Needed
The regular 3RC check-ins engage medical school staff during the curriculum implementation and require few resources beyond staff time, securing a meeting room, and identifying the AAR framework.
While the “development [of the AAR meeting series] takes a lot of time …I think we all recognize that the time is worth it because it gives you a better product in the end,” said Maietta.
Challenges and Lessons Learned
While the 3RC check-in is straightforward, the leadership team modified the process over time to better suit their needs. The team provided some insights that may benefit anyone who is interested in replicating some part of this practice at their own institution:
- The AAR format works only when all four questions are asked.
- Establish “priority setting” as a standing agenda item so staff can clearly communicate to school leaders what they believe should be most urgently addressed.
- Use some meeting time for shared problem-solving for issues staff can resolve themselves.
- Be transparent about the AAR process and get commitment from leaders to respond to staff feedback.
Moving Forward
The OMED will continue biweekly AAR meetings and will start incorporating more staff over time. For example, the next iteration of the curriculum rollout will impact the administration of clinical clerkships and future AARs will include clerkship administrators and other clerkship support staff.
Acknowledgements
This case study is based on a July 24, 2024, interview with Katie Maietta, MPPM, executive director of medical education, Greg Null, MA, director of program evaluation, CQI, and accreditation, and Allison Serra, MD, MPH, faculty director of program evaluation at University of Pittsburg School of Medicine.
Medical School Statistics
| Name of Institution | University of Pittsburgh School of Medicine |
| Location | Pittsburgh, Pennsylvania |
| School Size as of 2023: Number of medical students Number of full-time faculty Number of full-time staff |
7021* 2,5612 2,0523 |
| Public/Private | Public |
*Enrollment includes the number of students in medical school, including students on a leave of absence, on Oct. 31 of each year shown. Enrollment does not include students with graduated, dismissed, withdrawn, deceased, never enrolled, completed fifth pathway, did not complete fifth pathway, or degree revoked statuses.
1Source: AAMC. Student Records System. 2023. https://www.aamc.org/data-reports/students-residents/report/student-records-system-srs
2Source: AAMC. U.S. Medical School Faculty, 2023. AAMC; 2024.
3Source: Pitt Worx: Master Employee/Staff Report. University of Pittsburgh; 2023.
References
Fletcher A, Cline, PB, Hoffman M. A better approach to after-action reviews. Harv Bus Rev. Published online Jan. 12, 2023. https://hbr.org/2023/01/a-better-approach-to-after-action-reviews