Now that a major, 17-year-old medical licensing exam is gone, educators and students are feeling their way forward.
The daylong, in-person Step 2 Clinical Skills (CS) test — designed to assess aspiring doctors’ communication and physical exam techniques — was put on hold last March in response to the pandemic. Then, in a shift that shocked many observers, the exam’s sponsors — the Federation of State Medical Boards and the National Board of Medical Examiners (NBME) — announced on Jan. 26 that they were no longer exploring how to revive it.
The fact that this segment of the three-part United States Medical Licensing Examination (USMLE) is no longer has elicited varied responses, from utter dismay to sheer delight.
“Most students are completely overjoyed,” says Alex Lindqwister, past board chair of the AAMC’s Organization of Student Representatives. “They saw it as a source of stress with little actual value.”
But many educators mourn the loss. “We need to be able to say that our students are qualified in these incredibly foundational skills,” notes Toshi Uchida, MD, medical director of the Clinical Education Center at Northwestern University Feinberg School of Medicine in Chicago. “We’re not going to have that nationwide measure now.”
“Most students are completely overjoyed.”
Alex Lindqwister
Past board chair of the AAMC’s Organization of Student Representatives
In response, leaders in academic medicine are looking for new ways to assess the skills the test covered: taking a patient’s medical history, performing a physical exam, formulating possible diagnoses, and communicating effectively with patients and providers.
Meanwhile, what comes next remains unclear.
“We have an obligation to the public to make sure that critical skills are appropriately and uniformly assessed for basic competence,” says AAMC Chief Medical Education Officer Alison Whelan, MD. “I don’t yet know the best way to do that, but we have a duty to figure it out. Still, I’m confident the medical education community — schools, licensing and accreditation bodies, and learners — will tackle this thoughtfully.”
How did we get here?
Taking the USMLE Step exams is a major rite of passage, and all three exams are required for medical licensure in the United States. U.S. medical students usually tackle the daylong Step 1 exam, which covers the basics of medical practice and principles, at the end of their second year or during their third year. Step 2 Clinical Knowledge, also a daylong test, assesses applying medical knowledge and skills to patient care. Most students take that exam — as well as Step 2 CS — during their third or fourth year. Step 3, a two-day undertaking that evaluates one’s ability to practice medicine without supervision, usually gets handled during residency.
Almost as soon as it was created in 2004, Step 2 CS began drawing criticism.
High on the list of complaints was the test’s price tag, most recently set at $1,300. “The cost of the test alone could be your entire monthly budget as a medical student,” notes Zach St. Clair, a fourth-year student at University of Cincinnati (UC) College of Medicine. Some students also invested in fairly pricey study materials, he adds.
Other complaints involved some of the exam’s structure — which required working through several encounters with laypersons trained to portray patients — including that aspects of it felt artificial.
The number of test sites also drew fire: just six across the whole country. That meant many students were adding airfare and hotel to their test-related tabs — in addition to possibly missing three days of school.
And critics felt that all the effort ultimately offered little value given that the test was pass/fail and so many examinees passed on their first try. In 2019, for example, 95% of test-takers from U.S. and Canadian MD-granting schools succeeded on their first attempt.
When COVID-19 made the in-person exam impossible, the USMLE took the opportunity to explore revamping the test, which approximately 35,000 examinees took last year.
“We specifically stated that our goal was to deliver an exam that was appreciably better. … Now, we need to consider innovative alternatives.”
Chris Feddock, MD
Executive director of the Clinical Skills Evaluation Collaboration
As they began to tackle concerns, USMLE teams hosted focus groups, launched surveys, and conferred with a range of stakeholders from educators to examinees.
One option that the USMLE pursued but ultimately rejected was an online test. For one, going online would erase the physical exam component. There were also potential Wi-Fi connectivity issues, remote proctoring logistics, and difficulties expanding exam access broadly and fairly. Plus, it wouldn’t even lower exam costs much, partly due to the expense of building and maintaining a secure online platform.
“We specifically stated that our goal was to deliver an exam that was appreciably better,” says Chris Feddock, MD, executive director of the Clinical Skills Evaluation Collaboration, a collaboration between the NBME and the Educational Commission for Foreign Medical Graduates (ECFMG), which certifies international medical graduates for training in the United States. “There were definitely barriers in moving to a virtual platform in being able to demonstrate that the next version would be appreciably better. Now, we need to consider innovative alternatives.”
Are there downsides for students?
While many students are thrilled that the travel and expense of the exam are gone, not all will celebrate its removal, experts say.
In that group are those who didn’t pass and now won’t have the chance to retake the test. At this point, the USMLE is reporting failures on test transcripts but adding a note to explain these unusual circumstances, says Feddock.
Perhaps most affected are international medical graduates (IMGs), who relied on the test to prove they can make it in the U.S. health care system. This population is no small slice of trainees: Nearly 1 in 4 residents and fellows are IMGs, according to ECFMG President and CEO William Pinsky, MD.
That’s why the ECFMG quickly built alternative pathways to certification last spring. Requirements include passing the Occupational English Test — a patient communication and English language exam created specifically for doctors. Although some IMG students have called it unfair, the ECFMG says the test works well.
Meanwhile, the ECFMG has certified 2,600 applicants using its new approach. “We have an obligation to provide U.S. residency programs with a pool of IMG applicants that is ample, diverse, and highly qualified,” says Pinsky. “We feel these new pathways have been successful in ensuring that.”
The fallout for residency applications and medical education
Step 2 CS was created as a tool for licensing doctors, and state licensing boards are filling the gap in their own ways, according to Feddock. But medical educators have also come to rely on the test as they train tens of thousands of learners each year.
For residency program directors, the exam’s elimination isn’t terribly impactful given its high first-try pass rate. Still, for programs that receive 100 applications per slot, the test served as one bar, however low. And directors may bemoan the loss more starting in 2022, when another component of the USMLE series, Step 1, switches from a numeric score to pass/fail.
“Each year, it seems we get more applications and less information,” says Jessica Kovach, MD, director of the psychiatry residency program at Lewis Katz School of Medicine at Temple University in Philadelphia. “Some schools have even gone pass/fail for their course grades in recent years. I hope the education community will now come together to find a standardized way for medical schools to provide information that programs need to better evaluate candidates.”
“We’ll have to take back the responsibility for a high-stakes assessment. … As we do that, we have to be careful because schools can suffer from the phenomenon of failure to fail their own students.”
Rachel Yudkowsky, MD
Director of graduate studies at the University of Illinois College of Medicine
Meanwhile, medical schools are grappling with their own CS-related issues.
Most concerning to some experts is the potential impact on curricula. “Assessment drives learning,” argues Northwestern’s Uchida, who also serves as the president of Directors of Clinical Skills Courses, an international consortium of educators. “Having a high-stakes, rigorous licensing exam that’s required for everyone really pushed medical schools to increase their focus on clinical skills.” Now, she worries that some schools will cut back on the costly endeavor.
Another issue is whether schools that relied on Step 2 CS may need to ramp up their own assessments to ensure that students have the requisite skills to graduate, says Rachel Yudkowsky, MD, director of graduate studies in the Department of Medical Education at the University of Illinois College of Medicine in Chicago.
“We’ll have to take back the responsibility for a high-stakes assessment, and we have to make sure that it’s valid and reliable,” she says. “As we do that, we have to be careful because schools can suffer from the phenomenon of failure to fail their own students.”
She highlights one positive of the discontinuation of Step 2 CS: the flexibility to assess a wide variety of students’ clinical skills, rather than mirroring the test’s format.
“We can now explore all kinds of other skills,” Yudkowsky says. “Can students get informed consent, counsel someone for smoking cessation, handle end-of-life issues with a patient?” she says. “This move has really freed us up to experiment.”
What lies ahead?
Although Step 2 CS is no more, USMLE leaders are contemplating other approaches to assessing clinical skills. Students and educators are also mulling over what they think might work.
At the USMLE, the next step is to spend several months sorting through priorities and stakeholder feedback — as well as insights garnered from its aborted attempt to shift the test online.
“We have no plans at this time to replace the prior exam with another stand-alone, full-day exam,” says Feddock. “We don’t have any preconceived notions of the most ideal direction,” he adds.
Meanwhile, some argue that schools should assume the entire responsibility for assessing students’ clinical skills.
“I think it should fall to schools to ensure that our abilities are sufficient for graduation,” says Robbie Daulton, a fourth-year UC student. “We have a strong accreditation process that confirms the quality of medical schools. If accreditation includes a rigorous assessment of schools’ ability to assess clinical skills, then there's no need for a national clinical exam.” If schools do take over this role, experts suggest they could do so with a recognized rubric created with the input of faculty, USMLE leaders, and other experts.
Whatever lies ahead, students look forward to contributing to discussions.
“Now is a great time for students to be involved in pushing the conversation forward,” says Joseph Geraghty, a seventh-year MD-PhD student at the University of Illinois College of Medicine and the AAMC’s student liaison to the NBME. “It’s a great time to share bold, innovative ideas about how to improve clinical skills assessment.”
