Home Government Affairs Newsroom Meetings Publications	Shopping Cart Site Map

ACGME Duty Hours Standard Spark Regulatory Debate

What's in a Name? Philanthropists' Donations Giving New Names to Medical Schools

Proposed New Clinical Exam Draws Cost, Efficacy Concern

Leadership Q & A: A Pioneer Looks Back - And Ahead

Managing Editor
Scott Harris
sharris@aamc.org

Staff Writer
Elissa Fuchs
efuchs@aamc.org

Proposed New Clinical Exam Draws Cost, Efficacy Concerns

By Barbara A. Gabriel

The AAMC, American Medical Association (AMA), and student groups are concerned about the cost, content, and efficacy of a new exam designed to test the clinical skills of prospective physicians. They are responding to a new component of the United States Medical Licensing Exam (USMLE), currently in the final stages of assessment, which will aim to objectively measure the ability of medical students to deal with patients in clinical settings. It is specifically designed to gauge students' ability to take patient histories, perform physical examinations, and communicate effectively with patients and colleagues.

The AMA has asked for a suspension of the new exam, citing a lack of evidence of the examination's efficacy and concern over the estimated costs and logistics of its administration. The AAMC's Executive Council passed a resolution June 20 emphasizing the importance of not adding the estimated $1,000 cost of the exam to medical students' average debt load of $100,000. Medical student groups have also expressed doubt over the exam's capacity to objectively evaluate clinical competencies. Mirroring a typical day The new exam - in which students will examine 10 standardized patients in one day - is designed to mirror a physician's typical workday. To be administered at the end of the third or beginning of the fourth year of medical school, the exam is set to be introduced as part of the USMLE in mid-2004.

The groups assessing the exam's new component are the National Board of Medical Examiners (NBME), the Federation of State Medical Boards (FSMB), and the Educational Commission for Foreign Medical Graduates (ECFMG). The NBME has been studying the feasibility of such an exam for the past 15 years to ensure the test's objectivity and reliability, and has administered prototype exams to thousands of students at more than 40 medical schools.

From 1988 to 2001, Henry Pohl, M.D., vice dean for academic administration at Albany Medical College, was chair of the NBME's prototype subcommittee responsible for developing the principles supporting the clinical skills exam. Dr. Pohl says "many versions" of the exam have been pilot-tested over the past eight to nine years, and he believes the current version has been refined to the point that it will be able to identify those few students who lack the necessary clinical skills to practice medicine effectively.

Dr. Pohl points out that the ECFMG and the Medical Council of Canada, both of which already require students to take a similar exam, have proven effective in identifying a minority of students with deficiencies in the clinical skills arena. "By requiring students to pass an exam such as this," he says, "we are making sure that those people with weaknesses in examining or communicating with patients address them before they go into practice."

AMA calls for suspension

The AMA has outlined some serious concerns regarding the examination. Frank Simon, M.D., director of the AMA's Division of Undergraduate and Graduate Medical Education Policy and Standards, says the group has called on its members to contact the NBME and FSMB to request a suspension of the exam's implementation.

"The AMA endorses such a suspension until three standards have been met," says Dr. Simon. "First, that the exam be demonstrated to be valid in that it identifies people who would pass the USMLE but run into difficulty in the safe practice of medicine; second, that studies supporting that validity be published in peer-reviewed journals; and third, that the testing sites are available in more reasonable geographic locations than are currently proposed by the NBME."

Dr. Simon says the exam's "internal reliability" - which he believes to be proven by the NBME's extensive pilot-testing - is not at issue. "The issue for the AMA is the validity of the test in terms of its ability to identify students graduating from LCME-accredited medical schools who would run into difficulty in the practice of medicine," he explains. "That's a different question."

Students concerned

Michael Rosenthal, an M.D./Ph.D. candidate at the University of North Carolina at Chapel Hill School of Medicine and chair of the AAMC Organization of Student Representatives, concurs. "Everyone can agree on the importance of having extremely competent physicians in all aspects of medicine," Rosenthal says.

"My concern with the NBME lies in whether this is the right way to evaluate those skills. I personally don't feel as though they're really to the point of knowing enough about this exam that we can say, for example, the people who fail this test would have been bad doctors later on. It makes me nervous that theyare proceeding with the exam given that they really don't know the answer to that question," he adds.

The NBME estimates that 5 to 7 percent of U.S. medical students will not pass the exam on their first attempt, and that ultimately 1 to 2 percent - between 250 and 500 students - will be prevented from receiving medical licensure every year due to failure to pass the exam even after repeated attempts.

Rosenthal believes that the current method of "two years of direct observation by physicians with real patients" is a more reliable way to determine whether a medical student is competent in conducting physical exams and communicating with patients. "That approach is going to do a whole lot more for students and for building medical education as a whole rather than trying to 'outsource' medical education to external organizations like the NBME," he says.

Exam sites raise costs, concerns

The NBME says that in order to control the exam's standardization, its administration will be limited to five to seven testing sites around the country. The AAMC has voiced concern over the costs students would incur travelling to and from the test sites, compounding the high cost of the exam itself.

The AAMC's June 20 resolution urged the NBME to work with the FSMB to assure an external source of funding for the exam and to defer its implementation until such a funding mechanism is identified. The AAMC also suggested that the NBME and FSMB spread the cost of the exam across the entire community of practicing physicians through relatively small increments in licensing and re-licensing fees.

Rosenthal points out that if the exam is inevitable, such a funding mechanism would raise interest in the examination and licensure of medical students among practicing physicians.

"If they are going to be paying for it on a regular basis, they will be more likely to pay attention," he notes. "And I think that's a good model not only for the licensing exams but probably for medical education and medicine as a whole."