Setback in Medical Licensing Standards
One of the great benefits of standardized testing, is the ability to provide people without a privileged upbringing an opportunity to study at the greatest institutions. US healthcare has lost this ab
Background
Previously, I wrote about how the restrictions enacted within hospitals and medical schools in 2020 have negatively impacted the education & training of medical students and trainees throughout the country.
The reasons for this are many and varied, and for a full discussion please see the prior article:
This article is related to a change in the educational atmosphere that has been more insidious, and culminated in an announcement in February 2020. The proposed changes took effect in January 2022, and will impact the cohort of medical students applying to training programs (residencies) this fall. Since January, Step 1 has changed from a numerical score to a simple pass-fail exam.
We have already seen changes to standardized exams at every level of education:
Granted - each of these changes were implemented for varying alleged reasons, some of which may be valid. Before we discuss why the change to Step 1 is a problem, we should explore what the USMLE Step 1 examination is…
The USMLE Exams
The USMLE are a set of exams that medical trainees take throughout the years to:
Graduate medical school
Enter & complete training, and
Become licensed physicians.
There are 3 Steps: Step 1, Step 2 CK (Clinical Knowledge), Step 2 CS (Clinical Skills), and Step 3.
Step 1 is a test of basic medical sciences, and is taken at the end of Year 2 - just before starting clinical rotations.
Step 2 concerns material learned during the last 2 years of medical school (clinical rotations), and involves a written exam (CK) and a set of simulated patient encounters (CS).
Finally, Step 3 is the final exam taken around the time of internship year, and is also a written clinical knowledge test.
In the transition between the third and fourth year of medical school, students apply for training programs (residency). Their application includes: academic performance in the first 2 years of medical school, evaluations from the clinicians with whom they rotated during the 3rd year, any extracurricular work (such as research & volunteering), letters of recommendation, and licensing exam scores. Because of the timing, all applicants must have their Step 1 score included. Some will even have their Step 2 CK score.
USMLE Step 1
Until very recently, Step 1 was king. The simple reason is that Step 1 was the only universal and standardized score that the entire applicant pool could be differentiated by. Additionally, it is both more challenging and more foundational in the material that it tests. Step 2 CK notoriously looks for answers you may have not learned, simply based on what clinical or geographic setting you've been trained in - yes, it varies. There is a subjectivity to Step 2 CK that is analogous to bias. Apart from Step 1, almost every other aspect of the medical student's application is not standardized:
Academic performance in medical school will vary by institution
Evaluations from clinical rotations are highly subjective and can be a matter of luck
Research & volunteer exposure is highly dependent on the institution’s location and resources
The Step 1 score was the closest thing we had to an objective measure of a student’s knowledge of the medical sciences. Its importance is also reflected in the weight training programs gave this score when ranking their choice of medical students.
Importing True Diversity
Not only did Step 1 provide us with an ability to compare students across universities within the United States, but it also provided a standard by which foreign doctors could be measured if they wanted to train & practice in the United States. This includes American students who went to medical school in the Caribbean or abroad - typically students who underperformed academically early in the life. If the foreign medical graduates (FMGs) wish to undergo training within the US, they must apply for the same training programs.
Step 1 provides FMGs an opportunity to learn and work here. More importantly, it gives us the ability to import knowledge from abroad – akin to true diversity. With the help of Step 1 we can pick from what the world has to offer, whilst making sure that understanding of medical sciences meet our standards.
From my personal experience across several institutions, this method of selecting FMGs has provided us with access to some of the best doctors I have had the pleasure of working with.
Why change Step 1 to Pass/Fail?
To be honest, I don't know for certain.
The publicly stated aim was to stop medical students from spending too much time studying for Step 1. They claim that students were taking time away from their 'clinical studies' to focus on Step 1, so that they could score as high as possible. Additionally, they claim that Step 1 played too high a role in the selection process.
Less assertively, a claim is made surrounding racial equity and all that BS. There have been so many changes in education and medicine with respect to 'diversity, equity & inclusion' it is hard to ignore the possibility that this school of thought has played a role in these changes.
Regardless, the claims are both not convincing, nor realistic.
First, medical students in the US are allotted a set period of time near the end of Year 2 to study for Step 1. It is not a terribly large amount of time. The schools which allot more time toward Step 1 tend to be abroad, simply because FMGs are expected to score exceptionally high to have a hope in hell of landing a US residency spot. Thus, the shift away from numerical scores hurts non-traditional and foreign applicants the most (AKA students who are not from the mainstream path).
Second, the material tested on Step 1 are precisely the material taught during the first 2 years of medical school. If you want students to stay engaged, and perform well on Step 1 - just teach the basic medical sciences effectively. The students who take time away from their normal school curriculum, and study independently, often do so because their instructors are lacking.
Third - yes, Step 1 is one of the strongest single predictors of an invitation for an interview. This is not necessarily a bad thing, as it is the only non-subjective measure by which applicants can be compared. However, combined subjective measures still play a greater role in this decision process - in most places.
Lastly, Step 2 CK retains its numerical scoring system for now. Step 2 CK is also taken during medical school - usually after Year 3. In effect, what will happen now is that students will spend less time studying for Step 1, and more time focused on Step 2 CK.
The kicker? Less time spent on Step 1 material will lead to inferior understanding of the basic sciences underlying Western Medicine. I cannot stress enough how a lack of understanding of basic sciences contributed to the compliance of the medical professionals with the catastrophic policies implemented since 2020.
What Now?
This change was brought to my attention by the program director of the Radiology residency at my institution.
When I asked what will be done since Step 1 is now pass-fail, she simply said:
"Look at Step 2, I guess..." A sentiment echoed by colleagues at other training programs.
Important: What this now means is that medical students in their 3rd year, who are supposed to be on their clinical rotations, will spend more time studying for Step 2 CK.
Which invalidates the stated objective of the FSMB when they decided to make Step 1 pass-fail. Instead of losing time from the classroom, students will now lose time from their clinical rotations to study for the last remaining standardized measure of performance.
Fuckery Afoot
Apart from the usual complaints about Diversity & Equity changes implemented throughout the West...there's something quite strange about the timing of all this. This change was announced in February 2020.
February 2020 marked a time when medical students were severely restricted from clinical exposure. A formative experience for all budding physicians. Additionally, students in the typical classroom setting were also disrupted - illustrated by a shift towards Zoom-based lectures & conferences.
For the last two years we have seen a reduction in classroom attendance & academic performance, with an accompanying rise in mental health issues as well as substance abuse.
At a time when standardized testing would be most informative, and provide critical feedback about the consequences of our policy choices - we have opted to dispense with them.
At a time when the apparent motivation for 'diversity' is at its highest - we have opted to remove the one metric which allowed people from diverse backgrounds to compete in our healthcare training infrastructure.