Are USMLE® Score Predictors Accurate?

by Christopher Cimino, MD, FACMI
Vice President, Kaplan Medical

Everyone would like the reassurance of knowing in advance the USMLE® score they will get. Most of you would settle for any kind of score prediction. Surely it should be possible to take a USMLE practice test and get some kind of prediction from that, right?

Two factors make it difficult to calibrate predictive exam scores for any exam. They are:

Let’s first consider the mathematics behind USMLE score predictions: The typical approach would be to convert your score on the practice exam to the same score scale as the official USMLE. This assumes you have the mean and standard deviation of both the practice and official exams, that both exams have a normal distribution, and the people taking the exams are “equivalent.” As you can see, there are a lot of variables.

The means and standard deviations calculated from the two tests aren’t comparable. For example, if you sat for the USMLE Step 1 at two different times, we would predict that the mean score for the first time around would be lower than the mean score for the second time. Therefor, trying to calibrate the lower practice scores to match the higher target exam scores causes an upward shift. In a way, this target score is a built-in prediction that accounts for more studying to account for a higher score.

This type of prediction assumes an “average amount of studying” from a population of students will increase your score, but there is no way to know how much additional studying is added to cause this increase because it is based on an unknown population of students. For instance, if one student spent 3 weeks doing intensive studying and another did twelve weeks of leisurely studying, what does this say about recommending an ideal amount of studying? The expected improvement based on studying can’t be calculated.

The USMLE website publishes their mean scores “in the range of 215 to 235” and the standard deviation is “approximately 20.” Unless score information is collected by actual scores from actual test takers for a complete cohort, the published approximations are not accurate enough. The National Board of Medical Examiners (NBME®) has collects data from testing cohorts, so their predictions are only singly flawed (as described in Factor #1), while anyone other organizations’ predictions are destined to be doubly flawed.

For students who know they have a limited amount of time to study and must take their boards by a certain date, the answer is obvious: Ignore the score predictions. After all, the strategy is to study as much as possible in the time available and a predictive score may only cause more anxiety, however; if you insist on taking one, the best source is in the NBME’s Student Assessment Services (SAS).

For students who are more flexible, the key is to devise a study plan that maximizes your score, not to study until you reach some specific predicted score. After all, if you are shooting for your maximum score, you can’t do better than that, so the predicted score becomes moot.

For more information on building a personalized study plan, please reach out and request a free medical advising session with our experts!