This study explored regional and country-to-country variation in performance across basic science disciplines for examinee groups taking USMLE Step 1 for the first time from mid-2008 through mid-2011.

Study results were similar to findings from research done in the mid-1990s. Potential reasons for the sizable country-to-country variation in performance across disciplines include: differences in curricular focus, instructional methods, and emphasis on applying understanding of basic science in a clinical context; differences in the point at which examinee groups choose to take Step 1; facility with the English language; differences in culture and standards of care; relative emphasis on preparing students to take Step 1 and enter graduate training in the United States; and other factors.

A major limitation of the study is the use of a point-in-time, multiple-choice test as a criterion measure: Step1 scores cannot be viewed as a “gold standard” for assessing understanding of basic science. In addition, Step 1 test items are deliberately integrative, with individual items testing multiple disciplines, so assignment of items to discipline subscores is sometimes challenging.

Performance on USMLE Step 1 varies substantially from region to region and country to country, both on the total test and in individual basic science disciplines. More research is needed to understand the sources and implications of the variation in performance.

Percent-correct total scores and discipline subscores for 101,000+ examinees taking Step 1 for the first time during the study period were retrieved for analysis. For each examinee/subscore, we computed the difference between each examinee's subscore and the mean performance of examinees at US schools and then tabulated mean differences by country of medical school for countries with at least 300 examinees during the study period for the regions of interest. Depending upon the subscore, 500-3000 unique items contributed to calculation of subscore means.

Overall Step 1 performance was highly variable across regions and countries, with pass rates ranging from a high of 92% for first-time examinees from US schools to a low of 41% for first-time examinees educated in one of the Caribbean countries; Differences in mean total test percent-correct scores differed by up to 21%. Controlling for overall performance, international medical students and graduates (IMGs) did relatively better in Gross Anatomy (+2%) and Pathology (+2%) and relatively worse in Behavioral Sciences (-3%) and Genetics (-2%) compared with examinees from US schools. IMGs from Asian and Middle Eastern schools showed this pattern more strongly; performance of IMGs from Caribbean schools did not show the pattern.

1. Federation of State Medical Boards of the United States, Inc. and the National Board of Medical Examiners (2012) Step 1 Content Description and General Information, Philadelphia: FSMB and NBME.
2. Swanson DB, Case SM, Ripkey, D, Melnick, D, Bowles, LT, Gary, N (1997) Performance of examinees from foreign schools on the basic science component of United States Medical Licensing Examination, in Scherpbier A, van der Vleuten C, Rethans J, van der Steeg (eds), Advances in Medical Education, Dordrecht, Kluwer Academic Publishers, 187-190.
3. Case SM, Swanson DB, Ripkey DR, Bowles LT, Melnick DE (1997) Preliminary descriptive analyses of the performance of US citizens attending foreign medical schools on USMLE Steps 1 and 2,  in Scherpbier, A, van der Vleuten, C, Rethans, J, van der Steeg (eds), Advances in Medical Education, Dordrecht, Kluwer Academic Publishers, 135-138.