The impact of a targeted Arab Board of Emergency Medicine examination preparation course on resident success rates

Abstract

Background

The Arab Board of Emergency Medicine (ABEM) examination is a critical milestone for emergency medicine residents (EMRs) aiming to transition to independent practice. Prior to this study, no structured course had been administered to prepare EMRs for the ABEM examination. To address this gap, a five-day evening preparatory course was developed, designed to align with ABEM requirements and deliver updated, essential knowledge. This study aims to evaluate the impact of this newly introduced preparatory course on EMRs’ success rates on the ABEM examination from 2017 to 2021.

Methods

The five-day in-person course was conducted annually in the evenings to accommodate both EMRs and faculty, scheduled approximately six weeks before the ABEM examination. The course content was specifically aligned with ABEM standards, focusing on relevant and contemporary knowledge. EMRs participated in assessments modeled after the ABEM examination, using standardized rubrics. Pass rates were compared to historical data, and demographic variables were thoroughly analyzed. Feedback was collected from both EMRs and faculty after each course to continuously refine and improve course content and delivery.

Results

Data were collected over a five-year period (2017–2021), with 49 emergency medicine residents (EMRs) (100%) participating in the course. The majority of participants were male (69.4%), and the cohort included graduates from diverse regions, including Sudan (22.4%) and Pakistan (18.4%). The overall pass rate for the Arab Board of Emergency Medicine (ABEM) examination significantly increased to 91.8%, compared to a pre-course pass rate of 60%. No significant impact on outcomes was observed based on factors such as gender, country of graduation, year of participation, or initial confidence levels. Feedback from both participants and faculty indicated high levels of satisfaction with the course, with recommendations for further content refinement and the inclusion of additional Objective Structured Clinical Examination (OSCE) stations.

Conclusion

The findings demonstrate the significant impact of the newly introduced, face-to-face, five-day ABEM preparatory course on EMR pass rates, potentially serving as a model for similar programs in other specialties. The results highlight the importance of targeted, up-to-date instruction in improving the confidence and success of EMRs on the ABEM examination. Furthermore, the course’s development and the incorporation of ongoing feedback played key roles in further enhancing the quality and effectiveness of the training program.

Introduction

The Arab Board of Health Specialization, established by the Arab League’s Council in 1978, is a cornerstone of postgraduate medical education in the Middle East North Africa (MENA) region. Its expansion in 2009 to encompass a range of health specialties reflects a strategic response to physician emigration and specialist shortages in the Arab world. Headquartered in Damascus, Syria, the organization sets educational standards and assessments for more than 60 medical programs, aligning them with international norms to provide comprehensive training and regionally recognized certifications [1].

The Arab Board of Emergency Medicine (ABEM), integral to the Arab Board of Health Specialization, is crucial for advancing emergency medical care standards in the MENA region [1]. This program ensures physicians are well trained in emergency medicine, covering a broad range of procedures and scenarios for safe and effective clinical services for acute care. The successful completion of this program indicates high proficiency in emergency medicine, which is recognized throughout Arab nations [1, 2]. Emergency medicine training programs aim to train professionals in various aspects, including leadership, diagnostics, and medical interventions, crucial for emergency department efficacy [3,4,5].

The ABEM final examination is administered annually to emergency medicine residents (EMRs) in the MENA region. It is an expensive examination, with a fee of 426 JD (Jordanian dollars, 600 USD). It evaluates their progress toward attaining the fundamental knowledge necessary to function as independent emergency physicians [1, 5]. The ABEM examination is administered only once per year, however, which limits opportunities for taking the examination or seeking remediation. Although many commercial online board review courses are designed for certification examinations in emergency medicine, there is currently a notable absence of comprehensive face-to-face preparatory courses specifically tailored for ABEM examinations.

Before 2017, no organized or structured preparation course was available for the ABEM examination at the host hospital. During this period, the pass rate for the ABEM examination was approximately 60%, highlighting a clear need for a targeted intervention to improve emergency medicine residents’ (EMRs’) success rates and better equip them for the challenges of the examination. In response to this imperative demand, a specialized face-to-face preparatory course was developed in collaboration with the emergency medicine (EM) program director and core faculty members who possessed the requisite training and successful ABEM certification. This collaborative effort utilized the Delphi technique to establish a consensus on the course curriculum, which was designed to comprehensively prepare EMRs for ABEM examinations. The initial implementation of this course occurred in 2017, with sessions scheduled approximately six weeks prior to the ABEM examination date. As part of the evaluation process, EMRs were asked to retrospectively assess their self-reported confidence levels at the end of the course, comparing their confidence before and after participation. Additionally, verbal feedback was collected to gain deeper insights into their experiences and identify areas for potential improvements in the course design and delivery. Each EMR received individualized feedback on their performance at the conclusion of the course, enabling them to focus on areas for improvement in preparation for the ABEM examination. This study aims to evaluate the effectiveness of this specialized course in improving EMR pass rates on the final ABEM examination and to analyze the progression of self-reported confidence levels from 2017 to 2021.

Methodology

Subjects and settings

This five-year study involved 49 final-year EMRs from a four-year ACGME-I-accredited emergency medicine residency program. Participation in the five-day ABEM preparatory course was mandatory since passing the ABEM examination is a prerequisite for progression to the specialist grade within the emergency department. Annual enrollment ranged from 8 to 11 participants, all local EMRs in their final year of training. Variations in participant numbers differed due to intake sizes and instances of EMRs who required additional time to progress, joining the subsequent cohort. To accommodate both EMRs and faculty schedules, the course was held in the evenings, from 4:00 PM to 9:00 PM.

Table 1 Demographic composition of residents, including the arabic board examination year and resulting outcomes

Table 2 Impact analysis of gender, region or country of graduation, year of examination, and confidence levels on arabic board examination outcomes

Table 3 Sample schedule for emergency medicine training sessions with OSCE and case

Research design

This study employed a quantitative research design to assess the impact of the five-day face-to-face ABEM preparatory course on EMRs’ performance on the ABEM examination. The data collected focused on demographic characteristics and examination outcomes to analyze factors influencing the residents’ success.

Data collection

Demographic data, including participants’ gender, region or country of graduation, and year of the board examination, were collected at the start of the course to provide context for understanding any potential influence on the ABEM examination outcomes. Pass rates for the ABEM examination were obtained from the official Arab Board of Health Specializations website and served as the primary outcome measure for the study (Tables 1 and 2).

Course delivery

The ABEM preparatory course was conducted annually from 2017 to 2021, spanning five evenings, from 4:00 PM to 9:00 PM. The course content was delivered by ABEM-certified faculty, who also conducted the assessments. Each evening began with a joint session at 4:00 PM to introduce course objectives and expectations. From 4:30 PM to 7:30 PM, residents participated in case discussions and Objective Structured Clinical Examination (OSCE) stations, organized in rotating 10-minute intervals, with brief breaks between stations. After a 30-minute dinner break, the evening concluded with feedback sessions, including general feedback for all residents from 7:30 PM to 8:00 PM, followed by individual feedback from 8:00 PM to 9:00 PM (Table 3).

Feedback and evaluation

Clinical cases were used to assess residents’ knowledge and skills in managing critical conditions commonly encountered in the emergency department. Feedback sessions provided an opportunity for faculty to offer tailored feedback, addressing both strengths and areas for improvement. Verbal feedback from both residents and faculty was collected at the end of the course and used to enhance the quality of the course in subsequent years.

Quantitative data was collected using a five-item Likert scale survey to assess residents’ confidence in passing the ABEM examination. The survey was administered anonymously toward the end of the course and was informed by previous studies although it had not undergone independent validation. In addition to the survey, verbal feedback was collected to gain further insights into the residents’ experiences. Both the survey results and verbal feedback were reviewed at the end of the course in discussions with faculty and Emergency Medicine Residents (EMRs). These discussions aimed to identify opportunities for improvement in course design and delivery, ensuring the training program’s ongoing refinement and effectiveness.

Statistical analysis

Data collected over the five-year period (2017–2021) were compiled and analyzed via IBM SPSS Statistics (Version 27.0, IBM Corp., Armonk, NY, USA). Descriptive statistics were used to summarize the demographic data of the participants. The Shapiro‒Wilk test was used to assess the normality of the test score distributions, confirming a normal distribution. The results are presented as the means (standard deviations). Statistical analyses were performed via IBM SPSS Statistics for Windows, version 28.0. Armonk, NY, with significance set at p < 0.05 and 95% confidence intervals (CIs) reported. Additionally, the ABEM examination pass rates, sourced from the official Arab Board of Health Specialization website, were compared with historical data. An impact analysis was also performed to determine the influence of variables such as gender, region or country of graduation, year of participation, and initial confidence levels on examination outcomes.

Ethical considerations

The study was classified as “Exempt” under the Ministry of Public Health (MOPH) guidelines, specifically Category 3, which pertains to research involving the collection or study of existing data, documents, or records, where the information is recorded in a manner such that subjects cannot be identified directly or indirectly. The study received approval from the host hospital’s Institutional Review Board (MRC-01-24-015), adhering to the required ethical standards.

Results

The study included data from 49 emergency medicine residents (EMRs) who participated in a four-year ACGME-accredited residency program between 2017 and 2021. Table 1 summarizes EMR demographics, ABEM examination years, and outcomes, highlighting trends in success rates and demographic impacts on performance. All participants attended a five-day ABEM preparatory course and subsequently sat for the ABEM examination. The majority of participants were male (69.4%), with diverse educational backgrounds. The largest proportion graduated from Sudan (22.4%), followed by Pakistan (18.4%), China (16.3%), Egypt, and India (each 12.2%). Smaller percentages came from the Middle East (10.2%), Europe (6.1%), and Bangladesh (2.0%).

The ABEM examination pass rate was consistently high, with minimal variation across demographic and regional factors. Table 2 presents an impact analysis of gender, region or country of graduation, and year of examination on ABEM outcomes. Male participants achieved a pass rate of 91.2% while female participants achieved 93.3%; however, the difference was not statistically significant (p = 0.799). Participants from Pakistan, Egypt, China, the Middle East, Europe, and Bangladesh all achieved a 100% pass rate. In contrast, Indian graduates had a pass rate of 83.3% while Sudanese graduates had the lowest rate at 72.7%. Statistical analysis indicated that the region or country of graduation did not significantly influence pass rates (p = 0.269). Variations in success rates across examination years were also observed, ranging from 81.8% in 2019 to 100% in 2018 and 2021. These differences, however, were not statistically significant (p = 0.551).

Participants completed a five-item Likert scale survey at the end of the course to self-report their confidence levels in passing the examination. The survey results indicated a marked improvement in confidence after the course. The course’s practical focus, particularly through case discussions and OSCE stations, was frequently noted as beneficial. However, verbal feedback collected during the evening feedback sessions suggested areas for improvement, including more time allocated for OSCE and additional guidance in prioritizing emergency interventions.

Performance on case-based scenarios demonstrated strong resident skills in history-taking, formulating differential diagnoses, and ordering appropriate investigations. Some gaps were identified in prioritizing emergency treatments and adhering to clinical guidelines in specific scenarios, indicating areas for targeted training.

The findings revealed no significant correlation between demographic factors, region or country of graduation, and year of board examination with pass rates, suggesting the course’s broad effectiveness. Despite consistently high pass rates, the combination of confidence ratings, examination outcomes, and verbal feedback emphasized the need for continual refinement of the course to address specific learning needs and evolving trends in emergency medicine. These insights have helped the ABEM preparatory course to include new cases, teaching methods, and feedback mechanisms.

Discussion

The findings of this study underscore the effectiveness of the annual five-day face-to-face preparation courses for the Arab Board of Emergency Medicine (ABEM) examination, which were designed to accommodate the demanding schedules of emergency medicine residents (EMRs). Over the five-year period, the courses significantly enhanced ABEM examination pass rates, providing a promising model for similar programs in other specialties. The substantial improvement in pass rates post-course, from 60 to 91.8%, highlights the course’s critical role in enhancing EMRs’ readiness for the examination and independent practice.

The participant cohort was notably diverse, with a substantial representation of EMRs from Sudan and Pakistan, indicating the broad appeal and relevance of the course across different cultural contexts. This diversity suggests that the course’s effectiveness is not limited by regional or cultural differences, but rather by its adherence to the core competencies required for the ABEM examination. Furthermore, demographic variables such as gender, country of graduation, and year of participation had minimal impact on outcomes, reinforcing the idea that the course itself played a central role in preparing EMRs for the examination.

The course content was designed around pertinent clinical cases commonly encountered in emergency medicine practice (Table 3). Figures 1 and 2 showcase some of the cases used to evaluate the EMRs’ knowledge and skills. Proficiency in managing such cases is essential for success on the ABEM examination as well as for effective functioning as a specialist. Previous research supports the value of case-based learning in emergency medicine, noting that it fosters critical thinking and better decision-making, which are key for both examination performance and clinical practice [6]. Incorporating case discussions and Objective Structured Clinical Examinations (OSCEs) allowed the residents to engage in real-life scenarios, reinforcing their preparedness for the challenges they will face as specialists.

Fig. 1

Example case discussion station 1

Fig. 2

Example case discussion station 2

Herrmann et al. (2022) devised a new Pediatric Hospital Medicine (PHM) board review course aligned with the American Board of Pediatrics (ABP) PHM certifying examination outlines and core competencies [7]. Evaluation via the Kirkpatrick model revealed notable pass rates (89% in person, 93% online), substantial participant learning (mean correct response of 84% on post-presentation knowledge questions), and high ratings for faculty effectiveness (median score of 4.60). Similar benefits have been observed in other medical specialties, such as anatomic pathology, cardiovascular and thoracic surgery, and urology [8,9,10]. These studies employed various methodologies, including survey analysis, video ethnomethodology, and performance score correlations, to assess EMRs’ satisfaction with and perceptions of course efficacy.

Feedback from residents and faculty after each course cycle offered valuable insights into the course’s effectiveness and areas for enhancement. Both verbal feedback and responses to a five-item Likert scale survey administered at the course’s conclusion indicated a high level of satisfaction with the content, delivery, and structure. EMRs particularly emphasized the clarity of instruction, the relevance of case-based discussions, and the opportunity for personalized feedback, which significantly bolstered their confidence in preparing for the ABEM examination. Similarly, Cummings and Connelly (2016) demonstrated that structured and repeated simulation activities can substantially enhance learners’ confidence levels [11].

Looking forward, the course is planned to continue annually, with future modifications based on both participant feedback and evolving ABEM examination guidelines. One proposed change is to include more targeted OSCE stations for EMRs with lower initial confidence scores, which could help further individualize the educational experience. The EMRs’ scores from practice assessments could be used to inform their personalized education plans, addressing specific gaps in knowledge or areas of weakness before the examination. This approach could further enhance the preparation of EMRs, ensuring that each participant receives the support needed to succeed.

The success of the five-day course may be attributed to several factors. The course’s structure, which incorporated a blend of case discussions, OSCEs, and individualized feedback, allowed EMRs to actively engage with the material in a comprehensive and focused manner. Previous preparations for the ABEM examination were less structured and did not include such an immersive, hands-on approach. As a result, the course likely provided a more effective framework for reinforcing knowledge and skills, which contributed directly to the observed improvement in pass rates. This is consistent with findings from studies regarding other medical specialties, such as Eastin and Bernard’s (2013) research on in-training examinations and Herrmann et al.‘s (2022) study on pediatric hospital medicine board review courses. These studies support the value of structured, interactive board preparation courses in improving examination outcomes.

Further, the literature highlights the importance of customized courses that address the specific needs of EMRs. As highlighted by Mittal et al. (2012) and Velez et al. (2013), tailoring course content to align with both examination requirements and the learners’ evolving needs can significantly enhance the educational experience and improve outcomes [12, 13]. The EMRs in this study benefitted from a tailored course that directly addressed key areas in the ABEM examination, ensuring training that was both relevant and effective. Numerous other studies have also documented the advantages of specifically tailored courses in elevating final examination pass rates [14,15,16,17,18,19]. These studies suggest that the impact of examination prep courses on final examination success may vary depending on the implementation of specific strategies.

In conclusion, the annual five-day face-to-face ABEM preparation course has proven to be a highly effective means of improving EMRs’ performance on the ABEM examination. The significant increase in pass rates, the positive feedback from residents, and the value of the tailored approach to learning suggest that similar programs could be beneficial in other specialties. Future iterations of the course will continue to build on this success, incorporating ongoing feedback to ensure that EMRs are fully equipped to meet the demands of the ABEM examination and their future roles as specialists in emergency medicine.

Limitations and future recommendations

This single-center study evaluating the effectiveness of an ABEM examination preparation course has several important limitations. First, the course’s brief duration of five evenings may not fully cover the extensive examination syllabus, potentially resulting in knowledge gaps among participants. Second, the lack of a control group prevents definitive conclusions regarding the course’s impact on examination performance.

A third limitation is the potential influence of locally trained residents, whose prior education and training may have impacted their performance on the ABEM examination. The inclusion of residents from other regional countries within the MENA region could offer a broader perspective and help assess the course’s effectiveness across diverse educational backgrounds. Future studies may benefit from incorporating residents from various countries to better understand the course’s applicability across different training systems.

For future research, extending the course duration to ensure more comprehensive coverage of the examination content is recommended. Additionally, incorporating a control group would allow for a more robust comparative analysis. Combining objective assessments, conducted under realistic examination conditions, with self-reported evaluations immediately following the course would offer a more comprehensive understanding of its effectiveness in preparing EMRs.

Finally, our study’s relatively small sample size may have affected the validity of statistical analyses, particularly when examining results across demographic factors. Qualitative research methods, such as focus groups and interviews, could provide valuable insights into the strengths and limitations of the ABEM program, offering a more detailed and nuanced perspective.

Conclusion

The five-day, in-person ABEM examination preparation course significantly increased EMRs’ confidence levels and improved their ABEM examination pass rates. This study highlights the importance of developing meticulously crafted courses in emergency medicine to ensure that EMRs receive thorough and effective preparation for their certification examinations and future medical careers.