Is CME effective in improving Physicians` knowledge? Second Order Systematic Review

Authors

Mohammad Al-Onazi

Theme

Teaching and learning: the teachers toolkits and context for learning

Category

Postgraduate education and continuing medical education (CME)

INSTITUTION

RMH

Conclusion

 

In summary, pre-educational intervention need assessment and an interactive learning activity is required in designing CME to ensure a positive effect on physician`s knowledge. For sustaining such effect a follow up educational intervention should be planned 6 months to one year later

Background

Continuous medical education (CME) consists of educational activities in the medical field that is designed by faculty who are experts in certain clinical areas. These educational activities might be live programs, online courses, workshops, small group discussion, academic detailing, written materials, or any activity that serve to maintain, or increase the knowledge, skills, and professional competence of health professionals

Summary of Work

 

It is a 2nd Order Systematic Review of the literatures, conducted in an office setting. The study subjects are the Systematic Reviews of a well-designed and properly executed reviews which provide the best evidence on the efficacy of health care interventions.

Inclusion criteria:

The studies will be included if they (were):

  • Systematic reviews
  •  Written in English
  • Published after 2004.
  • Used CME interventions of a didactic or interactive nature.
  • Objectively determining the effect of CME on physician's knowledge.
  • More than 50% of the participants are practicing physicians, or residents in training.

 

 All included studies passed the following stages:

 Three screening phases.

Assessment of quality of the systematic review.

Collecting general study characteristics.

Collecting CME activity characteristics.

Compiling studies data for comparison.

We utilize the measurement tool developed by Shea and Colleagues in 2007 to assess the quality of reviews and reported ("A MeaSurment Tool to Assess Reviews"

(AMSTAR))

Take-home Messages

Recommendations& Implications for future policy and research:

 

An initial step in planning, and designing an activity for CME or CPD is to conduct a systematic process to determine and address the learning needs of the targeted audiences. This review shows clearly that interactive learning activities such as small group discussion, audit and feed back, and academic detailing are effective in improving physicians` knowledge as compared to didactic, and lecture-based activities. Due to the knowledge decay property of human being a follow-up intervention is advisable to sustain the knowledge gained. Most of the educational studies suggested 6 months to one year post-intervention as the appropriate timing for re-intervention. This review clearly found that the included primary studies in the systematic reviews are of poor quality. This finding calls for future researches of good quality to be able to build a firm inferences from such good quality researches


Summary of Results

 

The search of the electronic databases yielded 184 systematic reviews, while hand Searches yielded 14 systematic reviews. As a total we located 198 systematic reviews. After initial screening of the titles, 127 potentially relevant papers were considered for the second screening of the abstract. This process yielded 54 systematic reviews, which underwent a third screening of the full article to determine wither they satisfied the inclusion criteria. Nine of these studies are found to be relevant. The average quality of the of the included reviews according the AMSTAR were of moderate type. The resulting synthesis can therefore be described as sufficiently robust to sustain the subsequent interpretation and analysis. Participants were primary care physicians (four studies), clinicians ( two studies), resident in training (one study), and obstetrics and gynecologist (one study)

Acknowledgement

To My wife Dr. Wedad Al-Onazi, for her assisstance in retrieving researches, and to the secretarial staff in the department of Medical Education in College of Medicine in King Saud Ibn Abdulaziz for Health Sciences (COM, KSAU-HS), Mrs Rose and Annabelle, for their valuable administrative support.


References

Dr. Mohammad Al-Onazi


Dr. Rana Tamim


Prof. Mohi Eldin Majzoub


Conclusion
Background
Summary of Work

 

AMSTAR  

Yes. No. Can’t Answer. Not Applicable

1-Was a `priori` design provided?

The research question and inclusion criteria should be established before the conduct of the review

Yes. No. Can’t Answer. Not Applicable

2-Was there duplicate study selection and data extraction?

There should be at least two independent data extractors, and a consensus procedure for disagreements should be in place

Yes. No. Can’t Answer. Not Applicable

3-Was a comprehensive literature search performed?

At least two electronic sources should be searched. The report must include years and databases used (eg Central, EMBASE and MEDLINE). Key words and/or MESH terms should be stated and where feasible the search strategy should be provided …

Yes. No. Can’t Answer. Not Applicable

4-Was the status of publication (i.e. grey literature) used as an inclusion criterion?

The authors should state that they searched for reports regardless of their publication type

Yes. No. Can’t Answer. Not Applicable

5-Was a list of studies (included and excluded) provided?

A list of included and excluded studies should be provided

Yes. No. Can’t Answer. Not Applicable

6-Were the characteristics of the included studies provided?

In an aggregated form such as a table, data from the original studies should be provided on the participants, interventions and outcomes.

Yes. No. Can’t Answer. Not Applicable

7-Was the scientific quality of the included studies assessed and documented?

A priori’ methods of assessment should be provided (eg for effectiveness studies if the author(s) chose to include only randomized, double-blind, placebo-controlled studies, or allocation concealment as inclusion criteria)

Yes. No. Can’t Answer. Not Applicable

8-Was the scientific quality of the included studies used appropriately in formulating conclusions?

The results of the methodological rigor and scientific quality should be considered in the analysis and the conclusions of the review, and explicitly stated in formulating recommendations.

Yes. No. Can’t Answer. Not Applicable

9-Were the methods used to combine the findings of studies appropriate?

For the pooled results, a test should be done to ensure the studies were combinable, to assess their homogeneity

Yes. No. Can’t Answer. Not Applicable

10-Was the likelihood of publication bias assessed?

An assessment of publication bias should include a combination of graphical aids (eg funnel

plot, other available tests) and/or statistical tests

Yes. No. Can’t Answer. Not Applicable

11-Was the conflict of interest stated?

Potential sources of support should be clearly acknowledged in both the systematic review

and the included studies.

  Table: Study quality rating:

 

Quality Rating

Underlying Methodology

High

8 – 11

Moderate

6 - 7

Low

 < 6

Take-home Messages
Summary of Results

 

 

The result of this second order systematic review was in agreement with the previous reports (Maliheh et al, Bernard S. Bloom, and  Woosung et al) which indicate that educational interventions most likely to result in improvement of physician knowledge involve multiple interactive educational efforts that is clinically relevant such as, interactive small group sessions, workshops, and written materials or tool kits combined with audit or feedback, and strong communication channels between instructors and learner

 

Result table:

 

Evaluation in individual studies

Outcome Measure

Teaching Strategies

Target Audiences

Review Quality

(AMSTAR)

Supplement

Included Studies Design

Number of Studies Included, (Reported  quality)

Year

Author

Three studies shows some form of evaluation and follow-up.

Pre-/post test

Follow-up, mail survey, audit. and

Questionnaires

 

Improving Knowledge of practicing physicians

Improving practitioner behavior and practice towards older patients

Small –group learning

PBL

Feedback

Conferences

Didactic lectures

Academic detailing

Opinion leaders

Multiple methods

Primary Care Physicians and community internists

(1160+)#

Moderate (6/11)

Interviewing  22 directors of geriatrics CME Practicing Physician Education (PPE) programs that emphasized active-mode learning

RCT*, Systematic reviews, Meta-analysis.

(Details are not specified)

13

(Not specified)

 

2006

David C. Thomas17

Self-administered questionnaires in all studies except three

Interviews with GP in two interventions

Audits in three interventions

 

 

In 14 interventions measuring physicians` knowledge and attitude, an increase in the knowledge and attitude towards palliative care issues or cancer pain management. One RCT did not find differences in knowledge or attitude between the intervention and control group.

Role model educators, didactic courses, small group discussions, case studies, clinical rounds, home visits, outreach programs, opinion leaders, mini-fellowship, educational materials, guidelines, small group learning, and media events

Primary care physicians

(1653)

High

(9/11)

No

1 Systematic review

11 Interventions as before and after trials

3Educational courses

2 Evaluation comparing to guidelines

1 RCT

 

18

(Low)

 

(Consort Statement)

2006

Marta P. Alvarez18

Comparative

Effective intervention in improving the clinicians knowledge and their   prescription of recommended antibiotics for acute outpatients infections

 The median effect of interventions combining clinician education with audit and feedback was smaller than that for clinician education alone.

Academic detailing, educational outreach, workshops, printed materials, educational programs, feedback, mailed information, and symposiums,

Clinicians

(255350+)#

High

(9/11)

No

13 RCT

12 Controlled before-after trials

1 Interrupted time series

26

(Moderate)

 

(EPOC Recommendation)^

2006

Michael A. Steinman19

Questionnaires, knowledge tests, observations, surveys, pre-/post-intervention questionnaires

Changes in physicians knowledge, attitude, clinical practice, and patient outcome.

Training courses, written materials, programs, internet courses, seminars, outreach visits, multifaceted intervention, academic detailing, and in practice educational sessions

General Practitioners (GP)

(3378+)#

High

(8/11)

No

21 RCT

6 Clustered RCT

5 Quasi-experimental

32

(not specified)

2007

Jing Tian20

 

 

 

 

 

 

Comparative

Effectiveness of CME intervention on physician's knowledge.

Academic detailing, educational outreach, workshops, printed materials, educational programs, feedback, symposiums. and multifaceted programs.

Practicing physicians

High

(9/11)

No

Comparative studies with adequate control group

(RCT, Non RCT**, Observational studies)

28

(Low)

 

(Jadad Score)

2008

Georges Bordage21

Self evaluation, structured and standardized surveys, direct observation of teaching behavior, videotaped teaching sessions, objective structured teaching sessions, pre-and post-test, and resident questionnaires.

Effectiveness of resident-as-teacher interventional programs on the attitude, knowledge, skills, and behavior of the resident.

Lectures, small group discussion, debriefing, role plays, simulation, retreat, feedback, workshop, and seminars

Resident in training

Moderate

(8/11)

No

7 RCT

10 Non-RCT

10 Single group study

2 descriptive case series

29

(Low)

 

(Cochrane Handbook for Systematic Reviews of Intervention (Version 5.0.1))

2009

Andrew G. Hill22

Comparative results, feedback, questionnaires, self-assessment,

Learning achievement for knowledge, skills, attitude, and behavior

Educational meetings, interactive sessions, didactic sessions, e-learning

 

Low

(4/11)

No

3 RCT

7 Non RCT

13 Before and after comparison studies

23

(Moderate)

 

(Not specified)

 

 

 

2009

Amer Raza23

Comparative

1-Health care outcomes

2-health professional behavior.

3- PCPs` knowledge and learning

Multifaceted interventions, interactive seminars, outreach visits, small group workshops, and educational group meetings.

Primary care providers (PCP)

(1904)

High

(9/11)

No

5 Cluster RCT

1 Controlled before and after study

6

(Low)

 

(EPOC Quality Criteria) ^

2011

M. Perry24

Pre- and post-training test, audit, and questionnaires

Kirkpatrick`s four-level model

Computer-based training programs, audit with feedback, lectures, case studies, clinical supervision, simulated clinical scenarios,

Obstetrics and Gynecologist

Moderate

(8/11)

No

6 RCT

2 Non RCT

6 Non-comparative

6 Comparative time-series

20

(Low)

 

(Not specified)

 

2011

Pehrson C25

Acknowledgement
References
Send ePoster Link