Authors

Julia Freytag
Ulrike Sonntag

Institutions

Charité Universitätsmedizin Berlin - Department for Curriculum Management - Berlin - Germany

Logo
  • Currently 1.00/5
  • 1
  • 1
  • 2
  • 2
  • 3
  • 3
  • 4
  • 4
  • 5
  • 5

Rating: 1.0/5 (1 vote cast)

Theme

5AA Evaluation of the teacher

Title

Quality of teaching performance: Teachers' self-assessment in comparison to learners and observers views

Background

In medical education, both students and teachers are often asked to rate their own performance. However, research has proven self-assessment to be a difficult task.1 In this study, teaching performance is assessed by teachers, learners and observers. Their ratings are compared and checked against an objective measure, namely adherence to trained teaching standards.

Summary of Results

Distribution of teachers’, students’ and observers’ global rating of teaching performance (in %)

 

Very good

Good

Satisfactory

Weak

Teachers

7

61

28

4

Students

56

36

6

2

Observers

30

47

20

3

 

Correlations of global ratings of teaching performance, rated by students, teachers and observers (τb)

 

Teacher

Students

Observer

Teacher

1

-

-

Students

.097 (p = .03)

1

-

Observer

.25 (p = .04)

.15 (p < .001)

1

 

Central Tendency of global ratings of teaching performance, rated by students, teachers and observers

When comparing the central tendency of the global teaching performance ratings, each group differed significantly from the two others.

Correlations of global rating and fulfillment of teaching standards

The extent to which teachers fulfilled checklist standards correlated significantly with the global ratings of teaching performance of learners and observers (τb = .13 and .5, p < .001) but not with teachers’ self-assessment of teaching performance (τb = -.05, p = .67).

Take-home Messages

  • Teachers’ self-assessments should always be accompanied by views of learners and/or experts to obtain more accurate information.
  • Teaching standards and their importance need to be made clearer to teachers.

Summary of Work

60 sessions of KIT (a course for medical students about communication, interaction & teamwork in medicine) were observed by trained observers, using a recently developed checklist.2 Additionally, observers, teachers and learners were asked to rate overall teaching performance.

Correlations (Kendall's tau-b; τb) and Wilcoxon signed-rank tests (to test for differences in central tendency) were used to compare the assessments of the three groups. Furthermore, it was tested, if the results provided by the checklist correlated with the different global ratings (τb).

Conclusion

The significantly correlated global ratings of learners, observers and teachers indicate, that to some extent, the members of all three groups use similar criteria to rate teaching performance. Yet, the ratings differ significantly in central tendency. Teachers are more critical in their self-evaluation than observers, learners are least critical. Furthermore, our results show that teachers do not seem to relate to trained standards when assessing their performance. In contrast, students rate teaching performance the better, the more standards are fulfilled, without even explicitly knowing them. By doing so, they underline the quality of the teaching standards.

 

References

1. Davis,  Mazmanian,  Fordis, van Harrison, Thorpe & Perrier. (2006). Accuracy of physician self-assessment compared with observed measures of competence. Journal of the  American Medical Association, 296, 1094–1102.

2. Fach  & Moerschner (2014). Qualitätssicherung der Dozierenden-Schulung Umgang mit Simulationspatienten. Unpublished thesis, Charité Universitätsmedizin Berlin.

Background
Summary of Results

Exact results of the Wilcoxon signed-rank tests (central tendency)

Oberservers vs. teachers: Mdn(observers) = 2; Mdn(teachers) = 2; z = -2.26, p = .02 (two-tailed), r = -.21

Observers vs. students: Mdn(observers) = 2; Mdn(students) = 1; z = -8.52, p < .001 (two-tailed), r = -.29

Teachers vs. students: Mdn(teachers) = 2; Mdn(students) = 1; z = -12,74, p < .001 (two-tailed), r = -.44

Take-home Messages
Summary of Work

Parts of the observation checklist

 

More details about..

..KIT: KIT is a mandatory course for all medical students at Charité which follows a longitudinal curriculum. In small groups of about 8 students and a facilitator a broad spectrum of topics concerning social and communication skills needed as a physician, is covered.

..the development of our checklist: The checklist is based on internal teaching standards, which are taught in our faculty development courses. After a first version was developed, the checklist was piloted, checked for understandability, missing items, interrater agreement etc. Small changes were made afterwards.

..the training of observers: Observers were trained during a 2-hour training session, where they were introduced to the checklist and the accompanying guide (consisting of explanations and examples).Questions were discussed and agreement between observers was tested.

 

Conclusion
References
Send ePoster Link