Authors

Sheng-Po Kao
Mei-Lin Hsieh

Institutions

Buddhist Tzu Chi General Hospital - Taiwan

Tzu Chi University - Taiwan,

Logo
  • Currently nan/5
  • 1
  • 1
  • 2
  • 2
  • 3
  • 3
  • 4
  • 4
  • 5
  • 5

Rating: nan/5 (0 votes cast)

Theme

10BB Simulation 2

Title

A New Rating Scale for Assessing Standardized Patients' Performance

Background

Since the announcement of first article about quality assurance of SPs’ performances, most of the related articles focused on accuracy, consistency and replicability of SPs’ performances. Moreover, they needed to develop case-specific items of SPs’ performance for each SP case. Since the purpose of quality assurance is to assure the fairness of examinees’ score, examinee-centered quality assuring method may meet the goal more closely. We have developed a newly general rating scale for assessing standardized patients’ performance across different SP cases after reviewing and categorizing common errors during video clips in the Pilot Nationwide Clinical Skill Assessment at Tzu Chi University in Taiwan in 2011. This study is to assure the feasibility of this newly developed rating scale.

Summary of Work

Twenty-four video clips from 12 SPs with a range of performance levels were selected, by a staff member in SP training program, for expert/rating scale comparisons. SP performance was evaluated by two nurse raters who applied the developed rating scale and two extramural experts who used global judgment. We use Kappa value to estimate the concordance between expert opinions and our developed rating scale.

Summary of Results

With comparative analysis with Kappa Statistic between the two nurse raters and the two extramural experts, only one paired comparison revealed moderated agreement (Kappa coefficient = 0.571, P value= 0.028). The other three paired comparisons revealed chance agreement. However, both of the nurse rater and the expert with moderated agreement are stringent, while others showed leniency when rating.

Conclusion

Though only one paired comparison showed moderated agreement, it seems feasible to develop a newly general rating scale for assessing standardized patients’ performance. However, rater training for rating consensus will be recommended for applying this new score sheet effectively. Future research is needed for assuring the reliability and validity of the rating scale.

Take-home Messages

Since the purpose of quality assurance of SP's performance is to assure the fairness of examinees’ score, examinee-centered quality assuring method may meet the goal more closely. It is needed and seems feasible to develop a newly general rating scale for assessing standardized patients’ performance.

Acknowledgement

We would like to thank Ministry of Science and Technology, R.O.C. for funding this project under Grant MOST 103-2511-S-303-003.

References

  1. Vu, N.V., Steward, D.E., Marcy, M. (1987). An assessment of the consistency and accuracy of standardized patients’ simulations. Journal of Medical Education, 62, 1000-1002.
  2. Tamblyn, R.M., Klass, D.K., Schnabl, G.K., et al. (1990). Factors associated with the accuracy of standardized patient presentation. Academic Medicine, 65(9), S55-S56.
  3. Tamblyn, R.M., Klass, D.K., Schnabl, G.K., et al. (1991). The accuracy of standardized patient presentation. Medical Education, 25, 100-109.
  4. Erby, A.H., Roter, D.L., Biesecker, B.B. (2011). Examination of standardized patient performance: Accuracy and consistency of six standardized patients over time. Patient Education and Counseling, 85, 194-200.
  5. Barrows, H. S. (1993). An overview of the uses of standardized patients for teaching and evaluating clinical skills. Academic Medicine, 68(6), 443-451.
  6. Furman, G.E., Smee, S., Wilson, C. (2010). Quality Assurance Best Practices for Simulation-Based Examinations. Simulation in Healthcare, 5 (4), 226-231.
  7. Wallace, P. (2007). Coaching Standardized Patients for Use in the Assessment of Clinical Competence (1st ed.). Springer Publishing Company.
Background

 

Summary of Work

Summary of Results

Conclusion
Take-home Messages
Acknowledgement
References
Send ePoster Link