Systematic initial management of the acutely ill patient, standardised reporting and awareness of human factors improve communication and patient safety. Training and implementation of these procedures in hospitals, however, remain challenges.

St George's Hospital developed 'MEERKATS' (Medical Emergencies: Evaluation, Response and Keeping Attention on team communication and patient Safety), a mixed didactic and simulation-based training programme for new doctors and senior ward nurses.

We used 'Programme Theory' [1] and 'Contribution Analysis' [2] (tools used in the evaluation of large scale developmental projects which recognise that there are many contributions to outcomes) and applied it to this smaller scale context to construct and implement a 'theory of change' to revise and re-launch our course.

Programme Analysis can define success criteria within multiple and complex interdependencies and where traditional causality questions (to determine real evidence of efficacy in achieving a desired outcome) may be impossible to answer.We used this process to radically redesign our educational inputs as well as success indicators.

Theory-based evaluation is an iterative process of building, testing and refining the causal model. Programme Analysis may illuminate how an intervention is performing in terms of it's intended results.Achieving our original aim was more difficult than we thought. Locating our course within an array of other interventions promoted better design as well as more realistic evaluation.

Research Questions: To what extent are observed results due to programme activities rather than other factors?

1.Is the programme goal well understood?

2. What are the measurable changes attributable to these goals?

3. What assumptions underlie the programme's theory of change?

4. What additional factors, unrelated to the programme, influence results?

Our first step was to redevelop an existing course based on 'teacher assumptions' rather than educational needs. In order to establish the theory of change, the following steps were taken:-

Step 1: Focus on areas of acute care to be targeted (literature reviews)

Step 2: Understand specific targets for change within a complex system (governance of programme)

Step 3: Assess the contributions of other interventions (contextualising the programme and goals)

Step 4: Describe the goals of the course and how these are to be achieved (the performance story- by undertaking a focus group meeting with experts knowledgeable about the programme area)

Step 5: Assemble evidence of change within the complex acute care environment (interviews with ward managers)

Step 6: Revise goals, course structure and evaluation points.

The new course started in January 2010 and has included more than 250 participants.

Programme Analysis stimulated us to question our original assumptions. We then:-

1. Clarified goals (with course participants and their ward managers)

2. Defined responsibilities and interactions within faculty members

3. Changed teaching and learning activities within a new theory of change. (redesigning of lectures and simulation scenarios)

4. Implemented new programme activities (workshops/small group teaching sessions)

5. Identified other factors influencing success or failure (availability of both faculty and participants)

Many thanks to the critical care staff and the simulation department for making the MEERKATS course a success!

1. Funnell C, Rogers PJ. Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. John Wiley & Sons, Inc. NY. 2011

2. Mayne J. Addressing attribution through contribution analysis, Canadian Journal of Programme Evaluation. 2001; 16:1-24