Enhancing formative assessment in critical care training:

The use of a virtual patient

Timothy Willett, RCPSC Rakesh Patel, uOttawa Bernie McDonald, uOttawa Rick Hodder, uOttawa Angele Landriault, RCPSC Pierre Cardinal, RCPSC & uOttawa

Disclosure

• Employee of Royal College

• Royal College owns RCsim

3

Background

• Critical care: • Little time for decision-making during crises • High-stakes decisions • Few opportunities for safe practice with observation

and feedback • Formative assessment (Simulation and

feedback): • Standardized patients limited procedures • High-fidelity simulation limited fidelity

1. Rushton A. Formative assessment: a key to deep learning? Medical Teacher. 2005; 27(6):509-513.. 2. Yorke M. Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education. 2003; 45(4):477-501. 3. Aggarwal R, Mytton OT, Derbrew M, et al. Training and simulation for patient safety. Quality & safety in health care. 2010;19(Suppl 2):i34-43.

4

Alternative: Virtual Patients

• Can offer opportunities for deliberate practice

• Safe

• Reusable

• Best suited for clinical reasoning & decision-making

Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Medical education. 2009;43(4):303-11.

5

Objective

• Using a virtual patient for formative assessment during critical care training

• Feasibility • Perception of value for the use of RCsim as a tool

for formative assessment - Critical Care Fellow perspective - Facilitator perspective

6

RCsim: Open ended

7

Non-leading

8

Learner Operator

Facilitator

RCsim: Multimedia

9

Photo or

Video

Text Dynamic

vitals

Feedback during and/or after the case

10

Learner Operator

Facilitator

Qualitative approach

• Post session questionnaires for fellows • 2 cohorts (total n=13) • 4 open ended questions • completed immediately following the session

• Interviews for facilitator

• Responses anonymously transcribed

• Thematic analysis

11

Results

• Feasibility: • 1 hour session scheduled back to back

- Administrative assistant • Academic half day (3 or 4 back to back) • Available intensivist for facilitation • Available operator familiar with the application

and cases • Cases

12

Fellows’ perspective

• Realism: • “The case was realistic” (EF) • “Videos very useful. Case realistic” (EF) • “Not as realistic as the patient is not in front of you

at all times” (IF)

• EF = End of session Feedback • IF = Integrated feedback (throughout the session)

13

Fellows’ perspective

• Flow: • “Runs like real scenario” (EF) • “Allowed you to work through on your own, even

when stuck to challenge you to see what you know” (EF)

• “A lot of interruptions” (IF) • “Feedback is effective because it makes you think

out loud” (IF)

14

Fellows’ perspective

• Outcomes: • “I gained insight into the speed at which info

gathered/things ordered” (EF) • “Importance of closing the loop” (EF) • “Benefit of reasoning out loud to involve the

knowledge of the rest of the team” (EF)

• “[Will] make me a better teacher” (IF) • “To be more quick in decisions” (IF)

15

Fellows’ perspective

• Value: • “One of the best teaching activities I had” (EF) • “Different from other learning

opportunities…better prep. you for the real situation” (EF)

• “Overall a very useful learning experience” (IF) • “OK for a junior or senior resident but not a

fellow” (IF)

16

Facilitator s’ Perspective

• Insight into how residents would manage a critical situation on their own (EF)

• Additional opportunity to see them manage a case, which is infrequent in real life (EF)

• Fellows were safe & thorough, with good clinical reasoning, but were hesitant to commit to action (IF)

• Hesitancy could be because of exam-like formative assessment model (IF)

17

Conclusion

• VP sessions are generally perceived as realistic

• Sessions are enjoyable for learners and facilitators

• Learners can receive practical feedback that they can put into practice

• Unique opportunity for facilitators

• Further research is needed to determine strengths and weaknesses of the two facilitation/feedback models

18