Автор неизвестен - Mededworld and amee 2013 conference connect - страница 38
Results: This resulted in a 14 item peer rating scale, with three subscales (cognitive, social, and motivational contributions). Furthermore, a relationship between individual contributions and achievement was found. PF on these contributions, combined with reflection and goal setting, had a positive effect on achievement, but not on the quality of contributions. However, no differences were found between individual vs. collaborative reflection and goal setting. In the focus group, students indicated that the PF increased their awareness of 'appropriate tutorial behavior'. Discussion and Conclusion: These results indicated that students are able to evaluate their peers' cognitive, social, and motivational contributions to the group, which are related to individual achievement. PF on these contributions can be used to enhance awareness and achievement, but only if the students are stimulated to reflect upon the received PF and formulate goals for improvement. Although this thesis indicated the importance of PF in PBL groups, the question remains how reflection and goal setting can be facilitated optimally.
References: 1. Nieminen J., Sauri P., & Lonka K., On the relationship between group functioning and study success in problem-based learning. Medical Education,
2006. 40(1): p. 64-71.
2. Dolmans D. H. J. M., Wolfhagen I. H. A. P., & Van der Vleuten C. P. M., Motivational and cognitive processes influencing tutorial groups. Academic Medicine, 1998.
73(Suppl 10): p. S22-S24.
3. Moust J.H.C., Van Berkel H. J. M., & Schmidt H. G., Signs of erosion: Reflections on three decades of problem-based learning at Maastricht University. Higher
Education, 2005. 50: p. 665-683.
4. Dominick P. G., Reilly R. R. & McGourty J. W., The effects of peer feedback on team member motivation. Group & Organization Management, 1997. 22(4): p. 508520.
5. Hattie J. & Timperley H., The power of feedback. Review of educational research, 2007. 77(1): p. 81-112.
4E Research Papers: Simulation and Learning Technologies
Location: Meeting Hall V, PCC
Effective debriefing approaches in simulation based education
Kristian Krogh (Aarhus University, Centre for Medical Education, INCUBA Science Park Skejby Brendstrupgardsvej 102, Bygn. B, Aarhus N 8200, Denmark)
Margaret Bearman (Monash University, HealthPEER, Melbourne, Australia)
Debra Nestel (Monash University, School of Rural Health - Faculty of Medicine, Nursing and Health Sciences, Churchill, Australia)
Introduction: Debriefing facilitates participants' learning from the simulated experiences through reflection and feedback. The value of debriefing post scenario in simulation-based education (SBE) has been well documented. For example, a review by McGaghie et al (2010) identifies feedback (including debriefing) as the most important feature of SBE (McGaghie et al. 2010). There are many different models and approaches to facilitate debriefing in SBE within healthcare, building on research, experiences and practical application. However, there is little evidence supporting one approach over another. It is likely that several variables are important such as what is taught, the level of learners, their experience and not least the educator (Steinwachs 1992; B. S. Issenberg & Scalese 2007). Though the literature describes what constitutes effective debriefing, there are discrepancies as to what is actually being practiced (Dieckmann et al. 2009). There is limited information as to how experts or experienced debriefers practice (Dieckmann et al. 2009). This national study explores the practice of expert debriefers, who work within full-scale high-stakes immersive SBE environments. Methods: Individual semi structured interviews were conducted with experts in debriefing after immersive simulation based education. Respondents were nominated by peers through purposive sampling across Australian states. Interviews were audio recorded and transcribed for thematic analysis. Three researchers working independently each coded 3 transcripts and jointly developed a high level coding framework, used to guide interpretive thematic analysis. Triangulation through independent analysis continued throughout the thematic analysis process.
Results: A total of 24 interviews of 45-95 minutes were transcribed. Participants were from all states of Australia, with 20 different workplaces and centres, 6 different disciplines and 14 sub-disciplines. The participants had between 4 and 23 years of experience with debriefing in SBE, with an average of 9.7 years of experience.
The three high order categories are: features of expert practice; the development of expertise; and the
influence of context upon debriefing practice. Analysis is indicating that dominant practice features include: debriefing models used; video assisted debriefing; briefing; and continued professional development. Most participants used a blended approach to debriefing combining different models to fit the need of the learners. Key success factors identified by several participants were: the importance of showing genuine interest, being honest and continuing to strive for being better by continues professional development. Influences on debriefing practice included a range of peer interactions such as: peer feedback, observation of other debriefers, formalised courses in debriefing and conference workshop participation. Discussion and Conclusion: This study looks at the self-reported practices of expert debriefers. The purposive sampling covered a large range of disciplines and no new themes (saturation) were introduced within the final interview set. While there were many convergent features of expert practice and development; there were also context-dependent divergences. The need for peer interaction to develop and sustain expertise was marked. Interviewed expert debriefers do not use a single model for practice but have a blended approaches to debriefing with genuine interest and honesty as the main drivers.
References: Dieckmann P. et al., 2009. The art and
science of debriefing in simulation H: Ideal and practice,
31(7), pp.287-294. Available at: http://informahealthcare.com/doi/pdf/10.1080/014215 90902866218
Issenberg B.S. & Scalese R.J., 2007. Best evidence on high-fidelity simulation: what clinical teachers need to know. The Clinical Teacher, 4(2), pp.73-77. Available at:
McGaghie W.C. et al., 2010. A critical review of simulation-based medical education research: 20032009. Medical education, 44(1), pp.50-63. Available at: http://www.ncbi.nlm.nih.gov/pubmed/20078756 [Accessed August 6, 2011].
Steinwachs B., 1992. How to Facilitate a Debriefing. Simulation & Gaming, 23(2), pp.186-195. Available at: http://sag.sagepub.com/cgi/doi/10.1177/104687819223 2006 [Accessed March 10, 2012].
Considerations in learner-centered versus team-based collaborative online learning
Heather MacNeill (Bridgepoint Hospital, University of Toronto, Physiatry, 14 St Matthews Rd, rm 227, Toronto M4K1G1, Canada)
Deanna Telner (University of Toronto, Family and Community Medicine, Toronto, Canada) Elizabeth Hanna (Bridgepoint Hospital, University of Toronto, Speech Language Pathology, Toronto, Canada)
Introduction: Little is known about how online collaborative learning occurs in healthcare education (13). This exploratory, descriptive study examined the
perceived differences between individual and collaborative group online learning. Methods: Fifteen interdisciplinary healthcare professionals participated in a 12 week, blended online course consisting of online background information and a facilitated "case building" exercise (4) (modified "build-a-case method") (5) where learners were randomized into 2 groups. Group 1 (group learners, n=10) created their patient together synchronously using audio streaming/headsets and synchronous chat, Group 2 (individual learners, n=5) built their case independently online using the same software. Qualitative focus groups were held post intervention and at 4 month follow up and were triangulated with reflection pieces and online sessions. Results: Three themes emerged: Motivation and Stimulus to Learn, Collaboration, and Resources (time and technology). Group learners commented on peer pressure as a stimulus to learn and appreciated the rich feedback and fidelity to real life teamwork they experienced. They also seemed more apt to consider multiple psychosocial/barriers and how interrelated learners' interprofessional roles were when creating their case. They commented on developing a skill set in collaborative communication as well as knowledge acquisition during the course. However, there were "competing interests" about what individuals wanted to learn in a group setting. Individual learners liked the flexibility and control (of time and objectives), but found the breadth of information online overwhelming without immediate feedback. Although both groups reported equal time spent in the course, it was felt that group learning took more time. Working together online multiplied the time required to learn new software and collaborate in a novel way. The online synchronous method added 2 challenges for group learners: lack of visual communication, and time for technology integration.
Discussion: Individual online learning may provide flexible, individualized learning particularly suited for simple knowledge acquisition type of learning. However consideration should be given to provide focus/immediate feedback for the learner and consider motivation/ stimulus to learn given self-directed nature of this type of learning.
Collaborative/group online learning may allow for more exposure of individual unperceived learning needs and comprehension of complex multifaceted information that may require multiple perspectives and opinions to comprehend. It may seem more "true to application to practice" due to its interprofessional nature and allow the development of collaboration as a skill set. However, collaborative learning takes time and resources (facilitation and tech support), and can be overwhelming especially at beginning of a learning encounter, and therefore may not be appropriate for short interactions or simple information acquisition. Conclusion: This exploratory study describes areas for consideration, when matching learning objectives to online collaborative and individual learning methods.
ABSTRACT BOOK: SESSION 4 MONDAY 26 AUGUST: 1400-1530
(This presentation was also previously given at the Canadian Conference on Medical Education, 2012 and
the CME Congress 2012.)
References: (1) Triola MM, Huwendiek S, Levinson AJ, Cook DA. New directions in e-learning research in health professions education: report of two symposia. Med
Teach. 2012; 34(1):E15-E20.
(2) Oncu S, Cakir H. Research in online learning environments: priorities and methodologies. Comput
(3) Kirschner F, Pass F, Kirschner P. Individual and group-based learning from complex cognitive tasks: effects on retention and transfer efficiency. Comput Human Behav.
(4) Mac Neill H, Reeves S, Hanna E, Rankin S. The community of inquiry framework: a pertinent theory of online interprofessional education? In: Bromage A, Clouder L, Thistlewaite J, Gordon F, eds. Interprofessional E-learning and Collaborative Work: Practices and Technologies. Hershey, PA: Information Science Reference; 2010:75-89.
(5) Ryan DP, Marlow B. Build-a-case: a brand new continuing medical education technique that is peculiarly familiar. J Contin Educ Health. 2004;24(2):112-
Cognitive Appraisal as a Predictor of Stress, Patient Management, and Team Performance during Simulated Patient Crises
Carilynne Yarascavitch (University of Toronto, The Wilson Centre, 200 Elizabeth Street, Eaton South 1-565, Toronto M5G 2C4, Canada)
Daniel Haas (University of Toronto, Faculty of Dentistry, Discipline of Dental Anaesthesia, Toronto, Canada) Vicki LeBlanc (University of Toronto, The Wilson Centre, Toronto, Canada)
Introduction: Patient crises require healthcare teams to process high volumes of information, make appropriate decisions, and carry out multiple procedures under time constraints (1). These acute circumstances can evoke stress responses, which are known to impair attention, memory and decision making abilities (2). This has implications for patient safety. Proposed explanations for variations in individual responses to stress can be guided by the theory of Cognitive Appraisal (3). According to this theory, situations where perceived resources fail to meet perceived demands produce an imbalance which results in an appraisal of threat rather than challenge. Perceptions of threat have been associated with increased subjective and physiological stress (3-4), but have not been described in relation to team performance. The objective of this study was to describe the relationship between cognitive appraisals, stress responses and performance in the context of teams managing simulated medical emergencies. Methods: 22 teams of one general practice dentist and one assistant participated in four different simulated medical emergency scenarios of equal difficulty. Participants completed demographics and a knowledge
pre-test. Scenarios were video recorded and independently scored by four trained raters: patient management was assessed by checklists (Ck) previously developed through a Delphi method and a global rating scale (pGRS); teamwork was assessed using the Global Assessment of Obstetric Team Performance modified to the dental context (GATP) and a global rating scale (tGRS). Pre and post-scenario stress was measured by cognitive appraisal (ratio of perceived demands to resources; <1 = challenge appraisal; >1 = threat appraisal), self-reported anxiety (STAI-Y1) and salivary cortisol. Relationships between cognitive appraisal, stress and performance, knowledge and demographics were evaluated using stepwise logistic regression. Results: Dentists (D) and assistants (A) demonstrated a wide range of years of practice experience (D, M=19.2, R3.5-4.2; A, M=9.5, R=.5-25) and knowledge scores (D, M=78%, R55-100%; A, M=53%, R2-90%). Mean scores for threat, anxiety, and cortisol were above population norms for all scenarios. All checklists and scales demonstrated good inter-rater reliability (ICC Ck=0.85;
pGRS=0.76; GATP=0.759; tGRS=0.79). Cognitive
appraisal emerged as a consistent predictor of both stress response and performance: threat was associated with higher stress response (cortisol r=.30, p=.00; anxiety r=.50, p=.01) but lower performance (Ck, r=-.23,
p=.03; pGRS, r=-.22, p=.04; GATP, r=-.24, p=.03; tGRS, r=-
.25, p=.02). Knowledge and clinical experience were not consistent predictors of stress response or performance. Discussion and Conclusion: Healthcare providers can make mistakes during crisis events that can compromise patient safety. Patient crises can lead to subjective and physiologic stress responses that interfere with performance. Our pattern of findings suggests that cognitive appraisals of threat by team members are related to increased stress and degreased performance. Health professionals may benefit from simulated training experiences which address cognitive appraisal to reduce stress and enhance performance during medical emergencies.
References: (1) Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiolgica Scandinavia
(2) LeBlanc VR. The effects of acute stress on performance: Implications for health professions education. Academic Medicine 2009;84(10):S25-S33.
(3) Lazarus RS & Folkman S. Stress, appraisal, and coping. 1984. New York, NY: Springer.
(4) Tomaka J, Blascovich J, Kelsey RM, & Leitten CL. Subjective, physiological, and behavioural effects of threat and challenge appraisal. Journal of Personality and Social Psychology 1993;65(2):248-260.
(5) Dickerson S & Kemeny ME. Acute stressors and cortisol responses: A theoretical integration and synthesis of laboratory research. Psychological bulletin
ABSTRACT BOOK: SESSION 4 MONDAY 26 AUGUST: 1400-1530
The Impact of Massed versus Spaced Instruction on Learning and Self-efficacy in Pediatric Resuscitation
Catherine Patocka (McGill University, Emergency Medicine Residency Program & Centre for Medical Education, 687 Pine Ave West, Rm A4.62, Montreal H3A1A1, Canada)
Farooq Khan (McGill University, Emergency Medicine Residency Program, Montreal, Canada) Dubrovsky Sasha (McGill University, Pediatrics, Montreal, Canada)
Danny Brody (McGill University, Pediatrics, Montreal, Canada)
Ilana Bank (McGill University, Pediatrics, Centre for Medical Education& Steinberg Medical Simulation Centre, Montreal, Canada)
Farhan Bhanji (McGill University, Pediatrics, Centre for Medical Education & Steinberg Medical Simulation Centre, Montreal, Canada)
Introduction: Resuscitation skills are life-saving interventions provided by healthcare providers (HCP) in which survival is related to the quality of resuscitation (1). Unfortunately HCPs completing resuscitation courses demonstrate limited retention of skills in the weeks to months following training (2). HCP self-efficacy in resuscitation correlates with performance of critical skills and predicts application of life-saving skills in the simulation environment (3). Our study explored the impact of spaced instruction, compared to the usual massed instruction, on learner performance of pediatric resuscitation skills and self-efficacy in resuscitation. Methods: We delivered a previously published (4) resuscitation course to medical students in either a spaced (4 sessions of 75 minutes each, spaced over 4 weeks) or massed (one five-hour session) format. Four weeks following course completion trained, blinded observers used expert-developed checklists to evaluate student performance on 3 skills (bag valve mask ventilation (BVMV), intra-osseous insertion (IOI) and chest compressions (CC)). From the total of 32 checklist elements, 10 were deemed key. Self-efficacy was measured pre-, post- and 4 weeks post-course using a validated visual analogue scale (VAS). Results: Forty-five out of 48 students completed the study protocol. Students in both groups had similar overall checklist scores for IOI and CC, however students in the spaced group performed significantly better on BVMV ((6.9±1.4 points/10) vs. (5.8±1.9 points/10)(p=0.04)). Students in the intervention group performed key elements more frequently than those in the massed group for 4 of the 10 key elements (administering oxygen (OR 47.2, 95% CI 5.2-423, p<0.001), adhering to the target ventilation rate (OR 4.9, 95% CI 1.1-21 p<0.03), appropriate landmarking for IOI (OR 5.4, 95% CI 1.3-24.3, p<0.02), and using a stool when necessary for performing chest compressions (OR 8.3, 95%CI 1.2-59, p<0.03). Additionally students in the spaced group were faster at completing the potentially life-saving skill of IOI (30.2±34 seconds) vs. (62.1±30
seconds) (p=0.002). Both cohorts improved their self-efficacy in pediatric resuscitation from pre to post course, however only students in the massed group had decay in their self-efficacy 4 weeks following course completion: mean VAS score post course (70.9±12mm) vs. mean VAS score 4 weeks post-course
Discussion and Conclusion: Previous research suggests that spacing educational encounters over time results in improved learning but the literature is inconsistent, focuses on simple rather than complex tasks and reports immediate rather than long-term retention of skills (5). As well, little is known on the effect of spacing on learner self-efficacy, an important predictor of performance in critical life-threatening events. In our study, students taught pediatric resuscitation in a spaced format had improved skill performance on key steps one-month post-course completion and were more efficient to complete tasks. They also did not demonstrate decay in their self-efficacy while students in the control group did. Procedures learned in a spaced format may result in better retention of skills as compared to traditional massed training. Students' self-efficacy ratings suggest that learners taught in a massed format are less confident in their abilities when they might have to perform pediatric resuscitation. References: 1. Edelson DP, Abella BS, Kramer-Johansen J, Wik L, Myklebust H, Barry AM, et al. Effects of compression depth and pre-shock pauses predict defibrillation failure during cardiac arrest. Resuscitation 2006, Nov;71(2):137-45.<br>
2. Bhanji F, Mancini ME, Sinz E, Rodgers DL, McNeil MA, Hoadley TA, et al. Part 16: Education, implementation, and teams: 2010 american heart association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation 2010;122(18\_suppl\_3):S920.<br>
3. Turner NMB, Lukkassen I, Bakker N, Draaisma J, ten Cate OTJ. The effect of the apls-course on self-efficacy and its relationship to behavioural decisions in paediatric resuscitation. Resuscitation 2009;80(8):913-8.<br>
4. Bhanji F, Gottesman R, de Grave W, Steinert Y, Winer L. Paediatric resuscitation training--do medical students believe it should be a mandatory component of the curriculum? Resuscitation 2011;82(5):584-7.<br>
5. Donovan JJ, Radosevich DJ. A meta-analytic review of the distribution of practice effect: Now you see it, now you don't. Journal of Applied Psychology
4F Short Communications: Assessment: Written
Location: Chamber Hall, PCC 4F/1
Estimating insight and foresight from summative assessment
Mike Tweed (University of Otago Wellington, Department of Medicine, PO Box 7343, Wellington 6242, New Zealand)
Sarah Stein (University of Otago, Higher Education Devleopment Centre, Dunedin, New Zealand) Tim Wilkinson (University of Otago Christchurch, Department of the Dean, Christchurch, New Zealand) Jeff Smith (University of Otago, College of Education, Dunedin, New Zealand)
Background: When insight and foresight are lacking overconfidence and error can occur. Our pilot demonstrated how MCQs can be used to estimate insight, as determined by certainty in responses, and foresight, as determined by the safety of incorrect responses. This investigation extends this to summative assessment.
Summary of work: Fifth year medical students selected MCQ responses with certainty: low, moderate, high. Incorrect responses were classified: not, low, moderate, high unsafe. Analysis included response certainty and safety and student ability.
Summary of results: During 2011-2012, 15 students scored low(<8/20 correct), 64 substandard(8-11/20), 70 above standard(12-15/20), and 9 excellent(>15/20). A degree of insight and foresight were demonstrated. The proportion of responses: correct increased from a mean of 0.38 for low to 0.80 for high certainty(p<0.001); incorrect and not unsafe decreased from 0.28 for low to 0.07 for high certainty(p<0.001); incorrect and unsafe decreased from 0.27 for low to 0.10 for high certainty(p<0.001). A similar pattern was seen for ability groups. Of the incorrect responses the proportion that were unsafe increased from 0.41 for low to 0.55 for high certainty(p=0.004). This was seen for the above standard group only with 0.37 for low to 0.64 for high certainty responses(p=0.002).
Conclusions: Unlike others, we found that lower scoring students were insightful and the number of unsafe responses were not proportional to incorrect responses. The increase in unsafe responses may be due to the nature of clinical decision-making; developing insight before foresight; or risk-taking. Take-home messages: Certainty and safety response questions can be used in summative assessment and provide further information on and for students.
ABSTRACT BOOK: SESSION 4 MONDAY 26 AUGUST: 1400-1530
Meta-evaluation of multiple choice questions (MCQs) in clinical anatomy: prevalence, outcomes and guidelines
Milton Severo (Faculty of Medicine of the University of Porto, Department of Clinical Epidemiology, Predictive Medicine and Public Health, Alameda Prof. Hernani Monteiro, Porto 4200-319, Portugal) Bruno Guimaraes (Faculty of Medicine, University of Porto, Center for Medical Education, Porto, Portugal) Joao Pais (Faculty of Medicine, University of Porto, Center for Medical Education, Porto, Portugal) Ana Povo (Faculty of Medicine, University of Porto, Department of Anatomy, Porto, Portugal) Artur Silva (Faculty of Medicine of the University of Porto, Department of Anatomy, Porto, Portugal) Maria Amelia Ferreira (Faculty of Medicine of the University of Porto, Department of Anatomy, Porto, Portugal)
Background: The principles of writing effective multiple-choice test questions (MCQs) are well documented in educational measurement textbooks. However, few educators have formal instruction in writing MCQs and they often have item-writing flaws. The purpose of this study was to evaluate the effects of violations of standard multiple-choice item writing principles on item characteristics.