Автор неизвестен - Mededworld and amee 2013 conference connect - страница 39
Summary of work: 920 test items from 10 examinations were classified as either standard or flawed. If flawed the exact type of item flaw or flaws contained within the question (including options) was recorded. Four judges (2 teacher/2 students), blinded to all item performance data, independently classified each item. Summary of results: The flawed items prevalence was 54.7%. The most frequent flaws were: negative stem (18.1%), choice length not equal (11.6%), use none of the above (5.2%) and keep items dependent (4.8%) and not minimize reading (3.7%). Overall, the standard items are 4.5% easier than the flawed items (p=0.006) and the mean biserial correlation is 3.0% higher for the standard vs. the flawed items (p=0.016). However, the effect on difficulty and discrimination differs by each type of flaw. Conclusions: The prevalence of flaws was high and in general had a negative impact on item characteristics. Take-home messages: It is important to increase the formal instruction in MCQs item-writing in order to minimize item-writing flaws in future examinations. The project was supported by PP_IJUP2011 67 grant
MCQ vetting and students performance
Majed Wadi (University of Science and Technology, Centre for Medical Education and Training, Centre for Medical Education and Training, The 60th Road, P.O.Box 13064, Sana'a NA, Yemen)
Ahmad Fuad Abdul Rahim (Universiti Sains Malaysia, Medical Education Department, Kubang Kerian, Malaysia)
Muhamad Saiful Bahri Yusoff (Universiti Sains Malaysia, Medical Education Department, Kubang Kerian, Malaysia)
Kamarul Aryffin Baharuddin (Universiti Sains Malaysia, Medical Education Department, Kubang Kerian, Malaysia)
Background: MCQ vetting is important to ensure validity, reliability, and other quality indicators of such assessment tools. Faculty members invest a substantial amount of time and effort into the MCQ vetting process. However, there is a shortage of scientific evidence showing its effectiveness and at which level it needs to be focused on. This study aimed to provide evidence regarding the effects of question vetting process on students' examination performance by looking at their scores and pass-fail outcomes. Summary of work: A parallel randomized control trial was conducted on third year medical students in a medical school. They were randomly assigned into two equal groups (i.e. control and experimental). Two mock examinations were conducted (time I and time II). At time I, non-vetted MCQs were administered to both groups as a baseline measurement. At time II, vetted MCQs were administered to the experimental group, while the same non-vetted MCQs were administered to the control group.
Summary of results: Out of 203 students, 129 (63.5%) participated in both mock examinations. 65 students were in the control group and 64 students were in the experimental group. Statistical analysis showed no significant differences (p > 0.05) in mean examination scores and pass-fail outcomes between or within the control and experimental groups. Conclusions: This study indicated that the MCQ vetting process did not influence examination performance. Despite these findings, the MCQ vetting process should still be considered an important activity to ensure that test items are developed at the highest quality and standards.
Take-home messages: It can be suggested that such activity can be done at the departmental level rather than at the central level.
Standard Setting for Written Short Answer Question (SAQ) Examinations
Agnes Dodds (The University of Melbourne, Melbourne
Medical School, Level 7, North Wing Medical Building,
Parkville, Melbourne 3010, Australia)
Katharine Reid (The University of Melbourne, Melbourne
Medical School, Melbourne, Australia)
Fink Michael (The University of Melbourne, Department
of Surgery, Austin Health, Melbourne, Australia)
Background: The Angoff method of standard setting is widely used for Multiple Choice Question (MCQ) Examinations. However, it presents conceptual and logistic problems when applied to Short Answer Questions (SAQs). We wished to explore the usefulness
ABSTRACT BOOK: SESSION 4 MONDAY 26 AUGUST: 1400-1530
of the Borderline Regression method for establishing cut scores on written SAQs.
Summary of work: Examiners for six SAQs provided global ratings on a five-point scale in addition to total scores out of 20 for each SAQ assessed. We used total scores and global ratings from 360 second year medical students on each SAQ to develop cut scores using linear regression.
Summary of results: Average scores were relatively high across the SAQs (range 12.8-16.8), but with reasonable variation in scores and fairly consistent standard deviations. We predicted the scores of borderline students using a linear regression equation predicting total score from global rating categories. These scores ranged from 10.8 to 14.4 and served as the cut score for acceptable performance. More students fell below the Borderline Regression standard compared with a 50 per cent standard.
Conclusions: The Borderline Regression method can establish defensible standards in the context of written SAQs. With appropriate training, examiners can complete a global rating at the same time as they mark responses. Further exploration of standard setting methods is warranted to establish their usefulness for different assessments, and to consider the practical implications of their implementation. Take-home messages: Defensible standards can be achieved for written exams using the Borderline Regression method and its use overcomes a number of logistic problems.
Analysis of a Script Concordance Test (SCT) to evaluate treatment decisions in Physical Therapy: is item-response frequency analysis useful to improve test quality?
Claudia Gormaz (Oficina de Educacion en Ciencias de la Salud , Universidad Mayor - Facultad de Medicina, Guardia Vieja 255 oficina 606. Providencia, Santiago 7510186, Chile)
Luis Henriquez (Escuela de Kinesiologia, Universidad Mayor - Facultad de Medicina, Santiago, Chile) Maria Elisa Bazan (Oficina de Educacion en Ciencias de la Salud , Universidad Mayor - Facultad de Medicina, Santiago, Chile)
Carlos Brailovsky (The College of Family Physicians of Canada, Toronto, Canada)
Background: SCT is widely validated as reliable to evaluate decision-making in clinical reasoning (CR). Our aim was to develop and validate a SCT to evaluate CR in musculoskeletal disorders in Physical Therapy for 4th year students and recently graduated Physical Therapists (PT).
Summary of work: Five PT specialized in musculoskeletal disorders developed 38 vignettes with 155 nested items based on a blueprint of course objectives. Seventeen experts validated the test. The final SCT had 18 vignettes and 75 items. Cronbach and generalizability tests were run. Frequencies of item
responses of the expert panel were analyzed to better understand the way the experts answered. Summary of results: Forty-six students and 27 graduates wrote the SCT. Even though the Cronbach alpha of the experts' and graduates' results was a=0,733 and a=0,706 respectively, the students alpha was low (a=0,546). The analysis of the frequencies of the item responses showed dispersion among experts in almost half the items, which can explain the students' results. Conclusions: Even though the construction and validation of the SCT vignettes and items followed strict protocol, the students' Cronbach alpha was low. We postulate that the homogeneity of the distribution of answers from the expert panel partly explains the low alpha in this group. Our results reflect an inherent feature of the specialty.
Take-home messages: Improvement of the quality of evaluation instruments is crucial in all health professions. Validity and reliability analyses are important but not sufficient. Other aspects must be considered such as the concordance experts share on specific fields.
4G Short Communications: Curriculum:
Rural Medical Education
Location: Conference Hall, PCC
Do rural placements influence the attitudes of medical graduates towards rural practice?
John A Bushnell (University of Wollongong, Graduate School of Medicine, Norhtfields Avenue, Wollongong
Michael Jones (Macquarie University, Psychology Department, Sydney, Australia) John Humphreys (Monash University, School of Rural Health, Bendigo, Australia)
Background: In the face of a mal-distributed medical workforce in Australia, much attention has been paid to rural clinical placements a potential means of increasing interest in rural practice among medical graduates. Previous studies suggesting a positive effect of rural clinical placements have suffered from methodological limitations in study design, sampling, and failure to account for potentially confounding factors. We used data from a large national research project to examine the influence of rural and remote placements on attitudes to rural practice.
Summary of work: The Medical Schools Outcomes Database and Longitudinal Tracking Project (MSOD) is a national medical school multi-cohort longitudinal study that collects demographic and career intention measures, location and duration of clinical placements from all Australian medical schools. We explored the influence of placements, controlling for a number of demographic and contextual variables, using data from participants at commencement and exit from medical school (n=3268).
Summary of results: Rural/remote placement later in the program had a strongly positive effect upon future rural career intention. Students from rurally-focused medical schools were also more likely to endorse future rural career intention. However, the influence of these rural placements was overshadowed by the strong positive influence of prior rural background and rural intention at the time students commenced their studies. Conclusions: To influence future rural career choice, characteristics of students at selection for medical study, and the nature of medical school culture warrant more attention than they have had to date. Take-home messages: Rural clinical placement experiences are important, but may have more impact on future rural career choice if undertaken by students of rural origin.
ABSTRACT BOOK: SESSION 4 MONDAY 26 AUGUST: 1400-1530
Successful interventions to improve human resources for health: The Stellenbosch University Rural Medical Education Partnership Initiative
Marietjie De Villiers (Stellenbosch University, Medical Education, Franscie van Zijl Drive, Tygerberg, Cape Town
7505, South Africa)
Jean Nachega (Stellenbosch University/ University of Pittsburgh, Infectious Diseases, Cape Town, South Africa) Kalay Moodley (Stellenbosch University, Medical Education, Cape Town, South Africa) Lilian Dudley (Stellenbosch Unniversity, Public Health, Cape Town, South Africa)
Susan Van Schalkwyk (Stellenbosch University, Medical Education, Cape Town, South Africa)
Background: Stellenbosch University was among thirteen African Universities awarded the Medical Education Partnership Initiative grant aimed at improving human resources for health in Africa. SURMEPI specifically aims to improve the quality and quantity of health care workers; retention in areas of need and regionally relevant research. Summary of work: The project focuses on the development and evaluation of innovative medical education models that aim to support rural and underserved communities. The project uses two foci: medical curriculum renewal and health systems strengthening and development to render the delivery of transformative, systems based education. Summary of results: Year three of the grant has shown significant outputs in training approximately 550 health care workers and faculty in Public Health, PALSA PLUS, Evidence Based Health Care, Infection Prevention and Control and HIV/TB management. In addition SURMEPI has provided 21 undergraduate students with funding for rural electives in the African region. Eight PhD and 18 Masters students conducting research in rural areas have been supported through mentorship and funding. Interdepartmental collaborations have led to context-specific, systems-based approach to medical education integrally linked to the graduate attributes of SU. ELearning has been a cross-cutting theme resulting in a shift and recognition of the benefits of eLearning for medical students at central and rural sites. Conclusions: In the short time span of approximately three years, SURMEPI has achieved significant outputs in transforming medical education and improving human resources for health in rural Africa. Take-home messages: Innovative models adequately incorporated into institutional systems will promote the transformative education agenda at medical schools.
Rural electives - providing transformative learning opportunities and influencing choice of career: The Stellenbosch University experience
K Moodley (Stellenbosch Uuniversity, Medical Education, Tygerberg, Francie van Zijl Street, Cape Town 7505, South Africa)
B Willems (Stellenbosch University, Medical Education,
Cape Town, South Africa)
M De Villiers (Stellenbosch University, Medical
Education, Cape Town, South Africa)
S Van Schalkwyk (Stellenbosch University, Medical
Education, Cape Town, South Africa)
Background: Rural background and longitudinal rural clerkships for medical students including training at rural medical schools are established predictors for rural practice. Considering that less than a third of all South African medical students originate from rural areas and are educated in urban universities, additional strategies are necessitated to encourage rural career choices. This study explores the benefits of a one month rural elective in influencing further rural practice and is funded by the Stellenbosch University Rural Medical Education Partnership Initiative.
Summary of work: This qualitative study evaluated rural elective students' learning experiences and possible influences on future career choice. Online anonymised surveys and feedback from focus group discussions were analysed to identify recurring themes. Summary of results: Students described the rural elective as an enriching experience and a positive opportunity facilitating exposure to the context in which they will ultimately work. Adapting to environments with limited resources and populations with varying cultural backgrounds were embraced and valued by students. In addition, learning opportunities for community engagement and patient-centred care were seen to inspire students to return to rural areas. Conclusions: Rural electives provide transformative learning experiences that could influence urban educated medical students' choice of working in rural areas. Further investigation of this model is recommended.
Take-home messages: Innovative medical education models offering shorter periods of rural exposure may serve to enhance rural recruitment in South Africa.
What is the impact of an integrated community clerkship on students' attitude toward stressors of rural practice?
Wayne Woloschuk (University of Calgary, Undergraduate Medical Education, Health Sciences Centre, 3330 Hospital Drive NW, Calgary T2N 4N1, Canada)
Douglas Myhre (University of Calgary, Distributed Learning and Rural Initiatives, Calgary, Canada) Wesley Jackson (University of Calgary, Family Medicine, Calgary, Canada)
Kevin McLaughlin (University of Calgary, Undergraduate Medical Education, Calgary, Canada) Bruce Wright (University of Calgary, Undergraduate Medical Education, Calgary, Canada)
Background: In 2008 the University of Calgary Medical School implemented an integrated community clerkship (ICC) in the third and final year. ICC students spend 36
ABSTRACT BOOK: SESSION 4 MONDAY 26 AUGUST: 1400-1530
weeks in a rural community where family physicians act as primary preceptors. ICC students follow patients longitudinally and simultaneously complete mandatory rotations in family medicine, emergency medicine, obstetrics and gynecology, psychiatry and anesthesia. The clerkship of non-ICC students is rotation-based (RB) and their main community exposure is via family medicine (6 weeks) at rural, regional and urban practices.
Summary of work: We surveyed students (classes of 2009 - 2011) pre-post ICC and family medicine rotations to determine whether attitudes change over time. Responses to 6-point survey items that measured stressors of rural practice (dependent variable - lower mean scores reflect a positive attitude) were analysed according to location and time (independent variables) using a priori contrasts. Alpha was set at .05. Summary of results: Pre-post rotation data were
collected from 212 (48%) students. The mean (3.32)
reported by ICC students at post rotation was significantly lower than their pre rotation mean (3.60). The mean (4.02) reported by urban RB students at post rotation was significantly higher than their pre rotation mean (3.77). Means reported by students at rural RB and regional RB sites did not change. At post rotation the ICC mean was significantly lower than all other means.
Conclusions: Over time the initial concerns of ICC students about the stressful aspects of rural practice were alleviated.
Take-home messages: ICC students reported more favourable attitudes.
Teaching Primary Care Obstetrics: Insights and Recruitment Recommendations of Family Physicians
Sudha Koppula (University of Alberta, Family Medicine, 205 College Plaza, Edmonton T5X2G9, Canada) Judith B Brown (Western University, Family Medicine, London, Canada)
John M Jordan (Western University, Family Medicine, London, Canada)
Background: It has become challenging to recruit family medicine residents to primary care obstetrics practice upon their graduation. This is, in part, due to the lack of family physician teachers who practice intrapartum care, and who serve as positive role models for learners. Summary of work: One-on-one interviews were used to explore the experiences and recruitment recommendations of family physicians who practice and teach primary care obstetrics. Participants were all family physicians from Edmonton, Canada who practiced primary care obstetrics in group systems of care. There were 12 participants in this study, which is the number of participants at which saturation was reached. The interviews were audiotaped and transcribed verbatim. All transcripts were analysed individually by each investigator, and as a group of investigators, in an iterative and interpretive manner. Emergent themes
were analyzed to determine the primary care obstetrics teaching experiences of the participants in this study. Summary of results: Data analysis revealed findings categorized as follows: 1) poor confidence in teaching abilities, 2) challenges of having learners, 3) benefits of having learners, and 4) recommendations for recruiting learners to primary care obstetrics. Conclusions: Although the participants described limited confidence as teachers, and challenges with learners, they also identified positive influences which sustained their interest in teaching.
Take-home messages: Supporting these family medicine obstetrics educators, and recruiting more such positive role models, are strategies that could be employed to encourage family medicine learners into careers involving primary care obstetrics.
4H Short Communications: Clinical Teaching 2
Location: Club H, PCC
Adaptation during third-year clerkship: The lived experiences of medical students at the Northern Ontario School of Medicine
Tim Dube (Laurentian University, School of Rural and Northern Health, 935 Ramsey Lake Road, Sudbury P3E 2C6, Canada)
Roger Strasser (Northern Ontario School of Medicine, Rural Family Medicine, Sudbury, Canada) Robert Schinke (Laurentian University, School of Human Kinetics, Sudbury, Canada)
Background: Hafferty (1998) distinguished three interrelated concepts of what medical students learn: formal, informal, and hidden curriculum. Of the three concepts, the hidden curriculum has been the least explored in medical education. The authors who have explored notions of hidden curriculum have demonstrated how the experiences of medical training entrenched in the hidden curriculum can have a profound impact on medical student adaptation. Summary of work: The Northern Ontario School of Medicine's (NOSM) clerkship year consists of a mandatory eight-months of living and working in rural and northern communities throughout Northern Ontario and learning in the context of rural family practice. I explored how 12 third-year students described the challenges they had to manage and, in response, the strategies they employed to adapt to their clerkship. Data collection included: pre-clerkship interviews and a demographic questionnaire, mobile methods in the form of 'guided walks', and post-clerkship interviews. Vignettes were developed using the co-constructed narratives to present the participants' lived experiences. Summary of results: I will provide a rich description of events experienced such as training in one's hometown or familiar community, transitions including adjusting to the clinical setting and to the medical profession, the influence of clerkship on career path, personal well-being, and empathy.
Conclusions: The findings serve to advance our understanding of students' adaptation processes throughout a longitudinal integrated clerkship. Implications for medical students, community preceptors, and medical schools will be discussed. Take-home messages: I will propose recommendations regarding the suitability of authentic qualitative methods in medical education research, and discuss the implications for rural and northern health research.
ABSTRACT BOOK: SESSION 4 MONDAY 26 AUGUST: 1400-1530
Enhancing teaching and learning in service environments with rapid patient turnover
K Nathavitharana (Worcestershire Acute Hospitals NHS Trust, Education Centre, Alexandra Hospital, Woodrow Drive, Redditch B98 7UB, United Kingdom)
Background: Clinicians in acute settings face ever-increasing workloads and pressure to achieve early discharge. Time for teaching is limited due to service demand and high patient throughput. Opportunities for students to examine patients with a variety of physical signs and clinical features, as stipulated in the Curriculum, are limited due to the shortened length of patient stay. Many medical students complete their training, often with minimal experience of direct supervision of their clinical technique. Masterclasses were established to address this unmet need. Summary of work: Patients with illustrative case histories and physical signs were invited to attend focused teaching sessions on a voluntary basis. The visits did not form part of their care. Transport and refreshments were provided. Over two years, patients with a wide range of clinical features participated in a series of themed systems based sessions. No more than one or two students saw each patient whilst being directly observed by a consultant. This enabled constructive feedback on eliciting physical signs, discussion on differential diagnoses, further investigation and management. A tutorial approach fostered wider discussion, emphasising learning points. Summary of results: The Masterclasses were consistently highly rated by students against a range of criteria, including organisation, content, relevance, teaching style and patient selection. Students became more confident in clinical examination skills and were stimulated to practice these skills further. Patients not only valued the experience, but also commented on how much they learnt about their own medical conditions. Conclusions: Masterclasses deliver focused teaching based on direct supervision and constructive feedback to enhance learning.
Take-home messages: Teaching the clinical skills necessary for focused history taking and eliciting physical signs is challenging in settings with high patient throughout. Masterclasses provide an effective solution.
Does the number of patients seen by medical students relate to examination performance?
Miki Kiyokawa (University of Hawaii, Medicine, Hawaii,
Kalyani Premkumar (University of Saskatchewan, Community Health & Epidemiology, B103 HSc 107 Wiggins Road, Saskatoon S7N 5E5, Canada) Marcel D'Eon (University of Saskatchewan, Community Health & Epidemiology, Saskatoon, Canada) Krista Trinder (University of Saskatchewan, College of Medicine, Saskatoon, Canada)
Damon Sakai (University of Hawaii, Medical Education, Hawaii, United States)
Sheri Fong (University of Hawaii, Medical Education, Hawaii, United States)
Background: The association between number of patients seen by medical students and their performance in written and clinical skills exams is not clearly defined. Our study investigated this relationship. Summary of work: Log book data from sixty-four 3rd year medical students who completed their 12-week internal medicine clerkship at the University of Hawaii were analyzed. The relationship between the number of patients seen to board examination (NBME) scores, objective-structured clinical examination (OSCE) scores, type (longitudinal (L)/block (B)) and order of rotation (inpatients/outpatients) was analyzed. Summary of results: Overall, there was no significant relationship between total number of patients seen to examination scores. However, there was a positive correlation for students enrolled in a conventional B clerkship between NBME exam scores and the number of ambulatory patients seen (R=0.42, p=0.004). B students who did outpatient rotation first followed by inpatient, had positive correlation between numbers of ambulatory patients seen and NBME exam scores (R=046, p=0.019). Students in a longitudinal community clerkship who did outpatients first had positive correlation between number of inpatients seen and OSCE scores (R=0.70, p=0.016). Conclusions: A number of factors such as total patient numbers seen, clerkship pathway, and order of rotation seem to affect examination performance. Further research is needed to confirm these findings and explore possible explanations.