Prioritization / Relevance

Guiding questions:

Should the use of virtual patients in the curriculum be mandatory or voluntary?
How to improve the perceived relevance of virtual patients by students?
What is localization of virtual patients?

Integration of virtual patients in curricula differ by the weight the faculty put on their use by students. Some universities tend to play safe and introduce virtual patients as voluntary activities. Interested students are recommended to use virtual patients to deepen their knowledge as a self-study in their free time. Other universities make them mandatory which means their completion is a core requirement to obtain course credits. Which of those approaches is better?

There is no simple answer to this question and the evidence for recommendations is limited. What has been consistently observed is that when virtual patients are completely voluntary their use will be very low [Radon 2011, Kim 2018]. In a study by Hege et al., just around 10% of students used virtual patients when they were optional [Hege 2007]. Such behavior of students has been observed in implementation of other types of e-learning and simulation in general as well [Issenberg 2005].

The straightforward solution is to make the use of VP obligatory. This is even expected by some students who admit that they need external motivation to complete them [Huwendiek 2013]. Yet, there is a risk in mandatory integration scenarios. Teachers tend to set completion criteria easy to verify automatically, like minimal percentage of content (e.g. screen cards) displayed, time spent on exercise, success rate in multiple choice questions. But this can lead to superficial learning. There are always students who get tempted to game the system and click mindlessly through the content to reach the required threshold score or exchange answers to questions with other students [Hege 2007]. Kim et al. compared the mandatory and voluntary use of VPs in clinical clerkship and found that students in the mandatory integration mode completed more VPs and spent more time on them [Kim 2018]. Unfortunately, there was neither any difference in objectively measured knowledge by the end of the clerkship nor in student satisfaction which might be interpreted as evidence of superficial learning. Within the project consortium we have described different integration examples that you might find helpful.

The key to gain educational benefit from VPs seems to be to convince the students about the relevance of VPs to succeed in their health professional career [Edelbring 2012]. One way to achieve this is to show them explicitly how the learning objectives of VPs map to the curricular learning objectives. This is already discussed here.

Another solution is to take advantage of the rule that assessment drives learning and make the VPs exam-relevant. It means that the use of VPs remains voluntary, but it is explicitly stated how their content is covered in the examination blueprint. For instance, the teacher may declare that 20% of the questions in the final exam will address topics discussed in the VPs. In the study by Hege et al. more than 90% students used exam-relevant VPs [Hege07]. The result was comparable to the obligatory strategy, but the time the students spent on solving the cases was longer and there were indications of a more in depth approach to learning.

There are also other ways to improve the perceived relevance of the cases by students and by that their usage ratio. One of them is to involve students in the authoring of VPs. This approach is known as the “learning by teaching” strategy [Hege07]. Students often know best what is challenging for them in handling VPs and in collaboration with experts in the field focus on those issues when designing cases, learn from that, and make the VPs more relevant for peer students [Radon11].

Motivation to use VPs is low when students think they are not their target group. This may happen when VPs in the curriculum were developed in a context different to that of the students’ – e.g. as presented in external commercial products purchased by the university or in cases by authors from abroad in international projects [Fors 2009]. Some of the recommended diagnostic and treatment methods may be different and the cultural background feels foreign. The language of the VPs may be a barrier as well. Fors et al. has observed the use of VPs in a group of Romanian students who declared good or very good English skills. The same cases were randomly assigned to students either in English or Romanian language. Despite good declared English skills, students selected more diagnostic tests, made better decisions and spent more time on learning when the cases were available in their native language [Fors 2009].

The solution to the foreign context problem could be the language and cultural adaptation of VPs when implementing them in the curriculum. Some of the systems allow the content of the VPs to be translated in the native language of the student, which is worth the effort even if students (declare to) have good English skills. Medical tradition and cultural differences can be clarified and commented to the students by local specialists in expert comments connected with the cases. Improving intercultural understanding is important due to the growing mobility of patients and medical doctors and international VPs provide the opportunity to broaden students’ views and experience [Fors 2009, Radon 2011]. The iCoViP project has developed a workflow to handle the adaptation process and applied it in the development process of the 200 VPs initially designed in English to the local context and languages of the 5 participating countries.

What is also very important in convincing students about the relevance of VPs is the positive attitude of their teachers to the cases. Students are skillful in recognizing that. If the teachers disbelieve in the effectiveness of VPs, so will their students. If the teachers say this is just for the “young generation”, the students will doubt they will get feedback in case of problems and the topics presented in VPs will be relevant during their examinations. It is crucial the teachers walk the way they talk: know the content of VPs, are familiar with how to use them, and make reference to them while teaching.

Recommendations:
– When choosing an integration scenario for your institution, take all the advantages and disadvantages into consideration. Completely voluntary use may meet with low interest. When making VPs obligatory make sure you will not tend to set completion criteria too mechanistic which promotes superficial learning.
– Make sure your VPs will be relevant to the learning material by showing students explicitly how the learning objectives of VPs map to the curricular objectives.
– Making the VPs exam relevant may help to increase the number of cases completed and promote a more in-depth approach to learning.
– When using VPs created elsewhere (in foreign context), you may adapt their language and cultural background, before implementing them into the curriculum.

Phase of the curriculum	4th year of medical school
Goal in the curriculum	The iCoViP collection was implemented to practice selected VPs in the subject of Occupational Medicine (a voluntary subject in the University of Zaragoza in the 4th year of medical school).
Effective use of resources	The collection was used asynchronously so it was not necessary to have a computer infrastructure or a specific room and all students in Spain have laptops or access to the medical school computers. However, issue of updating content not yet solved.
VP alignment	The collection is used while students are doing their clinical placements having a teacher revising the responses and explaining to students most frequent mistakes.
Prioritization/relevance	For Occupational Medicine, the use of the iCoViP collection was a mandatory assignment needed to pass, but did not add points to the final grade.
Relation to other learning activities	None
Time allocation	One week was given to complete the assignment but more time was available for those who wanted to do the activity remote from home.
Group allocation	Students worked alone at home, when the educator discussed the solution of the VP, it was allocated in class (face-to-face or online).
Presence mode	Asynchronous
Technical Integration	It was integrated into Moodle.
VP use orientation/training	VPs were related to a specialty of medicine, but were randomly assigned to students regardless of where in the specialty they were rotating at the time.
Technical infrastructure	We used the technical infrastructure of the iCoViP project without any additions at the University of Zaragoza.
Learning activities around VPs	The teacher explained each VP and how to resolve them after getting the responses of all the students.
Assessment	Students had to complete the VPs, but there was no grade at the end. Only credit (pass/fail).
Quality assurance, maintenance, and sustainability	Frequent revision of the VPs by medical doctors.

Phase of the curriculum	Year 3 and 4 of medical school
Goal in the curriculum	Students learn basic steps of clinical reasoning including identifying relevant findings, developing differential diagnoses, deciding about a final diagnosis, ordering tests to rule out / confirm differentials, and suggesting treatment options.
Effective use of resources	Course tutors needed time to familiarize themselves with the VPs and time had to be planned during the synchronous meetings to discuss the VPs
VP alignment	VPs were part of the modules (e.g. Abdomen, cardiovascular system, pulmonary system, etc.) in year 3 and 4 of medical school and aligned with the objectives of these modules based on key symptoms. In addition a pool of 41 VPs was available for deliberate practice across key symptoms and diagnoses.
Time allocation	5 VPs / module
Group allocation	Students could choose whether to work in groups or individually.
Presence mode	Students could decide when and where to work on the VPs during the period of the module.
Technical Integration	VPs were integrated into the school's learning management system Moodle via a SingleSignOn interface.
VP use orientation/training	No specific familiarization, but general introduction at the beginning of year 3.
Learning activities around VPs	Depending on the modules other learning activities were embedded.
Assessment	The topics of the VPs were part of modules assessment.
Quality assurance, maintenance, and sustainability	We used the built-in feedback functionality to receive qualitative feedback from students and VPs were part of the regular evaluation activities of the medical school.

Phase of the curriculum	Year 1 and 2 of medical school
Goal in the curriculum	Students learn basic steps of clinical reasoning such as identifying & prioritizing findings and composing a summary statement. They also can follow the reasoning process of the VP author concerning differential diagnoses, ordered tests, and treatments.
Effective use of resources	Course tutors needed time to familiarize themselves with the VPs and time had to be planned during the synchronous meetings to discuss the VPs.
VP alignment	VPs were part of the longitudinal clinical course and aligned with the other modules in year 1 and 2 (In case of Augsburg this was Contact, Movement, and equilibrium). We aligned the key symptoms of the VPs with these modules, so that students worked in parallel on these VPs and the corresponding module.
Time allocation	15 VPs over two years / 5 VPs per module.
Group allocation	Students could choose whether to work in groups or individually.
Presence mode	Students could decide when and where to work on the VPs during the period of the module.
Technical Integration	VPs were integrated into the school's learning management system Moodle via a SingleSignOn interface.
VP use orientation/training	At the beginning of year 1 students were introduced into clinical reasoning and how they can train this ability with VPs.
Learning activities around VPs	Depending on the key symptoms during the longitudinal clinical course other learning activities were embedded.
Assessment	The topics of the VPs were part of the clinical longitudinal course assessment.
Quality assurance, maintenance, and sustainability	We used the built-in feedback functionality to receive qualitative feedback from students and VPs were part of the regular evaluation activities of the medical school.

Phase of the curriculum	Basic sciences/pre-clinical years: - At Jagiellonian University Medical College in Kraków we have integrated it in the “Introduction to Clinical Sciences” course in 2nd year of medicine. - At University of Porto in the “Propedeutics/Semiology” course in the 3rd year of medicine.
Goal in the curriculum	- To provide the students with an opportunity to challenge their knowledge at home in between classes with an interactive, clinical-oriented task. - To support learning by linking basic science knowledge with clinical reasoning. Students had the opportunity to practice on undiagnosed cases the skill of differentiating common symptoms, such as dyspnea, abdominal pain, headache.
Effective use of resources	Use of iCoViP VPs in native language of the students. No extra cost needed.
VP alignment	- In Kraków we have selected a few VPs (seven) with common diseases (e.g. pneumonia, pancreatitis, pulmonary embolism) with common symptoms. - In Porto, VPs are chosen according to the common symptoms to promote clinical reasoning.
Prioritization/relevance	- In Kraków the completion of all VPs is mandatory. - In Porto, VPs are introduced on a voluntary basis and are available on demand.
Relation to other learning activities	We used the time students had at home between seminars (Kraków: between on-campus based PBL sessions). VPs were spaced-activated (a new VP appeared biweekly synchronized with changing topics of the PBL seminars that focus on different leading symptoms).
Time allocation	Around 30 minutes biweekly, repeated 7 times in a semester.
Group allocation	Students worked alone at home to reflect but could consult the VPs with their peers or instructors in small groups of the face-to-face PBL seminars.
Presence mode	Students worked on the VPs asynchronously and self-directed at home to have time to reflect and consult textbooks.
Technical Integration	VPs were integrated with a course to which all students were enrolled on the official university learning management system Moodle using Learning Tools Interoperability (LTI) interface.
VP use orientation/training	Students were provided with an introductory email with instructions and had additionally the opportunity to technically practice using CASUS in a parallel “Telemedicine” (Medical informatics) course. Instructors received test-codes to practice the VPs at home.
Technical infrastructure	Students used their own computers from home. An email address was provided to a person responsible for technical support.
Learning activities around VPs	Students were asked to complete concept maps for all VPs they solve. Moreover, in Kraków students were provided with links to additional online articles to help them with topics difficult at this stage of education.
Assessment	Students were asked to complete the cases prior to the end of the term. They were not given grades for the activity - just credit. Their answers were randomly inspected to see common mistakes and provide general feedback to all students.
Quality assurance, maintenance, and sustainability	We checked the diagnostic accuracy of individual cases (and detected one case with imprecise diagnosis). Students evaluated the course using the iCoViP case collection evaluation questionnaire.