Effective use of Resources

Guiding questions:

How to deal with barriers due to limited funds?
How to deal when you do not have enough expertise?
What to do when you have no computer rooms or other elements of technical infrastructure?

One of the often quoted barriers in implementation of VPs are their costs. In a well known study carried out at U.S. and Canadian universities by Huang et al. [Huang 2007] the average costs of developing one virtual patient were reported to be in the range of $10,000 – 50,000 without maintenance costs. Other types of costs include: faculty development, teacher salaries, IT infrastructure and staff, classroom use, etc. The VPs and the software needs to be updated [Haag 2010]. Altogether it amounts to a total that appears to many teachers as an insurmountable barrier. How to deal with that?

The costs of VPs should of course not be ignored, but the good news is that with enough determination there are ways to deal with this issue. The best proof for that is the fact that VPs are reported to be in use on all continents (except Antarctica) both in countries with high and low income [Kononowicz 2019]. Below are a few tips on how to succeed if the budget for virtual patients is very limited.

First, it is important to reduce expectations of perfection in aesthetics and technical sophistication. The entertainment industry and press releases reporting on innovative virtual reality research projects create a demand unrealistic to satisfy on graphical fidelity in virtual patient presentation and the simulated clinical environment. Many expect VPs to contain very advanced artificial intelligence-based functions e.g. in natural language processing for history taking. This is of course all impressive and interesting, but to get a good learning outcome it is often unnecessary. In fact, there are studies published that fail to show any improvement in learning outcomes with growing technical sophistication of the VP simulation [Dankbaar 2016]. Some features of virtual worlds may actually be more a distraction than help. High technical requirements lead to problems in simulation accessibility (e.g. when external Internet connections are blocked, there is no possibility to install plugins or require expensive non-standard computer equipment like VR headsets) [Conradi 2009]. It might help to be clear from the very beginning that introducing VPs in the curriculum is not about impressing one with technology or entertaining, but about learning what is relevant in clinical reasoning in a flexible and interactive way.

The notion that VPs require dedicated computer labs to implement is nowadays largely outdated. The availability of computers or mobile devices at students’ homes is wide-spread. Taking advantage of this is an obvious way when you plan to implement v as part of the online preparatory activities in the flipped classroom model. If you need to be available in person for feedback while they are learning, you may suggest a “bring your own device” class when students have one shared laptop for a team brought by a group member. Of course this works only in the case you do not insist on the use of technically sophisticated VP solutions requiring for instance virtual reality headsets or high bandwidth Internet connections as discussed above. But, for instance CASUS, the VP system in use to host the iCoViP collection, requires for execution a plain web browser and works across all common operating system platforms. This is also true for several other VP platforms such as OpenLabyrinth or DecisionSim.

Another way of reducing costs is to collaborate and share both resources and experiences. The history of virtual patients is full of examples of initiatives in which universities from one or several countries team up to jointly create, repurpose or support themselves in using virtual patients. Good examples could be projects like CLIPP [Fall 2005], NetWoRM [Kolb 2007], eViP [Poulton 2011], the MEFANET network [Majernik 2016] and many more. Of course, the iCoViP project in the context of which these guidelines and the virtual patient collection have been developed, being a collaboration of six universities, is also an example of a shared effort. The results of such consortia are often publicly available on the Internet for several years after the project has ended. So the next tip is to search online for free content and tools. If not found, maybe you can reduce the cost of developing virtual patients by sharing the effort with partners in a project or network you join.

You can also learn the know-how in a cost-effective way from the experiences of other universities. Dewhurst et al. describe for instance how virtual patients have been implemented at several universities in Malawi [Dewhurst 2009]. The key to success was a series of workshops organized by visiting instructors from Scotland for those who just started the use of virtual patients in Africa. Another example of a long lasting international collaboration is the MEFANET initiative of Czech and Slovak universities who organize regular annual meetings to discuss experience in using e-learning resources, including virtual patients [Majernik 2016]. Kolb et al. reported about an international collaboration with many European universities in the area of virtual patients for occupational medicine [Kolb 2007]. MedBiquitous is a US-based international organization to support technical standards and good practice in the field of computer-aided medical education. In order to save time and money you should avoid doing repetitive work for example by joining a community that shares the same goals related to virtual patients as you have.

Some of the work can be completed in collaboration with residents and students who can help in development, translations and integration of virtual patients with minimal supervision of experts. For instance, in the project NetWoRM LA Students from Germany annually visited partner universities from Latin America (Brasil, Chile) to create virtual patients in collaboration with local experts. It turned out that those virtual patients were more interesting and motivating for users than VPs created by teachers [Radon 2011]. Some iCoViP partners (e.g. Jagiellonian University) collaborated closely with students while developing and translating VPs. This has significantly accelerated the project. For students, collaboration in such initiatives is rewarding as a learning by teaching scenario, an opportunity to collect international experiences and to improve intercultural understanding.

The tools you use to support teaching with virtual patients – for instance an e-portfolio system, tools for creation of concept maps or discussion boards do not need to be a commercial solution. There are several open source or freely available tools like Cmap, Moodle, Mahara or Padlet in use that can be used to support activities. If you need to enrich the existing virtual patients with new multimedia instead of developing those from scratch, you can also look for free images and videos under Creative Commons license (e.g. on Flickr) or in low cost stock photography services (e.g. Shutterstock).

Finally, some of those who complain about the costs of virtual patients forget that the alternative to virtual patients also costs money, sometimes more than what is required for virtual patients. As discussed in other parts of this guideline, medical curricula are often already filled to the brim and the introduction of VPs is only possible when some other activities are replaced. The money that is saved by discarding or reducing time in the curriculum of one type of education can be relocated to the new type. Traditional education also requires classrooms and teacher time for preparing and conducting the classes. The advantage of online resources is that after an initial investment to implement the resources, they can be used by many students. Virtual patients have already been used in Massive Open Online Courses (MOOCs) with close to 20.000 enrolled students [Kononowicz 2015], and with minimal scaling-up costs. You should take into account that alternatives to virtual patient simulation modalities are very expensive. For instance, the time of simulated patients is limited and requires training, salaries and on top of that the experience cannot be shared by many students simultaneously. In another study conducted by Haerling, it was shown that the cost associated with mannequin-based simulation is three times higher than for virtual patients without difference in terms of improvement in knowledge and satisfaction [Haerling 2018].

Recommendations:
– Share the effort of developing VPs by collaboration with other universities or reusing free available VPs as the one developed by the iCoViP project.
– Collaborate with residents and students on developing VPs with mutual benefit.
– Some costs are unavoidable while using VPs – like maintenance of the technical infrastructure, technical support, and content updates. But you can save money by lowering expectations on rich multimedia and sophisticated technical features with minimal impact on educational outcomes.
– Remember that other teaching methods, alternative to VPs, also cost money that can be saved when you decide to use VPs.

Phase of the curriculum	4th year of medical school
Goal in the curriculum	The iCoViP collection was implemented to practice selected VPs in the subject of Occupational Medicine (a voluntary subject in the University of Zaragoza in the 4th year of medical school).
Effective use of resources	The collection was used asynchronously so it was not necessary to have a computer infrastructure or a specific room and all students in Spain have laptops or access to the medical school computers. However, issue of updating content not yet solved.
VP alignment	The collection is used while students are doing their clinical placements having a teacher revising the responses and explaining to students most frequent mistakes.
Prioritization/relevance	For Occupational Medicine, the use of the iCoViP collection was a mandatory assignment needed to pass, but did not add points to the final grade.
Relation to other learning activities	None
Time allocation	One week was given to complete the assignment but more time was available for those who wanted to do the activity remote from home.
Group allocation	Students worked alone at home, when the educator discussed the solution of the VP, it was allocated in class (face-to-face or online).
Presence mode	Asynchronous
Technical Integration	It was integrated into Moodle.
VP use orientation/training	VPs were related to a specialty of medicine, but were randomly assigned to students regardless of where in the specialty they were rotating at the time.
Technical infrastructure	We used the technical infrastructure of the iCoViP project without any additions at the University of Zaragoza.
Learning activities around VPs	The teacher explained each VP and how to resolve them after getting the responses of all the students.
Assessment	Students had to complete the VPs, but there was no grade at the end. Only credit (pass/fail).
Quality assurance, maintenance, and sustainability	Frequent revision of the VPs by medical doctors.

Phase of the curriculum	Year 3 and 4 of medical school
Goal in the curriculum	Students learn basic steps of clinical reasoning including identifying relevant findings, developing differential diagnoses, deciding about a final diagnosis, ordering tests to rule out / confirm differentials, and suggesting treatment options.
Effective use of resources	Course tutors needed time to familiarize themselves with the VPs and time had to be planned during the synchronous meetings to discuss the VPs
VP alignment	VPs were part of the modules (e.g. Abdomen, cardiovascular system, pulmonary system, etc.) in year 3 and 4 of medical school and aligned with the objectives of these modules based on key symptoms. In addition a pool of 41 VPs was available for deliberate practice across key symptoms and diagnoses.
Time allocation	5 VPs / module
Group allocation	Students could choose whether to work in groups or individually.
Presence mode	Students could decide when and where to work on the VPs during the period of the module.
Technical Integration	VPs were integrated into the school's learning management system Moodle via a SingleSignOn interface.
VP use orientation/training	No specific familiarization, but general introduction at the beginning of year 3.
Learning activities around VPs	Depending on the modules other learning activities were embedded.
Assessment	The topics of the VPs were part of modules assessment.
Quality assurance, maintenance, and sustainability	We used the built-in feedback functionality to receive qualitative feedback from students and VPs were part of the regular evaluation activities of the medical school.

Phase of the curriculum	Year 1 and 2 of medical school
Goal in the curriculum	Students learn basic steps of clinical reasoning such as identifying & prioritizing findings and composing a summary statement. They also can follow the reasoning process of the VP author concerning differential diagnoses, ordered tests, and treatments.
Effective use of resources	Course tutors needed time to familiarize themselves with the VPs and time had to be planned during the synchronous meetings to discuss the VPs.
VP alignment	VPs were part of the longitudinal clinical course and aligned with the other modules in year 1 and 2 (In case of Augsburg this was Contact, Movement, and equilibrium). We aligned the key symptoms of the VPs with these modules, so that students worked in parallel on these VPs and the corresponding module.
Time allocation	15 VPs over two years / 5 VPs per module.
Group allocation	Students could choose whether to work in groups or individually.
Presence mode	Students could decide when and where to work on the VPs during the period of the module.
Technical Integration	VPs were integrated into the school's learning management system Moodle via a SingleSignOn interface.
VP use orientation/training	At the beginning of year 1 students were introduced into clinical reasoning and how they can train this ability with VPs.
Learning activities around VPs	Depending on the key symptoms during the longitudinal clinical course other learning activities were embedded.
Assessment	The topics of the VPs were part of the clinical longitudinal course assessment.
Quality assurance, maintenance, and sustainability	We used the built-in feedback functionality to receive qualitative feedback from students and VPs were part of the regular evaluation activities of the medical school.

Phase of the curriculum	Basic sciences/pre-clinical years: - At Jagiellonian University Medical College in Kraków we have integrated it in the “Introduction to Clinical Sciences” course in 2nd year of medicine. - At University of Porto in the “Propedeutics/Semiology” course in the 3rd year of medicine.
Goal in the curriculum	- To provide the students with an opportunity to challenge their knowledge at home in between classes with an interactive, clinical-oriented task. - To support learning by linking basic science knowledge with clinical reasoning. Students had the opportunity to practice on undiagnosed cases the skill of differentiating common symptoms, such as dyspnea, abdominal pain, headache.
Effective use of resources	Use of iCoViP VPs in native language of the students. No extra cost needed.
VP alignment	- In Kraków we have selected a few VPs (seven) with common diseases (e.g. pneumonia, pancreatitis, pulmonary embolism) with common symptoms. - In Porto, VPs are chosen according to the common symptoms to promote clinical reasoning.
Prioritization/relevance	- In Kraków the completion of all VPs is mandatory. - In Porto, VPs are introduced on a voluntary basis and are available on demand.
Relation to other learning activities	We used the time students had at home between seminars (Kraków: between on-campus based PBL sessions). VPs were spaced-activated (a new VP appeared biweekly synchronized with changing topics of the PBL seminars that focus on different leading symptoms).
Time allocation	Around 30 minutes biweekly, repeated 7 times in a semester.
Group allocation	Students worked alone at home to reflect but could consult the VPs with their peers or instructors in small groups of the face-to-face PBL seminars.
Presence mode	Students worked on the VPs asynchronously and self-directed at home to have time to reflect and consult textbooks.
Technical Integration	VPs were integrated with a course to which all students were enrolled on the official university learning management system Moodle using Learning Tools Interoperability (LTI) interface.
VP use orientation/training	Students were provided with an introductory email with instructions and had additionally the opportunity to technically practice using CASUS in a parallel “Telemedicine” (Medical informatics) course. Instructors received test-codes to practice the VPs at home.
Technical infrastructure	Students used their own computers from home. An email address was provided to a person responsible for technical support.
Learning activities around VPs	Students were asked to complete concept maps for all VPs they solve. Moreover, in Kraków students were provided with links to additional online articles to help them with topics difficult at this stage of education.
Assessment	Students were asked to complete the cases prior to the end of the term. They were not given grades for the activity - just credit. Their answers were randomly inspected to see common mistakes and provide general feedback to all students.
Quality assurance, maintenance, and sustainability	We checked the diagnostic accuracy of individual cases (and detected one case with imprecise diagnosis). Students evaluated the course using the iCoViP case collection evaluation questionnaire.