The role of 3D printed heart models in immediate and long-term knowledge acquisition in medical education

Background: The long-term effect of three-dimensional printed heart model (3DPHM) on knowledge acquisition of congenital heart disease (CHD) remains unknown. This prospective cohort study aims to investigate the role of 3DPHM in improving immediate knowledge gain and long-term knowledge retention on CHD among the medical students. Methods: Fifty-three second and third year medical students were assigned into two groups to compare their immediate knowledge acquisition and knowledge retention after an education session on anatomy and pathophysiology of CHD. During the 1.5 hour-long education session, both the control (n = 25) and study groups (n = 28) had access to identical teaching materials: digital 3D heart models, 2D diagrams, and medical images, except for 3DPHMwhich were only used in the study group. The immediate knowledge gain was assessed via an online quiz, whereas the long-term knowledge retention was assessed using another quiz in 6-weeks’ time post-intervention. A survey was also conducted to evaluate the participants’ learning experience. Results: There is no significant difference in the immediate knowledge acquisition and long-term knowledge retention between the groups (U = 272, p = 0.16 and r = –0.143, p = 0.15 respectively). Majority of the students (96% in control group and 85% in 3DPHM group) responded that the 3DPHM would have/had improved their learning experience. Despite that, there is no significant difference in the self-perceived knowledge improvement between the groups. Conclusions: This study concludes that the 3DPHM do not significantly improve both immediate knowledge acquisition and knowledge retention among the medical students. However, further research with larger sample size, as well as categorizing the type of questions in the quiz, is needed to better assess the role of 3DPHM in different educational components.


Background
Congenital heart disease (CHD) is the most common birth defect among the newborn [1][2][3]. It has a broad spectrum of severity and morphology that is largely variable between individuals [2]. A strong foundation for education in CHD is therefore crucial among the medical students and young clinicians for the development and training of future cardiac specialists. Current teaching approach of heart anatomy and physiology relies heavily on twodimensional (2D) medical images and cadaveric specimens, both of which have their own limitations and shortcomings. 2D medical images, or even three-dimensional (3D) digital reconstruction of heart presented on 2D plane requires the viewer to conceptually translate the information into 3D to understand the spatial relationship between the inter-and intra-cardiac structures. This is extremely challenging, especially for beginners who only just learn about CHD [1][2][3][4][5]. Cadaveric specimens are often inaccessible to students, expensive to maintain, and subjective to wear and tear overtime. Additionally, depending on the availability, the cadaveric specimens only demonstrate certain types of CHD (i.e., lesion-specific), and therefore cannot represent the whole spectrum of CHD [1,3,[6][7][8].
3D printed heart models (3DPHM) have shown to be a novel teaching approach that can improve students' and health providers' learning experience and satisfaction [9]. The 3DPHM were reported with the advantages of being easily accessible, affordable, and the ability to reproduce any type of lesions, including those which are rare and more complex [1,3,7,8]. Being tangible, the students can also assess the cardiac morphology up close at any viewing angle, achieving deeper understanding of the cardiac morphology without the need to master the knowledge of cardiac imaging [8]. A case-control study by Tan et al. [5] which involved 132 nursing students, reported that the 3DPHM group scored significantly higher in both objective and subjective evaluations. The study concluded that the 3DPHM were useful in improving the students' knowledge acquisition on atrial septal defect (ASD) as well as students' interest and satisfaction in their learning [5]. In a recently published randomized controlled trial which involved 5th year medical students, a significantly higher increase in test scores was observed in the 3DPHM group compared to the control group, regardless of the type of CHD being tested [9]. Similarly, in another study which involved 127 participants, Valverde et al. [10] reported that the use of 3DPHM have significantly improved the anatomical knowledge of criss-cross cardiac anatomy, compared to the use of imaging datasets as a teaching tool.
Despite these promising results, according to a recently published meta-analysis and systematic review, the use of 3D printed cardiac models do not necessarily increase the short-term knowledge gain among the students, even though their learning experience was enhanced [11]. To the best of our knowledge, none of the existing studies investigated the effect of 3D printed heart models on longterm knowledge gain among the students in the domain of CHD. This study aimed to investigate the immediate knowledge acquisition as well as long-term knowledge retention among the medical students with the use of 3DPHM incorporated in the teaching of CHD, compared to the conventional teaching approach.

Study design
A prospective cohort study was conducted to compare the knowledge retention on the topic of CHD between the medical students who were exposed to conventional teaching approach versus those who were exposed to teaching approach with the use of 3DPHM. The study was approved by the Curtin University Human Research Ethics Committee. This research was advertised twice in first semester 2020 and first semester 2021 respectively in the Perth Campus, Curtin University. The participant recruitment was primarily limited to second and third year medical students in the year 2020, and only second year group in the year 2021, as these were the groups who were considered as novice in cardiac anatomy and pathology.
From both of the recruitments, a total of 53 medical students (5 in their 3rd year, and 48 in their 2nd year) voluntarily participated in the study. There were 5 education sessions on the topic of CHD carried out each year, with a duration of 1.5 hour for each of them. The participants were required to attend one of them based on their availability. A maximum of 9 students were allowed per session. To minimize potential bias, the participants were not informed if they were in the control or study group. There were 28 students who attended the education sessions with the use of 3D printed heart models (study group), and 25 students attended the education sessions without being presented with the 3D printed heart models (control group). All 53 medical students had previously been taught of CHD during their curriculum.
In order to assess the difference in knowledge gain and knowledge retention between the groups, the students completed two sets of quizzes which comprised of 20 questions relevant to CHD. The first set was completed immediately following the education session (immediate knowledge gain), while the second set was completed 6 weeks after the education session (knowledge retention). They also completed a survey to rate their learning experience. Ap-preciation gifts were given to the study participants at the completion of second quiz to prevent drop-outs.

Generation of 3D printed heart models
In order for the participants to learn a range of CHD with different levels of complexity, four types of CHD were selected for generation of the 3D printed heart models based on their Aristotle Basic Complexity Level (ABCL). These include ASD (ABCL = 1), ventricular septal defect (VSD, ABCL = 2), Tetralogy of Fallot (ToF, ABCL = 3), and double outlet right ventricle (DORV, ABCL = 4). Anonymized cardiac computed tomography angiography (CCTA) images of four pediatric patients with the aforementioned CHD were used as the source data.
The first step to generate the 3DPHM was to convert the CCTA images into printable digital model in standard tessellation language (STL) format using Mimics Innovation Suite 22.0 (Materialise HQ, Leuven, Belgium) ( Fig. 1). The details of the conversion process were elucidated previously [12,13]. The required time for this conversion process was 45 minutes on average, however it was highly dependent on the image quality of the source data. Following that, the STL files were sent to the 3D printers for printing ( Fig. 1, Ref. [14]).
Two sets of four models were generated, each in a different printing material. One set was printed in Flexible V4 Resin from Formlabs (Somerville, MA, USA), while another was printed in TPU80A from Fabbxible Technology (Pulau Pinang, Malaysia) ( Fig. 2, Ref. [15]). Both printing materials are able to generate models that are flexible, which to some extent resemble the physical properties of heart muscles. Each cardiac model was divided into two components (transected at the right atrium-right ventricular plane) so that the structural defects of the heart can be clearly visualised. The average printing cost per model was approximately AUD 35.

Education session on CHD
The education sessions were designed to mainly focus on anatomy and pathophysiology of ASD, VSD, ToF, and DORV. For this purpose, 3DPHM, digital 3D heart models, 2D diagrams, and digital imaging and communications in medicine (DICOM) images were utilized as the teaching tools (Figs. 2,3). The education sessions were carried out in PowerPoint lecture format, incorporating the aforementioned teaching tools with each type of CHD presented. The session was designed to mainly provide the students with a basic understanding of the morphology and hemodynamics of each type of CHD and their imaging appearances on CCTA. Each education session ran for 1.5 hour by the same investigator (I.L.) to avoid variations in teaching style between different tutors. In order to ensure the consistency between each education session, both the study and control groups received the identical learning content, apart from the 3DPHM that were only used in the education sessions  for study group.
The education sessions took place in a computer laboratory. At the start of the session, the students were asked to download the digital teaching materials (i.e., DICOM dataset and digital 3D heart models), so that all of the participants would be able to access them on their own computer while listening to the tutor's explanation. After giving the students a quick revision of normal heart anatomy and gen-eral overview of acyanotic and cyanotic CHD, the session was followed by explanation of each type of chosen CHD in the following format and progression: (i) explanation of hemodynamic change using a 2D diagrams, (ii) subtypes of each CHD, (iii) possible complications if the CHD is left untreated, (iv) case study using DICOM images, digital 3D heart models, and 3DPHM. While the tutor was demonstrating the digital 3D heart models and the DICOM images on the projected screen, the participants were also encouraged to view and manipulate the digital 3D heart models and the DICOM images on their own computer using opensource software, Meshmixer (Autodesk, San Rafael, CA, USA) and RadiAnt (Medixant, Poznan, Poland). The digital 3D heart models were coloured according to the type of blood (oxygenated, deoxygenated or mixed) that the heart chamber is carrying to enable the students in achieving better understanding of the hemodynamic changes due to the CHD (Fig. 3). Using Meshmixer, the participants were allowed to rotate and zoom in the 3D models virtually. For the education sessions in which 3DPHM were used, each set of 3DPHM was shared among 3-5 students, and they were encouraged to refer to the 3DPHM freely throughout the session. 3DPHM were used to pinpoint the location of heart defects, to explain the hemodynamic changes, to highlight the malpositioning of the great arteries (if present), and to emphasize the difference in size between the major blood vessels, especially when a stenosis is present.

Assessment of participants' knowledge acquisition and learning experience
Immediately after the education session, the participants were asked to complete a set of online quiz (Quiz 1) which comprised of 20 multiple choice and short-answer questions on CHD, with a time limit of 15 minutes. The participants were also asked to complete a survey to rate their learning experience based on a 5-point Likert scale. In 6 weeks' time after the education session, the participants completed another online quiz (Quiz 2) of 20 questions, again with a time limit of 15 minutes. The score (out of 20) and the time taken for the participants to complete both of the quizzes were recorded. The process of the study is summarized in Fig. 4.
Both the online quizzes were designed to ensure that the students were tested on the content covered in the education session, whereas the survey was designed to address the students' learning experience with the teaching tools used and their self-rated knowledge acquisition. The questions for Quiz 1, Quiz 2, and the survey can be found in Supplementary files 1-3 respectively.

Statistical analysis
Data analyses were performed using IBM SPSS statistical package, version 26 (IBM Corp, Armonk, NY, USA). Shapiro-Wilk's and Levene's test were used to analyze the normality and homogeneity of the variance. Student t-test was used to analyze the score of anatomy unit that the students achieved in Year 1 between two groups to compare their baseline knowledge on anatomy. The following data were being analyzed using Mann-Whitney U test: score of Quiz 1 between two groups to compare the immediate knowledge acquisition of the participants, and survey responses for the students' learning experience. Linear regression was conducted to analyze score difference between Quiz 1 and Quiz 2 of each participant to measure the knowledge retention among the medical students, using the type of group (control or study) and year of recruitment as predictors. It is hypothesized that the 3DPHM group will experience smaller score difference between the two quizzes when compared to the control group. Pearson's correlation was also used to measure the strength of correlation between the predictors and the outcome variable. p-value of <0.05 (two-tailed) is considered statistically significant.

Results
Normality of test scores were met by Shapiro-Wilk's test for Quiz 2 score (p = 0.25 and p = 0.08, respectively), but not for Quiz 1 score (p = 0.014). Both quiz scores met the assumption of homogeneity of variance by Levene's test (p > 0.05).
The Quiz 1 score was marginally higher in the 3DPHM group, with a median score of 17.5 and compared to median score of 16 in the control group, however Mann-Whitney U test demonstrates no statistical significance (U = 272, p = 0.16). Fig. 5 presents a boxplot of the scores achieved by both student groups in Quizzes 1 and 2.
The 3DPHM group experienced slightly higher score difference between Quiz 1 and Quiz 2, with a mean score difference of 7.79 (±2.63), compared to the control group, 7.04 (±2.64). Fig. 6 illustrates the mean score differences between Quizzes 1 and 2 of both control and 3DPHM groups. The Pearson's correlation between the score difference and type of group and year of recruitment was very weak and not significantly different, r = -0.143 (p = 0.15) and r = 0.043 (p = 0.379), respectively. The fitted regres- sion equation for predicting the score difference of the two quizzes from the predictors was ŷ = -7.137 + 0.220χ -0.743z, where ŷ is the score difference of Quiz 1 and Quiz 2, χ is the year of recruitment (coded as 0 = year 2020, 1 = year 2021), and z is the type of group (coded as 0 = control group, 1 = 3DPHM group). The r 2 for the fitted regression was 0.022; that is, only 2% of the variance in score difference was predictable from the predictors. In other words, the result does not suggest improvement in long-term knowledge acquisition within the 3DPHM group when compared to the control group.
In comparing survey responses between the control and 3DPHM groups, the Mann-Whitney U test showed no significant difference in all of the questions. Figs. 7,8 illustrate the survey responses in percentage for the control group and 3DPHM group, respectively. Both groups of students indicated either 'strongly agree' or 'agree' that the teaching method was interactive and interesting (U = 346.5, p = 0.93). Slightly more students in 3DPHM group indicated 'strongly agree' that the education was easy to understand (82%) compared to the control group (76%), although this is not statistically significant (U = 331.5, p = 0.64). More students in 3DPHM indicated 'strongly agree' that their knowledge about CHD has improved (86%) compared to the control group (68%), and again this is not statistically significant (U = 292, p = 0.16). About 64% of the students in 3DPHM group indicated 'strongly agree' that the 3DPHM played an important role in helping them understand CHD. This is similar to the control group where 60% indicated 'strongly agree' that the 3DPHM would help them to understand CHD (U = 349.5, p = 0.99). However, one student in the 3DPHM group disagreed that the 3DPHM helped in his/her learning of CHD.

Discussion
This cohort study has evaluated the value of 3DPHM in both immediate and long-term knowledge acquisition among the medical students on the topic of CHD when compared to the conventional teaching tools. Even though there are published articles which assessed the role of 3DPHM in immediate knowledge gain on the topic of CHD, the present study is the first to investigate the effect of 3DPHM in longterm knowledge retention in the domain of CHD.
The results showed that the 3DPHM group scored slightly higher in Quiz 1, which indicated improvement in immediate knowledge acquisition compared to the control group. This difference however, did not achieve statistical significance. This finding echoes the finding in Jones et al.'s [16] study, in which 36 residents were divided into 3D printed model group and control group with their pretest and post-test scores analyzed. In their study, the 3D printed model group experienced slightly greater improvement in the post-test compared to the control group, how- ever the difference was not statistically significant [16]. In another study conducted by Loke et al. [2], the 3DPHM group did not show improvement in the immediate knowledge gain among the pediatric residents. Despite these positive reports, there are other studies documenting contrasting findings. In a study which involved 63 medical students, the 3DPHM group was reported to have a significant improvement in the test results when compared to the control group [17]. In another study by Lim et al. [7], 53 first year medical students were randomly assigned into three groups (cadaveric materials, 3DPHM, and combined materials) and were objectively tested in pre-and post-test for their knowledge acquisition on external cardiac anatomy. It was reported that the 3DPHM group had significant improvement in test scores (p = 0.003) [7]. This is not a surprising finding. In a meta-analysis conducted by Yammine et al. [18], physical models were reported to significantly improve the overall knowledge outcome and the spatial knowledge acquisition, but not the factual knowledge acquisition. In the present study, we focused on the topic of CHD using 3DPHM which feature 4 different types of CHD. Even though the result did not reach statistical significance, which likely was due to the small sample size, the role of 3DPHM in improving short-term knowledge gain on CHD should not be overlooked, as the effect of 3DPHM on immediate knowledge acquisition appears to vary with the complexity of the CHD. In the study by White et al. [19], residents were objectively tested on two types of CHD of different complexities to explore the usefulness of 3DPHM in improving residents' knowledge on VSD and ToF. They were divided into 3DPHM and control groups for both VSD and ToF lecture. It was reported that the 3DPHM group scored higher in the ToF post-test (p = 0.037), but lower in the VSD post-test (p = 0.02) [19]. Similarly, in another study by Smerling et al. [3], first year medical students were subjectively tested to assess their perceived knowledge acquisition using 3DPHM of different lesion complexity. It was reported that with the use of 3DPHM, as the complexity of CHD increases, the mean knowledge also increases [3]. This however, was not validated in our study, which could contribute to the findings of no significant difference. This limitation could be addressed in future studies.
Surprisingly, the 3DPHM group performed slightly inferior to the control group for long-term knowledge retention, even though this was not statistically significant. This is in contrast to another study by Lombardi et al., in which plastic heart models were compared to organ dissection of a sheep heart and virtual dissection using physiology software program, to assess the students' immediate knowledge acquisition and long-term knowledge retention on normal heart anatomy and physiology. It was found that the plastic model group performed significantly better during the initial exam and the 2-month follow-up exam [20]. This difference in findings might be due to the difference in methods of lecture being carried out in both studies. In Lombardi et al.'s [20] study, the students in plastic model group were allowed 45-minute of hands-on activity with the models as well as within-group discussions after 15-minute of Pow-erPoint lecture; whereas in our study, the students in the 3DPHM group learned individually and had full 1.5 hour of PowerPoint lecture with different teaching tools incorporated throughout the sessions. The amount of time that they spent on the models might not be enough. On the other hand, the study by Lombardi et al. [20] only focused on normal heart anatomy and physiology, whereas our study focused on CHD, which could justify the difference in findings.
As for the survey responses, although not statistically significant, greater percentage of students in 3DPHM group responded that the lecture was easy to understand, and that their knowledge on the topic of CHD has improved. In Loke et al.'s [2] study, the 3DPHM group rated significantly higher satisfaction score towards the teaching session, when compared to the control group. On the other hand, a meta-analysis found that the 3DPHM is associated Fig. 6. Mean changes in score between Quizzes 1 and 2 (Quiz 2 score -Quiz 1 score) for both control and 3DPHM groups. 3DPHM, three-dimensional printed heart model. with higher subjective evaluation scores and satisfactory level among the study participants [9]. This is also reflected in our study in which majority of the students responded (96% in the control group and 85% in the 3DPHM group) 'agree' or 'strongly agree' that the 3DPHM had helped them to gain a deeper understanding of CHD. Hence, the 3DPHM has contributed to improving students' learning experience.
It is worthwhile to report that in the survey when the students were asked to provide additional comments, some of them mentioned that the coloured digital 3D models were helpful for them to understand CHD compared to the CT images (total of 9 from the control group and 4 from the 3DPHM group). This is not surprising. As a novice, the students may not have the ability yet to mentally reconstruct the heart anatomy in 3D views based on the 2D images, not to mention when it comes to CHD when the heart mor- phology is more complex than the normal anatomy [2,4,21]. Hence, the digital 3D models were helpful for the students to visualise the heart defects in 3D, and the colour on the digital 3D models were helpful for them to understand the hemodynamic changes in the heart for each type of CHD. In fact, a recently published study by Liddle et al. [22] found that the use of digital 3D heart models over tele-consultation had significantly improved the CHD patients' knowledge about their disease as well as their cardiac surgical history. This has demonstrated the value of digital 3D heart models as an alternative or complementary teaching tool in medical education, especially when physical meeting is not possible during coronavirus disease 2019 (COVID-19) pandemic. In light of the pandemic, virtual education had formed an integral part of medical education [23]. While there are limited publications on virtual teaching of CHD, our group had compared the value of 3DPHM and virtual reality (VR) in medical teaching of CHD based on the opinions from radiologists, sonographers, and radiographers. It was found out that the VR heart models had comparable benefits with 3DPHM in medical teaching of CHD [15]. This is another area that can be explored further in future studies to investigate the use of virtual models in medical education.
This study has some limitations. Firstly, there were no pre-test done to test the students' actual baseline knowledge on CHD, hence it remains unknown that if both groups are comparable. Secondly, the selection criteria for the study participants could not exclude those who had pre-existing tertiary education, which could affect the results to some extent. Thirdly, the amount of time that each student in the 3DPHM group spent on assessing 3DPHM was not controlled. Even though they were encouraged to share the 3DPHM among themselves, some students may not spend as much time on the models as others, therefore reducing the level of benefits they could gain from the 3DPHM. Fourthly, both Quizzes 1 and 2 are a mixture of questions on factual and spatial knowledge on CHD, which might be too generalised to evaluate the actual effect of 3DPHM on CHD knowledge acquisition. It is suggested that future studies should group the questions into different educational components, for example, knowledge acquisition and structural conceptualization [17], to investigate the effect of 3DPHM on these two components separately. Additionally, in the last question of the survey, the control group was being asked speculatively about their opinion on 3DPHM. This could have led to bias in the results, hence should be interpreted with caution. Lastly, it is unknown if the students had done any extra studies or revision on CHD during the 6-week's duration between Quizzes 1 and 2, which could have affected their score in Quiz 2. Future similar studies should take this confounding variable into account.

Conclusions
The results of this study suggested no significant improvement in CHD knowledge acquisition and knowledge retention with the use of 3DPHM compared to the conventional teaching methods. The 3DPHM did not significantly enhance the self-perceived knowledge improvement either among the students when compared to the conventional teaching methods. However, the positive benefit that the 3DPHM could bring to enhance the students' learning experience should not be overlooked. Future research should include larger number of sample size, and categorize the quiz questions into different educational components, allowing measurement of the true effect of 3DPHM on knowledge acquisition of different aspects of CHD.

Author contributions
IL and ZHS designed the research study. IL performed the research investigation. IL analysed the data. ZHS provided supervision of the study. All authors contributed to editorial changes in the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate
The study was approved by the Human Research Ethics Committee of CURTIN UNIVERSITY (approval number: HRE 2017-0138). All participants signed inform consent prior to participating in the study.