The Factors Affecting The Utility Of Problem-Based Learning As An Educational Method
Introduction and Background
Problem-based learning (PBL) is an instructional, educational method that promotes, active, collaborative learning, based on the constructivist learning theory (Vygotksy 1980). PBL uses a problem as a trigger for students to create their own learning objectives. Since its introduction in McMaster’s University in Canada, it has been widely used either to replace, or in conjunction with traditional didactic methods to facilitate student learning, especially in medical education. Evidence suggests that PBL enhances long-term knowledge retention and skill-based assessment compared to traditional methods (Strobel and van Barneveld, 2009) and enhances physician competency especially in the cognitive and social domains (Koh et al, 2008). Within the Oxford deanery, PBL is not utilised as an educational tool. In my opinion, there are noticeable differences in terms of knowledge application of junior doctors trained using the PBL system versus those using traditional methods. We routinely have medical students attached to our firm for clinical exposure. Using PBL in those few weeks to enhance their exposure to other educational strategies may be advantageous to their development as learners, and future doctors. As PBL is a resource and time-intensive process in an already time-pressured clinical environment, it is important to be aware of the factors that may influence its utility so that student and clinician time is spent frugally. I conducted initial background reading on PBL and the factors deemed to affect its effectiveness. I then identified factors that would be useful to be aware of and/or modifiable in my clinical context and used these factors to direct my literature review.
To identify relevant studies, I searched the following databases: MEDLINE (1943), EMBASE (1974), ERIC (1999), BEI (1998), AMED (1985), PubMed and the Cochrane Databases (2005) through till 5th October 2018. Advanced search technique was used to identify papers with “problem based learning” in the “Title” in conjunction with the following words separately: “tutor”, “effective”, “gender”, “learning styles”, “ personality”, “stress”, also in the “Title” field. A snowball sampling technique was used to identify other papers from references that may be useful to include. The literature review will address factors that affect the utility of PBL. These will be subcategorised into Student factors ( Personality, Learning styles) and the PBL process (Case, Gender of participants, Tutor). When identifying literature surrounding the “case” and “tutor”, inclusion criteria of PBL in undergraduate medical students was used. No such restriction was placed when studying gender interactions/personality/learning styles. Similarly, no restrictions were placed on country, date or study design.
Student Factors
Personality
There are many models use to categorise personality. One such was proposed by Cloninger, Svrakic, and Przybeck (1993) which consists of four temperaments (novelty-seeking, harm avoidance, reward dependence and persistence) and three character dimension (self-directness, co-operativeness and self-transcendence). Jang & Park (2016) evaluated the personalities of eighty medical students in Korea using the validated Korean version of Temperament and Character Inventory (TCI-RS). Regression analysis of peer evaluated scores of four aspects reflecting collaborative behaviour (participation, preparedness, communication and contribution) revealed that co-operativeness, although postulated to be an advantageous characteristic did not show any relationship with the features of collaboration, a finding supported by El Skeikh et al (2014). Perhaps, this link would be present in situations, where the case lends itself to more diversity in opinion. Features of collaboration were seen more in individuals who are not reward dependent, implying intrinsically motivated, and “persistent”.
Although a propensity for collaborative behaviour did not translate to better academic performance in those who were reward independent, one could argue the use of peer evaluation as a methodology and a small sample size, limit the study reliability. Ethnicity was not accounted for in this study, which, in this context, may have impacted peer ratings as well as the students’ motivation to participate actively. Holen, A et al (2015) identified through their study of 449 students that extraversion and openness to experience were related to a preference for PBL. Neuroticism and conscientiousness were related positively and negatively to PBL preference. The authors propose that whilst neuroticism may promote anxiety in some, the interactive nature of PBL may give others social support. Similarly, being too conscientious may promote stress. Replacing the cross-sectional, survey-driven methodology by a longitudinal approach looking at changes in preferences over the course and relationship to academic achievement may have been useful. Furthermore, the results from the hybrid PBL environment in these centres may not be reflective of a pure PBL course. Lewis et al (2009) concluded that PBL courses can have additional stressors to non-PBL courses.
Therefore, personality traits more prone to stress should be recognised early. McManus, Keeling and Paice (2004) and Tyssen et al (2007) found that high levels of neuroticism, low levels of extraversion and low conscientiousness in combination, could lead to stress. A large sample size of 1668 in the study by McManus, Keeling and Paice (2004), coupled with 12 years of follow up provide validity to the results. Lievens et al (2002) showed conscientiousness to be an important trait for academic success, even after adjustment for possible selection bias due to high attrition rates between year one and three. This was partly supported by Ferguson et al (2003), who concluded that conscientiousness was a positive predictor for academic performance in the first three years, but negative predictor thereafter. Although the use of different personality scales makes direct comparison difficult, one possible speculation is that students had lost the protective effect of extraversion and were victims to stress in the latter years (Tyssen et al, 2007).
Learning Styles
Alghasham (2012) identified first year medical students with reflective and active learning styles and showed that active and reflective learners gather information differently: using multiple activities compared to multiple sources, respectively. As expected in the active learning environment created by PBL, active learners were better at all aspects of active participation. This contradicts findings from Hur & Kim 2007, where active learners did not outperform reflectors in any category. Academic scores were not affected by either learning style (Alghasham 2012). This is contrast to O’Toole (2001) which showed students with activist learning style had a negative correlation for the PBL module in year three.
This study also showed that the learning styles remained stable over the three-year follow-up and scores for activist learning styles were strengthened over the time period. The reliability of these results are questionable. The use of first year undergraduates in Alghasham’s study, who were not only PBL-naïve but new to the University learning environment, may have impacted the results. Furthermore, the sample size was only 24 in the study by O’Toole (2001). None of the studies studied the impact of socioeconomic/ race/ cultural factors– all of which can affect learning style. From a preferences standpoint, Pungente, Wasan and Moffett (2002) showed that, “Assimilators” (Kolb 1984) had a preference for PBL activities, whereas “Divergers” had the least preference. Although a good sample size of 116, the use of survey data and the abstract wording when using the Likert scale could have influenced the results. Very few studies were found studying PBL and learning styles. This coupled with the different learning tools and weak methodology poses difficulties in drawing firm conclusions.
PBL Factors
Case structure
Case structure is important in developing PBL scenarios but there is little literature surrounding it. Kim et al (2006, p) suggest cases should be “realistic, energising, challenging and instrumental”. Azer et al (2012) advise twelve tips on creating a good PBL case, including creating a template, engaging a trigger and building on prior knowledge. Dolmans et al (1997) add to this and suggest contextualising the case, promoting self -directed learning and interest is important. Sutyak, Lebeau and O’Donnell (1998) showed that the use of unstructured cases in students improved academic scores, even in those not motivated by career choice, presumably by promoting more cognitive skills.
Gender
Studies show that women prefer “connected” learning (Belenky, Clinchy, and Goldberger, 1986) which involves deeper understanding and appreciation of others’ views. This is supported by Reynolds et al (2003), who revealed that women enjoyed the responsibility of PBL and working with others. This confers to the self-directed, collaborative approach of PBL. They denied problems with voicing their opinion, although this may have been influenced by the cohort consisting of 85% females, with 80% female tutors and several female-only groups. The bias is suggested by Krupnick (1985) who found that women spoke more if led by a female tutor. Kaplowitz and Block (1998) showed retrospectively, that women felt more comfortable in single-sex groups and their confidence remained high when the groups were then mixed. Not only did they feel this contributed to their medical education, but qualitative data analysis suggested feeling “overpowered” or “ignored” when in mixed groups.
This suggestion of power is somewhat supported by Aries (1976) who found that males addressed the whole group rather an individual more frequently than women, signifying an assertion of power or dominance, especially when in male-only groups. Interaction analysis in Aries (1976) highlighted women were more likely to take it in turns to speak and were more self-aware of talking too much. Male lack of awareness of their mannerisms is suggested by Kaplowitz and Block (1998) when men felt “hurt” and “angry” that women wanted to be in a single-sex group. Women in the study seemed to develop self-awareness of their lack of assertiveness when put in the single sex group, which they developed and maintained when put back in the mixed gender group, thus improving their confidence. The small sample size of 16 coupled with only 60% male response rate makes the study underpowered but does highlight issues which we can all relate to. Although the studies are quite dated, group dynamics affected by a gender predominance is still present, especially in male-dominated roles where women adapt to be heard. Emphasis on turn-taking and not interrupting as part of social etiquette and change in educational culture has helped eradicate some of these issues.
Role of the tutor
Davies et al (1992) concluded that tutor expertise of the case affected exam scores. This mirrored the work of Hay and Katsikitis (2001) studying medical students learning about eating disorders, where the mean score was 72. 5% when taught by an expert compared to 57. 9% when not. Davies et al found in 1994, that by using a case that was faculty- assessed as being more focussed than the case used in 1992, the effect of tutor experience on exam scores was obliterated. The definition of expertise varied in both studies: Davies et al (1992), were very specific that the expert had advanced disciplinary and/or research experience in the topic. Hay and Katsikitis (2001) identified the psychiatrist as an expert, and the psychologist a non-expert, when in reality the psychologist probably had adequate knowledge of eating disorders through her training. The significant differences in exam scores despite the assumption of non-expert, suggests that other factors are involved. Schmidt (1994) showed the effect of tutor expertise on exam scores became especially relevant when students had little prior knowledge of the topic or if the “unit” was not structured, as evaluated through student questionnaires. Dolmans, Wolfhagen and Schmidt (1996) contradicted this result and found no effect of tutor expertise on exam scores, regardless of unit structure or students’ prior knowledge.
One could postulate that a smaller sample size of 119 (compared to 1800), meant the study was underpowered. Furthermore, analysis suggested less variation in students’ prior knowledge and structure in this study, perhaps demanding less tutor compensation through expertise. Davies et al (1994) used a robust methodology for interaction analysis using the Flanders system, and ensured four hundred observations were made (every three seconds for 20 minutes). This revealed teacher-directed activity with an expert tutor occurring 25. 8% vs 11. 4% and student-initiated topics only occurring 54. 5% of the time compared to 68. 7% in non-expert led groups. This, in theory, can be deleterious to learning, significantly affecting the self-directed learning goal of PBL. Gilkison (2003) compared two tutors – one humanities, the other medical, on two PBL tutorials and showed that the medical tutor spoke more often (22. 3% vs 5. 1%).
Although this was based on only two cases at a single institute, Silver and Wilkerson (1991) supported this claim and added that the duration of speak was longer too. Interestingly, Eagle, Harasym, and Mandin (1992) found the same trend but highlighted that almost double the learning issues and more objective-congruent learning issues were generated in the expert group. This lends to the argument that expert tutors speaking more, is not necessarily worrisome, as they may be stimulating learning. Schmidt, H. G. et al (1993) showed that content expertise improved exam scores particularly in the first year of study, perhaps when students may need more direction. Full transcription and exam performance in the study by Eagle, Harasym, and Mandin (1992) would have been useful for analysis.
Conclusion
There are student and PBL process related factors that can affect a student’s likelihood of engaging and benefitting from the PBL method. The evidence suggests that some personality traits may be linked with more collaborative behaviour. Whilst certain personality types may show a preference for PBL, this may not be reflected in exam scores. Conscientiousness is linked to academic success but no data is available to ascertain its effect depending on different curricula. What is clear is that some personality traits are more susceptible to stress, which PBL can add to and we should be mindful of this. There is very little data on the impact of learning styles on PBL and weak methodology and paucity of studies makes conclusions difficult do draw. The same issues extends to designing of the case. Gender of the group participants can have an impact on group dynamics and this is important to address.
Finally, the role of tutor as a facilitator is just as important as knowledge of the topic. There is no clear consensus on the effect of tutor expertise on academic achievements, but self-awareness that content experts may reverted to didactic methods and disrupt the self-directed learning is important. More directed learning may however, be required for first year students. The data varies in quality but in most subsections is dated, of a small sample size and is limited by weak methodologies. More longitudinal data, with concurrent assessment of academic achievement (comparing short answer and multiple-choice questions) as well as interview data and changes over the course of the degree would be helpful.
Furthermore, closer assessment of other learning styles could provide interesting findings. In my current clinical context, PBL is not utilized. I would therefore introduce students to PBL during clinical rotations, with the emphasis placed on developing cognitive skills rather than formal assessments. Whilst a short MCQs may motivate students or provide learning feedback, this will not impact on their overall University grade. Assessing students’ learning styles and personality traits beforehand, and utilising the knowledge from this literature review, especially my role as a tutor, I will be able to deliver an effective student-centred, PBL session.
