Development and evaluation of problem-based learning module in clinical dental hygiene

한국치위생학회
Jin-Sun Choi1Soo-Myoung Bae1Sun-Jung Shin1Bo-Mi Shin1Hyo-Jin Lee1*

Abstract

Objectives: The purpose of this study was to evaluate learners’ satisfaction with operating Problem-Based Learning (PBL), competency level for dental hygiene, and learners’ opinions through a reflection journal by developing a PBL class module and applying it to clinical dental hygiene classes. Methods: The subjects of the study were 31 students in the Clinical Dental Hygiene (oral health management for special patients) course in the second semester of the fourth grade. This study was conducted over the first semester from September to December 2020. The developed PBL learning module was applied for the 15 weeks class, and after all PBL classes were over, the overall satisfaction with the class and the change in student competency level was evaluated. Results: The overall satisfaction of learners with PBL was high, and the level of self-competence also increased compared to before class. In addition, the top three topics (Competencies learned through PBL, humanistic and sociological elements learned through PBL, and obstacles in the PBL) were derived through the reflection journal after PBL learning activities. Conclusions: It was confirmed that the PBL learning module developed in this study is a class that enables students to identify problems and solve them integrally and drives the improvement of humanities and sociological competencies.

Keyword



Introduction

Following the fourth industrial revolution, the paradigm of education has rapidly changed, and subsequently, the importance of different competencies such as integration and convergence, creativity, and problem solving and competency-based education [1] has been emphasized. As a result, it has been suggested that process knowledge must be taught more than formal subject knowledge [2]. Problem-based learning (PBL) is a systematic type of learning in which learners acquire knowledge and skills through the process of exploring complex and practical problems and performing tasks. PBL is widely used in academia where problem-solving ability is greatly emphasized [3]. In particular, the PBL model is increasingly used to develop reasoning and self-directed learning functions that have been overlooked in the traditional curriculum for medical students who need problem-solving abilities in real life situations [4].

PBL is a learning method that enables learners to cope with possible similar situations in the future by acquiring knowledge and learning skills and attitudes through exploration of a given problem [5]. PBL was first implemented in medical schools to improve the problems of medical school curriculum and has been applied to different areas of academia with increasing interest following demonstration of the educational effects. In health and medical academia that requires education on clinical reasoning and problem-solving ability in a clinical environment, interest in PBL has gradually increased. In certain dental schools, efforts have been continuously made to reorganize the curriculum with an emphasis on PBL [6], and in the Nursing, a learning method that combines PBL and simulation practice has significantly improved nursing performance of students [7].

Previous studies have reported that PBL effectively improves students’ problem-solving ability and increases comprehension, problem-solving, self-reflection, and self-directed learning [8]. In particular, PBL provides a wide range of opportunities to practice clinical reasoning in health and medical academia [9]. Sharing and discussing clinical experiences through team activities has been suggested as one of the main benefits of PBL [10].

A dental hygienist has to solve oral health problems and prevent oral disease by making pro-active decisions concerning the patient’s problems through critical thinking. A dental hygienist must also have integrated problem-solving skills as well as creative and flexible thinking skills according to the clinical situations. Jung et al. [11] developed and applied a PBL module to a clinical dentistry curriculum and suggested that students learning dental hygiene require a curriculum that can cultivate their problem-solving skills to develop professional competency. In fact, some studies have systematically and effectively utilized the PBL learning method in dental hygiene curriculums [11,12]. Furthermore, in Japan, where dental hygienists have similar roles and tasks as those in Korea, some dental hygiene curriculums implemented PBL to provide a course of approximately 100 hours [13].

Hence, it is necessary to provide a curriculum that helps students cultivate clinical problem-solving skills and professional clinical competency in clinical dental hygiene. This will enable dental hygienists to identify and pro-actively solve problems related to the oral health of their patients. For effective implementation of such a system, classes with active participation of students that seek solutions to a given problem through team-based cooperation must be applied than lecture-type classes.

Therefore, the purpose of this study was to develop and apply a PBL class module in the education of clinical dental hygiene students and evaluate student satisfaction, improvement of dental hygiene competency, and learner’s opinions through reflective journal.

Methods

1. Study participants

This study was approved by the Institutional Review Board of 00 university (IRB No: GWNUIRB-2020-25). The participants of this study were 31 students in their second semester of fourth year, enrolled in clinical dental hygiene 3 (special patient oral health management) course, which was a major elective course with theoretical contents to achieve integrated-thinking competency based on basic knowledge from major courses for dental hygiene students.

2. Study tools

In this study, a learning tool consisting of possible problems in clinical practice was developed. The problems required cooperation between students as well as integrated thinking focused on clinical dental hygiene. The PBL module was developed through the following processes: problem presentation (problem case and instruction manual), learning goals, learning resources, and references. The students were given the task of solving cases and problems, and learning was facilitated through small group self-study sessions and discussions [14]. In this study, a 15-week study plan, weekly guidelines for class operation (teaching-learning process plan), learning goals and competency according to problems and problem situations, reference materials, problem-solving guides, and an activity sheet template (team building, problem analysis, problem solving, task performance plan, individual learning result report) required for PBL were developed. The order and framework of this study are shown in <Fig. 1>.

http://dam.zipot.com:8080/sites/KSDH/images/N0220220201_image/Fig_KSDH_22_02_01_F1.png

Fig. 1. Process of PBL modules

3. Methods

1) PBL module development

(1) PBL problem development

In this study, three typical dental problem cases that dental hygienists often encounter in dental clinics were selected. Related learning contents and clinical tasks were identified and reflected in the problem development process.

To construct the problem scenarios, certain roles and problem situations were set for the learners. The learner was set as the main character of the problem situation to induce self-directed learning. A real-life situation was constructed to arouse interest of students in learning and clearly state the intention of the learning goal for motivation to actively participate in learning. In the problem-solving process, a dental hygiene management process chart and example of an oral photograph were developed as reference materials related to the problem situation for a realistic learning experience. The problem scenario specified the role of the learner for independent problem-solving. In addition, the scenario consisted of stories that required team cooperation activities, including problem situations that can be approached and solved in various ways. Two researchers reviewed the feasibility of the developed problems using the following four criteria: Can the problem be solved through various ways? Does the problem occur in real life? Is the problem based on the experiences of students? Does the task require cooperative learning?

(2) Expert validity evaluation for PBL problem scenarios

The developed problem scenarios were reviewed for expert feasibility by a professor working in the department of dental hygiene at a university in Korea and a professor working in the department of teaching (educational engineering major). The first validation review was conducted to correct and supplement the problem scenarios, which were then re-examined by a professor in education engineering major. An expert validity questionnaire was used for assessment of the problem scenarios and it consisted of three criteria (role of the problem-11 items, non-structurality-5 items, and practicality-11 items) based on problem validity evaluation items from a previous study [15]. Each item was evaluated on a five-point scale (five points: strongly agree, four points: agree, three points: neutral, two points: disagree, one point: strongly disagree). The experts also freely expressed their opinions on each item. In addition, they provided overall opinions on the problem scenario for each topic as well as examples of a task performance plan. Their ideas were reflected in the modification and supplementation of the items.

2) Formative evaluation of the PBL module

In the last stage of PBL module development, formative evaluation was conducted on the whole process of PBL. Formative evaluation was conducted using a formative evaluation checklist [15] by three professors in the department of dental hygiene in universities in Korea. The opinions were reflected to correct and supplement the problem scenarios.

3) PBL module application and evaluation

This study was conducted over one semester from September to December, 2020. A blended learning environment of face-to-face and online sessions was used to apply the developed PBL module. ‘Total PBL’ in which all 15-week major elective courses were provided in PBL format and ‘mixed PBL’ in which, the instructor intervened in the process of problem-solving to provide lectures, overall discussions, and structured group activities, were combined. The developed module was conducted by two professors. For consistency in the direction of instruction, class operation guidelines (teaching-learning process plan) and problem-solving guides were prepared through continuous discussions. The PBL process was applied for three topics over 15 weeks. For each topic, the tasks of ‘presenting and analyzing problems-writing a task plan and individual learning-finding solutions to problems-writing problem solutions and results-presenting final results’ were conducted for four weeks. The topics of PBL problems in this study were ‘the elderly, orthodontics, and implants,’ which were selected through discussions between four researchers based on dental clinical cases and oral health issues.

After completion of PBL classes, overall satisfaction of students was evaluated using the following five items which were evaluated on a five-point scale: overall satisfaction, learning achievement, usefulness of learning in work after graduation, improved understanding, and recommendations. To evaluate growth in competency, among the eight core dental hygiene competences suggested by Bae et al. [16], ‘professional behavior’ and ‘clinical dental hygiene’ were self-evaluated by students on a scale ranging from 0 to 10 before and after each subject. Professional behavior was evaluated using the following detailed items: Q1) “I can apply evidence-based problem-solving methods with critical thinking not only for dental situations, but also in health and medical situations” and Q2) “I can acquire and utilize the latest information through critical and scientific thinking.” Clinical dental hygiene was evaluated using the following detailed items: Q3) “I can identify the oral health-related needs of the patient and make dental hygiene decisions based on the collected information” and Q4) “I can determine dental hygiene implementation and education plans by dividing them into expert- and self-care areas based on dental hygiene judgment, oral health status, potential problems, causes and risk factors of problems, and possible implementation methods.”

4. Data analysis

Normality was tested to compare self-assessment before and after PBL module application. If normality was not satisfied, Wilcoxon signed rank test was conducted, and a p-value less than 0.05 was considered statistically significant. The SPSS (Statistical Package for the Social Sciences 25.0, Chicago, USA) program was used for statistical analysis.

In addition, after every class, the students were asked to complete reflective journals on the whole process of PBL. Opinions were collected for the following items: “What have you learned through the class?” (learning points), “How did you learn it?” (learning process), “How is it related to real life?” (generalization), ‘What have you contributed to learning?” (contribution), “What are the evaluations of team members during the problem-solving process?” (evaluation), “What can be improved in learning activities?” (alternative approach), and “Looking back at the entire course, what did you learn from it?” (feedback). The key terms were coded in a segmented manner to select meaningful units. As a result, three themes and ten sub-themes were derived.

Results

1. Module development

1) Selection of topics (situations) and scenario composition

Selecting the learning topics and situations is a crucial part of PBL. In this study, three topics were chosen for one semester, in addition to another topic selected as a preliminary problem before the introduction of the main topics, based on the clinical experience of four dental hygiene professors. The topics were: an elderly female patient with root carries symptoms, an adolescent male patient undergoing orthodontic treatment, and a postmenopausal female patient with implant placement. To facilitate learning immersion, more complex and realistic situations were devised.

After composing an overall story for the selected topics, an example of a dental hygiene chart that can be used by patients was prepared. In addition, oral photos or radiographs were provided as additional visual data of the problematic area.

In PBL, the learner is the protagonist of the problem situation and is given the opportunity to actively and practically solve the given problem. Moreover, the role of the learner and the given task are clearly specified to help the learner think about the actual role.

2) Presentation of problem-solving approaches

In the first week of PBL for each topic, scenarios and reference materials (charts and photos) for each topic were provided, followed by team activities for analysis of the problem. In this step, the instructor acted as a tutor to intervene in the discussion process and referred to the problem-solving guide to assist the students. The instructor observed the students writing the task performance plan and provided feedback on the results to guide the students on the core problems, assignment of roles, final results, and writing the reports. In particular, in the scenarios, implant patients were presented with two situations before and one year after implant placement. The students were asked to complete a task performance plan for each situation.

3) Use of instruction guidelines and study plan for course operation

The PBL classes were conducted by two instructors who applied the same topic to different classes. For the sake of standardization, a specific teaching-learning course plan was developed. The plan included the weekly topics and learning goals, class outline, detailed guidelines and precautions, roles of instructors and students, reference materials, detailed time required, and teaching method. The weekly learning content was as follows: PBL course introduction and team building activities in the 1st week, analysis of preliminary problem in the 2nd week, PBL activities in the 3rd to 14th weeks (problem analysis, task performance plan establishment, individual learning, problem solving, result production, final presentation and evaluation, mini lecture), and theoretical evaluation and final assessment (self-evaluation and reflective journals) in the 15th week. Prior to each class, the two instructors reviewed and discussed the class content and progress together using the teaching-learning process plan and topical-problem-solving guide for each week in order to minimize differences in the education content and progress.

2. Module application

Considering the COVID-19 pandemic, a blended lecture method involving both in-person and online classes was used. Team activities including discussion were held in-person while presentations of results and mini-lectures were held online. For in-person team activities, small team of five or six members were assigned. The class was divided into two clusters of three teams with six teams participating simultaneously in PBL. Each of the two professors instructed half the class (one cluster) and further instructed the other teams for an equal amount of time to minimize inconsistency in teaching. To help the students systematically learn by topic and participate, the instructors guided the problem-solving approach and conducted discussions. A mini lecture for each topic was organized based on the contents that the students could not access through activities during PBL or contents that required additional learning. The lectures were given after PBL was completed so that they did not interfere with the PBL process.

3. Post-module evaluation

1) Evaluation of satisfaction with PBL class

Student satisfaction with PBL was evaluated. The item “The PBL activities are helpful for actual work after graduation.” had the highest score at 4.61 points. In contrast, the item “Overall, the PBL was satisfactory.” had the lowest score at 4.13 points <Table 1>.

Table 1. Satisfaction with the problem-based learning class   (N=31)

http://dam.zipot.com:8080/sites/KSDH/images/N0220220201_image/Table_KSDH_22_02_01_T1.png

2) Evaluation of self-competency improvement before and after PBL

Among the nine core competencies for dental hygiene, self-competency was evaluated for ‘professional behavior’ and ‘clinical dental hygiene’ in this study before and after PBL. The detailed item “I can acquire and utilize the latest information through critical and scientific thinking” showed the greatest improvement from 5.83 points before PBL to 8.32 points after PBL (p<0.001). On the other hand, competency for the item “I can determine dental hygiene implementation and education plans by dividing them into expert- and self-care areas based on dental hygiene judgment, oral health status, potential problems, causes and risk factors of problems, and possible implementation methods.” increased by only 2.36 points (p<0.001), which was the least growth <Table 2>.

Table 2. Comparison of competency level before and after the PBL based on self-assessment   (N=31)

http://dam.zipot.com:8080/sites/KSDH/images/N0220220201_image/Table_KSDH_22_02_01_T2.png

*determined by Wilcoxon signed rank test

SD: standard deviation

3) Analysis of reflective journal for PBL

After PBL activities, reflective journal was completed by each student. Meaningful terms were derived for the three themes and ten sub-themes <Table 3>.

Table 3. Analysis of reflective journal after PBL

http://dam.zipot.com:8080/sites/KSDH/images/N0220220201_image/Table_KSDH_22_02_01_T3.png

*determined by Wilcoxon signed rank test

SD: standard deviation

(1) Theme 1: Competencies gained through PBL

In this theme, ‘integrated thinking’, ‘broad thinking’, ‘information delivery and explanation’, ‘prioritization ability’, and ‘understanding diversity of perspectives’ were derived. Through PBL activities, students faced complex problem situations with virtual patients and were able to think broadly to discuss solutions to the problems. The students discussed and communicated with others to explain their opinions and deliver information. Moreover, they showed an improved ability to set priorities in problem-solving methods and understand the diverse viewpoints of others on the given problems.

(2) Theme 2: Humanities and social sciences learned through PBL

In this theme, ‘cooperation’, ‘compromise’, ‘responsibility’, and ‘respect’ were derived. In the process of solving problems and deriving results as a team, the students compared to share ideas, listen to others, coordinate opinions, and respect others. The students developed a sense of responsibility as a team member of an organization.

(3) Theme 3: Obstructive factors in PBL

In this theme, ‘restrictions due to non-face-to-face activities’ was derived. Following the COVID-19 pandemic, the proportion of non-face-to-face activities, including online meetings, increased. This led to difficulties in sharing information and coordinating opinions among team members.

Discussion

In health care academia, the need for problem-solving ability and clinical reasoning and interest in curriculum related to PBL classes have gradually increased [6,7,9]. However, there has been a relative lack of studies that have applied PBL to dental hygiene curriculum. In this study, to improve integrated thinking and problem-solving skills in dental hygiene, a PBL module was developed and applied. Satisfaction and competency of students were assessed to provide basic data for improvement of dental hygiene PBL curriculum.

PBL is a teaching-learning method in which students solve real and situational problems as a team. It is operated in a different manner than the traditional lecture-style learning method [17]. The learning effects of PBL include improved problem-solving ability, creativity, and self-directed learning [18-20]. In this process, the student learns and solves problems based on knowledge and experience focused on the subject rather than the disease. The experience of problem-solving is objectified through discussion and reflection with transfer of knowledge [21]. In studies on medicine and nursing, PBL was associated with high student satisfaction and various benefits such as improved critical thinking, communication, problem solving, and self-learning [22,23]. In other studies on PBL in dentistry, PBL was found to be helpful in improving the quality of problem-solving skills compared to textbook-centered lectures [24] and provided unconventional problems, which strengthened hypothetical-deductive reasoning [25]. Following several attempts to improve problem-solving ability of dental hygienists with a focus on oral health problems based on oral symptoms, Jung and Hwang et al. [26] developed a PBL package focused on the role of dental hygienists. Bae et al. [27] applied PBL focused on clinical dental hygiene curriculum and reported satisfaction and content validity, suggesting that PBL is a new method to stimulate interest in students by encompassing previously learned contents.

Herein, we developed and presented a learning module in clinical dental hygiene that can improve problem-solving skills through the process of self-solving problems. The learner's satisfaction with PBL in this study was high at 4.13-4.61 out of 5 points. This was similar to that reported in previous studies related to medicine and dentistry [28,29]. Additionally, dental hygiene competency before and after PBL course was improved by 2.36-2.49 points from a normal range of 5.83-6.45 to a high level of 8.32-8.81 out of 10 points. In particular, the students showed great improvement in the ability to use the latest information in critical and scientific thinking for evidence-based problem-solving. This suggested that PBL may be an important process to develop professional competency of dental hygienists before graduation. This finding was observed by identifying changes in dental hygiene competency, which was rarely evaluated in previous studies. These results may be used as evidence for development and application of PBL learning process in the future.

Analysis of reflective journals showed that the students acquired the following competencies through PBL: integrated thinking, broad thinking, information delivery and explanation, prioritization ability, and understanding diversity of perspectives. All of these competencies have been suggested as essential elements for evidence-based dental hygiene management by dental hygienists [30], and these factors must be required and trained by dental hygienists prior to their completion of university curriculum. These competencies are important and basic requirements for dental professionals including dental hygienists. In addition, ‘collaboration’, ‘compromise’, ‘responsibility’, and ‘respect’ were humanities and sociological factors learned through PBL. In studies that evaluated participants after PBL, learners were able to participate in learning more pro-actively and learn the process of cooperation through team activities compared to theory-based lectures, which aligns with the findings of this study [31]. Based on the analysis of reflective journals, it is thought that PBL can be useful to identify the main problems of the patient (subject) and improve evidence-based problem-solving skills.

In PBL, the process of analyzing a given situation and deriving a solution is performed through team activities. Through this process, the participants not only acquire knowledge, but also develop teamwork and cooperate to solve problems, which lead to expansion of thoughts. The final results of each team were reviewed in consideration of the opinions of the learners, and it was observed that the group that had teamwork problems showed a low score for objective results. Thus, teamwork is the most important factor in PBL to solve problems through cooperation, and the instructors must consider teamwork in designing the module. In addition, it would be necessary to supplement the developed PBL module by considering the areas of improvement and supplementation suggested by the learners.

In PBL, the given ‘problem situation’ is more important than anything else. It is fundamental that the problem situation is a commonly experienced problem in dental clinics and requires integrated thinking and problem-solving skills. In this study, based on the research team's dental clinical experience, topics were selected through a review of related materials and discussions. Problems and reference materials were prepared to help the learners consider and learn various complex factors in problem situations. To increase the validity of the developed problem situation, reference materials, and task performance plan for problem solving, the items were reviewed to construct more reasonable problem situations and better reference materials. Additionally, formative evaluation was conducted on all final materials, methods, and processes, and supplementary work was performed to increase validity before the application of the module. In future studies, it would be important to supplement the module with the opinions of instructors as well as that of learners to increase the validity and effectiveness of PBL module for more efficient and effective learning of the students.

Several limitations must be considered in the interpretation of this study's findings. First, this study reported the development and application of PBL module in single university. Therefore, the findings of this study cannot be generalized. In future studies, it is important to apply PBL to all university students from first to fourth year students in dental hygiene courses to further assess the learning efficiency and effects of PBL. Moreover, due to the COVID-19 pandemic, there were restrictions in the learning process of students as a mix of online and in-person methods were used. Follow-up studies must evaluate and compare the efficiency, effect, and satisfaction of in-person and online PBL classes. However, this study is significant in that it developed a PBL module in dental hygiene to improve the management competency and professionalism of dental hygienists. In particular, this study assessed changes in student satisfaction and competency and collected student opinions through analysis of reflective journals, which may be usual basic data for improvement of dental hygiene PBL modules in the future.

Taken together, PBL provided an opportunity for dental hygiene students to develop problem-solving skills, integrative thinking, and clinical thinking in various problem situations that can be faced in dental clinics. In dental clinical practice, various complex problems cannot be solved with simple knowledge. Thus, problem-solving skills to take initiative and solve problems in a timely manner are essential for dental hygienists. Therefore, an adequate PBL process of solving problems through identification of problems and integrated thinking in given cases must be provided.

Conclusions

This study was conducted to develop and apply a PBL class module in clinical dental hygiene students and evaluate student satisfaction, improvement of dental hygiene competency, and learner’s opinions through reflective journals.

1. The learner’s satisfaction with PBL was generally high at 4.13-4.61 out of 5 points.

2. Self-competency was evaluated before and after PBL. The item “I can acquire and utilize the latest information through critical and scientific thinking.” was improved the most (p<0.001) while the item “I can determine dental hygiene implementation and education plans by dividing them into expert-and self-care areas based on dental hygiene judgment, oral health status, potential problems, causes and risk factors of problems, and possible implementation methods.” showed the least improvement (p<0.001) among different detailed items.

3. Reflective journals completed after PBL activities were analyzed to derive the following three themes and ten sub-themes: competency (integrated thinking, broad thinking, information delivery and explanation, prioritization ability, understanding diversity of perspectives), humanities and sociological factors (cooperation, compromise, responsibility, respect), intervening factors in PBL (restrictions due to non-face-to-face activities).

The students were satisfied with PBL activities in clinical dental hygiene courses. The developed PBL learning module enhanced humanities and sociological competency as well as problem-identifying and solving skills in an integrated manner. In future studies, strategies must be sought to apply and expand the PBL module to other subject areas.

Conflicts of Interest

The authors declared no conflicts of interest.

Acknowledgements

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2019R1G1A1009611).

Authorship

Conceptualization: HJ Lee, JS Choi, SM Bae, SJ Shin, BM Shin; Data collection: HJ Lee, JS Choi; Formal analysis: JS Choi, HJ Lee; Writing - original draft: HJ Lee, JS Choi, SM Bae, SJ Shin, BM Shin; Writing - review & editing: HJ Lee, JS Choi, SM Bae, SJ Shin, BM Shin

References

1 1. Korean Educational Development Institude. Education in the age of the fourth industrial revolution: the future of school. Chungbuk: Korean Educational Development Institude; 2018: 225.  

2 2. Drucker PF. Post - capitalist society. New York: Harper Collins; 1993: 2-198.  

3 3. Markham T, Larmer J, Ravitz J. Project - based learning handbook: a guide to standards focused project - based learning for middle and high school teachers. Novato, CA: Buck Institute for Education; 2003.  

4 4. Barrows HS. Practice - based learning problem - based learning applied to medical education. Biochem Educ 1994;22(4):128.  

5 5. Walton HJ, Mathews MB. Essentials of problem - based learning. Med Educ 1989;23(6):542-58. https://doi.org/10.1111/j.1365-2923.1989.tb01581.x    

6 6. Park BS, Kim HC, Cho BH, Park HR, Oh HJ. Case study of applicability of PBL - based instruction of integrated medicine and lifescience class in school of dentistry. Jour Sci Edu 2019;43(3):342-62. https://doi.org/10.21796/jse.2019.43.3.342  

7 7. Song YA. Effect of simulation - based practice by applying problem based learning on problem solving process, self - confidence in clinical performance and nursing competence. Korean J Women Health Nurs 2014;20(4):246-54. https://doi.org/10.4069/kjwhn.2014.20.4.246  

8 8. Choi JI. A case study for the application of PBL in higher - education: focused on the effectiveness of PBL presented in reflective journal. JET 2007;23(2):35-65. https://doi.org/10.17232/KSET.23.2.35  

9 9. Burgess A, Bleasel J, Haq I, Roberts C, Garsia R, Robertson T, et al. Team - based learning (TBL) in the medical curriculum: better than PBL?. BMC Med Educ 2017;17(1):243. https://doi.org/10.1186/s12909-017-1068-z      

10 10. Dolmans D, Michaelsen L, Van Merrienboer J, Van der Vleuten C. Should we choose between problem - based learning and team - based learning? no, combine the best of both worlds!. Med Teach 2015;37:354-9. https://doi.org/10.3109/0142159X.2014.948828    

11 11. Jung YR, Lee WS, Jo DJ. Development of PBL packages for the improvement of the problem - solving ability, self - directed learning capability and communicative competence of dental hygiene students. J Korean Soc Dent Hyg 2010;10(1):33-49.  

12 12. Jeong AY, Shin SJ, Shin BM, Bae SM. The development of problem - based learning module for clinical dentistry in dental hygiene. J Dent Hyg Sci 2017;17(5):383-97. https://doi.org/10.17135/jdhs.2017.17.5.383  

13 13. Shin BM, Bae SM, Shin SJ. Analysis of a clinical dentistry course to study the need for integrated curriculum : in the case of dental hygiene department in Korea and Japan. J Korean Soc Dent Hyg 2019;19(3):331-42. https://doi.org/10.13065/jksdh.20190035  

14 14. Kim YJ, Kang PS, Lee CK, Park JH. The principle and practice of PBL. Korean J Med Educ 2000;12(1):1-14. https://doi.org/10.3946/kjme.2000.12.1.1  

15 15. Na JY, Chung HM. Development of a PBL instructional design model for higher education. The Journal of Yeolin Education 2012;20(3):111-40.  

16 16. Bae SM, Shin SJ, Jang JH, Chung WG. Implementation of assessment in a competency - based dental hygiene science education. Changwon: Korean Association of Dental Hygiene Professors; 2014: 10-24.  

17 17. Barrows HS. Problem - based learning in medicine and beyond: a brief overview. New Directions for Teaching and Learning 1996;68:3-12. https://doi.org/10.1002/tl.37219966804  

18 18. Bae YS, Lee SH, Kim MH, Sun KS. Effects of PBL (Problem - Based Learning) on self - directed learning and critical thinking disposition of nursing students. J Korean Acad Soc Nurs Edu 2015;11(2):184-90.  

19 19. Yoon WH. (The) effects of problem - based learning program on the creative problem - solving ability of the gifted in elementary mathematics[Doctoral dissertation]. Seoul: Univ. of Konkuk, 2006.  

20 20. Yoo MR, Choi YJ, Kang MS. The effects of PBL (Problem - Based Learning) on the self - directed learning, critical thinking disposition, and problem solving process of nursing students. J Korean Acad Fundam Nurs 2009;16(1):46-55.  

21 21. Barrows HS. The essentials of problem - based learning. J Dent Educ 1998;62(9):630-3.    

22 22. Chung BY, Kim KH, Byun HS, Han JY. Application of problem based learning in nursing education. Journal of Kyungpook Nursing Science 2008;12(1):51-63.  

23 23. Kim YM, Park YS, Chung CW, Kim MJ. Application of Problem - Based Learning (PBL) for student’ practice in maternity nursing. Korean J Women Health Nurs 2006;12(4):326-37. https://doi.org/10.4069/kjwhn.2006.12.4.326  

24 24. Kim HA, Kim KK, Lee SW. A comparison of effect of lecture - based learning and problem - based learning on scientific reasoning in basic medicine. J Oral Med Pain 2005;30(1):35-44.  

25 25. Barrows HS, Pickell GC. Developing clinical problem solving skills: a guide to more effective diagnosis and treatment. New York: Norton; 1991: 103-24.  

26 26. Jung YR, Hwang YS. Development of PBL package - focusing on dental hygienist roles. J Korean Soc Dent Hyg 2004;4(1):119-32.  

27 27. Bae SM, Kim KM, Kim HJ. Satisfaction and content validity of learner’s satisfaction using Problem - Based Learning (PBL): focusing on dental hygiene department and clinical dental hygiene department. AJMAHS 2019;9(11):651-60. https://doi.org/10.35873/ajmahs.2019.9.11.058  

28 28. Park MJ. An evaluation on the implementation of Problem - Based Learning in medical education. Journal of Curriculum Studies 2010;28(2):225-53.  

29 29. Haghparast N, Sedghizadeh PP, Shuler CF, Ferati D, Christersso C. Evaluation of student and faculty perceptions of the PBL curriculum at two dental schools from a student perspective: a cross - sectional survey. Eur J Dent Educ 2007;11(1):14-22. https://doi.org/10.1111/j.1600-0579.2007.00423.x    

30 30. Bae SM, Shin SJ, Jang JH, Chung WG. Implementation of assessment in a competency - based dental hygiene science education. Changwon: Korean Association of Dental Hygiene Professors; 2014: 10-24.  

31 31. Kim SH. A study of problem based learning (PBL) experience in dental hygiene education - learning attitude, student assessment-. J Korean Soc Dent Hyg 2010;10(5):797-805.