Effectiveness of flipped classrooms in nursing education:Systematic review and meta-analysis

Cui Tn ,Wei-Gng Yue ,Yu Fu

aSichuan Nursing Vocational College,Chengdu,Sichuan 610100,China

bThe First Af f i liated Hospital of Lanzhou University,Lanzhou,Gansu 730000,China

1.Introduction

There is a gap between nursing education and actual practice such that newly graduated nursing students struggle to meet workplace requirements.1To address this gap,there has been a call for a radical transformation in nursing education involving educators designing learning experiences that enhance educationquality and ensure that graduates are prepared for today's complex nursing practice at different health care facilities.1-3New innovative teaching strategies are critical to integrate into nursing education.The literature tells us that one of the primary components of effective teaching is student engagement and that engagement is critical for learning.4,5Re flecting this change is the rise in the flipped classroom that is an alternative pedagogy that shifts from traditional lectures to active learning6and would offer opportunities to integrate clinical experiences within the classroom.

The flipped classroom is an instructional strategy that provides a new methodology and modality for teaching and learning.It involves a role change for instructors as a way of minimizing the amount of direct instruction in their teaching practice while maximizing one-to-one interaction and more cooperative and collaborative contribution to the teaching process,which can improve and encourage social interaction,teamwork and cultural diversity among students.The roles of students have a corresponding change from passive participants to active participants.1The modern flipped classroom began in 2007 in a high school chemistry course in Colorado.7It could be argued that the flipped classroom has been in existence within the broader educational sphere for a number of years.Flipped classrooms take what was previously class content and replace it with what was previously homework,which now takes place within the class.8In addition,this approach fosters student ownership of learning through the completion of preparatory work and being more interactive during actual class time.Proponents of the flipped class suggest that this pedagogy allows students to learn at their own pace,and they may have flexibility as to when they engage with electronic resources.It frees up actual class time for robust discussion and associated problem-solving activities,which could be initiated by the students,not the staff member.This pedagogy may help students to improve self-efficacy through self studying,problem-solving and communication skills,which are skills that are recommended by the Institute of Medicine report from 2010.2

Many studies have already reported that the flipped classroom had a positive effect on education outcomes,such as accelerating self-learning,improving academic performance,9,10developing critical thinking skills,11and increasing positive feedback.9,12However,some of the studies also showed no significant differences in final exam scores between students undertaking traditional lectures and those in the flipped classroom13,14and even had less satisfaction with the flipped class approach due to increasing the amount of out-of-class preparation time.6,15Meanwhile,students and staff are reluctant to abandon the lecture-based approach because it requires little active student participation.4

Despite acknowledgement of the positive attributes of the flipped classroom,few articles used a robust scienti fic method to verify educational outcomes as they relate to improved student learning and attitude.16,17Rigorous experimental research on the flipped classroom in higher education is very limited.14,18Other nurse scholars have conducted systematic reviews to address flipped classroom models in nursing education and found that a metaanalysis was impossible due to the limited number and heterogeneity of studies.19,20Therefore,research to date is insufficient to con firm the effectiveness of the flipped classroom approach.

The flipped classroom has generated considerable interest in nursing education in recent years.Although experimental research on the flipped classroom published in English is limited,14,18there has been a recent in flux of empirical evidence on the flipped classroom in Chinese nursing education.21Since Chinese articles may not be easily accessible to international researchers due to language barriers,this review aims to examine whether the flipped classroom is more effective than the traditional lecture through synthesizing both English and Chinese studies.The following research questions are addressed in this paper:

What effects of the flipped classroom compared with the traditional lecture have been reported in nursing education?

To what extent do the effects of flipped classrooms relate to knowledge,skills and attitudes compared with traditional lectures in nursing education?

What is the rating of self-report questionnaires as to the effectiveness of the flipped classroom?

2.Methods

We conducted this meta-analysis according to the guidelines proposed by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses(PRISMA)standards of quality22for reporting this review.

2.1.Protocol and registration

This protocol was registered in PRISMA with register No.CRD42016041826 and is available from http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42016041826.

2.2.Inclusion and exclusion criteria

Studies were selected for inclusion if the study had(a)participants were in a higher education program including associate degree students,undergraduates,and postgraduates,(b) a randomized controlled trial design,(c)a flipped classroom approach in the intervention group,(d)traditional lectures approach in the control group,(e)measurable outcome indexes,and(f)publication in Chinese and English.Exclusion criteria included that the study(a)was conducted with students in disciplines other than nursing or register nurses,(b)was a nonrandomized controlled trial,(c)had an intervention that utilized flipped classroom and at the same time accepted other teaching methods,or(d)had non-extractable outcome data.

2.3.Data sources and searches

The English databases searched included PubMed,EMBASE,Science Director,CINAHL,Google scholar,and the Chinese databases searched included CNKI,WanFang Data,VIP and CMB.We identi fied peer-reviewed studies in accordance with the inclusion criteria from inception to November 2016.To identify as many eligible studies as possible,we broadened search terms and strategies.Search terms were modi fied together with informatics and combined with Boolean operators as follows: ‘nurs*’AND(‘flipp*classroom’OR ‘invert*classroom’OR ‘revers*classroom’).The reference lists of the eligible articles included after the electronic search were also manually searched.Two researchers searched the databases together and independently selected the eligible articles.Articles with the same authors were included as long as the research participants were from different areas.Any discrepancies were resolved through discussion with the third researcher.

2.4.Data extraction process and quality assessment

Data extraction and article quality assessment were performed by two researchers independently.Disagreements were resolved by consensus.The following information from the studies was extracted based on a predesigned coding manual form:authors and years of publication,sample size,educational degrees,subjects,withdrawal rate,and intervention and outcome measures.

The quality of the included studies was based on the six general sources of bias described in the Cochrane Handbook for Systematic Reviews of Interventions.The following quality items were checked:adequacy of the generation of the allocation sequence,concealment of allocation,blinding procedures,incomplete outcome data,selective outcome reporting,and other sources of bias.These items were obtained from the published reports,and the authors were contacted if additional information was required.The methodological quality of these domains was assessed by using the following scoring:(1) ‘low risk of bias’,when plausible bias unlikely alters the results,(2) ‘unclear risk of bias’when plausible bias raises some doubt about the results and(3) ‘high risk of bias’when plausible bias seriously weakens con fidence in the results.23

2.5.Statistical analyses

Review Manager 5.3 software24was used to test the data for heterogeneity and to carry out the meta-analysis.As continuous data from different scales were extracted,the standardized mean difference(SMD)was calculated for effect size based on sample size25and 95%con fidence intervals for each study,and for the pooled studies using variance analysis.Weighted mean difference(WMD)and 95%con fidence intervals(CI)were calculated for continuous data from the same scale.A two-sidedP-value of less than 0.05 was regarded as significant for all analyses.There were two meta-analysis models.A fixed-effects model was used to pool data if there was no heterogeneity,and otherwise we used a random-effects model.Heterogeneity was considered significant for aP-value of Cochran's Q statistic＜0.1 or anI2＞ 50%.26I2is the percentage of variation attributed to heterogeneity and is easily interpreted.AnI2statistic of 25%-50%was considered low,50%-75%was considered moderate,and≥75%was considered high.If heterogeneity was found,we conducted a sensitivity analysis to assess if this heterogeneity significantly altered the results of the meta-analysis.We performed the sensitivity analysis by excluding studies one by one,and then recalculating the pooled estimates for the remaining studies.However,this process did not significantly alter the results.Subgroup analyses were planned according to preset variables,including educational degree and research setting.Publication bias was assessed through funnel plots,and funnel plot shapes did not reveal obvious evidence of asymmetry.

3.Results

3.1.Search results

The search yielded 497 relevant studies that were downloaded into EndNote X7.After the removal of duplicates,the 244 residual references were reviewed based on the title and abstract.Then,73 full text studies were assessed for eligibility.Subsequently,a total of 29 eligible studies were included in the meta-analysis.Fig.1 provided a flow chart illustrating our search strategy and output.

3.2.Characteristics of included studies

Twenty-nine articles9,10,12,27-52were included in this review.They were all Chinese articles published from 2014 to 2016.Sixteen studies measured theoretical examination scores,10,30,31,36,38-40,43-50,52 and sixteen studies measured skill examination scores.9,10,12,31,32,34-37,39,41,43,46,48,51,52 Fifteen studies investigated the students'self-learning through questionnaires.27-29,31,33-36,38-41,43,45,47 One study42 was notincluded in the meta-analysis because its outcome indexes were investigated only by questionnaires.The sample sizes ranged from 36 to 320 participants,and the pooled sample size was 3694(experimental group=1896,control group=1798).Fifteen studies were conducted in undergraduates,and 14 studies were conducted in associated degree students.Research settings included 25 schools(86.2%)and 4 clinical areas(13.8%).Table 1 shows the characteristics of the 29 included studies.

3.3.Study quality

Fig.2 shows the risk of bias assessment of the 29 included studies.The majority of the studies had a low risk of bias across the six domains.Most of the studies reported cluster randomized sampling or just mentioned it used “randomized”for allocation concealment without details.One study33adequately generated random numbers.Another two studies were separately generated by odd and even numbers based on the college entrance examination scores37and student ID numbers.49The allocation sequence of the two studies30,34was based on the preferences of the researchers.

Two studies10,33reported using blind implementation,but because of the nature of the given intervention,it was not feasible to blind the students and tutors.Therefore,performance bias was the only bias category causing high risk in nearly all studies included.One study reported missing data with reason,33the others reported complete outcome data.We assessed whether each study was free from selective outcome reporting by checking whether all outcomes mentioned were adequately reported in the results section.All studies adequately reported the results.All studies seemed free from “other sources of bias”as defined in the Cochrane Collaboration's domain-based evaluation.

3.4.The effectiveness of the flipped classroom in nursing education

The effects of the flipped classroom were measured by scores according to the teacher-constructed examination or self-report questionnaires.Major outcome indexes were(a)theoretical examination scores,(b)skills examination scores,(c)self-learning questionnaires,and(d)questionnaires used for teaching evaluation,student satisfaction,student attitudes,critical thinking and problem-solving skills.

3.4.1.The effectiveness of interventions on knowledge

We used the theoretical examination scores to check the students'academic performance in knowledge.Sixteen studies tested the theoretical examination scores with pooled 2103 participants(experimental group=1099,control group=1004).Thirteen studies showed statistically significant differences between the flipped classroom and traditional lecture.There was high heterogeneity(I2=93%,P＜0.00001).Thus,a random-effects model was used.The pooled effect size showed a significant difference in theoretical examination scores[SMD=1.13,95%CI(0.76,1.49),P＜0.00001]in favor of flipped classrooms compared with traditional lectures(Fig.3).The results indicated that the flipped classroom improved students'academic performance in knowledge.Due to the high heterogeneity among studies,we used subgroup analysis by education level.There was no significant difference in the subgroups(P=0.85,I2=0%),which means that the heterogeneity was from intra-group differences rather than inter-group differences.A sensitivity analysis was carried out to verify the reliability of the results.This analysis was performed using sequential omission of individual studies one by one,and the pooled effect size that favored the flipped classroom group did not change the effects observed in the initial analysis.The funnel plotof the 16 studies with theoretical examination scores is shown in Fig.4.The shape of the funnel plot is symmetrical indicating no significant publication bias.

Table 1Characteristics of included studies.

3.4.2.The effectiveness of interventions on skills

We used the skills scores to check the students'academic performance in skills.Sixteen studies tested the skill examination scores with a pooled 2349 participants (experimental group=1216,control group=1133).Thirteen studies showed statistically significant differences between the flipped classroom and the control groups in skills scores.There was high heterogeneity(I2=95%,P＜0.00001).Thus,a random-effects model was used.The pooled effect size showed a significant difference in skills scores[SMD=1.68,95%CI(1.24,2.12),P＜0.00001]in favor of a flipped classroom compared with traditional lectures.The results indicated that the flipped classroom improves students'academic performance in skills with an effect size of 1.68.Subgroups by education level and research setting were analyzed(Fig.5),and the results showed that there was small inter-group heterogeneity(I2=11.9%).Thus,the majority of heterogeneity were still from intra-group variability.A sensitivity analysis was performed using sequential omission of individual studies.After excluding one study,21the heterogeneity was obviously diminished(I2=66%)in undergraduates,but the pooled effect size favored the flipped classroom[SMD=1.65,95%CI(1.41,1.89),P＜0.00001]and did not change the effects observed in the initial analysis.The funnel plot for the 16 studies in the skill examination score analysis is shown in Fig.6.This shape was mostly symmetrical indicating no significant publication bias.

3.4.3.The effectiveness of interventions on self-learning skill

Fifteen of the articles tested the self-learning skill of students with pooled 2119 participants(experimental group=1099,control group=1020).The effect size of 1.51[95%CI(1.02,2.01),P＜0.00001]means that students agree that flipped classroom can improve their self-learning skill.There is a relatively small heterogeneity existing according to the evaluation instruments through subgroup analysis,however,it is not the main heterogeneity sources(Fig.7).The funnel plot for the 15 studies on the self-learning questionnaires analysis is shown in Fig.8.The shape of the funnel plot is symmetrical indicating no significant publication bias.

3.4.4.The effectiveness of interventions assessed by questionnaires

Overall,nineteen articles investigated the students'evaluations of the flipped classroom pedagogy through self-reported questionnaires in terms of teaching effectiveness,30,32,38,39,41,46,49,50student satisfaction,9,27-29,34,42,45,48student study attitudes,10,12,27,47critical thinking skills,42,44,47and problem-solving skills.10,27According to the questionnaire results,most students at the flipped classroom had positive comments about the flipped classroom.At least 90.4%of reported studies indicated an above average rating for students'study enthusiasm in the flipped classroom approach,and 93.8%of students selected above average that flipped classrooms accelerate the communication between faculty and students.In all,87%of the students reported that they preferred this approach and wanted to continue this approach in other subjects.There was also a statistically significant improvement in student satisfaction,29,34,48student interest,27,47and clinical thinking38,42in the flipped classroom compared with the traditional lecture.

4.Discussion

At first glance,evidence in favor of the flipped classroom seems strong because all studies indicated that flipped classroom students outperformed those in traditional lecture classrooms,especially in improving students'academic performance both in knowledge and skills and self-learning ability.These results are in accordance with the findings of the previous narrative review related to foreign nursing education.6,16,20In the evaluation of teaching satisfaction and study attitude,these results also showed a relatively higher rating for the flipped classroom compared with the traditional lecture,which was contrary to some Western studies6,53that found improved examination scores and diminished student satisfaction.Several possible reasons may explain why Chinese students prefer flipped classrooms.First,usually,in traditional Chinese nursing education,students are not allowed to use cellphones and computers during the lecture.Second,the traditional lecture is quite repressed since most students are not allowed to discuss their opinions and always remain quiet.Third,students have few opportunities to communicate with the faculty after class,and the flipped classroom may let students feel the faculty are more approachable.However,compared with the different culture in Western higher education,their traditional education has already adopted electronic equipment and encouraged them to share their opinions freely in the class,so they may not be curious about this pedagogy style,which may be why it received a low satisfaction rating due to the workload for class preparation.

Overall,in the pool of included studies,there was obvious heterogeneity,which may have resulted from the following factors.First,the different educational levels of the students resulted in the criteria for admission of nursing students being different.Second,the pooled studies focused on different nursing subjects.The third reason concerns the examination methods.Twenty-nine studies employed different appraisal tools.There were no uni fied criteria for the evaluation of the effectiveness of the flipped classroom.Fourth,each study employed a diverse range of preparatory and inclass active learning strategies as part of the flipped classroom intervention.These different teaching methods and objectives may have resulted in differenteffects,and this diversity may be the main cause of the heterogeneity among those studies.

Initially,the researcher was planning to synthesize the outcomes of questionnaires quantitatively.However,most of the studies used self-constructed questionnaires with a post-test and lacked comparison with traditional lectures.Meanwhile,the types of measurable data are different.Thus,the synthesis was not feasible.The researcher just described the aforementioned aspects related questionnaire surveys.However,most studies showed positive results for the flipped classroom.

5.Strengthens and limitations

In this review,we used a statistical method to validate the effectiveness of the flipped classroom in nursing education since all of the review papers before were narratives.It provided relatively robust evidence to address the concerns of prior researchers.Meanwhile,this review was reproducible in terms of data extraction and synthesis.In addition,we registered in PRISMA with a protocol before we conducted this review,which decreased the bias in this review.In terms of limitations,no studies published in English met the inclusion criteria,and some of the Chinese studies were of poor quality.Fortunately,the sensitivity analyses showed that the outcomes of the study were reasonably stable.

As a result,the effectiveness of the flipped classroom was positive.However,two studies in the final pool that found no signi f icant differences were excluded due to insufficient data.Therefore,we suppose that the extent of effectiveness may have diminished if we included them.On the other hand,in the measurable outcome indexes,knowledge can be objectively evaluated by theoretical examination,but the appraisal of skills and self-reported questionnaires are somehow subjective.Most studies did not report whether the implementation was blind or the surveys were anonymous,which is an important caveat when addressing survey results.

For future researchers,we recommend strengthening the instructional framework of flipped classroom pedagogy and formulating uni fied evaluation criteria.Additionally,there is a need for rigorous randomized controlled trial(RCT)designs for future studies to reduce the heterogeneity of the included articles.We also suggest that future reviewers perform meta-analyses to find sources of heterogeneity in this area.

6.Conclusions

This review indicated that the flipped classroom might help nursing students improve in knowledge,skills,self-learning,study satisfaction and enthusiasm.However,students'critical thinking,problem-solving,and decision-making skills should be further validated by robust studies.Flipped classroom pedagogy is recommended,especially in Chinese nursing education reform.Since all of the included articles were in Chinese,the review highlights the lack of evidence about flipped classrooms in Western nursing education.Compared with the outcomes of Western studies of the effectiveness of the flipped classroom,Chinese studies showed more positive feedback.

Most importantly,to explore the rationale behind flipped classroom models,factors that will in fluence student performance also need to be included.Nursing educators could then translate these models in more strategic ways to transform the teachinglearning process.

Con flicts of interest

The authors declare that there are no Conflicts of interest with regard to the present work.This research received no speci fic grant from any funding agency in the public,commercial or not-for-pro fit sectors.

Acknowledgements

The authors would like to thank Dr.Brian S.Bantugan,who gave guidance and always opened the door whenever I had questions about this study.We would also like to thank Ma'am Xiao Li,who provided the electronic libraries resources for the literature search and gave her suggestions about search strategies.

The authors would like to express gratitude to all who have given generously of their time to edit this paper.

1.Benner P,Sutphen M,Leonard V,Day L.Educating Nurses:A Call for Radical Transformation.San Francisco,CA:Jossey-Bass;2009.

2.IOM(Institute of Medicine).The Future of Nursing:Focus on Nursing Education.Washing ton,DC:The National Academic Press;2010.

3.Yacout DAESA,Shosha AAEFA.Nursing students'perceptions towards flipped classroom educational strategy.J Am Sci.2016;12:62-75.

4.Barkley EF.Student Engagement Techniques:A Handbook for College Faculty.San Francisco,CA:Jossey-Bass;2009.

5.Coates H.Student Engagement in Campus-based and Online Education:University Connections.New York:Routledge;2006.

6.Missildine K,Fountain R,Summers L,Gosselin K.Flipping the classroom to improve student performance and satisfaction.J Nurs Educ.2013;52:597-599.

7.Bergmann J,Aaron S.Remixing chemistry class:two Colorado teachers make podcasts of their lectures to free up class time for hands-on activities.Learn Leading Technol.2009;36:22-27.

8.Pierce R,Fox J.Vodcasts and active-learning exercises in a“ flipped classroom”model of a renal pharmacotherapy module.Am J Pharm Educ.2012;76:196.

9.Yan LX,Xie HY.Application of flipped classroom mode in comprehensive nursing technology.J Nurs Train.2015;7:626-628(in Chinese).

10.Gao Y.Flipped teaching with mobile teaching platform for fundamental nursing skill.J Nurs.2016;8:13-15(in Chinese).

11.Horn MB.The transformational potential of flipped classrooms.Educ Next.2013;13:78-79.

12.Tao WW,Liu YN,Ding SZ,Sha LY,Liu Y.Effect of flip classroom and microlectures on educational environment and academic emotions of nursing students.Chin Nurs Manage.2016;16:1221-1225(in Chinese).

13.Geist MJ,Larimore D,Rawiszer H,Al S,Abdellatif W.Flipped versus traditional instruction and achievement in a baccalaureate nursing pharmacology course.Nurs Educ Perspect.2015;36:114-115.

14.Harrington SA,Bosch MV,Schoofs N,Beel-Bates C,Anderson K.Quantitative outcomes for nursing students in a flipped classroom.Nurs Educ Perspect.2015;36:179-181.

15.Simpson V,Richards E.Flipping the classroom to teach population health:increasing the relevance.Nurse Educ Pract.2015;15:162-167.

16.O'Flaherty J,Phillips C.The use of flipped classrooms in higher education:a scoping review.Int Higher Educ.2015;25:85-95.

17.Bernard JS.Nurse Educators'Transition to Flipped Classroom:An Interpretive Description Study.East Tennessee State University;2015.Electronic Theses and Dissertations.Paper 2603 http://dc.etsu.edu/etd/2603.

18.Heyborne W.To flip or not to flip?Analysis of a flipped classroom pedagogy in a general biology course.J Coll Sci Teach.2016;45:31.

19.Betihavas V,Bridgman H,Kornhaber R,Cross M.The evidence for ‘ flipping out’:a systematic review of the flipped classroom in nursing education.Nurse Educ Today.2016;38:15-21.

20.Presti CR.The flipped learning approach in nursing education:a literature review.J Nurs Educ.2016;55:252-257.

21.Kang DQ,Yue SJ.Bibliometric analysis on application of flipped classroom in nursing teaching journal of nursing.J Nurs.2016;23:22-25(in Chinese).

22.Moher D,Liberati A,Tetzlaff J,Altman DG,PRISMA Group.Preferred reporting items for systematic reviews and meta-analyses:the PRISMA statement.PLoS Med.2009;6,e1000097.

23.Higgins JPT,Green S,eds.Cochrane handbook for Systematic Reviews of Interventions.vol.4.John Wiley&Sons;2011.

24.Cochrane Collaboration.Review Manager(RevMan).Copenhagen:The Nordic Cochrane Centre;2014.Version 5.2.3.

25.Cohen J.Statistical Power Analysis for the Behavioral Sciences.2nd ed.Hillsdale,NJ:Lawrence Erlbaum Associates;1988:20-26.

26.Higgins JP,Thompson SG,Deeks JJ,Altman DG.Measuring inconsistency in meta-analyses.BMJ.2003;327:557-560.

27.Yin WQ.The application value of flipped classroom in nursing education.New Curriculum Res.2016;3:83-85(in Chinese).

28.Feng R,Xu H,He SZ,Zhang YH.The implementation and evaluation of flipped classroom in gynecologic and obstetric nursing.J Nurs.2015;22:15-17(in Chinese).

29.Liu MX,Peng GH,Kong Y.Practice of flipped classroom model in teaching of nursing for interventional therapy.Hainan Med.2016;27:1013-1016(in Chinese).

30.Liu F.Application of flipped classroom teaching mode based on micro lecture in teaching of “introduction of nursing science”.Chin Nurs Res.2016;30:3406-3408(in Chinese).

31.Liu P,Xue M,Ma LD,LI J.The effect of flipping classroom teaching in improving the self-directed learning ability of higher vocational nursing students.Chin J Nurs Educ.2016;13:109-112(in Chinese).

32.Ye YS,Zhao J,Zhang ZX,Ye L,Wang WJ,Kong LL.The application of studentsbased flipping teaching in Fundamental Nursing technology.J Clin Nurs.2016;1:69-71(in Chinese).

33.Ye YS,Zhao J,Bao CY,Ye L,Wang WJ,Xie YQ.In fluence of student-based flipped teaching on clinical practical performance of nursing undergraduates.J Nurs.2016;1:16-19(in Chinese).

34.Song YL,Liu XT,Liu W,et al.Application of case method-based flipped classroom in practice teaching of fundamental of nursing and its evaluation.J Nurs.2016;9:8-11(in Chinese).

35.Zong MZ,Cong RZ,Li FJ,Liu YH.The practice of micro lecture and flipped classroom in fundamental nursing.World Latest Med Inf(Electro Version).2016;3:198(in Chinese).

36.Kou D,Gan H,Chen QH,He R,Li Y.The application of flipped classroom in intern nursing students.Tianjin J Nurs.2016;4:338-339(in Chinese).

37.Zhang XH.The evaluation of effectiveness of flipped teaching mode in clinical teaching of undergraduates nursing students.Curr Nurs.2016;6:131-133(in Chinese).

38.Yang SG.Evaluation of the flipped classroom model applied in preventive medicine teaching.China Med Educ Tech.2016;5:577-580(in Chinese).

39.Tian ZJ.Application of Flipped Instruction in the Course of Fundamental Nursing.Taiyuan:Shanxi Medical University;2016.

40.Tian ZJ,Jin RH,Liu CF.Application of flipped instruction in the course of geriatric nursing.J Chin Nurs Educ.2015;12:333-335(in Chinese).

41.Ji HX,Liang A,Zhang HY.Application of flipped classroom mode in surgical nursing technology.Mod Vocational Educ.2016;14,152-152(in Chinese).

42.Luo LR.A study of flipped classroom in critical thing and study satisfaction of associated degree nursing students.Health Vocational Educ.2016;34:106-108(in Chinese).

43.Hu HF.Flipped classroom in gynecologic and obstetric nursing.Time Educ.2015;20:86(in Chinese).

44.Xie ZM,Li YJ.The effective analysis of flipped classroom combine with micro video mode in strengthening critical thinking of nursing students.J Exam.2015;47:171-174(in Chinese).

45.Zhao MY.Application of micro lecture and flipped classroom in health assessment of nursing students.Health Vocational Educ.2016;34:83-85(in Chinese).

46.Zhao H,Yao PB,Liu YH,Wu MC.Application of micro lecture-based and flipped classroom mode in fundamental nursing.J Snake.2016;28:108-109(in Chinese).

47.Deng WF.Application of flipped classroom teaching mode based on constructive theory in the course of introduction of nursing.Chin J Prac Nurs.2016;32:1429-1433(in Chinese).

48.Guo W,Yu Y,Liu JL,Gai QM,Sang Y.Application of flipped classroom mode in vascular surgical teaching of nursing students.Today Nurse.2016;11:162-163(in Chinese).

49.Zhong ML,Jiang DY,Wei QM,Jiang ZX,Chen SZ,Hu RJ.The effects of flipped classroom in primary life support education.Chin J Nurs Educ.2016;13:273-276(in Chinese).

50.Chen W.Application of micro lecture based flipped classroom in physiology laboratory class of associated degree nursing students.J Med Theor Pract.2016;29:1255-1256(in Chinese).

51.Lu MY.The usefulness of flipped classroom in fundamental nursing of higher vocational education.Nurs Pract Res.2016;20:129-130(in Chinese).

52.Chen L,Cao XP,Yuan AD,Luo Y,Liang SY.A study of flipped classroom model on medical nursing of higher vocational education.Chin Gen Pract Nurs.2016;12:1277-1278(in Chinese).

53.Della-Ratta CB.Flipping the classroom with team-based learning in undergraduate nursing education.Nurse Educ.2015;40:71-74.