Mixed methods research in complementary and alternative medicine:a scoping review

DU Wanqing,WANG Liuding,JIA Min,LIANG Xiao,LI Bo,ZHANG Yunling,LIAO Xing

DU Wanqing,WANG Liuding,JIA Min,LIANG Xiao,ZHANG Yunling,Department of Neurology,Xiyuan Hospital,China Academy of Chinese Medical Sciences,Beijing100091,China

DU Wanqing,LIAO Xing,Center for Evidence-based Chinese Medicine,Institute of Basic Research in Clinical Medicine,China Academy of Chinese Medical Sciences,Beijing 100700,China

DU Wanqing,Xiyuan School of Clinical Medicine,Beijing University of Chinese Medicine,Beijing100029,China

LI Bo,Department of Clinical Epidemiology,Beijing Traditional Chinese Medicine Hospital,Capital Medical University,Beijing Institute of Traditional Chinese Medicine,Beijing 100010,China

Abstract OBJECTIVE:To update the current characteristics about the scope and quality of mixed methods research (MMR)in complementary and alternative medicine (CAM) after nearly 10 years.METHODS:A 5-stage approach for conducting a scoping review was adopted.Articles published on the top 10 journals in CAM with the highest impact factor in 2020 were screened for MMR.Information of included articles were extracted,and then synthesized to illustrate the current state.Methodological quality was evaluated according to the Mixed Method Appraisal Tool (MMAT)2018 version.RESULTS:A total of 55 (55/2991,2%) articles using mixed methods were retrieved,including 17 medical studies and 38 ethnobotanical studies.We performed an in-depth analysis on the 17 medical studies,which studied cancer,stress,pain,fatigue,exercises,mindfulness intervention,herbal medicine use,art and acupuncture.Thirteen pilot studies applied MMR to evaluate the feasibility of interventions or programs(13/17,76%);phenomenology was inferred as the most common philosophical assumptions (13/17,76%);the most applied type of MMR was convergent design (16/17,94%);integration often took place at integration (12/17,71%).Among the 16 eligible studies for quality appraisal,majority were rated as good (14/16,88%),whereas two studies were rated as poorly described.Primarily,a poor rating was due to incomplete reporting of data analysis and citations in qualitative components;lack of confounder controlling and the sampling strategy in quantitative components;poor description of integration and justification for mixed methods.Comparing with the previous review,fewer MMR were published in 2020 in CAM,but the proportion of studies that clearly reported MMR has increased by 4 times (4%→15%).CONCLUSION:CAM researchers need to realize the benefits that MMR can have on conducting further health care research.Our findings highlight that applying MMR will be helpful to understand the complex dynamics and interdisciplinary nature of complex intervention.In addition,addressing a standardized reporting criteria for MMR is recommended.

Keywords:complementary therapies;review;biomedical research;pilot projects;mixed methods

1.INTRODUCTION

Mixed methods research (MMR) combines the techniques of qualitative (QUAL) and quantitative(QUAN) research to address a range of complex research questions.1Since the development of MMR in the 1980s,it has become an important research method in the social sciences.2MMR combines the strengths of QUAL and QUAN research which can characterize complex phenomena more fully than either approach alone.3MMR brings the findings from different methods together into a dialogue that gives a balanced view of a phenomenon.4Integration is at the heart of MMR and should feature through the design,methods,interpretation,and reporting stages of the research process.MMR can quantify QUAL data to summarize findings,leverage QUAL data to interpret QUAN data,and simultaneously measure and analyze the effects,trends,and factors of a new phenomenon.5Based on a convergent or sequential design and the emphasis on which component,different connections between components in an MMR study formed 3 core MMR designs.They are the sequential explanatory design,the sequential exploratory design,and the convergent design.2Health researchers are increasingly using MMR in biomedical and health services research,6palliative care,7nursing,8health care policy making,9and medical education10in recent years.However,its development has been slow in CAM.11In the medicine discipline where QUAN research (especially the randomized controlled trials) dominates,the QUAL component in the MMR can examine whether the intervention was delivered as intended,describe implementation processes,and generate understanding of why the intervention failed to work or how its effectiveness was promoted or limited in the real world.It is prevalent that including a supplemental QUAL component within experimental or quasi-experimental studies of complex interventions in the area of health research.12Findings from QUAL study may moderate publication biases against QUAN study lacking intervention effectiveness by both explaining negative results and informing subsequent study.Over the years many medical researchers have justified the use of MMR as one that has the power to uncover important evidence that may otherwise be overlooked.

Complementary and Alternative Medicine (CAM) refers to a group of diverse medical practices and products that are not generally considered part of conventional medicine,such as acupuncture,herbal medicine,prayers and hypnotherapy.13CAM is an area with great public interest and activity,both nationally and worldwide.Patients have the right to make health care decisions that accord with their own preferences,wishes and values.This encompasses decisions to attain CAM either as an adjunct to conventional medicine,or instead of conventional medicine.Patients are looking for CAM for answers to complex medical problems,strategies for improved wellness,or relief from acute medical symptoms when conventional therapies are unable to relieve their health issues.CAM interventions are highly dependent on the interaction between practitioners and patients on an individual basis.14Hence there are specific challenges for singular QUAN methods in CAM:not easily applicable to conduct blinding;inadequate assessment for multimodal effect;difficulty to interpret personalized difference,15which making this area especially amenable to MMR.MMR may offer a means of capturing the complex nature of CAM because the integration of QUAL and QUAN approaches corresponds with the multidimensional practice that symbolizes CAM.Crucially,CAM researchers need to integrate these sources of knowledge to be able to articulate what comprises CAM and to practice CAM effectively.MMR has been handled as being useful in CAM covering diverse interventions such as music therapy,16Qigong exercise,17and herbal medicine use,18to quantify the effectiveness of treatments and qualify the practice experience.MMR may provide valuable insights into individual behaviors in the application of CAM which contribute to the uptake of CAM experience,and may help increase understanding of both the magnitude of efficacy (viaQUAN methods) and of patient-related issues,such as experiences and reasons for nonadherence of CAM (viaQUAL methods).

A previous review published in 2012 found only 4% of research in CAM journals using mixed method,and pointed out their poor quality of insufficient reporting philosophical tension,analytic methods,reflexivity and study procedures.11In view of the advances in MMR and quality appraisal tool,we find it necessary to update the profile of the application and qualities of MMR studies in the field of CAM.We highlighted the studies on clinical issues to promote the transformation from research to application.

2.MATERIALS AND METHODS

2.1.Screening Mixed Methods Research

We performed this review using Arksey and O'Malley's scoping review methodology as a guide,the 5-stage approach for conducting scoping reviews.19We organized a transdisciplinary research team by adopting the Rosenfield’s taxonomy of three levels of collaboration,that is researchers work jointly using a shared conceptual framework drawing together disciplinary-specific theories,concepts,and approaches to address common problems.20Our team integrated both varied methodological (research methodology) and disciplinary (CAM) expertise from the team members.

2.1.1.Defining the research question

We still kept the same research question with the previous review to evaluate the prevalence and quality of mixed methods studies in CAM.11In addition,we compare the results of the current review with the previous review.Among the included MMR studies in CAM,we aimed to a.examine the extent of use of MMR(by disease type);b.describe which researchers are using mixed methods designs (by country and discipline);c.determine the MMR designs commonly used (including rationale for using MMR,study sequence,priority,and time of integration);d.report the philosophy and study designs for both QUAL and QUAN components;and e.appraise the quality of MMR.

2.1.2.Searching relevant studies

We adopted a similar search strategy as the 2012 MMR review.The “Integrative and complementary medicine”category in the annual Web of Science Journal Citation Reports released by Clarivate in 2020 was searched to select the top 10 journals in CAM with the highest impact factor (Table 1).As the 2021 Early Access Policy Change was conducted,which affected the calculation of the impact factors in 2021,we did not update the search in 2021.All the articles published on these journals in 2020 were exported to literature manager software Endnote X8 and a temporary literature database was established.Title/abstract of each article was screened roughly and the related articles were kept.And then careful selecting for both the content area of interest and for the methodology of interest were conducted.

Search strategy:

1# (((((((((("Journal of Ginseng Research"[Journal]) OR("Phytomedicine"[Journal])) OR ("Journal of Ethnopharmacology"[Journal])) OR ("The American journal of Chinese medicine"[Journal])) OR ("Chinese Medicine"[Journal])) OR ("BMC complementary medicine and therapies"[Journal])) OR ("PLANTA MEDICA"[Journal])) OR ("Journal of Integrative Medicine"[Journal])) OR ("Integrative cancer therapies"[Journal])) OR ("Journal of alternative and complementary medicine (New York,N.Y.)"[Journal]))2# AND (("2020/01/01"[Date -Publication]:"2020/12/31"[Date -Publication]))

2.1.3.Selecting relevant studies

We screened relevant articles in three steps;first by screening title and abstract,second by the retrieval and checking of full-text articles,followed by verifying and discussion.

Firstly,one reviewer (DWQ) reviewed the titles and abstracts of each article,and preliminarily classified them into QUAL study,QUAN study,MMR study,and unclassifiable study.Secondly,another reviewer (WLD)reviewed the classification using abstract searching according to “distinctive label words” in MMR:mixed method,mixed-method,mixing method,quanlitative,survey,questionnaire,interview,field work,focus group,semi-structured,knowledge,perception,motivation,feedback,perspective,belief.Lastly,the reviewer verified these relevant articles based on a set of predefined criteria,determined which articles could be included.Divergences were solved by discussing with a more experienced researcher in MMR.The criteria for selecting eligible studies in this review were as following:a.articles reported mixed methods results (defined as at least one QUAL and one QUAN strand with clear integration) were included.Because the explicit focus of this review was health care study,we did not analyze ethnobotanical studies with MMR,as they focus on the interactions between plants and people,with particular emphasis on traditional or ethnic cultures.

2.1.4.Charting the Data

A data extraction form was developed and piloted on three randomly selected articles by two reviewers (DWQ and WLD).The data extraction form was developed and used to collect prespecified elements of each study.When differences arose,the reviewers consulted with a third reviewer,and where necessary,consensus was reached through group discussion.Extracted information included:(a) publication characteristics (journal,disease,discipline,country,sample size,focus of study);(b)mixed methods (justification for using MMR,QUAL methodology and design,QUAN design,and MMR design,priority and time of integration);(c) quality appraising information.

We adopted the classifications by Sandelowski for the purpose of undertaking MMR.21Purposes include complementarity (use of different methods to provide a more complete understanding of a given phenomenon),triangulation (the use of different methods to increase the credibility of the findings),and development (use different methods in ways to complement one another,e.g.,interviews inform developing a survey).

We sought to assess whether MMR researchers in CAM described how they reconciled research paradigmatic differences,for instance,constructionism in QUAL studies and post-positivism in QUAN studies.

In terms of the timing and priority of the 2 strands we classified the core design of included MMR studies into 3 kinds based on:2(a) convergent (data for both strands collected simultaneously);(b) sequential explanatory(QUAN data collection and analysis connects to and explained by QUAL data);(c) sequential exploratory(data from QUAL results inform the collection of QUAN data.These core designs (more than 2 strands) may intersect to more complex designs as multiphase design.Regarding the data from QUAN study and QUAL study that must be merged or integrated in MMR,which can take place at various points,we recorded the point of integration for each of the included MMR study.Integration took place at three stages:study design level,methods level,and interpretation and reporting level.22We applied the Mixed Methods Appraisal Tool (MMAT)2018 version23to assess the methodological quality of the included MMR studies.The MMAT Version 2018 consists of a screening block and 3 subsets:(a) QUAL component quality;(b) QUAN component quality(QUAN randomized controlled trials,or QUAN nonrandomized studies,or QUAN descriptive studies);(c)MM quality.Each subset has 5 items regarding study quality,and studies are rated as “yes”,“no”,or “cannot tell” for applicable item.According to a 2020 update document of MMAT24,the quality score of each subset is ranked from * to *****,and the overall score of each study equals to the score of the weakest subset.An overall score higher than or equal to 60% is recognized as good quality in this study,while others are determined as poor.

2.1.5.Collating,summarizing,and reporting the results

We completed descriptive statistics using Excel spreadsheets.Our analysis presented findings related to the prevalence,features,quality appraisal and the comparison between 2012 and 2020 in MMR use in the top ten CAM journals.We also described shifts in the frequency and trends,types of MMR,proportion of studies explicitly mentioned MMR,and major quality limitations.This presentation of results allows for comparison of the popularity of different application in MMR studies in CAM.

As is well known,reflexivity is essential in QUAL research.The primary goal of reflexivity is to reduce the likelihood of researcher bias and in turn it could improve the credibility of the study.Therefore,in this review,researcher reflexivity was maintained in order to minimize bias in interpretation and to maintain the integrity of work during the data collection and analyzing process.25All the group member of this review discussed their view-point and consensual decisions.

3.RESULTS

3.1.Use of Mixed Methods Research in CAM

A total of 2991 articles published in the year 2020 from the top 10 CAM journals were screened (Figure 1).We found QUAN studies dominates CAM research with a large proportion (2339/2991,78%) of all published papers,whereas,QUAL studies with very low proportion(20/2991,1%);remaining 632 related MMR articles or unclassifiable articles (review,response,editorial,etc.).After full-text review of 173 potentially relevant articles,55 reports were identified that met all criteria for inclusion in the review.Reasons for exclusion of 118 articles at full-text stage were:QUAN studies that involved an interview,survey or questionnaire,but no QUAL analysis (n=114),QUAL studies with simple counts of results (n=2),no integration between the two strands (n=1),failure to find the whole published information about the two strands (n=1).

Figure 1 Flowchart of screening mixed methods research

Of the 55 MMR studies,only 8 studies reported MMR explicitly,however,most researchers did not state in the use of MMR but in practice the design was observed to be applied.Among the 55 included MMR studies,38(38/55,69%) were ethnobotanical research.26-36Generally,they converted the species name appeared in QUAL interviews into QUAN citation frequency,so as to explore the knowledge and use of medicinal plants in a region.These botanical studies were intentionally left out as they were beyond the knowledge of the reviewers.The characteristics of the rest 17 MMR studies (31%)related with health care for narrative synthesis are summarized in Table 1.16-18,37-51All the 17 MMR studies derive from a total of 7 countries.The bulk of MMR in this area came from researchers from the United States(8/17,47%).Some journals seemed more friendly to MMR likeIntegrative cancer therapies(9/17,53%) andJournal of alternative and complementary medicine(5/17,29%).Cancer was the focus of 47% (8/17) of studies included,whereas stress (4/17,24%),pain (3/17,18%),and other allied disease fields (e.g.,fatigue,addiction,pediatrics,1/17,6% respectively) also published MMR studies.

Three quarters of articles (13/17,76%) evaluated an intervention,program,or service,such as various exercises (4/17,24%),mindfulness intervention (3/17,18%),art (1/17,6%),music (1/17,6%),acupuncture(1/17,6%),culinary course (1/17,6%),to combination therapies (1/17,6%).Other studies investigating the use of a therapy in a region,as well as the related factors contribute to this phenomenon,like herbal medicine(2/17,12%) and CAM use (Table 1).

3.2.Features of Mixed Methods Research in CAM

We referred to Sandelowski for the justification for using mixed methods,Creswell and Plano Clark for the type of mixed methods design,2and Fetters,Curry and Creswell for the integration feature to categorize the substantive content of the articles.22All the articles included enough information in their methods section to allow us to categorize the features used according to their typologies.

Disparities in the philosophical assumptions between different paradigms which underpin QUAL and QUAN traditions were described and reconciled or justified in 2 studies (12%) namely,phenomenology16or ethnography.48One study mentioned grounded theory,but was then judged by the reviewers as phenomenology.43The rest 15 studies provided no explicit philosophical assumptions,but the philosophy could still be inferred from the content of the articles.Finally,we divided 17 MMR studies into three kinds:13 articles with phenomenology,3 with ethnography and the rest one with grounded-theory was judged as phenomenology Table 1).

Both the QUAN and QUAL methods used in the included MMR were appropriately reported individually.The QUAL component of the included studies used semi-structured or in-depth interviews (n=8,47%),open-ended questionnaires (n=4,24%),focus group (n=2,12%) and combining of multiple methods (n=3,18%).Within the QUAN strands,survey (n=7,41%),prospective,single-group,pre-post study (n=5,29%),randomized controlled trial (n=2,12%),prospective cohort study (n=2,12%) and non-randomized trial (n=1,6%) were used.Most of the articles provided an adequate and complete description of the procedures followed in the sampling,data collection,and analysis stages of both components.

Only 3 (18%) studies named the MM design in the text,while others were judged by the reviewers.Justification for using MM was summarized by the reviewers into 3 types.The most common rationale was complementarity(15/17,88%),using different methods to provide a more complete understanding of a given phenomenon,for instance,exploring the feasibility (acceptability,experience,perspective,use,factors,satisfaction,effectiveness and safety) of a specific intervention.Triangulation was adopted in one study,which seek convergence of opinions from nurses,patients,and observers,48and development rationale was shown in a sequential explanatory study.

The timing of most studies was concurrent (16/17,94%),which entailed the independent collection of QUAN and QUAL data,the separate analysis of both types of data,and the final merging of the two datasets to see whether the findings converged,diverged or enhanced each other.Of sequentially conducted studies,explanatory form(QUAN study followed by QUAL exploration for depth)was reported in one study,whereas exploratory form was not used in any study.A multiphase design including >2 strands was used in none studies either.The priority of the strands was QUAN in most included studies,except for one study.48

Integration was most commonly undertaken in the interpretation phase of the study (12/17,71%),where the results were presented separately and there was some integration that took place in the discussion and conclusion sections of the report.Some studies used code counts to reveal the frequency of common responses that they transferred QUAL into QUAN data (4/17,24%).

These were considered to be integrated at the analysis stage.Integration at the study design stage was used in the only sequentially conducted mixed methods studies(1/17,6%).

3.3.Quality appraisal

Of the 17 included MMR studies,since one study was a protocol,50other studies were then assessed using MMAT.Due to only one QUAN study design(randomized controlled trial,or non-randomized study or survey) involved in each of the 16 MMR study,15 items of the 25 items and an overall quality (lowest score of the study components) assessing were performed.Only two of the 16 MMR studies fully conformed with all 15 items for good reporting of mixed methods studies.Table 1 classifies the quality of the 16 MMR studies in terms of their compliance with each of the MMAT criteria.

Among the QUAL items in MMAT,item 1,2 and 5 were fulfilled in most articles.We have not found obvious inconsistency in the induction from QUAL data to interpretation.Most studies used appropriate QUAL approach (94%) and data collection methods (88%).

More than half studies provide adequate data analyzing process (69%) or quotes of original resource (75%).There was no description of the development of the coding scheme,or the typical quotations of informants in two articles.39,44Eight studies illustrated the identity of therapists,interviewers or coders,16,37,41-43,47-49but none have specified the theoretical orientation of researchers.We assessed the QUAN components of the 16 MMR studies (two randomized controlled trial,seven nonrandomized studies and seven surveys) based on the MMAT.Among the two randomized controlled trials,both studies collected patient-report outcomes,so the outcome assessment could not be blinded to the patients.Adherence to intervention was a problem in one pragmatic trial with long follow-up,as the lost rate has climbed to even higher than 30%.Other items were sufficiently described in the two randomized controlled trials.Two poor-quality items in the non-randomized studies were the representativeness of target population and the control of confounding factors (only 3 studies met the criteria).Items of the measurements of outcomes and administration of intervention (or exposure) of these studies were,basically,appropriately reported.“Not Mentioned” occurred often in the seven surveys:4 studies did not provide a sampling strategy,and we could not judge the representativeness of respondents;5 studies had a high or unknown non-response rate.

In terms of MM quality,almost a half studies provided a clear rationale for using MM.In most cases,MMR studies scored no less than *** in overall quality score(14/16,88%),and were rated as good quality in this study.Only three studies met with all the criteria (*****).17,41,42Particularly,all the studies that explicitly stated using MMR,16,17,37,41,43,47,48were no less than ***.

3.4.Comparison between 2012 and 2020 studies

Overall,when comparing the quality of MMR studies included in our review with the results of Bishop FLetal,11who assessed 80 MMR studies (89 articles,9 studies published QUAL and QUAN components in separate articles) published in 2012 on the top ten CAM journals,three journals of which are different from our journals according to the new impact factor in 2020 (Suppl Table 1).QUAN studies still dominated CAM research,while fewer MMR were published in 2020 than 2012 in these journals (Table 3).11However,Bishop's research may overestimate the number of MMR in 2012.We reevaluated the included studies,and classified them into medical research and ethnobotanical research.We found that 6 articles did not meet our inclusion criteria for MMR (specific reasons listed in Suppl Table 2).5 studies were due to the lack of QUAL components,2 of which were surveys with one open-ended question only,and the other 3 were actually descriptive analysis of text data.No QUAL analysis or method was mentioned in these articles.One study was an independent QUAL study recruited a part of participants from a clinical study.In other words,when designing the research,these researchers only considered one component,rather than combing both methods.This may partly explain the decline of MMR.The ratio of proportion of medical studies to ethnobotanical studies almost stabilized at 1:2 in these years (medical 35%→31%,ethnobotanical 65%→69%).The proportion of studies in which researchers clearly reported using MMR have increased by 4 times(3→8,4%→15%).The quality of research is difficult to compare,because many items of Quality Appraisal Tools have changed,and we have not been able to obtain specific scores of each study in 2012.

Table 3 Comparison between 2012 research and 2020 research

4.DISCUSSION

The popularity of MMR in studies of biomedicine and CAM could be because of multiple reasons,including the publication of articles and researchers realizing the value of MMR and the growing acceptability of using QUAL methods to reveal information unavailable when using QUAN methods.52However,the rate of prevalence is still very low given the importance of the paradigm and the field of CAM itself.This review therefore,is important because it provides useful information regarding the extent to which CAM research are keeping abreast of MMR.MMR is not new,the field using MMR in health-related research summarized ten years ago included cardiology,pharmacy,family medicine,pediatric oncology nursing,mental health services,disabilities,and public health nutrition and the settings vary from the clinic to the social context of daily activities and relationships.53There is no doubt that MMR is valuable in medicine,and many important research questions in CAM need MMR (particularly synthesizing QUAN and QUAL findings).In the context of CAM,which is commonly associated with individualized medicine,QUAL studies allow a deeper understanding of subjectivity and complexity within human experience,turning them into a powerful tool to increase our knowledge of important processes within CAM.54In this review,we described the characteristics,methodological features,and quality of the MMR studies published in ten top CAM journals in 2020 and compared the results with the analysis of the 2012 review.11Our results have shed light on the prevalence,application,design and quality of MMR in CAM.Our findings showed that fewer than 2% (55/2991) of the empirical studies published in 2020 in CAM used a mixed methods design,of which only 15% explicitly stated MMR.This low frequency is even less than the results of previous reviews on MMR in other fields of health care.6,7,55It is believed that the usefulness of this methodological approach in answering the kind of complex questions that are being asked in CAM.The low frequency of MMR in CAM that in the ten top journals could be explained by a number of practical barriers.These include the pressure to generate evidence rapidly in the whole healthcare environments and build interdisciplinary teams of qualified researchers with varied methodological skills.56,57

An interesting phenomenon found in this review is that part from medical research,ethnobotanical studies account for a large portion of MMR in CAM.This could be explained by the inherent characteristic of ethnobotany,generally defined as the “science of people’s interaction with plants”.58It requires researchers to approach native people,and document abundant knowledge and usage of medicinal plants.59Then these QUAL data,fieldnotes and interviews,are scientifically identified as species and frequencies,and finally displayed as QUAN results.

The findings of this review highlight the need to improve reporting of MMR,in consistence with studies in other fields.7,60A 5-level overall grading system from MMAT amendment was adopted for the assessment to facilitate quantifying the quality of MMR in CAM.2488% of studies were rated as good quality (at least ***,but only 3 studies fulfilled the criteria of MMAT.

In accordance with findings reported in recent reviews,7,55,61we found that many studies reported a convergent design and adopted complementarity as the main motives for using MMR.In these studies,the findings of the QUAN and the QUAL components were used to enrich on the findings of the other component or to provide further insights.Many of these studies reported evaluation studies in which QUAN and the QUAL data were combined to assess the different components of an intervention while incorporating contextual information provided by CAM practitioners and patients.The high proportion of mixed methods evaluation studies found in this review demonstrates that CAM researchers are aware of the value of MMR in providing a more comprehensive evaluation of complex CAM interventions,just like other health care field argued by Farqhuar and colleagues.62

A key finding of this review is that three quarters of MMR studies in CAM medical studies were designed to test the feasibility of a CAM intervention as pilot studies.Pilot study,sometimes named as feasibility study,usually functions as a safeguard ahead of large-scale trial to test and optimize the design of main study.63,64A pilot study in this review had only 10 participants,but still provided rich feedback about the intervention.40

Besides,MMR is particularly popular in conditions with outcomes like pain,stress and fatigue.It is possible that this relates to the focus of CAM practitioners,where value is placed on soft outcomes understanding patients’experience of the illness and CAM intervention,compared with other fields where evaluation of hard clinical outcomes often remains the priority.Soft outcomes like narrative responses can provide concrete support to the quantified scores,and even highlight additional benefits beyond the scales.41This is particularly important in the patient centered outcomes research which focus on outcomes that are meaningful to patients.65Interestingly,cancer research is the most common field.This may be because cancer patients generally face varying degrees of pain,stress,and fatigue that impact their quality of life.66

There are various QUAN and QUAL methods used in the included MMR.We found that the methods of the QUAL strand in the most of the included studies was interview methods (individual or group),which hold theoretical and substantive depth than other methods by its strength in obtaining patient perspectives and insights.QUAL methods generate subjective evidence shaped by the researchers’ experience in collecting and analyzing.67However,almost a quarter of QUAL components failed to provide detailed analyzing process and original quotes,and very little,if any,description is provided as to theoretical background of researchers in all studies.In terms of QUAN components,efforts should be made to eliminate confounding by improving study design and data analyzing method,such as using an external control whenever possible,or using regression,propensity score,and subgroup analysis.68.Judgement could hardly be made about representative due to the unclear reporting why eligible patients did not participate and how the sample represent the target population.

Integration is regarded as the core of MMR.65However,studies in this review lack adequate description and justification for data integration,especially during analysis and discussion,which was supported by the findings of other MMR reviews.11,60Some studies elucidate the necessity of using QUAL and QUAN methods;16-18,37-39,41,42,47and how the two parts correlate,41-43or conflict with each other.16,17In most studies,authors reported findings separately for the QUAN and QUAL parts of their studies.As such,it was difficult to determine the appropriate integration.It should be noted that several ways of integrating findings from each strand in an iterative fashion,such as transformation,69parallel presenting results,or in a sequential study.70Researchers should specify how mixed methods are integrated across design,data sources,and/or data collection phases.65

It is also important to understand the stance they are using with respect to the paradigm controversy.However,few of the included studies explicitly stated their perspective on reconciling the paradigms of the two strands,The predominant QUAL methodology used in our review was recognized as phenomenology,due to the nature of these studies in identifying core facets of the phenomenon from patients’ perspective.67

As this is a scoping review,which is different from a systematic reviews or meta-analyses,do not narrow the parameters of the review to specific trials or require quality assessment.A scoping review holds a broader range of study types included with the aim of examining a broadly covered topic to comprehensively and systematically map the literature and identify key concepts,theories,evidence,or research gaps in a particular field.19The current scoping review aimed to map the status quo of MMR application in the field of CAM based on the analysis on the top ten CAM journals in 2020 and highlighted important implications for promoting and facilitating the use of MMS in CAM and improving the quality of published research.MMR will be inevitable in CAM due to the fact that the practical context demand for both generalizability and particularity.Although,MMR which is an established scientific approach to the individual,thus far has been explicitly used very rarely.

The status quo of MMR studies in CAM is fragmented and poorly indexed.However,we believe that the methodological structures we identify here start an important conversation that is relevant and applicable to other fields.Our study holds some useful insights and even within the limitations of its focus,we have been able to discern some interesting shifts across time.

5.RECOMMENDATIONS FOR FUTURE MMR IN CAM

MMR can be extraordinarily helpful to CAM research efforts,but the required effort,expertise,and mixing techniques should be taken into consideration. We believe that MMR in CAM could be optimized by aligning the three aspects listed below.

5.1.Reference standards for designing and conducting MMR in CAM

CAM researchers should make an effort to write concisely in order to be able to represent the complexity of the process and the findings of MMR with sufficient clarity.Guidance for designing and conducting MMR which focus on health care71and the report ‘Best Practices for Mixed Methods Research in the Health Sciences’ commissioned by the US NIH Office of Behavioral and Social Science Research are available to guide the design and conduct MMR in CAM.53

5.2.Critical considerations in the quality and rigor of conducting MMR in CAM

There are three important issues that should be considered as following:alignment of aims,methods,and research team capacity;methodological criteria for each component;articulation and implementation of plans for deliberate integration of both components.6Since MMR is an activity that demands specialized mixing techniques,CAM researchers should receive specific training in this methodology or else collaborate with researchers holding the expertise in this area.

5.3.Adequate reporting MMR in CAM

MMR should be more explicitly reported when researchers prepare reports of MMR findings,in a form of complete and transparent reporting,which could aid their critical appraisal and interpretation.Therefore,MMR researcher in CAM should report the ‘mixing’,as appropriate,rather than reporting QUAL-QUAN components separately;explicitly outline what has been mixed,implicitly acknowledging the required rigor,expertise,time,and effort;specifically discuss ‘mixing’findings particularly.The ‘Good Reporting of A Mixed Methods Study checklist’ which identifies the aspects of a study that should be addressed for appropriately transparent reporting,including the rationale for a MMR study and a description of the design,the methods of each component,the procedures for integration,the limitations of each method,and insights gained from MMR.60

In addition,the methodological standards reported recently by Patient-Centered Outcomes Research Institute for QUAL methods and MMR can be used to develop and evaluate proposals,conduct the research,and interpret findings.65

Finally,we would call on CAM journal editors to play their role in reviewing their practices.CAM Journals could give clear guidance for authors to report MMR.

The most important limitation of our review was the examination of MMR in CAM in only one year and in ten targeted CAM journals.It gave a snapshot profile that is constrained by the focused data analysis of one year in the ten journals which limited our ability to speculate the dynamic change over the past decades.It is possible that MMR studies were missed in our identification due to some controversial screening and that the number of these included studies underrepresented.However,our analysis gives additional and deeper insights into the patterns and problems in MMR and complements the previous review on the subject.11As the next step in our investigation of MMR in CAM,we have begun to conduct a new review at a different point in time,with a different set of eligible criteria,the conclusions would be different.We debated how to improve the quality of MMR that we deemed to exhibit poor quality of MMR in CAM.We do not intend to offend these researchers whose studies have been evaluated by giving the details of such studies.After all this review is reporting on second literature analysis which may hold implicit bias.Accepting the limitations of not having reported on a group of articles included in the review,we have however highlighted what we recommend for future MMR which is likely to be more helpful to readers,than criticizing those deemed to be of poorer quality in CAM.In conclusion,Mixed method research has wide-ranging applications in the field of CAM.Our findings have confirmed that MMR can be specifically valuable for addressing the multi-faceted nature of CAM,as well as for evaluating complex CAM interventions.Furthermore,CAM researchers are only beginning to familiarize themselves with and use MMR.This review makes a series of recommendations for researchers and journals that could enhance the quality of MMR.

6.ACKNOWLEDGMENTS

The authors thank Peng Fucong for her useful comments on QUAL methodology,Flora Wang for language proofreading and all the team members of the Evidencebased medicine platform.

7.REFERENCES

1.Palinkas LA,Mendon SJ,Hamilton AB.Innovations in mixed methods evaluations.Annu Rev Public Health 2019;40:423-42.

2.Creswell JW,Plano Clark VL.Designing and conducting mixed methods research.3rd ed.CA:Sage Publications,2017:34.

3.Wisdom JP,Cavaleri MA,Onwuegbuzie AJ,Green CA.Methodological reporting in qualitative,quantitative,and mixed methods health services research articles.Health Services Res 2012;47:721-45.

4.Fielding NG.Triangulation and mixed methods designs:data integration with new research technologies.J Mixed Methods Res 2012;6:124-36.

5.Pluye P,Hong QN.Combining the power of stories and the power of numbers:mixed methods research and mixed studies reviews.Annu Rev Public Health 2014;35:29-45.

6.Curry LA,Krumholz HM,O'Cathain A,Plano Clark VL,Cherlin E,Bradley EH.Mixed methods in biomedical and health services research.Circ Cardiovasc Qual Outcomes 2013;6:119-23.

7.Fàbregues S,Hong QN,Escalante-Barrios EL,Guetterman TC,Meneses J,Fetters MD.A methodological review of mixed methods research in palliative and end-of-life care (2014-2019).Int J Environ Res Public Health 2020;17:3853.

8.Sharma S,Rafferty AM,Boiko O.The role and contribution of nurses to patient flow management in acute hospitals:a systematic review of mixed methods studies.Int J Nurs Stud 2020;110:103709.

9.De Allegri M,Sieleunou I,Abiiro GA,Ridde V.How far is mixed methods research in the field of health policy and systems in Africa? A scoping review.Health Policy Plan 2018;33:445-55.

10.Maudsley G.Mixing it but not mixed-up:mixed methods research in medical education (a critical narrative review).Med Teach 2011;33:e92-104.

11.Bishop FL,Holmes MM.Mixed methods in CAM research:a systematic review of studies published in 2012.Evid Based Complement Alternat Med 2013;2013:187365.

12.Lewin S,Glenton C,Oxman AD.Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions:methodological study.Bmj 2009;339:b3496.

13.World Health Organization.WHO global report on traditional and complementary medicine 2019.Geneva:World Health Organization,2019:10-20.

14.Ahn AC,Nahin RL,Calabrese C,et al.Applying principles from complex systems to studying the efficacy of CAM therapies.J Altern Complement Med 2010;16:1015-22.

15.Veziari Y,Leach MJ,Kumar S.Barriers to the conduct and application of research in complementary and alternative medicine:a systematic review.BMC Complement Altern Med 2017;17:166.

16.Low MY,Lacson C,Zhang F,Kesslick A,Bradt J.Vocal music therapy for chronic pain:a mixed methods feasibility study.J Altern Complement Med 2020;26:113-22.

17.Osypiuk K,Ligibel J,Giobbie-Hurder A,et al.Qigong mind-body exercise as a biopsychosocial therapy for persistent post-surgical pain in breast cancer:a pilot study.Integr Cancer Ther 2020;19:1534735419893766.

18.Nsibirwa S,Anguzu G,Kamukama S,Ocama P,Nankya-Mutyoba J.Herbal medicine use among patients with viral and non-viral Hepatitis in Uganda:prevalence,patterns and related factors.BMC Complement Med Ther 2020;20:169.

19.Arksey H,O'Malley L.Scoping studies:towards a methodological framework.Int J Social Res Methodology 2005;8:19-32.

20.Rosenfield PL.The potential of transdisciplinary research for sustaining and extending linkages between the health and social sciences.Soc Sci Med 1992;35:1343-57.

21.Sandelowski M.Combining qualitative and quantitative sampling,data collection,and analysis techniques in mixed-method studies.Res Nurs Health.2000;23:246-55.

22.Fetters MD,Curry LA,Creswell JW.Achieving integration in mixed methods designs-principles and practices.Health Serv Res 2013;48:2134-56.

23.Hong QN,Pluye P,Fàbregues S,Bartlett G,Boardman F,Cargo M.Mixed Methods Appraisal Tool (MMAT),version 2018.Canadian Intellectual Property Office #1148552,Industry Canada,2018 Aug 1st.

24.Reporting the results of the MMAT.Mixed methods appraisal tool public,2020-12-09,cited 2022-05-27;1:2 screen.Available from URL:http://mixedmethodsappraisaltoolpublic.pbworks.com/w/browse/#view=ViewAllObjects.

25.Attia M,Edge J.Be(com)ing a reflexive researcher:a developmental approach to research methodology.Open Rev Edu Res 2017;4:33-45.

26.Abdusalam A,Zhang Y,Abudoushalamu M,et al.Documenting the heritage along the Silk Road:An ethnobotanical study of medicinal teas used in Southern Xinjiang,China.J Ethnopharmacol 2020;260:113012.

27.Ahoyo CC,Houéhanou TD,Yao?tcha AS,et al.Traditional medicinal knowledge of woody species across climatic zones in Benin (West Africa).J Ethnopharmacol 2021;265:113417.

28.Alqethami A,Aldhebiani AY,Teixidor-Toneu I.Medicinal plants used in Jeddah,Saudi Arabia:a gender perspective.J Ethnopharmacol 2020;257:112899.

29.Antonella Carabajal MP,Perea MC,Isla MI,Zampini IC.The use of jarilla native plants in a Diaguita-Calchaquí indigenous community from northwestern Argentina:an ethnobotanical,phytochemical and biological approach.J Ethnopharmacol 2020;247:112258.

30.Anywar G,Kakudidi E,Byamukama R,Mukonzo J,Schubert A,Oryem-Origa H.Indigenous traditional knowledge of medicinal plants used by herbalists in treating opportunistic infections among people living with HIV/AIDS in Uganda.J Ethnopharmacol 2020;246:112205.

31.Ataba E,Katawa G,Ritter M,et al.Ethnobotanical survey,anthelmintic effects and cytotoxicity of plants used for treatment of helminthiasis in the Central and Kara regions of Togo.BMC Complement Med Ther 2020;20:212.

32.Baldé MA,Tuenter E,Traoré MS,et al.Antimicrobial investigation of ethnobotanically selected guinean plant species.J Ethnopharmacol 2020;263:113232.

33.Barbosa F,Hlashwayo D,Sevastyanov V,et al.Medicinal plants sold for treatment of bacterial and parasitic diseases in humans in Maputo city markets,Mozambique.BMC Complement Med Ther 2020;20:19.

34.Chamorro MF,Ladio A.Native and exotic plants with edible fleshy fruits utilized in Patagonia and their role as sources of local functional foods.BMC Complement Med Ther 2020;20:155.

35.Dassou GH,Ouachinou JMS,Adomou AC,et al.Plant and natural product based homemade remedies for veterinary uses by the Peul community in Benin.J Ethnopharmacol 2020;261:113107.

36.Dhakal P,Chettri B,Lepcha S,Acharya BK.Rich yet undocumented ethnozoological practices of socio-culturally diverse indigenous communities of Sikkim Himalaya,India.J Ethnopharmacol 2020;249:112386.

37.Arden-Close E,Mitchell F,Davies G,et al.Mindfulness-based interventions in recurrent ovarian cancer:a mixed-methods feasibility study. Integr Cancer Ther 2020; 19:1534735420908341.

38.Bloom-Foster J,Mehl-Madrona L.An ultra-brief mindfulnessbased intervention for patients in treatment for opioid addiction with buprenorphine:a primary care feasibility pilot study.J Altern Complement Med 2020;26:34-43.

39.Conte E,Legacy M,Psihogios A,et al.A prospective outcomes pilot evaluation of inspire now:a program for people with lung cancer.Integr Cancer Ther 2020;19:1534735420983472.

40.Hirschberg R,Sylvia LG,Wright EC,et al.Collaborative songwriting intervention for veterans with post-traumatic stress disorder.J Altern Complement Med 2020;26:198-203.

41.Kaimal G,Carroll-Haskins K,Mensinger JL,Dieterich-Hartwell R,Biondo J,Levin WP.Outcomes of therapeutic artmaking in patients undergoing radiation oncology treatment:a mixedmethods pilot study.Integr Cancer Ther 2020;19:1534735420912835.

42.McDonnell KK,Gallerani DG,Newsome BR,et al.A prospective pilot study evaluating feasibility and preliminary effects of breathe easier:a mindfulness-based intervention for survivors of lung cancer and their family members (Dyads).Integr Cancer Ther 2020;19:1534735420969829.

43.Mehl A,Reif M,Zerm R,et al.Impact of a multimodal and combination therapy on self-regulation and internal coherence in german breast cancer survivors with chronic cancer-related fatigue:a mixed-method comprehensive cohort design study.Integr Cancer Ther 2020;19:1534735420935618.

44.Peters BC,Fields BE,Erdman EA.Hippotherapy practice and safety patterns in the united states:a descriptive survey study.J Altern Complement Med 2020;26:743-49.

45.Rahayu YYS,Araki T,Rosleine D.Factors affecting the use of herbal medicines in the universal health coverage system in Indonesia.J Ethnopharmacol 2020;260:112974.

46.Rothman JM,Bilici N,Mergler B,et al.A culinary medicine elective for clinically experienced medical students:a pilot study.J Altern Complement Med 2020;26:636-44.

47.Shewamene Z,Dune T,Smith CA.Use of traditional and complementary medicine for maternal health and wellbeing by African migrant women in Australia:a mixed method study.BMC Complement Med Ther 2020;20:60.

48.Stritter W,Rutert B,Eggert A,L?ngler A,Holmberg C,Seifert G.Evaluation of an integrative care program in pediatric oncology.Integr Cancer Ther 2020;19:1534735420928393.

49.Sylvia L,West E,Blackburn AM,et al.Acceptability of an adjunct equine-assisted activities and therapies program for veterans with posttraumatic stress disorder and/or traumatic brain injury.J Integr Med 2020;18:169-73.

50.Zhao Q,Zheng S,Delaney GP,et al.Acupuncture for cancer related pain:protocol for a pragmatic randomised wait-list controlled trial.Integr Cancer Ther 2020;19:1534735420976579.

51.Z?rg? S,Peters GY,Mkhitaryan S.Attitudes Underlying Reliance on Complementary and Alternative Medicine.Integr Cancer Ther 2020;19:1534735420910472.

52.Curry LA,Nembhard IM,Bradley EH.Qualitative and mixed methods provide unique contributions to outcomes research.Circulation 2009;119:1442-52.

53.Creswell JW,Klassen WA,Clark V,Smith KC.Best practices for mixed methods research in the health sciences.Qualitative Social Work 2013;12:541-45.

54.Alex,Broom.Using qualitative interviews in CAM research:A guide to study design,data collection and data analysis.Complement Ther Med 2005;13:65-73.

55.Campbell DJ,Tam-Tham H,Dhaliwal KK,et al.Use of mixed methods research in research on coronary artery disease,diabetes mellitus,and hypertension:a scoping review.Circ Cardiovasc Qual Outcomes 2017;10:e003310.

56.Tariq S,Woodman J.Using mixed methods in health research.Jrsm Short Reports 2013;4:2042533313479197.

57.Curry L,Nunez-Smith M.Mixed methods in health sciences research,CA:Sage Publications,2014:37-44.

58.Schultes RE,CReis SV.Ethnobotany:evolution of a discipline.Portland:Dioscorides Press,1995:264-83.

59.Thangaraj P.Pharmacological assays of plant-based natural products.Switzerland:Springer,2016:10-12.

60.O'Cathain A,Murphy E,Nicholl J.The quality of mixed methods studies in health services research.J Health Serv Res Policy 2008;13:92-8.

61.Irvine FE,Clark MT,Efstathiou N,et al.The state of mixed methods research in nursing:A focused mapping review and synthesis.J Adv Nurs 2020;76:2798-809.

62.Farquhar MC,Ewing G,Booth S.Using mixed methods to develop and evaluate complex interventions in palliative care research.Palliat Med 2011;25:748-57.

63.Arnold DM,Burns KE,Adhikari NK,Kho ME,Meade MO,Cook DJ.The design and interpretation of pilot trials in clinical research in critical care.Crit Care Med 2009;37(1 Suppl):S69-74.

64.In J.Introduction of a pilot study.Korean J Anesthesiol 2017;70:601-5.

65.Gaglio B,Henton M,Barbeau A,et al.Methodological standards for qualitative and mixed methods patient centered outcomes research.Bmj 2020;371:m4435.

66.Adam S,Thong MSY,Martin-Diener E,et al.Identifying classes of the pain,fatigue,and depression symptom cluster in long-term prostate cancer survivors-results from the multi-regional Prostate Cancer Survivorship Study in Switzerland (PROCAS).Support Care Cancer 2021;1:11.

67.Creswell JW,Poth CN.Qualitative inquiry &research designchoosing among five approaches.4th ed.London:SAGE Publications,2018:30-5.

68.Yan H,Karmur BS,Kulkarni AV.Comparing effects of treatment:controlling for confounding.Neurosurgery 2020;86:325-31.

69.Tashakkori A,Teddlie C.Mixed methodology:combining qualitative and quantitative approaches:SAGE Publications 1998:30-5.

70.Plano Clark VL.The adoption and practice of mixed methods:u.s.trends in federally funded health-related research.Qualitative Inquiry 2010;16:428-40.

71.Curry LA,Nembhard IM,Bradley EH.Qualitative and mixed methods provide unique contributions to outcomes research.Circulation 2009;119:1442-5