how to assess quality of literature review

• UNC Libraries
• HSL Academic Process
• Systematic Reviews
• Step 6: Assess Quality of Included Studies

Systematic Reviews: Step 6: Assess Quality of Included Studies

Created by health science librarians.

• Step 1: Complete Pre-Review Tasks
• Step 2: Develop a Protocol
• Step 3: Conduct Literature Searches
• Step 4: Manage Citations
• Step 5: Screen Citations

Assess studies for quality and bias

Critically appraise included studies, select a quality assessment tool, a closer look at popular tools, use covidence for quality assessment.

• Quality Assessment FAQs
• Step 7: Extract Data from Included Studies
• Step 8: Write the Review

  Check our FAQ's

   Email us

   Call (919) 962-0800

   Make an appointment with a librarian

  Request a systematic or scoping review consultation

About Step 6: Assess Quality of Included Studies

In step 6 you will evaluate the articles you included in your review for quality and bias. To do so, you will:

• Use quality assessment tools to grade each article.
• Create a summary of the quality of literature included in your review.

This page has links to quality assessment tools you can use to evaluate different study types. Librarians can help you find widely used tools to evaluate the articles in your review.

Reporting your review with PRISMA

If you reach the quality assessment step and choose to exclude articles for any reason, update the number of included and excluded studies in your PRISMA flow diagram.

Managing your review with Covidence

Covidence includes the Cochrane Risk of Bias 2.0 quality assessment template, but you can also create your own custom quality assessment template.

How a librarian can help with Step 6

• What the quality assessment or risk of bias stage of the review entails
• How to choose an appropriate quality assessment tool
• Best practices for reporting quality assessment results in your review

After the screening process is complete, the systematic review team must assess each article for quality and bias. There are various types of bias, some of which are outlined in the table below from the Cochrane Handbook.

The most important thing to remember when choosing a quality assessment tool is to pick one that was created and validated to assess the study design(s) of your included articles.

For example, if one item in the inclusion criteria of your systematic review is to only include randomized controlled trials (RCTs), then you need to pick a quality assessment tool specifically designed for RCTs (for example, the Cochrane Risk of Bias tool)

Once you have gathered your included studies, you will need to appraise the evidence for its relevance, reliability, validity, and applicability​.

Ask questions like:

Relevance:  ​.

• Is the research method/study design appropriate for answering the research question?​
• Are specific inclusion / exclusion criteria used? ​

Reliability:  ​

• Is the effect size practically relevant? How precise is the estimate of the effect? Were confidence intervals given?  ​

Validity: ​

• Were there enough subjects in the study to establish that the findings did not occur by chance?    ​
• Were subjects randomly allocated? Were the groups comparable? If not, could this have introduced bias?  ​
• Are the measurements/ tools validated by other studies?  ​
• Could there be confounding factors?   ​

Applicability:  ​

• Can the results be applied to my organization and my patient?   ​

What are Quality Assessment tools?

Quality Assessment tools are questionnaires created to help you assess the quality of a variety of study designs.  Depending on the types of studies you are analyzing, the questionnaire will be tailored to ask specific questions about the methodology of the study.  There are appraisal tools for most kinds of study designs.  You should choose a Quality Assessment tool that matches the types of studies you expect to see in your results.  If you have multiple types of study designs, you may wish to use several tools from one organization, such as the CASP or LEGEND tools, as they have a range of assessment tools for many study designs.

Click on a study design below to see some examples of quality assessment tools for that type of study.

Randomized Controlled Trials (RCTs)

• Cochrane Risk of Bias (ROB) 2.0 Tool Templates are tailored to randomized parallel-group trials, cluster-randomized parallel-group trails (including stepped-wedge designs), and randomized cross-over trails and other matched designs.
• CASP- Randomized Controlled Trial Appraisal Tool A checklist for RCTs created by the Critical Appraisal Skills Program (CASP)
• The Jadad Scale A scale that assesses the quality of published clinical trials based methods relevant to random assignment, double blinding, and the flow of patients
• CEBM-RCT A critical appraisal tool for RCTs from the Centre for Evidence Based Medicine (CEBM)
• Checklist for Randomized Controlled Trials (JBI) A critical appraisal checklist from the Joanna Briggs Institute (JBI)
• Scottish Intercollegiate Guidelines Network (SIGN) Checklists for quality assessment
• LEGEND Evidence Evaluation Tools A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.

Cohort Studies

• CASP- Cohort Studies A checklist created by the Critical Appraisal Skills Programme (CASP) to assess key criteria relevant to cohort studies
• Checklist for Cohort Studies (JBI) A checklist for cohort studies from the Joanna Briggs Institute
• The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses A validated tool for assessing case-control and cohort studies
• STROBE Checklist A checklist for quality assessment of case-control, cohort, and cross-sectional studies

Case-Control Studies

• CASP- Case Control Study A checklist created by the Critical Appraisal Skills Programme (CASP) to assess key criteria relevant to case-control studies
• Tool to Assess Risk of Bias in Case Control Studies by the CLARITY Group at McMaster University A quality assessment tool for case-control studies from the CLARITY Group at McMaster University
• JBI Checklist for Case-Control Studies A checklist created by the Joanna Briggs Institute

Cross-Sectional Studies

Diagnostic studies.

• CASP- Diagnostic Studies A checklist for diagnostic studies created by the Critical Appraisal Skills Program (CASP)
• QUADAS-2 A quality assessment tool developed by a team at the Bristol Medical School: Population Health Sciences at the University of Bristol
• Critical Appraisal Checklist for Diagnostic Test Accuracy Studies (JBI) A checklist for quality assessment of diagnostic studies developed by the Joanna Briggs Institute

Economic Studies

• Consensus Health Economic Criteria (CHEC) List 19 yes-or-no questions, one for each category to assess economic evaluations
• CASP- Economic Evaluation A checklist for quality assessment of economic studies by the Critical Appraisal Skills Programme

Mixed Methods

• McGill Mixed Methods Appraisal Tool (MMAT) 2018 User Guide See full site for additional information, including FAQ's, references and resources, earlier versions, and more

Qualitative Studies

• CASP- Qualitative Studies 10 questions to help assess qualitative research from the Critical Appraisal Skills Programme

Systematic Reviews and Meta-Analyses

• JBI Critical Appraisal Checklist for Systematic Reviews and Research Syntheses An 11-item checklist for evaluating systematic reviews
• AMSTAR Checklist A 16-question measurement tool to assess systematic reviews
• AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews A guide to selecting eligibility criteria, searching the literature, extracting data, assessing quality, and completing other steps in the creation of a systematic review
• CASP - Systematic Review A checklist for quality assessment of systematic review from the Critical Appraisal Skills Programme

Clinical Practice Guidelines

• National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Instrument A 15-item instrument using a scale of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trust worth guidelines
• AGREE-II Appraisal of Guidelines for Research and Evaluation The Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument evaluates the process of practice guideline development and the quality of reporting

Other Study Designs

• NTACT Quality Checklists Quality indicator checklists for correlational studies, group experimental studies, single case research studies, and qualitative studies developed by the National Technical Assistance Center on Transition (NTACT). (Users must make an account.)

Below, you will find a sample of four popular quality assessment tools and some basic information about each. For more quality assessment tools, please view the blue tabs in the boxes above, organized by study design.

Covidence uses Cochrane Risk of Bias (which is designed for rating RCTs and cannot be used for other study types) as the default tool for quality assessment of included studies. You can opt to manually customize the quality assessment template and use a different tool better suited to your review. More information about quality assessment using Covidence, including how to customize the quality assessment template, can be found below. If you decide to customize the quality assessment template, you cannot switch back to using the Cochrane Risk of Bias template.

More Information

• Quality Assessment on the Covidence Guide
• Covidence FAQs on Quality Assessment Commonly asked questions about quality assessment using Covidence
• Covidence YouTube Channel A collection of Covidence-created videos
• << Previous: Step 5: Screen Citations
• Next: Step 7: Extract Data from Included Studies >>
• Last Updated: May 16, 2024 3:24 PM
• URL: https://guides.lib.unc.edu/systematic-reviews

Quality of Literature Reviews

• First Online: 11 August 2022

Cite this chapter

• Rob Dekkers 4 ,
• Lindsey Carey 5 &
• Peter Langhorne 6  

1822 Accesses

Whereas the starting point of a literature review is presented in Chapter 2 —finding out more about what is written about a specific topic by evaluating it from a critical objective—, it leaves open what constitutes a good quality literature review, whether as review of scholarly knowledge before an empirical study or as stand-alone study. Keeping in mind that there are different archetypes of literature reviews, see Section  2.5 , also the way of looking at quality will vary across these types and with the objective of the literature review. Thus, it deserves a closer look at how quality of literature reviews can be assured.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Interestingly, the publication by Webster and Watson ( 2002 ) does not dwell on the implication of the title, even though it captures the essence of a literature review.

Note that this proposition by Yin et al. ( 1976 ) is related to the formalisation of the case survey method in Yin and Heald (1975); the case survey method appears in this book as method associated with qualitative synthesis in Section  10.3 . Also note that the latter publication is a precursor to what is known now as the case study methodology.

The topical survey is also addressed by Elisabeth Bergdahl in Section  11.1 .

There is some discussion about whom introduced or modified the concepts of nomothetic and idiographic forms of generating knowledge in its early stages. On this matter, Hurlburt and Knapp ( 2006 , pp. 287–9), and Salvatore and Valsiner ( 2010 , pp. 818–20) produce slightly different accounts.

Such is found by Steenhuis & de Bruijn ( 2006 ), too, in the case which journals gravitate toward nomothetic or ideographic research.

The common term ‘craftsmanship’ has been replaced with ‘academic mastery’ to avoid any unintended connotations.

Note that Boell and Cecez-Kecmanovic ( 2010 , p. 134 ff.) introduce a search strategy that is reminiscent of the iterative search strategy, presented in Section  5.3 , rather than representative of the hermeneutic approach as detailed in the current section. In their next writing (Boell and Cecez-Kecmanovic 2014 , p. 264), the search strategy is expanded with a cycle of analysis and interpretation, closer to the analysis stages in the systematic quantitative literature review (Section  9.5 ) and content analysis (see Section  10.3 ) than to hermeneutics.

A work in point noting this confusion about the use of the term ‘offshoring’ and related wording is Jahns et al. ( 2006 , pp. 222–3); to support their interpretation, they introduce a matrix to delineate the concepts for offshoring and outsourcing.

Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J (2008) Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC Med Res Methodol 8(1):21. https://doi.org/10.1186/1471-2288-8-21

Article   Google Scholar  

Avellar SA, Thomas J, Kleinman R, Sama-Miller E, Woodruff SE, Coughlin R, Westbrook TPR (2017) External validity: the next step for systematic reviews? Eval Rev 41(4):283–325. https://doi.org/10.1177/0193841x16665199

Babakus WS, Thompson JL (2012) Physical activity among South Asian women: a systematic, mixed-methods review. Int J Behav Nutr Phys Act 9(1):150. https://doi.org/10.1186/1479-5868-9-150

Bailey R, Pearce G, Smith C, Sutherland M, Stack N, Winstanley C, Dickenson M (2012) Improving the educational achievement of gifted and talented students: a systematic review. Talent Dev Excel 4(1):33–48

Google Scholar  

Bearman M (2016) Quality and literature reviews: beyond reporting standards. Med Educ 50(4):382–384. https://doi.org/10.1111/medu.12984

Bengtsson L, Elg U, Lind J-I (1997) Bridging the transatlantic publishing gap: how North American reviewers evaluate European idiographic research. Scand J Manag 13(4):473–492. https://doi.org/10.1016/S0956-5221(97)00022-5

Bergdahl E (2019) Is meta-synthesis turning rich descriptions into thin reductions? A criticism of meta-aggregation as a form of qualitative synthesis. Nurs Inq 26(1):e12273. https://doi.org/10.1111/nin.12273

Berkovich I (2018) Beyond qualitative/quantitative structuralism: the positivist qualitative research and the paradigmatic disclaimer. Qual Quant 52(5):2063–2077. https://doi.org/10.1007/s11135-017-0607-3

Bodolica V, Spraggon M (2018) An end-to-end process of writing and publishing influential literature review articles: do’s and don’ts. Manag Decis 56(11):2472–2486. https://doi.org/10.1108/MD-03-2018-0253

Boell SK, Cecez-Kecmanovic D (2010) Literature reviews and the hermeneutic circle. Aust Acad Res Libr 41(2):129–144. https://doi.org/10.1080/00048623.2010.10721450

Boell SK, Cecez-Kecmanovic D (2014) A hermeneutic approach for conducting literature reviews and literature searches. Commun Assoc Inf Syst 34:257–286. https://doi.org/10.17705/1CAIS.03412

Boell SK, Cecez-Kecmanovic D (2015) On being ‘systematic’ in literature reviews in IS. J Inf Technol 30(2):161–173. https://doi.org/10.1057/jit.2014.26

Bolderston A (2008) Writing an effective literature review. J Med Imaging Radiat Sci 39(2):86–92. https://doi.org/10.1016/j.jmir.2008.04.009

Borras SM, Hall R, Scoones I, White B, Wolford W (2011) Towards a better understanding of global land grabbing: an editorial introduction. J Peasant Stud 38(2):209–216. https://doi.org/10.1080/03066150.2011.559005

Brandstätter M, Baumann U, Borasio GD, Fegg MJ (2012) Systematic review of meaning in life assessment instruments. Psychooncology 21(10):1034–1052. https://doi.org/10.1002/pon.2113

Brereton P, Kitchenham BA, Budgen D, Turner M, Khalil M (2007) Lessons from applying the systematic literature review process within the software engineering domain. J Syst Softw 80(4):571–583. https://doi.org/10.1016/j.jss.2006.07.009

Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan J (2003) Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med 56(4):671–684. https://doi.org/10.1016/S0277-9536(02)00064-3

de Groot AD (1969) Methodology: foundations of inference and research in the behavioral sciences. The Hague, Mouton

de Moura DA, Botter RC (2017) Toyota production system—one example to shipbuilding industry. Indep J Manag Prod 8(3):874–897. https://doi.org/10.14807/ijmp.v8i3.626

Dekkers R (2005) (R)Evolution, organizations and the dynamics of the environment. Springer, New York

Dekkers R (2017) Applied systems theory, 2nd edn. Springer, Cham

Dekkers R, Kühnle H (2012) Appraising interdisciplinary contributions to theory for collaborative (manufacturing) networks: still a long way to go? J Manuf Technol Manag 23(8):1090–1128. https://doi.org/10.1108/17410381211276899

Delllinger AB (2005) Validity and the review of literature. Res Sch 12(2):41–54

Denzin NK, Lincoln YS (1994) Handbook of qualitative research. Sage, Thousands Oaks, CA

Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A (2005) Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 10(1):45–53b. https://doi.org/10.1258/1355819052801804

Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A-W, Cronin E, Williamson PR (2008) Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 3(8):e3081. https://doi.org/10.1371/journal.pone.0003081

Ellaway RH, O’Gorman L, Strasser R, Marsh DC, Graves L, Fink P, Cervin C (2016) A critical hybrid realist-outcomes systematic review of relationships between medical education programmes and communities: BEME Guide No. 35. Med Teach 38(3):229–245. https://doi.org/10.3109/0142159X.2015.1112894

Estabrooks CA, Field PA, Morse JM (1994) Aggregating qualitative findings: an approach to theory development. Qual Health Res 4(4):503–511. https://doi.org/10.1177/104973239400400410

Furunes T (2019) Reflections on systematic reviews: moving golden standards? Scand J Hosp Tour 19(3):227–231. https://doi.org/10.1080/15022250.2019.1584965

Gadamer H-G (1975) The problem of historical consciousness. Grad Fac Philos J 5(1):8–52. https://doi.org/10.5840/gfpj1975512

Galati G, Moessner R (2013) Macroprudential policy—a literature review. J Econ Surv 27(5):846–878. https://doi.org/10.1111/j.1467-6419.2012.00729.x

Gomez-Mejia LR, Cruz C, Berrone P, De Castro J (2011) The bind that ties: socioemotional wealth preservation in family firms. Acad Manag Ann 5(1):653–707. https://doi.org/10.5465/19416520.2011.593320

Gough D, Elbourne D (2002) Systematic research synthesis to inform policy, practice and democratic debate. Soc Policy Soc 1(3):225–236. https://doi.org/10.1017/S147474640200307X

Granello DH (2001) Promoting cognitive complexity in graduate written work: using Bloom’s taxonomy as a pedagogical tool to improve literature reviews. Couns Educ Superv 40(4):292–307. https://doi.org/10.1002/j.1556-6978.2001.tb01261.x

Grondin J (2015) The hermeneutical circle. In: Keane N, Lawn C (eds) The Blackwell companion to hermeneutics. Wiley, Chichester, pp 299–305

Chapter   Google Scholar  

Guba EG, Lincoln YS (1994) Competing paradigms in qualitative research. In: Denzin NK, Lincoln YS (eds) Handbook of qualitative research. Sage, Thousand Oaks, CA, pp 105–117

Hagedoorn J, Duysters G (2002) External sources of innovative capabilities: the preferences for strategic alliances or mergers and acquisitions. J Manag Stud 39(2):167–188

Hurlburt RT, Knapp TJ (2006) Münsterberg in 1898, not Allport in 1937, introduced the terms ‘idiographic’ and ‘nomothetic’ to American psychology. Theory Psychol 16(2):287–293. https://doi.org/10.1177/0959354306062541

Jahns C, Hartmann E, Bals L (2006) Offshoring: dimensions and diffusion of a new business concept. J Purch Supply Manag 12(4):218–231. https://doi.org/10.1016/j.pursup.2006.10.001

Janesick VJ (1994) The dance of qualitative research design: metaphor, methodolatry, and meaning. In: Denzin NK, Lincoln YS (eds) Handbook of qualitative research. Sage, Thousand Oaks, CA, pp 209–219

Jungherr A (2016) Twitter use in election campaigns: a systematic literature review. J Inform Tech Polit 13(1):72–91. https://doi.org/10.1080/19331681.2015.1132401

Karakas F (2010) Spirituality and performance in organizations: a literature review. J Bus Ethics 94(1):89–106. https://doi.org/10.1007/s10551-009-0251-5

Kennedy MM (2007) Defining a literature. Educ Res 36(3):139–147. https://doi.org/10.3102/0013189x07299197

Kvale S (1995) The social construction of validity. Qual Inq 1(1):19–40. https://doi.org/10.1177/107780049500100103

Lawrence M, Kerr S, McVey C, Godwin J (2012) The effectiveness of secondary prevention lifestyle interventions designed to change lifestyle behavior following stroke: summary of a systematic review. Int J Stroke 7(3):243–247. https://doi.org/10.1111/j.1747-4949.2012.00771.x

Lin AC (1998) Bridging positivist and interpretivist approaches to qualitative methods. Policy Stud J 26(1):162–180. https://doi.org/10.1111/j.1541-0072.1998.tb01931.x

Long HA, French DP, Brooks JM (2020) Optimising the value of the critical appraisal skills programme (CASP) tool for quality appraisal in qualitative evidence synthesis. Res Methods Med Health Sci 1(1):31–42. https://doi.org/10.1177/2632084320947559

Maier HR (2013) What constitutes a good literature review and why does its quality matter? Environ Model Softw 43:3–4. https://doi.org/10.1016/j.envsoft.2013.02.004

Mays N, Pope C (2000) Assessing quality in qualitative research. BMJ 320(7226):50–52. https://doi.org/10.1136/bmj.320.7226.50

McKercher B, Law R, Weber K, Song H, Hsu C (2007) Why referees reject manuscripts. J Hosp Tour Res 31(4):455–470. https://doi.org/10.1177/1096348007302355

Mohammed MA, Moles RJ, Chen TF (2016) Meta-synthesis of qualitative research: the challenges and opportunities. Int J Clin Pharm 38(3):695–704. https://doi.org/10.1007/s11096-016-0289-2

Montuori A (2005) Literature review as creative inquiry: reframing scholarship as a creative process. J Transform Educ 3(4):374–393. https://doi.org/10.1177/1541344605279381

Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati A (2005) Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ 330(7499):1053. https://doi.org/10.1136/bmj.38414.515938.8F

Münsterberg H (1899) Psychology and history. Psychol Rev VI(I):1–31. https://doi.org/10.1037/h0071306

Munthe-Kaas H, Nøkleby H, Lewin S, Glenton C (2020) The TRANSFER approach for assessing the transferability of systematic review findings. BMC Med Res Methodol 20(1):11. https://doi.org/10.1186/s12874-019-0834-5

Munthe-Kaas H, Nøkleby H, Nguyen L (2019) Systematic mapping of checklists for assessing transferability. Syst Rev 8(1):22. https://doi.org/10.1186/s13643-018-0893-4

Nakano D, Muniz Jr, J (2018) Writing the literature review for empirical papers. Production 28:e20170086. https://doi.org/10.1590/0103-6513.20170086

Oakes G (1980) History and Natural Science. Hist Theory 19(2):165–168. https://doi.org/10.2307/2504797

Oxman AD, Guyatt GH (1988) Guidelines for reading literature reviews. Can Med Assoc J 138(8):697–703

Petty R, Guthrie J (2000) Intellectual capital literature review: measurement, reporting and management. J Intellect Cap 1(2):155–176. https://doi.org/10.1108/14691930010348731

Pluye P, Gagnon M-P, Griffiths F, Johnson-Lafleur J (2009) A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud 46(4):529–546. https://doi.org/10.1016/j.ijnurstu.2009.01.009

Popper K (1999) All life is problem solving. Routledge, London

Ribeiro ÍJS, Pereira R, Freire IV, de Oliveira BG, Casotti CA, Boery EN (2018) Stress and quality of life among university students: a systematic literature review. Health Prof Educ 4(2):70–77. https://doi.org/10.1016/j.hpe.2017.03.002

Robinson OC (2011) The idiographic/nomothetic dichotomy: tracing historical origins of contemporary confusions. Hist Philos Psychol 13(2):32–39

Rowley J, Slack F (2004) Conducting a literature review. Manag Res News 27(6):31–39. https://doi.org/10.1108/01409170410784185

Salvatore S, Valsiner J (2010) Between the general and the unique: overcoming the nomothetic versus idiographic opposition. Theory Psychol 20(6):817–833. https://doi.org/10.1177/0959354310381156

Schwandt TA (1994) Constructivist, interpretivist approaches to human inquiry. In: Denzin NK, Lincoln YS (eds) Handbook of qualitative research. Sage, Thousand Oaks, CA, pp 118–137

Shephard K, Rieckmann M, Barth M (2019) Seeking sustainability competence and capability in the ESD and HESD literature: an international philosophical hermeneutic analysis. Environ Educ Res 25(4):532–547. https://doi.org/10.1080/13504622.2018.1490947

Smythe E, Spence D (2012) Re-viewing literature in hermeneutic research. Int J Qual Methods 11(1):12–25. https://doi.org/10.1177/160940691201100102

Snyder H (2019) Literature review as a research methodology: an overview and guidelines. J Bus Res 104:333–339. https://doi.org/10.1016/j.jbusres.2019.07.039

Spencer L, Ritchie J, Lewis J, Dillon L (2003) Quality in qualitative evaluation: a framework for assessing research evidence. Cabinet Office, London

Steenhuis HJ, de Bruijn EJ (2006) Publishing in OM: does scientific paradigm matter? In Annual Meeting of the Academy of Management, Atlanta, CA, 11–16 August 2016

Strang KD (2015) Articulating a research design ideology. In: Strang KD (ed) The Palgrave handbook of research design in business and management. Palgrave Macmillan, New York, NY, pp 17–30

Tilden T (2020) The idiographic voice in a nomothetic world: why client feedback is essential in our professional knowledge. In: Ochs M, Borcsa M, Schweitzer J (eds) Systemic research in individual, couple, and family therapy and counseling. Springer International Publishing, Cham, pp 385–399

van Laar E, van Deursen AJAM, van Dijk JAGM, de Haan J (2017) The relation between 21st-century skills and digital skills: a systematic literature review. Comput Hum Behav 72:577–588. https://doi.org/10.1016/j.chb.2017.03.010

Wagenmakers E-J, Dutilh G, Sarafoglou A (2018) The creativity-verification cycle in psychological science: new methods to combat old idols. Perspect Psychol Sci 13(4):418–427. https://doi.org/10.1177/1745691618771357

Walsh D, Downe S (2006) Appraising the quality of qualitative research. Midwifery 22(2):108–119. https://doi.org/10.1016/j.midw.2005.05.004

Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q 26(2):xiii–xxiii

Yin RK, Bingham E, Heald KA (1976) The difference that quality makes: the case of literature reviews. Sociol Methods Res 5(2):139–156. https://doi.org/10.1177/004912417600500201

Yin RK, Heald KA (1975) Using the case survey method to analyze policy studies. Adm Sci Q 20(3):371–381. https://doi.org/10.2307/2391997

Zhou J, Li X, Mitri HS (2018) Evaluation method of rockburst: state-of-the-art literature review. Tunn Undergr Space Technol 81:632–659. https://doi.org/10.1016/j.tust.2018.08.029

Download references

Author information

Authors and affiliations.

University of Glasgow, Glasgow, UK

Rob Dekkers

Glasgow Caledonian University, Glasgow, UK

Lindsey Carey

Prof. Peter Langhorne

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Rob Dekkers .

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Dekkers, R., Carey, L., Langhorne, P. (2022). Quality of Literature Reviews. In: Making Literature Reviews Work: A Multidisciplinary Guide to Systematic Approaches. Springer, Cham. https://doi.org/10.1007/978-3-030-90025-0_3

Download citation

DOI : https://doi.org/10.1007/978-3-030-90025-0_3

Published : 11 August 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-90024-3

Online ISBN : 978-3-030-90025-0

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• JABSOM Library

Systematic Review Toolbox

Quality assessment.

• Guidelines & Rubrics
• Databases & Indexes
• Reference Management
• Data Extraction
• Data Analysis
• Manuscript Development
• Software Comparison
• Systematic Searching This link opens in a new window
• Authorship Determination This link opens in a new window
• Critical Appraisal Tools This link opens in a new window

Critical Appraisal Questions

• Is the study question relevant?
• Does the study add anything new?
• What type of research question is being asked?
• Was the study design appropriate for the research question?
• Did the study methods address the most important potential sources of bias?
• Was the study performed according to the original protocol?
• Does the study test a stated hypothesis?
• Were the statistical analyses performed correctly?
• Do the data justify the conclusions?
• Are there any conflicts of interest?

The University of Sydney Library, Systematic Reviews: Assessment Tools and Critical Appraisal

Taylor, P., Hussain, J. A., & Gadoud, A. (2013). How to appraise a systematic review. British Journal of Hospital Medicine, 74(6), 331-334. doi: 10.12968/hmed.2013.74.6.331

Young, J. M., & Solomon, M. J. (2009). How to critically appraise an article. Nature Clinical Practice Gastroenterology and Hepatology, 6(2), 82-91. doi: 10.1038/ncpgasthep1331

Assessing the quality of evidence contained within a systematic review is as important as analyzing the data within. Results from a poorly conducted study can be skewed by biases from the research methodology and should be interpreted with caution. Such studies should be acknowledged as such in the systematic review or outright excluded. Selecting an appropriate tool to help analyze strength of evidence and imbedded biases within each paper is also essential. If using a systematic review manuscript development tool (e.g., RevMan), a checklist may be built into the software. Other software (e.g., Rayyan) may help with screening search results and discarding irrelevant studies. The following tools/checklists may help with study assessment and critical appraisal.

• Assessing the Methodological Quality of Systematic Reviews (AMSTAR 2) is widely used to critically appraise systematic reviews .
• Centre for Evidence-Based Medicine (CEBM) contains a collection of critical appraisal tools for studies of all types and examples of usage.
• Cochrane risk-of-bias (RoB 2) tool is the recommended tool for assessing quality and risk of bias in randomized clinical trials in Cochrane-submitted systematic reviews.
• Critical Appraisal Skills Programme (CASP) has 25 years of experience and expertise in critical appraisal and offers appraisal checklists for a wide range of study types .
• Joanna Briggs Institute (JBI) provides robust checklists for the appraisal and assessment of most types of studies .
• National Academies of Sciences, Health and Medicine Division provides standards for assessing bias in primary studies comprising systematic reviews of therapeutic or medical interventions.
• Newcastle-Ottawa Scale (NOS) is also used in non-observational studies of cohort and case-control varieties.
• Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool surveys diagnostic accuracy studies on four domains: index test, reference standard, patient selection, and flow and timing.
• Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) framework is often used to measure the quality of cohort, case-control and cross-sectional studies .

Requesting Research Consultation

The Health Sciences Library provides consultation services for University of Hawaiʻi-affiliated students, staff, and faculty. The John A. Burns School of Medicine Health Sciences Library does not have staffing to conduct or assist researchers unaffiliated with the University of Hawaiʻi. Please utilize the publicly available guides and support pages that address research databases and tools.

Before Requesting Assistance

Before requesting systematic review assistance from the librarians, please review the relevant guides and the various pages of the Systematic Review Toolbox . Most inquiries received have been answered there previously. Support for research software issues is limited to help with basic installation and setup. Please contact the software developer directly if further assistance is needed.

• << Previous: Data Extraction
• Next: Data Analysis >>
• Last Updated: Sep 20, 2023 9:14 AM
• URL: https://hslib.jabsom.hawaii.edu/systematicreview

Health Sciences Library, John A. Burns School of Medicine, University of Hawai‘i at Mānoa, 651 Ilalo Street, MEB 101, Honolulu, HI 96813 - Phone: 808-692-0810, Fax: 808-692-1244

Copyright © 2004-2024. All rights reserved. Library Staff Page - Other UH Libraries

Systematic Review: Quality assessment

• Which tool?
• Formulate the review question
• Search the Literature
• Medical Study Types
• Quality assessment
• Data extraction
• Analyse and synthesise
• Further reading
• Research Support This link opens in a new window
• EndNote for Systematic Reviews

Critical appraisal

What is critical appraisal?

Evaluation of literature is determining the value or worth of that piece of information, critical appraisal is a more advanced form of evaluation which asks the reader to consider the specifics of the literature in a structured way. Critically appraising a piece of research combines analysis of the design of the study, the validity of the findings in relation to the design of the study, the likelihood of bias, and the relevance of the overall results to other current research.

Specialist tools

Critical appraisal should be done systematically and objectively. To aid with this, there are a number of systems used for critical appraisal, many of which focus on using specific check lists. As there are a variety of available tools you should be able to choose the most appropriate tool for your research.

Scottish Intercollegiate Guidelines Network (SIGN) have detailed guidance on their website for the tools they use when undertaking systematic reviews for guideline development. This includes a flow chart to help you define the study type you are appraising and all relevant checklists and supporting guidance .

The Critical Appraisal Skills Programme provides a set of eight checklists which cover some study types which the SIGN checklists do not, such as Cohort Studies.

Center for Evidence-Based Management (CEBMa) :

This group provides a small selection of online checklists, but also has a mobile phone app (Android and IOS) which can help with critical appraisal on various published articles.

CEBM : The Centre for Evidence-Based Medicine at Oxford University has created critical appraisal guidance sheets for 4 main study types, which are available in 4 languages: English, German, Spanish and Lithuanian.

The checklists from STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) aim to provide support in critical appraisal for observational studies rather than clinical trials, and they have 5 checklists available, including one to appraise conference papers (STROBE, 2017).

Newcastle-Ottawa Scale :

Aimed at non-randomised study types, with a focus on case control studies, and designed to be easy to use while also providing a clear score for each paper.

PRISMA focus on the critical appraisal of systematic reviews and meta-analyses and have a standard checklist available covering both study types.

The University of Bristol have developed the QUADAS system as a tool to appraise the quality of diagnostic studies.

The GRADE system has been developed to combat the failings often seen in the check lists systems which are often limited to a single study type and encourages a thorough analysis of the paper as a whole via the creation of evidence tables and cross referencing (Grading of Recommendations Assessment Development and Evaluation (GRADE), 2017).

General tools

All the tools above are specific to designated study types, and at times there may be a requirement to critically appraise other forms of information. The following resources do not provide a numerical grade for the quality of the evidence. They do, however, give guidance and advice on critical appraisal and detailed evaluation which are applicable to a variety of resources.

Books and Articles:

Trisha Greenhalgh has published an excellent book on critical appraisal called ' How to read a paper ' which is available in the Library.

Additionally, the article from Nature Clinical Practice Gastroenterology & Hepatology by Jane Young and Michael Solomon covers the theory of critical appraisal as well as discussing ten valuable questions to ask when evaluating literature.

Students 4 Best Evidence :

This group have produced a general critical appraisal checklist which covers 20 questions to consider as you read through the different sections of a published journal article.

CARS Checklist :

Developed by academic publishers McGraw-Hill in 2001 the CARS checklist is designed to be applied to any type of information, not just scholarly articles. It covers the 4 main elements of evaluation while providing examples of questions to consider when appraising.

Understanding Health Research :

Created by the Social and Public Health Sciences Unit at the University of Glasgow, as well as having a step-by-step tool to walk you through critical appraisal of an article, this site also contains a wealth of information on methods used in health research and understanding bias.

Grading the literature

Some forms of critical appraisal of the literature result in each individual piece of literature receiving a score to rate its quality - this is often referred to as grading: the quality of literature is assessed and given a score. Evidence tables are used to list study characteristics and to help compare the literature. 

For example, SIGN checklists and the GRADE (Grading of Recommendations Assessment, Performance and Evaluation) system formalise the appraisal of literature by assigning an overall quality rating.

• << Previous: Medical Study Types
• Next: Data extraction >>
• Last Updated: May 21, 2024 3:20 PM
• URL: https://guides.lib.strath.ac.uk/systematic

Literature Review - what is a Literature Review, why it is important and how it is done

• Strategies to Find Sources

Evaluating Literature Reviews and Sources

Reading critically, tips to evaluate sources.

• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings
• Useful Resources

A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

• A Closer Look at Evaluating Literature Reviews Excerpt from the book chapter, “Evaluating Introductions and Literature Reviews” in Fred Pyrczak’s Evaluating Research in Academic Journals: A Practical Guide to Realistic Evaluation , (Chapter 4 and 5). This PDF discusses and offers great advice on how to evaluate "Introductions" and "Literature Reviews" by listing questions and tips. First part focus on Introductions and in page 10 in the PDF, 37 in the text, it focus on "literature reviews".
• Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.

To be able to write a good Literature Review, you need to be able to read critically. Below are some tips that will help you evaluate the sources for your paper.

Reading critically (summary from How to Read Academic Texts Critically)

• Who is the author? What is his/her standing in the field.
• What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, to critique or clarify?
• Note the experts in the field: are there specific names/labs that are frequently cited?
• Pay attention to methodology: is it sound? what testing procedures, subjects, materials were used?
• Note conflicting theories, methodologies and results. Are there any assumptions being made by most/some researchers?
• Theories: have they evolved overtime?
• Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?

Useful links:

• How to Read a Paper (University of Waterloo, Canada) This is an excellent paper that teach you how to read an academic paper, how to determine if it is something to set aside, or something to read deeply. Good advice to organize your literature for the Literature Review or just reading for classes.

Criteria to evaluate sources:

• Authority : Who is the author? what is his/her credentials--what university he/she is affliliated? Is his/her area of expertise?
• Usefulness : How this source related to your topic? How current or relevant it is to your topic?
• Reliability : Does the information comes from a reliable, trusted source such as an academic journal?

Useful site - Critically Analyzing Information Sources (Cornell University Library)

• << Previous: Strategies to Find Sources
• Next: Tips for Writing Literature Reviews >>
• Last Updated: Apr 10, 2024 3:27 PM
• URL: https://lit.libguides.com/Literature-Review

The Library, Technological University of the Shannon: Midwest

Literature Review: Assess your Literature Review

• Video Tutorial
• Research Question
• Select Resources
• Search Strategy

Assess your Literature Review

• Sample Literature Reviews
• Resource List
• Quick Links
• Avoid Plagiarism
• Use the rubric below to evaluate the quality of your literature review.  If your instructor has provided you with a rubric, you should use the criteria listed in that course or assignment rubric to ensure that your paper will meet the expectations for the course. ( Download a copy of the rubric.)

Adapted from Education 690: Assessment Rubric/Criteria for Literature Review, retrieved September 29,2010 from http://edweb.sdsu.edu/courses/ed690dr/grading/literaturereviewrubrique.html and Boote, D.N. & Biele, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher. 34(6) p. 8.

• << Previous: PLAGIARISM
• Next: Sample Literature Reviews >>
• Last Updated: Apr 24, 2024 3:03 PM
• URL: https://research.auctr.edu/literaturereview

About Systematic Reviews

Quality Assessment Tools for Systematic Reviews

Automate every stage of your literature review to produce evidence-based research faster and more accurately.

Systematic reviews, with or without meta-analysis are considered the highest level of evidence in medical literature [1]. There are different types of systematic reviews designed to help answer various research questions. For example, to answer broad questions, and identify key concepts in a research area, systematic scoping reviews are employed. For guidance on conducting systematic scoping reviews , you can learn more at the link provided. Since the intended purpose of a scoping systematic review is to gather information as opposed to recommending a clinical practice, quality assessment is either not undertaken or is not as stringent as in the case of a typical systematic review.

As systematic reviews are considered the highest form of evidence, conducting a thorough quality assessment of the study is required. In this article, we will look at the quality assessment tools which can be used to assess the internal validity of a systematic review.

What Is Quality Assessment and Why Is It Important?

Quality assessment is also known as quality appraisal, critical appraisal, and risk of bias assessment, with the terms sometimes being used interchangeably. They refer to the assessment of the methodological quality, and rigor of the trials or studies included in a systematic review. Although systematic reviews are designed in a way to produce robust, reliable, and reproducible results, they are still open to biases and errors. Errors in the study design and implementation have the potential to bias the results in favor of one intervention over the others. Given the importance of systematic reviews in evidence-based medicine, we need to be aware of these biases. Quality assessment, therefore, helps in minimizing the risk of bias and increases confidence in review findings.

Learn More About DistillerSR

(Article continues below)

Quality Assessment Tools For Systematic Reviews

There are various tools offered to assist quality assessment and critical appraisal of a systematic review. Some of them are listed below,

AMSTAR (Assessing the methodological quality of systematic reviews)

AMSTAR is a popular instrument for critically appraising systematic reviews of randomized controlled clinical trials. It was further developed to enable appraisal of systematic reviews of randomized, and non-randomized studies of healthcare interventions. The revised instrument AMSTAR 2 has 16 items, simpler response categories than the original AMSTAR, includes a more comprehensive user guide, and has an overall rating based on weaknesses in critical domains.

Cochrane Risk-Of-Bias (RoB 2) Tool

This is the recommended tool for assessing the quality and risk of bias within the randomized clinical trials included in the systematic review. Review authors assess risk of bias in six domains of potential bias as being either high, low, or unclear. The six domains of potential bias include selection bias, performance bias, detection bias, attrition bias, reporting bias, and other bias.

Joanna Briggs Institute (JBI) Checklist

The JBI Critical Appraisal Checklist for Systematic Reviews and Research Syntheses is an 11-item checklist for evaluating systematic reviews.

AHRQ Methods Guide For Effectiveness and Comparative Effectiveness Reviews

Provides detailed information on selecting eligibility criteria, searching the literature, data extraction, quality assessment, and other steps involved in the creation of a systematic review.

CASP-Systematic Review Checklist

The critical appraisal skills program has over 25 years of experience in developing detailed checklists for the quality assessment of different types of studies including systematic reviews.

Newcastle-Ottawa Scale (NOS)

This tool facilitates the appraisal of non-randomized studies included in the systematic review. Non Randomized studies, including case-control and cohort studies, can be challenging to implement and conduct. Assessment of the quality of such studies is essential for a proper understanding of non-randomized studies. The Newcastle-Ottawa Scale (NOS) is an ongoing collaboration between the Universities of Newcastle, Australia, and Ottawa, Canada. It was developed to assess the quality of nonrandomized studies with its design, content, and ease of use directed to the task of incorporating the quality assessments in the interpretation of meta-analytic results.

Other Tools And Resources

Along with the standardized checklists available for critically appraising the quality of the systematic reviews, researchers can also use systematic review management software tools that are specifically tailored to the review team’s needs. One such tool is DistillerSR. The software automates every stage of the process involved in a systematic review, reduces error or duplication, and by incorporating one or more of the aforementioned quality assessment checklists, it can assist in the critical appraisal of the review.

Assessing the quality of evidence contained in a systematic review is just as important as synthesizing the findings of the review. During quality assessment, one considers the relevance of the methods utilized in addressing review questions, the relevance and quality of methods used within individual studies, and the extent of evidence from reported findings [2]. Due to the importance of systematic reviews in evidence-based practice, it is crucial to conduct a stringent and thorough quality assessment of the review. There are a number of tools available to critically appraise a systematic review. By gaining an understanding of the tools and their implementation, researchers can publish robust, high-quality systematic reviews.

• Guyatt, G. H.; Sackett, D. L.; Sinclair, J. C.; Hayward, R.; Cook, D. J.; Cook, R. J. “Users’ guides to the medical literature IX. A method for grading health care recommendations.” JAMA, 274 (22) (1995): 1800-1804.
• Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L.M., Hróbjartsson, A., Altman, D.G. and Moher, D. (2017). Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study. Systematic Reviews, 6(1).

3 Reasons to Connect

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Indian J Dermatol
• v.59(2); Mar-Apr 2014

Understanding and Evaluating Systematic Reviews and Meta-analyses

Michael bigby.

From the Department of Dermatology, Harvard Medical School, Beth Israel Deaconess Medical Center, Boston, MA 02215, USA

A systematic review is a summary of existing evidence that answers a specific clinical question, contains a thorough, unbiased search of the relevant literature, explicit criteria for assessing studies and structured presentation of the results. A systematic review that incorporates quantitative pooling of similar studies to produce an overall summary of treatment effects is a meta-analysis. A systematic review should have clear, focused clinical objectives containing four elements expressed through the acronym PICO (Patient, group of patients, or problem, an Intervention, a Comparison intervention and specific Outcomes). Explicit and thorough search of the literature is a pre-requisite of any good systematic review. Reviews should have pre-defined explicit criteria for what studies would be included and the analysis should include only those studies that fit the inclusion criteria. The quality (risk of bias) of the primary studies should be critically appraised. Particularly the role of publication and language bias should be acknowledged and addressed by the review, whenever possible. Structured reporting of the results with quantitative pooling of the data must be attempted, whenever appropriate. The review should include interpretation of the data, including implications for clinical practice and further research. Overall, the current quality of reporting of systematic reviews remains highly variable.

Introduction

A systematic review is a summary of existing evidence that answers a specific clinical question, contains a thorough, unbiased search of the relevant literature, explicit criteria for assessing studies and structured presentation of the results. A systematic review can be distinguished from a narrative review because it will have explicitly stated objectives (the focused clinical question), materials (the relevant medical literature) and methods (the way in which studies are assessed and summarized).[ 1 , 2 ] A systematic review that incorporates quantitative pooling of similar studies to produce an overall summary of treatment effects is a meta-analysis.[ 1 , 2 ] Meta-analysis may allow recognition of important treatment effects by combining the results of small trials that individually might lack the power to consistently demonstrate differences among treatments.[ 1 ]

With over 200 speciality dermatology journals being published, the amount of data published just in the dermatologic literature exceeds our ability to read it.[ 3 ] Therefore, keeping up with the literature by reading journals is an impossible task. Systematic reviews provide a solution to handle information overload for practicing physicians.

Criteria for reporting systematic reviews have been developed by a consensus panel first published as Quality of Reporting of Meta-analyses (QUOROM) and later refined as Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA).[ 4 , 5 ] This detailed, 27-item checklist contains items that should be included and reported in high quality systematic reviews and meta-analyses. The methods for understanding and appraising systematic reviews and meta-analyses presented in this paper are a subset of the PRISMA criteria.

The items that are the essential features of a systematic review include having clear objectives, explicit criteria for study selection, an assessment of the quality of included studies, criteria for which studies can be combined, appropriate analysis and presentation of results and practical conclusions that are based on the evidence evaluated [ Table 1 ]. Meta-analysis is only appropriate if the included studies are conceptually similar. Meta-analyses should only be conducted after a systematic review.[ 1 , 6 ]

Criteria for evaluating a systematic review or the meta-analysis

A Systematic Review Should Have Clear, Focused Clinical Objectives

A focused clinical question for a systematic review should contain the same four elements used to formulate well-built clinical questions for individual studies, namely a Patient, group of patients, or problem, an Intervention, a Comparison intervention and specific Outcomes.[ 7 ] These features can be remembered by the acronym PICO. The interventions and comparison interventions should be adequately described so that what was done can be reproduced in future studies and in practice. For diseases with established effective treatments, comparisons of new treatments or regimens to established treatments provide the most useful information. The outcomes reported should be those that are most relevant to physicians and patients.[ 1 ]

Explicit and Thorough Search of the Literature

A key question to ask of a systematic review is: “Is it unlikely that important, relevant studies were missed?” A sound systematic review can be performed only if most or all of the available data are examined. An explicit and thorough search of the literature should be performed. It should include searching several electronic bibliographic databases including the Cochrane Controlled Trials Registry, which is part of the Cochrane Library, Medline, Embase and Literatura Latino Americana em Ciências da Saúde. Bibliographies of retrieved studies, review articles and textbooks should be examined for studies fitting inclusion criteria. There should be no language restrictions. Additional sources of data include scrutiny of citation lists in retrieved articles, hand-searching for conference reports, prospective trial registers (e.g., clinical trials.gov for the USA and clinical trialsregister.eu for the European union) and contacting key researchers, authors and drug companies.[ 1 , 8 ]

Reviews should have Pre-defined Explicit Criteria for what Studies would be Included and the Analysis should Include Only those Studies that Fit the Inclusion Criteria

The overwhelming majority of systematic reviews involve therapy. Randomized, controlled clinical trials should therefore be used for systematic reviews of therapy if they are available, because they are generally less susceptible to selection and information bias in comparison with other study designs.[ 1 , 9 ]

Systematic reviews of diagnostic studies and harmful effects of interventions are increasingly being performed and published. Ideally, diagnostic studies included in systematic reviews should be cohort studies of representative populations. The studies should include a criterion (gold) standard test used to establish a diagnosis that is applied uniformly and blinded to the results of the test(s) being studied.[ 1 , 9 ]

Randomized controlled trials can be included in systematic reviews of studies of adverse effects of interventions if the events are common. For rare adverse effects, case-control studies, post-marketing surveillance studies and case reports are more appropriate.[ 1 , 9 ]

The Quality (Risk of Bias) of the Primary Studies should be Critically Appraised

The risk of bias of included therapeutic trials is assessed using the criteria that are used to evaluate individual randomized controlled clinical trials. The quality criteria commonly used include concealed, random allocation; groups similar in terms of known prognostic factors; equal treatment of groups; blinding of patients, researchers and analyzers of the data to treatment allocation and accounting for all patients entered into the trial when analyzing the results (intention-to-treat design).[ 1 ] Absence of these items has been demonstrated to increase the risk of bias of systematic reviews and to exaggerate the treatment effects in individual studies.[ 10 ]

Structured Reporting of the Results with Quantitative Pooling of the Data, if Appropriate

Systematic reviews that contain studies that have results that are similar in magnitude and direction provide results that are most likely to be true and useful. It may be impossible to draw firm conclusions from systematic reviews in which studies have results of widely different magnitude and direction.[ 1 , 9 ]

Meta-analysis should only be performed to synthesize results from different trials if the trials have conceptual homogeneity.[ 1 , 6 , 9 ] The trials must involve similar patient populations, have used similar treatments and have measured results in a similar fashion at a similar point in time.

Once conceptual homogeneity is established and the decision to combine results is made, there are two main statistical methods by which results are combined: random-effects models (e.g., DerSimonian and Laird) and fixed-effects models (e.g., Peto or Mantel-Haenszel).[ 11 ] Random-effects models assume that the results of the different studies may come from different populations with varying responses to treatment. Fixed-effects models assume that each trial represents a random sample of a single population with a single response to treatment [ Figure 1 ]. In general, random-effects models are more conservative (i.e., random-effects models are less likely to show statistically significant results than fixed-effects models). When the combined studies have statistical homogeneity (i.e., when the studies are reasonably similar in direction, magnitude and variability), random-effects and fixed-effects models give similar results.

Fixed-effects models (a) assume that each trial represents a random sample (colored curves) of a single population with a single response to treatment. Random-effects models (b) assume that the different trials’ results (colored curves) may come from different populations with varying responses to treatment.

The point estimates and confidence intervals of the individual trials and the synthesis of all trials in meta-analysis are typically displayed graphically in a forest plot [ Figure 2 ].[ 12 ] Results are most commonly expressed as the odds ratio (OR) of the treatment effect (i.e., the odds of achieving a good outcome in the treated group divided by the odds of achieving a good result in the control group) but can be expressed as risk differences (i.e., difference in response rate) or relative risk (probability of achieving a good outcome in the treated group divided by the probability in the control group). An OR of 1 (null) indicates no difference between treatment and control and is usually represented by a vertical line passing through 1 on the x-axis. An OR of greater or less than 1 implies that the treatment is superior or inferior to the control respectively.

Annotated results of a meta-analysis of six studies, using random effects models reported as odd ratios using MIX version 1.7 (Bax L, Yu LM, Ikeda N, Tsuruta H, Moons KGM. Development and validation of MIX: comprehensive free software for meta-analysis of causal research data. BMC Med Res Methodol http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1626481/ ). The central graph is a typical Forest Plot

The point estimate of individual trials is indicated by a square whose size is proportional to the size of the trial (i.e., number of patients analyzed). The precision of the trial is represented by the 95% confidence interval that appears in Forest Plots as the brackets surrounding point estimate. If the 95% confidence interval (brackets) does not cross null (OR of 1), then the individual trial is statistically significant at the P = 0.05 level.[ 12 ] The summary value for all trials is shown graphically as a parallelogram whose size is proportional to the total number of patients analyzed from all trials. The lateral tips of the parallelogram represent the 95% confidence interval and if they do not cross null (OR of 1), then the summary value of the meta-analysis is statistically significant at the P = 0.05 level. ORs can be converted to risk differences and numbers needed to treat (NNTs) if the event rate in the control group is known [ Table 2 ].[ 13 , 14 ]

Deriving numbers needed to treat from a treatment's odds ratio and the observed or expected event rates of untreated groups or individuals

The difference in response rate and its reciprocal, the NNT, are the most easily understood measures of the magnitude of the treatment effect.[ 1 , 9 ] The NNT represents the number of patients one would need to treat in order to achieve one additional cure. Whereas the interpretation of NNT might be straightforward within one trial, interpretation of NNT requires some caution within a systematic review, as this statistic is highly sensitive to baseline event rates.[ 1 ]

For example, if a treatment A is 30% more effective than treatment B for clearing psoriasis and 50% of people on treatment B are cleared with therapy, then 65% will clear with treatment A. These results correspond to a rate difference of 15% (65-50) and an NNT of 7 (1/0.15). This difference sounds quite worthwhile clinically. However if the baseline clearance rate for treatment B in another trial or setting is only 30%, the rate difference will be only 9% and the NNT now becomes 11 and if the baseline clearance rate is 10%, then the NNT for treatment A will be 33, which is perhaps less worthwhile.[ 1 ]

Therefore, NNT summary measures within a systematic review should be interpreted with caution because “control” or baseline event rates usually differ considerably between studies.[ 1 , 15 ] Instead, a range of NNTs for a range of plausible control event rates that occur in different clinical settings should be given, along with their 95% confidence intervals.[ 1 , 16 ]

The data used in a meta-analysis can be tested for statistical heterogeneity. Methods to tests for statistical heterogeneity include the χ 2 and I.[ 2 , 11 , 17 ] Tests for statistical heterogeneity are typically of low power and hence detecting statistical homogeneity does not mean clinical homogeneity. When there is evidence of heterogeneity, reasons for heterogeneity between studies – such as different disease subgroups, intervention dosage, or study quality – should be sought.[ 11 , 17 ] Detecting the source of heterogeneity generally requires sub-group analysis, which is only possible when data from many or large trials are available.[ 1 , 9 ]

In some systematic reviews in which a large number of trials have been performed, it is possible to evaluate whether certain subgroups (e.g. children versus adults) are more likely to benefit than others. Subgroup analysis is rarely possible in dermatology, because few trials are available. Subgroup analyses should always be pre-specified in a systematic review protocol in order to avoid spurious post hoc claims.[ 1 , 9 ]

The Importance of Publication Bias

Publication bias is the tendency that studies that show positive effects are more likely to be published and are easier to find.[ 1 , 18 ] It results from allowing factors other than the quality of the study to influence its acceptability for publication. Factors such as the sample size, the direction and statistical significance of findings, or the investigators’ perception of whether the findings are “interesting,” are related to the likelihood of publication.[ 1 , 19 , 20 ] Negative studies with small sample size are less likely to be published.[ 1 , 19 , 20 ] Studies published are often dominated by the pharmaceutical company sponsored trials of new, expensive treatments often compared with the placebo.

For many diseases, the studies published are dominated by drug company-sponsored trials of new, expensive treatments. Such studies are almost always “positive.”[ 1 , 21 , 22 ] This bias in publication can result in data-driven systematic reviews that draw more attention to those medicines. Systematic reviews that have been sponsored directly or indirectly by industry are also prone to bias through over-inclusion of unpublished “positive” studies that are kept “on file” by that company and by not including or not finishing registered trials whose results are negative.[ 1 , 23 ] The creation of study registers (e.g. http://clinicaltrials.gov ) and advance publication of research designs have been proposed as ways to prevent publication bias.[ 1 , 24 , 25 ] Many dermatology journals now require all their published trials to have been registered beforehand, but this policy is not well policed.[ 1 ]

Language bias is the tendency for studies that are “positive” to be published in an English-language journal and be more quickly found than inconclusive or negative studies.[ 1 , 26 ] A thorough systematic review should therefore not restrict itself to journals published in English.[ 1 ]

Publication bias can be detected by using a simple graphic test (funnel plot), by calculating the fail-safe N, Begg's rank correlation method, Egger regression method and others.[ 1 , 9 , 11 , 27 , 28 ] These techniques are of limited value when less than 10 randomized controlled trials are included. Testing for publication bias is often not possible in systematic reviews of skin diseases, due to the limited number and sizes of trials.[ 1 , 9 ]

Question-driven systematic reviews answer the clinical questions of most concern to practitioners. In many cases, studies that are of most relevance to doctors and patients have not been done in the field of dermatology, due to inadequate sources of independent funding.[ 1 , 9 ]

The Quality of Reporting of Systematic Reviews

The quality of reporting of systematic reviews is highly variable.[ 1 ] One cross-sectional study of 300 systematic reviews published in Medline showed that over 90% were reported in specialty journals. Funding sources were not reported in 40% of reviews. Only two-thirds reported the range of years that the literature was searched for trials. Around a third of reviews failed to provide a quality assessment of the included studies and only half of the reviews included the term “systematic review” or “meta-analysis” in the title.[ 1 , 29 ]

The Review should Include Interpretation of the Data, Including Implications for Clinical Practice and Further Research

The conclusions in the discussion section of a systematic review should closely reflect the data that have been presented within that review. Clinical recommendations can be made when conclusive evidence is found, analyzed and presented. The authors should make it clear which of the treatment recommendations are based on the review data and which reflect their own judgments.[ 1 , 9 ]

Many reviews in dermatology, however, find little evidence to address the questions posed. The review may still be of value even if it lacks conclusive evidence, especially if the question addressed is an important one.[ 1 , 30 ] For example, the systematic review may provide the authors with the opportunity to call for primary research in an area and to make recommendations on study design and outcomes that might help future researchers.[ 1 , 31 ]

Source of Support: Nil

Conflict of Interest: Nil.

• Open access
• Published: 14 May 2024

Developing a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in Medical Assistance in Dying (MAiD): a mixed method modified e-Delphi study

• Jocelyn Schroeder 1 ,
• Barbara Pesut 1 , 2 ,
• Lise Olsen 2 ,
• Nelly D. Oelke 2 &
• Helen Sharp 2  

BMC Nursing volume  23 , Article number:  326 ( 2024 ) Cite this article

181 Accesses

Metrics details

Medical Assistance in Dying (MAiD) was legalized in Canada in 2016. Canada’s legislation is the first to permit Nurse Practitioners (NP) to serve as independent MAiD assessors and providers. Registered Nurses’ (RN) also have important roles in MAiD that include MAiD care coordination; client and family teaching and support, MAiD procedural quality; healthcare provider and public education; and bereavement care for family. Nurses have a right under the law to conscientious objection to participating in MAiD. Therefore, it is essential to prepare nurses in their entry-level education for the practice implications and moral complexities inherent in this practice. Knowing what nursing students think about MAiD is a critical first step. Therefore, the purpose of this study was to develop a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in MAiD in the Canadian context.

The design was a mixed-method, modified e-Delphi method that entailed item generation from the literature, item refinement through a 2 round survey of an expert faculty panel, and item validation through a cognitive focus group interview with nursing students. The settings were a University located in an urban area and a College located in a rural area in Western Canada.

During phase 1, a 56-item survey was developed from existing literature that included demographic items and items designed to measure experience with death and dying (including MAiD), education and preparation, attitudes and beliefs, influences on those beliefs, and anticipated future involvement. During phase 2, an expert faculty panel reviewed, modified, and prioritized the items yielding 51 items. During phase 3, a sample of nursing students further evaluated and modified the language in the survey to aid readability and comprehension. The final survey consists of 45 items including 4 case studies.

Systematic evaluation of knowledge-to-date coupled with stakeholder perspectives supports robust survey design. This study yielded a survey to assess nursing students’ attitudes toward MAiD in a Canadian context.

The survey is appropriate for use in education and research to measure knowledge and attitudes about MAiD among nurse trainees and can be a helpful step in preparing nursing students for entry-level practice.

Peer Review reports

Medical Assistance in Dying (MAiD) is permitted under an amendment to Canada’s Criminal Code which was passed in 2016 [ 1 ]. MAiD is defined in the legislation as both self-administered and clinician-administered medication for the purpose of causing death. In the 2016 Bill C-14 legislation one of the eligibility criteria was that an applicant for MAiD must have a reasonably foreseeable natural death although this term was not defined. It was left to the clinical judgement of MAiD assessors and providers to determine the time frame that constitutes reasonably foreseeable [ 2 ]. However, in 2021 under Bill C-7, the eligibility criteria for MAiD were changed to allow individuals with irreversible medical conditions, declining health, and suffering, but whose natural death was not reasonably foreseeable, to receive MAiD [ 3 ]. This population of MAiD applicants are referred to as Track 2 MAiD (those whose natural death is foreseeable are referred to as Track 1). Track 2 applicants are subject to additional safeguards under the 2021 C-7 legislation.

Three additional proposed changes to the legislation have been extensively studied by Canadian Expert Panels (Council of Canadian Academics [CCA]) [ 4 , 5 , 6 ] First, under the legislation that defines Track 2, individuals with mental disease as their sole underlying medical condition may apply for MAiD, but implementation of this practice is embargoed until March 2027 [ 4 ]. Second, there is consideration of allowing MAiD to be implemented through advanced consent. This would make it possible for persons living with dementia to receive MAID after they have lost the capacity to consent to the procedure [ 5 ]. Third, there is consideration of extending MAiD to mature minors. A mature minor is defined as “a person under the age of majority…and who has the capacity to understand and appreciate the nature and consequences of a decision” ([ 6 ] p. 5). In summary, since the legalization of MAiD in 2016 the eligibility criteria and safeguards have evolved significantly with consequent implications for nurses and nursing care. Further, the number of Canadians who access MAiD shows steady increases since 2016 [ 7 ] and it is expected that these increases will continue in the foreseeable future.

Nurses have been integral to MAiD care in the Canadian context. While other countries such as Belgium and the Netherlands also permit euthanasia, Canada is the first country to allow Nurse Practitioners (Registered Nurses with additional preparation typically achieved at the graduate level) to act independently as assessors and providers of MAiD [ 1 ]. Although the role of Registered Nurses (RNs) in MAiD is not defined in federal legislation, it has been addressed at the provincial/territorial-level with variability in scope of practice by region [ 8 , 9 ]. For example, there are differences with respect to the obligation of the nurse to provide information to patients about MAiD, and to the degree that nurses are expected to ensure that patient eligibility criteria and safeguards are met prior to their participation [ 10 ]. Studies conducted in the Canadian context indicate that RNs perform essential roles in MAiD care coordination; client and family teaching and support; MAiD procedural quality; healthcare provider and public education; and bereavement care for family [ 9 , 11 ]. Nurse practitioners and RNs are integral to a robust MAiD care system in Canada and hence need to be well-prepared for their role [ 12 ].

Previous studies have found that end of life care, and MAiD specifically, raise complex moral and ethical issues for nurses [ 13 , 14 , 15 , 16 ]. The knowledge, attitudes, and beliefs of nurses are important across practice settings because nurses have consistent, ongoing, and direct contact with patients who experience chronic or life-limiting health conditions. Canadian studies exploring nurses’ moral and ethical decision-making in relation to MAiD reveal that although some nurses are clear in their support for, or opposition to, MAiD, others are unclear on what they believe to be good and right [ 14 ]. Empirical findings suggest that nurses go through a period of moral sense-making that is often informed by their family, peers, and initial experiences with MAID [ 17 , 18 ]. Canadian legislation and policy specifies that nurses are not required to participate in MAiD and may recuse themselves as conscientious objectors with appropriate steps to ensure ongoing and safe care of patients [ 1 , 19 ]. However, with so many nurses having to reflect on and make sense of their moral position, it is essential that they are given adequate time and preparation to make an informed and thoughtful decision before they participate in a MAID death [ 20 , 21 ].

It is well established that nursing students receive inconsistent exposure to end of life care issues [ 22 ] and little or no training related to MAiD [ 23 ]. Without such education and reflection time in pre-entry nursing preparation, nurses are at significant risk for moral harm. An important first step in providing this preparation is to be able to assess the knowledge, values, and beliefs of nursing students regarding MAID and end of life care. As demand for MAiD increases along with the complexities of MAiD, it is critical to understand the knowledge, attitudes, and likelihood of engagement with MAiD among nursing students as a baseline upon which to build curriculum and as a means to track these variables over time.

Aim, design, and setting

The aim of this study was to develop a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in MAiD in the Canadian context. We sought to explore both their willingness to be involved in the registered nursing role and in the nurse practitioner role should they chose to prepare themselves to that level of education. The design was a mixed-method, modified e-Delphi method that entailed item generation, item refinement through an expert faculty panel [ 24 , 25 , 26 ], and initial item validation through a cognitive focus group interview with nursing students [ 27 ]. The settings were a University located in an urban area and a College located in a rural area in Western Canada.

Participants

A panel of 10 faculty from the two nursing education programs were recruited for Phase 2 of the e-Delphi. To be included, faculty were required to have a minimum of three years of experience in nurse education, be employed as nursing faculty, and self-identify as having experience with MAiD. A convenience sample of 5 fourth-year nursing students were recruited to participate in Phase 3. Students had to be in good standing in the nursing program and be willing to share their experiences of the survey in an online group interview format.

The modified e-Delphi was conducted in 3 phases: Phase 1 entailed item generation through literature and existing survey review. Phase 2 entailed item refinement through a faculty expert panel review with focus on content validity, prioritization, and revision of item wording [ 25 ]. Phase 3 entailed an assessment of face validity through focus group-based cognitive interview with nursing students.

Phase I. Item generation through literature review

The goal of phase 1 was to develop a bank of survey items that would represent the variables of interest and which could be provided to expert faculty in Phase 2. Initial survey items were generated through a literature review of similar surveys designed to assess knowledge and attitudes toward MAiD/euthanasia in healthcare providers; Canadian empirical studies on nurses’ roles and/or experiences with MAiD; and legislative and expert panel documents that outlined proposed changes to the legislative eligibility criteria and safeguards. The literature review was conducted in three online databases: CINAHL, PsycINFO, and Medline. Key words for the search included nurses , nursing students , medical students , NPs, MAiD , euthanasia , assisted death , and end-of-life care . Only articles written in English were reviewed. The legalization and legislation of MAiD is new in many countries; therefore, studies that were greater than twenty years old were excluded, no further exclusion criteria set for country.

Items from surveys designed to measure similar variables in other health care providers and geographic contexts were placed in a table and similar items were collated and revised into a single item. Then key variables were identified from the empirical literature on nurses and MAiD in Canada and checked against the items derived from the surveys to ensure that each of the key variables were represented. For example, conscientious objection has figured prominently in the Canadian literature, but there were few items that assessed knowledge of conscientious objection in other surveys and so items were added [ 15 , 21 , 28 , 29 ]. Finally, four case studies were added to the survey to address the anticipated changes to the Canadian legislation. The case studies were based upon the inclusion of mature minors, advanced consent, and mental disorder as the sole underlying medical condition. The intention was to assess nurses’ beliefs and comfort with these potential legislative changes.

Phase 2. Item refinement through expert panel review

The goal of phase 2 was to refine and prioritize the proposed survey items identified in phase 1 using a modified e-Delphi approach to achieve consensus among an expert panel [ 26 ]. Items from phase 1 were presented to an expert faculty panel using a Qualtrics (Provo, UT) online survey. Panel members were asked to review each item to determine if it should be: included, excluded or adapted for the survey. When adapted was selected faculty experts were asked to provide rationale and suggestions for adaptation through the use of an open text box. Items that reached a level of 75% consensus for either inclusion or adaptation were retained [ 25 , 26 ]. New items were categorized and added, and a revised survey was presented to the panel of experts in round 2. Panel members were again asked to review items, including new items, to determine if it should be: included, excluded, or adapted for the survey. Round 2 of the modified e-Delphi approach also included an item prioritization activity, where participants were then asked to rate the importance of each item, based on a 5-point Likert scale (low to high importance), which De Vaus [ 30 ] states is helpful for increasing the reliability of responses. Items that reached a 75% consensus on inclusion were then considered in relation to the importance it was given by the expert panel. Quantitative data were managed using SPSS (IBM Corp).

Phase 3. Face validity through cognitive interviews with nursing students

The goal of phase 3 was to obtain initial face validity of the proposed survey using a sample of nursing student informants. More specifically, student participants were asked to discuss how items were interpreted, to identify confusing wording or other problematic construction of items, and to provide feedback about the survey as a whole including readability and organization [ 31 , 32 , 33 ]. The focus group was held online and audio recorded. A semi-structured interview guide was developed for this study that focused on clarity, meaning, order and wording of questions; emotions evoked by the questions; and overall survey cohesion and length was used to obtain data (see Supplementary Material 2  for the interview guide). A prompt to “think aloud” was used to limit interviewer-imposed bias and encourage participants to describe their thoughts and response to a given item as they reviewed survey items [ 27 ]. Where needed, verbal probes such as “could you expand on that” were used to encourage participants to expand on their responses [ 27 ]. Student participants’ feedback was collated verbatim and presented to the research team where potential survey modifications were negotiated and finalized among team members. Conventional content analysis [ 34 ] of focus group data was conducted to identify key themes that emerged through discussion with students. Themes were derived from the data by grouping common responses and then using those common responses to modify survey items.

Ten nursing faculty participated in the expert panel. Eight of the 10 faculty self-identified as female. No faculty panel members reported conscientious objector status and ninety percent reported general agreement with MAiD with one respondent who indicated their view as “unsure.” Six of the 10 faculty experts had 16 years of experience or more working as a nurse educator.

Five nursing students participated in the cognitive interview focus group. The duration of the focus group was 2.5 h. All participants identified that they were born in Canada, self-identified as female (one preferred not to say) and reported having received some instruction about MAiD as part of their nursing curriculum. See Tables  1 and 2 for the demographic descriptors of the study sample. Study results will be reported in accordance with the study phases. See Fig.  1 for an overview of the results from each phase.

Fig. 1  Overview of survey development findings

Phase 1: survey item generation

Review of the literature identified that no existing survey was available for use with nursing students in the Canadian context. However, an analysis of themes across qualitative and quantitative studies of physicians, medical students, nurses, and nursing students provided sufficient data to develop a preliminary set of items suitable for adaptation to a population of nursing students.

Four major themes and factors that influence knowledge, attitudes, and beliefs about MAiD were evident from the literature: (i) endogenous or individual factors such as age, gender, personally held values, religion, religiosity, and/or spirituality [ 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ], (ii) experience with death and dying in personal and/or professional life [ 35 , 40 , 41 , 43 , 44 , 45 ], (iii) training including curricular instruction about clinical role, scope of practice, or the law [ 23 , 36 , 39 ], and (iv) exogenous or social factors such as the influence of key leaders, colleagues, friends and/or family, professional and licensure organizations, support within professional settings, and/or engagement in MAiD in an interdisciplinary team context [ 9 , 35 , 46 ].

Studies of nursing students also suggest overlap across these categories. For example, value for patient autonomy [ 23 ] and the moral complexity of decision-making [ 37 ] are important factors that contribute to attitudes about MAiD and may stem from a blend of personally held values coupled with curricular content, professional training and norms, and clinical exposure. For example, students report that participation in end of life care allows for personal growth, shifts in perception, and opportunities to build therapeutic relationships with their clients [ 44 , 47 , 48 ].

Preliminary items generated from the literature resulted in 56 questions from 11 published sources (See Table  3 ). These items were constructed across four main categories: (i) socio-demographic questions; (ii) end of life care questions; (iii) knowledge about MAiD; or (iv) comfort and willingness to participate in MAiD. Knowledge questions were refined to reflect current MAiD legislation, policies, and regulatory frameworks. Falconer [ 39 ] and Freeman [ 45 ] studies were foundational sources for item selection. Additionally, four case studies were written to reflect the most recent anticipated changes to MAiD legislation and all used the same open-ended core questions to address respondents’ perspectives about the patient’s right to make the decision, comfort in assisting a physician or NP to administer MAiD in that scenario, and hypothesized comfort about serving as a primary provider if qualified as an NP in future. Response options for the survey were also constructed during this stage and included: open text, categorical, yes/no , and Likert scales.

Phase 2: faculty expert panel review

Of the 56 items presented to the faculty panel, 54 questions reached 75% consensus. However, based upon the qualitative responses 9 items were removed largely because they were felt to be repetitive. Items that generated the most controversy were related to measuring religion and spirituality in the Canadian context, defining end of life care when there is no agreed upon time frames (e.g., last days, months, or years), and predicting willingness to be involved in a future events – thus predicting their future selves. Phase 2, round 1 resulted in an initial set of 47 items which were then presented back to the faculty panel in round 2.

Of the 47 initial questions presented to the panel in round 2, 45 reached a level of consensus of 75% or greater, and 34 of these questions reached a level of 100% consensus [ 27 ] of which all participants chose to include without any adaptations) For each question, level of importance was determined based on a 5-point Likert scale (1 = very unimportant, 2 = somewhat unimportant, 3 = neutral, 4 = somewhat important, and 5 = very important). Figure  2 provides an overview of the level of importance assigned to each item.

Ranking level of importance for survey items

After round 2, a careful analysis of participant comments and level of importance was completed by the research team. While the main method of survey item development came from participants’ response to the first round of Delphi consensus ratings, level of importance was used to assist in the decision of whether to keep or modify questions that created controversy, or that rated lower in the include/exclude/adapt portion of the Delphi. Survey items that rated low in level of importance included questions about future roles, sex and gender, and religion/spirituality. After deliberation by the research committee, these questions were retained in the survey based upon the importance of these variables in the scientific literature.

Of the 47 questions remaining from Phase 2, round 2, four were revised. In addition, the two questions that did not meet the 75% cut off level for consensus were reviewed by the research team. The first question reviewed was What is your comfort level with providing a MAiD death in the future if you were a qualified NP ? Based on a review of participant comments, it was decided to retain this question for the cognitive interviews with students in the final phase of testing. The second question asked about impacts on respondents’ views of MAiD and was changed from one item with 4 subcategories into 4 separate items, resulting in a final total of 51 items for phase 3. The revised survey was then brought forward to the cognitive interviews with student participants in Phase 3. (see Supplementary Material 1 for a complete description of item modification during round 2).

Phase 3. Outcomes of cognitive interview focus group

Of the 51 items reviewed by student participants, 29 were identified as clear with little or no discussion. Participant comments for the remaining 22 questions were noted and verified against the audio recording. Following content analysis of the comments, four key themes emerged through the student discussion: unclear or ambiguous wording; difficult to answer questions; need for additional response options; and emotional response evoked by questions. An example of unclear or ambiguous wording was a request for clarity in the use of the word “sufficient” in the context of assessing an item that read “My nursing education has provided sufficient content about the nursing role in MAiD.” “Sufficient” was viewed as subjective and “laden with…complexity that distracted me from the question.” The group recommended rewording the item to read “My nursing education has provided enough content for me to care for a patient considering or requesting MAiD.”

An example of having difficulty answering questions related to limited knowledge related to terms used in the legislation such as such as safeguards , mature minor , eligibility criteria , and conscientious objection. Students were unclear about what these words meant relative to the legislation and indicated that this lack of clarity would hamper appropriate responses to the survey. To ensure that respondents are able to answer relevant questions, student participants recommended that the final survey include explanation of key terms such as mature minor and conscientious objection and an overview of current legislation.

Response options were also a point of discussion. Participants noted a lack of distinction between response options of unsure and unable to say . Additionally, scaling of attitudes was noted as important since perspectives about MAiD are dynamic and not dichotomous “agree or disagree” responses. Although the faculty expert panel recommended the integration of the demographic variables of religious and/or spiritual remain as a single item, the student group stated a preference to have religion and spirituality appear as separate items. The student focus group also took issue with separate items for the variables of sex and gender, specifically that non-binary respondents might feel othered or “outed” particularly when asked to identify their sex. These variables had been created based upon best practices in health research but students did not feel they were appropriate in this context [ 49 ]. Finally, students agreed with the faculty expert panel in terms of the complexity of projecting their future involvement as a Nurse Practitioner. One participant stated: “I certainly had to like, whoa, whoa, whoa. Now let me finish this degree first, please.” Another stated, “I'm still imagining myself, my future career as an RN.”

Finally, student participants acknowledged the array of emotions that some of the items produced for them. For example, one student described positive feelings when interacting with the survey. “Brought me a little bit of feeling of joy. Like it reminded me that this is the last piece of independence that people grab on to.” Another participant, described the freedom that the idea of an advance request gave her. “The advance request gives the most comfort for me, just with early onset Alzheimer’s and knowing what it can do.” But other participants described less positive feelings. For example, the mature minor case study yielded a comment: “This whole scenario just made my heart hurt with the idea of a child requesting that.”

Based on the data gathered from the cognitive interview focus group of nursing students, revisions were made to 11 closed-ended questions (see Table  4 ) and 3 items were excluded. In the four case studies, the open-ended question related to a respondents’ hypothesized actions in a future role as NP were removed. The final survey consists of 45 items including 4 case studies (see Supplementary Material 3 ).

The aim of this study was to develop and validate a survey that can be used to track the growth of knowledge about MAiD among nursing students over time, inform training programs about curricular needs, and evaluate attitudes and willingness to participate in MAiD at time-points during training or across nursing programs over time.

The faculty expert panel and student participants in the cognitive interview focus group identified a need to establish core knowledge of the terminology and legislative rules related to MAiD. For example, within the cognitive interview group of student participants, several acknowledged lack of clear understanding of specific terms such as “conscientious objector” and “safeguards.” Participants acknowledged discomfort with the uncertainty of not knowing and their inclination to look up these terms to assist with answering the questions. This survey can be administered to nursing or pre-nursing students at any phase of their training within a program or across training programs. However, in doing so it is important to acknowledge that their baseline knowledge of MAiD will vary. A response option of “not sure” is important and provides a means for respondents to convey uncertainty. If this survey is used to inform curricular needs, respondents should be given explicit instructions not to conduct online searches to inform their responses, but rather to provide an honest appraisal of their current knowledge and these instructions are included in the survey (see Supplementary Material 3 ).

Some provincial regulatory bodies have established core competencies for entry-level nurses that include MAiD. For example, the BC College of Nurses and Midwives (BCCNM) requires “knowledge about ethical, legal, and regulatory implications of medical assistance in dying (MAiD) when providing nursing care.” (10 p. 6) However, across Canada curricular content and coverage related to end of life care and MAiD is variable [ 23 ]. Given the dynamic nature of the legislation that includes portions of the law that are embargoed until 2024, it is important to ensure that respondents are guided by current and accurate information. As the law changes, nursing curricula, and public attitudes continue to evolve, inclusion of core knowledge and content is essential and relevant for investigators to be able to interpret the portions of the survey focused on attitudes and beliefs about MAiD. Content knowledge portions of the survey may need to be modified over time as legislation and training change and to meet the specific purposes of the investigator.

Given the sensitive nature of the topic, it is strongly recommended that surveys be conducted anonymously and that students be provided with an opportunity to discuss their responses to the survey. A majority of feedback from both the expert panel of faculty and from student participants related to the wording and inclusion of demographic variables, in particular religion, religiosity, gender identity, and sex assigned at birth. These and other demographic variables have the potential to be highly identifying in small samples. In any instance in which the survey could be expected to yield demographic group sizes less than 5, users should eliminate the demographic variables from the survey. For example, the profession of nursing is highly dominated by females with over 90% of nurses who identify as female [ 50 ]. Thus, a survey within a single class of students or even across classes in a single institution is likely to yield a small number of male respondents and/or respondents who report a difference between sex assigned at birth and gender identity. When variables that serve to identify respondents are included, respondents are less likely to complete or submit the survey, to obscure their responses so as not to be identifiable, or to be influenced by social desirability bias in their responses rather than to convey their attitudes accurately [ 51 ]. Further, small samples do not allow for conclusive analyses or interpretation of apparent group differences. Although these variables are often included in surveys, such demographics should be included only when anonymity can be sustained. In small and/or known samples, highly identifying variables should be omitted.

There are several limitations associated with the development of this survey. The expert panel was comprised of faculty who teach nursing students and are knowledgeable about MAiD and curricular content, however none identified as a conscientious objector to MAiD. Ideally, our expert panel would have included one or more conscientious objectors to MAiD to provide a broader perspective. Review by practitioners who participate in MAiD, those who are neutral or undecided, and practitioners who are conscientious objectors would ensure broad applicability of the survey. This study included one student cognitive interview focus group with 5 self-selected participants. All student participants had held discussions about end of life care with at least one patient, 4 of 5 participants had worked with a patient who requested MAiD, and one had been present for a MAiD death. It is not clear that these participants are representative of nursing students demographically or by experience with end of life care. It is possible that the students who elected to participate hold perspectives and reflections on patient care and MAiD that differ from students with little or no exposure to end of life care and/or MAiD. However, previous studies find that most nursing students have been involved with end of life care including meaningful discussions about patients’ preferences and care needs during their education [ 40 , 44 , 47 , 48 , 52 ]. Data collection with additional student focus groups with students early in their training and drawn from other training contexts would contribute to further validation of survey items.

Future studies should incorporate pilot testing with small sample of nursing students followed by a larger cross-program sample to allow evaluation of the psychometric properties of specific items and further refinement of the survey tool. Consistent with literature about the importance of leadership in the context of MAiD [ 12 , 53 , 54 ], a study of faculty knowledge, beliefs, and attitudes toward MAiD would provide context for understanding student perspectives within and across programs. Additional research is also needed to understand the timing and content coverage of MAiD across Canadian nurse training programs’ curricula.

The implementation of MAiD is complex and requires understanding of the perspectives of multiple stakeholders. Within the field of nursing this includes clinical providers, educators, and students who will deliver clinical care. A survey to assess nursing students’ attitudes toward and willingness to participate in MAiD in the Canadian context is timely, due to the legislation enacted in 2016 and subsequent modifications to the law in 2021 with portions of the law to be enacted in 2027. Further development of this survey could be undertaken to allow for use in settings with practicing nurses or to allow longitudinal follow up with students as they enter practice. As the Canadian landscape changes, ongoing assessment of the perspectives and needs of health professionals and students in the health professions is needed to inform policy makers, leaders in practice, curricular needs, and to monitor changes in attitudes and practice patterns over time.

Availability of data and materials

The datasets used and/or analysed during the current study are not publicly available due to small sample sizes, but are available from the corresponding author on reasonable request.

Abbreviations

British Columbia College of Nurses and Midwives

Medical assistance in dying

Nurse practitioner

Registered nurse

University of British Columbia Okanagan

Nicol J, Tiedemann M. Legislative Summary: Bill C-14: An Act to amend the Criminal Code and to make related amendments to other Acts (medical assistance in dying). Available from: https://lop.parl.ca/staticfiles/PublicWebsite/Home/ResearchPublications/LegislativeSummaries/PDF/42-1/c14-e.pdf .

Downie J, Scallion K. Foreseeably unclear. The meaning of the “reasonably foreseeable” criterion for access to medical assistance in dying in Canada. Dalhousie Law J. 2018;41(1):23–57.

Nicol J, Tiedeman M. Legislative summary of Bill C-7: an act to amend the criminal code (medical assistance in dying). Ottawa: Government of Canada; 2021.

Google Scholar  

Council of Canadian Academies. The state of knowledge on medical assistance in dying where a mental disorder is the sole underlying medical condition. Ottawa; 2018. Available from: https://cca-reports.ca/wp-content/uploads/2018/12/The-State-of-Knowledge-on-Medical-Assistance-in-Dying-Where-a-Mental-Disorder-is-the-Sole-Underlying-Medical-Condition.pdf .

Council of Canadian Academies. The state of knowledge on advance requests for medical assistance in dying. Ottawa; 2018. Available from: https://cca-reports.ca/wp-content/uploads/2019/02/The-State-of-Knowledge-on-Advance-Requests-for-Medical-Assistance-in-Dying.pdf .

Council of Canadian Academies. The state of knowledge on medical assistance in dying for mature minors. Ottawa; 2018. Available from: https://cca-reports.ca/wp-content/uploads/2018/12/The-State-of-Knowledge-on-Medical-Assistance-in-Dying-for-Mature-Minors.pdf .

Health Canada. Third annual report on medical assistance in dying in Canada 2021. Ottawa; 2022. [cited 2023 Oct 23]. Available from: https://www.canada.ca/en/health-canada/services/medical-assistance-dying/annual-report-2021.html .

Banner D, Schiller CJ, Freeman S. Medical assistance in dying: a political issue for nurses and nursing in Canada. Nurs Philos. 2019;20(4): e12281.

Article   PubMed   Google Scholar  

Pesut B, Thorne S, Stager ML, Schiller CJ, Penney C, Hoffman C, et al. Medical assistance in dying: a review of Canadian nursing regulatory documents. Policy Polit Nurs Pract. 2019;20(3):113–30.

Article   PubMed   PubMed Central   Google Scholar  

College of Registered Nurses of British Columbia. Scope of practice for registered nurses [Internet]. Vancouver; 2018. Available from: https://www.bccnm.ca/Documents/standards_practice/rn/RN_ScopeofPractice.pdf .

Pesut B, Thorne S, Schiller C, Greig M, Roussel J, Tishelman C. Constructing good nursing practice for medical assistance in dying in Canada: an interpretive descriptive study. Global Qual Nurs Res. 2020;7:2333393620938686. https://doi.org/10.1177/2333393620938686 .

Article   Google Scholar  

Pesut B, Thorne S, Schiller CJ, Greig M, Roussel J. The rocks and hard places of MAiD: a qualitative study of nursing practice in the context of legislated assisted death. BMC Nurs. 2020;19:12. https://doi.org/10.1186/s12912-020-0404-5 .

Pesut B, Greig M, Thorne S, Burgess M, Storch JL, Tishelman C, et al. Nursing and euthanasia: a narrative review of the nursing ethics literature. Nurs Ethics. 2020;27(1):152–67.

Pesut B, Thorne S, Storch J, Chambaere K, Greig M, Burgess M. Riding an elephant: a qualitative study of nurses’ moral journeys in the context of Medical Assistance in Dying (MAiD). Journal Clin Nurs. 2020;29(19–20):3870–81.

Lamb C, Babenko-Mould Y, Evans M, Wong CA, Kirkwood KW. Conscientious objection and nurses: results of an interpretive phenomenological study. Nurs Ethics. 2018;26(5):1337–49.

Wright DK, Chan LS, Fishman JR, Macdonald ME. “Reflection and soul searching:” Negotiating nursing identity at the fault lines of palliative care and medical assistance in dying. Social Sci & Med. 2021;289: 114366.

Beuthin R, Bruce A, Scaia M. Medical assistance in dying (MAiD): Canadian nurses’ experiences. Nurs Forum. 2018;54(4):511–20.

Bruce A, Beuthin R. Medically assisted dying in Canada: "Beautiful Death" is transforming nurses' experiences of suffering. The Canadian J Nurs Res | Revue Canadienne de Recherche en Sci Infirmieres. 2020;52(4):268–77. https://doi.org/10.1177/0844562119856234 .

Canadian Nurses Association. Code of ethics for registered nurses. Ottawa; 2017. Available from: https://www.cna-aiic.ca/en/nursing/regulated-nursing-in-canada/nursing-ethics .

Canadian Nurses Association. National nursing framework on Medical Assistance in Dying in Canada. Ottawa: 2017. Available from: https://www.virtualhospice.ca/Assets/cna-national-nursing-framework-on-maidEng_20170216155827.pdf .

Pesut B, Thorne S, Greig M. Shades of gray: conscientious objection in medical assistance in dying. Nursing Inq. 2020;27(1): e12308.

Durojaiye A, Ryan R, Doody O. Student nurse education and preparation for palliative care: a scoping review. PLoS ONE. 2023. https://doi.org/10.1371/journal.pone.0286678 .

McMechan C, Bruce A, Beuthin R. Canadian nursing students’ experiences with medical assistance in dying | Les expériences d’étudiantes en sciences infirmières au regard de l’aide médicale à mourir. Qual Adv Nurs Educ - Avancées en Formation Infirmière. 2019;5(1). https://doi.org/10.17483/2368-6669.1179 .

Adler M, Ziglio E. Gazing into the oracle. The Delphi method and its application to social policy and public health. London: Jessica Kingsley Publishers; 1996

Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006;53(2):205–12.

Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. 1st ed. City: Wiley; 2011.

Willis GB. Cognitive interviewing: a tool for improving questionnaire design. 1st ed. Thousand Oaks, Calif: Sage; 2005. ISBN: 9780761928041

Lamb C, Evans M, Babenko-Mould Y, Wong CA, Kirkwood EW. Conscience, conscientious objection, and nursing: a concept analysis. Nurs Ethics. 2017;26(1):37–49.

Lamb C, Evans M, Babenko-Mould Y, Wong CA, Kirkwood K. Nurses’ use of conscientious objection and the implications of conscience. J Adv Nurs. 2018;75(3):594–602.

de Vaus D. Surveys in social research. 6th ed. Abingdon, Oxon: Routledge; 2014.

Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: A primer. Front Public Health. 2018;6:149. https://doi.org/10.3389/fpubh.2018.00149 .

Puchta C, Potter J. Focus group practice. 1st ed. London: Sage; 2004.

Book   Google Scholar  

Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. 5th ed. Oxford: Oxford University Press; 2015.

Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

Adesina O, DeBellis A, Zannettino L. Third-year Australian nursing students’ attitudes, experiences, knowledge, and education concerning end-of-life care. Int J of Palliative Nurs. 2014;20(8):395–401.

Bator EX, Philpott B, Costa AP. This moral coil: a cross-sectional survey of Canadian medical student attitudes toward medical assistance in dying. BMC Med Ethics. 2017;18(1):58.

Beuthin R, Bruce A, Scaia M. Medical assistance in dying (MAiD): Canadian nurses’ experiences. Nurs Forum. 2018;53(4):511–20.

Brown J, Goodridge D, Thorpe L, Crizzle A. What is right for me, is not necessarily right for you: the endogenous factors influencing nonparticipation in medical assistance in dying. Qual Health Res. 2021;31(10):1786–1800.

Falconer J, Couture F, Demir KK, Lang M, Shefman Z, Woo M. Perceptions and intentions toward medical assistance in dying among Canadian medical students. BMC Med Ethics. 2019;20(1):22.

Green G, Reicher S, Herman M, Raspaolo A, Spero T, Blau A. Attitudes toward euthanasia—dual view: Nursing students and nurses. Death Stud. 2022;46(1):124–31.

Hosseinzadeh K, Rafiei H. Nursing student attitudes toward euthanasia: a cross-sectional study. Nurs Ethics. 2019;26(2):496–503.

Ozcelik H, Tekir O, Samancioglu S, Fadiloglu C, Ozkara E. Nursing students’ approaches toward euthanasia. Omega (Westport). 2014;69(1):93–103.

Canning SE, Drew C. Canadian nursing students’ understanding, and comfort levels related to medical assistance in dying. Qual Adv Nurs Educ - Avancées en Formation Infirmière. 2022;8(2). https://doi.org/10.17483/2368-6669.1326 .

Edo-Gual M, Tomás-Sábado J, Bardallo-Porras D, Monforte-Royo C. The impact of death and dying on nursing students: an explanatory model. J Clin Nurs. 2014;23(23–24):3501–12.

Freeman LA, Pfaff KA, Kopchek L, Liebman J. Investigating palliative care nurse attitudes towards medical assistance in dying: an exploratory cross-sectional study. J Adv Nurs. 2020;76(2):535–45.

Brown J, Goodridge D, Thorpe L, Crizzle A. “I am okay with it, but I am not going to do it:” the exogenous factors influencing non-participation in medical assistance in dying. Qual Health Res. 2021;31(12):2274–89.

Dimoula M, Kotronoulas G, Katsaragakis S, Christou M, Sgourou S, Patiraki E. Undergraduate nursing students’ knowledge about palliative care and attitudes towards end-of-life care: A three-cohort, cross-sectional survey. Nurs Educ Today. 2019;74:7–14.

Matchim Y, Raetong P. Thai nursing students’ experiences of caring for patients at the end of life: a phenomenological study. Int J Palliative Nurs. 2018;24(5):220–9.

Canadian Institute for Health Research. Sex and gender in health research [Internet]. Ottawa: CIHR; 2021 [cited 2023 Oct 23]. Available from: https://cihr-irsc.gc.ca/e/50833.html .

Canadian Nurses’ Association. Nursing statistics. Ottawa: CNA; 2023 [cited 2023 Oct 23]. Available from: https://www.cna-aiic.ca/en/nursing/regulated-nursing-in-canada/nursing-statistics .

Krumpal I. Determinants of social desirability bias in sensitive surveys: a literature review. Qual Quant. 2013;47(4):2025–47. https://doi.org/10.1007/s11135-011-9640-9 .

Ferri P, Di Lorenzo R, Stifani S, Morotti E, Vagnini M, Jiménez Herrera MF, et al. Nursing student attitudes toward dying patient care: a European multicenter cross-sectional study. Acta Bio Medica Atenei Parmensis. 2021;92(S2): e2021018.

PubMed   PubMed Central   Google Scholar  

Beuthin R, Bruce A. Medical assistance in dying (MAiD): Ten things leaders need to know. Nurs Leadership. 2018;31(4):74–81.

Thiele T, Dunsford J. Nurse leaders’ role in medical assistance in dying: a relational ethics approach. Nurs Ethics. 2019;26(4):993–9.

Download references

Acknowledgements

We would like to acknowledge the faculty and students who generously contributed their time to this work.

JS received a student traineeship through the Principal Research Chairs program at the University of British Columbia Okanagan.

Author information

Authors and affiliations.

School of Health and Human Services, Selkirk College, Castlegar, BC, Canada

Jocelyn Schroeder & Barbara Pesut

School of Nursing, University of British Columbia Okanagan, Kelowna, BC, Canada

Barbara Pesut, Lise Olsen, Nelly D. Oelke & Helen Sharp

You can also search for this author in PubMed   Google Scholar

Contributions

JS made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and drafting and substantively revising the work. JS has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. BP made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and drafting and substantively revising the work. BP has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. LO made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and substantively revising the work. LO has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. NDO made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and substantively revising the work. NDO has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. HS made substantial contributions to drafting and substantively revising the work. HS has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.

Authors’ information

JS conducted this study as part of their graduate requirements in the School of Nursing, University of British Columbia Okanagan.

Corresponding author

Correspondence to Barbara Pesut .

Ethics declarations

Ethics approval and consent to participate.

The research was approved by the Selkirk College Research Ethics Board (REB) ID # 2021–011 and the University of British Columbia Behavioral Research Ethics Board ID # H21-01181.

All participants provided written and informed consent through approved consent processes. Research was conducted in accordance with the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Schroeder, J., Pesut, B., Olsen, L. et al. Developing a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in Medical Assistance in Dying (MAiD): a mixed method modified e-Delphi study. BMC Nurs 23 , 326 (2024). https://doi.org/10.1186/s12912-024-01984-z

Download citation

Received : 24 October 2023

Accepted : 28 April 2024

Published : 14 May 2024

DOI : https://doi.org/10.1186/s12912-024-01984-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical assistance in dying (MAiD)
• End of life care
• Student nurses
• Nursing education

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

• Open access
• Published: 22 May 2024

Prognosis prediction models for post-stroke depression: a protocol for systematic review, meta-analysis, and critical appraisal

• Lu Zhou   ORCID: orcid.org/0000-0002-8314-053X 1 ,
• Lei Wang 1 ,
• Gao Liu 1 &
• EnLi Cai   ORCID: orcid.org/0000-0002-9916-1829 1  

Systematic Reviews volume  13 , Article number:  138 ( 2024 ) Cite this article

Metrics details

Introduction

Post-stroke depression (PSD) is a prevalent complication that has been shown to have a negative impact on rehabilitation outcomes and quality of life and poses a significant risk for suicidal intention. However, models for discriminating and predicting PSD in stroke survivors for effective secondary prevention strategies are inadequate as the pathogenesis of PSD remains unknown. Prognostic prediction models that exhibit greater rule-in capacity have the potential to mitigate the issue of underdiagnosis and undertreatment of PSD. Thus, the planned study aims to systematically review and critically evaluate published studies on prognostic prediction models for PSD.

Methods and analysis

A systematic literature search will be conducted in PubMed and Embase through Ovid. Two reviewers will complete study screening, data extraction, and quality assessment utilizing appropriate tools. Qualitative data on the characteristics of the included studies, methodological quality, and the appraisal of the clinical applicability of models will be summarized in the form of narrative comments and tables or figures. The predictive performance of the same model involving multiple studies will be synthesized with a random effects meta-analysis model or meta-regression, taking into account heterogeneity.

Ethics and dissemination

Ethical approval is considered not applicable for this systematic review. Findings will be shared through dissemination at academic conferences and/or publication in peer-reviewed academic journals.

Systematic review registration

PROSPERO CRD42023388548.

Peer Review reports

Stroke is a serious public health concern worldwide, with elevated rates of mortality, disability, and recurrence [ 1 ]. Post-stroke depression (PSD) refers to any depressive state that occurs after a stroke, which is the most common neuropsychiatric disorder [ 2 ]. The prevalence of PSD ranges from 11 to 41%, with a cumulative incidence of 65%, of which roughly 14% are diagnosed with major depressive disorder (MDD) [ 1 , 3 , 4 ]. PSD is increasingly becoming a research hotspot due to its severe negative effects and economic burden [ 5 ].

While recovery from depression after a stroke within a year improves functional outcomes and quality of life [ 6 , 7 ], PSD is linked to higher mortality, poorer recovery, more pronounced cognitive impairments, heavier financial burden, and lower quality of life than stroke without depression [ 8 ], indicating that depression hinders functional recovery after a stroke [ 9 ]. PSD can manifest at any point following a cerebrovascular event. It affects roughly one-third of stroke survivors and is notably associated with compromised functional recovery and heightened mortality rates. Thus, early screening and risk stratification interventions for stroke survivors at risk for depression are essential to adequately understand the mechanisms and development of symptomatology and even to change the prognosis. However, PSD arises from the complex interplay of neurobiological and psychosocial factors [ 1 ], exhibiting differential effects across various time frames post-stroke. The intricate interaction mechanisms and dynamic evolution of these factors throughout the development of PSD have posed enduring challenges within academic discourse. Consequently, this complexity contributes to suboptimal predictive dynamics and precision in PSD assessment.

Specifically, the diagnosis of PSD is primarily reliant on the Diagnostic and Statistical Manual of Mental Disorders (DSM) guidelines, in conjunction with a range of instruments measuring depression [ 10 ], but instruments have limitations in screening for PSD, such as insufficient clinical applicability or poor specificity [ 11 ]. In addition, PSD has been frequently under-diagnosed and under-treated due to the pathophysiological mechanisms of PSD not being fully understood [ 12 ], causing a sub-optimal prognosis for stroke survivors [ 13 ]. Nevertheless, the heightened administration of pharmacological treatment involving antidepressants, specifically escitalopram and fluoxetine, has demonstrated effectiveness in individuals who exhibit a high risk of PSD. But it is essential to note that such treatment may pose an excessive risk of harm in those who exhibit a lower risk of PSD. Notably, fluoxetine cannot improve depressive symptoms in PSD patients [ 13 , 14 ], and these therapies lack risk stratification. Thus, identifying prediction variables (e.g., biomarkers or psychosocial factors, as well as demographic and clinical characteristics of patients) associated with an increased risk of PSD occurrence and then developing multivariable prediction models is one of the promising PSD prevention strategies [ 15 ].

Currently, the construction, validation, and updating of predictive models are gaining attention in clinical research [ 16 ]. Prediction models are formal combinations of multiple predictors that estimate the probability of an individual currently having a certain disease (diagnostic model) or having a certain outcome in the future (prognostic model) through a mathematical formula [ 17 ], from which risks for specific endpoints can be calculated for individual patients to facilitate the dissemination of preventive interventions, provide patient counseling, and establish clinical guidelines and policies [ 18 , 19 ]. This study will focus on prognostic models.

Previous work has shown prediction models provide more accurate and less variable estimates of risk compared to more subjectively made predictions [ 20 ], but the methodology of model development is key to ensuring predictive performance. Although an increasing number of prognostic prediction models for PSD have been published [ 21 , 22 ], there has been limited advancement in the development of prognostic models for the stratification of PSD and MDD in stroke survivors [ 9 ], which are mainly based on clinical characteristics and biological markers ignoring psychosocial data support [ 23 ] causing the limited clinical predictive value. In addition, most of the existing prediction models are opportunistic and have been rarely used or even mentioned in clinical guidelines [ 24 ]. Only a small proportion of these models have been evaluated for their performance in data from other participants. Further, research design flaws, insufficient statistical methods, and incomplete reporting hinder the clinical application of these models. According to the PROGRESS group, significant heterogeneity exists among studies, the inclusion and exclusion criteria are too narrow, stroke type (ischemic or hemorrhagic) is not reported, blinding is rarely reported, preset cutoff values are not reported, multiple predictive models are rarely compared in the same population, and the appraisal of models across different languages, races/ethnicities, and cultures is lacking [ 15 ]. These factors point to significant waste in research, including both financial and scientific resources [ 25 ].

As the research on PSD prognostic prediction models continues to grow annually, there are varying emphases on the content, format, performance, and modeling approaches. The abundance of available clinical research data poses challenges for clinicians in extracting evidence, making it difficult to discern the most targeted predictive prognostic models to assist clinical decision-making and determine best practices from independently published literature. Furthermore, after preliminary searches in the PROSPERO database, Cochrane systematic review database, and JBI evidence synthesis, no completed or ongoing systematic reviews or scoping reviews were identified.

Thus, a comprehensive review and overview of existing PSD models is necessary to clarify their predictive performance, advantages, disadvantages, usage characteristics, and methodology. This will provide evidence-based support for practitioners in selecting models, while also promoting the development, validation, and updating of prognosis prediction models for PSD.

Research aims

The planned study aims to conduct a systematic review of all available evidence regarding the current prognostic models for PSD and to identify which prognostic prediction models have been developed, establishing the most effective and best performance model to predict PSD, while informing clinical decision-making. The specific aims of this systematic review are:

To ascertain the existing prognostic prediction models for PSD.

To qualitatively characterize the qualitative properties of the included prognostic prediction models.

To summarize and compare the current prognostic models and their predictive performance.

To critically appraise the studies identified for inclusion, particularly the research methodology and reporting methods.

The present protocol was formulated in adherence to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocol (PRISMA-P) guidelines and was duly registered with PROSPERO, the international prospective register of systematic reviews (Supplementary Material 1 contains the PRISMA-P checklist for reference) [ 26 , 27 ].

A systematic review of prognostic prediction modeling studies for PSD will be conducted and will be in accordance with the guidelines established by the Cochrane Prognosis Methods Group (PMG) and PROGnosis RESearch Strategy (PROGRESS) throughout all stages of the process [ 28 , 29 , 30 ]. Certain specific steps and models, for instance, framing, critical appraisal, and the assessment of the risk of bias, will be conducted by employing the CHARMS checklist (critical appraisal and data extraction for systematic reviews of prediction modeling studies) [ 31 ] and the PROBAST (Prediction model Risk Of Bias ASsessment Tool) with four domains (i.e., participants, predictors, outcome, and analysis) [ 32 , 33 ]. Moreover, for predictive modeling studies applying machine learning techniques, study selection and evidence appraisal will be based on the metrics and statements highlighted and extended in the Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis—Artificial Intelligence (TRIPOD-AI) and PROBAST-AI being developed [ 34 ].

Eligibility criteria

The outline of the review data and study selection were defined according to the CHARMS checklist (key items to guide the framing of the review aim, search strategy, and study inclusion and exclusion criteria) [ 31 ] detailed in Table  1 .

The inclusion criteria are as follows: (1) studies that develop or validate prognostic models (e.g., machine learning and Cox models), whether or not they include external validation; (2) study populations that involve research on ischemic or hemorrhagic stroke; (3) primary outcome measures indicating whether PSD occurred; (4) secondary outcome measures related to PSD, such as functional status, health status, quality of life, or mortality; (5) studies with a sample size of adequate power to detect small effects, with a goodness-of-fit statistic of over 0.99 for both closed and non-closed models; (6) cross-sectional and longitudinal primary research or literature research; (7) studies that report statistical models or instruments and their prediction indicators for predicting an individual’s risk of a future outcome (i.e., prognostic prediction model); (8) other names for prediction models include prognostic models, prognostic or prediction indices or rules, risk or clinical prediction models, and predictive models; and (9) prediction models are used to estimate the probability of a specific outcome occurring and can be reported using either absolute probability or relative risk score terms [ 35 ].

The exclusion criteria are as follows: (1) diagnostic prediction models; (2) evaluation of the predictive value of more than one variable, but without reporting subgroups or evaluation outcomes; (3) study population not related to stroke or studying a combined population with missing grouping results, or the study population only includes patients with individual or multiple complications of vascular damage (infarction, WMH, atrophy), such as cognitive impairment or dementia; (4) targeting depression occurring before stroke onset; and (5) presented as literature reviews, meta-reviews, protocols, theses, quality improvement activities, editorial comments, or letters, or not available in full text.

There will be no restrictions on year or language. In instances where multiple studies reported results from the identical cohort concerning a specific outcome measure, the data from the study that encompassed the largest patient population will be selected for analysis. Alternatively, if the studies involved an equal number of patients, the data from the earliest published study will be utilized.

Information sources

A search will be conducted in the following electronic databases: Ovid MEDLINE® Epub Ahead of Print, In-Process, Other Non-Indexed Citations, Daily Update; Embase Classic—Ovid®; Coverage: 1946 to present. The reference list of the included studies will undergo a meticulous manual search to identify any additional potentially relevant citations and a manual search will be conducted with the Google Scholar web search engine.

Search strategy

The search strategy will be devised for MEDLINE using the OvidSP platform, incorporating Medical Subject Headings (MeSH) and relevant keywords to enhance the efficacy of the search process (MeSH terms are available in Supplementary Material 2 ). Specifically, subject indexing terms will include a combination of the following five aspects of the PICOS system search construct [ 35 , 36 ]: #1 Population search AND #2 Index search AND #3 Comparator search AND #4 Outcomes search NOT #5 Study design-exclusion filter.

All model development studies will be back-citation-searched to identify potentially relevant external validation studies. Subsequently, a comprehensive review of all retrieved studies will be performed to ascertain their suitability for inclusion in the analysis. References identified by the search strategy will be entered into Endnote bibliographic software to screen the selected articles.

Study records

Data management.

Upon exportation from electronic databases, all search results will be subsequently imported into Covidence, a systematic review management platform, to facilitate efficient and organized review and analysis [ 37 ], available at https://www.covidence.org , and duplicates will be removed.

Study selection

Based on the established eligibility criteria for article selection, one author (L.Z.) will test the retrieve strategy across all the databases while two authors (G.L. and L.W.) will independently screen the titles and abstracts. The search results will be then screened a second time, in duplicate. Potential disagreements regarding the inclusion of an article will be resolved through a discussion but, in case of differences, a third researcher (EL.C.) decides whether to include an article. If there is no sufficient data to determine eligibility, additional information will be obtained from the study authors; if missing data cannot be obtained, studies will be excluded from the analysis. But the report with the highest risk of bias will also be removed if data from the identical samples are related to the same model testing.

Data collection process

The data will be extracted independently across the included studies by two reviewers (L.Z. and G.L.) using a standardized electronic form developed with reference to the CHARMS checklist (relevant items to extract from individual studies in a systematic review of prediction models for purposes of description or assessment of risk of bias or applicability) that is available in Supplementary Material 3 [ 31 ]. Moreover, the data items in the checklist will adapt to the specific clinical question, for instance, aims; data source; participants; stakeholders; algorithms; predicted outcomes; potential predictors; sample size; missing data; model development; model performance, including properties of discrimination with confidence intervals, calibration, classification, and overall performance; final multivariable models; interpretation of presented models; and model evaluation. Through discussions between the co-investigator (EL.C.) and two reviewers (L.Z and L.W.), the two data collection sheets will be reconciled into one data set. Any disagreement or uncertainty will be resolved by discussion among reviewers to reach a consensus, if required, by consulting another author of the review team (EL.C.).

Critical appraisal

PROBAST will be used to analyze the methodological quality and relevance of participants, predictors, and outcomes from each included study to the review topic in a systematic assessment [ 16 ]. With a total of 20 signaling questions, this instrument comprises four domains: participants, predictors, results, and analysis. Domains were scored as “high,” “low,” or “unclear” risk of bias. Two reviewers (L.Z. and G.L.) will independently apply the tool to rate the risk of bias and applicability of each included study of the 10 studies. The kappa coefficient for inter-rater reliability should be over 0.8 [ 38 ]. Any disagreement will be resolved by discussion. Graphical representations will be utilized to present the findings of each study.

Data synthesis

Evidence synthesis.

The initial methodology will involve utilizing a narrative synthesis approach to systematically detail the characteristics and quantitative data obtained from the studies that have been included. Specifically, the qualitative/heterogeneous outcomes of studies, including predictors, performance measures, classification measures, measures of uncertainty, and a descriptive analysis of key items [ 30 ], will be summarized qualitatively. Results will be presented in tabular form with each study to facilitate comparison.

Meta-analysis

The homogeneous outcomes of the same prediction model which meet the following criteria will be statistically analyzed in meta-analysis: (1) across ≥ 2 studies; (2) the identical category of prediction modeling study, specifically either development or validation; and (3) the follow-up periods for the primary outcome(s) are considered similar. While conducting the meta-analysis, it is possible to combine re-scaled measures of model performance which have similar outcomes. It will be typically accomplished via a random-effects meta-analysis approach, using restricted maximum likelihood estimation. Additionally, the Hartung-Knapp-Siddik-Jonkman method will be used to derive confidence intervals. Where feasible, 95% prediction intervals will be estimated. The performance of the prognostic prediction model will be based on the following measures [ 30 , 39 ], detailed in Table  2 . Additionally, where possible, we will employ multivariate meta-analysis for jointly synthesizing calibration and discrimination performance, while accounting for their correlation.

Sensitivity analysis and investigation of heterogeneity

To ascertain the robustness of the findings, sensitivity analyses will be conducted, wherein studies deemed to have a significant or uncertain risk of bias will be excluded. The I 2 statistic for univariate meta-analysis models and sub-group analyses will be employed to explore heterogeneity between studies. Between-effects heterogeneity will be estimated via restricted maximum-likelihood I 2 and tau 2 statistics. Potential sources of considerable between-effects heterogeneity will be investigated by conducting a meta-regression analysis ( p  < 0.05). If possible, the sub-group analysis will be based on:

Stroke types—ischemic or hemorrhagic.

Risk factors—biomarkers or psychosocial factors.

Depression types—PSD or MDD.

Modeling techniques—machine learning or non-machine learning.

Follow-up duration.

Region—based on the Organisation for Economic Co-operation and Development classification, that is, low/middle-income and high-income countries.

The meta-analysis process will be conducted in the metareg module in Stata 13.0 regarding the Meta-analysis of Observational Studies in Epidemiology (MOOSE) group guidelines [ 40 ].

Reporting findings

The findings of this systematic review will be reported in adherence to the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) guideline [ 41 , 42 ] and the PRISMA statement (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [ 43 ].

The planned study will be the first systematic review to evaluate existing evidence regarding prognostic prediction models (including machine learning algorithms, statistical models, and clinical risk scales) aimed at post-stroke depression for secondary prevention. The occurrence mechanism of PSD is complex and diverse. Currently, there is a lack of a gold standard for diagnosing PSD, and screening instruments have certain limitations, resulting in a relatively high rate of missed diagnoses. Although numerous PSD prediction models have been developed at this stage, most of the prediction models are not developed, validated, and assessed based on guidelines for predictive research [ 34 ]. This has led to significant biases in risk estimation and serious deficiencies in statistical methods, as well as a lack of internal and external validation [ 11 ], affecting the performance and applicability of the models and resulting in less-than-ideal accuracy and precision in clinical PSD prediction. Additionally, at present, there is a lack of systematic reviews and evaluations of PSD prediction models, which hinders relevant practitioners in selecting, promoting, and applying these models. This systematic review refers to details of the foundation and evidence for further studies, which aimed at developing, verifying, implementing, and assessing prognostic prediction models for PSD within the four domains of the PROGRESS prognosis research framework [ 44 ]. Regarding the TRIPOD-AI and PROBAST-AI tool, incorporating insights from these forthcoming extensions could enhance the review’s comprehensiveness and relevance, especially concerning machine learning-based prognostic models, ultimately contributing to more robust and applicable prognostic models for PSD in secondary prevention.

The findings will facilitate the early identification of people at high risk for PSD, the identification of the most effective current prognostic prediction models based on the shown predictive accuracy, and the stratification of PSD severity to estimate the risk of MDD after stroke. This will be a significant step towards informing the clinical management of patients with an established stroke diagnosis. It is essential for accurate identification of PSD, translation of clinical research of high-quality evidence, and savings in healthcare resources. Additionally, it will promote the consideration of the broad continuum of risk related to this condition in routine clinical practice. At a health service level, prediction models with good performance and high clinical applicability would support a personalized risk-stratified model of care, which would ultimately better direct finite health resources to stroke survivors at high risk of PSD and most likely to benefit from intervention.

Availability of data and materials

This content has been supplied by the authors.

Guo J, Wang J, Sun W, Liu X. The advances of post-stroke depression: 2021 update. J Neurol. 2022;(269):1–14.

Villa RF, Ferrari F, Moretti A. Post-stroke depression: mechanisms and pharmacological treatment. Pharmacol Ther. 2018;184:131–44.

Article   CAS   PubMed   Google Scholar  

Mortensen JK, Andersen G. Pharmacological management of post-stroke depression: an update of the evidence and clinical guidance. Expert Opin Pharmacother. 2021;22(9):1157–66.

Article   PubMed   Google Scholar  

Hackett ML, Pickles K. Part I: frequency of depression after stroke: an updated systematic review and meta-analysis of observational studies. Int J Stroke. 2014;9(8):1017–25.

James SL, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, Abbastabar H, Abd-Allah F, Abdela J, Abdelalim A. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1789–858.

Article   Google Scholar  

Medeiros GC, Roy D, Kontos N, Beach SR. Post-stroke depression: a 2020 updated review. Gen Hosp Psychiatry. 2020;66:70–80.

Nys G, Van Zandvoort M, Van Der Worp H, De Haan E, De Kort P, Jansen B, Kappelle L. Early cognitive impairment predicts long-term depressive symptoms and quality of life after stroke. J Neurol Sci. 2006;247(2):149–56.

Shi YZ, Xiang YT, Yang Y, Zhang N, Wang S, Ungvari GS, Chiu HF, Tang WK, Wang YL, Zhao XQ. Depression after minor stroke: the association with disability and quality of life–a 1-year follow-up study. Int J Geriatr Psychiatry. 2016;31(4):421–7.

Hirt J, van Meijeren LC, Saal S, Hafsteinsdóttir TB, Hofmeijer J, Kraft A, Meyer G, Janneke M. Predictive accuracy of the Post-Stroke Depression Prediction Scale: a prospective binational observational study✰. J Affect Disord. 2020;265:39–44.

Guo J, Wang J, Sun W, Liu X. The advances of post-stroke depression: 2021 update. J Neurol. 2022;269(3):1236–49.

Pollett S, Johansson MA, Reich NG, Brett-Major D, Del Valle SY, Venkatramanan S, Lowe R, Porco T, Berry IM, Deshpande A. Recommended reporting items for epidemic forecasting and prediction research: the EPIFORGE 2020 guidelines. PLoS Med. 2021;18(10):e1003793.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Ladwig S, Ziegler M, Südmeyer M, Werheid K. The Post-Stroke Depression Risk Scale (PoStDeRiS): development of an acute-phase prediction model for depression 6 months after stroke. J Acad Consult Liaison Psychiatry. 2022;63(2):144–52.

Choi-Kwon S, Han SW, Kwon SU, Kang D-W, Choi JM, Kim JS. Fluoxetine treatment in poststroke depression, emotional incontinence, and anger proneness: a double-blind, placebo-controlled study. Stroke. 2006;37(1):156–61.

Yi Z, Liu F, Zhai S. Fluoxetine for the prophylaxis of poststroke depression in patients with stroke: a meta-analysis. Int J Clin Pract. 2010;64(9):1310–7.

Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, Riley RD, Hemingway H, Altman DG, Group P. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10(2):e1001381.

Article   PubMed   PubMed Central   Google Scholar  

Wolff RF, Moons KG, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S. Group† P: PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2019;170(1):51–8.

Riley RD, Moons KG, Snell KI, Ensor J, Hooft L, Altman DG, Hayden J, Collins GS, Debray TP. A guide to systematic review and meta-analysis of prognostic factor studies. BMJ. 2019;364:k4597.

Collins GS, de Groot JA, Dutton S, Omar O, Shanyinde M, Tajar A, Voysey M, Wharton R, Yu L-M, Moons KG. External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med Res Methodol. 2014;14(1):1–11.

Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, Malats N, Briggs A, Schroter S, Altman DG. Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013;10(2):e1001380.

Kattan MW, Yu C, Stephenson AJ, Sartor O, Tombal B. Clinicians versus nomogram: predicting future technetium-99m bone scan positivity in patients with rising prostate-specific antigen after radical prostatectomy for prostate cancer. Urology. 2013;81(5):956–61.

Liu R, Yue Y, Jiang H, Lu J, Yuan Y, Wang Q. A risk prediction model of PSD in stroke survivors. Eur Psychiatry. 2015;30(S1):1–1.

Google Scholar  

Ryu YH, Kim SY, Kim TU, Lee SJ, Park SJ, Jung H-Y, Hyun JK. Prediction of poststroke depression based on the outcomes of machine learning algorithms. J Clin Med. 2022;11(8):2264.

Liu R, Yue Y, Jiang H, Lu J, Wu A, Geng D, Wang J, Lu J, Li S, Tang H. A risk prediction model for post-stroke depression in Chinese stroke survivors based on clinical and socio-psychological features. Oncotarget. 2017;8(38):62891.

Vickers AJ, Cronin AM. Everything you always wanted to know about evaluating prediction models (but were too afraid to ask). Urology. 2010;76(6):1298–301.

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. The Lancet. 2009;374(9683):86–9.

Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, Stewart L. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev. 2012;1(1):1–9.

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1–9.

Riley RD, Ridley G, Williams K, Altman DG, Hayden J, De Vet H. Prognosis research: toward evidence-based results and a Cochrane methods group. J Clin Epidemiol. 2007;60(8):863–5.

Moons KG, Hooft L, Williams K, Hayden JA, Damen JA, Riley RD. Implementing systematic reviews of prognosis studies in Cochrane. Cochrane Database Syst Rev. 2018;10:ED000129.

PubMed   Google Scholar  

Debray TP, Damen JA, Snell KI, Ensor J, Hooft L, Reitsma JB, Riley RD, Moons KG. A guide to systematic review and meta-analysis of prediction model performance. BMJ. 2017;356:i6460.

Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, Reitsma JB, Collins GS. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 2014;11(10):e1001744.

Moons KG, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med. 2019;170(1):W1–33.

Fernandez-Felix BM, López-Alcalde J, Roqué M, Muriel A, Zamora J. CHARMS and PROBAST at your fingertips: a template for data extraction and risk of bias assessment in systematic reviews of predictive models. BMC Med Res Methodol. 2023;23(1):1–8.

Collins GS, Dhiman P, Navarro CLA, Ma J, Hooft L, Reitsma JB, Logullo P, Beam AL, Peng L, Van Calster B. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. 2021;11(7):e048008.

Cooray SD, Boyle JA, Soldatos G, Wijeyaratne LA, Teede HJ. Prognostic prediction models for pregnancy complications in women with gestational diabetes: a protocol for systematic review, critical appraisal and meta-analysis. Syst Rev. 2019;8(1):1–10.

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14(1):1–10.

Babineau J. Product review: Covidence (systematic review software). J Can Health Libr Assoc. 2014;35(2):68–71.

Kvalseth TO. A coefficient of agreement for nominal scales: an asymmetric version of Kappa. Educ Psychol Meas. 1991;51(1):95–101.

Huang C, Li S-X, Caraballo C, Masoudi FA, Rumsfeld JS, Spertus JA, Normand SLT, Mortazavi BJ, Krumholz HM. Performance metrics for the comparative analysis of clinical risk prediction models employing machine learning. Circ Cardiovasc Qual Outcomes. 2021;14(10):e007526.

Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. J British Surg. 2015;102(3):148–58.

Article   CAS   Google Scholar  

Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162(1):W1–73.

Moher D, Liberati A, Tetzlaff J, Altman DG. Group* P: Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.

Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, Briggs A, Udumyan R, Moons KG, Steyerberg EW. Prognosis Research Strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ. 2013;346:e5595.

Download references

Acknowledgements

None declared.

Patient and public involvement

The study was conducted without the involvement of patients and the general public in its design and conception.

The study was supported by the Yunnan University of Chinese Medicine Research Program (202101AZ070001-221), and a grant was provided to the corresponding author (Enli Cai) to support stroke-related research.

Author information

Authors and affiliations.

School of Nursing, Yunnan University of Chinese Medicine, Kunming, China

Lu Zhou, Lei Wang, Gao Liu & EnLi Cai

You can also search for this author in PubMed   Google Scholar

Contributions

EL.C. and L.Z. conceived the study idea, and L.Z. was responsible for developing and writing the first draft of the systematic review protocol and manuscript. G.L. and L.W. contributed to the data curation. EL.C., L.Z., G.L., and L.W. provided critical insights at all stages. All authors approved and contributed to the final manuscript.

Corresponding author

Correspondence to EnLi Cai .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval is not applicable to this study. The findings will be disseminated by publishing in peer-reviewed publications and/or academic conferences.

Consent for publication

All authors approved and contributed to the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Zhou, L., Wang, L., Liu, G. et al. Prognosis prediction models for post-stroke depression: a protocol for systematic review, meta-analysis, and critical appraisal. Syst Rev 13 , 138 (2024). https://doi.org/10.1186/s13643-024-02544-x

Download citation

Received : 01 September 2023

Accepted : 23 April 2024

Published : 22 May 2024

DOI : https://doi.org/10.1186/s13643-024-02544-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Systematic Review
• Open access
• Published: 20 May 2024

Bioelectrical impedance analysis—derived phase angle (PhA) in lung cancer patients: a systematic review

• Melania Prete 1 ,
• Giada Ballarin 2 ,
• Giuseppe Porciello 3 ,
• Aniello Arianna 4 ,
• Assunta Luongo 3 ,
• Valentina Belli 5 ,
• Luca Scalfi 4 &
• Egidio Celentano 3  

BMC Cancer volume  24 , Article number:  608 ( 2024 ) Cite this article

41 Accesses

1 Altmetric

Metrics details

Lung cancer is the second most diagnosed cancer in the world. Up to 84% of diagnosed patients have malnutrition, which can negatively affect quality of life and survival and may worsen with neoadjuvant treatment. Bioelectrical Impedance Analysis-Derived Phase Angle (PhA) in these patients could be a valid tool to assess the nutritional status in order to improve their condition.

This review provides an update on PhA assessment in lung cancer patients over the past twenty years. We searched PubMed, Embase, Scopus, Web of Science, and Cochrane, for articles regarding the PhA obtained from Bioelectrical Impedance Analysis in lung cancer patients. The authors independently performed a literature search: sample size, patient population, study type, study dates, survival and interventions were evaluated. The final review included 11 studies from different countries.

Eight studies only considered patients with lung cancer, while three studies considered patients with different kind of cancer, including lung. Correlation data between PhA and age are conflicting. In patients undergoing clinical treatment and patients undergoing surgical treatment lower PhA was observed. A lower PhA is associated with a shorter survival. In three studies emerged a relationship between Karnofski Performance Status and Handgrip Strenght with PhA. From one study, univariate logistic regression analysis showed that higher PhA values represent a protective factor for sarcopenia.

Our research underlined interesting, but not conclusive, results on this topic; however more researches are needed to understand the clinical meaning of PhA.

Peer Review reports

Introduction

Lung Cancer (LC) is the second most diagnosed cancer worldwide, especially in males. Most recent data have shown an incidence of 2.2 million of new cases (11.4%) and 1,8 million of deaths (18.0%) occurred in 2020. It represents leading cause of cancer death in 93 countries [ 1 ]. Following diagnosis, 5-year survival rates ranges from 10 to 20% in most countries, with higher rates in Japan (33%), Israel (27%), and Korea (25%) [ 1 ]. In Italy, LC showed a 5-year survival of 16% in men and 23% in women [ 2 ]. LC aetiology is multifactorial and complex. In addition to a family history of LC, tobacco smoke currently represents the leading risk factor [ 2 , 3 ]. Second-hand tobacco use may also increase LC risk, causing more than 3.000 deaths each year [ 4 ]. Other lung carcinogens include inhaled chemicals such as arsenic, cadmium and asbestos [ 5 ].

LCs are traditionally classified in small cell lung carcinoma (SCLC) and non-small cell lung carcinoma (NSCLC) divided into four major classes (adenocarcinoma, squamous cell carcinoma and large cell carcinoma) [ 6 ]. Conventional LC therapies include surgical intervention for resectable diseases and, in selected cases, a combination of radiotherapy (RT) and chemotherapy (CHT) for locally advanced or metastatic disease. Advancements in the understanding of LC molecular pathogenesis has led to the development of targeted strategies like immune checkpoint inhibitor (ICI) in first and later lines of treatment [ 7 ].

In addition to cancer-related symptoms, including chronic cough, dyspnoea, pain and adverse effects from anti-neoplastic treatments, LC patients’ may experience fatigue, weight loss or nutritional status alterations, such as malnutrition [ 8 , 9 ]. Cancer patients are more likely to become malnourished, with a prevalence ranging from 20.0% to 80.0% [ 10 ]. Recent studies indicate that dietary nutrient deficiency in cancer patients may induce unintentional body weight loss to sarcopenia, up to cachexia [ 11 ]. It is known that malnutrition was prevalent in advanced LC patients [ 12 ]: up to 84% of them showed malnutrition status during illness, which can be worsened by ongoing neoadjuvant treatment [ 12 , 13 , 14 , 15 , 16 , 17 ]. This condition has been associated with poorer prognosis, decreased treatment response, poorer tolerance to treatment, lower quality of life (QoL) and increased healthcare costs [ 12 , 18 ]. Additionally, sarcopenia, defined as progressive loss of muscle mass and functioning, is highly prevalent among LC patients ranging from 42.8% to 45.0%, in association with increased postoperative complications and increased risk of mortality, regardless of cancer stage and treatment [ 19 ]. Furthermore, LC is more commonly linked to cancer cachexia than other types of cancer [ 16 ], characterized mainly by a decrease in muscle strength, due to the loss of adipose tissue and skeletal mass [ 20 , 21 ].

Body Composition (BC) is a crucial requirement for the overall body assessment of cancer patients: it can reflect the nutritional status of patients and predict clinical outcomes and prognosis [ 22 ]. Bioelectrical Impedance Analysis (BIA) is a simple, cost-effective and non-invasive method that measures electrical characteristics of human body, i.e. impedance (Z), through application of four electrodes and an applied alternate current, using single (SF-BIA) or multiple (MF-BIA) current frequencies. Z derives from resistive component (Resistance, R) and capacitive component (Reactance, X c ), by equation Z = R 2  + Xc 2 ​. R indicates how much a substance opposes the flow of electric current: greater resistance indicates greater difficulty of passage. It can be affected by factors such as tissue density or hydration and cell membrane permeability. Reactance reflects the ease with which electricity can flow through tissues: high reactance means that there is more resistance of tissues and less conductivity. It can therefore indicate cellular and membrane integrity.

BIA-derived Phase Angle (PhA) is obtained as [arctangent (X c /R) × 180°/π]. PhA represent an indicator of cellular health, cell membrane integrity, and better cell function: low values are indicators of apoptosis and cell matrix alteration [ 23 , 24 ]. Therefore, BIA analysis and the use of the PhA have a good consistency in the application in cancer patients to evaluate nutritional and hydration status [ 25 , 26 ]. Despite importance of nutritional status and BC for the clinical evaluation of cancer patients, these conditions remain in part an undertreated issue [ 27 ]. Thus, nutritional status assessment of these patients is essential for adequate nutritional support, and may also improve QoL and consequently survival post-diagnosis [ 28 , 29 ]. In the literature, a large number of publications on BIA and in particular of PhA assessment, confirm its prognostic role in different types of cancer (e.g., breast, pancreatic and colon) [ 30 , 31 , 32 , 33 ]. Although PhA is a useful prognosis tool even in patients with LC [ 34 ], this has not been discussed in detail in scientific literature. So, our review aims to critically report and discuss available clinical data relating to PhA in LC patients, to provide a broad and clear picture of topic.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed for performing the present review, considering the possibility of including both cross-sectional and longitudinal studies. Further details about PRISMA checklist and study protocol were provided in Supplementary File. Due to the study type, ethical approval was not required. This systematic revision is not currently registered in any database.

Data sources

Authors independently performed a systematic literature research between 2000 and 2023 of the electronic databases PubMed, Embase, Scopus, Web of Science and Cochrane. The following terms were used as search strategy string on full texts: phase angle" AND (lung or pulmonary) AND (bioelectrical OR impedance OR bia) AND cancer”, "phase angle" AND (bia OR bioelectrical OR bioimpedance) AND cancer”. Zotero and EndNote X7 citation management software were used to manage citations.

Criteria for analysis

In this review, cross-sectional, case–control, and longitudinal studies were included. The measurement of PhA values by BIA was a necessary and indispensable condition for an article to be included. The studies had to be present in the literature in English form and had to be published no earlier than 2000 to include the most recent evidence. Outcome of interest included associations between PhA and survival, mortality, or other variables related to LC patients. Furthermore, it was necessary that studies must have been conducted in the health field. Articles that did not meet these requirements were excluded from the review. All studies include a phase-sensitive BC measurement tool. In all selected studies PhA is calculated by the ratio of R to X c equal to [arctangent (X c /R) × 180°/π. BC parameters were not obtained by predictive regression models.

Data extraction and analysis

To assess the suitability of the articles obtained from the literature search, authors carefully and meticulously examined all the titles and abstracts. Subsequently, the authors independently extracted the data from the papers and reported in an excel file. The data included: first author, year of publication, country of origin, design of study, sample size, age, sex, presence of control groups, type of tumour, methods used (BIA, BIVA), focusing on PhA.

Selected studies

One hundred and sixty-five papers were identified from the systematic search, 92 from Pubmed, 32 from Embase, 6 from Scopus and 38 from Web of Science. After removing duplicates ( n  = 38), 120 articles were excluded, including 2 reviews, 1 symposium and 117 that did not concern LC. A total of 11 full-text articles were selected for eligibility, as shown in Fig.  1 .

General characteristics of included studies

This systematic review includes several types of studies: five prospective studies [ 35 , 36 , 37 , 38 , 39 ], three observational studies [ 40 , 41 , 42 ], two retrospective studies [ 34 , 43 ] and one cross-sectional study [ 44 ]. Shi et al. showed the highest number of patients involved (804). The remaining selected scientific papers included 30 to 204 patients. Five studies included both males and females, six studies included males only [ 36 , 38 , 40 , 41 , 42 , 44 ]. However, in two studies [ 36 , 37 ], we do not know if the percentages of women and men present in the study referred to LC patients or patients with other kind of cancers. General characteristics of studies included in this review are shown completely in Table  1 .

Relationship between PhA, general characteristics and cancer features

Four studies investigated the potential relationship between age and PhA. In the study by Ji et al. [ 36 ], Pearson’s analysis showed an inverse correlation for age ( r  = -0.238 p  =  < 0.001). Shi et al. [ 39 ], showed both for men and women, higher PhA in younger (for both, p  < 0.001); Spearman’s correlation analysis also showed that PhA was significantly correlated with age (men, r  = -0.46, p  < 0.001; women, r  = -0.24, p  < 0.001). In Castanho et al. [ 44 ], Pearson’s correlation showed no significant results between PhA and age ( r  = -0.32). Suzuki et al. showed a negative correlation between age and PhA ( r  = -0.51; p  < 0.001); Spearman’s correlation was used to assess the correlation between age and PhA.

Only 8 of 11, considered data on patients with LC, while 3 studies considered patients with different types of cancer, including LC. Three other studies investigate PhA in different types of cancer, including LC: they respectively include 8, 244 and 26 patients with LC [ 36 , 37 , 45 ]. All studies showed patients with confirmed cancer diagnosis: five studies evaluated NSCLC patients with stage III and IV [ 34 , 38 , 40 , 41 , 42 ]. Shi et al. included adenocarcinoma, squamous cell carcinoma, SCLC and other type of LC patients in different stage of disease (from I to IV) [ 39 ]. Suzuki et al. and Hui et al. showed data on LC patients but did not specify cancer type or stage [ 43 , 45 ]. Cancer stage was not specified in two studies [ 36 , 37 ]. Castanho et al. [ 44 ] considered LC patients presenting from stage IB to IIIB. Results showed, by multifactorial analysis of variance, correlations between PhA and tumour size ( r  = -0.55; p  < 0.001) or Karnofski Performance Status (KPS) ( r  = 0.44; P  < 0.05). Three studies included patients not undergoing CHT, RT and without specific treatment information [ 37 , 39 , 45 ] while newly diagnosed patients and/or patients with ongoing therapies were evaluated in remaining studies [ 34 , 36 , 38 , 42 ]. Suzuki et al. [ 43 ] evaluated LC patients after surgery while patients were evaluated after cancer treatment in two studies [ 40 , 41 ]. Castanho et al. [ 44 ] evaluated patients after surgery and after cancer treatment (CHT, RT).

Relationship between PhA and body composition

The main associations identified in selected studies, between PhA and different variables, are shown in Table  2 . In seven studies parameters related to BC have been evaluated [ 36 , 39 , 40 , 41 , 43 , 44 ]. Castanho et al. [ 44 ] correlates PhA with arm circumference and weight loss over time. Authors also indicate that between patients undergoing surgery, those with lowest survival (52 days) showed a lower PhA (3.8°) and a High Extracellular Mass/Body Cell Mass (ECM:BCM) ratio (1:5), independent from tumour size. Whereas, in those treated medically, patients with a lower survival also had lowest PhA (3.9°) and highest ECM/BCM ratio (1:6), this were related to tumour volume (849 ml).

Shi et al. [ 39 ] shows that male patients < 65 years, with a lower Body Mass Index (BMI) and lower cancer stage have higher PhA values, but the differences were not statistically significant. In Suzuki et al. [ 43 ], Spearman’s correlation showed a positive correlation between PhA and BMI (rho = 0.29; p  < 0.001); no correlation with body Fat Mass (FM) was found. In Hui et al. [ 45 ] study, PhA was associated with several known prognostic variables, including Fat-Free Mass (FFM) and FFM index (FFMI). However, the Spearman correlation was weak (rho < 0.4; p  < 0.001). Two articles assessed different aspect within the same LC patients' population (Stage IV, male patients), including PhA. In Detopoulou et al. [ 40 ] were found significant correlations between PhA and FFM (rho = 0.247; p  = 0.02), but no significant correlation for waist and hip circumference (cm), waist-hip ratio, body fat (%), BCM (Kg), total body water (TBW), extra-cellular and intra-cellular water (ECW, ICW). In the other study, Spearman’s correlation shows no significance between PhA, anthropometric and BC variables [ 41 ]. In Wei Ji et al. [ 36 ], in addition to PhA, other variables were also examined, such as appendicular skeletal muscle mass (ASMM), BMI and skeletal muscle mass index (SMI). Pearson’s correlations showed a moderate correlation between PhA values and variables considered (ASMM r  = 0.301, p  < 0.001; BMI r  = 0.450, p  < 0.001; SMI r  = 0.463, p  < 0.001.

Relationship between PhA, nutritional status and nutritional risk

PhA relationships with nutritional status and malnutrition screening scores were evaluated. Shi et al. indicated a significant correlation between PhA and some nutritional index: results of Spearman’s rank correlation test showed correlation with Nutritional Risk Score-2002 (NRS-2002) (men, r  =  − 0.25, p  < 0.001; women, r  =  − 0.15, p  < 0.001). No correlation between PhA and Patient-Generated Subjective Global Assessment (PG-SGA) score was found. Then, logistic regression analysis showed significant correlation between PhA, NRS-2002 score (men, p  < 0.001; women, p  < 0.001) and PG-SGA score (men, p  < 0.001; women, p  < 0.001) in both men and women, indicating an association with secondary clinical outcomes such as nutrition and well-being [ 39 ]. In Suzuki et al. [ 43 ], Spearman’s correlation showed a positive correlation between PhA and albumin (rho = 0.33; p  < 0.001), considered a useful biochemical markers for nutrition assessment, and Prognostic Nutritional Index (PNI) (rho = 0.32; p  < 0.001), a simple index obtained from serum albumin concentration and total peripheral blood lymphocyte count, used to assess the immune-nutritional status of patients who undergo surgery. In Hui et al. [ 45 ] study, PhA was associated with several known prognostic variables, including serum albumin, but correlation was weak (γ < 0.4, p  < 0.001; Spearman correlation test). Detopoulou et al. [ 40 ] found significant correlations between PhA and dietary pattern (Food pattern 2) rich in potatoes, meat and poultry (rho = 0.254, p  = 0.02). No significant results with PhA and other dietary patterns (food pattern 1: whole grains, fruits, vegetables; food pattern 3: high olive oil, low alcohol; food pattern 4: legumes, fish). Finally, in the same patient’s sample, Mediterranean Diet Score (MedDiet Score) was positively related to PhA changes (rho = 0.251; p  = 0.02).

Relationship between PhA, prognostic indices, quality of life and survival

Some of the selected studies evaluated the correlation between PhA and some prognostic indices, QoL scores and survival in patients with LC. Five out of eleven studies indicate patient survival data in relation to the PhA [ 34 , 35 , 38 , 39 , 42 ]; in two studies, indicators associated with survival were evaluated [ 34 , 35 , 38 , 39 , 42 , 43 , 44 ]. Sanchez-Lara et al. and Shi et al. have evaluated QoL in relation to PhA.

Multifactorial analysis of variance showed correlations between PhA and KPS ( r  = 0.44; P  < 0.05) in cross-sectional study by Castanho et al. [ 44 ].

In Hui et al. [ 45 ] study, PhA was associated with several known prognostic variables, including the Palliative Performance Scale (γ = 0.18; p  = 0.007), KPS (γ = 0.18; p  = 0.007), Palliative Prognostic Score (γ = -0.21; p  = 0.002), and Palliative Prognostic Index (γ = -0.22; p  = 0.001). Unadjusted PhA ( P  = 0.001) was found to be significantly associated with overall survival, as indicated by Kaplan-Meyer curves analysis: a lower value is associated with poor survival (PhA < 3°, median 35 days; 95% CI, 29–41 days).

Sanchez-Lara et al. [ 38 ] evaluated the association of PhA, QoL’s dimensions EORTC QLQ C30 (QLQ-C30) and survival in patients with advanced NSCLC. No significant association between PhA and QoL scores were found. The bivariate survival analysis shows that PhA ≤ 5.8° was significantly associated with low overall survival; multivariate analysis indicate for highest PhA values a higher survival rate (HR = 3.02; 95% CI, 1.2–7.11; p  = 0.011).

The results of Spearman’s rank correlation test in Shi et al. [ 39 ] showed correlation between PhA and QoL. It was found a L-shaped association between PhA and LC survival in both sexes (men p  = 0.019 and women p  = 0.121); an association between higher PhA and better survival resulted for men and women ( p  = 0.007 and p  < 0.001, respectively). Kaplan–Meier survival curves for patients with high and low PhA values in different cancer stages showed longer OS in patients with high PhA than patients with low PhA, without taking account stage. Univariate Cox regression analysis showed that continuous PhA was significantly associated with mortality in men with LC ( p  = 0.015); also in women, PhA was significantly associated with survival ( p  = 0.029). After adjusting for several covariates, in a multivariate-adjusted Cox regression analysis PhA was identified as an independent risk factor for mortality in men (HR = 0.79, 95% CI = 0.65–0.95, p  = 0.015), but not in women ( p  = 0.105) [ 39 ].

In Gupta et al., univariate Kaplan–Meier survival analysis showed statistically significant differences ( p  = 0.02) between patients with PhA <  = 5.3 (median survival = 7.6 months; 95% CI: 4.7 to 9.5; n  = 81) and those with > 5.3 (12.4 months; 95% CI: 10.5 to 18.7; n  = 84) [ 34 ].

No correlation with Charlson Comorbidity Index was found in Suzuki et al. (rho = -0.09; p  = 0.16). Also, in this study, multivariate logistic analysis reveals that PhA (OR = 0.51, 95% CI: 0.29–0.90, p  = 0.018) was an independent predictor of Clavien-Dindo grade ≥ II, index used for surgical complications [ 43 ].

Data from univariate survival analysis of Toso et al., stratified by the cancer stage, indicated that LC patients with a PhA ≤ 4.5° had significantly shorter survival compared to those who have a higher PhA ( p  = 0.01) (median of 3.7 vs 12.1 months in patients with a PhA ≤ 4.5° vs > 4.5°, respectively, from stage IIIB, and 1.4 vs 5.0 months in in patients with a PhA ≤ 4.5° vs > 4.5, respectively from stage IV) [ 42 ].

Relationship between PhA, muscle strength and physical efficiency

Navigante et al. [ 37 ] evaluated weakness assessed with Hand-grip strength (HGS). In patients with LC only statistically significant result was linear correlation between grip work and PhA ( p  = 0.007), which was considered very significant (95% CI: 0.3843 to 0.9717).

In Hui et al., PhA was also associated with HGS, but correlation was not very strong (Spearman’s correlation γ < 0.4; p  > 0.001) [ 35 ].

Wei Ji et al. have evaluated muscle strength and ASMM to define sarcopenia diagnosis: PhA had strongest correlation with SMI ( r  = 0.463) and HGS ( r  = 0.354). Logistic regression analysis adjusted for potential confounders showed that higher PhA values represent a protective factor for sarcopenia (OR 0.309, 95% CI, 0.246 0.617; p  < 0.001) [ 36 ].

Comparison between different groups and PhA

In the study of Toso et al. were reported differences between healthy subjects, patients with IIIB and IV stages.

Comparing patients with healthy controls was found a reduction in PhA value (resulting in a reduction in capacity, but not R. No significant differences between two groups of patients with IIIB and IV stages were found. However, a significant difference between patients with different stages (statistically lower in patients with higher stages) was observed for survival [ 42 ].

In Navigante et al. was carried out a comparison between different groups (healthy volunteers vs patients), but in reference to muscle strength (maximal muscle strength, mean muscle strength, median muscle strength) and not to PhA [ 37 ].

In the Hui et al. patients’ cohort ( n  = 204) with different types of cancer (including breast, gastrointestinal, head and neck and gynaecological) two different groups have been distinguished: patients with edema and without edema. Univariate analysis showed a reduced survival for PhA ≤ 3° vs PhA > 3° for total patients ( p  = 0.045) and no edema patients ( p  < 0.001). PhA ≤ 3° was associated with shorter survival in the non-oedematous cohort (HR 4.42, 95% CI 2.09–9.36, p  < 0.001), but this association did not occur in the whole cohort (HR 1.44, 95% CI 0.99–2.09, p  = 0.054) and in the cohort with edema (HR 1.04, 95% CI 0.67–1, 62, p  = 0.85) (Cox regression analysis) [ 35 ].

Wei Ji et al. evaluated the association between PhA in older male patients with different types of malignancies, with sarcopenia (22.0%) and without. PhA in patients without sarcopenia was 5.02° (SD ± 0.72°), while in sarcopenic was 4.18° (SD ± 0.85°); this difference was statistically significative ( p  < 0.001) [ 36 ].

The present review aims to investigate the current data regarding PhA in LC patients. We did not find a large number of studies focused on the assessment of PhA, which made it difficult to reach a comprehensive conclusion on this topic. We found 11 studies evaluating the PhA value obtained from BIA in patients with LC. In order to choose the right cancer treatment and plan carefully, survival prediction is crucial. In any case, new tools are necessary to be applied in daily clinical practice.

In the latest years, a growing body of studies have evaluated the prognostic role of PhA not only in patients with LC, but also in patients with respiratory disease. Indeed, De Benedetto et al. reported that lower PhA is associated with a decreased muscle mass, muscle strength and exercise capacity in patients with idiopathic pulmonary fibrosis, regardless of body weight. Moreover, patients with Chronic Obstructive Pulmonary Disease (COPD) and lower PhA have reduced cell mass, evident skeletal muscle depletion, worsening gas exchange and an increased risk of all-cause mortality [ 46 ]. Similarly, a lower PhA has been associated with an increased risk of malnutrition, sarcopenia, fluid retention, systemic inflammation, symptoms, and poorer QoLin patients with cancer. Moreover, a lower PhA may be a novel prognostic factor of poorer overall survival and higher risk of postoperative complications in cancer patients [ 47 ].

Overall, lower PhA levels have been associated with poorer physical condition and shorter survival in patients with LC: in Sanchez-Lara et al. [ 38 ], Gupta et al. [ 30 , 31 , 32 , 34 , 48 ] and Toso et al. [ 42 ] LC patients with lower PhA had a shorter survival than patients with higher PhA. In addition, in Shi et al. [ 39 ] patients with a higher PhA had a better survival and PhA was an independent risk factor for mortality in men with LC. Navigante et al. [ 37 ] in cancer patients without edema, PhA values ≤ 3° were associated with mortality within three days of BIA analysis, while in sarcopenic patients the PhA value was reduced compared to non-sarcopenic patients.

Patients with a lower PhA also had a higher risk of complications after surgical procedures [ 36 ]. In the prospective observational study of Uccella et al., it was observed that PhA was an independent prognostic factor of optimal cytoreduction and postoperative complications among patients with primary diagnosis of advanced ovarian cancer [ 49 ]. Similarly, in the prospective observational study of Inci et al. (The Risk-Gin trial) the authors observed that patients undergoing surgery for gynecological cancer with PhA < 4.75° and HGS < 44 kg in both hands had a three-fold increased risk of 30 days severe postoperative complications [ 50 ].

Thus, in addition to being a marker of cellular function, muscle mass and nutritional status, PhA may be a predictive factor of acute catastrophic complications risk. Interestingly, PhA was weakly but significantly associated with other prognostic variables, suggesting that it captures some additional information compared to existing prognostic factors. Further studies are needed to examine physiological and cellular changes associated with PhA.

Gupta et al. have evaluated role of PhA in the prognosis of 52 patients with advanced colorectal cancer: patients with PhA ≤ 5.7º had an 8-months average survival rate (Kaplan–Meier method), while those with PhA > 5.57º had a 40-month average survival rate [ 32 ]. Bosy-Westphal et al. [ 51 ] showed that patients with PhA < the fifth percentile had a deterioration in nutritional and functional status, decreased QoL and increased morbidity and mortality. The fifth percentile was a clinically relevant indicator of cancer cachexia. In this context, Hui et al. [ 35 ]investigated the association between PhA and survival in individuals with terminal cancer, where the increment of 1 degree in PhA was associated with higher survival rates.

In the prospective observational study by Paiva et al. PhA is reported as an independent prognostic factor, and Norman et al. [ 52 ]showed that in cancer patients, PhA values (stratified by age, sex, and BMI) below the fifth percentile of reference corresponded to a significant deterioration in nutritional status. In addition, these patients showed decreased HGS, increased incidence of weight loss, dyspnoea, fatigue, and increased risk of mortality. In three articles positive correlation has emerged between HGS and PhA, so weakness is related to the reduction in PhA in cancer patients. In other patient’s populations, it was shown that impaired muscle strength was associated with a poorer prognosis [ 53 , 54 ]. In addition, correlations between HGS, PhA and other BIA variables have emerged in adolescents and young adults [ 55 ].

In Navigante et al. [ 37 ], univariate logistic regression analysis showed that higher PhA values represent a protective factor for sarcopenia. In Pérez-Camargo et al. [ 56 ] palliative care patients with cancer (the most frequently were gastric cancer, gynecological cancer, LC, and haematological malignancies) and severe sarcopenia had a lower mean PhA (3.9°) compared to patients without sarcopenia (mean PhA was 4.1°) showing that PhA is an independent measurement that can be associated to detect sarcopenia. Moreover, authors found that sarcopenic patients had a shorter overall survival and an increased risk of death compared to patients without sarcopenia. A recent review [ 57 ] indicated that PhA and sarcopenia are related to LC prognosis through different mechanisms including inflammation and oxidative stress. Detection of sarcopenia and the evaluation of BC and PhA can be a valuable tool for identifying and timely intervention of the state of malnutrition of the cancer patient. Timely analysis of patient’s nutritional status is essential, as it allows to avoid significant loss of cell mass and lean mass, but ensures a proper nutritional approach in order to avoid aggravation of the general condition of the patient. Moreover, to obtain an accurate clinical interpretation of PhA, simultaneous assessment of hydration and BCM status is required [ 58 ]. These informations could be derived from the analysis of vector length on the R/Xc plot using Bioelectrical Impedance Vectorial Analysis (BIVA) (a scatterplot that represents R in X-axis and Xc in Y-axis divided by height in meters). In healthy subjects, a balance is observed between BCM and hydration status of FFM: malnutrition, sarcopenia and cachexia can lead to a loss of BCM and cell membrane surface area, provoking cell damage and a reduction in PhA [ 58 ].

According to the American Cancer Society (ACS), one of the first hallmarks of several types of cancer, including LC, is unexplained weight loss. In addition, treatments such as RT and CHT could generate side effects that cause inappetence to patients (e.g., mouth ulcers, nausea, vomiting). Tumour growth leading to extreme loss of appetite and weight in association with systemic signs of inflammation may be breeding grounds for cachexia [ 59 ]. Cachexia leads to a considerable loss of skeletal muscle mass (SMM) that cannot be completely compensated with traditional nutritional supports. Furthermore, it may be an underlying condition in patients with sarcopenia. We know that malnutrition and cachexia are often present in cancer patients [ 13 ]: this state can worsen the effects of anticancer therapies, with premature discontinuation of treatment, patient's QoL decreased and higher risk of mortality [ 60 , 61 ]. Nutritional status in cancer patients, especially the elderly, should be evaluated before and during CHT [ 61 ]. However, evaluation and detection of malnutrition status is not simple and instantaneous in patients. Therefore, it is preferable and useful to use the different variables defined by the BIA, to evaluate the changes in cellular membrane and body water levels. Future research on the use of PhA in clinical practice will be valuable in establishing cut-off values to better categorize oncology patients [ 58 ].

Limitations

To our knowledge, this is the most recent review focusing on the assessment of BIA-derived PhA in LC patients, however, we had to consider several limitations. Although several search engines outside PubMed were used, the review included only 11 studies of which only 8 were aimed at analysing PhA exclusively on patients with LC. Only four studies related to this topic were published in 2022, the other articles we included in our work were published between 2000 and 2013, so we do not have a large number of recent evidence.

Finally, accurate data on PhA relationships and other variables such as anthropometric values, HGS test, ECM/BCM ratio, tumour stage, tumour volume, etc., are absolutely necessary for a broader understanding of how PhA can be beneficial for this population. No information about PhA timing measurement by bioimpedance were provided in selected studies Monitoring the health of these patients is very important in order to be able to act promptly with targeted integrated therapies involving nutrition and physical activity, so further studies are needed. Patients with LC are at greater risk of malnutrition, sarcopenia and cachexia, with implications for physical function and overall QoL. Therefore, before making an assessment of the BC and PhA, it would be useful to consider the stage of the disease, cancer therapies carried out and the side effects, comorbidities and possible states of inflammation. All measurement should be performed in standard conditions and, in these specific cases, in days that do not correspond with cancer therapy. Patients with chronic respiratory disease could experience dysfunction in skeletal muscle mass (SMM) and BC changes as consequence of smoking, alcohol abuse, systemic inflammation, systemic oxidative stress, hormonal deficiencies, comorbidities, aging, and inappropriate diet [ 46 , 47 ]. Moreover, fat-free mass depletion and decreased muscle strength are common features of these patients. In this context, usefulness of PhA as a health status marker was investigated by a growing body of study. Similarly, patients with cancer patients have multiple comorbidities (chronic kidney disease) which may impact BC and cellular function. Moreover, patients with cancer frequently have fluid retention including edema, ascites, and pleural effusions. These factors may complicate the interpretation of PhA because this parameter is affected by altered ECW/ICW, or fluid disruption. Therefore, given the few studies currently available and the high number of factors that can affect BC measurements (hydration status, concomitant intake of food, alcohol use, physical activity, menstrual cycle, use of specific drugs that increase cell retention and catabolism, etc.) it should maintain accurate standardization in measurement.

Conclusions

The evaluation of PhA by BIA analysis in LC patients is not widely discussed in scientific literature. However, early identification in nutritional status changes in cancer patients represents a crucial aspect to improve patient’s quality of life both post-diagnosis and during and after anticancer therapies, avoiding possible states of malnutrition and sarcopenia, which can aggravate patient’s status. This systematic review shows an association between a very low PhA and an increased risk of a deficent physical condition, linked to reduced survival in lung cancer patients. In the selected studies, various cut-offs point for PhA have been reported that need to be interpreted with caution: to date, it is not possible to define a single threshold or cut off point for PhA due to technical differences in commercial BIA devices (single-, multiple-frequency and BIS). Given the high incidence of this cancer and the low number of studies on this issue, it would be important and necessary to make greater use of this screening method in clinical practice.

Availability of data and materials

All the data analysed for this review are present in the tables; there are no additional files.

Abbreviations

American Cancer Society

Appendicular Skeletal Muscle Mass

• Body composition

Body Cellular Mass

Bioelectrical Impedance Analysis

Bioelectrical Impedance Vectorial Analysis

Body Mass Index

Calf Circumference

Chemotherapy

Chronic Obstructive Pulmonary Disease

Extra-cellular Mass

Extra-cellular Water

Fat-Free Mass

Fat-Free Mass index

Hand-grip strength

Health Related Quality of Life

Immune checkpoint inhibitor

Intra-Cellular Water

Karnofski Performance Status

Lung Cancer

Mediterranean Dietary Score

Multiple-frequency Bioelectrical Impedance Analysis

Nutritional Risk Score-2002

Non-small cell lung carcinoma

Overall Survival

Phase Angle

Patient-Generated Subjective Global Assessment

Prognostic Nutritional Index

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Quality of Life Questionnaire – Cancer 30

Quality of Life

Radiotherapy

Small cell lung carcinoma

Single-frequency Bioelectrical Impedance Analysis

Skeletal Muscle Mass Index

Skeletal Muscle Mass

Total Lymphocyte Count

Total Body Water

Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71(3):209–49.

Article   PubMed   Google Scholar  

I Numeri del Cancro in Italia 2022 [ https://www.aiom.it/i-numeri-del-cancro-in-italia/ ].

Zhang P, Chen PL, Li ZH, Zhang A, Zhang XR, Zhang YJ, Liu D, Mao C. Association of smoking and polygenic risk with the incidence of lung cancer: a prospective cohort study. Br J Cancer. 2022;126(11):1637–46.

Article   PubMed   PubMed Central   Google Scholar  

Kim CH, Lee YC, Hung RJ, McNallan SR, Cote ML, Lim WY, Chang SC, Kim JH, Ugolini D, Chen Y, et al. Exposure to secondhand tobacco smoke and lung cancer by histological type: a pooled analysis of the International Lung Cancer Consortium (ILCCO). Int J Cancer. 2014;135(8):1918–30.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Luo J, Hendryx M, Ducatman A. Association between six environmental chemicals and lung cancer incidence in the United States. J Environ Public Health. 2011;2011: 463701.

Zheng M. Classification and Pathology of Lung Cancer. Surg Oncol Clin N Am. 2016;25(3):447–68.

Mele MC, Rinninella E, Cintoni M, Pulcini G, Di Donato A, Grassi F, Trestini I, Pozzo C, Tortora G, Gasbarrini A, et al. Nutritional support in lung cancer patients: the state of the art. Clin Lung Cancer. 2021;22(4):e584–94.

Beckles MA, Spiro SG, Colice GL, Rudd RM. Initial evaluation of the patient with lung cancer: symptoms, signs, laboratory tests, and paraneoplastic syndromes. Chest. 2003;123(1 Suppl):97S–104S.

Polanski J, Jankowska-Polanska B, Rosinczuk J, Chabowski M, Szymanska-Chabowska A. Quality of life of patients with lung cancer. Onco Targets Ther. 2016;9:1023–8.

PubMed   PubMed Central   Google Scholar  

Arends J, Baracos V, Bertz H, Bozzetti F, Calder PC, Deutz NEP, Erickson N, Laviano A, Lisanti MP, Lobo DN, et al. ESPEN expert group recommendations for action against cancer-related malnutrition. Clin Nutr. 2017;36(5):1187–96.

Article   CAS   PubMed   Google Scholar  

Porro C, La Torre ME, Tartaglia N, Benameur T, Santini M, Ambrosi A, Messina G, Cibelli G, Fiorelli A, Polito R, et al. The potential role of nutrition in lung cancer establishment and progression. Life (Basel). 2022;12(2):270.

CAS   PubMed   Google Scholar  

Pilikidou M, Palyvou F, Papadopoulou SK, Tsiouda T, Tsekitsidi E, Arvaniti K, Miziou A, Tsingerlioti Z, Apostolidis G, Ntiloudis R, et al. Lung cancer, treatment and nutritional status. Mol Clin Oncol. 2021;15(6):248.

Dewys WD, Begg C, Lavin PT, Band PR, Bennett JM, Bertino JR, Cohen MH, Douglass HO, Jr., Engstrom PF, Ezdinli EZ et al.: Prognostic effect of weight loss prior to chemotherapy in cancer patients. Eastern Cooperative Oncology Group. Am J Med 1980, 69(4):491–497.

Ravasco P, Monteiro-Grillo I, Vidal PM, Camilo ME. Nutritional deterioration in cancer: the role of disease and diet. Clin Oncol (R Coll Radiol). 2003;15(8):443–50.

Prado CM, Lieffers JR, Bowthorpe L, Baracos VE, Mourtzakis M, McCargar LJ. Sarcopenia and physical function in overweight patients with advanced cancer. Can J Diet Pract Res. 2013;74(2):69–74.

Fortunati N, Manti R, Birocco N, Pugliese M, Brignardello E, Ciuffreda L, Catalano MG, Aragno M, Boccuzzi G. Pro-inflammatory cytokines and oxidative stress/antioxidant parameters characterize the bio-humoral profile of early cachexia in lung cancer patients. Oncol Rep. 2007;18(6):1521–7.

Duguet A, Bachmann P, Lallemand Y, Blanc-Vincent MP. Good clinical practice in nutritional management in cancer patients: malnutrition and nutritional assessment. Bull Cancer. 1999;86(12):997–1016.

Polanski J, Chabowski M, Swiatoniowska-Lonc N, Dudek K, Jankowska-Polanska B, Zabierowski J, Mazur G: Relationship between Nutritional Status and Clinical Outcome in Patients Treated for Lung Cancer. Nutrients 2021;13(10):3332.

Lin TY, Chen YF, Wu WT, Han DS, Tsai IC, Chang KV, Ozcakar L. Impact of sarcopenia on the prognosis and treatment of lung cancer: an umbrella review. Discov Oncol. 2022;13(1):115.

Bapuji SB, Sawatzky JA. Understanding weight loss in patients with colorectal cancer: a human response to illness. Oncol Nurs Forum. 2010;37(3):303–10.

Argiles JM, Lopez-Soriano FJ, Busquets S. Mechanisms and treatment of cancer cachexia. Nutr Metab Cardiovasc Dis. 2013;23(Suppl 1):S19–24.

Wang J, Tan S, Gianotti L, Wu G. Evaluation and management of body composition changes in cancer patients. Nutrition. 2023;114:112132.

Yamada Y, Buehring B, Krueger D, Anderson RM, Schoeller DA, Binkley N. Electrical Properties Assessed by Bioelectrical Impedance Spectroscopy as Biomarkers of Age-related Loss of Skeletal Muscle Quantity and Quality. J Gerontol A Biol Sci Med Sci. 2017;72(9):1180–6.

PubMed   Google Scholar  

Barbosa-Silva MC, Barros AJ. Bioelectrical impedance analysis in clinical practice: a new perspective on its use beyond body composition equations. Curr Opin Clin Nutr Metab Care. 2005;8(3):311–7.

Almada-Correia I, Neves PM, Makitie A, Ravasco P. Body Composition Evaluation in Head and Neck Cancer Patients: A Review. Front Oncol. 2019;9:1112.

Branco MG, Mateus C, Capelas ML, Pimenta N, Santos T, Makitie A, Ganhao-Arranhado S, Trabulo C, Ravasco P: Bioelectrical Impedance Analysis (BIA) for the Assessment of Body Composition in Oncology: A Scoping Review. Nutrients 2023, 15(22):792.

Article   Google Scholar  

Caccialanza R, Cotogni P, Cereda E, Bossi P, Aprile G, Delrio P, Gnagnarella P, Mascheroni A, Monge T, Corradi E, et al. Nutritional support in cancer patients: update of the Italian Intersociety Working Group practical recommendations. J Cancer. 2022;13(9):2705–16.

Shintani Y, Ikeda N, Matsumoto T, Kadota Y, Okumura M, Ohno Y, Ohta M. Nutritional status of patients undergoing chemoradiotherapy for lung cancer. Asian Cardiovasc Thorac Ann. 2012;20(2):172–6.

Marin Caro MM, Laviano A, Pichard C. Nutritional intervention and quality of life in adult oncology patients. Clin Nutr. 2007;26(3):289–301.

Gupta D, Lis CG, Dahlk SL, Vashi PG, Grutsch JF, Lammersfeld CA. Bioelectrical impedance phase angle as a prognostic indicator in advanced pancreatic cancer. Br J Nutr. 2004;92(6):957–62.

Gupta D, Lammersfeld CA, Vashi PG, King J, Dahlk SL, Grutsch JF, Lis CG. Bioelectrical impedance phase angle as a prognostic indicator in breast cancer. BMC Cancer. 2008;8:249.

Gupta D, Lammersfeld CA, Burrows JL, Dahlk SL, Vashi PG, Grutsch JF, Hoffman S, Lis CG. Bioelectrical impedance phase angle in clinical practice: implications for prognosis in advanced colorectal cancer. Am J Clin Nutr. 2004;80(6):1634–8.

Grundmann O, Yoon SL, Williams JJ. The value of bioelectrical impedance analysis and phase angle in the evaluation of malnutrition and quality of life in cancer patients–a comprehensive review. Eur J Clin Nutr. 2015;69(12):1290–7.

Gupta D, Lammersfeld CA, Vashi PG, King J, Dahlk SL, Grutsch JF, Lis CG. Bioelectrical impedance phase angle in clinical practice: implications for prognosis in stage IIIB and IV non-small cell lung cancer. BMC Cancer. 2009;9:37.

Hui D, Bansal S, Morgado M, Dev R, Chisholm G, Bruera E. Phase angle for prognostication of survival in patients with advanced cancer: preliminary findings. Cancer. 2014;120(14):2207–14.

Ji W, Liu X, Zheng K, Yang H, Cui J, Li W. Correlation of phase angle with sarcopenia and its diagnostic value in elderly men with cancer. Nutrition. 2021;84:111110.

Navigante A, Morgado PC, Casbarien O, Delgado NL, Giglio R, Perman M. Relationship between weakness and phase angle in advanced cancer patients with fatigue. Support Care Cancer. 2013;21(6):1685–90.

Sanchez-Lara K, Turcott JG, Juarez E, Guevara P, Nunez-Valencia C, Onate-Ocana LF, Flores D, Arrieta O. Association of nutrition parameters including bioelectrical impedance and systemic inflammatory response with quality of life and prognosis in patients with advanced non-small-cell lung cancer: a prospective study. Nutr Cancer. 2012;64(4):526–34.

Shi J, Xie H, Ruan G, Ge Y, Lin S, Zhang H, Zheng X, Liu C, Song M, Liu T, et al. Sex differences in the association of phase angle and lung cancer mortality. Front Nutr. 2022;9:1061996.

Detopoulou P, Tsiouda T, Pilikidou M, Palyvou F, Mantzorou M, Perzirkianidou P, Kyrka K, Methenitis S, Kondyli FS, Voulgaridou G, et al. Dietary habits are related to phase angle in male patients with non-small-cell lung cancer. Curr Oncol. 2022;29(11):8074–83.

Detopoulou P, Tsiouda T, Pilikidou M, Palyvou F, Tsekitsidi E, Mantzorou M, Pezirkianidou P, Kyrka K, Methenitis S, Voulgaridou G et al: Changes in Body Weight, Body Composition, Phase Angle, and Resting Metabolic Rate in Male Patients with Stage IV Non-Small-Cell Lung Cancer Undergoing Therapy. Medicina (Kaunas) 2022;58(12):1779.

Toso S, Piccoli A, Gusella M, Menon D, Bononi A, Crepaldi G, Ferrazzi E. Altered tissue electric properties in lung cancer patients as detected by bioelectric impedance vector analysis. Nutrition. 2000;16(2):120–4.

Suzuki Y, Kushimoto Y, Ishizawa H, Kawai H, Ito A, Matsuda Y, Hoshikawa Y. The phase angle as a predictor of postoperative complications in patients undergoing lung cancer surgery. Surg Today. 2023;53(3):332–7.

Castanho IA, Lopes AJ, Koury JC, Tessarollo B, Silva AC, Nunes RA. Relationship between the phase angle and volume of tumours in patients with lung cancer. Ann Nutr Metab. 2013;62(1):68–74.

Hui D, Moore J, Park M, Liu D, Bruera E. Phase Angle and the Diagnosis of Impending Death in Patients with Advanced Cancer: Preliminary Findings. Oncologist. 2019;24(6):e365–73.

De Benedetto F, Marinari S, De Blasio F. Phase angle in assessment and monitoring treatment of individuals with respiratory disease. Rev Endocr Metab Disord. 2023;24(3):491–502.

Amano K, Bruera E, Hui D. Diagnostic and prognostic utility of phase angle in patients with cancer. Rev Endocr Metab Disord. 2023;24(3):479–89.

Naughton MJ, Weaver KE. Physical and mental health among cancer survivors: considerations for long-term care and quality of life. N C Med J. 2014;75(4):283–6.

Uccella S, Mele MC, Quagliozzi L, Rinninella E, Nero C, Cappuccio S, Cintoni M, Gasbarrini A, Scambia G, Fagotti A. Assessment of preoperative nutritional status using BIA-derived phase angle (PhA) in patients with advanced ovarian cancer: Correlation with the extent of cytoreduction and complications. Gynecol Oncol. 2018;149(2):263–9.

Inci MG, Richter R, Woopen H, Rasch J, Heise K, Anders L, Mueller K, Nasser S, Siepmann T, Sehouli J. Role of predictive markers for severe postoperative complications in gynecological cancer surgery: a prospective study (RISC-Gyn Trial). Int J Gynecol Cancer. 2020;30(12):1975–82.

Bosy-Westphal A, Danielzik S, Dorhofer RP, Later W, Wiese S, Muller MJ. Phase angle from bioelectrical impedance analysis: population reference values by age, sex, and body mass index. JPEN J Parenter Enteral Nutr. 2006;30(4):309–16.

Norman K, Stobaus N, Pirlich M, Bosy-Westphal A. Bioelectrical phase angle and impedance vector analysis–clinical relevance and applicability of impedance parameters. Clin Nutr. 2012;31(6):854–61.

Hodgev VA, Kostianev SS. Maximal inspiratory pressure predicts mortality in patients with chronic obstructive pulmonary disease in a five-year follow-up. Folia Med (Plovdiv). 2006;48(3–4):36–41.

Cooper R, Kuh D, Hardy R, Mortality Review G. Falcon, Teams HAS: Objectively measured physical capability levels and mortality: systematic review and meta-analysis. BMJ. 2010;341: c4467.

Sacco AM, Valerio G, Alicante P, Di Gregorio A, Spera R, Ballarin G, Scalfi L. Raw bioelectrical impedance analysis variables (phase angle and impedance ratio) are significant predictors of hand grip strength in adolescents and young adults. Nutrition. 2021;91–92: 111445.

Google Scholar  

Perez Camargo DA, Allende Perez SR, Verastegui Aviles E, Rivera Franco MM, Meneses Garcia A, Herrera Gomez A, Urbalejo Ceniceros VI. Assessment and impact of phase angle and sarcopenia in palliative cancer patients. Nutr Cancer. 2017;69(8):1227–33.

Detopoulou P, Voulgaridou G, Papadopoulou S. Cancer, Phase Angle and Sarcopenia: The Role of Diet in Connection with Lung Cancer Prognosis. Lung. 2022;200(3):347–79.

Bellido D, Garcia-Garcia C, Talluri A, Lukaski HC, Garcia-Almeida JM. Future lines of research on phase angle: Strengths and limitations. Rev Endocr Metab Disord. 2023;24(3):563–83.

Blauwhoff-Buskermolen S, Versteeg KS, de van der Schueren MA, den Braver NR, Berkhof J, Langius JA, Verheul HM. Loss of Muscle Mass During Chemotherapy Is Predictive for Poor Survival of Patients With Metastatic Colorectal Cancer. J Clin Oncol 2016;34(12):1339-1344.

Aapro M, Arends J, Bozzetti F, Fearon K, Grunberg SM, Herrstedt J, Hopkinson J, Jacquelin-Ravel N, Jatoi A, Kaasa S, et al. Early recognition of malnutrition and cachexia in the cancer patient: a position paper of a European School of Oncology Task Force. Ann Oncol. 2014;25(8):1492–9.

Caillet P, Liuu E, Raynaud Simon A, Bonnefoy M, Guerin O, Berrut G, Lesourd B, Jeandel C, Ferry M, Rolland Y, et al. Association between cachexia, chemotherapy and outcomes in older cancer patients: A systematic review. Clin Nutr. 2017;36(6):1473–82.

Download references

This research was funded by Ministry of Health (5X1000_2021_2).

Author information

Authors and affiliations.

Division of Radiotherapy, Istituto Nazionale Tumori IRCCS Fondazione G. Pascale, Naples, 80131, Italy

Melania Prete

Department of Medical, Movement Sciences and Wellbeing, University of Naples “Parthenope”, Naples, 80133, Italy

Giada Ballarin

Epidemiology and Biostatistics Unit, Istituto Nazionale Tumori, IRCCS Fondazione G. Pascale, Naples, 80131, Italy

Giuseppe Porciello, Assunta Luongo & Egidio Celentano

Department of Public Health, Federico II University Hospital, Via Pansini 5, Naples, 80131, Italy

Aniello Arianna & Luca Scalfi

Scientific Direction, Istituto Nazionale Tumori, IRCCS Fondazione G. Pascale, Naples, 80131, Italy

Valentina Belli

You can also search for this author in PubMed   Google Scholar

Contributions

M.P., G.B. and L.S. have proposed, designed and edited data analysis of this systematic review; M.P. and G.P. wrote the main manuscript text and prepared Fig.  1 and Tables 1 and 2 . All work was supervised by E.C and L.S. All authors have read and revised the manuscript.

Corresponding author

Correspondence to Giuseppe Porciello .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Prete, M., Ballarin, G., Porciello, G. et al. Bioelectrical impedance analysis—derived phase angle (PhA) in lung cancer patients: a systematic review. BMC Cancer 24 , 608 (2024). https://doi.org/10.1186/s12885-024-12378-4

Download citation

Received : 21 December 2023

Accepted : 13 May 2024

Published : 20 May 2024

DOI : https://doi.org/10.1186/s12885-024-12378-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Lung cancer
• Phase angle

ISSN: 1471-2407

• Submission enquiries: [email protected]
• General enquiries: [email protected]