what are the different types of literature review

Literature Review: Types of literature reviews

• Traditional or narrative literature reviews
• Scoping Reviews
• Systematic literature reviews
• Annotated bibliography
• Keeping up to date with literature
• Finding a thesis
• Evaluating sources and critical appraisal of literature
• Managing and analysing your literature
• Further reading and resources

Types of literature reviews

The type of literature review you write will depend on your discipline and whether you are a researcher writing your PhD, publishing a study in a journal or completing an assessment task in your undergraduate study.

A literature review for a subject in an undergraduate degree will not be as comprehensive as the literature review required for a PhD thesis.

An undergraduate literature review may be in the form of an annotated bibliography or a narrative review of a small selection of literature, for example ten relevant articles. If you are asked to write a literature review, and you are an undergraduate student, be guided by your subject coordinator or lecturer.

The common types of literature reviews will be explained in the pages of this section.

• Narrative or traditional literature reviews
• Critically Appraised Topic (CAT)
• Scoping reviews
• Annotated bibliographies

These are not the only types of reviews of literature that can be conducted. Often the term "review" and "literature" can be confusing and used in the wrong context. Grant and Booth (2009) attempt to clear up this confusion by discussing 14 review types and the associated methodology, and advantages and disadvantages associated with each review.

Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26 , 91–108. doi:10.1111/j.1471-1842.2009.00848.x

What's the difference between reviews?

Researchers, academics, and librarians all use various terms to describe different types of literature reviews, and there is often inconsistency in the ways the types are discussed. Here are a couple of simple explanations.

• The image below describes common review types in terms of speed, detail, risk of bias, and comprehensiveness:

"Schematic of the main differences between the types of literature review" by Brennan, M. L., Arlt, S. P., Belshaw, Z., Buckley, L., Corah, L., Doit, H., Fajt, V. R., Grindlay, D., Moberly, H. K., Morrow, L. D., Stavisky, J., & White, C. (2020). Critically Appraised Topics (CATs) in veterinary medicine: Applying evidence in clinical practice. Frontiers in Veterinary Science, 7 , 314. https://doi.org/10.3389/fvets.2020.00314 is licensed under CC BY 3.0

• The table below lists four of the most common types of review , as adapted from a widely used typology of fourteen types of reviews (Grant & Booth, 2009).  

Grant, M.J. & Booth, A. (2009).  A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26 (2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

See also the Library's  Literature Review guide.

Critical Appraised Topic (CAT)

For information on conducting a Critically Appraised Topic or CAT

Callander, J., Anstey, A. V., Ingram, J. R., Limpens, J., Flohr, C., & Spuls, P. I. (2017).  How to write a Critically Appraised Topic: evidence to underpin routine clinical practice.  British Journal of Dermatology (1951), 177(4), 1007-1013. https://doi.org/10.1111/bjd.15873 

Books on Literature Reviews

• << Previous: Home
• Next: Traditional or narrative literature reviews >>
• Last Updated: May 12, 2024 12:18 PM
• URL: https://libguides.csu.edu.au/review

Charles Sturt University is an Australian University, TEQSA Provider Identification: PRV12018. CRICOS Provider: 00005F.

• Locations and Hours
• UCLA Library
• Research Guides
• Biomedical Library Guides

Systematic Reviews

• Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

• Planning Your Systematic Review
• Database Searching
• Creating the Search
• Search Filters and Hedges
• Grey Literature
• Managing and Appraising Results
• Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Home
• Next: Planning Your Systematic Review >>
• Last Updated: Apr 17, 2024 2:02 PM
• URL: https://guides.library.ucla.edu/systematicreviews

Literature Reviews

• Types of reviews
• Getting started

Types of reviews and examples

Choosing a review type.

• 1. Define your research question
• 2. Plan your search
• 3. Search the literature
• 4. Organize your results
• 5. Synthesize your findings
• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

• Meta-analysis
• Systematized

Definition:

"A term used to describe a conventional overview of the literature, particularly when contrasted with a systematic review (Booth et al., 2012, p. 265).

Characteristics:

• Provides examination of recent or current literature on a wide range of subjects
• Varying levels of completeness / comprehensiveness, non-standardized methodology
• May or may not include comprehensive searching, quality assessment or critical appraisal

Mitchell, L. E., & Zajchowski, C. A. (2022). The history of air quality in Utah: A narrative review.  Sustainability ,  14 (15), 9653.  doi.org/10.3390/su14159653

Booth, A., Papaioannou, D., & Sutton, A. (2012). Systematic approaches to a successful literature review. London: SAGE Publications Ltd.

"An assessment of what is already known about a policy or practice issue...using systematic review methods to search and critically appraise existing research" (Grant & Booth, 2009, p. 100).

• Assessment of what is already known about an issue
• Similar to a systematic review but within a time-constrained setting
• Typically employs methodological shortcuts, increasing risk of introducing bias, includes basic level of quality assessment
• Best suited for issues needing quick decisions and solutions (i.e., policy recommendations)

Learn more about the method:

Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: the evolution of a rapid review approach.  Systematic reviews, 1 (1), 1-9.  https://doi.org/10.1186/2046-4053-1-10

Virginia Commonwealth University Libraries. (2021). Rapid Review Protocol .

Quarmby, S., Santos, G., & Mathias, M. (2019). Air quality strategies and technologies: A rapid review of the international evidence.  Sustainability, 11 (10), 2757.  https://doi.org/10.3390/su11102757

Grant, M.J. & Booth, A. (2009). A typology of reviews: an analysis of the 14 review types and associated methodologies.  Health Information & Libraries Journal , 26(2), 91-108. https://www.doi.org/10.1111/j.1471-1842.2009.00848.x

Developed and refined by the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), this review "map[s] out and categorize[s] existing literature on a particular topic, identifying gaps in research literature from which to commission further reviews and/or primary research" (Grant & Booth, 2009, p. 97).

Although mapping reviews are sometimes called scoping reviews, the key difference is that mapping reviews focus on a review question, rather than a topic

Mapping reviews are "best used where a clear target for a more focused evidence product has not yet been identified" (Booth, 2016, p. 14)

Mapping review searches are often quick and are intended to provide a broad overview

Mapping reviews can take different approaches in what types of literature is focused on in the search

Cooper I. D. (2016). What is a "mapping study?".  Journal of the Medical Library Association: JMLA ,  104 (1), 76–78. https://doi.org/10.3163/1536-5050.104.1.013

Miake-Lye, I. M., Hempel, S., Shanman, R., & Shekelle, P. G. (2016). What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products.  Systematic reviews, 5 (1), 1-21.  https://doi.org/10.1186/s13643-016-0204-x

Tainio, M., Andersen, Z. J., Nieuwenhuijsen, M. J., Hu, L., De Nazelle, A., An, R., ... & de Sá, T. H. (2021). Air pollution, physical activity and health: A mapping review of the evidence.  Environment international ,  147 , 105954.  https://doi.org/10.1016/j.envint.2020.105954

Booth, A. (2016). EVIDENT Guidance for Reviewing the Evidence: a compendium of methodological literature and websites . ResearchGate. https://doi.org/10.13140/RG.2.1.1562.9842 . 

Grant, M.J. & Booth, A. (2009). A typology of reviews: an analysis of the 14 review types and associated methodologies.  Health Information & Libraries Journal , 26(2), 91-108.  https://www.doi.org/10.1111/j.1471-1842.2009.00848.x

"A type of review that has as its primary objective the identification of the size and quality of research in a topic area in order to inform subsequent review" (Booth et al., 2012, p. 269).

• Main purpose is to map out and categorize existing literature, identify gaps in literature—great for informing policy-making
• Search comprehensiveness determined by time/scope constraints, could take longer than a systematic review
• No formal quality assessment or critical appraisal

Learn more about the methods :

Arksey, H., & O'Malley, L. (2005) Scoping studies: towards a methodological framework.  International Journal of Social Research Methodology ,  8 (1), 19-32.  https://doi.org/10.1080/1364557032000119616

Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science: IS, 5, 69. https://doi.org/10.1186/1748-5908-5-69

Example : 

Rahman, A., Sarkar, A., Yadav, O. P., Achari, G., & Slobodnik, J. (2021). Potential human health risks due to environmental exposure to nano-and microplastics and knowledge gaps: A scoping review.  Science of the Total Environment, 757 , 143872.  https://doi.org/10.1016/j.scitotenv.2020.143872

A review that "[compiles] evidence from multiple...reviews into one accessible and usable document" (Grant & Booth, 2009, p. 103). While originally intended to be a compilation of Cochrane reviews, it now generally refers to any kind of evidence synthesis.

• Compiles evidence from multiple reviews into one document
• Often defines a broader question than is typical of a traditional systematic review

Choi, G. J., & Kang, H. (2022). The umbrella review: a useful strategy in the rain of evidence.  The Korean Journal of Pain ,  35 (2), 127–128.  https://doi.org/10.3344/kjp.2022.35.2.127

Aromataris, E., Fernandez, R., Godfrey, C. M., Holly, C., Khalil, H., & Tungpunkom, P. (2015). Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare , 13(3), 132–140. https://doi.org/10.1097/XEB.0000000000000055

Rojas-Rueda, D., Morales-Zamora, E., Alsufyani, W. A., Herbst, C. H., Al Balawi, S. M., Alsukait, R., & Alomran, M. (2021). Environmental risk factors and health: An umbrella review of meta-analyses.  International Journal of Environmental Research and Public Dealth ,  18 (2), 704.  https://doi.org/10.3390/ijerph18020704

A meta-analysis is a "technique that statistically combines the results of quantitative studies to provide a more precise effect of the result" (Grant & Booth, 2009, p. 98).

• Statistical technique for combining results of quantitative studies to provide more precise effect of results
• Aims for exhaustive, comprehensive searching
• Quality assessment may determine inclusion/exclusion criteria
• May be conducted independently or as part of a systematic review

Berman, N. G., & Parker, R. A. (2002). Meta-analysis: Neither quick nor easy. BMC Medical Research Methodology , 2(1), 10. https://doi.org/10.1186/1471-2288-2-10

Hites R. A. (2004). Polybrominated diphenyl ethers in the environment and in people: a meta-analysis of concentrations.  Environmental Science & Technology ,  38 (4), 945–956.  https://doi.org/10.1021/es035082g

A systematic review "seeks to systematically search for, appraise, and [synthesize] research evidence, often adhering to the guidelines on the conduct of a review" provided by discipline-specific organizations, such as the Cochrane Collaboration (Grant & Booth, 2009, p. 102).

• Aims to compile and synthesize all known knowledge on a given topic
• Adheres to strict guidelines, protocols, and frameworks
• Time-intensive and often takes months to a year or more to complete
• The most commonly referred to type of evidence synthesis. Sometimes confused as a blanket term for other types of reviews

Gascon, M., Triguero-Mas, M., Martínez, D., Dadvand, P., Forns, J., Plasència, A., & Nieuwenhuijsen, M. J. (2015). Mental health benefits of long-term exposure to residential green and blue spaces: a systematic review.  International Journal of Environmental Research and Public Health ,  12 (4), 4354–4379.  https://doi.org/10.3390/ijerph120404354

"Systematized reviews attempt to include one or more elements of the systematic review process while stopping short of claiming that the resultant output is a systematic review" (Grant & Booth, 2009, p. 102). When a systematic review approach is adapted to produce a more manageable scope, while still retaining the rigor of a systematic review such as risk of bias assessment and the use of a protocol, this is often referred to as a  structured review  (Huelin et al., 2015).

• Typically conducted by postgraduate or graduate students
• Often assigned by instructors to students who don't have the resources to conduct a full systematic review

Salvo, G., Lashewicz, B. M., Doyle-Baker, P. K., & McCormack, G. R. (2018). Neighbourhood built environment influences on physical activity among adults: A systematized review of qualitative evidence.  International Journal of Environmental Research and Public Health ,  15 (5), 897.  https://doi.org/10.3390/ijerph15050897

Huelin, R., Iheanacho, I., Payne, K., & Sandman, K. (2015). What’s in a name? Systematic and non-systematic literature reviews, and why the distinction matters. https://www.evidera.com/resource/whats-in-a-name-systematic-and-non-systematic-literature-reviews-and-why-the-distinction-matters/

• Review Decision Tree - Cornell University For more information, check out Cornell's review methodology decision tree.
• LitR-Ex.com - Eight literature review methodologies Learn more about 8 different review types (incl. Systematic Reviews and Scoping Reviews) with practical tips about strengths and weaknesses of different methods.
• << Previous: Getting started
• Next: 1. Define your research question >>
• Last Updated: May 8, 2024 8:12 AM
• URL: https://guides.library.duke.edu/lit-reviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

Systematic Reviews: Types of literature review, methods, & resources

• Types of literature review, methods, & resources
• Protocol and registration
• Search strategy
• Medical Literature Databases to search
• Study selection and appraisal
• Data Extraction/Coding/Study characteristics/Results
• Reporting the quality/risk of bias
• Manage citations using RefWorks This link opens in a new window
• GW Box file storage for PDF's This link opens in a new window

Analytical reviews

GUIDELINES FOR HOW TO CARRY OUT AN ANALYTICAL REVIEW OF QUANTITATIVE RESEARCH

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Randomised trials, Systematic reviews, Study protocols, Diagnostic/prognostic studies, Case reports, Clinical practice guidelines, Animal pre-clinical studies, etc). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

When comparing therapies :

PRISMA (Guideline on how to perform and write-up a systematic review and/or meta-analysis of the outcomes reported in multiple clinical trials of therapeutic interventions. PRISMA  replaces the previous QUORUM statement guidelines ):  Liberati, A,, Altman, D,, Moher, D, et al. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.  Plos Medicine, 6 (7):e1000100. doi:10.1371/journal.pmed.1000100 

When comparing diagnostic methods :

Checklist for Artificial Intelligence in Medical Imaging (CLAIM). CLAIM is modeled after the STARD guideline and has been extended to address applications of AI in medical imaging that include classification, image reconstruction, text analysis, and workflow optimization. The elements described here should be viewed as a “best practice” to guide authors in presenting their research. Reported in Mongan, J., Moy, L., & Kahn, C. E., Jr (2020). Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers.  Radiology. Artificial intelligence ,  2 (2), e200029. https://doi.org/10.1148/ryai.2020200029

STAndards for the Reporting of Diagnostic accuracy studies (STARD) Statement. (Reporting guidelines for writing up a study comparing the accuracy of competing diagnostic methods)  http://www.stard-statement.org/

When evaluating clinical practice guidelines :

AGREE Research Trust (ART) (2013).  Appraisal of Guidelines for Research & Evaluation (AGREE-II) . (A 23-item instrument for as sessing th e quality of Clinical Practice Guidelines. Used internationally for evaluating or deciding which guidelines could be recommended for use in practice or to inform health policy decisions.)

National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Instrument (2019). (A 15-item instrument using scales of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trustworthy guidelines. It has good external validity among guideline developers and good interrater reliability across trained reviewers.)

When reviewing genetics studies

Human genetics review reporting guidelines.  Little J, Higgins JPT (eds.). The HuGENet™ HuGE Review Handbook, version 1.0 . 

When you need to re-analyze individual participant data

If you wish to collect, check, and re-analyze individual participant data (IPD) from clinical trials addressing a particular research question, you should follow the  PRISMA-IPD  guidelines as reported in  Stewart, L.A., Clarke, M., Rovers, M., et al. (2015). Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement. JAMA, 313(16):1657-1665. doi:10.1001/jama.2015.3656 .

When comparing Randomized studies involving animals, livestock, or food:

O’Connor AM, et al. (2010).  The REFLECT statement: methods and processes of creating reporting guidelines for randomized controlled trials for livestock and food safety by modifying the CONSORT statement.  Zoonoses Public Health. 57(2):95-104. Epub 2010/01/15. doi: 10.1111/j.1863-2378.2009.01311.x. PubMed PMID: 20070653.

Sargeant JM, et al. (2010).  The REFLECT Statement: Reporting Guidelines for Randomized Controlled Trials in Livestock and Food Safety: Explanation and Elaboration.  Zoonoses Public Health. 57(2):105-36. Epub 2010/01/15. doi: JVB1312 [pii] 10.1111/j.1863-2378.2009.01312.x. PubMed PMID: 20070652.

GUIDELINES FOR HOW TO WRITE UP FOR PUBLICATION THE RESULTS OF ONE QUANTITATIVE CLINICAL TRIAL

When reporting the results of a Randomized Controlled Trial :

Consolidated Standards of Reporting Trials (CONSORT) Statement. (2010 reporting guideline for writing up a Randomized Controlled Clinical Trial).  http://www.consort-statement.org . Since updated in 2022, see Butcher, M. A., et al. (2022). Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension . JAMA : the Journal of the American Medical Association, 328(22), 2252–2264. https://doi.org/10.1001/jama.2022.21022

Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010). Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biology, 8(6), e1000412–e1000412. https://doi.org/10.1371/journal.pbio.1000412 (A 20-item checklist, following the CONSORT approach, listing the information that published articles reporting research using animals should include, such as the number and specific characteristics of animals used; details of housing and husbandry; and the experimental, statistical, and analytical methods used to reduce bias.)

Narrative reviews

GUIDELINES  FOR HOW TO CARRY OUT  A  NARRATIVE REVIEW / QUALITATIVE RESEARCH /  OBSERVATIONAL STUDIES

Campbell, M. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ, 368. doi: https://doi.org/10.1136/bmj.l6890  (guideline on how to analyse evidence for a narrative review, to provide a recommendation based on heterogenous study types).

Community Preventive Services Task Force (2021).  The Methods Manual for Community Guide Systematic Reviews . (Public Health Prevention systematic review guidelines)

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Observational studies, Qualitative research, Quality improvement studies, and Economic evaluations). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from  https://methods.cochrane.org/qi/training-resources . (Training materials for how to do a meta-synthesis, or qualitative evidence synthesis). 

Cornell University Library (2019). Planning worksheet for structured literature reviews. Retrieved 4/8/22 from  https://osf.io/tnfm7/  (offers a framework for a narrative literature review).

Green, B. N., Johnson, C. D., & Adams, A. (2006).  Writing narrative literature reviews for peer-reviewed journals: secrets of the trade . Journal of Chiropractic Medicine, 5(3): 101-117. DOI: 10.1016/ S0899-3467 (07)60142-6.  This is a very good article about what to take into consideration when writing any type of narrative review.

When reviewing observational studies/qualitative research :

STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement. (Reporting guidelines for various types of health sciences observational studies).  http://www.strobe-statement.org 

Meta-analysis of Observational Studies in Epidemiology (MOOSE)  http://jama.jamanetwork.com/article.aspx?articleid=192614

RATS Qualitative research systematic review guidelines.  https://www.equator-network.org/reporting-guidelines/qualitative-research-review-guidelines-rats/

Methods/Guidance

Right Review , this decision support website provides an algorithm to help reviewers choose a review methodology from among 41 knowledge synthesis methods.

The Systematic Review Toolbox , an online catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Maintained by the UK University of York Health Economics Consortium, Newcastle University NIHR Innovation Observatory, and University of Sheffield School of Health and Related Research.

Institute of Medicine. (2011).  Finding What Works in Health Care: Standards for Systematic Reviews . Washington, DC: National Academies  (Systematic review guidelines from the Health and Medicine Division (HMD) of the U.S. National Academies of Sciences, Engineering, and Medicine (formerly called the Institute of Medicine)).

International Committee of Medical Journal Editors (2022).  Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals . Guidance on how to prepare a manuscript for submission to a Medical journal.

Cochrane Handbook of Systematic Reviews of Interventions (International Cochrane Collaboration systematic review guidelines). The various Cochrane review groups comporise around 30,000 physicians around the world working in the disciplines on reviews of interventions with very detailed methods for verifying the validity of the research methods and analysis performed in screened-in Randmized Controlled Clinical Trials. Typically published Cochrane Reviews are the most exhaustive review of the evidence of effectiveness of a particular drug or intervention, and include a statistical meta-analysis. Similar to practice guidelines, Cochrane reviews are periodically revised and updated.

Joanna Briggs Institute (JBI) Manual of Evidence Synthesis . (International systematic review guidelines). Based at the University of Adelaide, South Australia, and collaborating with around 80 academic and medical entities around the world. Unlike Cochrane Reviews that strictly focus on efficacy of interventions, JBI offers a broader, inclusive approach to evidence, to accommodate a range of diverse questions and study designs. The JBI manual provides guidance on how to analyse and include both quantitative and qualitative research.

Cochrane Methods Support Unit, webinar recordings on methodological support questions 

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from https://methods.cochrane.org/qi/training-resources . (How to do a meta-synthesis, or qualitative evidence synthesis). 

Center for Reviews and Dissemination (University of York, England) (2009).  Systematic Reviews: CRD's guidance for undertaking systematic reviews in health care . (British systematic review guidelines). 

Agency for Health Research & Quality (AHRQ) (2013). Methods guide for effectiveness and comparative effectiveness reviews . (U.S. comparative effectiveness review guidelines)

Hunter, K. E., et al. (2022). Searching clinical trials registers: guide for systematic reviewers.  BMJ (Clinical research ed.) ,  377 , e068791. https://doi.org/10.1136/bmj-2021-068791

Patient-Centered Outcomes Research Institute (PCORI).  The PCORI Methodology Report . (A 47-item methodology checklist for U.S. patient-centered outcomes research. Established under the Patient Protection and Affordable Care Act, PCORI funds the development of guidance on the comparative effectivess of clinical healthcare, similar to the UK National Institute for Clinical Evidence but without reporting cost-effectiveness QALY metrics). 

Canadian Agency for Drugs and Technologies in Health (CADTH) (2019). Grey Matters: a practical tool for searching health-related grey literature. Retrieved from https://www.cadth.ca/resources/finding-evidence/grey-matters . A checklist of N American & international online databases and websites you can use to search for unpublished reports, posters, and policy briefs, on topics including general medicine and nursing, public and mental health, health technology assessment, drug and device regulatory, approvals, warnings, and advisories.

Hempel, S., Xenakis, L., & Danz, M. (2016). Systematic Reviews for Occupational Safety and Health Questions: Resources for Evidence Synthesis. Retrieved 8/15/16 from http://www.rand.org/pubs/research_reports/RR1463.html . NIOSH guidelines for how to carry out a systematic review in the occupational safety and health domain.

A good source for reporting guidelines is the  NLM's  Research Reporting Guidelines and Initiatives .

Grading of Recommendations Assessment, Development and Evaluation (GRADE). (An international group of academics/clinicians working to promote a common approach to grading the quality of evidence and strength of recommendations.) 

Phillips, B., Ball, C., Sackett, D., et al. (2009). Oxford Centre for Evidence Based Medicine: Levels of Evidence. Retrieved 3/20/17 from https://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf . (Another commonly used criteria for grading the quality of evidence and strength of recommendations, developed in part by EBM guru David Sackett.) 

Systematic Reviews for Animals & Food  (guidelines including the REFLECT statement for carrying out a systematic review on animal health, animal welfare, food safety, livestock, and agriculture)

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26(2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x. (Describes 14 different types of literature and systematic review, useful for thinking at the outset about what sort of literature review you want to do.)

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements . Health information and libraries journal, 36(3), 202–222. doi:10.1111/hir.12276  (An updated look at different types of literature review, expands on the Grant & Booth 2009 article listed above).

Garrard, J. (2007).  Health Sciences Literature Review Made Easy: The Matrix Method  (2nd Ed.).   Sudbury, MA:  Jones & Bartlett Publishers. (Textbook of health sciences literature search methods).

Zilberberg, M. (2012).  Between the lines: Finding the truth in medical literature . Goshen, MA: Evimed Research Press. (Concise book on foundational concepts of evidence-based medicine).

Lang, T. (2009). The Value of Systematic Reviews as Research Activities in Medical Education . In: Lang, T. How to write, publish, & present in the health sciences : a guide for clinicians & laboratory researchers. Philadelphia : American College of Physicians.  (This book chapter has a helpful bibliography on systematic review and meta-analysis methods)

Brown, S., Martin, E., Garcia, T., Winter, M., García, A., Brown, A., Cuevas H.,  & Sumlin, L. (2013). Managing complex research datasets using electronic tools: a meta-analysis exemplar . Computers, Informatics, Nursing: CIN, 31(6), 257-265. doi:10.1097/NXN.0b013e318295e69c. (This article advocates for the programming of electronic fillable forms in Adobe Acrobat Pro to feed data into Excel or SPSS for analysis, and to use cloud based file sharing systems such as Blackboard, RefWorks, or EverNote to facilitate sharing knowledge about the decision-making process and keep data secure. Of particular note are the flowchart describing this process, and their example screening form used for the initial screening of abstracts).

Brown, S., Upchurch, S., & Acton, G. (2003). A framework for developing a coding scheme for meta-analysis . Western Journal Of Nursing Research, 25(2), 205-222. (This article describes the process of how to design a coded data extraction form and codebook, Table 1 is an example of a coded data extraction form that can then be used to program a fillable form in Adobe Acrobat or Microsoft Access).

Elamin, M. B., Flynn, D. N., Bassler, D., Briel, M., Alonso-Coello, P., Karanicolas, P., & ... Montori, V. M. (2009). Choice of data extraction tools for systematic reviews depends on resources and review complexity .  Journal Of Clinical Epidemiology ,  62 (5), 506-510. doi:10.1016/j.jclinepi.2008.10.016  (This article offers advice on how to decide what tools to use to extract data for analytical systematic reviews).

Riegelman R.   Studying a Study and Testing a Test: Reading Evidence-based Health Research , 6th Edition.  Lippincott Williams & Wilkins, 2012. (Textbook of quantitative statistical methods used in health sciences research).

Rathbone, J., Hoffmann, T., & Glasziou, P. (2015). Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers. Systematic Reviews, 480. doi:10.1186/s13643-015-0067-6

Guyatt, G., Rennie, D., Meade, M., & Cook, D. (2015). Users' guides to the medical literature (3rd ed.). New York: McGraw-Hill Education Medical.  (This is a foundational textbook on evidence-based medicine and of particular use to the reviewer who wants to learn about the different types of published research article e.g. "what is a case report?" and to understand what types of study design best answer what types of clinical question).

Glanville, J., Duffy, S., Mccool, R., & Varley, D. (2014). Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? Journal of the Medical Library Association : JMLA, 102(3), 177–183. https://doi.org/10.3163/1536-5050.102.3.007

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan a web and mobile app for systematic reviews.  Systematic Reviews, 5 : 210, DOI: 10.1186/s13643-016-0384-4. http://rdcu.be/nzDM

Kwon Y, Lemieux M, McTavish J, Wathen N. (2015). Identifying and removing duplicate records from systematic review searches. J Med Libr Assoc. 103 (4): 184-8. doi: 10.3163/1536-5050.103.4.004. https://www.ncbi.nlm.nih.gov/pubmed/26512216

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. (2016). De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. 104 (3):240-3. doi: 10.3163/1536-5050.104.3.014. Erratum in: J Med Libr Assoc. 2017 Jan;105(1):111. https://www.ncbi.nlm.nih.gov/pubmed/27366130

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–46. doi: 10.1016/j.jclinepi.2016.01.021 . PRESS is a guideline with a checklist for librarians to critically appraise the search strategy for a systematic review literature search.

Clark, JM, Sanders, S, Carter, M, Honeyman, D, Cleo, G, Auld, Y, Booth, D, Condron, P, Dalais, C, Bateup, S, Linthwaite, B, May, N, Munn, J, Ramsay, L, Rickett, K, Rutter, C, Smith, A, Sondergeld, P, Wallin, M, Jones, M & Beller, E 2020, 'Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial',  Journal of the Medical Library Association , vol. 108, no. 2, pp. 195-207.

Journal articles describing systematic review methods can be searched for in PubMed using this search string in the PubMed search box: sysrev_methods [sb] . 

Software tools for systematic reviews

• Covidence GW in 2019 has bought a subscription to this Cloud based tool for facilitating screening decisions, used by the Cochrane Collaboration. Register for an account.
• NVIVO for analysis of qualitative research NVIVO is used for coding interview data to identify common themes emerging from interviews with several participants. GW faculty, staff, and students may download NVIVO software.
• RedCAP RedCAP is software that can be used to create survey forms for research or data collection or data extraction. It has very detailed functionality to enable data exchange with Electronic Health Record Systems, and to integrate with study workflow such as scheduling follow up reminders for study participants.
• SRDR tool from AHRQ Free, web-based and has a training environment, tutorials, and example templates of systematic review data extraction forms
• RevMan 5 RevMan 5 is the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews
• Rayyan Free, web-based tool for collecting and screening citations. It has options to screen with multiple people, masking each other.
• GradePro Free, web application to create, manage and share summaries of research evidence (called Evidence Profiles and Summary of Findings Tables) for reviews or guidelines, uses the GRADE criteria to evaluate each paper under review.
• DistillerSR Needs subscription. Create coded data extraction forms from templates.
• EPPI Reviewer Needs subscription. Like DistillerSR, tool for text mining, data clustering, classification and term extraction
• SUMARI Needs subscription. Qualitative data analysis.
• Dedoose Needs subscription. Qualitative data analysis, similar to NVIVO in that it can be used to code interview transcripts, identify word co-occurence, cloud based.
• Meta-analysis software for statistical analysis of data for quantitative reviews SPSS, SAS, and STATA are popular analytical statistical software that include macros for carrying out meta-analysis. Himmelfarb has SPSS on some 3rd floor computers, and GW affiliates may download SAS to your own laptop from the Division of IT website. To perform mathematical analysis of big data sets there are statistical analysis software libraries in the R programming language available through GitHub and RStudio, but this requires advanced knowledge of the R and Python computer languages and data wrangling/cleaning.
• PRISMA 2020 flow diagram generator The PRISMA Statement website has a page listing example flow diagram templates and a link to software for creating PRISMA 2020 flow diagrams using R software.

GW researchers may want to consider using Refworks to manage citations, and GW Box to store the full text PDF's of review articles. You can also use online survey forms such as Qualtrics, RedCAP, or Survey Monkey, to design and create your own coded fillable forms, and export the data to Excel or one of the qualitative analytical software tools listed above.

Forest Plot Generators

• RevMan 5 the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews.
• Meta-Essentials a free set of workbooks designed for Microsoft Excel that, based on your input, automatically produce meta-analyses including Forest Plots. Produced for Erasmus University Rotterdam joint research institute.
• Neyeloff, Fuchs & Moreira Another set of Excel worksheets and instructions to generate a Forest Plot. Published as Neyeloff, J.L., Fuchs, S.C. & Moreira, L.B. Meta-analyses and Forest plots using a microsoft excel spreadsheet: step-by-step guide focusing on descriptive data analysis. BMC Res Notes 5, 52 (2012). https://doi-org.proxygw.wrlc.org/10.1186/1756-0500-5-52
• For R programmers instructions are at https://cran.r-project.org/web/packages/forestplot/vignettes/forestplot.html and you can download the R code package from https://github.com/gforge/forestplot
• << Previous: Home
• Next: Protocol and registration >>

• Last Updated: May 8, 2024 11:07 AM
• URL: https://guides.himmelfarb.gwu.edu/systematic_review

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
• Himmelfarb Health Sciences Library
• 2300 Eye St., NW, Washington, DC 20037
• Phone: (202) 994-2850
• [email protected]
• https://himmelfarb.gwu.edu

• Chester Fritz Library
• Library of the Health Sciences
• Thormodsgard Law Library

Literature Reviews

• Get started

Literature Reviews within a Scholarly Work

Literature reviews as a scholarly work.

• Finding Literature Reviews
• Your Literature Search
• Library Books
• How to Videos
• Communicating & Citing Research
• Bibliography

Literature reviews summarize and analyze what has been written on a particular topic and identify gaps or disagreements in the scholarly work on that topic.

Within a scholarly work, the literature review situates the current work within the larger scholarly conversation and emphasizes how that particular scholarly work contributes to the conversation on the topic. The literature review portion may be as brief as a few paragraphs focusing on a narrow topic area.

When writing this type of literature review, it's helpful to start by identifying sources most relevant to your research question. A citation tracking database such as Web of Science can also help you locate seminal articles on a topic and find out who has more recently cited them. See "Your Literature Search" for more details.

A literature review may itself be a scholarly publication and provide an analysis of what has been written on a particular topic without contributing original research. These types of literature reviews can serve to help keep people updated on a field as well as helping scholars choose a research topic to fill gaps in the knowledge on that topic. Common types include:

Systematic Review

Systematic literature reviews follow specific procedures in some ways similar to setting up an experiment to ensure that future scholars can replicate the same steps. They are also helpful for evaluating data published over multiple studies. Thus, these are common in the medical field and may be used by healthcare providers to help guide diagnosis and treatment decisions. Cochrane Reviews are one example of this type of literature review.

Semi-Systematic Review

When a systematic review is not feasible, a semi-systematic review can help synthesize research on a topic or how a topic has been studied in different fields (Snyder 2019). Rather than focusing on quantitative data, this review type identifies themes, theoretical perspectives, and other qualitative information related to the topic. These types of reviews can be particularly helpful for a historical topic overview, for developing a theoretical model, and for creating a research agenda for a field (Snyder 2019). As with systematic reviews, a search strategy must be developed before conducting the review.

Integrative Review

An integrative review is less systematic and can be helpful for developing a theoretical model or to reconceptualize a topic. As Synder (2019) notes, " This type of review often re quires a more creative collection of data, as the purpose is usually not to cover all articles ever published on the topic but rather to combine perspectives and insights from di ff erent fi elds or research traditions" (p. 336).

Source: Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research. 104. 333-339. doi: 10.1016/j.jbusres.2019.07.039

• << Previous: Get started
• Next: Finding Literature Reviews >>
• Last Updated: Dec 5, 2023 8:31 PM
• URL: https://libguides.und.edu/literature-reviews

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 15, 2024 9:53 AM
• URL: https://libguides.usc.edu/writingguide

How to Write a Literature Review

What is a literature review.

• What Is the Literature
• Writing the Review

A literature review is much more than an annotated bibliography or a list of separate reviews of articles and books. It is a critical, analytical summary and synthesis of the current knowledge of a topic. Thus it should compare and relate different theories, findings, etc, rather than just summarize them individually. In addition, it should have a particular focus or theme to organize the review. It does not have to be an exhaustive account of everything published on the topic, but it should discuss all the significant academic literature and other relevant sources important for that focus.

This is meant to be a general guide to writing a literature review: ways to structure one, what to include, how it supplements other research. For more specific help on writing a review, and especially for help on finding the literature to review, sign up for a Personal Research Session .

The specific organization of a literature review depends on the type and purpose of the review, as well as on the specific field or topic being reviewed. But in general, it is a relatively brief but thorough exploration of past and current work on a topic. Rather than a chronological listing of previous work, though, literature reviews are usually organized thematically, such as different theoretical approaches, methodologies, or specific issues or concepts involved in the topic. A thematic organization makes it much easier to examine contrasting perspectives, theoretical approaches, methodologies, findings, etc, and to analyze the strengths and weaknesses of, and point out any gaps in, previous research. And this is the heart of what a literature review is about. A literature review may offer new interpretations, theoretical approaches, or other ideas; if it is part of a research proposal or report it should demonstrate the relationship of the proposed or reported research to others' work; but whatever else it does, it must provide a critical overview of the current state of research efforts. 

Literature reviews are common and very important in the sciences and social sciences. They are less common and have a less important role in the humanities, but they do have a place, especially stand-alone reviews.

Types of Literature Reviews

There are different types of literature reviews, and different purposes for writing a review, but the most common are:

• Stand-alone literature review articles . These provide an overview and analysis of the current state of research on a topic or question. The goal is to evaluate and compare previous research on a topic to provide an analysis of what is currently known, and also to reveal controversies, weaknesses, and gaps in current work, thus pointing to directions for future research. You can find examples published in any number of academic journals, but there is a series of Annual Reviews of *Subject* which are specifically devoted to literature review articles. Writing a stand-alone review is often an effective way to get a good handle on a topic and to develop ideas for your own research program. For example, contrasting theoretical approaches or conflicting interpretations of findings can be the basis of your research project: can you find evidence supporting one interpretation against another, or can you propose an alternative interpretation that overcomes their limitations?
• Part of a research proposal . This could be a proposal for a PhD dissertation, a senior thesis, or a class project. It could also be a submission for a grant. The literature review, by pointing out the current issues and questions concerning a topic, is a crucial part of demonstrating how your proposed research will contribute to the field, and thus of convincing your thesis committee to allow you to pursue the topic of your interest or a funding agency to pay for your research efforts.
• Part of a research report . When you finish your research and write your thesis or paper to present your findings, it should include a literature review to provide the context to which your work is a contribution. Your report, in addition to detailing the methods, results, etc. of your research, should show how your work relates to others' work.

A literature review for a research report is often a revision of the review for a research proposal, which can be a revision of a stand-alone review. Each revision should be a fairly extensive revision. With the increased knowledge of and experience in the topic as you proceed, your understanding of the topic will increase. Thus, you will be in a better position to analyze and critique the literature. In addition, your focus will change as you proceed in your research. Some areas of the literature you initially reviewed will be marginal or irrelevant for your eventual research, and you will need to explore other areas more thoroughly. 

Examples of Literature Reviews

See the series of Annual Reviews of *Subject* which are specifically devoted to literature review articles to find many examples of stand-alone literature reviews in the biomedical, physical, and social sciences. 

Research report articles vary in how they are organized, but a common general structure is to have sections such as:

• Abstract - Brief summary of the contents of the article
• Introduction - A explanation of the purpose of the study, a statement of the research question(s) the study intends to address
• Literature review - A critical assessment of the work done so far on this topic, to show how the current study relates to what has already been done
• Methods - How the study was carried out (e.g. instruments or equipment, procedures, methods to gather and analyze data)
• Results - What was found in the course of the study
• Discussion - What do the results mean
• Conclusion - State the conclusions and implications of the results, and discuss how it relates to the work reviewed in the literature review; also, point to directions for further work in the area

Here are some articles that illustrate variations on this theme. There is no need to read the entire articles (unless the contents interest you); just quickly browse through to see the sections, and see how each section is introduced and what is contained in them.

The Determinants of Undergraduate Grade Point Average: The Relative Importance of Family Background, High School Resources, and Peer Group Effects , in The Journal of Human Resources , v. 34 no. 2 (Spring 1999), p. 268-293.

This article has a standard breakdown of sections:

• Introduction
• Literature Review
• Some discussion sections

First Encounters of the Bureaucratic Kind: Early Freshman Experiences with a Campus Bureaucracy , in The Journal of Higher Education , v. 67 no. 6 (Nov-Dec 1996), p. 660-691.

This one does not have a section specifically labeled as a "literature review" or "review of the literature," but the first few sections cite a long list of other sources discussing previous research in the area before the authors present their own study they are reporting.

• Next: What Is the Literature >>
• Last Updated: Jan 11, 2024 9:48 AM
• URL: https://libguides.wesleyan.edu/litreview

Types of Literature Review

There are many types of literature review. The choice of a specific type depends on your research approach and design. The following types of literature review are the most popular in business studies:

Narrative literature review , also referred to as traditional literature review, critiques literature and summarizes the body of a literature. Narrative review also draws conclusions about the topic and identifies gaps or inconsistencies in a body of knowledge. You need to have a sufficiently focused research question to conduct a narrative literature review

Systematic literature review requires more rigorous and well-defined approach compared to most other types of literature review. Systematic literature review is comprehensive and details the timeframe within which the literature was selected. Systematic literature review can be divided into two categories: meta-analysis and meta-synthesis.

When you conduct meta-analysis you take findings from several studies on the same subject and analyze these using standardized statistical procedures. In meta-analysis patterns and relationships are detected and conclusions are drawn. Meta-analysis is associated with deductive research approach.

Meta-synthesis, on the other hand, is based on non-statistical techniques. This technique integrates, evaluates and interprets findings of multiple qualitative research studies. Meta-synthesis literature review is conducted usually when following inductive research approach.

Scoping literature review , as implied by its name is used to identify the scope or coverage of a body of literature on a given topic. It has been noted that “scoping reviews are useful for examining emerging evidence when it is still unclear what other, more specific questions can be posed and valuably addressed by a more precise systematic review.” [1] The main difference between systematic and scoping types of literature review is that, systematic literature review is conducted to find answer to more specific research questions, whereas scoping literature review is conducted to explore more general research question.

Argumentative literature review , as the name implies, examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. It should be noted that a potential for bias is a major shortcoming associated with argumentative literature review.

Integrative literature review reviews , critiques, and synthesizes secondary data about research topic in an integrated way such that new frameworks and perspectives on the topic are generated. If your research does not involve primary data collection and data analysis, then using integrative literature review will be your only option.

Theoretical literature review focuses on a pool of theory that has accumulated in regard to an issue, concept, theory, phenomena. Theoretical literature reviews play an instrumental role in establishing what theories already exist, the relationships between them, to what degree existing theories have been investigated, and to develop new hypotheses to be tested.

At the earlier parts of the literature review chapter, you need to specify the type of your literature review your chose and justify your choice. Your choice of a specific type of literature review should be based upon your research area, research problem and research methods.  Also, you can briefly discuss other most popular types of literature review mentioned above, to illustrate your awareness of them.

[1] Munn, A. et. al. (2018) “Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach” BMC Medical Research Methodology

  John Dudovskiy

Types of Literature Reviews : Home

• Health Science Information Consortium
• University Health Network - New
• Types of Literature Reviews

Need More Help?

• Knowledge Synthesis Support
• Literature Search Request
• The Right Review for You - Workshop Recording (YouTube) 27min video From UHN Libraries Recorded Nov 2021
• The Screening Phase for Reviews Tutorial This tutorial presents information on the screening process for systematic reviews or other knowledge syntheses, and contains a variety of resources for successfully preparing to complete this important research stage.
• Workshops Find more UHN Libraries workshops, live and on-demand, and other learning opportunities helpful for knowledge synthesis projects.

How to Choose Your Review Method

TREAD* Lightly and Consider...

• Available T ime for conducting your review
• Any R esource constraints within which you must deliver your review
• Any requirements for specialist E xpertise in order to complete the review
• The requirements of the A udience for your review and its intended purpose
• The richness, thickness and availability of D ata within included studies

* Booth A, Sutton A, Papaioannou D.  Systematic approaches to a successful literature review.  2nd edition.  Los Angeles, CA:  Sage, 2016.  (p.36)

How do I write a Review Protocol?

• What is a Protocol? (UofT)
• Guidance on Registering a Review with PROSPERO

Writing Resources

• Advice on Academic Writing (University of Toronto)
• How to write a great research paper using reporting guidelines (EQUATOR Network)
• Instructions to Authors in the Health Sciences
• Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals (ICMJE)
• Writing resources guide (BMC)

What is a Literature Review?

A literature review provides an overview of what's been written about a specific topic. It is a generic term. There are many different types of literature reviews which can cover a wide range of subjects at various levels of completeness and comprehensiveness. Choosing the type of review you wish to conduct will depend on the purpose of your review, and the time and resources you have available.

This page will provide definitions of some of the most common review types in the health sciences and links to relevant reporting guidelines or methodological papers.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal . 2009 Jun 1;26(2):91-108. 

• Summary of Five Types of Reviews Table summarizing the characteristics, guidelines etc. of 5 common types of review article.

Traditional (Narrative) Review

Traditional (narrative) literature reviews provide a broad overview of a research topic with no clear methodological approach. Information is collected and interpreted unsystematically with subjective summaries of findings. Authors aim to describe and discuss the literature from a contextual or theoretical point of view. Although the reviews may be conducted by topic experts, due to preconceived ideas or conclusions, they could be subject to bias. This sort of literature review can be appropriate if you have a broad topic area, are working on your own, or have time constraints.

Agarwal S, Charlesworth M, Elrakhawy M. How to write a narrative review . Anaesthesia . 2023;78(9):1162-1166. doi:10.1111/anae.16016

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade . Journal of Chiropractic Medicine . 2006;5(3):101-117. doi:10.1016/S0899-3467(07)60142-6.

Ferrari R. Writing narrative style literature reviews . Medical Writing. 2015 Dec 1;24(4):230-5.

Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews ? European journal of clinical investigation . 2018;48:e12931.

Knowledge Synthesis

• What is Knowledge Synthesis?
• Learn more!

CIHR Definition of Knowledge Syntheses:

“The contextualization and integration of research findings of individual research studies within the larger body of knowledge on the topic. A synthesis must be reproducible and transparent in its methods, using quantitative and/or qualitative methods.”  - A Guide to Knowledge Synthesis, CIHR

Grimshaw J. A Guide to Knowledge Synthesis [Internet]. CIHR. Canadian Institutes of Health Research; 2010.

Canadian Institutes of Health Research. Synthesis Resources [Internet]. CIHR. Canadian Institutes of Health Research; 2013.

Booth A, Noyes J, Flemming K, Gerhardus A, Wahlster P, van der Wilt, Gert Jan, et al.  Structured methodology review identified seven (RETREAT) criteria for selecting qualitative evidence synthesis approaches . Journal of clinical epidemiology. 2018;99:41-52.

Kastner M, Tricco AC, Soobiah C, et al. What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review . BMC Medical Research Methodology . 2012;12:114. doi:10.1186/1471-2288-12-114.

Kastner M, Antony J, Soobiah C, Straus SE, Tricco AC. Conceptual recommendations for selecting the most appropriate knowledge synthesis method to answer research questions related to complex evidence . Journal of Clinical Epidemiology . 2016;73:43-49.

Knowledge Synthesis Support at UHN

Common Types of Knowledge Syntheses

• Systematic Reviews
• Meta-Analysis
• Scoping Reviews
• Rapid or Restricted Reviews
• Clinical Practice Guidelines
• Realist Reviews
• Mixed Methods Reviews
• Qualitative Synthesis
• Narrative Synthesis

A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making. (See Section 1.2 in the Cochrane Handbook for Systematic Reviews of Interventions .)

A systematic review is not the same as a traditional (narrative) review or a literature review. Unlike other kinds of reviews, systematic reviews must be as thorough and unbiased as possible, and must also make explicit how the search was conducted. Systematic reviews may or may not include a meta-analysis.

On average, a systematic review project takes a year. If your timelines are shorter, you may wish to consider other types of synthesis projects or a traditional (narrative) review. See suggested timelines for a Cochrane Review for reference.

Systematic Review Overview (UHN)

Systematic Review Overview workshop recording (UHN)

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)

Greyson D, Rafferty E, Slater L, et al. Systematic review searches must be systematic, comprehensive, and transparent: a critique of Perman et al. BMC public health . 2019;19:153.

Ioannidis J. P. (2016). The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses . The Milbank quarterly , 94 (3), 485-514.

A subset of systematic reviews. Meta-analysis is a technique that statistically combines the results of quantitative studies to provide a more precise effect of the results.

"..a form of knowledge synthesis that addresses an exploratory research question aimed at mapping key concepts, types of evidence, and gaps in research related to a defined area or field by systematically searching, selecting and synthesizing existing knowledge." (Colquhoun, HL et al., 2014)

Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework .   International Journal of Social Research Methodology: Theory and Practice , 8 (1), 19-32. doi:10.1080/1364557032000119616.

Levac, D., Colquhoun, H. & O'Brien, K.K. Scoping studies: advancing the methodology . Implementation Sci 5 , 69 (2010). https://doi.org/10.1186/1748-5908-5-69

Colquhoun, H. L., Levac, D., O'Brien, K. K., Straus, S., Tricco, A. C., Perrier, L., . . . Moher, D. (2014). Scoping reviews: Time for clarity in definition, methods, and reporting . Journal of Clinical Epidemiology , 67(12), 1291-1294. doi:10.1016/j.jclinepi.2014.03.013.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews . Int.J.Evid Based.Healthc . 2015 Sep;13(3):141-146.

Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping Reviews (2020 version). In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis , JBI, 2020.

Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation . Ann Intern Med . [Epub ahead of print ] doi: 10.7326/M18-0850.

“…a type of knowledge synthesis in which systematic review processes are accelerated and methods are streamlined to complete the review more quickly than is the case for typical systematic reviews. Rapid reviews take an average of 5–12 weeks to complete, thus providing evidence within a shorter time frame required for some health policy and systems decisions.” (Tricco AC et al., 2017)

Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and implications of rapid reviews . Implementation Science : IS . 2010;5:56. doi:10.1186/1748-5908-5-56.

Langlois EV, Straus SE, Antony J, King VJ, Tricco AC. Using rapid reviews to strengthen health policy and systems and progress towards universal health coverage . BMJ Global Health . 2019;4:e001178.

Tricco AC, Langlois EV, Straus SE, editors. Rapid reviews to strengthen health policy and systems: a practical guide . Geneva: World Health Organization; 2017. Licence: CC BY-NC-SA 3.0 IGO.

Watt A, Cameron A, Sturm L, Lathlean T , Babidge W, Blamey S, et al. Rapid reviews versus full systematic reviews: An inventory of current methods and practice in health technology assessment . International Journal of Technology Assessment in Health Care . 2008;24(2):133-9.

“Clinical practice guidelines are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.” Source: Institute of Medicine. (1990). Clinical Practice Guidelines: Directions for a New Program, M.J. Field and K.N. Lohr (eds.) Washington, DC: National Academy Press. Page 38.

–Disclosure of any author conflicts of interest

AGREE Reporting Checklist

Alonso-Coello, P., Oxman, A. D., Moberg, J., Brignardello-Petersen, R., Akl, E. A., Davoli, M., ... & Guyatt, G. H. (2016). GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines . BMJ , 353 , i2089.

Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review - a new method of systematic review designed for complex policy interventions . Journal of Health Services Research & Policy . 2005;10:21-34.

Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: illustrating the method for implementation research . Implementation science : IS . 2012;7:33.

Wong G, Greenhalgh T, Westhorp G, Pawson R. Realist methods in medical education research: what are they and what can they contribute? Medical Education . 2012;46(1):89-96.

"Mixed-methods systematic reviews can be defined as combining the findings of qualitative and quantitative studies within a single systematic review to address the same overlapping or complementary review questions." (Harden A, 2010)

Harden A.   Mixed-Methods Systematic Reviews: Integrating quantitative and qualitative findings .   NCDDR:FOCUS. 2010.

Lizarondo L, Stern C, Apostolo J, et al. Five common pitfalls in mixed methods systematic reviews: lessons learned . J Clin Epidemiol . 2022;148:178-183. doi:10.1016/j.jclinepi.2022.03.014

Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews . Annual review of public health . 2014;35:29-45.

Pearson A, White H, Bath-Hextall F, Salmond S, Apostolo J, Kirkpatrick P. A mixed-methods approach to systematic reviews . International journal of evidence-based healthcare . 2015;13:121-131.

The Joanna Briggs Institute 2014 Reviewers Manual: Methodology for JBI Mixed Methods Systematic Reviews .

There are various methods for integrating the results from qualitative studies. "Systematic reviews of qualitative research have an important role in informing the delivery of evidence-based healthcare. Qualitative systematic reviews have investigated the culture of communities, exploring how consumers experience, perceive and manage their health and journey through the health system, and can evaluate components and activities of health services such as health promotion and community development." (Lockwood C et al., 2015)

Booth A, Noyes J, Flemming K, Gerhardus A, Wahlster P, van der Wilt, Gert Jan, et al. Structured methodology review identified seven (RETREAT) criteria for selecting qualitative evidence synthesis approaches . Journal of clinical epidemiology. 2018;99:41-52.

Ring N, Jepson R, Ritchie K. Methods of synthesizing qualitative research studies for health technology assessment . International Journal of Technology Assessment in Health Care . 2011;27:384-390.

Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation . International journal of evidence-based healthcare . 2015;13:179-187.

France EF, Cunningham M, Ring N, et al. Improving reporting of meta-ethnography: The eMERGe reporting guidance . Journal of advanced nursing . 2019.

Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review . BMC Med Res Methodol . 2009;9:59. Published 2009 Aug 11. doi:10.1186/1471-2288-9-59.

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews . BMC Med Res Methodol . 2008;8:45. Published 2008 Jul 10. doi:10.1186/1471-2288-8-45

Lewin S, Booth A, Glenton C, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series . Implementation science : IS . 2018;13:2.

"Narrative synthesis refers to an approach to the systematic review and synthesis of findings from multiple studies that relies primarily on the use of words and text to summarise and explain the findings of the synthesis. Whilst narrative synthesis can involve the manipulation of statistical data, the defining characteristic is that it adopts a textual approach to the process of synthesis to ‘tell the story’ of the findings from the included studies." (Popay J, 2006)

Tricco AC, Soobiah C, Antony J, et al. A scoping review identifies multiple emerging knowledge synthesis methods, but few studies operationalize the method . Journal of Clinical Epidemiology . 2016;73:19-28.

Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews . Lancaster: ESRC Research Methods Programme; 2006.

Snilstveit B, Oliver S, Vojtkova M. Narrative approaches to systematic review and synthesis of evidence for international development policy and practice . Journal of development effectiveness . 2012 Sep 1;4(3):409-29. 

Lucas PJ, Baird J, Arai L, Law C, Roberts HM. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews . BMC medical research methodology . 2007 Dec;7(1):4.

Ryan R. Cochrane Consumer sand Communication Review Group. Cochrane Consumers and Communication Review Group: data synthesis and analysis . June 2013.

• Last Updated: May 8, 2024 10:29 AM
• URL: https://guides.hsict.library.utoronto.ca/c.php?g=705263

We acknowledge this sacred land on which the University Health Network operates. For thousands of years it has been the traditional territory of the Huron-Wendat, the Haudenosaunee, and most recently, the Mississaugas of the Credit River. This territory was the subject of the Dish With One Spoon Wampum Belt Covenant, an agreement between the Haudenosaunee Confederacy and the Confederacy of the Ojibwe and allied nations to peaceably share and care for the resources around the Great Lakes. Today, the meeting place of Toronto is still the home to many Indigenous people from across Turtle Island and we are grateful to have the opportunity to work and learn on this territory

UHN Library and Information Services

University Libraries

• Ohio University Libraries
• Library Guides

Evidence-based Practice in Healthcare

• Performing a Literature Review
• EBP Tutorials
• Question- PICO
• Definitions
• Systematic Reviews
• Levels of Evidence
• Finding Evidence
• Filter by Study Type
• Too Much or Too Little?
• Critical Appraisal
• Quality Improvement (QI)
• Contact - Need Help?

Hanna's Performing a qualitity literature review presentation slides

• Link to the PPT slides via OneDrive anyone can view

Characteristics of a Good Literature Review in Health & Medicine

Clear Objectives and Research Questions : The review should start with clearly defined objectives and research questions that guide the scope and focus of the review.

Comprehensive Coverage : Include a wide range of relevant sources, such as research articles, review papers, clinical guidelines, and books. Aim for a broad understanding of the topic, covering historical developments and current advancements. To do this, an intentional and minimally biased search strategy.

• Link to relevant databases to consider for a comprehensive search (search 2+ databases)
• Link to the video "Searching your Topic: Strategies and Efficiencies" by Hanna Schmillen
• Link to the worksheet "From topic, to PICO, to search strategy" to help researchers work through their topic into an intentional search strategy by Hanna Schmillen

Transparency and Replicability : The review process, search strategy, should be transparent, with detailed documentation of all steps taken. This allows others to replicate the review or update it in the future.

Appraisal of Studies Included : Each included study should be critically appraised for methodological quality and relevance. Use standardized appraisal tools to assess the risk of bias and the quality of evidence.

• Link to the video " Evaluating Health Research" by Hanna Schmillen
• Link to evaluating and appraising studies tab, which includes a rubric and checklists

Clear Synthesis and Discussion of Findings : The review should provide a thorough discussion of the findings, including any patterns, relationships, or trends identified in the literature. Address the strengths and limitations of the reviewed studies and the review itself. Present findings in a balanced and unbiased manner, avoiding over interpretation or selective reporting of results.

Implications for Practice and Research : The review should highlight the practical implications of the findings for medical practice and policy. It should also identify gaps in the current literature and suggest areas for future research.

Referencing and Citation : Use proper citation practices to credit original sources. Provide a comprehensive reference list to guide readers to the original studies.

• Link to Citation Style Guide, includes tab about Zotero

Note: A literature review is not a systematic review. For more information about systematic reviews and different types of evidence synthesis projects, see the Evidence Synthesis guide .

• << Previous: Quality Improvement (QI)
• Next: Contact - Need Help? >>

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Literature Review: Types of Literature Reviews

• Literature Review
• Purpose of a Literature Review
• Work in Progress
• Compiling & Writing
• Books, Articles, & Web Pages

Types of Literature Reviews

• Departmental Differences
• Citation Styles & Plagiarism
• Know the Difference! Systematic Review vs. Literature Review

It is important to think of knowledge in a given field as consisting of three layers.

• First, there are the primary studies that researchers conduct and publish.
• Second, are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies.
• Third, there are the perceptions, conclusions, opinions, and interpretations that are shared informally that become part of the lore of the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomenon emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomenon. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147.

All content is from The Literature Review created by Dr. Robert Larabee USC

• << Previous: Books, Articles, & Web Pages
• Next: Departmental Differences >>
• Last Updated: Oct 19, 2023 12:07 PM
• URL: https://uscupstate.libguides.com/Literature_Review

• Duke NetID Login
• 919.660.1100
• Duke Health Badge: 24-hour access
• Accounts & Access
• Databases, Journals & Books
• Request & Reserve
• Training & Consulting
• Request Articles & Books
• Renew Online
• Reserve Spaces
• Reserve a Locker
• Study & Meeting Rooms
• Course Reserves
• Digital Health Device Collection
• Pay Fines/Fees
• Recommend a Purchase
• Access From Off Campus
• Building Access
• Computers & Equipment
• Wifi Access
• My Accounts
• Mobile Apps
• Known Access Issues
• Report an Access Issue
• All Databases
• Article Databases
• Basic Sciences
• Clinical Sciences
• Dissertations & Theses
• Drugs, Chemicals & Toxicology
• Grants & Funding
• Interprofessional Education
• Non-Medical Databases
• Search for E-Journals
• Search for Print & E-Journals
• Search for E-Books
• Search for Print & E-Books
• E-Book Collections
• Biostatistics
• Global Health
• MBS Program
• Medical Students
• MMCi Program
• Occupational Therapy
• Path Asst Program
• Physical Therapy
• Researchers
• Community Partners

Conducting Research

• Archival & Historical Research
• Black History at Duke Health
• Data Analytics & Viz Software
• Data: Find and Share
• Evidence-Based Practice
• NIH Public Access Policy Compliance
• Publication Metrics
• Qualitative Research
• Searching Animal Alternatives

Systematic Reviews

• Test Instruments

Using Databases

• JCR Impact Factors
• Web of Science

Finding & Accessing

• COVID-19: Core Clinical Resources
• Health Literacy
• Health Statistics & Data
• Library Orientation

Writing & Citing

• Creating Links
• Getting Published
• Reference Mgmt
• Scientific Writing

Meet a Librarian

• Request a Consultation
• Find Your Liaisons
• Register for a Class
• Request a Class
• Self-Paced Learning

Search Services

• Literature Search
• Systematic Review
• Animal Alternatives (IACUC)
• Research Impact

Citation Mgmt

• Other Software

Scholarly Communications

• About Scholarly Communications
• Publish Your Work
• Measure Your Research Impact
• Engage in Open Science
• Libraries and Publishers
• Directions & Maps
• Floor Plans

Library Updates

• Annual Snapshot
• Conference Presentations
• Contact Information
• Gifts & Donations
• What is a Systematic Review?

Types of Reviews

• Manuals and Reporting Guidelines
• Our Service
• 1. Assemble Your Team
• 2. Develop a Research Question
• 3. Write and Register a Protocol
• 4. Search the Evidence
• 5. Screen Results
• 6. Assess for Quality and Bias
• 7. Extract the Data
• 8. Write the Review
• Additional Resources
• Finding Full-Text Articles

Review Typologies

There are many types of evidence synthesis projects, including systematic reviews as well as others. The selection of review type is wholly dependent on the research question. Not all research questions are well-suited for systematic reviews.

• Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice and information is widely applicable.

Review the table to peruse review types and associated methodologies. Librarians can also help your team determine which review type might be appropriate for your project. 

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108.  doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: What is a Systematic Review?
• Next: Manuals and Reporting Guidelines >>
• Last Updated: May 16, 2024 2:10 PM
• URL: https://guides.mclibrary.duke.edu/sysreview
• Duke Health
• Duke University
• Duke Libraries
• Medical Center Archives
• Duke Directory
• Seeley G. Mudd Building
• 10 Searle Drive
• [email protected]

Faculty / Staff Search

Department / unit search.

• Student Services

• Catalogue & Collections
• Research Support
• My Library Account
• Videos and How-Tos
• typesofliteraturereviews

Videos & How-Tos

• YouTube Playlist
• Email us about Videos & How-Tos

Literature Reviews, Introduction to Different Types of

There are many different types of literature reviews, each with its own approach, analysis, and purpose. To confuse matters, these types aren't named consistently. The following are some of the more common types of literature reviews.

These are more rigorous, with some level of appraisal:

• The Systematic Review is important to health care and medical trials, and other subjects where methodology and data are important. Through rigorous review and analysis of literature that meets a specific criteria, the systematic review identifies and compares answers to health care related questions. The systematic review may include meta-analysis and meta-synthesis, which leads us to...
• The Quantitative or Qualitative Meta-analysis Review can both make up the whole or part of systematic review(s). Both are thorough and comprehensive in condensing and making sense of a large body of research. The quantitative meta-analysis reviews quantitative research, is objective, and includes statistical analysis. The qualitative meta-analysis reviews qualitative research, is subjective (or evaluative, or interpretive), and identifies new themes or concepts.

These don't always include a formal assessment or analysis:

• The Literature Review (see our Literature Review video) or Narrative Review often appears as a chapter in a thesis or dissertation. It describes what related research has already been conducted, how it informs the thesis, and how the thesis fits into the research in the field. (See https://student.unsw.edu.au/writing-critical-review for more information.)
• The Critical Review is like a literature review, but requires a more detailed examination of the literature, in order to compare and evaluate a number of perspectives.
• The Scoping Review is often used at the beginning of an article, dissertation or research proposal. It is conducted before the research begins, and sets the stage for this research by highlighting gaps in the literature, and explaining the need for the research about to be conducted, which is presented in the remainder of the article.
• The Conceptual Review groups articles according to concepts, or categories, or themes. It identifies the current 'understanding' of the given research topic, discusses how this understanding was reached, and attempts to determine whether a greater understanding can be suggested. It provides a snapshot of where things are with this particular field of research.
• The State-of-the-Art Review is conducted periodically, with a focus on the most recent research. It describes what is currently known, understood, or agreed upon regarding the research topic, and highlights where are there still disagreements.

Source: Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal , 26 (2), 91-108. http://doi.org/10.1111/j.1471-1842.2009.00848.x

Contact us for more assistance

1151 Richmond Street London, Ontario, Canada, N6A 3K7 Tel: 519-661-2111 Privacy | Web Standards | Terms of Use | Accessibility

About the Libraries

Library Accessibility

Library Privacy Statement

Land Acknowledgement

Support the Libraries

Types of Literature Reviews

Barry Mauer and John Venecek

Strategies for Getting Started

Composition guidelines, how to locate reviews by discipline.

We also provide the following activities:

Types of Literature Reviews [Refresher]

The previous page provided an introduction to literature reviews and guidelines for determining the scope and purpose of your review. Next, we take a look at the different types of literature reviews and why a researcher might select one type over another.

A literature review helps your reader understand the relationship of your research project to the work of other scholars. It covers the existing knowledge about a problem, and allows you to show the relevance/significance of your contribution to the discussion. Your reader may or may not have read scholarly literature about the theories, methodologies, and literary works you are discussing. But they want to know that you have read it and have thought about it. Your literature review provides not only a summary of the existing scholarship for readers; it also offers your perspective on it.

In general, literature reviews occur in the first half of a research paper, after the introduction. The literature review familiarizes your reader with the relevant scholarly discussion that has occurred to this time, and in the second half of your paper, you introduce your contribution to the discussion.

Not all humanities research projects contain literature reviews, but many do. Keep in mind that the type of literature review you choose (see list below) pertains to the secondary research – other scholarly sources – and not to the primary literary work. For instance, a literature review about Kate Chopin’s writing will be your thoughts about the scholarship on Chopin and not about Chopin’s text itself. You are summarizing what you see in the scholarly literature about Chopin’s writing. The literature review puts you in the position of authority not just on Chopin’s writing but on the scholarship about her writing. You are seeking to understand what scholars have said about her work. Scholars might belong to different schools of thought (psychoanalytic, feminist, Marxist, etc.). They might make different arguments about Chopin. They might use different methodological approaches. 

If your research involves two or more theories, such as psychology and genre studies, you may need to create multiple literature reviews, one for each theory or methodology. If the theories overlap with each other significantly (i.e., Marxism and Cultural Studies), you may combine them. Your literature review need not include everything about the subject area – you would need to write a book to cover a single theory – but only those concepts and methods that are most relevant to your research problem.

Factors to Consider When Developing Your Literature Review

• Determine the Scope : How broad or narrow should your literature review be? You may want to focus on recent scholarship only, or on a particular school of thought in the literature. Your scope is determined by your purpose; what is it you aim to achieve with your research?
• Establish Criteria : We discussed the importance of defining the purpose and scope of your review on the previous page, but it’s worth reviewing here as well. This step will help you establish important criteria and focus your searching. For example, how many sources will you need? What types of sources (primary, secondary, statistics, media)? Is currency important? Do you know who the prominent authors or theorists are in your subject area? Take some time to map out these or other important factors before you begin searching journals and databases.
• Consider Your Audience : Unlike a work cited page or an annotated bibliography, both of which are lists of sources, a literature review is essayistic and can be considered a precursor to your final paper. Therefore, it should be written in your own voice, and it should be geared toward a specific audience. Considering audience during this early stage will help focus your final paper as well.
• Find Models : We’ll discuss the different types of literature reviews and how to locate examples in the section below. However, even if you’re undecided about what type of review will work best for you, you may want to review some example literature reviews to get a sense of what they look like before you begin your own.

One piece of advice before starting: look for existing literature reviews on your area of scholarship. You can build on the work that other scholars have put into reviewing the scholarly literature. There’s no need to completely “reinvent the wheel” if some of the work is already done.

Scholars sometimes publish “stand-alone” literature reviews that are not part of a larger work; such literature reviews are valuable contributions to the field, as they summarize the state of knowledge for other scholars.

Maria J. Grant and Andrew Booth’s “A Typology of Reviews” identifies 14 distinct types of literature reviews. Further, the UCLA library created a chart to complement the article and for easy comparison of those 14 types of reviews. This section provides a brief summary of the most common literature reviews. For a more complete analysis, please see the full article and the chart .

To choose the most appropriate structure, put yourself in your reader’s shoes and think through their need for information. The literature review is about providing context for your contribution. How much context do people need? Keep it to the minimum necessary; compressing a lot of information into a small amount of text is a must.

These structures are not meant to be straightjackets but tools to help you organize your research. If you find that the tool is working, then keep using it. If not, switch tools or modify the one you are using. Keep in mind that the types of literature reviews are just different ways of organizing information. So, you can discuss literary trends without organizing your review of secondary literature by trend; your discussion can be organized by theory or theme, for examples. In our literature reviews, we are not recounting other scholars’ arguments at length but merely providing key concepts so we can summarize the discussion so far and position our own claims. You don’t have to adhere strictly to one structure or another. They are just organizing tools that help you manage your material (and help your reader make sense of it).

Types of Reviews

• Traditional or narrative reviews : This approach will generate a comprehensive, critical analysis of the published research on your topic. However, rather than merely compiling as many sources as possible, use this approach to establish a theoretical framework for your paper, establish trends, and identify gaps in the research. This process should bring your research question into clearer focus and help define a thesis that you will argue for in your paper. This is perhaps the most common and general type of literature review. The examples listed below are all designed to serve a more specific purpose.
• Argumentative : The purpose of an argumentative literature review is to select sources for the purpose of supporting or refuting a specific claim. While this type of review can help the author make a strong case for or against an issue, they can also be prone to claims of bias. Later in this textbook, we will read about the distinction between warranted and unwarranted bias . One is ok and the other is not.
• Chronological : A chronological review is used when the author wants to demonstrate the progression of how a theory, methodology, or issue has progressed over time. This method is most effective when there is a clear chronological path to the research about a specific historical event or trend as opposed to a more recursive theoretical concept.
• By trend : This is similar to the chronological approach except it focuses on clearly-defined trends rather than date ranges. This would be most appropriate if you want to illustrate changing perspectives or attitudes about a given issue when specific date ranges are less important than the ebb and flow of the trend.
• Thematic : In this type of literature review, the author will select specific themes that he or she feels are important to understanding a larger topic or concept. Then, the author will organize the sources around those themes, which are often based on relevance or importance. The value of this method is that the process of organizing the review by theme is similar to constructing an argument. This can help the author see how resources connect to each other and determine how as well as why specific sources support their thesis.
• Theoretical : The goal of this type of review is to examine how theory has shaped the research on a given topic. It establishes existing theoretical models, their connections, and how extensively they have been developed in the published research. For example, Jada applied critical race theory to her analysis of Sonny’s Blues , but she might also consider conducting a more comprehensive review of other theoretical frameworks such as feminism, Marxism, or postmodernism. Doing so could provide insight into alternate readings, and help her identify theoretical gaps such as unexplored or under-developed approaches to Baldwin’s work.
• Methodological : The approach focuses on the various methodologies used by researchers in a specific area rather than an analysis of their findings. In this case, you would create a framework of approaches to data collection related to your topic or research question. This is perhaps more common in education or the social and hard sciences where published research often includes a methods section, but it is sometimes appropriate for the digital humanities as well.
• Scoping : The aim of a scoping review is to provide a comprehensive overview or map of the published research or evidence related to a research question. This might be considered a prelude to a systematic review that would take the scoping review one step further toward answering a clearly defined research question. See below for more details.
• Systematic : The systematic review is most appropriate when you have a clearly-defined research question and have established criteria for the types of sources you need. In this way, the systematic review is less exploratory than other types of reviews. Rather, it is comprehensive, strategic, and focused on answering a specific research question. For this reason, the systematic review is more common in the health and social sciences, where comprehensiveness is more important. Literature reviews in the Humanities are not usually exhaustive but tend to show only the most representative or salient developments in the scholarship.
• Meta-analysis : Does your research deal with statistics or large amounts of data? If so, then a meta-analysis might be best for you rather than providing a critical review, the meta-analysis will summarize and synthesize the results of numerous studies that involve statistics or data to provide a more comprehensive picture than would be possible from just one study.

An argumentative literature review presents and takes sides in scholarly arguments about the literary work. It makes arguments about other scholars’ work. It does not necessarily involve a claim that the literary work is itself making an argument. Likewise, a chronological literature review presents the scholarly literature in chronological order.

You don’t need to keep strictly to one type. Scholars often combine features from various types of literature reviews. A sample review that combines the follow types –

• Argumentative
• Theoretical
• Methodological

– is the excellent work of Eiranen, Reetta, Mari Hatavara, Ville Kivimäki, Maria Mäkelä & Raisa Maria Toivo (2022) “ Narrative and Experience: Interdisciplinary Methodologies between History and Narratology , ” Scandinavian Journal of History , 47:1, 1-15

When writing your literature review, please follow these pointers:

• Conduct systematic searches
• Use Evidence
• Be Selective
• Use Quotes Sparingly
• Summarize & Synthesize
• Use Caution when Paraphrasing
• Use Your Own Voice

Advice from James Mason University’s “Literature Reviews: An Overview”

A note on synthesizing : Don’t make the common mistake of summarizing individual studies or articles one after the other. The goal is to synthesize — that is, to make observations about groups of studies. Synthesis often uses language like this:

• Much of the literature on [topic x ] focuses on [major themes].
• In recent years, researchers have begun investigating [facets a , b , and c ] of [topic x ].
• The studies in this review of [topic x ] confirm / suggest / call into question / support [idea / practice / finding / method / theory / guideline y ].
• In the reviewed studies [variable x ] was generally associated with higher / lower rates of [outcome y ].
• A limitation of some / most / all of these studies is [ y ].

Please see this sample annotated literature review  from James Mason University.

Structure of a literature review [1]

• Problematization: The 2 to 3 pages of problematization are a distinct, iterative, step. It may take doing such a statement a few times before moving forward to writing the actual paper.
• Search: Write down your keyword sets, your updated keyword sets, and databases. It is perfectly within a reviewer’s rights to ask for these details.
• Summary: Really getting to know major themes requires some annotation of articles. You want to identify core papers and themes and write about them. This helps you really learn the material. [ChatGPT or Wikipedia are no substitute for deep engagement with a paper.]
• Argument: Either outline or create a slide deck that help you express the arguments in your paper. Read them out loud. Have friends look at them. Present them. [Every literature review has an argument. If not, it’s a summary. A summary does not merit publication in a top outlet.]
• Unpacking: Once you’ve nailed the short pitch, unpack the full argument. [ a) Take time in each major section to map out a) the argument, b) the supporting evidence, and the takeaway. b) Take those major sections, reconcile them, make sure they don’t overlap, then move on to writing. c) Sketch out the paper’s sections, tables, figures, and appendices.]
• Writing: Writing is the easy part. You can always put words to the screen. [Revising and improving is hard. Make time to write every day. Improving requires feedback. Find a writing partner to give feedback. Create your tables and figures. Write to them. Make sure the words in the paper align to the visuals.]
• Communicate: When the paper is done, go back and create a paper presentation. [I do this for the papers that I’m most serious about. The act of storyboarding helps me sort out the small pieces of the story that don’t fit together. If I really want it to succeed, I present it. The act of presenting helps me get it right. My best papers sometimes take seven or eight presentations to get it right. Then I return to the paper and fine tune it. Only then, does it have a shot at a top outlet.]

Literature reviews can be published as part of a scholarly article, often after the introduction and sometimes with a header, but they can also be published as a standalone essay. To find examples of what reviews look like in your discipline, choose an appropriate subject database (such as MLA for literary criticism) and conduct a keyword search with the term “Literature Review” added in quotes:

Not only do these examples demonstrate how to structure different types of literature reviews, but some offer insights into trends and directions for future research. In the next section, we’ll take a closer look at some reading strategies to help guide you through this process.

Since scholars already have produced literature reviews on various scholarly conversations, you don’t always need to “reinvent the wheel” (start a literature review from nothing). You can find a published literature review and update it or amend it; scholars do that all the time. However, you must properly cite work you incorporate from others.

• What types of literature review will you be using for your paper? Why did you make this selection over others? If you haven’t made a selection yet, which types are you considering?
• What specific challenges do you face in following a literature review structure?
• If there are any elements of your assignment that need clarification, please list them.
• What was the most important lesson you learned from this page? What point was confusing or difficult to understand?
• Richard West, Brigham Young University, amended by Jason Thatcher, Temple University - https://www.linkedin.com/posts/jason-thatcher-0329764_academicwriting-topten2023-activity-7146507675021766656-BB0O ↵

Types of Literature Reviews Copyright © 2021 by Barry Mauer and John Venecek is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

ON YOUR 1ST ORDER

Different Types of Literature Review: Which One Fits Your Research?

By Laura Brown on 13th October 2023

You might not have heard that there are multiple kinds of literature review. However, with the progress in your academic career you will learn these classifications and may need to use different types of them. However, there is nothing to worry if you aren’t aware of them now, as here we are going to discuss this topic in detail.

There are approximately 14 types of literature review on the basis of their specific objectives, methodologies, and the way they approach and analyse existing literature in academic research. Of those 14, there are 4 major types. But before we delve into the details of each one of them and how they are useful in academics, let’s first understand the basics of literature review.

What is Literature Review?

A literature review is a critical and systematic summary and evaluation of existing research. It is an essential component of academic and research work, providing an overview of the current state of knowledge in a particular field.

In easy words, a literature review is like making a big, organised summary of all the important research and smart books or articles about a particular topic or question. It’s something scholars and researchers do, and it helps everyone see what we already know about that topic. It’s kind of like taking a snapshot of what we understand right now in a certain field.

It serves with some specific purpose in the research.

• Provides a comprehensive understanding of existing research on a topic.
• Identifies gaps, trends, and inconsistencies in the literature.
• Contextualise your own research within the broader academic discourse.
• Supports the development of theoretical frameworks or research hypotheses.

4 Major Types Of Literature Review

The four major types include, Narrative Review, Systematic Review, Meta-Analysis, and Scoping Review. These are known as the major ones because they’re like the “go-to” methods for researchers in academic and research circles. Think of them as the classic tools in the researcher’s toolbox. They’ve earned their reputation because they have a unique style for literature review introduction , clear steps and specific qualities that make them super handy for different research needs.

1. Narrative Review

Narrative reviews present a well-structured narrative that reads like a cohesive story, providing a comprehensive overview of a specific topic. These reviews often incorporate historical context and offer a broad understanding of the subject matter, making them valuable for researchers looking to establish a foundational understanding of their area of interest. They are particularly useful when a historical perspective or a broad context is necessary to comprehend the current state of knowledge in a field.

2. Systematic Review

Systematic reviews are renowned for their methodological rigour. They involve a meticulously structured process that includes the systematic selection of relevant studies, comprehensive data extraction, and a critical synthesis of their findings. This systematic approach is designed to minimise bias and subjectivity, making systematic reviews highly reliable and objective. They are considered the gold standard for evidence-based research as they provide a clear and rigorous assessment of the available evidence on a specific research question.

3. Meta Analysis

Meta analysis is a powerful method for researchers who prefer a quantitative and statistical perspective. It involves the statistical synthesis of data from various studies, allowing researchers to draw more precise and generalisable conclusions by combining data from multiple sources. Meta analyses are especially valuable when the aim is to quantitatively measure the effect size or impact of a particular intervention, treatment, or phenomenon.

4. Scoping Review

Scoping reviews are invaluable tools, especially for researchers in the early stages of exploring a topic. These reviews aim to map the existing literature, identifying gaps and helping clarify research questions. Scoping reviews provide a panoramic view of the available research, which is particularly useful when researchers are embarking on exploratory studies or trying to understand the breadth and depth of a subject before conducting more focused research.

Different Types Of Literature review In Research

There are some more approaches to conduct literature review. Let’s explore these classifications quickly.

5. Critical Review

Critical reviews provide an in-depth evaluation of existing literature, scrutinising sources for their strengths, weaknesses, and relevance. They offer a critical perspective, often highlighting gaps in the research and areas for further investigation.

6. Theoretical Review

Theoretical reviews are centred around exploring and analysing the theoretical frameworks, concepts, and models present in the literature. They aim to contribute to the development and refinement of theoretical perspectives within a specific field.

7. Integrative Review

Integrative reviews synthesise a diverse range of studies, drawing connections between various research findings to create a comprehensive understanding of a topic. These reviews often bridge gaps between different perspectives and provide a holistic overview.

8. Historical Review

Historical reviews focus on the evolution of a topic over time, tracing its development through past research, events, and scholarly contributions. They offer valuable context for understanding the current state of research.

9. Methodological Review

Among the different kinds of literature reviews, methodological reviews delve into the research methods and methodologies employed in existing studies. Researchers assess these approaches for their effectiveness, validity, and relevance to the research question at hand.

10. Cross-Disciplinary Review

Cross-disciplinary reviews explore a topic from multiple academic disciplines, emphasising the diversity of perspectives and insights that each discipline brings. They are particularly useful for interdisciplinary research projects and uncovering connections between seemingly unrelated fields.

11. Descriptive Review

Descriptive reviews provide an organised summary of existing literature without extensive analysis. They offer a straightforward overview of key findings, research methods, and themes present in the reviewed studies.

12. Rapid Review

Rapid reviews expedite the literature review process, focusing on summarising relevant studies quickly. They are often used for time-sensitive projects where efficiency is a priority, without sacrificing quality.

13. Conceptual Review

Conceptual reviews concentrate on clarifying and developing theoretical concepts within a specific field. They address ambiguities or inconsistencies in existing theories, aiming to refine and expand conceptual frameworks.

14. Library Research

Library research reviews rely primarily on library and archival resources to gather and synthesise information. They are often employed in historical or archive-based research projects, utilising library collections and historical documents for in-depth analysis.

Each type of literature review serves distinct purposes and comes with its own set of strengths and weaknesses, allowing researchers to choose the one that best suits their research objectives and questions.

Choosing the Ideal Literature Review Approach in Academics

In order to conduct your research in the right manner, it is important that you choose the correct type of review for your literature. Here are 8 amazing tips we have sorted for you in regard to literature review help so that you can select the best-suited type for your research.

• Clarify Your Research Goals: Begin by defining your research objectives and what you aim to achieve with the literature review. Are you looking to summarise existing knowledge, identify gaps, or analyse specific data?
• Understand Different Review Types: Familiarise yourself with different kinds of literature reviews, including systematic reviews, narrative reviews, meta-analyses, scoping reviews, and integrative reviews. Each serves a different purpose.
• Consider Available Resources: Assess the resources at your disposal, including time, access to databases, and the volume of literature on your topic. Some review types may be more resource-intensive than others.
• Alignment with Research Question: Ensure that the chosen review type aligns with your research question or hypothesis. Some types are better suited for answering specific research questions than others.
• Scope and Depth: Determine the scope and depth of your review. For a broad overview, a narrative review might be suitable, while a systematic review is ideal for an in-depth analysis.
• Consult with Advisors: Seek guidance from your academic advisors or mentors. They can provide valuable insights into which review type best fits your research goals and resources.
• Consider Research Field Standards: Different academic fields have established standards and preferences for different forms of literature review. Familiarise yourself with what is common and accepted in your field.
• Pilot Review: Consider conducting a small-scale pilot review of the literature to test the feasibility and suitability of your chosen review type before committing to a larger project.

Bonus Tip: Crafting an Effective Literature Review

Now, since you have learned all the literature review types and have understood which one to prefer, here are some bonus tips for you to structure a literature review of a dissertation .

• Clearly Define Your Research Question: Start with a well-defined and focused research question to guide your literature review.
• Thorough Search Strategy: Develop a comprehensive search strategy to ensure you capture all relevant literature.
• Critical Evaluation: Assess the quality and credibility of the sources you include in your review.
• Synthesise and Organise: Summarise the key findings and organise the literature into themes or categories.
• Maintain a Systematic Approach: If conducting a systematic review, adhere to a predefined methodology and reporting guidelines.
• Engage in Continuous Review: Regularly update your literature review to incorporate new research and maintain relevance.

Some Useful Tools And Resources For You

Effective literature reviews demand a range of tools and resources to streamline the process.

• Reference management software like EndNote, Zotero, and Mendeley helps organise, store, and cite sources, saving time and ensuring accuracy.
• Academic databases such as PubMed, Google Scholar, and Web of Science provide access to a vast array of scholarly articles, with advanced search and citation tracking features.
• Research guides from universities and libraries offer tips and templates for structuring reviews.
• Research networks like ResearchGate and Academia.edu facilitate collaboration and access to publications. Literature review templates and research workshops provide additional support.

Some Common Mistakes To Avoid

Avoid these common mistakes when crafting literature reviews.

• Unclear research objectives result in unfocused reviews, so start with well-defined questions.
• Biased source selection can compromise objectivity, so include diverse perspectives.
• Never miss on referencing; proper citation and referencing are essential for academic integrity.
• Don’t overlook older literature, which provides foundational insights.
• Be mindful of scope creep, where the review drifts from the research question; stay disciplined to maintain focus and relevance.

While Summing Up On Various Types Of Literature Review

As we conclude this classification of fourteen distinct approaches to conduct literature reviews, it’s clear that the world of research offers a multitude of avenues for understanding, analysing, and contributing to existing knowledge.

Whether you’re a seasoned scholar or a student beginning your academic journey, the choice of review type should align with your research objectives and the nature of your topic. The versatility of these approaches empowers you to tailor your review to the demands of your project.

Remember, your research endeavours have the potential to shape the future of knowledge, so choose wisely and dive into the world of literature reviews with confidence and purpose. Happy reviewing!

Laura Brown, a senior content writer who writes actionable blogs at Crowd Writer.

Assessment Types

• Critical Review

Literature Review

• Presentations

What is a Literature Review?

A literature review is different to an essay. It provides a snapshot of what we know about the research on a particular topic. The author is required to do a comprehensive search of relevant and recent literature (journal articles, text books, websites) to identify what is known about a topic. The main themes are then identified and crtically analysed. A literature review should cover:

• Compare and contrast the research, what are the common themes in the research?
• Are there any debates or disagreements in findings?
• Are there any gaps or limitations in the research reviewed?
• What are the strengths and limitations of the research?

What are the steps to write a literature review?

Adapted from Scribbr. Click here to learn more

Tips for Success Writing a Literature Review

• If it is your first time allow more time that you would for an essay or report.
• Finding quality research will make every step easier. Contact the library for help finding research.
• It can be difficult keeping track of sources. Use a note taking guide.
• Be flexible. Your structure may change as you write.

How do I Structure a Literature Review?

Organising Body Paragraphs

Do not arrange by articles. 

Arrange by themes and ideas across the articles. 

What does it look like?

Adapted from:

Ni Shé, C., Farrell, O., Brunton, J., & Costello, E. (2021). Integrating design thinking into instructional design: The #OpenTeach case study.  Australasian Journal of Educational Technology ,  38 (1), 33–52. https://doi.org/10.14742/ajet.6667

What language do I use in a Literature Review?

• << Previous: HELI
• Next: Presentations >>
• Last Updated: May 10, 2024 4:43 PM
• URL: https://eca.libguides.com/assessments

• University of Texas Libraries
• UT Libraries

Systematic Reviews & Evidence Synthesis Methods

Types of reviews.

• Formulate Question
• Find Existing Reviews & Protocols
• Register a Protocol
• Searching Systematically
• Supplementary Searching
• Managing Results
• Deduplication
• Critical Appraisal
• Glossary of terms
• Librarian Support
• Video tutorials This link opens in a new window
• Systematic Review & Evidence Synthesis Boot Camp

Not sure what type of review you want to conduct?

There are many types of reviews ---  narrative reviews ,  scoping reviews , systematic reviews, integrative reviews, umbrella reviews, rapid reviews and others --- and it's not always straightforward to choose which type of review to conduct. These Review Navigator tools (see below) ask a series of questions to guide you through the various kinds of reviews and to help you determine the best choice for your research needs.

• Which review is right for you? (Univ. of Manitoba)
• What type of review is right for you? (Cornell)
• Review Ready Reckoner - Assessment Tool (RRRsAT)
• A typology of reviews: an analysis of 14 review types and associated methodologies. by Grant & Booth
• Meeting the review family: exploring review types and associated information retrieval requirements | Health Info Libr J, 2019

Reproduced from Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies . Health Info Libr J. 2009 Jun;26(2):91-108. doi: 10.1111/j.1471-1842.2009.00848.x

• Last Updated: May 16, 2024 11:05 AM
• URL: https://guides.lib.utexas.edu/systematicreviews

Trends in mathematics education and insights from a meta-review and bibliometric analysis of review studies

• Original Paper
• Open access
• Published: 15 May 2024

Cite this article

You have full access to this open access article

• Mustafa Cevikbas   ORCID: orcid.org/0000-0002-7844-4707 1 ,
• Gabriele Kaiser 2 , 3 &
• Stanislaw Schukajlow 4  

1 Altmetric

Review studies are vital for advancing knowledge in many scientific fields, including mathematics education, amid burgeoning publications. Based on an extensive consideration of existing review typologies, we conducted a meta-review and bibliometric analysis to provide a comprehensive overview of and deeper insights into review studies within mathematics education. After searching Web of Science, we identified 259 review studies, revealing a significant increase in such studies over the last five years. Systematic reviews were the most prevalent type, followed by meta-analyses, generic literature reviews, and scoping reviews. On average, the review studies had a sample size of 99, with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines commonly employed. Despite certain studies offering nuanced distinctions among review types, ambiguity persisted. Only about a quarter of the studies explicitly reported employing specific theoretical frameworks (particularly, technology, knowledge, and competence models). Co-authored publications were most common within American institutions and the leading countries are the United States, Germany, China, Australia, and England in publishing most review studies. Educational review journals, educational psychology journals, special education journals, educational technology journals, and mathematics education journals provided platforms for review studies, and prominent research topics included digital technologies, teacher education, mathematics achievement, and learning disabilities. In this study, we synthesised a range of reviews to facilitate readers’ comprehension of conceptual congruities and disparities across various review types, as well as to track current research trends. The results suggest that there is a need for discipline-specific standards and guidelines for different types of mathematics education reviews, which may lead to more high-quality review studies to enhance progress in mathematics education.

Avoid common mistakes on your manuscript.

1 Introduction

Comprehensive literature reviews serve as foundational pillars for advancing scholarly discourse, offering critical insights into existing research and shaping future inquiries across disciplines. In the realm of academic writing, spanning from journal articles to dissertations, literature reviews are highly regarded for their capacity to synthesize knowledge, identify gaps, and provide a cohesive framework for understanding complex topics (Boote & Beile, 2005 ). Moreover, reviews play a significant role in academia by setting new research agendas and informing decision-making processes in practice, policy, and society (Kunisch et al., 2023 ).

As empirical and theoretical research burgeons in diverse fields, the need for literature review studies has become even more pronounced, facilitating a deeper understanding of specific research areas or themes (Hart, 2018 ; Nane et al., 2023 ). Additional factors contributing to the popularity of review studies in recent years include the rise of specialized review journals (Kunisch et al., 2023 ), challenges associated with conducting various types of empirical studies during the prolonged COVID-19 crisis (Cevikbas & Kaiser, 2023 ), and a competitive research climate wherein factors such as impact factors and citations hold significant weight (Ketcham & Crawford, 2007 ). Review studies are particularly attractive as they often garner a substantial number of citations, thereby enhancing researchers’ visibility and scholarly impact (Grant & Booth, 2009 ; Taherdoost, 2023 ).

The importance of review studies has been duly acknowledged in mathematics education, as evidenced by the inclusion of review papers in thematically oriented special issues of journals such as ZDM– Mathematics Education (Kaiser & Schukajlow, 2024 ), which has been originally founded as review journal. Several upcoming or already published special issues of ZDM– Mathematics Education , which emphasise ‘reviews on important themes in mathematics education’, highlight the importance of review studies as valuable contributions to the field.

The proliferation of literature reviews has increased interest in developing typologies to categorise them and understand different literature review approaches (Grant & Booth, 2009 ; Paré et al., 2015 ; Schryen & Sperling, 2023 ). Despite its significance, there remains a notable lack of research aimed at comprehensively understanding review studies within the field of mathematics education from a meta-perspective. In response to this gap, we conducted a systematic meta-review with the aim of providing an overview of different types of review studies in mathematics education over the past few decades and consolidating insights from multiple high-level review studies (Becker & Oxman, 2008 ; Schryen & Sperling, 2023 ). Meta-reviews offer concise yet comprehensive synopses and curated lists of pertinent reviews, adeptly addressing the perennial challenge of balancing thorough coverage with focused specificity (Grant & Booth, 2009 ).

In addition, we applied bibliometric analysis as a valuable tool for identifying research trends, progress, reliable sources, and future directions within the field. The bibliometric analysis aids in identifying hot research topics and trends (Song et al., 2019 ), assessing progress, identifying reliable sources, recognising major contributors, and predicting future research success (Geng et al., 2017 ). Furthermore, it helps researchers to pinpoint potential topics, suitable institutions for cooperation, and potential scholars for scientific collaboration (Martínez et al., 2015 ). By combining a meta-review and bibliometric analysis, we aim to offer a comprehensive overview of and deeper insights into state-of-the-art review studies within mathematics education.

Specifically, we seek to understand how the distribution and development of literature review studies in mathematics education have evolved over the years, examining factors such as publication years, publishers, review types, sample sizes, and the use of theoretical or conceptual frameworks. Additionally, we aim to assess adherence to review study guidelines and protocols, providing insights into the rigor and quality of research methodologies employed, particularly in light of the lack of clear guidance on producing rigorous and impactful literature reviews (Kunisch et al., 2023 ).

Furthermore, we endeavour to identify authors who have made contribution to the field of mathematics education through review studies, as well as those whose work is most frequently cited. We also identify co-authorship network analysis as understanding research networks allows researchers to identify potential collaborators and build partnerships with other scholars in various countries. Collaborative research endeavours can lead to enhanced research outcomes, broader dissemination of findings, and increased opportunities for funding and professional development. It can also highlight interdisciplinary connections and collaborations within and across fields, leading to innovative approaches and solutions to complex research questions (RQs) that transcend disciplinary boundaries.

Moreover, we analysed the distribution of common keywords across review studies, identifying focal subjects and thematic areas prevalent in mathematics education research. This analysis can provide valuable insights into key topics and trends shaping the field, guiding future research directions and priorities.

Lastly, we identified the most cited review papers in mathematics education and the journals in which they have been published, recognizing seminal works and influential publications that have contributed to the advancement of the field.

Overall, in light of the preceding discourse, we addressed the following RQs to uncover the characteristics of review studies, identify research trends, and delineate future research directions in mathematics education:

How can the distribution and development of review studies in mathematics education over time be characterised according to the number of manuscripts, publishers, review types, sample sizes, the use of theoretical or conceptual frameworks, and adherence to review study guidelines and protocols?

Which authors have contributed the largest number of review studies in mathematics education, and which authors’ review papers are most frequently cited in the literature?

From which countries are the authors of the review studies in mathematics education?

Which author keywords can be identified in the review studies in mathematics education, how are these keywords distributed across the analysed review studies, and which focal topics do these keywords indicate?

What are the most cited review papers in mathematics education, and in which journals have they been published?

2 Literature review studies and review typologies– background information

In this chapter, we provide a thorough analysis of different typologies for review studies, as we seek to elucidate the primary characteristics of various review studies conducted within mathematics education (Sect.  2.1 ). This effort led to the identification of 28 review types presented in Table  1 , which were used in the current study’s literature search processes to access existing review studies and the analysis of identified studies in the field of mathematics education. Furthermore, we discuss the advancement of guidelines and protocols, highlighting their role in shaping the conduct of review studies (Sect. 2.2). Finally, we conclude the chapter by underscoring the importance and potential impact of meta-reviews and bibliometric analyses in the context of mathematics education (Sect. 2.3).

2.1 Literature review typologies

Researchers have defined and emphasized different review types with distinct features, objectives, and methodologies. To address the challenge of ambiguous review categorisations, we conducted an extensive search and analysis of the literature on Web of Science (WoS) using the search strings ‘typology of reviews’ and ‘taxonomy of reviews’ to search the titles of studies. We focused particularly on influential theoretical, conceptual, and review papers discussing the taxonomy and typology of review studies and recent advances driven by scholars across diverse fields.

2.1.1 Seminal work by Grant and Booth ( 2009 ) on the discourse of literature review typologies

The categorisation of literature reviews has been profoundly influenced by the seminal work of Grant and Booth ( 2009 ), on which typologies of literature reviews are often based. Their paper garnered significant attention, with over 10,304 citations as of 20 April 2024 according to Google Scholar. Originally in the field of health information theory and practice, these authors founded their work on earlier approaches, notably Cochrane’s ( 1979 ) approach. Grant and Booth ( 2009 ) claimed that the developed typology could standardise the diverse terminology used. They distinguished 14 review types, which we summarise below, highlighting the main scope and search methodologies (Grant & Booth, 2009 , pp. 94–95):

A critical review ‘goes beyond mere description of identified articles and includes a degree of analysis and conceptual innovation’; no formalised or systematic approach is required because the aim of such a review is ‘to identify conceptual contributions to embody existing or derive new theory’.

A generic literature review incorporates ‘published materials that provide examination of recent of current literature’; comprehensive searching may or may not be necessary.

A mapping review/systematic mapping is used to ‘categorize existing literature’ and identify gaps in the research literature. The completeness of a search is important, but no formal quality assessment is needed.

A meta-analysis is a ‘technique that statistically combines the results of quantitative studies to provide a more precise effect of the results’; a comprehensive search is conducted based on the inclusion and exclusion criteria.

A mixed-studies review/mixed-methods review incorporates ‘a combination of review approaches, for example combining quantitative with qualitative research… and requires a very sensitive search’.

An overview is a generic term describing a ‘summary of the… literature that attempts to survey the literature and describe its characteristics’; it may or may not include comprehensive searching and quality assessment.

A qualitative systematic review/qualitative evidence synthesis is a ‘method for integrating or comparing the findings from qualitative studies’, and it may involve selective sampling.

A rapid review comprises an ‘assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research’; a characteristic of such a review is that the ‘completeness of searching is determined by time constraints’.

A scoping review is a ‘preliminary assessment of the potential size and scope of available research literature’, with the ‘completeness of searching determined by time/scope constraints’.

A state-of-the-art review ‘tend[s] to address more current matters in contrast to other combined retrospective and current approaches’ and ‘aims for comprehensive searching of current literature’.

A systematic review ‘seeks to systematically search for, appraise and synthesise research evidence’ and should be comprehensive and based on inclusion/exclusion criteria.

A systematic search and review ‘combines [the] strengths of critical review with a comprehensive search process’, typically addressing broad questions to produce ‘best evidence synthesis’ based on ‘exhaustive, comprehensive searching’.

A systematised review ‘include[s] elements of systematic review process while stopping short of systematic review’, ‘typically conducted as postgraduate student assignment’; it ‘may or may not include comprehensive searching’.

An umbrella review ‘specifically refers to review compiling evidence from multiple reviews into one accessible and usable document’ via ‘identification of component reviews, but no search for primary studies’. ‘Primary studies’ refer to original research studies or individual studies conducted by researchers to gather data first-hand.

Booth with colleagues later expanded the typology by introducing the concept of a review family construct and amalgamating various types of reviews for further refinement, such as traditional reviews, systematic reviews, review of reviews, rapid reviews, mixed-methods reviews, and purpose-specific reviews (for details, see Sutton et al., 2019 ).

2.1.2 Further development of the review typologies

Many classifications for review studies have been developed, and in the following section, we present more recent approaches. Paré et al. ( 2015 ), in another highly cited study (2,059 Google Scholar citations as of 20 April 2024) considered seven recurrent dimensions: the goal of the review, the scope of the review questions, the search strategy, the nature of the primary sources, the explicitness of the study selection, quality appraisal, and the methods used to analyse/synthesise the findings. Based on these dimensions, they formulated nine different literature review types: narrative reviews, descriptive reviews, scoping/mapping reviews, meta-analyses, qualitative systematic reviews, umbrella reviews, critical reviews, theoretical reviews, and realist reviews.

In Paré et al.’s ( 2015 ) classification, the review categories that differ from Grant and Booth’s ( 2009 ) classification are theoretical reviews, realist reviews, narrative reviews, and descriptive reviews, which we therefore describe them briefly. A theoretical review draws on conceptual and empirical studies to develop a conceptual framework or model using structured approaches, such as taxonomies, to discover patterns or commonalities. The aim of a realist review (also called a meta-narrative review) is to formulate explanations; such reviews ‘are theory-driven interpretative reviews which were developed to inform, enhance, extend, or alternatively supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision making’ (Paré et al., 2015 , p. 188). The purpose of a narrative review is to survey the existing literature on a particular subject or topic without necessarily seeking generalisations or cumulative insights from the material reviewed (Davies, 2000 ). Typically, such reviews do not detail the underpinning review processes or involve systematic and exhaustive searches of all pertinent literature. This category resembles Grant and Booth’s ( 2009 ) description of ‘literature reviews’ and overlaps with Samnani et al.’s ( 2017 ) narrative reviews, literature reviews, and overviews, resulting in a somewhat ambiguous typology. The aim of a descriptive review is to identify patterns and trends across a set of empirical studies within a specific research field, encompassing pre-existing propositions, theories, methodological approaches, or findings. To accomplish this objective, descriptive reviews collect, structure, and analyse numerical data that reflect the frequency distribution of research elements.

MacEntee ( 2019 ), Samnani et al. ( 2017 ), Schryen et al. ( 2020 ), and Taherdoost ( 2023 ) corroborated Grant and Booth’s ( 2009 ) and Paré et al.’s ( 2015 ) classifications, identifying various common review categories (see Table  1 ). In Samnani et al.’s ( 2017 ) classification, a distinct review type based on the previously mentioned categories is meta-synthesis , the aim of which is to provide explanations for phenomena, in contrast to meta-analysis, which focuses on quantitative outcomes.

Later, Schryen and Sperling ( 2023 ) introduced a slightly revised typology of literature review studies, which they applied to a meta-review of operations research. Their study distinguished nine types of literature reviews, newly introduced categories included tutorial reviews, selective reviews, algorithmic reviews, computational reviews, and meta-reviews. The objective of a tutorial review is to offer a research-oriented summary of principles, mathematical fundamentals, and concepts, aiming to inspire and direct future research endeavours. The authors’ emphasis on foundational aspects has often provided a launching pad for research advances. A selective review typically has a limited scope because it is not based on a thorough search of all relevant literature. This type of review concentrates on specific segments of the literature, such as journals, time periods, methodologies, or issues, to delve deeper into specific questions and phenomena. An algorithmic review focuses on advances in algorithms and frameworks in the literature that address a spectrum of problems. It employs either selective or comprehensive search strategies, predominantly examining algorithm-related sources. A computational review investigates algorithms and/or parameterisations proposed in the literature, largely considering implementations and computational studies, measurement efficiency, effectiveness, and different forms of robustness. Finally, Schryen and Sperling ( 2023 ) defined a meta-review as an overview of systematic reviews or a systematic review of reviews and pointed out that a meta-review can also be called an umbrella review (which is the case by Grant and Booth), again confirming the fuzzy nature of the currently available typologies. According to Schryen and Sperling ( 2023 ), meta-reviews primarily aim to furnish descriptive overviews of literature reviews, serving as tertiary studies that integrate evidence from multiple (qualitative or quantitative) reviews into unified and user-friendly documents (Becker & Oxman, 2008 ; Paré et al., 2015 ). In contrast to the previously mentioned perspectives, Schryen and Sperling ( 2023 ) argued that meta-reviews are not limited to addressing specific research questions but can also address a wide range of enquiries.

Chigbu et al. ( 2023 , pp. 5–6) emphasised that there ‘is a continuum of literature types’ (p. 4) and distinguished twelve different types of literature reviews, six of which were not covered by the classifications provided by previously mentioned studies: integrated reviews, interpretative reviews, iterative reviews, semi-systematic reviews, and bibliometric reviews. According to their approach, an integrative review builds ‘new knowledge based on the existing body of literature following a rationalist perspective’, an interpretative review ‘interprets what other scholars have written to put into specific perspectives’, and an iterative review is an ‘algorithm-based approach performed to collate all studies in a specific field of research’. Moreover, a meta-synthesis review examines and analyses qualitative study findings and is often employed to clarify specific concepts. Additionally, a semi-systematic review analyses the data and findings of other studies to address specific research inquiries, using a partial systematic review methodology. Lastly, a bibliometric review systematically examines the literature on a specific subject or research discipline by quantitatively measuring indicators such as authors, citations, journals, countries, and years of publications.

As previously noted in this paper, this detailed description of review types is instrumental in facilitating our investigation of various review studies in the realm of mathematics education.

2.2 Advancements in guidelines and protocols for review studies

Various researchers have developed guidelines, protocols, and statements to assist authors in conducting, evaluating, and reporting their review studies. This academic endeavour has predominantly focused on enhancing the rigour and transparency of systematic reviews, meta-analyses, and, more recently, scoping reviews. For instance, the population, intervention, comparison, and outcomes (PICO) model, originally conceived to support evidence-based healthcare, serves as a cornerstone for establishing review criteria, crafting research questions and search strategies, and delineating the characteristics of included studies or meta-analyses (Richardson et al., 1995 ). In response to the observed deficiencies in reporting standards within meta-analyses, an international consortium introduced the Quality of Reporting of Meta-Analyses (QUOROM) statement in 1996, primarily to enhance the reporting quality of meta-analyses involving randomised controlled trials (Moher et al., 1999 ). Subsequently, Moher et al. ( 2009 ) updated these guidelines, which are now known as the PRISMA guidelines, and incorporated various conceptual and methodological advances in systematic reviews and meta-analyses. Additionally, Shea et al. ( 2007 ) introduced the Assessment of Multiple Systematic Reviews (AMSTAR) checklist to evaluate methodological quality and guide the conduct of systematic reviews, while Grant and Booth ( 2009 ) developed the search, appraisal, synthesis, and analysis (SALSA) framework to analyse and characterise review types. Most recently, Page et al. ( 2021 ) updated the PRISMA guidelines, providing updated reporting standards that reflect advances in methods for identifying, selecting, appraising, and synthesising studies, with the aim of promoting more transparent, complete, and accurate reporting of systematic reviews and meta-analyses. An extension of PRISMA guidelines for scoping reviews, known as PRISMA-ScR, aids readers in understanding relevant terminology, core concepts, and key items for reporting scoping reviews (Tricco et al., 2018 ). Despite the value of these efforts, further studies are warranted, particularly comprehensive guidelines for each type of review studies.

2.3 Literature reviews in mathematics education

The preceding section delineates various types of review studies, underscoring their key methodological attributes. Within the realm of mathematics education, akin to other disciplines, literature review studies, particularly systematic reviews, and meta-analyses, received considerable attention (Cevikbas et al., 2022 ; Cevikbas & Kaiser, 2023 ; Kaiser & Schukajlow, 2024 ). However, the understanding of the prevailing characteristics of review studies in mathematics education, including prevalent review types, trends, gaps, and avenues for future improvement, remains limited.

Meta-reviews can offer a promising avenue for pinpointing research gaps, evaluating evidence quality, and informing policy and intervention strategies and guiding evidence-based decision-making processes by synthesizing findings from multiple review studies (Schryen & Sperling, 2023 ). In addition to meta-reviews, the bibliometric analyses serve to ascertain the scope of prior research, discern contemporary review trends, identify literature gaps, and propose future research agendas (Chigbu et al., 2023 ). While meta-reviews provide a comprehensive assessment of the literature, bibliometric analyses aid in systematically screening literature on a specific subject, topic, or research discipline by quantitatively measuring various indicators such as authors, citations, journals, countries, and years of publication. These methodological approaches hold promise for instituting a systematic, transparent, and reproducible review process, thereby augmenting the overall quality of reviews in mathematics education. Bibliometric techniques serve as valuable tools in literature reviews, guiding researchers by pinpointing influential works and impartially mapping the research landscape prior to in-depth exploration (Zupic & Cater, 2015 ).

Despite their significance, meta-reviews and bibliometric analyses remain seldom within the domain of mathematics education, signifying a substantial gap in the literature. Our comprehensive literature review underscores an urgent need for meta-review studies encompassing literature review studies in the realm of mathematics education. Additionally, while no bibliometric analysis study specifically focusing on review studies in mathematics education was identified, several bibliometric studies in mathematics education on various topics were noted, such as mathematics anxiety (Radevic & Milovanovic, 2023 ), problem-solving (Suseelan et al., 2022 ), and teacher noticing (Wei et al., 2023 ).

Overall, there exists a compelling need for meta-reviews enriched by bibliometric analyses to explore the current state of literature review research in mathematics education, and the current study aims to address this gap in a timely manner.

3 Methodology

3.1 literature search and manuscript selection process.

In this study, following the latest PRISMA guidelines (Page et al., 2021 ), we aimed to conduct a systematic review of previous review studies in mathematics education. Specifically, we employed the meta-review (umbrella review) method supplemented by bibliometric analyses. We processed the manuscript selection under three stages: identification, screening, and included.

3.1.1 Identification

On 10 January 2024 (last access), we conducted an extensive literature search using the WoS electronic database, which includes publications in high-ranking peer-reviewed journals and is widely acknowledged as a primary source of review and bibliometric data that meet high quality standards (Korom, 2019 ). WoS facilitates effective literature searches, supports various information purposes, and aids research topic mapping, trend monitoring as well as scholarly activity analysis (Birkle et al., 2020 ).

To comprehensively identify potentially relevant review studies in mathematics education, we developed an inclusive search query targeting specific terms in the titles, abstracts, and keywords of papers. The query comprised terms that we extracted from the typologies of literature reviews described in Chap. 2, particularly the more general, commonly used types of reviews:

( TOPIC ) ((literature review*OR literature survey* OR systematic review* OR rapid review* OR scoping review* OR critical review* OR meta-analysis OR narrative review* OR umbrella review* OR meta review* OR meta-review OR bibliometric review OR bibliometric analysis OR mapping review OR mixed-methods review OR integrative review OR interpretative review OR iterative review OR meta-synthesis OR descriptive review OR theoretical review OR realist review OR selective review OR algorithmic review OR computational review)) AND ( TOPIC ) ((math* OR geometry OR algebra OR calculus OR probability OR statistics OR arithmetic).

Based on these search strings, we conducted an online search that initially yielded 63,462 records.

3.1.2 Screening

In this stage, we applied data cleaning filters based on the manuscript inclusion and exclusion criteria (see Table  2 ). First, we electronically filtered the identified records based on language, resulting in the retention of 61,787 papers published in English. Subsequently, we narrowed down the selection to 10,098 papers using the following five categories of research areas within the WoS: ‘education/educational research, psychology, social sciences other topics, mathematics, or science technology other topics’. Following this categorisation, we further refined the dataset by excluding non-review papers and accessing 3,344 records within the ‘review article’ and ‘early access’ categories of the WoS database. We categorised records lacking a final publication date that had undergone peer review and acceptance as ‘early access’. Notably, to comprehensively capture publication trends, we imposed no restrictions on the publication years of the studies. In the subsequent phase, a meticulous manual screening of the titles, abstracts, and keywords of 3,344 papers led to the identification of 357 studies in mathematics education.

3.1.3 Included

Ultimately, after an extensive review of the full-text versions of initially identified 357 papers, 259 eligible review articles remained for analysis as these papers fulfilled our criteria comprehensively (see the Appendix for the list of included studies; see Fig.  1 for the flow diagram of the entire manuscript selection process). Subsequently, as detailed below, the data analysis process commenced with the inclusion of these eligible review papers in mathematics education.

Flow diagram of the manuscript selection process

3.2 Data analysis

After incorporating 259 studies into this meta-review and bibliometric analysis, we compiled the identified records into a marked list on WoS. Subsequently, we exported the records into Excel, EndNote, and plain text file formats for analysis. The analysis consisted of content analysis and bibliometric analysis (see Fig.  2 , adapted from Wei et al., 2023 ).

For the content analysis, we meticulously organised the records using EndNote reference management software and Excel worksheets. We scrutinised the full-text versions of all included articles, coding them based on (1) publication year, (2) publisher, (3) review type, (4) number of included studies (sample size), (5) guidelines and protocols for the article selection process, and (6) the theoretical and conceptual framework of the studies.

Our coding manual, informed by prior studies (Cevikbas et al., 2022 , 2024 ), guided this process (see appendix for a sample of the coding manual). After completing the content analysis coding procedure, 20% of the papers ( n  = 52) were double-coded based on the initial coding protocol. The intercoder reliability, gauged at 0.92, signifies the presence of a coding system that exhibits satisfactory reliability (Creswell, 2013 ). Any discrepancies were addressed through discussions among the coders until consensus was reached.

For the bibliometric analysis, we employed VOSviewer software (version 1.6.20), which is widely recognised and extensively used in various fields, including the educational sciences (van Eck & Waltman, 2010 ). Chigbu et al. ( 2023 ) pointed out that the WoS database plays a pivotal role in facilitating bibliometric analyses across various disciplines. These analyses help establish trends in the development and application of knowledge within specific subjects and disciplines.

In our study, the bibliometric network presented in the results chapter consists of nodes and edges, with nodes representing entities such as publications, journals, researchers, or keywords. Edges denote relationships between pairs of nodes, indicating not only the presence or absence of connections but also conveying the intensity or strength of relationships (van Eck & Waltman, 2010 ). For distance-based approaches, the positioning of nodes in a bibliometric network reflects their approximate relatedness based on proximity.

Utilising VOSviewer software, we conducted (1) co-authorship analysis (authors and countries) to elucidate collaboration patterns and contributions, (2) co-occurrence analysis (focusing Author Keywords) to scrutinise knowledge structures and the distribution and development of key research topics in mathematics education, and (3) citation analysis to delve deeper into research influences and citation networks, drawing insights from the documents and sources.

This multifaceted approach allowed us to gain a comprehensive understanding of the bibliometric landscape and unravel collaborative structures, thematic foci, and the influence of key works on mathematics education.

Analytical process for this study

In this chapter, we present the key results of the meta-review and bibliometric analyses divided into two main categories: an overview of the review studies in mathematics education based on the content analysis, addressing RQ1, and the results of the bibliometric analysis, addressing RQ2 – RQ5.

4.1 Overview of review studies in mathematics education (RQ1)

To discern the research trends and essential attributes of review studies in mathematics education, we conducted a content analysis within our meta-review to examine the 259 included review studies. Our analysis encompassed publication years, publishers, review types, guidelines, protocols used, sample sizes, and the theoretical and conceptual frameworks employed in these review studies. A general overview of the included studies is presented in Table  3 .

Our literature search with no restriction on the publication years yielded review studies published between 1996 and 2023, with a notable increase within the last five years (2019–2023, see Fig.  3 ).

Distribution of publications from 1996 to 2023

The analysis showed that the Springer Group is the primary publisher of review articles in mathematics education, followed by Taylor & Francis, Elsevier, Sage, Frontiers, Wiley, MDPI, and the American Psychological Association (APA) (see Table  4 ). Other publishers published the remaining review articles ( n  = 43). This result may be attributed to the predominance of mathematics education journals published by Springer within the WoS database.

To explore the prevailing types of review studies in mathematics education, we scrutinised the review methodologies of the included studies, considering the review types presented earlier in Table  1 . The findings revealed that researchers conducted (according to their own classification) 10 different types of reviews in mathematics education as outlined in Fig.  4 .

Types of review studies Note: *systematic reviews and meta-analyses ( n  = 6), systematic reviews and bibliometric analyses ( n  = 3), meta-analyses and narrative reviews ( n  = 2), and meta-analysis and critical review ( n  = 1)

Our analysis did not yield further review types in mathematics education. Time-related analysis showed that recent studies were systematic reviews, meta-analyses, literature reviews, and scoping reviews, whereas early examples of review studies in mathematics education were primarily narrative or critical reviews or were not explicitly classified according to review type by their authors. Figure  4 shows that some researchers ( n  = 18) described their studies as literature reviews using Grant and Booth’s ( 2009 ) generic term, without providing further details about the type of review.

To comprehend the methodologies employed by researchers to conduct reviews and select eligible studies, we conducted an analysis of the guidelines and protocols the researchers used. The findings revealed that the PRISMA guidelines were the most frequently employed ( n  = 121), aligning with the distribution of review types—PRISMA guidelines are basically recommended for systematic reviews and meta-analyses (Page et al., 2021 ). For scoping reviews, the guidelines developed by Arksey and O’Malley ( 2005 ) were the most prevalent and were used in seven studies. In six instances, researchers applied various guidelines (e.g. PICO or SALSA guidelines) sourced from the literature. Almost half of the studies ( n  = 125) did not specify the use of guidelines for conducting literature searches and selecting eligible studies. Additionally, three studies aimed to provide protocols for conducting review studies. Furthermore, seven studies were preregistered as review studies, following the Open Science Framework (OSF) and/or the International Prospective Register of Systematic Reviews (PROSPERO) protocol.

A prevalent discourse among researchers in review studies revolved around determining the most suitable number of studies to include in reviews. Our results revealed that the sample sizes of the included studies (i.e. the number of primary studies) in the field of mathematics education ranged from 8 to 3,485. Unfortunately, this information was not reported in 19 review articles. In the remaining 240 review articles, the average was 99 included studies, with an overall total of 23,761. Most of the studies ( n  = 202) had sample sizes of less than 100, with an average of 34 (see Table  5 ). Although we harboured concerns that the review studies identified in this investigation might not have been aptly named and conceptualised by their authors, we deliberately refrained from addressing this issue because it fell outside the scope of our study. While including a substantial number of studies is common and potentially suitable for bibliometric analyses and meta-analyses, conducting a systematic review, scoping review, or narrative review that critically analyses exceptionally high volumes of studies may pose challenges. In this meta-review, for example, we observed that five articles included more than 1,000 studies in the review process. Two studies, enriched by bibliometric analysis, took this approach, while another study was identified by the authors as a scoping review with a sample size of 2,433. Additionally, two studies were labelled as systematic reviews with sample sizes of 1,968, and 3,485, respectively.

Finally, we conducted a content analysis to scrutinise the theoretical and conceptual frameworks underpinning the included review studies in mathematics education. The findings revealed that out of 259 review studies, only 61 incorporated any theoretical or conceptual framework. Notably, a subset of studies ( n  = 14) was based on technology-related conceptual frameworks, such as Technological Pedagogical Content Knowledge (TPACK), frameworks pertaining to augmented and virtual reality, embodied design, artificial intelligence, big data, and the European Framework for the Digital Competence for Educators (DigCompEdu). Another prevalent category ( n  = 10) relied on frameworks related to the knowledge and competence of individuals (e.g. teachers and/or students), encompassing models such as the competence as continuum framework, TPACK, the didactic-mathematical knowledge and competencies model, mathematical content knowledge, pedagogical content knowledge, mathematical knowledge for teaching, teacher noticing competence, and an integrative model for the study of developmental competencies in minority children. Bronfenbrenner’s ecological theories (e.g. ecological theory of human development, bioecological model of human development, ecological systems theory, and ecological dynamics—a blend of dynamic-systems theory and ecological psychology) were employed by researchers in five review studies in mathematics education. In a limited subset of the studies, social and cultural theories (e.g. sociocultural theory, social learning theory, and cultural activity theory ( n  = 3)), cognitive theories (e.g. cognitive developmental theory ( n  = 2)), affective theories (e.g. self-determination theory and expectancy-value theory ( n  = 2)), linguistic theories ( n  = 2), and constructivist theories ( n  = 2) were used as frameworks. Additionally, researchers used conceptual frameworks concerning computational thinking ( n  = 2) and engagement ( n  = 3) alongside a few less frequently reported frameworks.

4.2 Results of the bibliometric analysis (RQ2–RQ5)

To identify productive and most cited authors, important journals, and countries of origin of the authors, along with the underlying research collaborations between researchers and countries, as well as research trends and key topics of review studies in mathematics education, we conducted a bibliometric analysis based on co-authorship, co-occurrence, and citations.

4.2.1 Co-authorship analysis

We conducted a co-authorship analysis according to authors and countries within the units of analysis.

Co-authorship and author analysis

The bibliometric analysis, using VOSviewer, revealed that 761 authors contributed to mathematics education, each of whom conducted at least one review study. The review papers were predominantly authored through collaboration, with most being written by two authors (30,2%), followed by three authors (20,2%), four authors (19,4%), a single author (10,1%), five authors (8,9%), six authors (6,2%), seven authors (3,5%), eight authors (1,6%), and nine authors (0,4%). These results showed that researchers primarily collaborate with their colleagues in conducting review studies—a practice vital for reducing workload and enhancing the quality of analyses—with the advantage of incorporating the various perspectives of different authors.

Table  6 highlights the top 17 authors who published a minimum of three review papers each. Notably, Lieven Verschaffel is the only scholar present in both lists of prolific and highly cited authors. The researchers listed in Table  7 , except Lieven Verschaffel, contributed to the field with a single review study. Consequently, while these researchers rank among most cited authors, the low total link strength (TLS) values indicate their limited collaboration with other scholars. The TLS was automatically calculated by VOSviewer and represents the overall intensity of co-authorship connections between a particular researcher and others. According to the co-authorship analysis, it is also noteworthy that many of the highly cited authors’ review studies typically date back over ten years, which is expected as citations tend to accumulate gradually over time. The results from the detailed citation analyses provided in Sect. 4.2.3.

Upon examining the research domains of prolific and highly cited authors, we found a diverse range of topics spanning mathematics education, psychology, educational psychology, special education, and neuroscience. This diversity highlights the interdisciplinary nature of research in mathematics education, with contributions to the literature review studies from psychologists and special education and neuroscience scholars alongside mathematics educators.

Figure  5 shows a co-authorship network map for the authors of the included review studies based on the TLS. We set the minimum number of documents for an author as one, which encompassed 761 authors who contributed to review papers in mathematics education. This bibliometric co-authorship analysis yielded 51 clusters, each containing a minimum of five items (researchers). The prominent co-authorship clusters included a green cluster (led by Lieven Verschaffel), a blue cluster (led by Gabriele Kaiser and Mustafa Cevikbas), a red cluster (led by Nelson Gena), and a yellow cluster (led by Diane P. Bryant). Nelson Gena had the highest number of collaboration links, with a TLS of 26, followed by Lieven Verschaffel (TLS = 22), Gabriele Kaiser (TLS = 16), Soyoung Park (TLS = 16), Tassia Bradford (TLS = 13), Diane P. Bryant (TLS = 12), Johannes König (TLS = 12), Mikyung Shin (TLS = 12), Min Wook Ok (TLS = 12), Bert de Smedt (TLS = 10), Fred Spooner (TLS = 10), Jihyun Lee (TLS = 10), Mustafa Cevikbas (TLS = 10), Rosella Santagata (TLS = 10), Sarah R. Powell (TLS = 10), and Thorsten Scheiner (TLS = 10).

Co-authorship and author networks

Co-authorship and country analysis

We conducted a co-authorship–country analysis, setting the minimum number of documents for a country as one, and identified 50 countries. This selection resulted in five clusters, each containing a minimum of five items (countries).

The most prominent cluster was the green cluster, encompassing eight countries from various global regions: the United States (US; TLS = 30), Germany (TLS = 23), Australia (TLS = 21), China (TLS = 11), South Korea (TLS = 6), Sweden (TLS = 4), New Zealand (TLS = 2), and Jordan (TLS = 1). The US dominated research collaborations both within this cluster and overall.

The red cluster included nine countries, predominantly Nordic and European countries: Norway (TLS = 13), Finland (TLS = 7), Belgium (TLS = 6), the Netherlands (TLS = 6), Lithuania (TLS = 1), Portugal (TLS = 1), Luxembourg (TLS = 1), Scotland (TLS = 1), and Israel (TLS = 1).

The yellow cluster contained seven countries: Canada (TLS = 7), Malaysia (TLS = 7), Denmark (TLS = 3), Libya (TLS = 2), Singapore (TLS = 2), Indonesia (TLS = 1), and the United Arab Emirates (TLS = 1).

The blue cluster primarily highlighted European collaborations and included seven countries: England (TLS = 22), Switzerland (TLS = 4), Italy (TLS = 3), France (TLS = 3), Greece (TLS = 1), Chile (TLS = 1), and Saudi Arabia (TLS = 1).

Lastly, the purple cluster represented a network of predominantly South and North American countries featuring, among others, Brazil (TLS = 6), Ireland (TLS = 5), Mexico (TLS = 4), Ecuador (TLS = 2), and Cuba (TLS = 2)(See Fig. 6 ).

Co-authorship and country networks

4.2.2 Co-occurrence analysis

To explore the research hotspots within mathematics education, we ran a keyword co-occurrence analysis using Author Keywords.

Co-occurrence analysis based on author keywords

The author keyword co-occurrence analysis indicated that our repository contained 691 keywords (see Fig.  7 , left side), of which 23 met the minimum occurrence threshold of five occurrences ( n  = 5) (see Fig.  7 , right side). In the figure, the size of a node corresponds to the frequency of a keyword co-selected in review studies in mathematics education. The distance between any two keywords reflects their relative strength and topic similarity. Nodes within the same colour cluster indicate similar topics among these publications.

The red cluster comprises 11 closely related items, including ‘mathematics, meta-analysis, mathematics achievement, intervention, scoping review, bibliometric analysis, review, technology, learning disabilities, children, and math anxiety’. The green cluster emerges as the second prominent cluster, featuring 8 interrelated items such as ‘mathematics education, systematic review, systematic literature review, literature review, teacher education, education, teaching, and flipped classroom’. Lastly, the blue cluster consists of 4 items, namely ‘math, science, early childhood, and identity’.

Co-occurrence analysis of author keywords

Notably, the most frequently cited author keyword was ‘mathematics education’ ( n  = 55), followed by ‘systematic review’ ( n  = 44), ‘mathematics’ ( n  = 41), ‘meta-analysis’ ( n  = 34), ‘systematic literature review’ ( n  = 14), ‘literature review’ ( n  = 11), ‘teacher education’ ( n  = 9), ‘mathematics achievement’ ( n  = 8), ‘intervention’ ( n  = 6), ‘education’ ( n  = 6), ‘teaching’ ( n  = 6), ‘science’ ( n  = 6), ‘scoping review’ ( n  = 5), ‘bibliometric analysis’ ( n  = 5), ‘review’ ( n  = 5), ‘math’ ( n  = 5), ‘technology’ ( n  = 5), ‘flipped classroom’ ( n  = 5), ‘early childhood’ ( n  = 5), ‘children’ ( n  = 5), ‘identity’ ( n  = 5), ‘learning disabilities’ ( n  = 5), and ‘math anxiety’ ( n  = 6).

The keywords chosen by the authors highlighted the focus areas of reviews in mathematics education, emphasising themes such as mathematics achievement, teacher education, interventions, technology, and technology-enhanced approaches (e.g. flipped classrooms), special education, and early childhood education. Furthermore, the author keywords reflected the prevalent review types in mathematics education, specifically systematic reviews and meta-analyses. Additionally, they highlighted the interdisciplinary nature of reviews in mathematics education, encompassing both mathematics education and science education.

Furthermore, we conducted distinct author keyword co-occurrence analyses for review studies published within the periods of 2019 to 2023 and those preceding 2019, aiming to discern temporal trends in author keywords, particularly in recent years. The analysis yielded 606 keywords for the 2019–2023 period and 144 keywords for the period before 2019 (see Table  8 for the most popular 15 author keywords). A noteworthy disparity in prevalent keywords was observed between the two temporal segments. While predominant keyword regarding the review types prior to 2019 was meta-analysis, followed by literature review and systematic review, over the past five years, additional keywords such as scoping review and bibliometric analysis emerged, signalling an augmentation in the diversity of review types and methodologies. The findings indicated a notable increase in the popularity of systematic reviews over the past five years.

4.2.3 Citation analysis

To explore the most cited publications and journals in mathematics education, we conducted a citation analysis based on the units of analysis in documents and sources.

Citation and document analysis

The analysis of the 259 review papers in mathematics education included in this study indicated that they received a total of 7,050 citations between 1996 and 2023, averaging 251.79 citations per year and 27.22 citations per paper. Notably, 67% of these citations were received in the last five years (2019–2023).

The threshold for the minimum number of citations of documents was set at one, which 221 review studies out of 259 met. Figure  8 visualises the network between these review papers with the largest citation links and Table  9 shows the most cited documents. Not all the studies listed in Table  9 are among the top 10 studies with the highest TLS. Among them, only Gersten et al. ( 2009 ), Cheung and Slavin ( 2008 ), and Slavin and Lake ( 2008 ) are within the top 10 review studies in mathematics education with the highest TLS. While highly cited documents are influential in terms of direct references, the TLS metric provides additional insights into the collaborative relationships and connections between researchers and their work, which may not always correlate perfectly with citation counts as seen in our findings.

Our results showed that the largest number of citation links were for meta-analyses and systematic review studies. The most prominent review type among the most cited studies listed in Table  9 is meta-analysis ( n  = 6), followed by literature review ( n  = 2), systematic review ( n  = 1), and narrative review ( n  = 1). This result indicates the potential of meta-analysis studies in terms of citation performance. Most of these review studies were primarily published in high-ranking educational review journals ( n  = 6). Other review papers published in teacher education ( n  = 2), psychology ( n  = 1), and behavioural science and neuroscience journals ( n  = 1). These ten most cited review articles were all published in SSCI journals over a decade ago. Regarding research topics in the most cited papers, the dominant topics were mathematics achievement, content knowledge, working memory, learning disabilities, and educational technologies.

Specifically, we analysed the citation trends of the most cited 10 review papers over time and separately for the first five years after publication and the past five years (2019–2023). The results indicate a significant increase in the citations review studies have received in the last five years. We found that eight out of the ten most cited papers received more citations in the past five years (2019–2023) than in the first five years after their publication. The analysis revealed that the average annual citations for each paper ranged from 7 to 30. While the majority of these review studies ( n  = 8) received the least citations in the year of their publication, they received the most citations on average approximately 12 years after publication. This indicates that the peak citation period for review articles in mathematics education extends beyond the first decade following their publication.

Additionally, we investigated the ‘Enriched Cited References’ feature, which provides insight into why an author cited a particular reference; this beta enhancement is only available in selected journals (Clarivate, 2024 ). These references are presented to aid readers in quickly assessing sections of a review paper, allowing them to identify the most closely related or impactful references and infer their purpose. Articles containing enriched cited references are marked with the following labels (Clarivate, 2024 ):

Previously published research that contextualizes the current study within an academic domain.

References that supply the datasets, methodologies, concepts, and ideas directly utilized by the author or upon which the author’s work relies.

References introduced because the current study engages in a more thorough discussion.

References cited by the current study as yielding similar results. This may encompass methodological similarities or, in certain instances, replication of findings.

References noted by the current study as presenting contrasting results. This may also involve disparities in methodology or sample differences, influencing the outcomes.

The results, displayed in Table  10 , pertain to the classification of references based on the Enriched Cited References analysis conducted automatically by WoS. These results suggest that the most cited review studies in mathematics education were predominantly utilized by researchers to establish the background for their own research. Furthermore, these reviews also frequently employed to shape the discussion within the papers. In addition, some researchers utilize the mentioned most cited review studies to establish a conceptual, theoretical, or methodological basis. While the limited number of the studies cited these reviews to support their findings, they were not used to present opposing evidence. This suggests a reliance on existing literature review studies to inform, validate, or potentially challenge new research within the field.

Citation and source analysis

We conducted a citation source analysis and present the citation network map for the journals in Fig.  9 , listing the top 15 journals in Table  11 based on the citation and TLS metrics to represent the frequency of citations between articles in any two journals. The threshold for the minimum number of documents citing a source was one, and 103 records met the minimum number of citations of a source, also set at one. The network map shown in Fig.  9 indicates prominent clusters. The red cluster included 23 items (mostly special education, educational psychology, and educational review journals). The blue cluster included 16 items (predominantly educational psychology, educational technology, and educational review journals). The green cluster comprised 17 items (including mathematics and mathematics education journals, educational technology journals, and educational psychology journals).

The number of articles and the distribution of journals across various research fields were as follows: 25 educational sciences journals (43 papers), 20 psychology and educational psychology journals (41 papers), 15 special education journals (32 papers), 12 mathematics education journals (52 papers), 10 educational review journals (41 papers), 9 educational technology journals (28 papers), 3 mathematics journals (14 papers), and 9 other journals (8 articles).

Our findings indicate that ZDM– Mathematics Education ( n  = 16) has, so far, published the most review studies focusing on mathematics education, which is not unexpected due to the origin of the journal as a review journal publishing only special issues, for which a review article is compulsory in each issue. This was followed by Frontiers in Psychology ( n  = 14), Educational Research Review ( n  = 13), and Mathematics ( n  = 10) (see Table  11 for the top 15 journals).

The results highlighted that the most frequently cited papers were often published in specific educational review journals (e.g. Review of Educational Research , Educational Research Review , and Educational Psychology Review ), psychology and educational psychology journals (e.g. Frontiers in Psychology , Educational Psychology Review , European Journal of Cognitive Psychology , and Psychological Bulletin ), special education journals (e.g. Exceptional Children , Learning Disabilities Research & Practice , Learning Disability Quarterly , and Remedial and Special Educati on), educational technology journals (e.g. Computers & Education , Journal of Computer Assisted Learning , and Education and Information Technologies ), and mathematics and mathematics education journals (e.g. ZDM– Mathematics Education , Educational Studies in Mathematics , and Mathematics ).

Although the most visible mathematics education journals in citation network map were ZDM– Mathematics Education and Educational Studies in Mathematics (see Fig.  9 ), as mentioned earlier, twelve mathematics education journals provided platforms for review studies. These were ZDM– Mathematics Education ( n  = 16), Educational Studies in Mathematics ( n  = 5), International Journal of Science and Mathematics Education ( n  = 5), International Journal of Mathematical Education in Science and Technology ( n  = 5), International Electronic Journal of Mathematics Education ( n  = 3), Mathematics Education Research Journal ( n  = 3), International Journal for Technology in Mathematics Education ( n  = 3), International Journal of Education in Mathematics, Science and Technology ( n  = 3), Journal for Research in Mathematics Education ( n  = 2), Canadian Journal of Science, Mathematics and Technology Education ( n  = 1), Journal für Mathematik-Didaktik ( n  = 1), and Research in Mathematics Education ( n  = 1).

5 Discussion, conclusions, and limitations

In this study, we conducted a meta-review of literature review studies in mathematics education, enriched by a comprehensive bibliometric analysis. This paper significantly contributes to scholarly discourse by unravelling nuanced research trends, the most common review methodologies, and prevalent theoretical approaches in review studies in mathematics education. Based on content and bibliometric analysis, it delves into the research foci, providing an understanding of the relevant academic landscape. Additionally, it illuminates intricate connections among researchers, countries, and journals, elucidating collaborative networks in mathematics education research.

5.1 Insights from the meta-review and implications

The findings revealed a significant increase in the number of literature reviews in mathematics education, particularly in the past five years; 79% of the reviews we examined were published during this period. Multiple factors may have contributed to this surge, including researchers’ increased publication output during the pandemic (Cevikbas & Kaiser, 2023 ; Nane et al., 2023 ), challenges in collecting empirical data during the pandemic crisis (Uleanya & Yu, 2023 ), the relatively high citation rates associated with literature review studies, the growing prestige of educational review journals based on their increased impact factors (Miranda & Garcia-Carpintero, 2018 ), and the publication of review-oriented special issues in mathematics education journals.

Our findings revealed a prevalence of systematic reviews and meta-analyses; however, researchers also conducted diverse types of reviews, including scoping reviews, critical reviews, narrative reviews, theoretical reviews, and tutorial reviews. This methodological diversity is important as the advantages of one method can potentially overcome the disadvantages of another and combining different approaches can mitigate disadvantages (Taherdoost, 2023 ). Furthermore, our study revealed that rapid reviews, meta-reviews, umbrella reviews, mapping reviews, mixed-methods reviews, integrative reviews, interpretative reviews, iterative reviews, meta-syntheses, descriptive reviews, realist reviews, selective reviews, algorithmic reviews, and computational reviews indexed in WoS were not represented in mathematics education. The well-established PRISMA guidelines offer a defined framework for systematic reviews and meta-analyses to assist researchers in conducting reviews while adhering to quality and transparency criteria (Moher et al., 2009 ; Page et al., 2021 ). This adherence may have encouraged researchers to undertake such reviews, and future advancements in the development of specific guidelines and methodologies for each review type may further motivate researchers to conduct other types of reviews in mathematics education more frequently.

There were nuanced overlaps between the review types, leading to ambiguous distinctions. For instance, the structural similarity between systematic reviews and scoping reviews has led to misunderstandings. Munn et al. ( 2018 ) confirm inconsistency and confusion regarding the differentiation between scoping reviews and systematic reviews and offered guidelines for this decision-making process: a systematic review is preferable when addressing specific questions regarding the feasibility, appropriateness, significance, or efficacy of a specific treatment or practice. However, if the authors intend to demarcate the research field and explore its potential size and scope, a scoping review is more appropriate. Grant and Booth ( 2009 ) and Munn et al. ( 2018 ) clarified that a scoping review is preparation for a systematic literature review, not a deep study for a systematic literature review. The diverse taxonomies proposed by researchers have contributed to this complexity, with some employing various terms for similar review characteristics, and others applying the same terms to studies with distinct review attributes. Consequently, a consensus regarding the categorisation of review studies, both in a broad context and specifically in mathematics education, remains elusive. We also observed instances of researchers labelling their reviews inaccurately. However, we refrained from judging the appropriateness of these terminologies as they fall outside the scope of our study and may be difficult to justify due to the ambiguity of the current typologies. Borges Migliavaca et al. ( 2020 ) expressed a similar concern, highlighting substantial disparities in review studies concerning their conceptualisation, conduct, reporting, risk of bias assessment, and data synthesis. They called for the evidence synthesis community to promptly develop guidance and reporting standards for review studies. Future researchers could potentially examine inconsistencies in the conducting of review studies and their categorisation in mathematics education. In this study, we distilled the various existing types of review studies to provide clear explanations of the main review types and to help researchers and readers understand the key characteristics of various review studies (see Chap. 2).

An additional noteworthy consideration pertains to the sample sizes of review studies. A prevalent discourse considers the appropriate number of studies to be included in a review, but establishing such a minimum or maximum number may be challenging and not appropriate because this depends on various contextual factors, such as the research area, topic, inclusion/exclusion criteria, and applied protocols. For example, in technical terms, a systematic review can be conducted with as few as two studies or as many as a thousand. A review study with a small sample (e.g. two or three studies) may be due to the literature search methods used or insufficient number of existing studies in a particular field, suggesting a limited demand for such a review. As previously noted, the primary function of review studies is to inform readers in the relevant field about published studies to address the challenge posed by an increasing number of studies and to identify trends and research gaps (Fusar-Poli & Radua, 2018 ). Conversely, although it is technically feasible to include a substantial number of studies in a review (e.g. 1,000 or 2,000), conducting a comprehensive analysis (e.g. content analysis) of such a large dataset can present major time, cost, storage, memory, bias, and security challenges (Cohen et al., 2015 ). Nevertheless, the findings of our study provide insight into this issue. Notably, the sample size of the studies we analysed varied from 8 to 3,485, with an average of 99. Notably, most of these studies (78%) had sample sizes of less than 100, with an average of 34. Although this observation does not serve as a prescriptive recommendation, it offers valuable insights into the typical sample sizes with which mathematics education researchers have tended to work in the past.

Furthermore, as evidenced by our findings, literature reviews may serve various purposes, such as assessing the use of theoretical models or conceptual and methodological approaches, or advancing new theories, concepts, or research models through critical appraisal of previous research within a specific subject area (Cooper, 1988 ). However, our findings also indicate that it is not common in practice to use or develop a theoretical or conceptual framework in mathematics education review studies. Only 24% of the reviewed studies explicitly reported employing a specific framework, and very few sought to formulate a framework based on the literature under scrutiny. The results highlighted the researchers’ interest in frameworks related to technology, knowledge, and competence models. A few studies incorporated grand theories, such as constructivism, sociocultural theory, and cognitive development theory.

It is remarkable that despite focusing on mathematics education, there is a notable scarcity of review studies employing content-specific frameworks in mathematics education, such as those centred on problem-solving, reasoning, and mathematical thinking. Only a minority of the studies used frameworks related to mathematical modelling and mathematical content knowledge. This observation may reflect a gap in the literature, suggesting a need for greater integration of domain-specific frameworks into review studies in mathematics education to enhance the depth and specificity of the studies. Moreover, this trend prompts a critical examination of potential underlying factors. One plausible explanation lies in the interdisciplinary nature of review studies in mathematics education, which draws contributions from diverse fields including psychology, educational technology, special education, and neuroscience. The diverse disciplinary backgrounds of the researchers may influence their preferences for frameworks that are not necessarily specific to mathematics education but rather draw from broader fields.

5.2 Insights from the bibliometric analyses and implications

The bibliometric analysis revealed contributions to mathematics education, with 761 authors from 50 countries conducting review studies. In future studies, researchers may consider conducting detailed analyses of how these initiatives have influenced the landscape of mathematics education, examining their specific impacts on various subfields, and assessing their overall influence.

Our findings reveal a notable participation in literature review studies within mathematics education by scholars from diverse backgrounds, including educational psychologists, mathematics educators, and specialists in special education and neuroscience. This multidisciplinary engagement underscores the broader interest of researchers beyond the field of mathematics education. Notably, co-authorship connections within US institutions were the most extensive. The leading countries that published review studies included the US, Germany, China, Australia, and England. A robust network emerged among researchers in North America, Europe, Asia, and Australia, emphasising collaboration opportunities that warrant exploration by African and South American researchers.

Systematic reviews and meta-analyses stood out as the predominant review types in mathematics education, both in terms of the number of publications and citation counts. Systematic reviews offer rigorous and comprehensive syntheses of existing literature on specific research questions, providing valuable insights, identifying gaps in knowledge, and informing evidence-based decision-making in various fields. Moreover, meta-analyses enhance statistical power, resolve conflicting findings, and offer more precise estimates of effect sizes by combining data from various sources. However, there is a discernible need to diversify the types of reviews conducted in mathematics education.

The findings underscore a significant surge in both the quantity of review studies and their citation counts within mathematics education especially over the recent five-year period (2019–2023). This trend suggests a prevalent practice among authors to draw upon previously published reviews to contextualize their own studies, frequently engaging in discussions and citing references to corroborate or challenge existing findings. Such reliance on established literature highlights the discipline’s emphasis on leveraging prior knowledge to inform and substantiate new research endeavours.

The most cited review papers were associated with specific educational review journals, educational psychology journals, special education journals, educational technology journals, and mathematics education journals, further highlighting the interdisciplinary nature of impactful research in the field. The results revealed that ZDM– Mathematics Education , Educational Studies in Mathematics , International Journal of Science and Mathematics Education , and International Journal of Mathematical Education in Science and Technology were the key mathematics education journals committed to publishing review studies. The performance of these journals, particularly in recent years, reflects the escalating significance of review studies in mathematics education. Nevertheless, the limited visibility of some mathematics education journals in publishing review studies could be attributed, among other factors, to their restricted representation in the WoS database or to the overall small number of studies published yearly in particular mathematics education journals.

Prominent research topics in mathematics education review studies are digital technologies, technology-enhanced approaches (e.g. flipped classrooms), teacher education, mathematics achievement, early childhood education, and learning disabilities. Recent technological advances, including artificial intelligence and augmented/virtual reality, may soon attract mathematics education researchers’ attention to emerging technologies (Cevikbas, Bulut et al., 2023 ; Cevikbas, Greefrath et al., 2023 ). In addition to technology-enhanced mathematics education and special education, researchers have also explored the cognitive and affective aspects of learning and teaching mathematics.

In short, the absence of high-quality research syntheses may impede theoretical and conceptual advances within mathematics education (Webster & Watson, 2002 ). Therefore, future researchers may endeavour to develop discipline-specific standards and guidelines for conducting various types of review studies in mathematics education. Moreover, they could focus on expanding the content of mathematics education journals to accommodate a greater number of review studies. The scientific influence of review journals may also provide an opportunity to establish a dedicated review journal with a pronounced focus on mathematics education.

5.3 Limitations and conclusion

Finally, we want to point out that in this comprehensive meta-review, enriched by bibliometric analysis, we meticulously compiled and scrutinised the largest dataset of reviews in mathematics education available within the WoS database. Although this was a substantial sample ( n  = 259) that was reasonably representative of published review studies in mathematics education, it is important to acknowledge certain limitations. Our search was confined to WoS, and we specifically focused on review articles published in English. It is worth noting that the characteristics of review studies published in journals, international handbooks, or conference proceedings not indexed in WoS or published in a language other than English could potentially differ from those we examined. In addition, despite studies indexed in WoS theoretically being of high quality, we identified inconsistencies and variability in the review studies we examined, and it is possible that a more extensive search would have yielded different results.

In conclusion, we advocate producing high-quality review papers that adeptly synthesise available knowledge to improve professional practice (Templier & Paré, 2015 ). Such efforts may further advance mathematics education and contribute to the continuous improvement of teaching and learning activities, despite the demanding nature of comprehensive review studies.

Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology , 8 (1), 19–32.

Article   Google Scholar  

Becker, L. A., & Oxman, A. D. (2008). Overviews of reviews. In J. P. T. Higgins, & S. Green (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 607–631). Wiley.

Birkle, C., Pendlebury, D. A., Schnell, J., & Adams, J. (2020). Web of Science as a data source for research on scientific and scholarly activity. Quantitative Science Studies , 1 (1), 363–376.

Boote, D., & Beile, N. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher , 34 (6), 3–15.

Borges Migliavaca, C., Stein, C., Colpani, V., et al. (2020). How are systematic reviews of prevalence conducted? A methodological study. BMC Medical Research Methodology , 20 , 96.

Cevikbas, M., & Kaiser, G. (2023). Can flipped classroom pedagogy offer promising perspectives for mathematics education on pandemic-related issues? A systematic literature review. ZDM–Mathematics Education , 55 , 177–191.

Cevikbas, M., Kaiser, G., & Schukajlow, S. (2022). A systematic literature review of the current discussion on mathematical modelling competencies: State-of-the-art developments in conceptualizing, measuring, and fostering. Educational Studies in Mathematics , 109 (2), 205–236.

Cevikbas, M., Greefrath, G., & Siller, H. S. (2023). Advantages and challenges of using digital technologies in mathematical modelling education–a descriptive systematic literature review. Frontiers in Education , 8 , 1142556.

Cevikbas, M., Bulut, N., & Kaiser, G. (2023). Exploring the benefits and drawbacks of AR and VR technologies for learners of mathematics: Recent developments. Systems , 11 (5), 244.

Cevikbas, M., König, J., & Rothland, M. (2024). Empirical research on teacher competence in mathematics lesson planning: Recent developments. ZDM–Mathematics Education , 56 , 101–113.

Cheung, A. C., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review , 9 , 88–113.

Chigbu, U. E., Atiku, S. O., & Du Plessis, C. C. (2023). The science of literature reviews: Searching, identifying, selecting, and synthesising. Publications , 11 (1), 2.

Clarivate (2024). Citation context in Web of Science . URL.

Cochrane, A. L. (1979). 1931–1971: A critical review with particular reference to the medical profession. In G. Teeling Smith, & N. Wells (Eds.), Medicines for the year 2000 (pp. 2–12). Office of Health Economics.

Cohen, B., Vawdrey, D. K., Liu, J., Caplan, D., Furuya, E. Y., Mis, F. W., & Larson, E. (2015). Challenges associated with using large data sets for quality assessment and research in clinical settings. Policy Politics & Nursing Practice , 16 (3–4), 117–124.

Cooper, H. M. (1988). Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society , 1 (1), 104–126.

Google Scholar  

Cooper, H., Nye, B., Charlton, K., Lindsay, J., & Greathouse, S. (1996). The effects of summer vacation on achievement test scores: A narrative and meta-analytic review. Review of Educational Research , 66 (3), 227–268.

Creswell, J. W. (2013). Qualitative inquiry and research design; choosing among five approaches . Sage.

Davies, P. (2000). The relevance of systematic reviews to educational policy and practice. Oxford Review of Education , 26 (3–4), 365–378.

Depaepe, F., Verschaffel, L., & Kelchtermans, G. (2013). Pedagogical content knowledge: A systematic review of the way in which the concept has pervaded mathematics educational research. Teaching and Teacher Education , 34 , 12–25.

DeStefano, D., & LeFevre, J. A. (2004). The role of working memory in mental arithmetic. European Journal of Cognitive Psychology , 16 (3), 353–386.

Friso-van den Bos, I., Van der Ven, S. H., Kroesbergen, E. H., & Van Luit, J. E. (2013). Working memory and mathematics in primary school children: A meta-analysis. Educational Research Review , 10 , 29–44.

Fusar-Poli, P., & Radua, J. (2018). Ten simple rules for conducting umbrella reviews. Evidence-Based Mental Health , 21 (3), 95.

Geary, D. C. (1996). Sexual selection and sex differences in mathematical abilities. Behavioral and Brain Sciences , 19 (2), 229–247.

Geng, Y., Chen, W., Liu, Z., Chiu, A. S., Han, W., Liu, Z., & Cui, X. (2017). A bibliometric review: Energy consumption and greenhouse gas emissions in the residential sector. Journal of Cleaner Production , 159 , 301–316.

Gersten, R., Chard, D. J., Jayanthi, M., Baker, S. K., Morphy, P., & Flojo, J. (2009). Mathematics instruction for students with learning disabilities: A meta-analysis of instructional components. Review of Educational Research , 79 (3), 1202–1242.

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal , 26 (2), 91–108.

Hart, C. (2018). Doing a literature review: Releasing the research imagination (2nd edition). Sage.

Kaiser, G., & Schukajlow, S. (2024). Literature reviews in mathematics education and their significance to the field. ZDM–Mathematics Education , 56 , 1–3.

Ketcham, C. M., & Crawford, J. M. (2007). The impact of review articles. Laboratory Investigation , 87 (12), 1174–1185.

Korom, P. (2019). A bibliometric visualization of the economics and sociology of wealth inequality: A world apart? Scientometrics , 118 , 849–868.

Kunisch, S., Denyer, D., Bartunek, J. M., Menz, M., & Cardinal, L. B. (2023). Review research as scientific inquiry. Organizational Research Methods , 26 (1), 3–45.

Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review , 22 , 215–243.

MacEntee, M. I. (2019). A typology of systematic reviews for synthesising evidence on health care. Gerodontology , 36 (4), 303–312.

Martin, D. B. (2009). Researching race in mathematics education. Teachers College Record , 111 (2), 295–338.

Martínez, M. A., Cobo, M. J., Herrera, M., & Herrera-Viedma, E. (2015). Analyzing the scientific evolution of social work using science mapping. Research on Social Work Practice , 25 (2), 257–277.

Miranda, R., & Garcia-Carpintero, E. (2018). Overcitation and overrepresentation of review papers in the most cited papers. Journal of Informetrics , 12 (4), 1015–1030.

Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reporting of meta-analysis of randomized controlled trials: The QUOROM statement. Lancet , 354 , 1896–1900.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Plos Medicine , 6(7), e1000097.

Munn, Z., Peters, M. D., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology , 18 , 1–7.

Nane, G. F., Robinson-Garcia, N., van Schalkwyk, F., & Torres-Salinas, D. (2023). COVID-19 and the scientific publishing system: Growth, open access and scientific fields. Scientometrics , 128 (1), 345–362.

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., McDonald, S., & Moher, D. (2021). (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ , 372 (71), 1–9.

Paré, G., Trudel, M. C., Jaana, M., & Kitsiou, S. (2015). Synthesizing information systems knowledge: A typology of literature reviews. Information & Management , 52 (2), 183–199.

Radevic, L., & Milovanovic, I. (2023). Current trends in math anxiety research: A bibliometric approach. International Journal of Science and Mathematics Education . https://doi.org/10.1007/s10763-023-10424-4 .

Richardson, W. S., Wilson, M. C., Nishikawa, J., & Hayward, R. S. (1995). The well-built clinical question: A key to evidence-based decisions. ACP Journal Club , 123 (3), A12–A13.

Samnani, S. S., Vaska, M., Ahmed, S., & Turin, T. C. (2017). Review typology: The basic types of reviews for synthesizing evidence for the purpose of knowledge translation. Journal of the College of Physicians and Surgeons Pakistan , 27 (10), 635–641.

Schryen, G., & Sperling, M. (2023). Literature reviews in operations research: A new taxonomy and a meta review. Computers & Operations Research , 106269.

Schryen, G., Wagner, G., Benlian, A., & Paré, G. (2020). A knowledge development perspective on literature reviews: Validation of a new typology in the IS field. Communications of the AIS , 46 , 134–186.

Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., & Bouter, L. M. (2007). Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology , 7 , 1–7.

Slavin, R. E., & Lake, C. (2008). Effective programs in elementary mathematics: A best-evidence synthesis. Review of Educational Research , 78 (3), 427–515.

Song, Y., Chen, X., Hao, T., Liu, Z., & Lan, Z. (2019). Exploring two decades of research on classroom dialogue by using bibliometric analysis. Computers & Education , 137 , 12–31.

Suseelan, M., Chew, C. M., & Chin, H. (2022). Research on Mathematics Problem solving in Elementary Education conducted from 1969 to 2021: A bibliometric review. IJEMST , 10 (4), 1003–1029.

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: Exploring review types and associated information retrieval requirements. Health Information & Libraries Journal , 36 (3), 202–222.

Taherdoost, H. (2023). Towards nuts and bolts of conducting literature review: A typology of literature review. Electronics , 12 (4), 800.

Templier, M., & Paré, G. (2015). A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems , 37 (1), 6.

Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D., Horsley, T., Weeks, L., Hempel, S., et al. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine , 169 (7), 467–473.

Uleanya, C., & Yu, K. (2023). Data collection in times of pandemic: A self-study and revisit of research practices during a crisis. Sage Open , 13 (1), 21582440231160698.

van Eck, N., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics , 84 (2), 523–538.

Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly , 26 (2), xiii–xxiii.

Wei, Y., Zhang, Q., Guo, J., & Chen, M. (2023). Learning to teach through noticing: A bibliometric review of teacher noticing research in mathematics education during 2006–2021. Humanities and Social Sciences Communications , 10 (1), 1–15.

Zupic, I., & Cater, T. (2015). Bibliometric methods in management and organization. Organizational Research Methods , 18 (3), 429–472.

Download references

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Humboldt-Universität zu Berlin Institute of Educational Sciences, Unter den Linden 6, 10099, Berlin, Germany

Mustafa Cevikbas

Faculty of Education, Universität Hamburg, Von-Melle-Park 8, 20146, Hamburg, Germany

Gabriele Kaiser

Nord University, Bodø, Norway

Institute of Mathematics Education and Computer Science Education, Universität Münster, Henriette-Son Str. 19, 48149, Münster, Germany

Stanislaw Schukajlow

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mustafa Cevikbas .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The electronic supplementary material includes the list of the reviewed studies and a sample of the coding manual.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cevikbas, M., Kaiser, G. & Schukajlow, S. Trends in mathematics education and insights from a meta-review and bibliometric analysis of review studies. ZDM Mathematics Education (2024). https://doi.org/10.1007/s11858-024-01587-7

Download citation

Accepted : 02 May 2024

Published : 15 May 2024

DOI : https://doi.org/10.1007/s11858-024-01587-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Bibliometric analysis
• Critical review
• Literature review
• Mapping review
• Mathematics education
• Meta-review
• Narrative review
• Systematic review
• Scoping review
• Theoretical review
• Tutorial review
• Umbrella review
• Find a journal
• Publish with us
• Track your research

Enhancing data integrity in Electronic Health Records: Review of methods for handling missing data

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Amin Vahdati
• For correspondence: [email protected]
• ORCID record for Sarah Cotterill
• ORCID record for Antonia Marsden
• ORCID record for Evangelos Kontopantelis
• Info/History
• Preview PDF

Introduction Electronic Health Records (EHRs) are vital repositories of patient information for medical research, but the prevalence of missing data presents an obstacle to the validity and reliability of research. This study aimed to review and category ise methods for handling missing data in EHRs, to help researchers better understand and address the challenges related to missing data in EHRs.

Materials and Methods This study employed scoping review methodology. Through systematic searches on EMBASE up to October 2023, including review articles and original studies, relevant literature was identified. After removing duplicates, titles and abstracts were screened against inclusion criteria, followed by full-text assessment. Additional manual searches and reference list screenings were conducted. Data extraction focused on imputation techniques, dataset characteristics, assumptions about missing data, and article types. Additionally, we explored the availability of code within widely used software applications.

Results We reviewed 101 articles, with two exclusions as duplicates. Of the 99 remaining documents, 21 underwent full-text screening, with nine deemed eligible for data extraction. These articles introduced 31 imputation approaches classified into ten distinct methods, ranging from simple techniques like Complete Case Analysis to more complex methods like Multiple Imputation, Maximum Likelihood, and Expectation-Maximization algorithm. Additionally, machine learning methods were explored. The different imputation methods, present varying reliability. We identified a total of 32 packages across the four software platforms (R, Python, SAS, and Stata) for imputation methods. However, it’s significant that machine learning methods for imputation were not found in specific packages for SAS and Stata. Out of the 9 imputation methods we investigated, package implementations were available for 7 methods in all four software platforms.

Conclusions Several methods to handle missing data in EHRs are available. These methods range in complexity and make different assumptions about the missing data mechanisms. Knowledge gaps remain, notably in handling non-monotone missing data patterns and implementing imputation methods in real-world healthcare settings under the Missing Not at Random assumption. Future research should prioritize refining and directly comparing existing methods.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

Amin Vahdati, Pre-Doctoral Fellow (NIHR 129296), is funded by the National Institute for Health and Care Research (NIHR) for this research project.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Data Availability

All data produced in the present work are contained in the manuscript.

View the discussion thread.

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One

Subject Area

• Epidemiology
• Addiction Medicine (323)
• Allergy and Immunology (627)
• Anesthesia (163)
• Cardiovascular Medicine (2367)
• Dentistry and Oral Medicine (288)
• Dermatology (206)
• Emergency Medicine (379)
• Endocrinology (including Diabetes Mellitus and Metabolic Disease) (835)
• Epidemiology (11765)
• Forensic Medicine (10)
• Gastroenterology (702)
• Genetic and Genomic Medicine (3732)
• Geriatric Medicine (348)
• Health Economics (633)
• Health Informatics (2393)
• Health Policy (930)
• Health Systems and Quality Improvement (896)
• Hematology (340)
• HIV/AIDS (781)
• Infectious Diseases (except HIV/AIDS) (13303)
• Intensive Care and Critical Care Medicine (767)
• Medical Education (365)
• Medical Ethics (104)
• Nephrology (398)
• Neurology (3493)
• Nursing (198)
• Nutrition (523)
• Obstetrics and Gynecology (673)
• Occupational and Environmental Health (662)
• Oncology (1820)
• Ophthalmology (535)
• Orthopedics (218)
• Otolaryngology (287)
• Pain Medicine (232)
• Palliative Medicine (66)
• Pathology (446)
• Pediatrics (1032)
• Pharmacology and Therapeutics (426)
• Primary Care Research (420)
• Psychiatry and Clinical Psychology (3172)
• Public and Global Health (6137)
• Radiology and Imaging (1280)
• Rehabilitation Medicine and Physical Therapy (746)
• Respiratory Medicine (825)
• Rheumatology (379)
• Sexual and Reproductive Health (372)
• Sports Medicine (322)
• Surgery (401)
• Toxicology (50)
• Transplantation (172)
• Urology (145)

Long-term levels of protection of different types of immunity against the Omicron variant: a rapid literature review

Affiliations.

• 1 Institute of Global Health, University of Geneva, Geneva, Switzerland.
• 2 The GRAPH Network, Geneva, Switzerland.
• 3 HIV/AIDS Unit Division of Infectious Diseases, Geneva University Hospitals, Geneva, Switzerland.
• 4 Scuola Superiore Sant'Anna, Pisa, Italy.
• PMID: 38749028
• DOI: 10.57187/s.3732

Introduction: With the emergence of newer SARS-CoV-2 variants and their substantial effects on the levels and duration of protection against infection, an understanding of these characteristics of the protection conferred by humoral and cellular immunity can aid in the proper development and implementation of vaccine and safety guidelines.

Methods: We conducted a rapid literature review and searched five electronic databases weekly from 1 November 2021 to 30 September 2022. Studies that assessed the humoral or cellular immunity conferred by infection, vaccination or a hybrid (combination of both) in adults and risk groups (immunocompromised and older populations) were identified. Studies were eligible when they reported data on immunological assays of COVID-19 (related to vaccination and/or infection) or the effectiveness of protection (related to the effectiveness of vaccination and/or infection).

Results: We screened 5103 studies and included 205 studies, of which 70 provided data on the duration of protection against SARS-CoV-2 infection. The duration of protection of adaptive immunity was greatly impacted by Omicron and its subvariants: levels of protection were low by 3-6 months from exposure to infection/vaccination. Although more durable, cellular immunity also showed signs of waning by 6 months. First and second mRNA vaccine booster doses increased the levels of protection against infection and severe disease from Omicron and its subvariants but continued to demonstrate a high degree of waning over time.

Conclusion: All humoral immunities (infection-acquired, vaccine-acquired and hybrid) waned by 3-6 months. Cellular immunity was more durable but showed signs of waning by 6 months. Hybrid immunity had the highest magnitude of protection against SARS-CoV-2 infection. Boosting may be recommended as early as 3-4 months after the last dose, especially in risk groups.

Publication types

• COVID-19 Vaccines* / immunology
• COVID-19* / immunology
• COVID-19* / prevention & control
• Immunity, Cellular* / immunology
• Immunity, Humoral / immunology
• Immunization, Secondary
• SARS-CoV-2* / immunology
• Vaccination