literature review on organization

Organizing and Creating Information

• Citation and Attribution

What Is a Literature Review?

Review the literature, write the literature review, further reading, learning objectives, attribution.

This guide is designed to:

• Identify the sections and purpose of a literature review in academic writing
• Review practical strategies and organizational methods for preparing a literature review

A literature review is a summary and synthesis of scholarly research on a specific topic. It should answer questions such as:

• What research has been done on the topic?
• Who are the key researchers and experts in the field?
• What are the common theories and methodologies?
• Are there challenges, controversies, and contradictions?
• Are there gaps in the research that your approach addresses?

The process of reviewing existing research allows you to fine-tune your research question and contextualize your own work. Preparing a literature review is a cyclical process. You may find that the research question you begin with evolves as you learn more about the topic.

Once you have defined your research question , focus on learning what other scholars have written on the topic.

In order to  do a thorough search of the literature  on the topic, define the basic criteria:

• Databases and journals: Look at the  subject guide  related to your topic for recommended databases. Review the  tutorial on finding articles  for tips. 
• Books: Search BruKnow, the Library's catalog. Steps to searching ebooks are covered in the  Finding Ebooks tutorial .
• What time period should it cover? Is currency important?
• Do I know of primary and secondary sources that I can use as a way to find other information?
• What should I be aware of when looking at popular, trade, and scholarly resources ? 

One strategy is to review bibliographies for sources that relate to your interest. For more on this technique, look at the tutorial on finding articles when you have a citation .

Tip: Use a Synthesis Matrix

As you read sources, themes will emerge that will help you to organize the review. You can use a simple Synthesis Matrix to track your notes as you read. From this work, a concept map emerges that provides an overview of the literature and ways in which it connects. Working with Zotero to capture the citations, you build the structure for writing your literature review.

How do I know when I am done?

A key indicator for knowing when you are done is running into the same articles and materials. With no new information being uncovered, you are likely exhausting your current search and should modify search terms or search different catalogs or databases. It is also possible that you have reached a point when you can start writing the literature review.

Tip: Manage Your Citations

These citation management tools also create citations, footnotes, and bibliographies with just a few clicks:

Zotero Tutorial

Endnote Tutorial

Your literature review should be focused on the topic defined in your research question. It should be written in a logical, structured way and maintain an objective perspective and use a formal voice.

Review the Summary Table you created for themes and connecting ideas. Use the following guidelines to prepare an outline of the main points you want to make. 

• Synthesize previous research on the topic.
• Aim to include both summary and synthesis.
• Include literature that supports your research question as well as that which offers a different perspective.
• Avoid relying on one author or publication too heavily.
• Select an organizational structure, such as chronological, methodological, and thematic.

The three elements of a literature review are introduction, body, and conclusion.

Introduction

• Define the topic of the literature review, including any terminology.
• Introduce the central theme and organization of the literature review.
• Summarize the state of research on the topic.
• Frame the literature review with your research question.
• Focus on ways to have the body of literature tell its own story. Do not add your own interpretations at this point.
• Look for patterns and find ways to tie the pieces together.
• Summarize instead of quote.
• Weave the points together rather than list summaries of each source.
• Include the most important sources, not everything you have read.
• Summarize the review of the literature.
• Identify areas of further research on the topic.
• Connect the review with your research.
• DeCarlo, M. (2018). 4.1 What is a literature review? In Scientific Inquiry in Social Work. Open Social Work Education. https://scientificinquiryinsocialwork.pressbooks.com/chapter/4-1-what-is-a-literature-review/
• Literature Reviews (n.d.) https://writingcenter.unc.edu/tips-and-tools/literature-reviews/ Accessed Nov. 10, 2021

This guide was designed to: 

• Identify the sections and purpose of a literature review in academic writing 
• Review practical strategies and organizational methods for preparing a literature review​

Content on this page adapted from: 

Frederiksen, L. and Phelps, S. (2017).   Literature Reviews for Education and Nursing Graduate Students.  Licensed CC BY 4.0

• << Previous: EndNote
• Last Updated: May 24, 2024 10:37 AM
• URL: https://libguides.brown.edu/organize

Brown University Library  |  Providence, RI 02912  |  (401) 863-2165  |  Contact  |  Comments  |  Library Feedback  |  Site Map

Library Intranet

Literature Reviews

• "How To" Books
• Examples of Literature Reviews
• Collecting Resources for a Literature Review
• Organizing the Literature Review
• Writing the Literature Review
• Endnote This link opens in a new window
• Evaluating Websites

Organization

Organization of your Literature Review

What is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? What order should you present them?

Just like most academic papers, literature reviews must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper.

Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.

Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).

Conclusions/Recommendations: Discuss what you have drawn from reviewing the literature so far. Where might the discussion proceed?

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario and then three typical ways of organizing the sources into a review:

You've decided to focus your literature review on materials dealing with sperm whales. This is because you've just finished reading Moby Dick, and you wonder if that whale's portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980's. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Chronological

If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.

By publication

Order your sources chronologically by publication if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.

Another way to organize sources chronologically is to examine the sources under a trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Using this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.

Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a "chronological" and a "thematic" approach is what is emphasized the most: the development of the harpoon or the harpoon technology.

More authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as "evil" in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.

Methodological

A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the "methods" of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Once you've decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

Current Situation: Information necessary to understand the topic or focus of the literature review.

History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.

Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

• << Previous: Collecting Resources for a Literature Review
• Next: Writing the Literature Review >>
• Last Updated: Nov 2, 2021 12:11 PM
• URL: https://guides.library.stonybrook.edu/literature-review
• Request a Class
• Hours & Locations
• Ask a Librarian
• Special Collections
• Library Faculty & Staff

Library Administration: 631.632.7100

• Stony Brook Home
• Campus Maps
• Web Accessibility Information
• Accessibility Barrier Report Form

Comments or Suggestions? | Library Webmaster

Except where otherwise noted, this work by SBU Libraries is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License .

Literature Review Guide: How to organise the review

• What is a Literature Review?
• How to start?
• Search strategies and Databases
• Examples of Literature Reviews
• How to organise the review
• Library summary
• Emerald Infographic

How to structure your literature review (ignore the monotone voice as advice is good)

How to structure and write your literature review

• Chronological, ie. by date of publication or trend
• Methodological
• Use Cooper's taxonomy to explore and determine what elements and categories to incorporate into your review
• Revise and proofread your review to ensure your arguments, supporting evidence and writing is clear and precise

Cronin, P., Ryan, F. & Coughlan, M. (2008). Undertaking a literature review: A step-by-step approach . British Journal of Nursing, 17 (1), pp.38-43.

Different ways to organise a Literature Review

CHRONOLOGICAL (by date): This is one of the most common ways, especially for topics that have been talked about for a long time and have changed over their history. Organise it in stages of how the topic has changed: the first definitions of it, then major time periods of change as researchers talked about it, then how it is thought about today.

BROAD-TO-SPECIFIC : Another approach is to start with a section on the general type of issue you're reviewing, then narrow down to increasingly specific issues in the literature until you reach the articles that are most specifically similar to your research question, thesis statement, hypothesis, or proposal. This can be a good way to introduce a lot of background and related facets of your topic when there is not much directly on your topic but you are tying together many related, broader articles.

MAJOR MODELS or MAJOR THEORIES : When there are multiple models or prominent theories, it is a good idea to outline the theories or models that are applied the most in your articles. That way you can group the articles you read by the theoretical framework that each prefers, to get a good overview of the prominent approaches to your concept.

PROMINENT AUTHORS : If a certain researcher started a field, and there are several famous people who developed it more, a good approach can be grouping the famous author/researchers and what each is known to have said about the topic. You can then organise other authors into groups by which famous authors' ideas they are following. With this organisation it can help to look at the citations your articles list in them, to see if there is one author that appears over and over.

CONTRASTING SCHOOLS OF THOUGHT : If you find a dominant argument comes up in your research, with researchers taking two sides and talking about how the other is wrong, you may want to group your literature review by those schools of thought and contrast the differences in their approaches and ideas.

Ways to structure your Literature Review

Different ways to organise your literature review include:

• Topical order (by main topics or issues, showing relationship to the main problem or topic)
• Chronological order (simplest of all, organise by dates of published literature)
• Problem-cause-solution order
• General to specific order
• Known to unknown order
• Comparison and contrast order
• Specific to general order
• << Previous: Videos
• Next: Library summary >>
• Last Updated: Feb 27, 2024 4:07 PM
• URL: https://ait.libguides.com/literaturereview

The Literature Review: 5. Organizing the Literature Review

• 1. Introduction
• 2. Why Do a Literature Review?
• 3. Methods for Searching the Literature
• 4. Analysing the Literature
• 5. Organizing the Literature Review
• 6. Writing the Review

1. Organizing Principles

A literature review is a piece of discursive prose, not a list describing or summarizing one piece of literature after another. It should have a single organizing principle:

• Thematic - organize around a topic or issue
• Chronological - sections for each vital time period
• Methodological - focus on the methods used by the researchers/writers

4. Selected Online Resources

• Literature Review in Education & Behavioral Sciences This is an interactive tutorial from Adelphi University Libraries on how to conduct a literature review in education and the behavioural sciences using library databases
• Writing Literature Reviews This tutorial is from the Writing section of Monash University's Language and Learning Online site
• The Literature Review: A Few Tips on Conducting It This guide is from the Health Services Writing Centre at the University of Toronto
• Learn How to Write a Review of the Literature This guide is part of the Writer's Handbook provided by the Writing Center at the University of Wisconsin-Madison

2. Structure of the Literature Review

Although your literature review will rely heavily on the sources you read for its information, you should dictate the structure of the review. It is important that the concepts are presented in an order that makes sense of the context of your research project.

There may be clear divisions on the sets of ideas you want to discuss, in which case your structure may be fairly clear. This is an ideal situation. In most cases, there will be several different possible structures for your review.

Similarly to the structure of the research report itself, the literature review consists of:

• Introduction

Introduction - profile of the study

• Define or identify the general topic to provide the context for reviewing the literature
• Outline why the topic is important
• Identify overall trends in what has been published about the topic
• Identify conflicts in theory, methodology, evidence, and conclusions
• Identify gaps in research and scholarlship
• Explain the criteria to be used in analysing and comparing the literature
• Describe the organization of the review (the sequence)
• If necessary, state why certain literature is or is not included (scope)

Body - summative, comparative, and evaluative discussion of literature reviewed

For a thematic review:

• organize the review into paragraphs that present themes and identify trends relevant to your topic
• each paragraph should deal with a different theme - you need to synthesize several of your readings into each paragraph in such a way that there is a clear connection between the sources
• don't try to list all the materials you have identified in your literature search

From each of the section summaries:

• summarize the main agreements and disagreements in the literature
• summarize the general conclusions that have been drawn
• establish where your own research fits in the context of the existing literature

5. A Final Checklist

• Have you indicated the purpose of the review?
• Have you emphasized recent developments?
• Is there a logic to the way you organized the material?
• Does the amount of detail included on an issue relate to its importance?
• Have you been sufficiently critical of design and methodological issues?
• Have you indicated when results were conflicting or inconclusive and discussed possible reasons?
• Has your summary of the current literature contributed to the reader's understanding of the problems?

3. Tips on Structure

A common error in literature reviews is for writers to present material from one author, followed by information from another, then another.... The way in which you group authors and link ideas will help avoid this problem. To group authors who draw similar conclusions, you can use linking words such as:

• additionally

When authors disagree, linking words that indicate contrast will show how you have analysed their work. Words such as:

• on the other hand
• nonetheless

will indicate to your reader how you have analysed the material. At other times, you may want to qualify an author's work (using such words as specifically, usually, or generally ) or use an example ( thus, namely, to illustrate ). In this way you ensure that you are synthesizing the material, not just describing the work already carried out in your field.

Another major problem is that literature reviews are often written as if they stand alone, without links to the rest of the paper. There needs to be a clear relationship between the literature review and the methodology to follow.

• << Previous: 4. Analysing the Literature
• Next: 6. Writing the Review >>
• Last Updated: May 9, 2024 10:36 AM
• URL: https://libguides.uwi.edu/litreviewsoe

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE: Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 30, 2024 9:38 AM
• URL: https://libguides.usc.edu/writingguide

× The Libraries will be closed for Memorial Day Weekend from Friday-Monday, May 24-27. Enjoy the holiday!

× sherrill library will be closed from may 18 - september 2 due to construction. services by appointment: research consultations, library instruction, pick up appointments moriarty library will be open may 28 - august 16, monday - thursday from 10am-6pm, friday from 9am-noon. closed weekends and holidays. as always, our web resources are available 24/7. questions our chat and ask us services are available monday-thursday, 10am-6pm and friday 9am-noon., × spring break: monday, 03/13/2023 - sunday, 03/19/2023: library pickups are by appointment. need an appointment email us at sherrill library: [email protected] or moriarty library: [email protected], × alert mm/dd/yyyy: something is broken please contact us with questions., × alert 12/14/2023: ebsco allsearch is unavailable. we are working to fix this as quickly as we can. in the meanwhile, please try searching for articles from our proquest central database and for ebooks and books from our flo catalog . we're very sorry for the inconveniance. --> × welcome back our remote services guide has everything you need to know about library services we're offering this semester, including research help, study spaces, and more for other campus plans, see the lesley university covid-19 response. any other questions ask us, × welcome back our remote services guide has everything you need to know about library services we're offering this semester, including research help, study spaces, and more any other questions ask us.

• Literature Review
• What type of literature review should you write?
• Hide Menu Thingy Widget
• Additional Guides & Reading Materials

Library & Research Help

• Sherrill Library: 617-349-8850
• Moriarty Library: 617-349-8070
• Text: 617-340-8008
• Research Help:  Ask Us!
• Email:  [email protected]
• Art & Design Research Email: [email protected]

There are different methods to organize and present the materials collected for the literature review.

The list below goes over different organizational frameworks that can be used to present the research conducted. If you are not sure what method to use, check with your professor.​

• Chronological:  The chronological framework organizes the literature in the order in which they are published. For example, if you were writing about a specific teaching method, you would begin with the materials that first introduced the method. You would then follow with case studies applying that method. You would conclude your review with contemporary papers that may even give a historical perspective on the method from when it was first conceived and how it is applied today.
• by publication:  This framework is useful if you notice a series of articles that are written in response to one another that are all within one publication. You still follow chronological order, but you break it so that the articles responding to one another are grouped together.
• by trend:  This framework looks at specific trends and organizes them chronologically. For example, if you were looking at the history of assistive technology in helping students with disabilities, you may organize the reviews by what disability was being treated, and then present the history of using assistive technology to treat that particular disability in chronological order.
• Thematic:  The thematic framework is similar to organizing by trend, except that you are not organizing the reviews in the order that they were published. This does not mean that you do not consider the timeline for how a topic or issue developed, but that you will not focus on organizing your reviews chronologically. Rather, the emphasis will be on the themes you find within the topic or issue — such as commonalities — and from there you fit your reviews into the separate ideas in which they fit. For example, if the review topic was arts-based research, your review may focus on different ways artistic inquiry was used to understand the creative process, focusing then on the concepts rather than the development.
• Methodological:  The method or practice applied in a case study can be the basis for organizing a literature review. This framework focuses on how the author(s) or the person(s) administering a study applied similar methods as another study. As a result, the types of literature in a literature review that applies the methodological framework tends to review similar materials. For example, if you reviewed methods used to treat post traumatic stress syndrome, the review would organize the studies by the methods used to treat the patients and not the order that the studies were published.

Note: After choosing the organizational framework for the literature review, it should be easier to write because you should have a clear idea of what sections you need to include in the paper. For example, a chronological review will have subsections for each vital time period. A thematic review will have subtopics based upon factors that relate to the theme or issue.

In some cases the literature does not quite fit the framework you have chosen. In this case, you should determine where it makes sense to place the literature and confirm this choice with your professor.

• << Previous: Literature Review
• Next: Additional Guides & Reading Materials >>
• Last Updated: Dec 15, 2022 2:41 PM
• URL: https://research.lesley.edu/LitReview

Moriarty Library

Porter Campus 1801 Massachusetts Avenue Cambridge, MA 02140 617-349-8070

Sherrill Library

South Campus 89 Brattle Street Cambridge, MA 02138 617-349-8850

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

• Roughly how many sources should you include?
• What types of sources (books, journal articles, websites)?
• Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should you evaluate your sources?
• Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

• Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
• Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
• Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

• Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
• By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
• By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
• Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
• Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

• Current Situation: Information necessary to understand the topic or focus of the literature review.
• History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

How To Structure Your Literature Review

3 options to help structure your chapter.

By: Amy Rommelspacher (PhD) | Reviewer: Dr Eunice Rautenbach | November 2020 (Updated May 2023)

Writing the literature review chapter can seem pretty daunting when you’re piecing together your dissertation or thesis. As  we’ve discussed before , a good literature review needs to achieve a few very important objectives – it should:

• Demonstrate your knowledge of the research topic
• Identify the gaps in the literature and show how your research links to these
• Provide the foundation for your conceptual framework (if you have one)
• Inform your own  methodology and research design

To achieve this, your literature review needs a well-thought-out structure . Get the structure of your literature review chapter wrong and you’ll struggle to achieve these objectives. Don’t worry though – in this post, we’ll look at how to structure your literature review for maximum impact (and marks!).

But wait – is this the right time?

Deciding on the structure of your literature review should come towards the end of the literature review process – after you have collected and digested the literature, but before you start writing the chapter. 

In other words, you need to first develop a rich understanding of the literature before you even attempt to map out a structure. There’s no use trying to develop a structure before you’ve fully wrapped your head around the existing research.

Equally importantly, you need to have a structure in place before you start writing , or your literature review will most likely end up a rambling, disjointed mess. 

Importantly, don’t feel that once you’ve defined a structure you can’t iterate on it. It’s perfectly natural to adjust as you engage in the writing process. As we’ve discussed before , writing is a way of developing your thinking, so it’s quite common for your thinking to change – and therefore, for your chapter structure to change – as you write. 

Need a helping hand?

Like any other chapter in your thesis or dissertation, your literature review needs to have a clear, logical structure. At a minimum, it should have three essential components – an  introduction , a  body   and a  conclusion . 

Let’s take a closer look at each of these.

1: The Introduction Section

Just like any good introduction, the introduction section of your literature review should introduce the purpose and layout (organisation) of the chapter. In other words, your introduction needs to give the reader a taste of what’s to come, and how you’re going to lay that out. Essentially, you should provide the reader with a high-level roadmap of your chapter to give them a taste of the journey that lies ahead.

Here’s an example of the layout visualised in a literature review introduction:

Your introduction should also outline your topic (including any tricky terminology or jargon) and provide an explanation of the scope of your literature review – in other words, what you  will   and  won’t   be covering (the delimitations ). This helps ringfence your review and achieve a clear focus . The clearer and narrower your focus, the deeper you can dive into the topic (which is typically where the magic lies). 

Depending on the nature of your project, you could also present your stance or point of view at this stage. In other words, after grappling with the literature you’ll have an opinion about what the trends and concerns are in the field as well as what’s lacking. The introduction section can then present these ideas so that it is clear to examiners that you’re aware of how your research connects with existing knowledge .

2: The Body Section

The body of your literature review is the centre of your work. This is where you’ll present, analyse, evaluate and synthesise the existing research. In other words, this is where you’re going to earn (or lose) the most marks. Therefore, it’s important to carefully think about how you will organise your discussion to present it in a clear way. 

The body of your literature review should do just as the description of this chapter suggests. It should “review” the literature – in other words, identify, analyse, and synthesise it. So, when thinking about structuring your literature review, you need to think about which structural approach will provide the best “review” for your specific type of research and objectives (we’ll get to this shortly).

There are (broadly speaking)  three options  for organising your literature review.

Option 1: Chronological (according to date)

Organising the literature chronologically is one of the simplest ways to structure your literature review. You start with what was published first and work your way through the literature until you reach the work published most recently. Pretty straightforward.

The benefit of this option is that it makes it easy to discuss the developments and debates in the field as they emerged over time. Organising your literature chronologically also allows you to highlight how specific articles or pieces of work might have changed the course of the field – in other words, which research has had the most impact . Therefore, this approach is very useful when your research is aimed at understanding how the topic has unfolded over time and is often used by scholars in the field of history. That said, this approach can be utilised by anyone that wants to explore change over time .

For example , if a student of politics is investigating how the understanding of democracy has evolved over time, they could use the chronological approach to provide a narrative that demonstrates how this understanding has changed through the ages.

Here are some questions you can ask yourself to help you structure your literature review chronologically.

• What is the earliest literature published relating to this topic?
• How has the field changed over time? Why?
• What are the most recent discoveries/theories?

In some ways, chronology plays a part whichever way you decide to structure your literature review, because you will always, to a certain extent, be analysing how the literature has developed. However, with the chronological approach, the emphasis is very firmly on how the discussion has evolved over time , as opposed to how all the literature links together (which we’ll discuss next ).

Option 2: Thematic (grouped by theme)

The thematic approach to structuring a literature review means organising your literature by theme or category – for example, by independent variables (i.e. factors that have an impact on a specific outcome).

As you’ve been collecting and synthesising literature , you’ll likely have started seeing some themes or patterns emerging. You can then use these themes or patterns as a structure for your body discussion. The thematic approach is the most common approach and is useful for structuring literature reviews in most fields.

For example, if you were researching which factors contributed towards people trusting an organisation, you might find themes such as consumers’ perceptions of an organisation’s competence, benevolence and integrity. Structuring your literature review thematically would mean structuring your literature review’s body section to discuss each of these themes, one section at a time.

Here are some questions to ask yourself when structuring your literature review by themes:

• Are there any patterns that have come to light in the literature?
• What are the central themes and categories used by the researchers?
• Do I have enough evidence of these themes?

PS – you can see an example of a thematically structured literature review in our literature review sample walkthrough video here.

Option 3: Methodological

The methodological option is a way of structuring your literature review by the research methodologies used . In other words, organising your discussion based on the angle from which each piece of research was approached – for example, qualitative , quantitative or mixed  methodologies.

Structuring your literature review by methodology can be useful if you are drawing research from a variety of disciplines and are critiquing different methodologies. The point of this approach is to question  how  existing research has been conducted, as opposed to  what  the conclusions and/or findings the research were.

For example, a sociologist might centre their research around critiquing specific fieldwork practices. Their literature review will then be a summary of the fieldwork methodologies used by different studies.

Here are some questions you can ask yourself when structuring your literature review according to methodology:

• Which methodologies have been utilised in this field?
• Which methodology is the most popular (and why)?
• What are the strengths and weaknesses of the various methodologies?
• How can the existing methodologies inform my own methodology?

3: The Conclusion Section

Once you’ve completed the body section of your literature review using one of the structural approaches we discussed above, you’ll need to “wrap up” your literature review and pull all the pieces together to set the direction for the rest of your dissertation or thesis.

The conclusion is where you’ll present the key findings of your literature review. In this section, you should emphasise the research that is especially important to your research questions and highlight the gaps that exist in the literature. Based on this, you need to make it clear what you will add to the literature – in other words, justify your own research by showing how it will help fill one or more of the gaps you just identified.

Last but not least, if it’s your intention to develop a conceptual framework for your dissertation or thesis, the conclusion section is a good place to present this.

Example: Thematically Structured Review

In the video below, we unpack a literature review chapter so that you can see an example of a thematically structure review in practice.

Let’s Recap

In this article, we’ve  discussed how to structure your literature review for maximum impact. Here’s a quick recap of what  you need to keep in mind when deciding on your literature review structure:

• Just like other chapters, your literature review needs a clear introduction , body and conclusion .
• The introduction section should provide an overview of what you will discuss in your literature review.
• The body section of your literature review can be organised by chronology , theme or methodology . The right structural approach depends on what you’re trying to achieve with your research.
• The conclusion section should draw together the key findings of your literature review and link them to your research questions.

If you’re ready to get started, be sure to download our free literature review template to fast-track your chapter outline.

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

27 Comments

Great work. This is exactly what I was looking for and helps a lot together with your previous post on literature review. One last thing is missing: a link to a great literature chapter of an journal article (maybe with comments of the different sections in this review chapter). Do you know any great literature review chapters?

I agree with you Marin… A great piece

I agree with Marin. This would be quite helpful if you annotate a nicely structured literature from previously published research articles.

Awesome article for my research.

I thank you immensely for this wonderful guide

It is indeed thought and supportive work for the futurist researcher and students

Very educative and good time to get guide. Thank you

Great work, very insightful. Thank you.

Thanks for this wonderful presentation. My question is that do I put all the variables into a single conceptual framework or each hypothesis will have it own conceptual framework?

Thank you very much, very helpful

This is very educative and precise . Thank you very much for dropping this kind of write up .

Pheeww, so damn helpful, thank you for this informative piece.

I’m doing a research project topic ; stool analysis for parasitic worm (enteric) worm, how do I structure it, thanks.

comprehensive explanation. Help us by pasting the URL of some good “literature review” for better understanding.

great piece. thanks for the awesome explanation. it is really worth sharing. I have a little question, if anyone can help me out, which of the options in the body of literature can be best fit if you are writing an architectural thesis that deals with design?

I am doing a research on nanofluids how can l structure it?

Beautifully clear.nThank you!

Lucid! Thankyou!

Brilliant work, well understood, many thanks

I like how this was so clear with simple language 😊😊 thank you so much 😊 for these information 😊

Insightful. I was struggling to come up with a sensible literature review but this has been really helpful. Thank you!

You have given thought-provoking information about the review of the literature.

Thank you. It has made my own research better and to impart your work to students I teach

I learnt a lot from this teaching. It’s a great piece.

I am doing research on EFL teacher motivation for his/her job. How Can I structure it? Is there any detailed template, additional to this?

You are so cool! I do not think I’ve read through something like this before. So nice to find somebody with some genuine thoughts on this issue. Seriously.. thank you for starting this up. This site is one thing that is required on the internet, someone with a little originality!

I’m asked to do conceptual, theoretical and empirical literature, and i just don’t know how to structure it

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• UWF Libraries

Literature Review: Conducting & Writing

• Organizing/Writing
• Steps for Conducting a Lit Review
• Finding "The Literature"

Consider Organization

Literature review synthesis matrix, composing your literature review, managing citations / zotero.

• APA Style This link opens in a new window
• Chicago: Notes Bibliography This link opens in a new window
• MLA Style This link opens in a new window
• Sample Literature Reviews

Presentation on Synthesizing a Literature Review

You've got a focus, and you've narrowed it down to a thesis statement. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. Introduction:  Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern. Body:  Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each). Conclusions/Recommendations:  Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed? Organizing the body Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further. To help you come up with an overall organizational framework for your review, consider the following scenario and then three typical ways of organizing the sources into a review: You've decided to focus your literature review on materials dealing with sperm whales. This is because you've just finished reading  Moby Dick , and you wonder if that whale's portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 2020's. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 2021 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in  Moby Dick , so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel. Chronological If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (2021), and finally the biology articles (2000s) and the recent articles (last five years) on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus. By publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies. By trend A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote more than a century apart.

Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a "chronological" and a "thematic" approach is what is emphasized the most: the development of the harpoon or the harpoon technology.

But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as "evil" in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.

Methodological

A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the "methods" of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Once you've decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

Current Situation : Information necessary to understand the topic or focus of the literature review.

History : The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.

Methods and/or Standards : The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

(Adapted from  "Literature Reviews" from The Writing Center, University of North Carolina at Chapel Hill )

This synthesis matrix in Excel can help you get a jumpstart on finding ways in which the literature differs and is the same.

• Synthesis Matrix

O nce you've settled on a general pattern of organization, you're ready to write each section. There are a few guidelines you should follow during the writing stage. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

  However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as "writer," "pedestrian," and "persons." The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine "generic" condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, "Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense," Women and Language19:2.

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review's focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton's study, but then synthesize it by rephrasing the study's significance and relating it to their own work.

Keep your own voice

While the literature review presents others' ideas, your voice (the writer's) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil's. For more information, please see our handout on plagiarism .

Use a citation manager to manage citations from journals, books, documents, and internet sites.

A good one to use is Zotero. Instructions on using it can be found in the following guide:

• Zotero Guide

Content for this section of the guide was taken from  Literature Reviews from The Writing Center, University of North Carolina at Chapel Hill , under the guidelines of their Creative Commons License.

• << Previous: Finding "The Literature"
• Next: APA Style >>
• Last Updated: Mar 22, 2024 9:37 AM
• URL: https://libguides.uwf.edu/litreview

• University Libraries
• Research Guides
• Subject Guides

What is a literature review?

• Organization
• Getting Started
• Finding the Literature

A Process for Organizing Your Review of the Literature

Tools to help organize your literature review.

• Connect with Your Librarian
• More Information

Citation Management

• Zotero: A Beginner's Guide by Todd Quinn Last Updated Mar 29, 2024 20327 views this year
• Zotero v. EndNote v. Mendeley by Todd Quinn Last Updated Mar 13, 2023 473 views this year
• EndNote Software: Beginner's Guide by Todd Quinn Last Updated Jan 24, 2024 3669 views this year

More information

• Literature review assignments A video on how to organize literature reviews.
• Select  the most relevant material from your sources  With your research question in mind, read each of your sources and identify the material that is most relevant to that question. This might be material that answers the question directly, but it might also be material that helps explain why it’s important to ask the question or that is otherwise relevant to your question. When you pull this material from your source, you can extract it as a direct quotation, or you can paraphrase the passage or idea. (Make sure you enclose direct quotations in quotation marks!) A single source may have more than one idea relevant to your question.  
• Arrange  that material so you can focus on it apart from the source text itself Many writers put the material they have selected into a grid. They place each quotation or paraphrase in a cell in that grid. Arranging your selected material in a grid has two benefits: first, you can view your relevant material away from the source text (meaning you are now working with fewer words and pages!). Second, you can view all of the material that will go into your lit review in one place.  

Once you have created these groups of ideas, approaches, or themes, give each one a label. The labels describe the points, themes, or topics that are the backbone of your paper’s structure.

Now that you have identified the topics you will discuss in your lit review, look them over as a whole. Do you see any gaps that you should fill by finding additional sources? If so, do that research and add those sources to your groupings.

Once you have an assertion for each of your groupings, put those assertions in the order that you want to use in the lit review. This may be the order that has the best logical flow, or the order that tells the story you want to tell in the lit review.

Source:  Organizing Literature Reviews: The Basics from George Mason University

• Bubbl.us Bubbl.us makes it easy to organize your ideas visually in a way that makes sense to you and others.
• Coggle Coggle is online software for creating and sharing mindmaps and flowcharts.
• Google Sheets Create a matrix with author names across the top (columns), themes on the left side (rows).
• Microsoft Excel Create a matrix with author names across the top (columns), themes on the left side (rows).
• Mind42 Mind42 is a free online mind mapping software. In short: Mind42 offers you a software that runs in your browser to create mind maps - a special form of a structured diagram to visually organize information.
• Popplet Mind maps, flow charts, timelines, story boards and more.
• Scapple Ever scribbled ideas on a piece of paper and drawn lines between related thoughts? Then you already know what Scapple does. It's a virtual sheet of paper that lets you make notes anywhere and connect them using lines or arrows.
• XMind The full-featured mind mapping and brainstorming app.

• << Previous: Finding the Literature
• Next: Connect with Your Librarian >>
• Last Updated: Aug 17, 2023 9:45 AM
• URL: https://libguides.unm.edu/litreview

Eugene McDermott Library

Literature review.

• Collecting Resources for a Literature Review
• Organizing the Literature Review
• Writing the Literature Review
• Examples of Literature Reviews

Organization

Organization of your Literature Review

What is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? What order should you present them?

Basic Outline of a Literature Review

Just like most academic papers, literature reviews must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper.

Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.

Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).

Conclusions/Recommendations: Discuss what you have drawn from reviewing the literature so far. Where might the discussion proceed ? ( " Literature Reviews" from The Writing Center, University of North Carolina at Chapel Hill )

Organizing the Body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario and then three typical ways of organizing the sources into a review:

You've decided to focus your literature review on materials dealing with sperm whales. This is because you've just finished reading Moby Dick, and you wonder if that whale's portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980's. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Chronological

If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.

By publication

Order your sources chronologically by publication if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.

Another way to organize sources chronologically is to examine the sources under a trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Using this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.

Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a "chronological" and a "thematic" approach is what is emphasized the most: the development of the harpoon or the harpoon technology.

More authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as "evil" in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.

Methodological

A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the "methods" of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Once you've decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

( "Literature Reviews" from The Writing Center, University of North Carolina at Chapel Hill )

Other Sections to Include in a Literature Review

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

Current Situation: Information necessary to understand the topic or focus of the literature review.

History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.

Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

( " Literature Reviews" from The Writing Center, University of North Carolina at Chapel Hill )

• << Previous: Collecting Resources for a Literature Review
• Next: Writing the Literature Review >>
• Last Updated: Mar 22, 2024 12:28 PM
• URL: https://libguides.utdallas.edu/literature-review

Conducting a Literature Review

• Getting Started
• Define your Research Question
• Finding Sources
• Evaluating Sources

Organizing the Review

• Cite and Manage your Sources

Introduction

Organizing your literature review involves examining the sources you have and determining how they best fit together to form a coherent and complete narrative. However you choose to do this, the goal should be to organize your literature in a way that naturally flows and makes sense to your reader.

Additional Resources

• Literature Review: Conducting & Writing by the University of West Florida Libraries
• Literature Reviews: Organizing Your Research by the Embry-Riddle Aeronautical University Hunt Library

A literature review is structured similarly to other research essays, opening with an introduction that explains the topic and summarizes how the review will be conducted, several body paragraphs organized to share your findings, and a concluding paragraph.

There are many different ways to organize the body of your review. Some possible approaches are listed below.

Subtopic/Theme

While they all share the same overarching topic, each source approaches it in a slightly different way, valuing certain aspects or methods more than others. For example, with a literature review about the impacts of the Affordable Care Act, some literature might focus on the demographic changes in access to healthcare, or the actions taken by private health insurance companies, or even the way healthcare is discussed in politics. By combining sources that discuss the same subtopics, you can organize your review to show how the articles overlap and complement one another to create a more complete view of the existing research.

For a thematic literature review, each body paragraph would consist of one of these themes, or subtopics, and the literature associated with it.

Alternatively, you can group your resources by their relevance to your research question. Again using the ACA as an example, it might be a good idea to begin with the sources that most broadly address the impacts of the Affordable Care Act and then order the literature by increasing specificity. 

Methodology

It may be the case that your literature can be neatly defined into different types of research, such as different methods to treat an illness or ways to test a hypothesis. Examining the literature by the ways in which the authors tried to answer questions associated with the topic is a useful way to compare and contrast research results, as well as identify potential strengths or weaknesses in the methodologies used.

Varying Opinions/Problem & Solution(s)

Your various sources might not all come to the same conclusions about the topic; in fact, especially with controversial subject matter, there may be widely differing opinions on the issues and how best to approach them. Related to the thematic review, this type of literature review structure uses the first body paragraph to pose a question, then each of the body paragraphs illustrating the differing answers found in the literature. It is an excellent way to address arguments and counter-arguments if your topic is hotly contested in academic and popular works.

If you find yourself struggling to differentiate your sources by topic or relevance because they are all about equal in these regards, it might be a good idea to organize them chronologically.

There are two major types of chronology literature reviews tend to be grouped by:

• Publication date : Start with the earliest-published research and finish with the most current
• By trend : Organize sources into eras based on the time period and relative events associated with the topic. For example, regarding the Affordable Care Act, it could be split into the time before the ACA was passed, the immediate aftermath (2010-2011), Obama's second term (2012-2016), etc.

Using a Synthesis Matrix

A literature review doesn't merely summarize the current research on a topic: part of your responsibility is to take this information and make something new out of it that can be used by future researchers. This process of combining other sources of information and making an original argument out of them is called  synthesis , which literally means "the combination of ideas to form a theory or system." You will synthesize the literature you've selected for review to form an argument about where more research needs to be done on your topic.

One of the most important elements of synthesis in a literature review is analysis: rather than simply repeating the results of each source you've found, you are going to analyze it for similarities to your other resources, limitations and strengths of the methodology, and an examination of the conclusions drawn by the author(s) compared to the rest of the research on the topic. This is why proper organization of the literature is so important; it will allow you to group your sources by theme so that they can be more easily compared and contrasted.

In addition to the recommendations elsewhere on this page, a common method for preparing to organize your literature is by using a synthesis matrix. This is a tool to help pick out the most important aspects of each source and see where the most common themes lie. 

With the major information organized like this, it is easy to see which resources used similar methods of research, which had similar or differing results, and when chronologically the research was conducted. Grouping the literature by any of these similarities could be a useful way to organize your review.

• How to Synthesize Your Literature Review by Britt McGowan & UWF Libraries
• Synthesizing Sources by Purdue Online Writing Lab

Questions the Literature Review Should Answer

The University of the West Indies (linked below) provides a useful checklist of questions that a good literature review should address. When outlining your review, pay attention to how you will answer the following:

These will likely be answered throughout your body paragraphs, but it might be worthwhile to address some of these in the conclusion instead or in addition.

• Organizing the Literature Review by the University of the West Indies

• << Previous: Evaluating Sources
• Next: Writing the Review >>
• Last Updated: Apr 12, 2024 10:37 AM
• URL: https://libraryservices.acphs.edu/lit_review

• Process: Literature Reviews
• Literature Review
• Managing Sources

Ask a Librarian

Does your assignment or publication require that you write a literature review? This guide is intended to help you understand what a literature is, why it is worth doing, and some quick tips composing one.

Understanding Literature Reviews

What is a literature review  .

Typically, a literature review is a written discussion that examines publications about  a particular subject area or topic. Depending on disciplines, publications, or authors a literature review may be: 

A summary of sources An organized presentation of sources A synthesis or interpretation of sources An evaluative analysis of sources

A Literature Review may be part of a process or a product. It may be:

A part of your research process A part of your final research publication An independent publication

Why do a literature review?

The Literature Review will place your research in context. It will help you and your readers:  

Locate patterns, relationships, connections, agreements, disagreements, & gaps in understanding Identify methodological and theoretical foundations Identify landmark and exemplary works Situate your voice in a broader conversation with other writers, thinkers, and scholars

The Literature Review will aid your research process. It will help you to:

Establish your knowledge Understand what has been said Define your questions Establish a relevant methodology Refine your voice Situate your voice in the conversation

What does a literature review look like?

The Literature Review structure and organization may include sections such as:  

An introduction or overview A body or organizational sub-divisions A conclusion or an explanation of significance

The body of a literature review may be organized in several ways, including:

Chronologically: organized by date of publication Methodologically: organized by type of research method used Thematically: organized by concept, trend, or theme Ideologically: organized by belief, ideology, or school of thought

• Find a focus
• Find models
• Review your target publication
• Track citations
• Read critically
• Manage your citations
• Ask friends, faculty, and librarians

Additional Sources

• Reviewing the literature. Project Planner.
• Literature Review: By UNC Writing Center
• PhD on Track
• CU Graduate Students Thesis & Dissertation Guidance
• CU Honors Thesis Guidance

• Next: Managing Sources >>
• University of Colorado Boulder Libraries
• Research Guides
• Research Strategies
• Last Updated: May 31, 2024 4:45 PM
• URL: https://libguides.colorado.edu/strategies/litreview
• © Regents of the University of Colorado

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Grad Med Educ
• v.8(3); 2016 Jul

The Literature Review: A Foundation for High-Quality Medical Education Research

a  These are subscription resources. Researchers should check with their librarian to determine their access rights.

Despite a surge in published scholarship in medical education 1 and rapid growth in journals that publish educational research, manuscript acceptance rates continue to fall. 2 Failure to conduct a thorough, accurate, and up-to-date literature review identifying an important problem and placing the study in context is consistently identified as one of the top reasons for rejection. 3 , 4 The purpose of this editorial is to provide a road map and practical recommendations for planning a literature review. By understanding the goals of a literature review and following a few basic processes, authors can enhance both the quality of their educational research and the likelihood of publication in the Journal of Graduate Medical Education ( JGME ) and in other journals.

The Literature Review Defined

In medical education, no organization has articulated a formal definition of a literature review for a research paper; thus, a literature review can take a number of forms. Depending on the type of article, target journal, and specific topic, these forms will vary in methodology, rigor, and depth. Several organizations have published guidelines for conducting an intensive literature search intended for formal systematic reviews, both broadly (eg, PRISMA) 5 and within medical education, 6 and there are excellent commentaries to guide authors of systematic reviews. 7 , 8

• A literature review forms the basis for high-quality medical education research and helps maximize relevance, originality, generalizability, and impact.
• A literature review provides context, informs methodology, maximizes innovation, avoids duplicative research, and ensures that professional standards are met.
• Literature reviews take time, are iterative, and should continue throughout the research process.
• Researchers should maximize the use of human resources (librarians, colleagues), search tools (databases/search engines), and existing literature (related articles).
• Keeping organized is critical.

Such work is outside the scope of this article, which focuses on literature reviews to inform reports of original medical education research. We define such a literature review as a synthetic review and summary of what is known and unknown regarding the topic of a scholarly body of work, including the current work's place within the existing knowledge . While this type of literature review may not require the intensive search processes mandated by systematic reviews, it merits a thoughtful and rigorous approach.

Purpose and Importance of the Literature Review

An understanding of the current literature is critical for all phases of a research study. Lingard 9 recently invoked the “journal-as-conversation” metaphor as a way of understanding how one's research fits into the larger medical education conversation. As she described it: “Imagine yourself joining a conversation at a social event. After you hang about eavesdropping to get the drift of what's being said (the conversational equivalent of the literature review), you join the conversation with a contribution that signals your shared interest in the topic, your knowledge of what's already been said, and your intention.” 9

The literature review helps any researcher “join the conversation” by providing context, informing methodology, identifying innovation, minimizing duplicative research, and ensuring that professional standards are met. Understanding the current literature also promotes scholarship, as proposed by Boyer, 10 by contributing to 5 of the 6 standards by which scholarly work should be evaluated. 11 Specifically, the review helps the researcher (1) articulate clear goals, (2) show evidence of adequate preparation, (3) select appropriate methods, (4) communicate relevant results, and (5) engage in reflective critique.

Failure to conduct a high-quality literature review is associated with several problems identified in the medical education literature, including studies that are repetitive, not grounded in theory, methodologically weak, and fail to expand knowledge beyond a single setting. 12 Indeed, medical education scholars complain that many studies repeat work already published and contribute little new knowledge—a likely cause of which is failure to conduct a proper literature review. 3 , 4

Likewise, studies that lack theoretical grounding or a conceptual framework make study design and interpretation difficult. 13 When theory is used in medical education studies, it is often invoked at a superficial level. As Norman 14 noted, when theory is used appropriately, it helps articulate variables that might be linked together and why, and it allows the researcher to make hypotheses and define a study's context and scope. Ultimately, a proper literature review is a first critical step toward identifying relevant conceptual frameworks.

Another problem is that many medical education studies are methodologically weak. 12 Good research requires trained investigators who can articulate relevant research questions, operationally define variables of interest, and choose the best method for specific research questions. Conducting a proper literature review helps both novice and experienced researchers select rigorous research methodologies.

Finally, many studies in medical education are “one-offs,” that is, single studies undertaken because the opportunity presented itself locally. Such studies frequently are not oriented toward progressive knowledge building and generalization to other settings. A firm grasp of the literature can encourage a programmatic approach to research.

Approaching the Literature Review

Considering these issues, journals have a responsibility to demand from authors a thoughtful synthesis of their study's position within the field, and it is the authors' responsibility to provide such a synthesis, based on a literature review. The aforementioned purposes of the literature review mandate that the review occurs throughout all phases of a study, from conception and design, to implementation and analysis, to manuscript preparation and submission.

Planning the literature review requires understanding of journal requirements, which vary greatly by journal ( table 1 ). Authors are advised to take note of common problems with reporting results of the literature review. Table 2 lists the most common problems that we have encountered as authors, reviewers, and editors.

Sample of Journals' Author Instructions for Literature Reviews Conducted as Part of Original Research Article a

Common Problem Areas for Reporting Literature Reviews in the Context of Scholarly Articles

Locating and Organizing the Literature

Three resources may facilitate identifying relevant literature: human resources, search tools, and related literature. As the process requires time, it is important to begin searching for literature early in the process (ie, the study design phase). Identifying and understanding relevant studies will increase the likelihood of designing a relevant, adaptable, generalizable, and novel study that is based on educational or learning theory and can maximize impact.

Human Resources

A medical librarian can help translate research interests into an effective search strategy, familiarize researchers with available information resources, provide information on organizing information, and introduce strategies for keeping current with emerging research. Often, librarians are also aware of research across their institutions and may be able to connect researchers with similar interests. Reaching out to colleagues for suggestions may help researchers quickly locate resources that would not otherwise be on their radar.

During this process, researchers will likely identify other researchers writing on aspects of their topic. Researchers should consider searching for the publications of these relevant researchers (see table 3 for search strategies). Additionally, institutional websites may include curriculum vitae of such relevant faculty with access to their entire publication record, including difficult to locate publications, such as book chapters, dissertations, and technical reports.

Strategies for Finding Related Researcher Publications in Databases and Search Engines

Search Tools and Related Literature

Researchers will locate the majority of needed information using databases and search engines. Excellent resources are available to guide researchers in the mechanics of literature searches. 15 , 16

Because medical education research draws on a variety of disciplines, researchers should include search tools with coverage beyond medicine (eg, psychology, nursing, education, and anthropology) and that cover several publication types, such as reports, standards, conference abstracts, and book chapters (see the box for several information resources). Many search tools include options for viewing citations of selected articles. Examining cited references provides additional articles for review and a sense of the influence of the selected article on its field.

Box Information Resources

• Web of Science a
• Education Resource Information Center (ERIC)
• Cumulative Index of Nursing & Allied Health (CINAHL) a
• Google Scholar

Once relevant articles are located, it is useful to mine those articles for additional citations. One strategy is to examine references of key articles, especially review articles, for relevant citations.

Getting Organized

As the aforementioned resources will likely provide a tremendous amount of information, organization is crucial. Researchers should determine which details are most important to their study (eg, participants, setting, methods, and outcomes) and generate a strategy for keeping those details organized and accessible. Increasingly, researchers utilize digital tools, such as Evernote, to capture such information, which enables accessibility across digital workspaces and search capabilities. Use of citation managers can also be helpful as they store citations and, in some cases, can generate bibliographies ( table 4 ).

Citation Managers

Knowing When to Say When

Researchers often ask how to know when they have located enough citations. Unfortunately, there is no magic or ideal number of citations to collect. One strategy for checking coverage of the literature is to inspect references of relevant articles. As researchers review references they will start noticing a repetition of the same articles with few new articles appearing. This can indicate that the researcher has covered the literature base on a particular topic.

Putting It All Together

In preparing to write a research paper, it is important to consider which citations to include and how they will inform the introduction and discussion sections. The “Instructions to Authors” for the targeted journal will often provide guidance on structuring the literature review (or introduction) and the number of total citations permitted for each article category. Reviewing articles of similar type published in the targeted journal can also provide guidance regarding structure and average lengths of the introduction and discussion sections.

When selecting references for the introduction consider those that illustrate core background theoretical and methodological concepts, as well as recent relevant studies. The introduction should be brief and present references not as a laundry list or narrative of available literature, but rather as a synthesized summary to provide context for the current study and to identify the gap in the literature that the study intends to fill. For the discussion, citations should be thoughtfully selected to compare and contrast the present study's findings with the current literature and to indicate how the present study moves the field forward.

To facilitate writing a literature review, journals are increasingly providing helpful features to guide authors. For example, the resources available through JGME include several articles on writing. 17 The journal Perspectives on Medical Education recently launched “The Writer's Craft,” which is intended to help medical educators improve their writing. Additionally, many institutions have writing centers that provide web-based materials on writing a literature review, and some even have writing coaches.

The literature review is a vital part of medical education research and should occur throughout the research process to help researchers design a strong study and effectively communicate study results and importance. To achieve these goals, researchers are advised to plan and execute the literature review carefully. The guidance in this editorial provides considerations and recommendations that may improve the quality of literature reviews.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Welcome to the Purdue Online Writing Lab

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

The Online Writing Lab at Purdue University houses writing resources and instructional material, and we provide these as a free service of the Writing Lab at Purdue. Students, members of the community, and users worldwide will find information to assist with many writing projects. Teachers and trainers may use this material for in-class and out-of-class instruction.

The Purdue On-Campus Writing Lab and Purdue Online Writing Lab assist clients in their development as writers—no matter what their skill level—with on-campus consultations, online participation, and community engagement. The Purdue Writing Lab serves the Purdue, West Lafayette, campus and coordinates with local literacy initiatives. The Purdue OWL offers global support through online reference materials and services.

A Message From the Assistant Director of Content Development 

The Purdue OWL® is committed to supporting  students, instructors, and writers by offering a wide range of resources that are developed and revised with them in mind. To do this, the OWL team is always exploring possibilties for a better design, allowing accessibility and user experience to guide our process. As the OWL undergoes some changes, we welcome your feedback and suggestions by email at any time.

Please don't hesitate to contact us via our contact page  if you have any questions or comments.

All the best,

Social Media

Facebook twitter.

The four building blocks of change

Large-scale organizational change has always been difficult, and there’s no shortage of research showing that a majority of transformations continue to fail. Today’s dynamic environment adds an extra level of urgency and complexity. Companies must increasingly react to sudden shifts in the marketplace, to other external shocks, and to the imperatives of new business models. The stakes are higher than ever.

So what’s to be done? In both research and practice, we find that transformations stand the best chance of success when they focus on four key actions to change mind-sets and behavior: fostering understanding and conviction, reinforcing changes through formal mechanisms, developing talent and skills, and role modeling. Collectively labeled the “influence model,” these ideas were introduced more than a dozen years ago in a McKinsey Quarterly article, “ The psychology of change management .” They were based on academic research and practical experience—what we saw worked and what didn’t.

Digital technologies and the changing nature of the workforce have created new opportunities and challenges for the influence model (for more on the relationship between those trends and the model, see this article’s companion, “ Winning hearts and minds in the 21st century ”). But it still works overall, a decade and a half later (exhibit). In a recent McKinsey Global Survey, we examined successful transformations and found that they were nearly eight times more likely to use all four actions as opposed to just one. 1 1. See “ The science of organizational transformations ,” September 2015. Building both on classic and new academic research, the present article supplies a primer on the model and its four building blocks: what they are, how they work, and why they matter.

Fostering understanding and conviction

We know from research that human beings strive for congruence between their beliefs and their actions and experience dissonance when these are misaligned. Believing in the “why” behind a change can therefore inspire people to change their behavior. In practice, however, we find that many transformation leaders falsely assume that the “why” is clear to the broader organization and consequently fail to spend enough time communicating the rationale behind change efforts.

This common pitfall is predictable. Research shows that people frequently overestimate the extent to which others share their own attitudes, beliefs, and opinions—a tendency known as the false-consensus effect. Studies also highlight another contributing phenomenon, the “curse of knowledge”: people find it difficult to imagine that others don’t know something that they themselves do know. To illustrate this tendency, a Stanford study asked participants to tap out the rhythms of well-known songs and predict the likelihood that others would guess what they were. The tappers predicted that the listeners would identify half of the songs correctly; in reality, they did so less than 5 percent of the time. 2 2. Chip Heath and Dan Heath, “The curse of knowledge,” Harvard Business Review , December 2006, Volume 8, Number 6, hbr.org.

Therefore, in times of transformation, we recommend that leaders develop a change story that helps all stakeholders understand where the company is headed, why it is changing, and why this change is important. Building in a feedback loop to sense how the story is being received is also useful. These change stories not only help get out the message but also, recent research finds, serve as an effective influencing tool. Stories are particularly effective in selling brands. 3 3. Harrison Monarth, “The irresistible power of storytelling as a strategic business tool,” Harvard Business Review , March 11, 2014, hbr.org.

Even 15 years ago, at the time of the original article, digital advances were starting to make employees feel involved in transformations, allowing them to participate in shaping the direction of their companies. In 2006, for example, IBM used its intranet to conduct two 72-hour “jam sessions” to engage employees, clients, and other stakeholders in an online debate about business opportunities. No fewer than 150,000 visitors attended from 104 countries and 67 different companies, and there were 46,000 posts. 4 4. Icons of Progress , “A global innovation jam,” ibm.com. As we explain in “Winning hearts and minds in the 21st century,” social and mobile technologies have since created a wide range of new opportunities to build the commitment of employees to change.

Reinforcing with formal mechanisms

Psychologists have long known that behavior often stems from direct association and reinforcement. Back in the 1920s, Ivan Pavlov’s classical conditioning research showed how the repeated association between two stimuli—the sound of a bell and the delivery of food—eventually led dogs to salivate upon hearing the bell alone. Researchers later extended this work on conditioning to humans, demonstrating how children could learn to fear a rat when it was associated with a loud noise. 5 5. John B. Watson and Rosalie Rayner, “Conditioned emotional reactions,” Journal of Experimental Psychology , 1920, Volume 3, Number 1, pp. 1–14. Of course, this conditioning isn’t limited to negative associations or to animals. The perfume industry recognizes how the mere scent of someone you love can induce feelings of love and longing.

Reinforcement can also be conscious, shaped by the expected rewards and punishments associated with specific forms of behavior. B. F. Skinner’s work on operant conditioning showed how pairing positive reinforcements such as food with desired behavior could be used, for example, to teach pigeons to play Ping-Pong. This concept, which isn’t hard to grasp, is deeply embedded in organizations. Many people who have had commissions-based sales jobs will understand the point—being paid more for working harder can sometimes be a strong incentive.

Despite the importance of reinforcement, organizations often fail to use it correctly. In a seminal paper “On the folly of rewarding A, while hoping for B,” management scholar Steven Kerr described numerous examples of organizational-reward systems that are misaligned with the desired behavior, which is therefore neglected. 6 6. Steven Kerr, “On the folly of rewarding A, while hoping for B,” Academy of Management Journal , 1975, Volume 18, Number 4, pp. 769–83. Some of the paper’s examples—such as the way university professors are rewarded for their research publications, while society expects them to be good teachers—are still relevant today. We ourselves have witnessed this phenomenon in a global refining organization facing market pressure. By squeezing maintenance expenditures and rewarding employees who cut them, the company in effect treated that part of the budget as a “super KPI.” Yet at the same time, its stated objective was reliable maintenance.

Even when organizations use money as a reinforcement correctly, they often delude themselves into thinking that it alone will suffice. Research examining the relationship between money and experienced happiness—moods and general well-being—suggests a law of diminishing returns. The relationship may disappear altogether after around $75,000, a much lower ceiling than most executives assume. 7 7. Belinda Luscombe, “Do we need $75,000 a year to be happy?” Time , September 6, 2010, time.com.

Would you like to learn more about our People & Organizational Performance Practice ?

Money isn’t the only motivator, of course. Victor Vroom’s classic research on expectancy theory explained how the tendency to behave in certain ways depends on the expectation that the effort will result in the desired kind of performance, that this performance will be rewarded, and that the reward will be desirable. 8 8. Victor Vroom, Work and motivation , New York: John Wiley, 1964. When a Middle Eastern telecommunications company recently examined performance drivers, it found that collaboration and purpose were more important than compensation (see “Ahead of the curve: The future of performance management,” forthcoming on McKinsey.com). The company therefore moved from awarding minor individual bonuses for performance to celebrating how specific teams made a real difference in the lives of their customers. This move increased motivation while also saving the organization millions.

How these reinforcements are delivered also matters. It has long been clear that predictability makes them less effective; intermittent reinforcement provides a more powerful hook, as slot-machine operators have learned to their advantage. Further, people react negatively if they feel that reinforcements aren’t distributed fairly. Research on equity theory describes how employees compare their job inputs and outcomes with reference-comparison targets, such as coworkers who have been promoted ahead of them or their own experiences at past jobs. 9 9. J. S. Adams, “Inequity in social exchanges,” Advances in Experimental Social Psychology , 1965, Volume 2, pp. 267–300. We therefore recommend that organizations neutralize compensation as a source of anxiety and instead focus on what really drives performance—such as collaboration and purpose, in the case of the Middle Eastern telecom company previously mentioned.

Developing talent and skills

Thankfully, you can teach an old dog new tricks. Human brains are not fixed; neuroscience research shows that they remain plastic well into adulthood. Illustrating this concept, scientific investigation has found that the brains of London taxi drivers, who spend years memorizing thousands of streets and local attractions, showed unique gray-matter volume differences in the hippocampus compared with the brains of other people. Research linked these differences to the taxi drivers’ extraordinary special knowledge. 10 10. Eleanor Maguire, Katherine Woollett, and Hugo Spires, “London taxi drivers and bus drivers: A structural MRI and neuropsychological analysis,” Hippocampus , 2006, Volume 16, pp. 1091–1101.

Despite an amazing ability to learn new things, human beings all too often lack insight into what they need to know but don’t. Biases, for example, can lead people to overlook their limitations and be overconfident of their abilities. Highlighting this point, studies have found that over 90 percent of US drivers rate themselves above average, nearly 70 percent of professors consider themselves in the top 25 percent for teaching ability, and 84 percent of Frenchmen believe they are above-average lovers. 11 11. The art of thinking clearly, “The overconfidence effect: Why you systematically overestimate your knowledge and abilities,” blog entry by Rolf Dobelli, June 11, 2013, psychologytoday.com. This self-serving bias can lead to blind spots, making people too confident about some of their abilities and unaware of what they need to learn. In the workplace, the “mum effect”—a proclivity to keep quiet about unpleasant, unfavorable messages—often compounds these self-serving tendencies. 12 12. Eliezer Yariv, “‘Mum effect’: Principals’ reluctance to submit negative feedback,” Journal of Managerial Psychology , 2006, Volume 21, Number 6, pp. 533–46.

Even when people overcome such biases and actually want to improve, they can handicap themselves by doubting their ability to change. Classic psychological research by Martin Seligman and his colleagues explained how animals and people can fall into a state of learned helplessness—passive acceptance and resignation that develops as a result of repeated exposure to negative events perceived as unavoidable. The researchers found that dogs exposed to unavoidable shocks gave up trying to escape and, when later given an opportunity to do so, stayed put and accepted the shocks as inevitable. 13 13. Martin Seligman and Steven Maier, “Failure to escape traumatic shock,” Journal of Experimental Psychology , 1967, Volume 74, Number 1, pp. 1–9. Like animals, people who believe that developing new skills won’t change a situation are more likely to be passive. You see this all around the economy—from employees who stop offering new ideas after earlier ones have been challenged to unemployed job seekers who give up looking for work after multiple rejections.

Instilling a sense of control and competence can promote an active effort to improve. As expectancy theory holds, people are more motivated to achieve their goals when they believe that greater individual effort will increase performance. 14 14. Victor Vroom, Work and motivation , New York: John Wiley, 1964. Fortunately, new technologies now give organizations more creative opportunities than ever to showcase examples of how that can actually happen.

Role modeling

Research tells us that role modeling occurs both unconsciously and consciously. Unconsciously, people often find themselves mimicking the emotions, behavior, speech patterns, expressions, and moods of others without even realizing that they are doing so. They also consciously align their own thinking and behavior with those of other people—to learn, to determine what’s right, and sometimes just to fit in.

While role modeling is commonly associated with high-power leaders such as Abraham Lincoln and Bill Gates, it isn’t limited to people in formal positions of authority. Smart organizations seeking to win their employees’ support for major transformation efforts recognize that key opinion leaders may exert more influence than CEOs. Nor is role modeling limited to individuals. Everyone has the power to model roles, and groups of people may exert the most powerful influence of all. Robert Cialdini, a well-respected professor of psychology and marketing, examined the power of “social proof”—a mental shortcut people use to judge what is correct by determining what others think is correct. No wonder TV shows have been using canned laughter for decades; believing that other people find a show funny makes us more likely to find it funny too.

Today’s increasingly connected digital world provides more opportunities than ever to share information about how others think and behave. Ever found yourself swayed by the number of positive reviews on Yelp? Or perceiving a Twitter user with a million followers as more reputable than one with only a dozen? You’re not imagining this. Users can now “buy followers” to help those users or their brands seem popular or even start trending.

The endurance of the influence model shouldn’t be surprising: powerful forces of human nature underlie it. More surprising, perhaps, is how often leaders still embark on large-scale change efforts without seriously focusing on building conviction or reinforcing it through formal mechanisms, the development of skills, and role modeling. While these priorities sound like common sense, it’s easy to miss one or more of them amid the maelstrom of activity that often accompanies significant changes in organizational direction. Leaders should address these building blocks systematically because, as research and experience demonstrate, all four together make a bigger impact.

Tessa Basford is a consultant in McKinsey’s Washington, DC, office; Bill Schaninger is a director in the Philadelphia office.

Explore a career with us

Related articles.

Winning hearts and minds in the 21st century

Changing change management

Digital hives: Creating a surge around change

A Thematic Literature Review on International Public Sector Accounting Standards (IPSAS)

• Open access
• Published: 29 May 2024

Cite this article

You have full access to this open access article

• Sean Scannell 1 &
• Vincent Tawiah   ORCID: orcid.org/0000-0003-1679-387X 1  

265 Accesses

Explore all metrics

This paper is a thematic review of the existing literature on International Public Sector Accounting Standards (IPSAS) between 2000 and 2023. The review finds various advantages associated with the high adoption rates in the literature in the developed world. It also identified challenges associated with implementation and the effects of the cash-basis standard in developing countries. The study finds that uniform application may not be feasible at this stage of the diffusion process given the inherent limitations in developing countries. This review also provides a platform for identifying strands of the literature which have been underexplored and require further studies.

Similar content being viewed by others

Corporate Social Responsibility (CSR) Implementation: A Review and a Research Agenda Towards an Integrative Framework

Mandatory CSR and sustainability reporting: economic analysis and literature review

The impact of corporate governance on financial performance: a cross-sector study.

Avoid common mistakes on your manuscript.

Introduction

The International Public Sector Accounting Standards are a new phenomenon aimed at reforming public sector financial reporting and have been established and governed by the International Public Sector Accounting Standards Board (IPSASB) since 1986. However, many of the technical reforms have been accelerated in the wake of the sovereign debt crisis in 2008 which created distrust and the need to demonstrate greater transparency in the public sector (Schmidthuber et al., 2022 ; Rossi et al., 2016 ; Brusca et al., 2013 ). IPSAS is an initiative stemming from New Public Management principles which seeks to apply accrual reporting techniques established in the private sector to the public sector. The IPSASB has formulated these standards with a long-term view to unifying governmental reporting and moving away from traditional cash-based reporting systems.

According to the International Labour Organisation over 80 bodies including the UN have adopted IPSAS either directly or indirectly (ILO, 2021 ). The revolution in public sector reporting is widespread with adopters in Europe (Christaens et al., 2015 ; Argento et al., 2018 ; Jorge et al., 2019 ), Africa (Whitefield & Savvas, 2016 ; Olaoye & Talabi, 2018 ) and Asia (Abdulkarim et al., 2023 ; Alshujairi, 2014 ). Other governments have developed their own governmental accounting standards “GAS” as seen in Indonesia. IPSAS has experienced some resistance as the benefits of implementation are not understood (Bekiaris & Paraponti, 2023 ). Further, there is inadequate education on the use and interpretation of accrual accounts (Fahmid et al., 2020 ). There is also a lack of evidence of accrual accounting being used for decision-making purposes (Polzer et al., 2019 ; Hepworth, 2015 ). A review of transnational studies is included in the review, which investigates drivers of adoption (Schmidthuber et al., 2022 ), corruption (Tawiah, 2023a ; Hamed-Sidhom et al., 2022 ; Cuadrado-Ballasteros et al., 2020 ) and other consequences.

Much of the extant research on adoption rates has been conducted in Europe, with many of the benefits advertised by proponents applying in this context. For the purposes of this research, it was determined that a holistic review of both developing and developed countries was required to obtain a clear picture of IPSAS. This would allow us to capture the nuances associated with its implementation in various settings. This review provides evidence as to the current state of IPSAS adoption and how it can enhance governmental financial reporting. This review also provides a platform for identifying strands of the literature which have been underexplored and require further studies. The review may attract the attention of policymakers to form a better understanding of the cultural and institutional factors inhibiting adoption globally, as well as the impact IPSAS implementation has had on adopting bodies.

The remainder of the paper is structured as follows. The second section presents how the review was conducted. The third section discusses New Public Management and IPSAS adoption. The fourth section considers the consequences of adoption. The fifth section presents the challenges to IPSAS adoption and implementation. The sixth section contains the review of developing countries and cash-basis IPSAS. The paper concludes in the seventh section and suggestions for future research are presented in the eighth section.

How the Review was Done

Boundaries of the review.

IPSAS and public sector financial management reform have attracted commentary from academics and practitioners alike, particularly in the wake of the 2009 global financial crisis. Researchers have conducted various studies over the past two decades exploring IPSAS adoption rates, the reasons why countries adopt, and the practical implications and issues associated with adoption. A Google Scholar search indicated that there are over 300 academic articles available which cover the topic of IPSAS. The methodology of these studies is wide-ranging including qualitative analysis and discussions along with empirical and theoretical studies.

To ensure the most accurate and relevant information was selected for this review, the search was narrowed to include papers in the public sector and governmental studies, financial management and accountancy journals. IPSAS is a relatively new development in public sector financial management, and therefore, this review only includes articles from the past 23-year period from 2000 to 2023. The aim of this constraint is to ensure information is not outdated and reflects the current landscape in public sector reforms.

Selection of Papers

Various keywords (e.g. public sector reforms, IPSAS, government accounting standards, new public sector financial management, etc.) were used in the search to allow for the gathering of literature on various topics associated with IPSAS, including adoption, consequences, corruption, cultural factors and cash-basis. The papers were collected from reputable databases such as the Web of Science and Scopus. The quality of the journals is based on the Association of Business School, ABS-ranked journals. However, due to the nature of IPSAS and its prominence as a research topic in developing countries, not all papers met this criterion. However, if they were deemed to be of high quality, they were included.

The review approach is broken down as follows; the authors, titles, a summary of the findings and journal titles were collected to form an archive of articles examined. This database is categorized further by the topic to which each article relates. Table  1 presents articles collected in the Adoption and NPM section. Table  2 highlights the consequences of adoption which include budgeting, corruption and cultural barriers. Table  3 presents papers which examine the limitations associated with IPSAS implementation. Finally, Table  4 shows papers collected on the developing countries and cash-basis IPSAS which has become a popular alternative to the traditional accrual-basis IPSAS as less-developed countries transition to a full accrual financial reporting system.

New Public Management and IPSAS Adoption

IPSAS is underpinned by the concept of New Public Management which is a movement driven by practitioners to improve public administration and make the public sector more accountable to the stakeholders they serve. This is achieved by institutionalizing accrual accounting for budgetary and reporting to provide enhanced information about liabilities and debt (Pina & Torres, 2003 ; Olson et al., 2000 ). However, the IPSASB have also issued a cash-basis standard which is designed for transitional and developing economies with the view to full accrual adoption in the future. The OECD cites this trend as being responsible for making the public sector more responsive to citizen’s needs while promoting enhanced governance (Groot & Budding, 2008 ; Lapsley et al., 2009 ).

Professional accountants have embraced the transition to accrual and NPM techniques in public sector reporting and some have suggested that much of their enthusiasm around the reform is driven by self-interest and the desire to attain more clients and greater business (Vivian & Maroun, 2018 ). This has forced some to question the objectivity of professional bodies in the IPSAS diffusion process.

The accrual accounting technique is a development in the private sector and its application to the public sector is driven in part by the desire to overcome bureaucratic obstacles to allow for a more efficient allocation of resources (Pina & Torres, 2003 ; ACCA, 2017 ; Gomes et al., 2023 ; Obara & Nangih, 2017 ). However, the advantages of accrual accounting to the public sector have been refuted by many academics in the literature as the concepts of stewardship and accountability associated with public sector governance are emphasised less than information relevance which is a key characteristic of financial reporting in the private sector (Whittington, 2008 ; Murphy et al., 2013 ; Zhang & Andrew, 2014 ).

IPSAS is modelled on IFRS which is applied in the private sector. Many argue that using IFRS as a basis for IPSAS is unsuitable due to the complexities and nuances of government accounting. IFRS is based on transactions, risks and presenting information to capital market participants (Rincón-Soto & Gómez-Villegas, 2020 ), factors that are less relevant in public sector reporting. Critics argue that IPSAS adoption is merely a technical revolution, however, in practice, it is much more complex (Hepworth, 2017 ). In addition, it is contended that IPSAS fail to reflect public sector institutional dynamics which contribute to decoupling, the production of unsuitable information for users’ needs (Rincón-Soto & Gómez-Villegas, 2020 ; Baker & Rennie, 2006 ; Biondi, 2018 ; Toudas et al., 2013 ).

Some of the extant literature has looked at other driving forces behind the adoption of IPSAS besides the benefits of improved governance, transparency and accountability. The theory of isomorphism has attracted the most discourse among academics as an explanation for IPSAS adoption in case study settings. There are three strands to isomorphism; coercive (driven by international institutions), mimetic (a search for legitimacy), and normative (related to political groups, lobbyists and professionals) (Rincón-Soto & Gómez-Villegas, 2020 ). Some countries, particularly with developing economies are subject to coercive isomorphism and recognise the need to implement IPSAS to access financial aid as is the case in Jordan and other developing countries (Alghizzawi & Masruki, 2020 ; Mnif Sellami & Gafsi, 2019 ; Amiri & Hamza, 2020 ). These claims have been further reinforced by empirical evidence suggesting that IPSAS adoption shares a strong positive correlation with access to aid (Tawiah & Soobaroyen, 2023 ).

Mimetic forces driving IPSAS implementation have been highlighted in Malta, Spain and Portugal (Jones & Caruana, 2016 ; Gomes et al., 2023 ). The Maltese government’s search for trust drove IPSAS implementation. Their new EU membership and small economic power have made them more susceptible to mimetic forces sparking public sector reform. The aim to be recognised as legitimate internationally shaped their stance on IPSAS.

The final element of isomorphism discussed in the literature is the normative aspect. An Estonian case study (Argento et al., 2018 ) provides insight into how influential accounting professionals were a major contributory factor in adopting IPSAS in Estonia. Normative forces along with the nation’s history of a centrally planned economy formed a broader supportive context for implementing radical reforms in public sector accounting.

IPSAS adoption has been the subject of various studies in the literature, especially given that implementation is not compulsory. Adoption itself can be further broken down into full accrual adoption, partial adoption and the use of the cash-basis IPSAS. This has prompted researchers to investigate the landscape globally in public financial management reform. One of the cited benefits of the standards is increased comparability across jurisdictions (Brusca et al., 2016 ; Benito et al., 2007 ; Fahmid et al., 2020 ; Gkouma & Filos, 2022 ), however, the numerous options available have contradicted these claims.

EU countries have been at the forefront of reforms in the public sector and as such much attention has been devoted to investigating adoption rates and the reasons behind their decisions. Christiaens et al. ( 2010 ) was one of the first major studies that examined adoption at both local and central government levels in Europe. None of the respondents planned to fully adopt at both local and central government. Only 21.1% were adopted in local governments, while 36.8% had adopted or planned to adopt IPSAS at the central government level. Accrual accounting was valued by respondents to questionnaires, however, confidence in full IPSAS adoption remained low as respondents raised concerns about the role IFRS played in the development process. Conversely, this stance was contradicted by Brusca et al. ( 2013 ) who found within the Iberian context, the EU’s strategy of promoting IFRS has strengthened the credibility of IPSAS as a parallel system. Other studies of Spanish and Portuguese authorities have re-emphasised that IFRS has played a role in promoting adoption with their desire to align standards with their private sector (Jorge et al., 2019 ).

Christiaens et al. ( 2015 ) conducted a comparative study to their previous investigation whereby adoption rates and those considering adoption increased significantly, indicating that uncertainty around the standards is beginning to dissipate. However, questionnaires denoted that institutional factors such as legal systems and economic conditions dictated the ability to implement IPSAS. This slow trend toward harmonization is highlighted again in more recent literature. Bekiaris and Paraponti ( 2023 ) report that only 64% of OECD countries have medium to high levels of IPSAS implementation. The remainder reporting low levels of adoption lacked understanding of the benefits which has acted as a barrier to dissemination in these countries.

Many of the above studies looked at IPSAS in the context of the developed world, however many of the benefits cited such as improving governance and reducing corruption are marketed to the developing countries.

Consequences of Adoption

The benefits of IPSAS diffusion are widely reported by both practitioners and academics. Tighter fiscal control, sound financial management and government stability are advantages documented both in the literature and by professional bodies (ACCA, 2017 ; Gourfinkel, 2021 ; Casteñeda-Rodríguez, 2022 ). These advantages are predicted to come in tandem with societal benefits, improved data consistency and international comparability.

IPSAS has also been seen to promote transparency (Ijeoma, 2014 ), providing greater information on public finances and decision-making which helps to develop a more informed electorate. This is done by bridging information asymmetry between public officers and the citizens they govern (Gkouma & Filos, 2022 ; Fahmid et al., 2020 ). This has even been the case in developing countries which are characterized by inadequate regulation and low transparency (Tawiah, 2023a , b ; Abdulkarim et al., 2023 ).

Accrual accounting techniques are intended to provide greater clarity about future liabilities, assets and debts than is available in traditional cash-based financial reporting by governments (Pina & Torres, 2003 ). Greater information, both in volume and quality enhances the ability of decision-makers to track performance and take corrective action if necessary (Bergmann, 2012 ; Gomes et al., 2023 ).

Having agile systems that portray an accurate picture of public finances is a means of avoiding sovereign debt crises like those experienced in the late 2000’s (Rossi et al., 2016 ). The Greek economy was one of the most adversely affected economies by the crash and IPSAS adoption is seen as a way of enhancing financial reporting by producing high-quality, understandable financial statements. This enables citizens to be more informed when selecting officers to represent them in parliament. Ball ( 2015 ) suggest that restoring trust to the public is the main benefit of IPSAS in the Greek context.

Finally, comparability is another significant driver of adoption. Harmonization is not as prioritized in the public sector as in the private sector due to the nature of capital markets. However, in an EU context, there are benefits of having ubiquitous standards and presentation of financial information (Lopez et al., 2002 ). This is especially relevant given a European standard based on IPSAS (EPSAS) is in the process of development (Caruana et al., 2019 ).

Transparency is one of the main aims of IPSAS, and researchers have investigated whether IPSAS can curtail corruption in the public sector. Much of the literature supports these claims with accrual IPSAS reducing corruption and cash-basis IPSAS proven to be a controller of corruption. However, the accrual basis was highlighted to be more impactful (Tawiah 2023a ). The absence of a financial reporting framework in some foreign aid beneficiary countries was noted to facilitate corruption and budget waste. However, IPSAS and heightened disclosures significantly reduced perceived levels of corruption in these countries (Hamed-Sidom et al., 2022 ). Similarly, Cuadrado-Ballasteros et al. ( 2020 ) found that accrual systems reduce corruption, and user understanding is intrinsic to greater transparency. The reduction of information asymmetry between the government and the public forces politicians to act in the public interest.

Researchers in Nigeria have paid particular attention to the topic of corruption and IPSAS given its history of poor budget implementation and a lack of accountability. Two distinct and separate studies in the Nigerian context (Olaoye & Olaniyan, 2018 ; Enofe et al., 2017 ) have highlighted no significant association between IPSAS adoption and reduced corruption, even going as far as to say that IPSAS adoption was unsuitable and an imposition for developing countries. The researchers believe that given their economic landscape, IPSAS adoption is unsuccessful in attracting FDI, fostering development and controlling corruption, despite improved financial reporting. However, it is important to note that the vast majority of IPSAS implementation projects have yielded positive results in the area of corruption. Broader and more comprehensive institutional overhauls may be necessary to realise the benefits of IPSAS (Gkouma & Filos, 2022 ; Christaens et al., 2015 ; Abdulkarim et al., 2023 ) in developing economies.

Budgeting is another innate duty of public sector finance departments (Zarei et al., 2022 ) and the transition to accrual-basis reporting will continue to impact the budget process which has been traditionally cash-based (Benito et al., 2007 ; Christiaens & Rommel, 2008 ) and has required some amendments to budgetary law in certain jurisdictions (Alshujairi, 2014 ). According to López et al. ( 2002 ), information derived from financial reporting must be appropriately linked through the preparation of various reconciliations including from budgeted expenditure to actual fiscal performance. These reconciliations are vital to maintaining transparency and accountability as well as reducing budgetary waste as advertised by IPSAS (Sanderson & van Schaik, 2008 ; Hamed-Sidhom et al., 2022 ).

Much attention has been paid to the challenges associated with portraying an accurate picture of the budgetary and financial results of government bodies through reconciliation and the practicality of IPSAS 24, the standard on budgeting. Heiling ( 2019 ) and van Schaik ( 2021 ) highlighted reconciliations that would improve the presentation of budget and financial information under IPSAS, particularly when using the cash-basis of budgeting and accrual financial reporting. These included flagging cash flows under both activities and functional categories, complexities that generally arise when different charts of accounts are used. Other recommendations involve clarifying material differences between financial statements and budget figures. Differences in reconciliation should also be presented separately and not subtracted from one another. Despite the intricacies, another layer of credibility will be added to IPSAS by preparing these suggested reconciliations while also demonstrating the relationship between cash budgeting and accrual reporting.

Various case studies have been carried out in European settings on the topic of budgeting and accrual reporting. Van Der Hoerk ( 2005 ) was one of the first academics to explore this topic in the literature, identifying the contextual differences between the private and public sectors, and the dangers associated with recognising citizens as customers under a new basis of financial reporting. While the above study showed how The Netherlands transitioned to full accruals and encountered few challenges, the same cannot be said for Italy as officials there were less concerned with optimizing their plans for resource allocation, but more with the preparation of a “legally correct” budget (Grandis & Mattei, 2014 ).

These contrasting experiences are mirrored in a case study on the Polish and Maltese governments respectively. While Malta is committed to IPSAS adoption and less developed budgeting practices, Poland utilises sophisticated budgeting techniques and demonstrates no inclination to adopt IPSAS (Kowalczyk & Caruana, 2022 ).

The comparative viewpoints listed above provide a basis for standard-setters to develop a system that not only harmonises budgeting but also optimizes it, in tandem with accrual-basis reporting under IPSAS. While much of the literature presented above reflects glowingly on IPSAS, in the interest of being complete, it is important to highlight challenges associated with accrual reporting in the public sector as will be discussed in the subsequent section on issues and cultural barriers.

Challenges to IPSAS Adoption

IPSAS is undoubtedly a transformative step toward modernizing and harmonising public sector financial reporting, however, academics have identified general implementation issues and cultural barriers that may hamper a smooth transition to accrual public sector reporting.

Critical and specific government situations have not been accounted for in the development of standards as well as the suitability within European legal systems (Grossi & Soverchia, 2011 ; Gourfinkel, 2021 ). This paired with a lack of stakeholder willingness and the human resources required acts as a barrier to broader IPSAS diffusion. Equally, within the Belgian public sector, Christiaens and Rommel ( 2008 ) discussed some of the deficiencies in applying an accrual framework to the public sector. They argue that spending should not be related to the level of public services provided as in the private sector and that accruals would only be appropriate for “business-like” departments in government.

Other operational challenges have been documented in the literature such as asset measurement which gives rise to additional complexities than those experienced in the private sector (Lapsley et al., 2009 ; Brusca et al., 2016 ; Biondi & Lapsley, 2014 ). Typically, assets are recognised if probable future cashflows are expected. Caruana ( 2021 ) concludes that standards should draw less from private sector practices and proposes four potential alternatives to provide greater clarity for preparing IPSAS statements: historical cost, fair value, fulfilment value and replacement cost. Practical issues identified in the literature include implementation costs, lack of pressure, and alignment with public sector specialities in addition to the reliance on IFRS in standard development (Scmitdhuber et al., 2022 ). These concerns are valid and require careful consideration by the IPSASB to create a comprehensive framework that satisfies user and government needs.

Another point for consideration for the IPSASB is the cultural landscape in adopting countries. Baskerville and Grossi ( 2019 ) put forward the idea of “Glocalization” which is adapting universal standards to local cultural norms to facilitate the retention of identity. These slight changes make standard diffusion more palatable for those implementing IPSAS, provided these changes to standards do not compromise the quality. If adaptations do not reduce transparency or stewardship in public sector reporting, these variations should ease scepticism and make for a smoother transition around the world.

Economic factors such as the aftermath of the financial crisis (Schmidthuber et al., 2022 ) have played a significant role, particularly in Portugal, with the Portuguese government succumbing to pressure from the Troika after availing of aid after the financial crash (Jorge et al., 2019 ), reinforcing the findings of Mnif Sellami and Gafsi ( 2019 ) who found coercive factors to influence IPSAS implementation. The cultural context differed in neighbouring Spain with the Spanish public sector eager to mirror IFRS used in the private sector. Ultimately, institutions are unlikely to advance in the adoption of IPSAS without a thorough analysis of the external and macroeconomic factors including access to foreign aid, public perception, institutional dynamics and the availability of accounting professionals (Argento et al., 2018 ) as well as other cost-benefit analyses (Abdulkarim et al., 2020 ).

There is an added responsibility for higher education, particularly in the developed world, to supply an educated labour force to implement IPSAS in the civil service. Adam et al. ( 2020 ) found that universities are giving limited attention to public sector accounting, financial management and IPSAS, a trend requiring change if IPSAS is to be successful, particularly as on-the-job training is costly both financially and in terms of time.

Developing Countries and Cash-Basis IPSAS

Developing countries have attracted much attention to public financial reporting and management reforms. However, IPSAS is more than a technical accounting innovation and there are many barriers to its implementation, particularly in countries striving for economic independence and self-sufficiency. The application of IPSAS in the developing countries is more complex as developing countries are typically characterised by lower transparency, corruption and weak financial regulations (Tawiah, 2023a , b ). This shows that despite the good intentions of standard setters, IPSAS may not hold all the answers to fostering development in these countries.

One of the major barriers to IPSAS success in developing countries is the lack of access to technology (Whitefield & Savvas, 2016 ; Alshujairi, 2014 ; Gkouma & Filos, 2022 ) and education for those operating the systems (Fahmid et al., 2020 ; Adam et al., 2020 ; Amiri & Hamza, 2020 ) which makes the transition to accrual accounting more seamless. In addition to these drawbacks, Polzer et al. ( 2019 ) concluded that there was little evidence of politicians using accrual-based information for decision-making in these countries. This issue is then exacerbated by the lack of education, whereby citizens display little interest in the new information produced, leading to decoupling issues and a lack of information usefulness for the public.

Hepworth ( 2015 ) also provides an interesting perspective on how IPSAS may be an enabler of corruption in developing countries. The mechanics of accrual accounting and the prudence concept may allow politicians to overinflate provisions, recognise losses and misappropriate public funds for personal gain. To realize the full benefits, the author argues that there is a need for a country’s civil service to be independent of its political system, ensuring integrity is maintained.

Studies have been conducted on the topic of IPSAS in countries where integrity, transparency and governance have been questioned previously. Iraq and Qatar are interesting cases for different reasons. Qatar has embarked on a search for legitimacy as it looks to modernise elements of its economy, legal and accounting systems to Western standards while maintaining its unique culture and values. They are moving to a less conservative culture and revolutionising their public sector is a means of achieving this legitimacy (Abdulkarim et al., 2023 ). A survey of professional accountants in Qatar shows that they support the disclosure of more detailed and comprehensive information associated with accrual-based public sector accounting practices. However, standard setters and governments must be conscious and understand the cultural backgrounds of the country to support the transition to ubiquitous public sector financial reporting (Abdulkarim et al., 2023 ). Similarly, Iraq provides a fascinating landscape given the country’s move from a macro-economic-focused and centrally planned economy to one which promotes free-market interests. Unsurprisingly, these reforms have been initiated following the American occupation of the nation in 2003 (Alshujairi, 2014 ).

Other obstacles to increased efficiency in financial reporting as a result of IPSAS include relationships between central and local governments as identified by Ada and Christiaens ( 2018 ) in Turkey. Collaboration between local authorities and the Supreme Audit Institution is seen as key to operationalising IPSAS in Turkey. Co-operation between central and local bodies has been a long-standing issue in developing countries and hiders the implementation of international rules locally (Christiaens et al., 2015 ).

Managerial reforms and system changes are critical in helping countries of the developing countries realise the benefits of IPSAS. Accrual-basis reports and systems are not always feasible for economies and governments in transition from cash-basis budgeting and financial reporting. The cash-basis standard has been issued to ease this transition, an option which many developing countries have availed of.

As discussed in prior sections the reliance on foreign aid for developing countries has prompted them to incorporate the cash-basis standard as it is deemed to be suitable for their accounting systems and stage of development (Amiri & Hamza, 2020 ). However, Chan ( 2008 ), contended that the mere presence of a cash-basis standard questions the validity of accrual-basis IPSAS entirely. It is counter-productive as some governments are under the impression that cash-basis and accrual-basis IPSAS are equally acceptable.

Cash-basis IPSAS has been a discussion point for many developing countries such as Bangladesh (Rajib et al., 2019 ). The role played by accountants in institutionalising changes in Bangladesh determines the trajectory of reforms, like the mimetic forces highlighted in Estonia (Argento et al., 2018 ).

A case study set in Pakistan found that cash-basis IPSAS did not cover all elements of the government’s financial position as non-cash transactions were ignored, despite the value they hold in the economy and the link they share with long-term sustainability. Overall, it was found that the true financial position was not represented on cash basis (Javed & Zhuquan, 2018 ). Similarly, it was found in the Nepali context that other, more effective alternatives to cash-basis IPSAS were required (Adhikari et al., 2015 ).

Other concerns have been raised concerning the comparability across jurisdictions. Sutcliffe ( 2009 ) argues that homogeneity is impossible due to the lack of general principles underpinning the cash-basis standard, a gap which was filled by Mustapha et al. ( 2017 ) who proposed guidelines for creating a cash-basis format which enhances the understanding of government financial statements.

Currently, the developing countries are in transition in public financial reporting, and the literature highlights that cash-basis IPSAS is not the long-term solution, with economic development and institutional amendments required. It also reinforces the credibility of accrual IPSAS as countries continue their journey to long-term growth, sustainability and self-sufficiency.

Most professionals agreed that public financial management changes are needed, listing corruption, lack of public services and transparency surrounding budget implementation as the main reasons. However, as discussed above, barriers exist in the form of inadequate IT systems and a lack of human resources to implement such changes (Amiri & Hamza, 2020 ; Gkouma & Filos, 2022 ). Financial support to aid further development has been recognised in Iraq as a driver in favour of IPSAS implementation, reinforcing extant literature (Mnif Sellami & Gafsi, 2019 ; Alghizzawi & Masruki, 2020 ; Tawiah, 2023a ).

The research on IPSAS in the developing countries is inherently positive, with researchers concluding that improved information quality and heightened transparency were found in many nations including the Indian and Romanian settings (Narsaiah, 2019 ; Ilie & Miose, 2012 ). The desire for transparency and increased openness of accountancy professionals as discussed above can act as a catalyst for the long-term success of IPSAS in the developing countries.

The literature supports IPSAS as an innovative, more comprehensive and potentially transformative approach to public sector reporting. Accrual accounting techniques in the public sector will provide greater information about future liabilities, assets, and debts (Pina & Torres, 2003 ) while supporting long-term sustainability by recognising non-cash transactions. (Javed & Zhuquan, 2018 ). Proponents of New Public Management techniques refer to the increased efficiencies that will be realised, following the introduction of accrual accounting (ACCA, 2017 ; Lapsley et al., 2009 ; Fahmid et al., 2020 ).

The IPSASB’s approach to standard setting, mirroring IFRS, the private sector’s international framework has attracted mixed reactions, with some commending the increased legitimacy of using an already established accrual framework (Brusca et al., 2013 ; Jones & Caruana, 2016 ). Others, however, feel the basis for IPSAS ignores the complexities and institutional dynamics within the public sector (Rincón-Soto & Gómez-Villegas, 2020 ; Baskerville & Grossi, 2019 ).

Adoption rates have soared significantly in recent years (Christiaens et al., 2015 ; Bekiaris & Paraponti, 2023 ), increasing convergence in public sector reporting as has been desired by standard-setters. Many forces have been at play in prompting countries to implement the standards, both at local and central government levels, namely coercive, mimetic and normative forces. Access to aid (Ada & Christiaens, 2018 ; Alghizzawi & Masruki, 2020 ), the search for legitimacy (Jones & Caruana, 2016 ) and direction from professionals (Argento et al., 2018 ; Gomes et al., 2023 ).

This review has also focused on the developing countries, for which the standards have been advertised to reduce corruption. This is an issue which is prominent due to low fiscal transparency and weak regulations (Tawiah & Soobaroyen, 2023 ). IPSAS are not looked upon favourably in countries such as Nigeria (Olaoye & Talabi, 2018 ; Enofe et al., 2017 ), however, nations such as Qatar and Iraq view IPSAS as a means of creating a more accountable public sector and restoring public trust, nationally and globally (Alshujairi, 2014 ; Abdulkarim et al., 2023 ). On the contrary, IPSAS accrual programmes in Indonesia (Fahmid et al., 2020 ) were found to undermine other government programmes designed to facilitate growth in rural communities. This together with lack of education contributed to minimal understanding of the new information produced by the system. In addition, evidence of politicians using accrual information for decision-making in developing countries is limited (Polzer et al., 2019 ), conflicting with claims of professional bodies such as ACCA ( 2017 ).

Developing countries are seen by some to be limited in implementing full IPSAS and a modified cash basis of reporting has been suggested (Wynne, 2013) as being more appropriate. IPSASB has issued a cash-basis standard, to facilitate a smooth transition to accrual reporting, however, it may be detrimental to the credibility of the standards as decision-makers may view cash and accrual frameworks as equally acceptable in the long term (Chan, 2008 ).

Finally, IPSAS are not without their drawbacks including enabling manipulation due to accrual and prudence concepts (Christiaens & Rommel, 2008 ), lack of education (Adam et al., 2020 ), technology (Fahmid et al., 2020 ) and human resources (Alshujairi, 2014 ).

Suggestion for Future Research

It is evident from the literature review that, research in IPSAS is still in the early stages and requires more attention from scholars. Indeed, research on international public sector accounting standards (IPSAS) is essential to improve the transparency, accountability, and comparability of financial reporting in the public sector worldwide. Therefore, based on the review, future studies can explore the following areas.

First, researchers can undertake comparative studies between developed and developing countries on the adoption and consequences of IPSAS. Current studies have examined these groups of countries separately at different periods and provided mixed findings. It will be interesting to know how the adoption process in developed countries differs from developing countries. Also, will the benefits of adopting IPSAS be the same for developed and developing countries at the same time?

Second, other studies can investigate how culture and people’s perception of the government influence the relevance of IPSAS in the country. Arguably culture shaped how people see the role of government in the country. In a socialist culture, the government is seen as a key institution and manager of public funds, hence they expect the government to be more accountable and transparent. On the contrary in a capitalist culture, the government is more of a regulator and referee whose mandate is to make and enforce the laws. Some cultures may prioritize strict compliance with detailed rules, while others may be more inclined toward a principles-based approach. IPSAS, being principles-based, may face resistance or adaptation based on cultural orientations.

Third, the key theme for the adoption of IPSAS is the improvement of government reporting and accountability. If that is the case, then, can IPSAS influence the cost of borrowing and valuation of government financial instruments such as bonds? Although Tawiah and Soobaroyen ( 2023 ) show how the IPSAS adoption affects the different sources of finance, it does not cover the cost of borrowing and valuation of financial instruments.

Fourth, there is growing concern about government commitment to environmental sustainability and climate change actions. Therefore, future studies can explore the potential inclusion of environmental and social reporting within IPSAS to address sustainability and non-financial performance disclosure in the public sector.

Fifth, as highlighted in the review, developing countries are largely pressured to adopt international standards including the IPSAS, yet the evidence on the benefits is limited partly due to implementation issues. Therefore, future studies can examine the unique challenges faced by developing countries in adopting and implementing IPSAS, including institutional capacity, data availability, and political factors. Such studies can suggest strategies to support the gradual transition to IPSAS in resource-constrained environments.

Last but not least, future studies can investigate the role of emerging technologies such as blockchain, artificial intelligence, and data analytics in improving the accuracy and efficiency of IPSAS implementation and compliance. Assessing the potential for digital financial reporting and its impact on public sector transparency is also an area worth investigating.

Abdulkarim, M. E., Umlai, M. I., & Al-Saudi, L. F. (2020). Exploring the role of innovation in the level of readiness to adopt IPSAS. Journal of Accounting & Organizational Change, 16 (3), 469–495.

Article   Google Scholar  

Abdulkarim, M. E., Umlai, M. I., & Al-Saudi, L. F. (2023). Culture, language, and accounting reform: A new perspective on IPSAS implementation. Journal of Accounting & Organizational Change, 19 (2), 376–400.

ACCA. (2017). IPSAS implementation: Current status and challenges . The Association of Chartered Certified Accountants

Ada, S. S., & Christiaens, J. (2018). The magic shoes of IPSAS: Will they fit Turkey? Transylvanian Review of Administrative Sciences, 14 (54), 5–21.

Adam, B., Brusca, I., Caperchione, E., Heiling, J., Jorge, S. M. F., & Manes Rossi, F. (2020). Are higher education institutions in Europe preparing students for IPSAS? International Journal of Public Sector Management, 33 (2/3), 363–378.

Adhikari, P., Kuruppu, C., Wynne, A., & Ambalangodage, D. (2015). Diffusion of the cash basis international public sector accounting standard (IPSAS) in less developed countries (LDCs)–the case of the Nepali central government. In  The public sector accounting, accountability and auditing in emerging economies . Emerald Group Publishing Limited.

Alghizzawi, M. A., & Masruki, R. (2020). Issues on public sector accounting reform in Jordan. Journal of Modern Accounting and Auditing, 16 (3), 130–139.

Google Scholar  

Alshujairi, M. H. A. (2014). Government accounting system reform and the adoption of IPSAS in Iraq. Research Journal of Finance and Accounting, 5 (24), 1–20.

Amiri, A., & Hamza, S. E. (2020). The transition to IPSAS standards: The extent of adoption and the influence of institutional, contingency and economic network factors. Accounting and Management Information Systems, 19 (2), 215–251.

Amor, D. B., & Ayadi, S. D. (2019). The profile of IPSAS-adopters. Accounting and Management Information Systems, 18 (2), 262–282.

Argento, D., Peda, P., & Grossi, G. (2018). The enabling role of institutional entrepreneurs in the adoption of IPSAS within a transitional economy: The case of Estonia. Public Administration and Development, 38 (1), 39–49.

Baker, R. O. N., & Rennie, M. D. (2006). Canadian Accounting Perspectives, 5 (1), 83–112.

Ball, I. (2015). Debate: Would IPSAS help Greece? Public Money& Management, 35 (6), 397–398

Baskerville, R., & Grossi, G. (2019). Glocalization of accounting standards: Observations on neo-institutionalism of IPSAS. Public Money & Management, 39 (2), 95–103.

Bekiaris, M., & Paraponti, T. (2023). Examining the status of IPSAS adoption at the country level: An analysis of the OECD member states. Journal of Accounting & Organizational Change, 19 (2), 296–325.

Benito, B., Brusca, I., & Montesinos, V. (2007). The harmonization of government financial information systems: The role of the IPSASs. International Review of Administrative Sciences, 73 (2), 293–317.

Bergmann, A. (2012). The influence of the nature of government accounting and reporting in decision-making: Evidence from Switzerland. Public Money &Management, 32 (1), 5–20.

Biondi, Y. (2018). Fair value and the formation of financial market prices through ignorance and hazard. In  The Routledge Companion to Fair Value in Accounting  (Chap. 16, pp. 288–295). Routledge. Retrieved from https://doi.org/10.4324/9781315621876-16

Biondi, L., & Lapsley, I. (2014). Accounting, transparency and governance: The heritage assets problem. Qualitative Research in Accounting & Management, 11 (2), 146–164.

Brusca, I., Montesinos, V., & Chow, D. S. (2013). Legitimating international public sector accounting standards (IPSAS): The case of Spain. Public Money & Management, 33 (6), 437–444.

Brusca, I., Gómez-villegas, M., & Montesinos, V. (2016). Public financial management reforms: The role of IPSAS in Latin-America. Public Administration and Development, 36 (1), 51–64.

Caruana, J. (2021). The proposed IPSAS on measurement for public sector financial reporting—recycling or reiteration? Public Money & Management, 41 (3), 184–191.

Caruana, J., Dabbicco, G., Jorge, S., & Jesus, M. A. (2019). The development of EPSAS: Contributions from the literature. Accounting in Europe, 16 (2), 146–176.

Castañeda-Rodríguez, V. (2022). Is IPSAS implementation related to fiscal transparency and accountability?  BAR-Brazilian Administration Review ,  19 .

Chan, J. L. (2008). International public sector accounting standards: Conceptual and institutional issues. The Harmonization of Government, 21 , 1–15.

Christiaens, J., & Rommel, J. (2008). Accrual accounting reforms: Only for businesslike (parts of) governments. Financial Accountability & Management, 24 (1), 59–75.

Christiaens, J., Reyniers, B., & Rollé, C. (2010). Impact of IPSAS on reforming governmental financial information systems: A comparative study. International Review of Administrative Sciences, 76 (3), 537–554.

Christiaens, J., Vanhee, C., Manes-Rossi, F., Aversano, N., & Van Cauwenberge, P. (2015). The effect of IPSAS on reforming governmental financial reporting: An international comparison. International Review of Administrative Sciences, 81 (1), 158–177.

Cuadrado-Ballesteros, B., Citro, F., & Bisogno, M. (2020). The role of public-sector accounting in controlling corruption: An assessment of organisation for economic co-operation and development countries. International Review of Administrative Sciences, 86 (4), 729–748.

Enofe, A., Afiangbe, S., & Agha, D. (2017). Financial management reforms and corruption in Nigeria public sector. Financial Management, 3 (7), 1–22.

Fahmid, I. M., Harun, H., Graham, P., Carter, D., Suhab, S., An, Y., Zheng, X., & Fahmid, M. M. (2020). New development: IPSAS adoption, from G20 countries to village governments in developing countries. Public Money & Management, 40 (2), 160–163.

Gkouma, O. G., & Filos, J. (2022). Assessing the impact of IPSAS on financial reporting and public management in Greece. SPOUDAI-Journal of Economics and Business, 72 (1–2), 56–79.

Gomes, P., Brusca, I., Fernandes, M. J., & Vilhena, E. (2023). The IPSAS implementation and the use and usefulness of accounting information: A comparative analysis in the Iberian Peninsula. Journal of Public Budgeting, Accounting & Financial Management, 35 (1), 12–40.

Gourfinkel, D. (2021). The main challenges of Public Sector Accounting reforms and World Bank’s Public Sector Accounting and Reporting (PULSAR) Program. Journal of Public Budgeting, Accounting & Financial Management, 34 (2), 347–352.

Grandis, F. G., & Mattei, G. (2014). The authorising function of budgets in public administration. Applicability of IPSAS 24 in Italy.  Open Journal of Accounting ,  3 (02), 45.

Groot, T., & Budding, T. (2008). New public management’s current issues and future prospects. Financial Accountability & Management, 24 (1), 1–13.

Grossi, G., & Soverchia, M. (2011). European Commission adoption of IPSAS to reform financial reporting. Abacus, 47 (4), 525–552.

Hamed-Sidhom, M., Hkiri, Y., & Boussaidi, A. (2022). Does IPSAS adoption reduce corruption’s level? New evidence from ODA beneficiary countries. Journal of Financial Crime, 29 (1), 185–201.

Heiling, J. (2019). The presentation of budget information in financial statements–challenges in applying IPSAS 24. The International Journal on Governmental Financial Management, 19 (2), 54.

Hepworth, N. (2015). Debate: Implementing advanced public financial management reform in developing countries. Public Money & Management, 35 (4), 251–253.

Hepworth, N. (2017). Is implementing the IPSASs an appropriate reform? Public Money & Management, 37 (2), 141–148.

Humphrey, C., & Miller, P. (2012). Rethinking impact and redefining responsibility: The parameters and coordinates of accounting and public management reforms.  Accounting, Auditing & Accountability Journal .

Ijeoma, N. B. (2014). The impact of international public sector accounting standard (IPSAS) on reliability, credibility and integrity of financial reporting in state government administration in Nigeria. International Journal of Technology Enhancements and Emerging Engineering Research, 2 (3), 2347–4289.

Ilie, E., & Miose, N. M. (2012). IPSAS and the application of these standards in the Romania. Procedia-Social and Behavioral Sciences,  62 , 35–39.  https://doi.org/10.1016/j.sbspro.2012.09.008

ILO. (2021).  International Public Sector Accounting Standards (IPSAS) . Available at: https://www.ilo.org/caribbean/events-and-meetings/WCMS_818562/lang--en/index.htm . Accessed 10 June 2023.

Javed, M., & Zhuquan, W. (2018). Analysis of accounting reforms in the public sector of Pakistan and adoption of cash basis IPSAS. Universal Journal of Accounting and Finance, 6 (2), 47–53.

Jones, R., & Caruana, J. (2016). Governmental accounting in Malta towards IPSAS within the context of the European Union. International Review of Administrative Sciences, 82 (4), 745–762.

Jorge, S., Brusca, I., & Nogueira, S. P. (2019). Translating IPSAS into national standards: An illustrative comparison between Spain and Portugal. Journal of Comparative Policy Analysis: Research and Practice, 21 (5), 445–462.

Kowalczyk, M., & Caruana, J. (2022). Governmental accounting and budgeting in Malta and Poland–a different dance to the same tune.  International Journal of Public Sector Management , 35 (7), 783–805.

Lapsley, I., Mussari, R., & Paulsson, G. (2009). On the adoption of accrual accounting in the public sector: a self-evident and problematic reform.

López, B. B., Alijarde, I. B., & Julve, V. M. (2002). The IPSAS approach: a useful tool for accounting reform in Europe?  Innovations in governmental accounting  (pp. 85–97).

Mnif, Y., & Gafsi, Y. (2023). The influence of the socioeconomic and politico-administrative environment on the extent of compliance with IPSAS in developing countries.  Journal of Accounting in Emerging Economies .

Mnif Sellami, Y., & Gafsi, Y. (2019). Institutional and economic factors affecting the adoption of international public sector accounting standards. International Journal of Public Administration, 42 (2), 119–131.

Murphy, T., O’Connell, V., & Ó hÓgartaigh, C. (2013). Discourses surrounding the evolution of the IASB/FASB conceptual framework: What they reveal about the “living law” of accounting. Accounting, Organizations and Society, 38 (1), 72–91.

Mustapha, M., Ismail, K. N. I. K., & Ahmad, H. N. (2017). The adoption of cash-basis IPSAS: A conceptual framework for enhancing decision-useful financial reporting. Journal of Business Management and Accounting, 7 (1), 51–68.

Narsaiah, N. (2019). Adoption of International Public Sector Accounting Standards (IPSAS) in India: Feasibility, benefits and challenges. Indian Journal of Accounting (IJA), 51 , 1.

Obara, L. C., & Nangih, E. (2017). International Public Sector Accounting Standards (IPSAS) Adoption and governmental financial reporting in Nigeria-An empirical investigation. Journal of Advance in Social Science and Humanities ISSN, 2395 , 6542.

Olaoye, F. O., & Olaniyan, N. O. (2018). An empirical evaluation of IPSAS adoption and public sector financial management. International Journal of Innovative Research and Advanced Studies (IJIRAS), 5 (5), 3–16.

Olaoye, F. O., & Talabi, A. O. (2018). International Public Sector Accounting Standards (IPSAS) and credibility of financial reporting in Nigeria public sector: An improvement or a ruse. Research Journal of Finance and Accounting, 9 (14), 68–74.

Olson, 0.. Humphrey, C., & Guthrie, J. (2000). Caught in an evaluatory trap: The dilemma of “public services” under NPFM. ElASM International Conference on Accounting, Auditing and Management in Public Sector Reforms .

Parry-michael, M., & Wynne, A. (2009). The cash basis IPSAS: An alternative view.  International Journal of Government Financial Management , 23.

Peter Van Der Hoek, M. (2005). From cash to accrual budgeting and accounting in the public sector: The Dutch experience. Public Budgeting & Finance, 25 (1), 32–45.

Pina, V., & Torres, L. (2003). Reshaping public sector accounting: An international comparative view. Canadian Journal of Administrative Sciences/revue Canadienne Des Sciences De L’administration, 20 (4), 334–350.

Polzer, T., Gårseth-Nesbakk, L., & Adhikari, P. (2019). “Does your walk match your talk?” Analyzing IPSASs diffusion in developing and developed countries. International Journal of Public Sector Management, 33 (2/3), 117–139.

Rajib, S.U., Adhikari, P., Hoque, M., & Akter, M. (2019). Institutionalisation of the cash basis International Public Sector Accounting Standard in the central government of Bangladesh: An example of delay and resistance.  Journal of Accounting in Emerging Economies .

Rincón-Soto, C. A., & Gómez-Villegas, M. (2020). Institucional isomorphism in IPSAS adoption. Cuadernos De Administración (universidad Del Valle), 36 (68), 204–218.

Rossi, F. M., Cohen, S., Caperchione, E., & Brusca, I. (2016). Harmonizing public sector accounting in Europe: Thinking out of the box. Public Money & Management, 36 (3), 189–196.

Sanderson, I., & van Schaik, F. (2008). Public sector accounting standards: Strengthening accountability and improving governance. Accountancy Ireland, 40 , 22–24.

Schmidthuber, L., Hilgers, D., & Hofmann, S. (2022). International Public Sector Accounting Standards (IPSASs): A systematic literature review and future research agenda. Financial Accountability & Management, 38 (1), 119–142.

Sutcliffe, P. (2009). International public sector accounting standards board review the cash basis IPSAS: An opportunity to influence developments. International Journal on Governmental Financial Management, 9 (2), 15–22.

Tawiah, V. (2023a). The impact of IPSAS adoption on corruption in developing countries. Financial Accountability & Management, 39 (1), 103–124.

Tawiah, V. (2023b). The effect of IPSAS adoption on governance quality: Evidence from developing and developed countries. Public Organization Review, 23 (1), 305–324.

Tawiah, V., & Soobaroyen, T. (2023). The relationship between the adoption of international public sector accounting standards and sources of government financing: evidence from developing countries. In  Accounting Forum  (pp. 1–24). Routledge.

Toudas, K., Poutos, E., & Balios, D. (2013). Concept, Regulations and Institutional Issues of IPSAS: A critical review. European Journal of Business and Social Sciences, 2 (1), 43–54.

van Schaik, F. D. (2021). Reconciliation of budgeting and accounting.  Public Money & Management  (pp. 1–10).

Vivian, B., & Maroun, W. (2018). Progressive public administration and new public management in public sector accountancy: An international review.  Meditari Accountancy Research .

Whitefield, A. A., & Savvas, P. (2016). The Adoption and Implementation of the International Public Sector Accounting Standards: The challenges faced by the United Nation in producing UN-IPSAS compliant financial reports in Kenya. International Journal of Finance and Accounting, 1 (1), 75–91.

Whittington, G. (2008). Fair value and the IASB/FASB conceptual framework project: An alternative view. Abacus, 44 (2), 139–168.

Wynne, A. (2013). International public sector accounting standards: compilation guide for developing countries, International Consortium on Governmental Financial Management. https://www.scribd.com/doc/134603499/ICGFM-Compilation-Guide-to-Financial-Reportingby-Governments . Accessed 29 May 2023.

Zarei, H., Yazdifar, H., Dahmarde Ghaleno, M., & Namazi, N. (2022). National culture and public-sector budgeting: The mediating role of country-level institutions using a structural equation modeling approach. Journal of Applied Accounting Research, 23 (3), 686–714.

Zhang, Y., & Andrew, J. (2014). Financialisation and the conceptual framework. Critical Perspectives on Accounting, 25 (1), 17–26.

Download references

Open Access funding provided by the IReL Consortium.

Author information

Authors and affiliations.

Dublin City University, Dublin, Ireland

Sean Scannell & Vincent Tawiah

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Vincent Tawiah .

Ethics declarations

Ethical approval, informed consent, conflict of interest.

There is no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Scannell, S., Tawiah, V. A Thematic Literature Review on International Public Sector Accounting Standards (IPSAS). Public Organiz Rev (2024). https://doi.org/10.1007/s11115-024-00773-1

Download citation

Accepted : 27 April 2024

Published : 29 May 2024

DOI : https://doi.org/10.1007/s11115-024-00773-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• International accounting
• Developing countries
• Public sector accounting
• Government accounting
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 04 June 2024

Sharpening the lens to evaluate interprofessional education and interprofessional collaboration by improving the conceptual framework: a critical discussion

• Florian B. Neubauer 1 ,
• Felicitas L. Wagner 1 ,
• Andrea Lörwald 1 &
• Sören Huwendiek 1  

BMC Medical Education volume  24 , Article number:  615 ( 2024 ) Cite this article

75 Accesses

Metrics details

It has been difficult to demonstrate that interprofessional education (IPE) and interprofessional collaboration (IPC) have positive effects on patient care quality, cost effectiveness of patient care, and healthcare provider satisfaction. Here we propose a detailed explanation for this difficulty based on an adjusted theory about cause and effect in the field of IPE and IPC by asking: 1) What are the critical weaknesses of the causal models predominantly used which link IPE with IPC, and IPE and IPC with final outcomes? 2) What would a more precise causal model look like? 3) Can the proposed novel model help us better understand the challenges of IPE and IPC outcome evaluations? In the format of a critical theoretical discussion, based on a critical appraisal of the literature, we first reason that a monocausal, IPE-biased view on IPC and IPC outcomes does not form a sufficient foundation for proper IPE and IPC outcome evaluations; rather, interprofessional organization (IPO) has to be considered an additional necessary cause for IPC; and factors outside of IPC additional causes for final outcomes. Second, we present an adjusted model representing the “multi-stage multi-causality” of patient, healthcare provider, and system outcomes. Third, we demonstrate the model’s explanatory power by employing it to deduce why misuse of the modified Kirkpatrick classification as a causal model in IPE and IPC outcome evaluations might have led to inconclusive results in the past. We conclude by applying the derived theoretical clarification to formulate recommendations for enhancing future evaluations of IPE, IPO, and IPC. Our main recommendations: 1) Focus should be placed on a comprehensive evaluation of factual IPC as the fundamental metric and 2) A step-by-step approach should be used that separates the outcome evaluation of IPE from that of IPC in the overarching quest for proving the benefits of IPE, IPO and IPC for patients, healthcare providers, and health systems. With this critical discussion we hope to enable more effective evaluations of IPE, IPO and IPC in the future.

Peer Review reports

There is scant knowledge on the extent to which the quality of interprofessional education (IPE) and interprofessional collaboration (IPC) at healthcare institutions influences the patient care quality [ 1 , 2 , 3 ], the cost effectiveness of patient care, the job satisfaction of healthcare professionals [ 1 ] and, as a result, their retention [ 4 , 5 ]. Patients, people who organize and finance healthcare, policy makers, tax payers, and arguably societies as a whole have a reasonable interest in an answer to this question.

According to the peer-reviewed literature, relevant knowledge gaps persist about the benefits of IPE and IPC despite multiple studies on IPE and IPC outcomes covering a period of almost 50 years [ 2 , 3 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 ]. Several explanations as to how this can be possible are proposed: The number of evaluation studies is still too low [ 10 ]; the time periods typically covered by evaluations is too short to detect final outcomes of IPE/IPC interventions [ 2 , 8 , 11 , 14 , 15 ]; too much focus is placed on immediate results without including measures for final outcomes from the outset [ 10 ]; or, ultimately, positive effects of IPE and IPC simply might not exist [ 6 , 9 , 10 ]. Another frequent and non-contradictory explanation proposes that a lack of clarity in theory and terminology of IPE and IPC and an insufficient use of conceptual frameworks are major deficits which obscure evaluation results [ 8 , 12 , 13 , 16 , 17 , 18 , 19 , 20 , 21 ].

In this article, we argue the latter: That an insufficient use of conceptual frameworks has obscured evaluation results. We propose that the persistence of the knowledge gap relating to patient outcomes, satisfaction of healthcare professionals, and cost effectiveness of IPE and IPC activities (briefly, “patient, healthcare provider, and system outcomes”) is rooted in a lack of accuracy in the theoretical models used for mapping causes and effects in IPE and IPC. Our objective is to contribute to overcoming the inconclusiveness in IPE and IPC outcome evaluations by achieving the missing accuracy through the lens of a novel “multi-stage multi-causality” model. Specifically, our research questions are: 1) What are the critical weaknesses of the causal models predominantly used which link IPE with IPC, and IPE and IPC with final outcomes? 2) What would a more precise causal model look like? 3) Can the proposed novel model help us better understand the challenges of IPE and IPC outcome evaluations?

In answering these questions, we first show evidence from the literature that the existing causal models of IPE and IPC exhibit a crucial imprecision. Second, we present the “multi-stage multi-causality model of patient, healthcare provider, and system outcomes” which fixes this imprecision by making a small but important modification to the causal role of IPO. Third, we demonstrate the explanatory power of the multi-stage multi-causality model showing why evaluations using the modified Kirkpatrick classification of interprofessional outcomes (MKC) [ 11 , 22 , 23 ] — a tool commonly used to evaluate outcomes of IPE activities — have failed to substantiate positive outcomes of IPE and IPC; namely, we show how the misuse of MKC leads to inconclusiveness and difficulties in evaluating final patient, healthcare provider, and system outcomes. We conclude with recommendations for future evaluations in the field of IPE, IPO and IPC.

With this theoretical investigation, we hope to contribute to a deeper understanding of the causal factors in IPE, IPO and IPC and to enable more precise evaluations in the future.

Based on our research questions, we performed iterative literature searches (detailed below) followed by critical appraisal by the authors, and transformed the resulting insights into the critical discussion presented in the main section of the present article by applying the 6 quality criteria of the SANRA scale [ 24 ]:

Justification of the article's importance for the readership: Our target audience consists of researchers whose goal is to evaluate whether IPE, IPO or IPC improve patient, healthcare provider, and system outcomes. For our target audience the present study is meaningful because it advances the understanding of the theoretical foundations of evaluations in this field. Further, in local contexts where the potential of IPE, IPO, and IPC is still neglected, clear evidence demonstrating substantial benefits would help to foster programs aimed at implementing better IPE, IPO, or IPC.

Statement of concrete/specific aims or formulation of questions : We set out to explore the following questions: 1) What are the critical weaknesses of the causal models predominantly used which link IPE with IPC, and IPE and IPC with final outcomes? 2) What would a more precise causal model look like? 3) Can the proposed novel model help us better understand the challenges of IPE and IPC outcome evaluations?

Description of literature searches: We searched for existing definitions, causal models, relevant indicators, and evaluation instruments for IPE, IPO, and IPC using PubMed, Google and Google Scholar with the following search terms in different combinations: “interprofessional education”, “interprofessional collaboration”, “interprofessional organization”, “interprofessional team work”, “evaluation”, “outcome evaluation”, “process evaluation”, “modified Kirkpatrick”, “conceptual framework”, “theory”, “model”, “instrument”, “assessment scale”, “survey”, “review”. We conducted all searches in English, covering the time period from 1950 to 2023. We augmented the initial body of literature found by this strategy with citation tracking: for backward tracking, we followed the references provided in articles which we deemed relevant for our research questions; for forward searches, we used the "cited by" feature of PubMed and Google Scholar. The subchapter-specific literature search used in the development of our definition of IPC is described under “Definition of factual IPC”.

Referencing: We consistently back key statements by references.

Scientific reasoni ng: We enable the reader to easily follow our narrative by structuring the present article around the three research questions as stated above, following a logical flow of arguments.

Appropriate presentation of data: We present the data by distinguishing which findings were taken from the literature and which novel arguments for answering the research questions were derived by us.

Definitions

Definition of ipe.

Occasions when two or more healthcare/social care professions learn with, from and about each other to improve collaboration and the quality of care for patients/clients [ 2 ] (slightly refining the CAIPE definition [ 25 ]).

These occasions can happen formally or informally, in dedicated educational settings or at the workplace of healthcare/social care professions, and at any stage along the learning continuum, i.e. foundational education, graduate education, and post-licensure continuing professional development [ 8 , 26 ]. The central concept in IPE is learning [ 13 ], the gain of knowledge, skills, and attitudes, or — from a constructivist’s perspective — changes in the brains of individuals.

Definition of factual IPC

Presence of activities in the following 7 dimensions:

Patient-centered care, including a shared treatment plan and effective error management;

Shared creation of the treatment plan and coordination of its execution;

Mutual respect between professions;

Communication, including shared decision-making, sharing of information, appropriate communication tools, and accessibility of team members;

Shared definition and acceptance of roles and responsibilities;

Effective conflict management; and

Leadership, including outcome orientation.

How did we arrive at this definition? IPC has to be distinguished from traditional “multiprofessional collaboration”. In multiprofessional collaboration, patient care is organized in a discipline-oriented way, affecting its organization, leadership, communication, and decision-making. Different professions work separately, each with their own treatment goals; the physician delegates treatment options to the other healthcare professionals in one-way, mostly bilateral communication [ 27 , 28 ]. IPC, in contrast, is defined as the occasions “when multiple health workers from different professional backgrounds work together with patients, families, carers and communities to deliver the highest quality of care” [ 29 ]. This definition by the WHO remains in use today [ 30 ]. However, we found that, in order to talk about specific effects of IPE on IPC and to tailor evaluations towards less ambiguous results, an operationalized definition of IPC is required which provides a higher level of applicability. To create such a definition, we searched the literature to collect a comprehensive list of IPC dimensions which covers all possible settings of IPC. In an iterative process of content-based thematic clustering, reviews, original articles and preexisting questionnaires on the evaluation of IPE and IPC were added until there was agreement between the authors that saturation was reached with regard to all relevant IPC dimensions. This resulted in the following list of publications: [ 3 , 7 , 9 , 19 , 26 , 28 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 ]. Next, we clustered the terms for IPC dimensions found in this body of literature by consensus agreement on sufficient equivalence between three of the authors (FBN, FLW, SH). Clustering was required due to a lack of consistent terminology in the literature and resulted in the comprehensive set of 7 IPC dimensions used in our definition of IPC provided above. Finally, we needed to differentiate IPC from IPE and learning: At the workplace, informal learning happens all the time. As a result, interprofessional work processes can comprise both IPC and IPE at the same time; however, interprofessional learning is only a possible , not a necessary element of IPC and hence was not included in our definition of IPC. For example, a healthcare professional who is fully equipped with all competencies required for factual IPC could proficiently work in an established team in an interprofessional way without having to learn any additional IPC-related skills.

In order to stress that our definition of IPC includes all the healthcare-related interprofessional work processes actually taking place but excludes the activities required to create them (those fall in the domains of IPE or IPO), we use the term “factual IPC” throughout the present article. Factual IPC not only happens in formal interprofessional work processes like regular, scheduled meetings but also “on the fly”, i.e. during informal and low-threshold communication and collaboration.

Definition of IPO

All activities at a healthcare institution which create, improve, or maintain regular work processes of factual IPC or create, improve, or maintain institutional conditions supporting formal and informal parts of factual IPC, but excluding activities related to IPE.

There is no agreed upon definition of IPO in the literature, so we propose this refined one here that is broad enough to encompass the full variety of IPC-supporting activities at a healthcare institution while, at the same time, being narrow enough to exclude all manifestations of IPE.

According to this definition, IPO complements IPE within the set of jointly sufficient causes of factual IPC. IPO comprises all conditions required for the realization of factual IPC which are not related to interprofessional learning. It includes the actions of healthcare managers to implement work processes for IPC and to create supportive conditions for IPC (cf. the definitions of IPO in [ 6 , 8 , 13 , 17 , 23 , 26 , 30 , 31 , 33 , 40 , 41 , 42 ]). All interventions which establish or improve interprofessional work processes, i.e. which change how things are done in patient care, or which improve the conditions for factual IPC at an institution, belong in the domain of IPO. IPO is also the continued support for factual IPC by management like encouragement, clarification of areas of responsibility, incentives, staffing, room allocation, other resources, or funding. In contrast, established and regular interprofessional tasks themselves, after they have become part of the day-to-day work life of healthcare teams, without requiring further actions by management, would be categorized as factual IPC, not IPO.

Taken together, IPE is the umbrella term for planning, organizing, conducting, being subject to, and the results of interprofessional learning activities, whereas IPO is the umbrella term for all other activities that, in addition to individual competencies of team members, are necessary to cause factual IPC of high quality.

Critical discussion

What are the critical weaknesses of the causal models predominantly used which link ipe with ipc, and ipe and ipc with final outcomes.

We start by exploring the models in the literature that describe causes and effects of interprofessional activities in the context of patient care. We will derive evidence from the literature that the existing models exhibit a crucial imprecision regarding the causal role of IPO.

The causal model of IPC proposed by the WHO [ 29 , 43 ] (Fig.  1 ) was the model predominantly used in past evaluations of IPE and IPC. The WHO model suggests that IPE-related learning leads to IPC competence (knowledge, skills, and attitudes) in the “health workforce” that is “IPC-ready” post-IPE. This readiness “automatically” leads (as the long diagonal arrow in Fig.  1 suggests) to factual IPC. “The World Health Organization and its partners acknowledge that there is sufficient evidence to indicate that effective interprofessional education enables effective collaborative practice” [ 29 ]. Factual IPC, in turn “strengthens health systems and improves health outcomes” [ 29 ]. As a result, this model suggests a kind of “transitivity” between first causes and last effects: effective IPE activities are expected to ultimately yield positive patient, healthcare provider, and system outcomes on their own.

The WHO model of causes and effects in IPE and IPC (from [ 29 ], with permission)

After its publication, the WHO model was regularly cited and endorsed by IPE experts and continues to exert broad influence today. As of October 6, 2023, the “Google Scholar” search engine showed the original publication [ 29 ] to have 4393 citations, 367 of them in 2023 alone.

It is important to note that the WHO model is monocausal with respect to IPC, i.e. IPE is the sole necessary cause for factual IPC. While the model acknowledges that, next to IPE, there are further “mechanisms that shape how collaborative practice is introduced and executed”, it only ranks them as supportive: “ Once a collaborative practice-ready health workforce is in place [emphasis added], these [additional] mechanisms will help them [policy-makers] determine the actions they might take to support [emphasis added] collaborative practice” [ 29 ]. The following quotes by Reeves and colleagues further illustrate the strong emphasis causal models used to put on IPE: «It is commonly argued that IPE can promote the skills and behaviours required for effective IPC, which in turn can improve quality of health care and patient outcomes” [ 17 ] and “National organisations have created core competencies for interprofessional collaborative practice, positioning IPE as fundamental to practice improvement [emphasis added]” [ 10 ]. A couple of years later, Paradis and colleagues even state: “During this wave [of IPE; 1999–2015], advocates suggested IPE as the solution to nearly every health care problem that arose (…)” [ 6 ].

However, a scoping review by Reeves and colleagues aimed at improving “conceptualization of the interprofessional field” published soon after the WHO model, already acknowledged that the monocausal picture of factual IPC is incomplete [ 17 ]. Based on a broad analysis of the literature, their review offers a theoretical “Interprofessional framework” that includes the notion of IPO as an additional and different possible cause for desired interprofessional outcomes (Fig.  2 ). They define IPO interventions as “changes at the organizational level (e.g. space, staffing, policy) to enhance collaboration and the quality of care”. The explicit inclusion of IPO in this causal model of IPC was a very important step forward. The authors position IPO interventions parallel to IPE interventions, clearly indicating that IPO is an additional possible cause for desired interprofessional objectives and outcomes. However, in their framework, the capacity of IPO to be a second necessary cause in addition to IPE had not been clearly worked out yet.

The Interprofessional Framework (from [ 17 ], reprinted by permission of the publisher (Taylor & Francis Ltd, [ 44 ]). Note that, next to IPE, IPO is listed as a different, additional cause for desired interprofessional objectives and outcomes, but the crucial concept that it also is a necessary cause has not yet been worked out here

Side note: This model and publications using it (e.g. [ 45 ]) specify “Interprofessional Practice” (IPP) as a fourth domain, different from IPO (Fig.  2 , middle column). However, the IPP elements describe interventions that support work processes of factual IPC, and support for work processes of factual IPC is fully included in our definition of IPO. As a result, we see no necessity to set IPP apart from IPO and do not include IPP as an additional domain in our model below.

For completeness’ sake, we want to mention another explicit model by D’amour and Oandasan [ 26 ] with a comparable level of causal clarity which similarly claims that “there are many factors that act as determinants for collaborative practice to be realized”. As this model does not alter our line of argument it is not shown here.

The ongoing imprecision about the causal role of IPO naturally led to the next iteration of models. The authors of a 2015 review, commissioned by the Institute of Medicine of the National Academy of Sciences (IOM), provide the most recent influential model of causes and effects in IPC which they call “Interprofessional learning continuum model” [ 8 ] (Fig.  3 ).

The Interprofessional learning continuum model (from [ 8 ], with permission). Under the labels of “Institutional culture”, “Workforce policy”, and “Financing policy” it not only comprises IPO but assigns to IPO the crucial property of being an “enabling” factor, i.e. being co-causal for factual IPC (here labeled as “Collaborative behavior” and “Performance in practice”, lower left row). Despite this important improvement, the hierarchy of causes and effects remains partially vague: a The green arrow seems to imply direct effects of IPO on health and system outcomes without acknowledging that if IPO is supposed to have an effect on those at all, it necessarily must improve factual IPC first. b The impression remains that factual IPC mainly belongs on the left-hand side, being primarily an effect of IPE. IPO seems less effective on IPC, depending on how one interprets the influence of the green arrow on the larger red box which groups learner, health and system outcomes. c The left tip of the red double arrow in the center, indicating an effect of health and system outcomes on learning outcomes, is not discussed in the publication

In comparison to the WHO model (Fig.  1 ) and the Interprofessional Framework (Fig.  2 ), this causal model acknowledges that IPO is not just an additional but also a necessary cause of IPC and thus provides the most elaborate description of the causal relationships between IPE, IPO and IPC in the literature so far. The authors state, “Diverse and often opaque payment structures and differences in professional and organizational cultures generate obstacles to innovative workforce arrangements, thereby impeding interprofessional work. On the other hand, positive changes in workforce and financing policies could enable [emphasis added] more effective collaboration (…)” [ 8 ]. The word “enable” implies causal necessity : if an enabling factor is absent, the effect is disabled, hence the enabling factor is necessary . The key insight that IPO is a further necessary cause of IPC next to IPE can be found in several other, partly less recent publications, with the only difference that these publications do not embed this insight in a formal model [ 6 , 7 , 13 , 23 , 31 , 33 , 41 , 42 ]. The causal necessity of IPO becomes evident if one considers the extreme case: imagine a healthcare team whose individual members have all learned through IPE the skill set necessary for high quality IPC, i.e. they are optimally trained for IPC. However, they work at an institution that does not support proper IPC work processes, e.g. there is no dedicated time for team discussions of treatment plans and no electronic tools that allow all team members equal access to patient data. Consequently, there effectively cannot be an optimal manifestation of factual IPC, and it is impossible to expect that the IPE that the team members experienced during their training will significantly affect the quality of patient care in this setting.

What would a more precise causal model look like?

As we have seen, the notion of IPO in causal models of interprofessionality in the literature progressed from “IPO supportive” (Fig.  1 ) to “IPO possible but optional” (Fig.  2 ) to “IPO enabling, i.e. necessary” (Fig.  3 ). The key result of our study is a refinement missing from the existing causal model of IPE/IPO/IPC. It is the explicit statement that IPO is an equally necessary factor next to IPE in the causation of factual IPC. Only jointly are IPE and IPO sufficient to cause factual IPC of high quality. We deem this small modification crucial to reach the conceptual resolution required to fully understand the causes of factual IPC. The fully adjusted causal model is presented in Fig.  4 . In this “multi-stage multi-causality model of patient, healthcare provider, and system outcomes”, IPO is now unequivocally labeled as co-necessary for factual IPC alongside IPE-caused individual competencies.

Multi-stage multi-causality model of patient, healthcare provider, and system outcomes. Key ideas: IPO is an equally necessary co-factor in the causation of high-quality factual IPC, in addition to IPE. And the entire realm of interprofessional activities (red-outlined box), of which factual IPC is the final and active ingredient, is in turn only one of several causes leading to final outcomes of interest. Orange boxes: Domain of IPE, the domain of acquisition of competencies for IPC by an individual person through learning. Blue boxes: Domain of IPO, defined as the institutional domain of implementation, improvement, and maintenance of work processes of factual IPC and of IPC-supportive institutional conditions. Green box: Domain of factual IPC at a healthcare institution. Green-gray box, bottom row: Final outcomes of interest, i.e. patient care quality, job satisfaction of healthcare professionals, and cost effectiveness of patient care

Much more explicitly than previous ones, the multi-stage multi-causality model further shows that there are additional necessary causes for beneficial patient, healthcare provider, and system outcomes that lie entirely outside of the realm of IPC-related activities (i.e. outside of IPE/IPO/IPC). It is important to understand that not only factual IPC, but also the final patient, healthcare provider, and system outcomes have more than only one necessary cause, as reflected in the concept of “multi-causality on multiple stages”. This means that optimizing factual IPC is necessary but still not sufficient to optimize patient, healthcare provider, and system outcomes. Examples for necessary co-factors on the same level as factual IPC but from outside the realm of IPE/IPO/IPC are a) profession-specific (“uniprofessional”) competencies for aspects of a task that can only be accomplished by members of a specific healthcare profession ( task work vs. team work in [ 41 ]), b) details of health insurance policies, which can affect the cost effectiveness of patient care [ 46 ], salaries paid to health professionals by a healthcare institution, a factor which can influence job satisfaction [ 47 ], or good management decisions at an institution of patient care in general which comprise much more than just full support for factual IPC [ 46 ].

It should be noted that the co-causality in this conceptual framework is not compatible with the transitivity of the WHO model, where IPE ultimately leads to patient and healthcare provider outcomes via a predefined chain of “self-sustaining” secondary effects.

In sum, the adjusted causal model proposes that patient, healthcare provider, and system outcomes depend on multi-stage multi-causality. Stage 1: IPE + IPO = factual IPC: competencies for IPC in the workforce, the final result of interprofessional learning (IPE), plus creating and maintaining IPC work processes and supportive institutional conditions (IPO) together cause factual IPC. Stage 2: Factual IPC + non-interprofessional factors = patient, healthcare provider, and system outcomes: Factual IPC of high quality plus additional necessary but interprofessionality-independent factors together cause the final outcomes of interest.

The intention of our notion of “multi-stage multi-causality” is not to devalue the arrow-less “causal halos” of contextual factors in other models but rather to emphasize that even in “complex” systems (systems with multiple interacting elements) the actual sequence of causes and effects should be understood as precisely as possible for optimizing evaluations.

Brandt and colleagues, after reviewing the impact of IPE and IPC, note in their outlook on IPE, “given the complexity of the healthcare world, training learners in effective team work may not ultimately lead to improved health outcomes or reduce the cost of care” [ 9 ]. We don’t share this degree of pessimism; above we have shown that a monocausal, IPE-biased view on IPC simply might be insufficient for proper outcome evaluation of IPE and IPC. There is hope that by considering IPO, evaluations will become more conclusive. Wei and colleagues state in a systematic meta-review of systematic reviews about IPC, “Effective IPC is not linear; it does not occur naturally when people come together but takes a whole system’s efforts, including organizations, teams, and individuals” [ 30 ]. As we have explained, IPO has to be factored in as an additional necessary cause for IPC, and factors from outside the realm of IPE/IPO/IPC contribute to the “hard” outcomes of interest as well. We presented an adjusted causal model which explicitly acknowledges this multi-stage multi-causality of patient, healthcare provider, and system outcomes.

Can the proposed novel model help us better understand the challenges of IPE and IPC outcome evaluations?

We claim that the multi-stage multi-causality model exhibits strong explanatory power with regards to the difficulties of showing positive consequences of IPE and IPC in outcome evaluations in the past. To illustrate this, we must first describe the prominent role the modified Kirkpatrick classification of interprofessional outcomes [ 11 , 22 , 23 ] plays in outcome evaluations of IPE and IPC.

The modified Kirkpatrick classification (MKC)

MKC is regularly used to classify outcomes of IPE learning activities, curricula and programs [ 2 , 8 , 14 , 20 , 42 , 45 , 48 , 49 ]. It is a derivative of the original Kirkpatrick model for evaluating training results, named after its author, Donald L. Kirkpatrick, which distinguishes four categories of learning outcomes (Level 1: Reaction, Level 2: Learning, Level 3: Behavior, Level 4: Results) [ 50 , 51 ]. Expanding the original model, MKC assigns outcomes of IPE activities to six categories [ 11 ]:

Level 1: Reaction

Level 2a: Modification of perceptions & attitudes

Level 2b: Acquisition of knowledge & skills

Level 3: Behavioural change

Level 4a: Change in organisational practice (wider changes in the organization and delivery of care)

Level 4b: Benefits to patients/clients

In 2007, the authors of MKC claimed, “We have used these categories since 2000. They have proved useful and, contrary to our initial expectations, sufficient to encompass all outcomes in the hundreds of studies reviewed to date” [ 11 ]. This completeness has made MKC a useful tool for authors of review articles as it allows a retrospective classification of IPE outcomes not labeled in the original literature. As a result, MKC was quickly adopted by IPE evaluators around the world to describe the effectiveness of IPE interventions. As Thistlethwaite and colleagues put it in 2015, “This (…) model is now ubiquitous for health professional education evaluation” [ 42 ].

At first glance, the existence of such a clear and simple classification of IPE outcomes which not only covers all possible IPE outcomes but also is widely embraced in the literature, seems to be good news. What exactly is the problem then? Why did the introduction of MKC more than twenty years ago, plus the conceptual clarification provided by it, not resolve the difficulty in demonstrating IPE-caused patient, healthcare provider, and system outcomes (i.e. effects on MKC levels 4a and 4b)? In the following, we unfold a detailed answer to this question after application of the multi-stage multi-causality model.

To achieve progress, IPE and IPC outcome evaluations need to be complemented with process evaluations

MKC classifies outcomes but is agnostic about how these outcomes come into existence. For an evaluator using MKC, the effects of IPE-related interventions unfold inside a black box. The input into the black box is the intervention, the output constitutes 6 different classes of outcomes, i.e. the 6 levels of MKC described above. Naturally, such solely outcome-focused evaluations cannot explain functional interdependencies between the elements of the system. As we have seen, the benefits of IPE and IPC do not unfold as trivially as initially thought. Therefore, after two decades of (overall rather) inconclusive results of applying MKC to the outcomes of interprofessional interventions, the “why” should have moved to the center of the IPE evaluation efforts. This question is posed variously under well-known labels: Authors aware of said stagnancy either call for “formative evaluation” [ 52 ], “process evaluation” [ 14 ], or “realist evaluation” [ 42 ] in order to understand why interventions work as intended or not. In the following, we use the term “process evaluation” because we focus on understanding the underlying mechanisms.

Process evaluations require a causal model

Process evaluations require a causal model for the system under study to be able to select relevant indicators from a potentially much larger number of conceivable indicators. Appropriately selected indicators, which reflect the inner mechanisms of the system, then replace the black box, reveal bottlenecks, and allow explanations as to why interventions did or did not have the expected or intended outcomes. To explicitly demand the use of a causal model in an evaluation is a core principle, for example, of the “realistic evaluation” approach [ 53 ]. By directly criticizing the (original) Kirkpatrick model, Holton similarly suggests that a “researchable evaluation model” is needed which should “account for the effects of intervening variables that affect outcomes, and indicate causal relationships” [ 54 ]. Specifically for the domain of IPE and IPC, Reeves and colleagues [ 20 ] recommend “the use of models which adopt a comprehensive approach to evaluation” and the IOM authors conclude, “Having a comprehensive conceptual model provides a taxonomy and framework for discussion of the evidence linking IPE with learning, health, and system outcomes. Without such a model, evaluating the impact of IPE on the health of patients and populations and on health system structure and function is difficult and perhaps impossible [emphasis added]” [ 8 ].

MKC is not a causal model

Aliger and Yanak note that when Donald Kirkpatrick first proposed his model, he did not assert that each level is caused by the previous level [ 55 ]. Similarly, the developers of MKC acknowledge that “Kirkpatrick did not see outcomes in these four areas as hierarchical.” Rather, most likely in an attempt to avoid indicating causality in MKC themselves, they talk about “categories” not “levels” throughout the majority of their abovementioned paper [ 11 ]. They even knew from the outset that besides IPE the domain which we now call IPO influences outcomes on MKC levels 4a and 4b (but did not include IPO in MKC): “(…) impact of one professional’s changes in behavior depend[s] on [a] number of organisational constraints such as individual’s freedom of action (…) and support for innovation within the organisation” [ 13 ]. This means that by design neither the original Kirkpatrick model nor MKC are intended to be or to include causal models. MKC simply doesn’t ask at all whether additional causes besides an IPE intervention might be required for creating the outcomes it classifies, especially those of levels 3, 4a and 4b. In case such additional causes exist, MKC neither detects nor reflects them. Yardley and Dornan conclude that Kirkpatrick’s levels “are unsuitable for (…) education interventions (…) in which process evaluation is as important as (perhaps even more important than) outcome evaluation” [ 14 ].

Nevertheless, MKC continues to be misunderstood as implying a causal model

The numbered levels in the original Kirkpatrick model have drawn criticism for implying causality [ 14 , 54 , 55 ]. Originally, Kirkpatrick had used the term “steps” not “levels” [ 15 , 42 , 55 ] whereas all current versions of the Kirkpatrick model, including MKC, now use the term “levels”. Bates [ 52 ] cites evidence that Kirkpatrick himself, in his later publications, started to imply causal relationships between the levels of his model. Bates bluntly declares: “Kirkpatrick’s model assumes that the levels of criteria represent a causal chain such that positive reactions lead to greater learning, which produces greater transfer and subsequently more positive organizational results” [ 52 ]. Alliger and Janak [ 55 ] provide other examples from the secondary literature which explicitly assume direct causal links between the levels and continue to show that this assumption is highly problematic. Most strikingly, the current (2023) version of the Kirkpatrick model [ 51 ], created by Donald Kirkpatrick’s successors, explicitly contains a causal model which uses the exact same causal logic Alliger and Janak had proposed as underlying it almost 3 decades earlier [ 55 ].

As a derivative of the Kirkpatrick model, MKC has inherited just that unfortunate property of implying causality between levels. While starting their above-mentioned publication with the carefully chosen term “categories”, the authors of MKC, in the same publication, later fall back on using “levels” [ 11 ]. In earlier publications, they even had explicitly assigned explanatory causal power to MKC: “Level 4b: Benefits to patients/clients. This final level covers any improvements in the health and well being of patients/clients as a direct result [emphasis added] of an education programme” [ 22 ]. Taken together, the authors of MKC themselves, while acknowledging that the original Kirkpatrick model didn’t imply a causal hierarchy, at times contradictorily fuel the notion that MKC provides a viable causal model for the mechanisms of IPE and IPC. As Roland observes, it became common in the literature in general to see the levels of MKC as building on each other, implying a linear causal chain from interprofessional learning to collaborative behavior to patient outcomes [ 15 ].

Why has the wrong attribution of being a causal model to MKC remained stable for so long?

Why has this misunderstanding of MKC as a causal model not drawn more criticism and why has it been so stable? We speculate that a formal parallelism between the transitive relations in the WHO causal model (Fig.  1 ) and the numbered levels of MKC, if wrongly understood as a linear chain of subsequent causes and effects, strengthens the erroneous attribution of a causal model to MKC (Fig.  5 ). Our reasoning: The continued use of the mono-causal WHO model, as opposed to switching to a model incorporating multiple causes for patient, healthcare provider, and system outcomes, stabilizes the misunderstanding of the monothematic (IPE-constricted) MKC as a causal model. (In defense of this mistake, one could say, if the transitivity assumption associated with the WHO causal model was true, i.e. if the causal chain actually was mono-linear, then MKC would be a valid causal model because intermediate outcomes would be the sole causes of subsequent outcomes, covering the entire, linear chain of causes. As a result, there would be no difference between outcome evaluation and process evaluation, and MKC would be an appropriate tool for process evaluations.) Conversely, we suspect that the wrong but established use of MKC as a conceptual framework in IPE and IPC outcome evaluations stabilizes the continued use of the mono-causal linear WHO model, reinforcing the wrong impression that IPE is the only cause of interprofessional outcomes. The “transitivity” of the WHO model strongly resonates with the observation that the (original) Kirkpatrick model implies the assumption that “all correlations among levels are positive” [ 55 ]. If the most upstream event (an IPE activity) is positively correlated with the most downstream elements (patient, healthcare provider, and system outcomes) anyway, why should one bother evaluating intermediate steps? The same fallacy holds true for MKC. When its authors state that “Level 4b (…) covers any improvements in the health and well being of patients/clients as a direct result of an education programme” [ 22 ], they not only assign causal explanatory power to MKC, but also neglect the “multi-causality on multiple stages” of outcomes. They assume the same causal transitivity for MKC as is present in the WHO model and thereby expect an “automatic” tertiary effect from an IPE intervention on patient outcomes without considering at all whether the quality of factual IPC – as a necessary intermediate link in the causal chain – has changed due to the intervention or not.

“Unhealthful alliance” between the WHO causal model and MKC. MKC as an outcome classification does not contain a causal model, but uses the term “level” and has numbers attached to each, suggesting causal hierarchy nonetheless. The “levels” of MKC resonate with the causal chain of the WHO model. We speculate that this formal similarity stabilizes the false assignment of a causal structure to MKC (red arrows in the lower row) and, at the same time, as MKC is widely used, perpetuates the use of the WHO model

If misused as a causal model, MKC does not function and can hinder progress in IPE and IPC evaluations

So far we have established that a) Pure outcome evaluations do not answer the question why it is so hard to detect patient, healthcare provider, and system outcomes of IPE and IPC interventions; b) Process evaluations are required to address this “why” question and to achieve progress in IPE and IPC evaluations; c) A theoretical causal model is required for such process evaluations; d) MKC is not such a causal model; e) Nevertheless, MKC falsely keeps being used as such a causal model; and f) The misuse of MKC has remained rather stable, possibly due to a formal parallelism between the WHO causal model and MKC.

The multi-staged multi-causality model of patient, healthcare provider, and system outcomes now makes it clear why evaluations which implicitly or explicitly treat MKC as a causal model are bound to fail in their process evaluation part: MKC, when used as a causal model, is crucially incomplete: In terms of the causes of factual IPC (cf. Figure  4 , orange and blue boxes), MKC sees IPE but is blind to IPO; and in terms of the direct causes of patient, healthcare provider, and system outcomes (cf. Figure  4 , green and grey boxes), MKC sees factual IPC but is blind to the complementary non-interprofessional causes because none of its levels covers them. MKC is a classification limited to detecting outcomes of IPE, and neither IPO nor non-interprofessional factors are such outcomes. When speaking about the original model (but with his statement being transferable to MKC), Bates notes that “Kirkpatrick’s model implicitly assumes that examination of (…) [contextual] factors is not essential for effective evaluation” [ 52 ]. Citing Goldstein and Ford [ 56 ], he continues, “when measurement is restricted to (…) the four (…) levels no formative data about why training was or was not effective is generated” [ 52 ]. Specifically targeting the MKC version, Thistlethwaite and colleagues imply that MKC lacks IPO: “When thinking of applying of Kirkpatrick’s framework to IPE, we must remember the importance of the clinical environment (…) and consider how conducive it is to, and facilitative of, any potential change in behaviour arising from interprofessional learning activities” [ 42 ].

Bordage calls conceptual frameworks “lenses” through which scientists see the subjects of their studies [ 57 ]. Following this metaphor, we conclude that the resolution of the “conceptual lens” of MKC, if misused as a causal model, is too low for process evaluations. In our perspective, this, in turn, is the most likely reason why outcome evaluations of the past have failed to reliably demonstrate terminal benefits of IPE and IPC.

It is important to note that MKC by design solely, agnostically and successfully measures outcomes of interprofessional education in different dimensions. Therefore, its failure to detect bottlenecks in IPE and IPC is not its own fault, but the fault of evaluators who continue to use it as a causal model while failing to acknowledge the multi-staged multi-causality of patient, healthcare provider, and system outcomes.

We next take a closer look at how exactly MKC fails. In the mono-linear, low-resolution view of MKC, if a study that evaluates the effects of an intervention fails to detect final outcomes, the only logical possible conclusion is to question the effectiveness of previous levels. If there are changes in interprofessional behavior (level 3) but there is no benefit to patients (level 4b), the conclusion is that changes in interprofessional behavior are not beneficial to patients; if there are interprofessional competencies acquired by learners (level 2) but no subsequent change in interprofessional behavior (level 3), then interprofessional competencies do not translate into behavior. Using MKC as the conceptual lens, the logical answer to “why” is that “the training program was not designed in ways that fostered effective transfer or (…) other input factors blocked skill application” [ 52 ], and a straightforward overall conclusion with regards to the knowledge gap about the benefits of IPE and IPC would be that IPE is not very effective in terms of patient, healthcare provider, and system outcomes. While this disappointing result has actually been considered as a possibility [ 6 , 9 , 10 ], more often alternative explanations are sought in an attempt to rescue IPE efforts and to avoid the conclusion that IPE is ineffective while sticking with MKC as the causal model.

One of these “escape routes” is to claim that it is methodologically too difficult to measure outcomes on MKC levels 3, 4a and 4b by using different variants of a temporal argument. Paraphrasing Belfield et al. [ 58 ], Roland [ 15 ] states that “patient outcomes may only become apparent over a protracted period of time due to the time needed for the learner to acquire and implement new skills [emphasis added by us, also in the following quotations]” whereas Hammick and colleagues state, “It is unsurprising that all but one of the studies (…) evaluated IPE for undergraduate students. The time gap between their interprofessional learning and qualification clearly presents a challenges [sic] associated with evaluating levels 3, 4a and 4b outcomes” [ 11 ]. Yardley and Dornan add, “early workplace experience (…) might take months or even years to have any demonstrable effect on learners, let alone patients” [ 14 ]. The IOM comments that “Efforts to generate this evidence are further hindered by the relatively long lag time between education interventions and patient, population, and system outcomes” [ 8 ] while Reeves and colleagues note that “ the time gap between undergraduates receiving their IPE and them qualifying as practitioners presents challenges with reporting outcomes at Levels 3, 4a, and 4b” [ 2 ]. The core argument here is always that undergraduate IPE happens in educational institutions whereas IPC happens at the workplace at healthcare institutions much later . By this logic, the causal chain assumed by MKC might be fully intact but the time lag between an IPE intervention and effects on levels 3, 4a and 4b constitutes an insurmountable methodological difficulty and renders comprehensive evaluations of IPE outcomes impossible.

Another “escape route” is to invoke “complexity” of IPE as the reason why its final outcomes are hard to detect. Thistlethwaite and colleagues [ 42 ] agree with Yardley and Dornan [ 14 ] that the MKC is not suited to evaluate “the complexity of health profession education and practice.” The authors from the IOM state that “The lack of a well-defined relationship between IPE and patient and population health and health care delivery system outcomes is due in part to the complexity of the learning and practice environments” [ 8 ]. The term “complexity” usually refers to systems which are cognitively difficult to understand because they have many elements or because science has not figured out yet how to model their interactions [ 59 ]. In our opinion, the term “complexity” in the context of IPE is ill-defined and a placeholder for saying that the set of causes of patient, healthcare provider, and system outcomes is not being understood well and that a more precise causal model is required to figure out what is going on.

Compare and contrast: “multi-stage multi-causality” as causal model

If we use “multi-stage multi-causality” as the conceptual lens instead of MKC we increase the available resolution and can see more elements of the system. If evaluations fail to show beneficial outcomes of IPE or IPC, we now can do much better asking the right sub-questions to find an answer to “why”. Viewed through the high-resolution lens of the multi-stage multi-causality model, the list of possible failure points on this trajectory significantly expands. The resulting high-resolution picture provides an exquisite set of novel testable hypotheses (Table  1 ). Collecting data on different levels, including the level of factual IPC, should enable decisions as to which of these scenarios were attributable to an IPE intervention having no multi-level effect.

Taken together, we argue that the answers to “why” allowed by the low resolution lens of MKC when misused as a causal model might sometimes be wrong and should be replaced with more detailed explanations.

It is premature to conclude that IPE has no effects on patient, healthcare provider, and system outcomes unless the presence or absence of all co-causes has been considered.

The deeper cause of the temporal argument might be to mistakenly use MKC as a causal model because the use of MKC masks any problems with IPO or other co-causes. Given the higher resolution of the multi-stage multi-causality model, it is now possible to conceptually distinguish between the known challenge arising from the passage of time (creating various confounders) and the case in which a lack of IPO blocks the effects of IPE. It should be possible, in principle, to assess at any later point in time, for example by means of a survey, how much and which types of IPE members of an interprofessional team had experienced earlier in their career and how much they remember; or even to assess their current competencies for IPC in a practical exam. Such measurements might reveal that individual competencies for IPC are present, no matter how much time has passed since their acquisition, and that IPO is the actual bottleneck.

Likewise, alleged methodological perplexity due to IPE “complexity” is de-emphasized if we swap the low-resolution lens of MKC for the high-resolution lens of the multi-staged multi-causality model. The high-resolution picture (Fig.  4 ; Table  1 ) replaces the fuzzy placeholder of “complexity” by adding missing elements of the system to the model.

In sum we have demonstrated that when MKC is misused as a causal model it neglects co-causing factors with essential influence on IPE outcomes, is therefore an insufficient tool to detect bottlenecks, and edges out any better-suited, viable causal model. This miscast hampers meaningful process evaluations, the subsequent improvement of indicators and interventions, and thereby ultimately the progress in proving beneficial patient, healthcare provider, and system outcomes of IPE and IPC.

Limitations

One limitation of our theoretical critical discussion is that we did not illuminate how hard it is to quantify patient, healthcare provider, and system outcomes from a methodological point of view (e.g. document-based patient data analysis). Neither did we address the extent to which this limits the meaningfulness of IPE/IPC outcome evaluations. However, we claim that the conceptual weakness of missing co-causalities is the deeper root of the evaluation problem, not particular methods, and that methodological issues are solvable as soon as relevant co-causalities are appropriately considered.

Another limitation is that a model is always a simplification. For example, the multi-stage multi-causality model does not include personality traits of team members, intra-personal abilities like self-regulation, or the harmony of personality types within a team, which also play a role in factual IPC. These traits would be difficult to incorporate into the model and gathering such information for evaluations might even be unethical. Similarly, the model does not reflect the influence which the behavior and health literacy of patients (and their families, caregivers, and communities) might have on factual IPC.

A third limitation is that we did not discuss a particular setting in which the use of MKC as mono-linear causal model could work, namely, if IPE champions themselves become IPO managers and subsequently establish factual IPC in their institutions through an appropriate combination of IPE and IPO. In this scenario, the roles of health professionals (as carriers of IPE-induced competence for factual IPC) and managers (as IPO decision makers) overlap – obviously potentially optimal to foster factual IPC. In a certain sense, in this particular case, IPE would lead to IPO and to factual IPC with the potential of “transitively” improving patient, healthcare provider, and system outcomes. However, as we believe that there is no fixed relationship between undergoing IPE and becoming a healthcare manager, we did not pursue this line of argument further, regarding it as an exception.

Conclusions

In our critical discussion we have analyzed previous models of causes and effects in IPC based on the existing literature, proposed a novel “multi-stage multi-causality” model, and demonstrated its explanatory power by establishing that MKC is not suited to foster progress in proving or disproving beneficial final outcomes of IPE and IPC. We conclude with 6 practical, applicable recommendations for future IPE, IPO, and IPC outcome evaluations.

Recommendation 1: stop (mis-)using MKC as a causal model

We have pointed out that the continued use of MKC as causal model seems to severely inhibit the scientific exploration of the co-necessity of IPO and non-interprofessional factors and therefore delays answering the important question whether IPE and IPC actually improve patient, healthcare provider, and system outcomes. As early as 1989, the use of the original Kirkpatrick model as a causal model was questioned [ 55 ]. In 2004, Bates took the position that the continued use of this model is unethical if beneficial results are missed by evaluations due to the narrow focus on outcomes [ 52 ]. Today, we conclude that using MKC as a causal model in IPE, IPO or IPC outcome evaluations should be discontinued.

Recommendation 2: state the causal model under which evaluations of IPE/IPO/IPC operate

Evaluators should make an explicit statement about the causal model under which they design interventions and interpret results, including their additional assumptions about the chain of causes and effects. Knowledge of these assumptions allows the reader to detect inconsistencies – an important element for causal clarification – and should prevent the field of IPE, IPO, and IPC outcome evaluations from getting mired down for even more decades.

Recommendation 3: always include some process evaluation

Even if the primary goal of a study is summative outcome evaluation, evaluators should always include some process evaluation to test the causal model they assume and under which they designed their evaluation, and do so at least until the topic of causality in IPE, IPO, and IPC is fully settled. For example, if an IPE intervention aims at improving factual IPC, evaluators who assume multi-causality would co-evaluate IPO to make sure that IPO is no bottleneck in the evaluated setting.

Recommendation 4: strive for specificity in IPE, IPO, or IPC interventions

If the only goal of an intervention is to improve a certain outcome metric like patient safety, one might initiate a broad, non-specific intervention using best-practice guidelines and all available resources. However, if a goal of the intervention is also to show the existence of specific benefits of IPE, IPO, or IPC in a scientific way, then the multi-causality of outcomes must be taken into account. Intervention designs that change both, interprofessional and non-interprofessional causes of outcomes, must be avoided. For example, if uniprofessional training (a cause outside the domain of IPE/IPO/IPC) is also part of an intervention (e.g. the re-design of the entire workflow in an emergency department in order to enhance patient safety), then this mix of causes obscures the contribution of IPE, IPO, or IPC to the desired effect. Reeves and colleagues euphemistically and aptly call measuring the particular influence of IPE on patient outcomes in such multifaceted interventions a “challenge” [ 10 ]. This example shows why theoretical clarity about the causal model is required to effectively evaluate beneficial outcomes of IPE, IPO, or IPC. Respecting the multi-stage multi-causality of patient, healthcare provider, and system outcomes means designing interventions that improve interprofessional elements only, or, if other components inevitably change as well, to control for those components through comprehensive measurements and/or by adding qualitative methods that allow final outcomes to be causally attributed to IPE, IPO, or IPC.

Recommendation 5: always quantify factual IPC

Recommendations 5 and 6 are our most important recommendations. It is self-explanatory that without the emergence of factual IPC there cannot be any final, globally desirable outcomes of upstream IPE or IPO activities; not until IPE or IPO activities improve factual IPC, does the attempt to evaluate their effects on patient, healthcare provider, and system outcomes start to make any sense. Further, if a positive correlation exists between the quality of factual IPC and patient, healthcare provider, and system outcomes, then correlating factual IPC with final outcomes is the most conclusive way to show it. While the notion that factual IPC is the minimum necessary condition for final outcomes of interprofessional efforts is not new [ 8 , 19 ], the realization that the attached transitivity assumption (that IPE automatically creates the necessary IPC) is wrong, certainly is. As shown above, dismissing transitivity is a cogent consequence of embracing the multi-stage multi-causality of final outcomes. In future evaluations, the quantification of IPE therefore should no longer serve as a surrogate for the quantification of factual IPC. Rather, factual IPC, as an intermediate necessary step towards final outcomes and their most direct cause within the realm of IPE/IPO/IPC, always needs to be evaluated on its own. The same holds true for future evaluations of IPO. IPO interventions do not automatically lead to factual IPC, but first must be shown to improve factual IPC before they can be expected to cause any changes in patient, healthcare provider, and system outcomes. Taken together, a comprehensive measurement of the quality of factual IPC needs to be the centerpiece of any meaningful evaluation of final outcomes achieved by IPE interventions, IPO interventions, combined IPE + IPO interventions, or of factual IPC itself.

From the large number of dimensions of factual IPC (see “Methods”) arises the necessity to evaluate it in detail. Such completeness in the evaluation of factual IPC is important for several reasons:

Obtaining a meaningful sum score: Evaluating factual IPC in a given setting against a hypothetical optimum requires integration of all of its subdimensions into one sum score.

Not missing correlations: If an IPC score does not cover all dimensions of factual IPC, correlations between factual IPC and its effects (or causes) might be missed, even if these relationships truly exist. Example: An evaluation which only includes the dimensions of “mutual respect” and “conflict management” might miss an actually existing correlation between factual IPC and cost effectiveness, mainly driven, say by the dimension of “shared creation of the treatment plan and coordination of its execution”. The result of this evaluation could cast substantial doubt on the existence of positive effects of factual IPC despite them actually being there. Similarly, only a complete set of IPC indicators is suited to reveal potentially diverging effects of different subdimensions of factual IPC on different final outcomes. For example, optimal interprofessional team behavior that maximizes patient safety, might, at the same time, turn out to be less cost effective than multiprofessional team behavior that compromises on patient safety.

Optimizing process evaluation: A complete IPC coverage further provides valuable information for process evaluations aimed at identifying weaknesses in factual IPC. Significant correlations between outcomes and specific subdimensions of IPC can suggest causal relationships and uncover crucial components for successful IPC in a given setting. Focusing on strengthening these subdimensions could help optimize patient, healthcare provider, and system outcomes.

Enabling setting independence and comparisons: Factual IPC is setting-specific [ 11 , 19 , 35 , 60 , 61 ], i.e. the needs of patients for specific medical services differ across different contexts of patient care (e.g. emergency care; acute care; rehabilitation; chronic care; multimorbid patients; palliative care). As a consequence, different subdimensions of factual IPC contribute to the outcomes of interest to a variable degree depending on the specific healthcare setting. Even within a specific setting, requirements and behaviors necessary for effective IPC can vary due to the specifics of the case, e.g. the particular rareness or severity of the patient’s condition. Assumptions made prior to an evaluation about which subdimensions of factual IPC are most important in a specific setting therefore should not preclude the exploratory evaluation of the other subdimensions. If an evaluation grid misses IPC subdimensions, it may work well in one setting but fail in others. Hence, the completeness of indicators for factual IPC in an evaluation instrument creates setting independence, eliminates the burden of adjusting the included IPC subdimensions every time a new healthcare setting is evaluated, and allows unchanged evaluation instruments to be re-used in subsequent studies (called for by e.g. [ 16 ]) as well as multi-center studies (called for by e.g. [ 2 ]). A starting point for the operationalization of factual IPC including all of its subdimensions is provided in our definition of factual IPC (see “Methods”; a validated evaluation toolbox based on this operationalization will be published elsewhere; a published tool which also covers all subdomains of factual IPC, with a focus on adaptive leadership, is the AITCS [ 39 ]).

Recommendation 6: use a step-by-step approach for proving benefits of IPE and IPO

The multi-stage multi-causality of patient, healthcare provider, and system outcomes naturally implies that the process of proving that IPE or IPO benefits final outcomes could be broken down into discrete steps. The key idea is to evaluate the impact of interprofessional activities on each of the subsequent levels in the causal chain while controlling for non-interprofessional factors. Showing the effects of IPE on IPC competencies, the effects of IPC competencies on factual IPC, and the effects of factual IPC on patient, healthcare provider, and system outcomes then becomes three different research agendas that can be processed independently. If it can be shown in the first of these research agendas that IPE leads to learning (by controlling for non-interprofessional learning-related factors), and in the second, independent research agenda, that learning leads to improved factual IPC (controlling for IPO), and in the third research agenda that factual IPC leads to desired final outcomes (controlling for co-conditions for final outcomes like uniprofessional competencies), then the benefit of IPE on patient, healthcare provider, and system outcomes is ultimately proved. If this approach fails, then at least it will be exactly revealed where the chain of effects breaks down. The same holds true for IPO: Show that IPO interventions lead to work processes and/or favorable institutional conditions which support factual IPC, separately show that these work processes and conditions lead to improved factual IPC (if co-conditions for factual IPC like IPE are present), and show that better factual IPC leads to an improvement of final outcomes; then the positive impact of IPO is verified.

By covering the entire process, this “step-by-step” approach could build a compelling case for how interprofessional interventions lead to desired final outcomes. It further could markedly simplify the agenda of interprofessional research because it takes the burden of showing the effect of one particular IPE or IPO intervention on one particular final outcome off the shoulders of evaluators. After breaking down the evaluation task into separate steps that prove the impact from link to link, researchers are free to work on one step at a time only.

The presented critical discussion advances the theoretical foundations of evaluations in the field of IPE, IPO and IPC. To improve patient-centered care by means of IPC, one needs to think bigger than just training of healthcare professionals in the competencies and mindsets required for effective IPC; work processes also have to be established and optimized in a setting-dependent manner to allow for factual IPC to happen. Besides IPC, factors like discipline-specific knowledge of health professionals or administrative aspects of patient management have to be optimized, too, to achieve optimal patient, healthcare provider, and system outcomes.

By sharing the multi-stage multi-causality model and its pertinent theoretical clarification we hope to contribute to a deeper understanding of causes and effects in interprofessional collaboration, to answer the repeated call in the research community for improved theory in this field, to explain difficulties faced by past evaluations, and to provide helpful guidance for future research studies. Our key recommendations for future evaluations of interprofessional outcomes are to focus on a comprehensive evaluation of factual IPC as the most fundamental metric and to deploy a step-by-step research agenda with the overarching goal of proving beneficial patient, healthcare provider, and system outcomes related to IPE, IPO, and IPC. With these contributions, we hope to help healthcare institutions improve their evaluations of IPE, IPO, and IPC, ultimately benefiting health, healthcare provider, and system outcomes.

Availability of data and materials

No datasets were generated or analysed during the current study.

Abbreviations

• Interprofessional collaboration
• Interprofessional education
• Interprofessional organization
• Modified Kirkpatrick classification

World Health Organization

Körner M, Bütof S, Müller C, Zimmermann L, Becker S, Bengel J. Interprofessional teamwork and team interventions in chronic care: A systematic review. J Interprof Care. 2016;30(1):15–28.

Article   Google Scholar  

Reeves S, Fletcher S, Barr H, Birch I, Boet S, Davies N, et al. A BEME systematic review of the effects of interprofessional education: BEME Guide No. 39. Med Teach. 2016;38(7):656–68.

Reeves S, Pelone F, Harrison R, Goldman J, Zwarenstein M. Interprofessional collaboration to improve professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2017. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD000072.pub3/full . Accessed 23 Feb 2024.

Sibbald B, Bojke C, Gravelle H. National survey of job satisfaction and retirement intentions among general practitioners in England. BMJ. 2003;326(7379):22.

Cowin LS, Johnson M, Craven RG, Marsh HW. Causal modeling of self-concept, job satisfaction, and retention of nurses. Int J Nurs Stud. 2008;45(10):1449–59.

Paradis E, Whitehead CR. Beyond the lamppost: a proposal for a fourth wave of education for collaboration. Acad Med. 2018;93(10):1457.

Holly C, Salmond S, Saimbert M. Comprehensive Systematic Review for Advanced Practice Nursing. 2nd ed. New York: Springer Publishing Company; 2016.

Google Scholar  

IOM (Institute of Medicine). Measuring the impact of interprofessional education on collaborative practice and patient outcomes. Washington, DC: The National Academies Press; 2015.

Brandt B, Lutfiyya MN, King JA, Chioreso C. A scoping review of interprofessional collaborative practice and education using the lens of the Triple Aim. J Interprof Care. 2014;28(5):393–9.

Reeves S, Perrier L, Goldman J, Freeth D, Zwarenstein M. Interprofessional education: effects on professional practice and healthcare outcomes (Review) (Update). Cochrane Database Syst Rev. 2013. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD002213.pub3/full . Accessed 23 Feb 2024.

Hammick M, Freeth D, Koppel I, Reeves S, Barr H. A best evidence systematic review of interprofessional education: BEME Guide no. 9. Med teach. 2007;29(8):735–51.

Zwarenstein M, Reeves S, Perrier L. Effectiveness of pre-licensure interprofessional education and post-licensure collaborative interventions. J Interprof Care. 2005;19(Suppl 1):148–65.

Barr H, Hammick M, Koppel I, Reeves S. Evaluating interprofessional education: two systematic reviews for health and social care. Br Edu Res J. 1999;25(4):533–44.

Yardley S, Dornan T. Kirkpatrick’s levels and education ‘evidence’. Med Educ. 2012;46(1):97–106.

Roland D. Proposal of a linear rather than hierarchical evaluation of educational initiatives: the 7Is framework. J Educ Eval Health Profess. 2015;12:35.

Thannhauser J, Russell-Mayhew S, Scott C. Measures of interprofessional education and collaboration. J Interprof Care. 2010;24(4):336–49.

Reeves S, Goldman J, Gilbert J, Tepper J, Silver I, Suter E, et al. A scoping review to improve conceptual clarity of interprofessional interventions. J Interprof Care. 2011;25(3):167–74.

Suter E, Goldman J, Martimianakis T, Chatalalsingh C, DeMatteo DJ, Reeves S. The use of systems and organizational theories in the interprofessional field: Findings from a scoping review. J Interprof Care. 2013;27(1):57–64.

Havyer RD, Wingo MT, Comfere NI, Nelson DR, Halvorsen AJ, McDonald FS, et al. Teamwork assessment in internal medicine: a systematic review of validity evidence and outcomes. J Gen Intern Med. 2014;29(6):894–910.

Reeves S, Boet S, Zierler B, Kitto S. Interprofessional education and practice guide No. 3: Evaluating interprofessional education. J Interprofess Care. 2015;29(4):305–12.

McNaughton SM, Flood B, Morgan CJ, Saravanakumar P. Existing models of interprofessional collaborative practice in primary healthcare: a scoping review. J Interprof Care. 2021;35(6):940–52.

Barr H, Freeth D, Hammick M, Koppel I, Reeves S. Evaluations of Interprofessional Education: A United Kingdom Review for Health and Social Care. Centre for the Advancement of Interprofessional Education and The British Educational Research Association. 2000. https://www.caipe.org/resources/publications/barr-h-freethd-hammick-m-koppel-i-reeves-s-2000-evaluations-of-interprofessional-education . Accessed 23 Feb 2024.

Barr H, Koppel I, Reeves S, Hammick M, Freeth D. Effective Interprofessional Education: Argument, assumption, and evidence. Wiley-Blackwell; 2005.

Baethge C, Goldbeck-Wood S, Mertens S. SANRA—a scale for the quality assessment of narrative review articles. Research integrity and peer review. 2019;4(1):1–7.

CAIPE (Centre for the Advancement of Interprofessional Education). About CAIPE, s. v. “Defining Interprofessional Education”. https://www.caipe.org/about . Accessed 23 Feb 2024.

D’amour D, Oandasan I. Interprofessionality as the field of interprofessional practice and interprofessional education: An emerging concept. J Interprof Care. 2005;19(Suppl 1):8–20.

Körner M, Wirtz MA. Development and psychometric properties of a scale for measuring internal participation from a patient and health care professional perspective. BMC Health Serv Res. 2013;13(1):374.

Körner M, Wirtz MA, Bengel J, Göritz AS. Relationship of organizational culture, teamwork and job satisfaction in interprofessional teams. BMC Health Serv Res. 2015;15(1):243.

World Health Organization. Framework for Action on Interprofessional Education & Collaborative Practice. 2010. https://www.who.int/publications/i/item/framework-for-action-on-interprofessional-education-collaborative-practice . Accessed 23 Feb 2024.

Wei H, Horns P, Sears SF, Huang K, Smith CM, Wei TL. A systematic meta-review of systematic reviews about interprofessional collaboration: facilitators, barriers, and outcomes. J Interprof Care. 2022;36(5):735–49.

Morey JC, Simon R, Jay GD, Wears RL, Salisbury M, Dukes KA, et al. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res. 2002;37(6):1553–81.

D’Amour D, Ferrada-Videla M, San Martin Rodriguez L, Beaulieu M-D. The conceptual basis for interprofessional collaboration: core concepts and theoretical frameworks. J interprofess care. 2005;19(1):116–31.

San Martín-Rodríguez L, Beaulieu M-D, D’Amour D, Ferrada-Videla M. The determinants of successful collaboration: a review of theoretical and empirical studies. J Interprof Care. 2005;19(Suppl 1):132–47.

Guise J-M, Deering SH, Kanki BG, Osterweil P, Li H, Mori M, et al. Validation of a tool to measure and promote clinical teamwork. Simulation in Healthcare. 2008;3(4):217–23.

Orchard C, Bainbridge L, Bassendowski S.. A national interprofessional competency framework. Vancouver: Canadian Interprofessional Health Collaborative; University of British Columbia; 2010; Available online: http://ipcontherun.ca/wp-content/uploads/2014/06/National-Framework.pdf . Accessed 29 May 2024.

Oishi A, Murtagh FE. The challenges of uncertainty and interprofessional collaboration in palliative care for non-cancer patients in the community: a systematic review of views from patients, carers and health-care professionals. Palliat Med. 2014;28(9):1081–98.

Valentine MA, Nembhard IM, Edmondson AC. Measuring teamwork in health care settings: a review of survey instruments. Med Care. 2015;53(4):e16–30.

Lie DA, Richter-Lagha R, Forest CP, Walsh A, Lohenry K. When less is more: validating a brief scale to rate interprofessional team competencies. Med Educ Online. 2017;22(1):1314751.

Orchard C, Pederson LL, Read E, Mahler C, Laschinger H. Assessment of interprofessional team collaboration scale (AITCS): further testing and instrument revision. J Contin Educ Heal Prof. 2018;38(1):11–8.

Begun JW, White KR, Mosser G. Interprofessional care teams: the role of the healthcare administrator. J Interprof Care. 2011;25(2):119–23.

Weaver SJ, Salas E, King HB. Twelve best practices for team training evaluation in health care. The Joint Commission Journal on Quality and Patient Safety. 2011;37(8):341–9.

Thistlethwaite J, Kumar K, Moran M, Saunders R, Carr S. An exploratory review of pre-qualification interprofessional education evaluations. J Interprof Care. 2015;29(4):292–7.

Gilbert JH, Yan J, Hoffman SJ. A WHO report: framework for action on interprofessional education and collaborative practice. J Allied Health. 2010;39(3):196–7.

Taylor & Francis Online. http://www.tandfonline.com (2024). Accessed 23 Feb 2024.

Sockalingam S, Tan A, Hawa R, Pollex H, Abbey S, Hodges BD. Interprofessional education for delirium care: a systematic review. J Interprof Care. 2014;28(4):345–51.

Folland S, Goodman AC, Stano M. The Economics of Health and Health Care. 7th ed. New York, NY: Routledge; 2016.

Book   Google Scholar  

Hayes B, Bonner A, Pryor J. Factors contributing to nurse job satisfaction in the acute hospital setting: a review of recent literature. J Nurs Manag. 2010;18(7):804–14.

Curran V, Reid A, Reis P, Doucet S, Price S, Alcock L, et al. The use of information and communications technologies in the delivery of interprofessional education: A review of evaluation outcome levels. J Interprof Care. 2015;29(6):541–50.

Danielson J, Willgerodt M. Building a theoretically grounded curricular framework for successful interprofessional education. A J Pharmaceut Educ. 2018;82(10):1133–9.

Kirkpatrick D, Kirkpatrick J. Evaluating Training Programs: The Four Levels. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers; 2006.

Kirkpatrick J, Kirkpatrick W. An Introduction to The New World Kirkpatrick Model. Kirkpatrick Partners. 2021. http://www.kirkpatrickpartners.com/wp-content/uploads/2021/11/Introduction-to-the-Kirkpatrick-New-World-Model.pdf . Accessed 30 Nov 2023.

Bates R. A critical analysis of evaluation practice: the Kirkpatrick model and the principle of beneficence. Eval Program Plann. 2004;27(3):341–7.

Pawson R, Tilley N. Realistic evaluation. 1st ed. London: Sage Publications Ltd; 1997.

Holton EF III. The flawed four-level evaluation model. Hum Resour Dev Q. 1996;7(1):5–21.

Alliger GM, Janak EA. Kirkpatrick’s levels of training criteria: Thirty years later. Pers Psychol. 1989;42(2):331–42.

Goldstein IL, Ford JK. Training in organisations: Needs assessment, development, and evaluation. 4th ed. Belmont, CA: Wadsworth; 2002.

Bordage G. Conceptual frameworks...: What lenses can they provide to medical education? Investigación en educación médica. 2012;1(4):167–9.

Belfield C, Thomas H, Bullock A, Eynon R, Wall D. Measuring effectiveness for best evidence medical education: a discussion. Med Teach. 2001;23(2):164–70.

Johnson N. Simply complexity: A clear guide to complexity theory. London: Oneworld Publications; 2009.

Retchin SM. A conceptual framework for interprofessional and co-managed care. Acad Med. 2008;83(10):929–33.

Schmitz C, Atzeni G, Berchtold P. Challenges in interprofessionalism in Swiss health care: the practice of successful interprofessional collaboration as experienced by professionals. Swiss Med Wkly. 2017;147: w14525.

Download references

Acknowledgements

Not applicable.

The Swiss Federal Office of Public Health partly funded this study with the contractual mandate “Bildung und Berufsausübung: Evaluationsinstrumente”.

Author information

Authors and affiliations.

Institute for Medical Education, Department for Assessment and Evaluation, University of Bern, Bern, Switzerland

Florian B. Neubauer, Felicitas L. Wagner, Andrea Lörwald & Sören Huwendiek

You can also search for this author in PubMed   Google Scholar

Contributions

FBN, FLW, AL and SH made substantial contributions to the conception of the work and to the literature searches. FBN wrote the first full draft of the manuscript. All authors improved the draft. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Florian B. Neubauer .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Neubauer, F.B., Wagner, F.L., Lörwald, A. et al. Sharpening the lens to evaluate interprofessional education and interprofessional collaboration by improving the conceptual framework: a critical discussion. BMC Med Educ 24 , 615 (2024). https://doi.org/10.1186/s12909-024-05590-0

Download citation

Received : 29 November 2023

Accepted : 22 May 2024

Published : 04 June 2024

DOI : https://doi.org/10.1186/s12909-024-05590-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Outcome evaluation
• Process evaluation
• Causal model
• Conceptual framework
• Terminology

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Organizing Academic Research Papers: 5. The Literature Review

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

A literature review surveys scholarly articles, books and other sources relevant to a particular issue, area of research, or theory, and by so doing, providing a description, summary, and critical evaluation of these works. Literature reviews are designed to provide an overview of sources you have explored while researching a particular topic and to demonstrate to your readers how your research fits into the larger field of study.

Importance of a Good Literature Review

A literature review may consist of simple a summary of key sources, but it usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

The purpose of a literature review is to:

• Place each work in the context of its contribution to the understanding of the research problem being studied,
• Describe the relationship of each work to the others under consideration,
• Identify new ways to interpret, and shed light on any gaps in previous research,
• Resolve conflicts amongst seemingly contradictory previous studies,
• Identify areas of prior scholarship to prevent duplication of effort,
• Point the way in fulfilling a need for additional research, and
• Locate your own research within the context of existing literature.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination. Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques. Los Angeles, CA: SAGE, 2011; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students. 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers.* First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination. Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following :

• An overview of the subject, issue or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories (e.g. works that support of a particular position, those against, and those offering alternative approaches entirely),
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence (e.g. primary historical material, case studies, narratives, statistics, recent scientific findings)?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most/least convincing?
• Value -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Stages

• Problem formulation -- which topic or field is being examined and what are its component issues?
• Literature search -- finding materials relevant to the subject being explored.
• Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic.
• Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not very specific about what form your literature review should take, seek clarification from your professor by asking these questions:

• Roughly how many sources should I include?
• What types of sources should I review (books, journal articles, websites)?
• Should I summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should I evaluate the sources?
• Should I provide subheadings and other background information, such as definitions and/or a history?

Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature reviews. Read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. The bibliography or reference section of sources you've already read are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make your job easier if you first limit scope of the research problem. A good strategy is to begin by searching the HOMER catalog for books about the topic and review their contents for chapters that focus on more specific issues. You can also review the subject indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is very common in the sciences where research conducted only two years ago could be obsolete. However, when writing a review in the social sciences, a survey of the history of the literature may be what is needed because what is important is how perspectives have changed over the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is consider by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronological of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic (“conceptual categories”) Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it will still be organized chronologically reflecting technological developments in media. The only difference here between a "chronological" and a "thematic" approach is what is emphasized the most: the role of the Internet in presidential politics. Note however that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you but include only what is necessary for the reader to locate your study within the larger scholarship framework.

Here are examples of other sections you may need to include depending on the type of review you write:

• Current Situation : information necessary to understand the topic or focus of the literature review.
• History : the chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : the criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed articles and journals.
• Standards : the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Use Quotes Sparingly Some short quotes are okay if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Sometimes you may need to quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute your own summary and interpretation of the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to their own work. Keep Your Own Voice While the literature review presents others' ideas, your voice (the writer's) should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevent sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in the literature review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination. Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques. Los Angeles, CA: London : SAGE, 2011; Literature Review Handout . Online Writing Center. Liberty University; Literature Reviews .  The Writing Center. University of North Carolina; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students. 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation . vol. 14, June 2009; Taylor, Dena. The Literature Review: A Few Tips On Conducting It . University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.  

Writing Tip

Break Out of Your Disciplinary Box!

Thinking in an interdisciplinary way about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support of their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was constructed because it lays a foundation for  developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've adequately reviewed the literature:

• Look for repeating patterns in the research findings. If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work. If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been introduced in addressing the research question.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. If the same authors are being cited again and again, this may indicate no new literature has been written on the topic.
• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

• Skip to main content
• Keyboard shortcuts for audio player

Weekend Edition Sunday

• Latest Show

Sunday Puzzle

• Corrections

Listen to the lead story from this episode.

Politics chat: How voters are responding to Trump's felony conviction

by  Ayesha Rascoe ,  Mara Liasson

The Americas

Mexico votes for a new president after a campaigning season plagued by violence.

by  Eyder Peralta ,  Ayesha Rascoe

Middle East

Aid workers in gaza say nowhere is safe after israeli attacks on 'humanitarian zones'.

by  Hadeel Al-Shalchi

Girls in the U.S. are getting their period earlier. Here's what parents should know

by  Ayesha Rascoe ,  Maria Godoy

Bookstores have come under attack in Ukraine. But interest in reading is only growing

by  Joanna Kakissis

25 years ago, Napster changed how we listen to music forever

by  Ayesha Rascoe

What locals think of the proposal to build U.S.'s tallest building in Oklahoma City

by  Graycen Wheeler

Sunday Puzzle NPR hide caption

Sunday Puzzle: Second in Line

by  Will Shortz

Movie Interviews

A new animated film follows a lonely dog and his robot friend in new york city.

by  Ayesha Rascoe ,  Matthew Schuerman ,  Andrew Craig

Conservative media sows doubt about the verdict in Trump's felony convictions

by  Ayesha Rascoe ,  David Folkenflik

Supreme Court judge accused of bias towards Trump declines to recuse himself from case

by  Ayesha Rascoe ,  Matthew Schuerman ,  Hiba Ahmad

Some states are adopting a new form of reading instruction to combat falling scores

by  Juma Sei

A new movie tells the story of Kemba Smith Pradia, race and incarceration

Strange news, meet abby lampe, two-time champion of the cheese-wheel-chasing race, meet abby lampe, two-time champion of the chees-wheel-chasing race, 100 years ago, indigenous people were granted u.s. citizenship by law.

by  Sandhya Dirks

The first professional women's hockey league in the U.S. has a winner

Music interviews, jon lampley, a veteran of stephen colbert's talk show, releases his debut album.

by  D. Parvaz ,  Ayesha Rascoe ,  Ryan Benk

Searching for a song you heard between stories? We've retired music buttons on these pages. Learn more here.