system model research paper

Last updated 27/06/24: Online ordering is currently unavailable due to technical issues. We apologise for any delays responding to customers while we resolve this. For further updates please visit our website: https://www.cambridge.org/news-and-insights/technical-incident

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings .

Login Alert

• > Journals
• > Design Science
• > Volume 10
• > Utilization of system models in model-based systems...

Article contents

• Introduction
• Literature review
• System model utilization
• Limitations
• Discussion and conclusions

Utilization of system models in model-based systems engineering: definition, classes and research directions based on a systematic literature review

Published online by Cambridge University Press:  14 February 2024

• Supplementary materials

The use of system models within model-based systems engineering (MBSE) is essential for improved communication or system documentation. Previous publications have investigated further reuse of these system models, for example, transforming them directly into discipline-specific models for reuse. The authors refer to this as the term “Utilization” of system models. It aims the compensation of modelling efforts and a further integration of linked models within MBSE. Motivated by a lack of common understanding of this term, a systematic literature review of the state of the art is presented. With this systematic overview, a definition and classification system for different use cases and system life cycle stages are created. These are key results to support engineers and researchers in adopting existing or discovering new utilization approaches. This supports the mission of advanced systems engineering and aims the identification of new research directions coming along with SysML v2 and the advanced systems engineering methods.

1. Introduction

Descriptive system models have various benefits for the development of technical as well as non-technical systems. In the beginning, these models, for example, with representing dynamics (Forrester Reference Forrester 2013 ), focused on grasping the system’s complexity and existing interdependencies between its own elements and with other systems. Hick, Bajzek & Faustmann ( Reference Hick, Bajzek and Faustmann 2019 ) describe the system model as an allocation of central hubs, which are used at different phases in a project (horizontal) at different levels in the system hierarchy (vertical) for the horizontal and vertical distribution of information. With that goal, the system modelling language (SysML) was created. This enables system developers to integrate multiple disciplines and domains, as well as different views and perspectives on a system, into one model (Friedenthal, Moore & Steiner Reference Friedenthal, Moore and Steiner 2015 ; Albers et al. Reference Albers, Bursac, Scherer, Birk, Powelske and Muschik 2019 ). The main purpose of these models is defined by their use and the improvement of interdisciplinary communication (Weilkiens Reference Weilkiens 2019 ). This contrasts with the usually domain-specific models, such as computer-aided design (CAD) models for mechanical engineers and allows to manage system complexity, which is a main area of interest within advanced systems engineering (ASE) (Dumitrescu et al. Reference Dumitrescu, Albers, Riedel, Stark and Gausemeier 2021 ). Fuelled by the increasing complexity of technical products, for example, advanced systems like a Digital Twin and accompanying new business models, this complexity management is a crucial aspect to ensure holistic approaches for the system development of future products on future markets (Albers et al. Reference Albers, Dumitrescu, Gausemeier, Lindow, Riedel and Stark 2022 ). It requires the usage of advanced engineering methods to handle this complexity and support the system development (Albers et al. Reference Albers, Dumitrescu, Gausemeier, Riedel and Stark 2018 ). This interaction of the three fields advanced systems, systems engineering and advanced engineering defines the field of ASE which creates a frame of actions and a holistic thinking and application of methods.

Within model-based systems engineering (MBSE), the use of system models is an essential aspect to clarify the whole system context and identify possible interdependencies between subsystems and components or with the system environment. Besides the improvement of interdisciplinary communication throughout the system development, it serves for documentation purposes of the system as well. However, several studies investigating the quantifiable benefits of MBSE and system models have revealed that many of the vowed benefits can only be quantified to a limited extent (Campo et al. Reference Campo, Teper, Eaton, Shipman, Bhatia and Mesmer 2023 ; Henderson et al. Reference Henderson, McDermott, van Aken and Salado 2023 ). Especially the main benefit of system models through communication improvement was identified as an often only presumed benefit in literature that is not yet fully measurable (Henderson & Salado Reference Henderson and Salado 2021 ). Along value chains, this quantification aspect gets lost and reveals further obstacles to identify and receive a benefit from using system models (Wilking, Schleich & Wartzack Reference Wilking, Schleich and Wartzack 2020 ). While system integrators might benefit from the use of system models to manage complexity, suppliers that are forced to comply with the MBSE approach of their customer might not because of the limited complexity of their own products. Especially these forced users of MBSE approaches are seeking for new opportunities to compensate the effort coming along with the approach, for example, creating system models of their products. This reveals that the sole usage of system models is not significant enough to provide the necessary justification for it. Hence, advanced engineering methods are necessary to improve the benefit of system models and increase the feasibility to fully unfold their potentials. Therefore, system models require reuse, combined with these methods, during the product development at different stages, for example, for systems design (Mendieta et al. Reference Mendieta, La Vara, Llorens and Álvarez-Rodríguez 2017 ; Mahboob & Husung Reference Mahboob and Husung 2022 ) or consistent data models (Schwede et al. Reference Schwede, Hanna, Wortmann and Krause 2019 ). These two examples show the reuse in terms of recycling of system models, a common and feasible approach for other models such as CAD. Approaches within the last years have shown another type of reusing system models. They focus on how to transform system models into other model artefacts, to enable an authoritative source of truth for other models or derive directly executable code from it, such as shown by Vogel-Heuser et al. ( Reference Vogel-Heuser, Schuetz, Frank and Legat 2014 ) or Schumacher & Inkermann ( Reference Schumacher and Inkermann 2022 ). But it is a crucial aspect to consider data availability at specific stages of the system development, for example, by synchronizing models at specific milestones. The system model in this case is not a single source of truth, but supports the model and data management for information that is stored in other models in more detail. Overall, there is a variety of use cases of system model utilization as exemplarily shown by Umeda et al. ( Reference Umeda, Takatsuki, Kobayashi, Ueda, Wada, Komatsu, Ishihama and Iwata 2022 ). The term of utilization used but not clearly specified by Umeda et al. ( Reference Umeda, Takatsuki, Kobayashi, Ueda, Wada, Komatsu, Ishihama and Iwata 2022 ) might therefore address the whole variety of use cases within this context.

Concluding from that, two types of advanced use of system models are noticeable. Those describe use forms, such as reuse, that exceed common use scenarios, like documentation and communication purposes. First, the reuse in terms of recycling existing content within system models to share information across projects, for example, for the next product generation development. Second, the reuse in terms of system model utilization , which involves the vertical as well as horizontal distribution of information and processing of system models, such as the derivation of executable code or development artefact, in order to facilitate advanced engineering methods. The distribution of information is motivated by the aim, to extract information from the system model and support the coping with other challenges along the life cycle or within the system model, for example, by making information available that is stored within the system model but required in a different tool. This initial understanding of utilization and its contextualisation in system model reuse forms the basis for the conducted literature review (see Section 2).

With respect to ASE, the holistic application of MBSE is vital as a systematic approach to handle the complexity of a system. Nevertheless, advanced engineering methods are needed to support its application. With the efforts invested in system modelling, the reuse of those models enables a compensation of such efforts and, therefore, the successful utilization of system models contributes to the fundamentals of ASE, as highlighted in Figure 1 . With its mission 2035 described by Albers et al. ( Reference Albers, Bursac, Scherer, Birk, Powelske and Muschik 2019 ), the community of ASE follows different goals to strive for excellence in engineering and in order to enable the development of successful products. To fulfil this, stakeholders from industry, economy, politics as well as research need to be involved and therefore fields of actions were defined (Albers et al. Reference Albers, Bursac, Scherer, Birk, Powelske and Muschik 2019 ). Especially with regards to those for research, the utilization seems a promising approach in the following defined fields of actions:

▪ Integrate systems and software engineering by combining different disciplines through model utilization.

▪ Enabling a cross-cutting activity that can affect the whole life cycle of a system.

▪ Support data competency of engineering by using a cross-discipline source for information.

Figure 1. The three main aspects of ASE regarding Dumitrescu et al. ( Reference Dumitrescu, Albers, Riedel, Stark and Gausemeier 2021 ) and the contribution of system model reuse.

Beyond that, utilizing system models promises to be an approach to significantly increase the value of system models and use them throughout the lifecycle of a system. These system model utilizations have a variety of application scenarios that were published over the last years. However, a bibliographical characterization and definition of the term is yet missing. As a foundation for an analysis in this context, the extensive literature review on SysML of Wolny et al. ( Reference Wolny, Mazak, Carpella, Geist and Wimmer 2020 ) until 2017 can be used. This addresses a research direction identified by them, which concerns the deeper analysis of their results to answer future research questions (Wolny et al. Reference Wolny, Mazak, Carpella, Geist and Wimmer 2020 ). Therefore, the present contribution builds on their results and shifts the focus of the analysis towards the utilization of system models. In this regard, they additionally conclude that future approaches need to focus more on the whole life cycle rather than the implementation phase. Utilizing system models, based on the preliminary definition, is aiming this enlargement towards the whole lifecycle. Coming along with a special interest of software engineering into SysML since 2013 and new visions such as the Digital Twin and the Internet of Things (IoT), recent publications already utilize SysML in different use cases along the life cycle, for example, the operation of Digital Twins or other life cycle stages (Fursin, Reference Fursin 2019 ; Zhang et al., Reference Zhang, Hoepfner, Berroth, Pasch and Jacobs 2021 ; Wilking et al. Reference Wilking, Sauer, Schleich and Wartzack 2022b ).

Based on the derived initial understanding of system model utilization and taking the general literature study on SysML by Wolny et al. ( Reference Wolny, Mazak, Carpella, Geist and Wimmer 2020 ) into account, it can be seen that the term of utilization and its characteristic are not considered explicitly within their review. However, the given overview of relevant SysML papers and the derived research directions as key findings of the literature review provide a good insight for further investigations towards the utilization of system models. This complies with their identified research direction which motivates a further investigation of the different approaches and to answer new research questions coming along with them. The creation of a new search string supports the identification of relevant research specified on SysML utilization, even if the favoured term of utilization is not used. In addition, the definition supports a common understanding and could be enriched with a classification system to structure existing research and identify similar approaches to take recourse to existing knowledge. Such a classification system might help to create a foundation for future modelling guidelines and conventions as well as methodical recommendations. While SysML is one language to model systems, other languages, and methods such as ARCADIA (Bajzek et al. Reference Bajzek, Faustmann, Krems, Kranabitl and Hick 2021a ) or OPM (Dori Reference Dori 2016 ) are not considered in this review, as SysML is a widely spread modelling language. It is later discussed how transferable the results are for other languages. The main research questions (RQ) of this contribution are therefore:

▪ RQ1: Which bibliographical characteristics of a literature database search string enable engineers and researchers to identify relevant literature in the context of system model utilization?

▪ RQ2: How can the use cases within the relevant papers be provided to support engineers in finding similar approaches to their desired utilization scenario and take recourse to existing knowledge?

▪ RQ3: Which initial methodical implications can be provided to support engineers and researchers during the process of creating system models that can be utilized within the product life cycle?

To answer the derived research questions, this contribution is divided into two main sections. First, a systematic literature review is conducted in Section 2 to identify relevant use cases and scenarios for the utilization of system models. Second, based on this a definition of the term utilization is presented in Section 3 as well as a derived classification system to identify relevant classes of utilization along the life cycle of the system. This is further used to create a basis for guidelines towards the utilization that support engineers who are aiming to utilize system models.

1.2. Significance

Yet, there is no common understanding of the term utilization, which comprises a variety of use and reuse scenarios for system models. By contributing to this understanding, the communication of current and future research in this context can be supported. For example, through a classification system to separate the different approaches and to derive further recommendations towards modelling and guidelines. Therefore, the results of a literature review within this area, which includes the analysis of bibliographical characteristics and is enriched with a definition and a classification system for the approaches, can guide further research in this field. This will support the way towards a common understanding and harmonized modelling approaches. With this, the utilization of system models is a valuable contribution towards the ASE portfolio. While it is based on systems engineering principles by using notation compliant models, the utilization of system models rethinks previous modelling use cases and defines new approaches of how to model and how to further use these models. The complex nature of systems, developed by using systems engineering methods, comes along with a variety of benefits for utilizing system models, as later showed in the examples. Nevertheless, advanced systems like Digital Twins are a likely use case for the utilization as they are highly based on models and require a sophisticated integration of multiple disciplines and domains throughout the life cycle (Schleich et al. Reference Schleich, Dittrich, Clausmeyer, Damgrave, Erkoyuncu, Haefner, Lange, Plakhotnik, Scheidel and Wuest 2019 ).

2. Literature review

The methodical approach of this publication follows the depicted steps in Figure 2 . It begins with the literature review of Wolny et al. ( Reference Wolny, Mazak, Carpella, Geist and Wimmer 2020 ), which offered a first set of relevant SysML papers. Yet, the review was restricted to publications prior to 2017. Therefore, the review was updated by the authors of this contribution to 2022 by adapting the search string of the previous review, receiving relevant papers from 2005 until the end of 2022. With the initial understanding of system model utilization in Section 1 in mind, relevant papers from the study results were identified. This approach generated 48 relevant papers from the enlarged SysML literature review, which foster the initial understanding of the term utilization.

Figure 2. Methodical approach for the systematic literature study on utilization of system models.

To receive the bibliographical characteristics of the utilization, this preliminary set was analysed for relevant search items as well as synonyms of the terms, building up a matrix for verifying the match of the search string with all relevant papers. Resulting from that, the main search string for the literature review on utilization was developed as shown in Figure 2 . Similar to Wolny et al. ( Reference Wolny, Mazak, Carpella, Geist and Wimmer 2020 ) the title was the main search field for SysML papers, specifically searching for “SysML” and written-out versions. Besides, the authors of this contribution decided to add author keywords in combination with relevant synonyms or close terms for the utilization, such as “utilis*” or “utiliz*” or connected terms like “generat*” or “automat*.” These synonyms were identified by using various English language dictionaries, such as Oxford and Cambridge dictionary. In addition, the identified papers within the previous literature study were taken as source for relevant terms. Thus, the search string combines two aspects. First, the identification of relevant SysML papers by searching for them through title and author keyword. The second part includes all relevant and identified search items, resulting from the initial set of identified papers.

Although stated by the International Council on Systems Engineering (INCOSE) and other publications (Weilkiens Reference Weilkiens 2019 ), MBSE does not necessarily include the usage of SysML. However, many relevant publications do not specify the occurrence of SysML models within their publications but refer to MBSE models or generic system models (e.g., Karban et al. Reference Karban, Dekens, Herzig, Elaasar, Jankevičius, Angeli and Dierickx 2016 ; Horber et al. Reference Horber, Wilking, Schleich, Wartzack, Binz, Bertsche, Spath and Roth 2021 ; Zerwas et al. Reference Zerwas, Jacobs, Kowalski, Husung, Gerhard, Rumpe, Zeman, Vafaei, König and Höpfner 2022 ). Therefore, a reasonable trade-off is the inclusion of author keywords for “SysML,” as these are actively chosen by the author. Adding MBSE or systems engineering into the search string would lead to a significant increase of the received results without contributing significantly to more relevant papers. With the search string of Figure 2 the literature review was conducted following PRISMA (Page et al. Reference Page, McKenzie, Bossuyt, Boutron, Hoffmann, Mulrow, Shamseer, Tetzlaff, Akl, Brennan, Chou, Glanville, Grimshaw, Hróbjartsson, Lalu, Li, Loder, Mayo-Wilson, McDonald, McGuinness, Stewart, Thomas, Tricco, Welch, Whiting and Moher 2021 ). The literature search was conducted on 02/16/2023 and used the scientific database Scopus to cover high-quality and peer-reviewed journal articles, conference papers and book chapters in English language. The publication years were limited to the period up to the year 2022 and after 2005, the latter grounds on the first public release of SysML as described by Wolny et al. ( Reference Wolny, Mazak, Carpella, Geist and Wimmer 2020 ). The results from Scopus were compared to other databases like IEEE and no additional results were identified. From the resulting 258 contributions, 6 duplicates were removed. After reading the papers’ titles as well as abstracts and comparing the content to the initial understanding of utilization, 54 papers were classified as out of scope and therefore excluded. Out of scope summarizes three different exclusion reasons. First, the majority of the excluded papers describe scenarios in which SysML is used to model a specific system. These papers often contain the part “[…] using SysML” in their titles and therefore are captured with the search string. A second exclusion reason is the use of other modelling languages. These papers use the abstract to introduce another modelling language that is similar to SysML. However, by focusing on SysML only, these papers were excluded from the list of relevant papers. In addition, the term “reuse” is used ambivalently, containing cases which describe the reuse in terms of utilization or recycling of models. With the focus on utilization, all papers that focus the relevant topic of model recycling were not part of this literature review and therefore were excluded. With the included 198 papers, the analysis and discussion of the results is performed in the following and shown in Figure 2 leading to the detailed definition of utilization of SysML system models.

2.1. Statistical analysis of relevant papers

Based on the conducted systematic literature review described in the previous section, 198 paper with reference to utilization were identified. In advance to a detailed analysis and discussion of the contributions in the following section, the overall statistics of the results are analysed first. In order to enable a comparison of the results from the study within this paper, a comparative study on the general use of SysML in scientific papers was performed. In contrast to the study of Wolny et al. ( Reference Wolny, Mazak, Carpella, Geist and Wimmer 2020 ), the search string was not limited to the paper titles, but also considered author keywords. As a result of this comparative study, 1501 contributions concerning SysML were identified. Reviewing those results regarding their years of publication ( Figure 3a ), a rising trend between 2005 and 2013 as well as a declining trend between 2014 and 2017 can be observed, which matches the findings of Wolny et al. ( Reference Wolny, Mazak, Carpella, Geist and Wimmer 2020 ). Since the year 2018, the number of contributions focusing SysML increased again to approximately the level of 2017 in the year 2021. Comparing those numbers to the resulting 198 publications within this systematic literature review, it can be observed that the rising trend of SysML papers in general is not represented in the results of utilization papers. In the years after 2009 the number per year declines, but rises again after 2011. After a drop in papers per year in 2015 and a high in 2016, the number averages at about 12 papers per year ( Figure 3a ).

Figure 3. Comparison of publications per year (a) resulting from systematic literature review on utilization (198 papers) and general search on SysML (1501 papers) as well as the share of publications per year (b) concerning utilization.

In order to emphasize this result, the share of publications of general SysML and utilization paper is shown in Figure 3b . Thus, about 11% of the total SysML papers per year address the utilization within the past five years. This leads to the finding in the context of the conducted literature review, that there is a continuous but not rising interest in this aspect of SysML model reuse.

2.2. Clustering and distribution

The 198 papers of the literature review show all facets of the term utilization. With the generic initial understanding and the identification of relevant bibliographical characteristics for the search string, a broad field for the further reuse and the utilization is given. However, each of them describes an individual use case to answer a specific research question. For example, while the specific use case is focused on directly creating a Failure Mode and Effects Analysis (FMEA) from a SysML model (Girard et al. Reference Girard, Baeriswyl, Hendriks, Scherwey, Müller, Hönig, Lunde, Baraldi, Di Maio and Zio 2020 ), the general aim is to derive a development artefact from a system model. Insights of these derived artefacts or documents can lead to manual feedback towards the system model. For creating these classes, the relevant papers were labelled with their specific use cases. Hence, these individual approaches can be clustered into larger classes (see Table 1 , where a short description of each is given), which define the overall aim of the paper assigned to those classes. A detailed description of the classes with examples is given in Section 2.3.

Table 1. Definition of classes for the utilization of SysML models and assessed papers

The analysis of the results of the conducted literature study regarding the aforementioned classifications reveals an uneven distribution of research efforts towards the utilization, see Table 1 . Since the resulting classes in section only use selected contributions as examples, Table 1 lists all 196 papers within their respective classes. The complete list of literature is provided in the Supplementary Material . For this literature review, the relevant papers were labelled with specific use cases. Merging these use cases led to the creation of the presented classes.

While the majority with 57% of the publications is classified as model transformations, only a fraction of the papers (3%) considers model synchronization, which was simplified as a form of bidirectional model transformation. Interestingly, model transformation could be assumed as a pre-stage for bidirectional synchronization and explain the missing publications in this class as they require preceded works. However, this cannot be observed by analysing the allocation of classes over the years. Generally the term “Single Source of Truth” or “Authoritative Source of Truth” is often mentioned with system models and MBSE approaches (Kruse & Blackburn Reference Kruse and Blackburn 2019 ; Henderson et al. Reference Henderson, McDermott, van Aken and Salado 2023 ). It is considered as one of the main aims for introducing an MBSE approach. Although there is no official definition of this term, it goes along with improved accessibility for information. This could be achieved by synchronizing models bidirectional and defining the system model as the source for relevant information throughout the system development. Nevertheless, the model transformation is the dominant research direction within the utilization of SysML models since 2006.

2.3. Detailed class description and interaction with the system model

After labelling all relevant papers, the presented classes in Table 1 were identified as common thread to describe the format of reusing and utilizing system models. In addition to these classes, different approaches for building up the system model can be observed as depicted in Figure 4 . While some use the initial system model directly, others are creating new sub-models and packages with the only purpose of utilizing these. The utilization process itself can also be differentiated into three different categories. First, the system model is feeding a database that can then be further used for the different approaches. Second, the tool API is used to connect different tools, for example, the connection between other software, such as Cameo or Matlab Simulink (Chabibi et al. Reference Chabibi, Douche, Anwar and Nassar 2016 ). Third, the system model’s XMI structure is utilized to extract the relevant information from the model (Wilking et al. Reference Wilking, Sauer, Schleich and Wartzack 2022a ). The approaches are not restricted on the usage of only one class but can be assigned to multiple classes. The individual classes with examples are explained individually in the following sections whilst referring to Figure 4 as indicated by lower case characters.

Figure 4. Derived classes of utilization with automated and manual tasks and use cases. The assigned characters (a–j) are described individually within the textual class description.

2.3.1. Artefact derivation

SysML is an integrational part of MBSE (Friedenthal et al. Reference Friedenthal, Moore and Steiner 2015 ) defining a contrast to document-based approaches by using models. However, documents are still an essential part of product development and cannot be left out, for example, for documentation purposes. Therefore, a convenient automation ( Figure 4a ) is the derivation of documents from existing models. This supports the engineer in everyday situations and prevents the creation of manual engineering artefacts throughout the product development. A common scenario for the artefact derivation is an automatic creation of Failure Modes and Effects Analyses (Hecht, Dimpfl & Pinchak Reference Hecht, Dimpfl and Pinchak 2014 ; Girard et al. Reference Girard, Baeriswyl, Hendriks, Scherwey, Müller, Hönig, Lunde, Baraldi, Di Maio and Zio 2020 ; Hecht et al. Reference Hecht, Chuidian, Tanaka and Raymond 2020 ) or Fault Trees (Mhenni, Nguyen & Choley Reference Mhenni, Nguyen and Choley 2014 ). However, artefacts are not limited to static documents but can also be extended to other artefacts such as test cases (Dahlweid, Brauer & Peleska Reference Dahlweid, Brauer and Peleska 2015 ) or Design Structure Matrices (McLellan et al. Reference McLellan, Maier, Fadel and Mocko 2009 ). Insights from these documents can be manually transferred back into the system model ( Figure 4b ).

2.3.2. Execution

SysML is based on UML (Friedenthal et al. Reference Friedenthal, Moore and Steiner 2015 ). While the generation of usable code from UML diagrams is a common approach for over 20 years (Herrington Reference Herrington 2003 ; Ciccozzi, Malavolta & Selic Reference Ciccozzi, Malavolta and Selic 2019 ), similar approaches based on SysML models appeared over the last years. These approaches describe a direct generation of executable code from individual SysML diagrams, packages or whole models, for example, to be used as core input for a software tool (Vogel-Heuser et al. Reference Vogel-Heuser, Schuetz, Frank and Legat 2014 ) or for the definition of the behaviour of a system (Godart et al. Reference Godart, Gross, Mukherjee and Ubellacker 2017 ). However, this category does not involve a semantic integration of information within a SysML diagram to an already existing code frame, but describes the direct execution of a software code generated through the SysML model ( Figure 4c ). The approach has been identified as helpful to create a single source of truth regarding the software code, as abnormal behaviours of the system as well as failures are easy to identify (Wilking et al. Reference Wilking, Sauer, Schleich and Wartzack 2022b ). In addition, the derivation of executable code saves manual coding effort and the structure of the code is more accessible through the graphical visualization of a SysML diagram.

2.3.3. Model simulation

SysML models are still mainly created by humans and even though their initial aim is the improvement of the overall system understanding, large system models are difficult to overview and check manually. A common use case is the simulation of SysML models, which is often executed in the SysML tool itself ( Figure 4e ). Typically, model simulations focus on the verification of the model, for example, verifying requirements and designs (Morkevicius & Jankevicius Reference Morkevicius and Jankevicius 2015 ), checking consistency (Bankauskaite & Morkevicius Reference Bankauskaite and Morkevicius 2018 ) or event simulation to check the behaviour of a system (Liu et al. Reference Liu, Irudayaraj, Zhou, Jiao and Goodman 2014 ). Insights from these simulations are likely used to enhance the system model itself ( Figure 4d ). Unlike the execution, the model simulation does not aim the derivation of executable code outside of the SysML tool.

2.3.4. Model synchronization

The system model in literature is often considered as the single source of truth (Hick et al. Reference Hick, Bajzek and Faustmann 2019 ). Shown by the class of semantic integration and model transformation, the information stored in the system model can be used to compensate development efforts in other stages of the product creation or along the value chain. However, change scenarios and their handling require switching between the layers in the V-Model (Hick et al. Reference Hick, Sanladerer, Trautner, Ryan, Piguet, Wilking, Horber, Faustmann, Kranabitl, Kolleger, Bajzek, Schleich and Wartzack 2023 ). For example, a change of a requirement will lead to an impact propagation analysis, often conducted on the physical layer to analyse the effect on specific subsystems and components. However, this requires a connection between discipline-specific models and a deep connection between the other layers of the V-Model. As shown in the class of model transformation, there are already many approaches, which transform a system model into other models ( Figure 4f ), but not the other way around. Hence, the class of model synchronization describes any cases in which a bidirectional model transformation is conducted to bring information back into the system model and enrich the single source of truth ( Figure 4g ). This enables consistency throughout the different models in an MBSE approach. In addition, this class involves any case, in which a system model is created from outside of the modelling tool but was not necessarily transformed from it before, such as shown by Chami, Zoghbi & Bruel ( Reference Chami, Zoghbi and Bruel 2019 ) and Dworschak et al. ( Reference Dworschak, Zirngibl, Schleich and Wartzack 2019 ).

2.3.5. Model transformation

The successful integration of MBSE is not restricted on the usage of SysML system models but involves all relevant models throughout the development (Bajzek et al. Reference Bajzek, Fritz, Hick, Maletz, Faustmann, Stieglbauer, Hick, Küpper and Sorger 2021b ). Hence, a consistent toolchain is crucial for the application of MBSE (Ma et al. Reference Ma, Wang, Lu, Vangheluwe, Kiritsis and Yan 2022 ). However, multidisciplinary systems will require synchronous modelling activities during the development. This requires domain-specific models, which are based on a holistic system model that allows interoperability (Brahmi et al. Reference Brahmi, Hammadi, Aifaoui and Choley 2021 ). To enable a consistent transformation of a system model into a domain-specific model, an automated approach ( Figure 4h ) reduces the risk of biased human interpretation. Model Transformation enables the use of SysML models or individual diagrams within another modelling tool, for example, by defining first parameters for a CAD tool (Schumacher & Inkermann Reference Schumacher and Inkermann 2022 ). Other examples are the transformation into digital signal processing and control simulation such as Simulink (Paredis et al. Reference Paredis, Bernard, Burkhart, Koning, Friedenthal, Fritzson, Rouquette and Schamai 2010 ; Palachi, Cohen & Takashi Reference Palachi, Cohen and Takashi 2013 ) or even the transformation from a SysML into a UML model. It involves the direct transformation of the whole SysML model or diagram into a model within another tool, for example, for further modelling activities (Mahboob et al. Reference Mahboob, Husung, Weber, Liebal and Krömker 2019 ), simulations (Barbau, Bock & Dadfarnia Reference Barbau, Bock and Dadfarnia 2019 ) or execution (Kapos et al. Reference Kapos, Tsadimas, Kotronis, Dalakas, Nikolaidou and Anagnostopoulos 2021 ). It does not involve the return of information into the initial system model.

2.3.6. Semantic integration

A whole model transformation is not always necessary. In particular cases, information just needs to be extracted from a system model ( Figure 4i ), for example, for the integration of design requirements into CAD (Brahmi et al. Reference Brahmi, Hammadi, Aifaoui and Choley 2021 ) or product architecture design (Schwede et al. Reference Schwede, Winter, Lödding and Krause 2020 ). This allows the storage of system information in a model that can be used, for example, in domain or discipline-specific applications. This is significantly relevant for information that is used by multiple applications or is transferred along the value chain (Wilking et al. Reference Wilking, Schleich and Wartzack 2020 ) and is exchanged through several system models. In fact, the extracted information can later be used to build up another model as shown by Kerzhner & Paredis ( Reference Kerzhner and Paredis 2011 ). In addition, the extracted and processed information leads to new insights about the system that can be manually handed back into the system model ( Figure 4j ). For example, for the OEM or for suppliers, it can be crucial to store the information in a more generic system model rather than in a discipline-specific model. The recipient can read the necessary information in the system model instead of manually searching for it, which often comes along with human errors and misinterpretation. However, this class describes the specific search for information within the model and does not attempt to transform the whole model into another.

2.3.7. Other

Few papers describe cases of SysML utilization that are not classifiable in any of the aforementioned classes but for example, describe methodical approaches for the utilization (Fu, Liu & Wang Reference Fu, Liu and Wang 2021 ).

3. System model utilization

3.1. definition and classification system.

The literature review enabled the analysis of bibliographical characteristics for the reuse and utilization of SysML system models. Within the introduction in Section 1, an initial understanding was given, that was based on the subjectively and implicitly usage of the term “utilization”. To enable a common understanding of the term as well as giving a basis for harmonized modelling strategies, a definition is presented in this section. Shortly after the introduction of SysML first attempts were made to further reuse the models. In fact, reuse describes the generic reuse of the models and therefore can also mean that elements of a system model are reused, for example, throughout several product generations (Albers, Bursac & Wintergerst Reference Albers, Bursac, Wintergerst, Mastorakis and Solomon To 2015 ). This reuse is defined as an advanced use of the three purposes for system modelling, that is, improvement of communication, analysis of the technical system and the documentation of it (Friedenthal, Dori & Mordecai Reference Friedenthal, Dori and Mordecai 2021 ).

More specific the utilization of system models describes the reuse for use cases or several application scenarios along the product life cycle of the system by creating machine-readable system models. The authors of this contribution therefore define utilization as a subcategory of reusing system models. Those models are used for automated vertical and horizontal distribution of their included information as well as interfacing domain-specific tools (e.g., requirements management system). Therefore, utilization adds new objectives for existing or adapted system models, whereas recycling of system models does not change the objectives of the modelled information. This definition of the utilization is shown in Figure 5 . However, utilization is only one form of reusing system models, as mentioned before.

Figure 5. Definition of the reuse in terms of system model utilization.

The utilization of system models in the context of SysML can therefore address the introduced classes of artefact derivation, execution, model simulation, model transformation, semantic integration and/or model synchronization ( Figure 5 ). Although this review was conducted for SysML models only, the classification system is also suitable for other languages and utilization scenarios, such as within Capella and the connection with discipline-specific models. However, other approaches might lead to additional use cases that cannot be grasped by SysML.

In addition to the classes, every single use case of utilization, such as verification and validation, Digital Twins, and so on, and stages within the life cycle according to ISO/IEC/IEEE 15288 ( 2015 ) can be assigned to a class. This leads to a three-dimensional classification system for the utilization of system models, as shown in Figure 6 . The orange indications within this figure are relevant for the exemplary application for two different scenarios of the proposed classification system within Section 3.2. Interdependencies between the three axes are possible. While classes can be found in any life cycle stage, specific use cases might predefine a class and a stage of the life cycle. This does not lead to a limitation of the classification system, but reveals applied and potentially more feasible approaches.

Figure 6. Proposed classification system for the utilization with integration of scenarios S1.1 and S1.2 (see Table 2 ) with upstream modelling activities and scenario S2 (see Table 3 ) into the classification system; visual design based on VDI/VDE Society Measurement and Automatic Control ( 2015 ).

For an application of the classification system and finally the derivation of connected methodical modelling approaches, further considerations have to be stated. Since machine-readability and interoperability between SysML and other models are highly depending on a tool support, many of the described scenarios are using the XMI structure of SysML models (Handley et al. Reference Handley, Khallouli, Huang, Edmonson and Kibret 2021 ). This requires awareness in the modelling process of the system model. As humans are able to interpret text fields on a diagram and might make the correct association with nearby diagram objects, the machine-readable context is gone and information is lost in the visualization of the model, but not in the model itself. This is not necessarily caused by a false application of SysML, but rather by the given interpretation spaces coming along with this language. The further utilization of system models therefore requires modelling recommendations and guidelines or simplified APIs that enable the utilization described in this definition. Universal methodical approaches are not yet existing, since the applications are use case specific and might require different methods. Therefore, a thorough classification of the approach, for example, through the presented classification system, might help to develop certain guidelines and methods to enable a harmonized approach for the utilization. However, the allocation within the classification system must not be limited to a distinct point but can be enlarged to multiple sections, for example, for relevancy along different life cycle stages. Nevertheless, a separation of the class is recommended, as these significantly influence the purpose of the utilization. Yet, multiple classes can be combined to a connected application scenario that involves different classes but their purposes are dissociated from each other. The recommended classification system therefore offers a separation of the described classes in layers, as they are strictly separated. In addition, the context of the life cycle stage is given based on ISO/IEC/IEEE 15288 ( 2015 ). Models have different purposes regarding their usage within the life cycle of a system. Nevertheless, almost every class of the utilization can be discovered in every stage of the life cycle regarding the analysed paper, which can be seen in the Supplementary Material . Furthermore, the separation of use cases for the utilization supports the definition of a goal for the approach that might be achieved in a later stage of the life cycle but must be conducted in a previous one, for example, through the creation of models for the operation phase that are created during the development. This shift of the required effort can be useful to create the right modelling approach in the right context of the product creation. The given examples for use cases, such as the Digital Twin, show this shift. While Digital Twins might benefit from the utilization of the SysML model during operation, the required modelling and utilization effort will happen during its development and concept phase as well as the twinning during the production and instantiation (Stark et al. Reference Stark, Anderl, Thoben and Wartzack 2020 ). But the assignment to a use case can be shaped by different implications and can have intersections with other use cases. The shown use cases in Figure 6 are examples for published work, such as a system model-driven student project for engineering education (Wilking et al. Reference Wilking, Behringer, Fett, Goetz, Kirchner and Wartzack 2023 ). While some use cases reveal technical implications for methodical recommendations, others can simply provide similar approaches to the designer. Used within product development, the classification of the system model towards utilization reveals insights in the application scenario as well as required modelling activities. This is necessary to develop further methodical approaches for the specific utilization scenario.

3.2. Application of the proposed classification system for utilization

For a further description of the definition of the term and the integrated classification system, as well as first recommendations towards guidelines for a methodical approach, two scenarios are given which describe different use cases, classes, and stages of the product life cycle. The variety of use cases for the utilization of system models is huge. However, the two use cases can give first valuable feedback on the requirements for modelling guidelines or methodical approaches towards the utilization. Figure 6 shows their integration into the classification system, where they are indicated in orange colour.

3.2.1. S1: SysML 4 Digital Twins

Wilking et al. ( Reference Wilking, Sauer, Schleich and Wartzack 2022 b ) describe an approach to utilize SysML models that are visualizing the behaviour of a Digital Twin system. Since models are a significant part of a Digital Twin (Schleich et al. Reference Schleich, Anwer, Mathieu and Wartzack 2017 ), MBSE is a promising approach for the design of Digital Twins (Wang et al. Reference Wang, Steinbach, Klein and Anderl 2021 ). System models add a special value to the development and operation of Digital Twins by defining the general behaviour of the physical and digital counterparts as well as connecting relevant information and other models that are used within the Digital Twin concept. Examples for this are the integration of the system into its environment or the inclusion of model decay regarding the physical counterpart throughout its lifecycle into the behaviour of the system. In this scenario, a system model was created as a central model that visualizes the interaction of multiple relevant models for the Digital Twin as well as the detected and stored operational data from the physical counterpart. Packages of that model were directly derived and used as executable code to base the actual behaviour of the Digital Twin completely on the SysML model, see Table 2 . In addition, a machine learning algorithm as well as relevant information for its execution was implemented into the system model to directly connect physical elements of the model with relevant elements for machine learning (Wilking et al. Reference Wilking, Sauer, Schleich and Wartzack 2022a ). This enabled parallel activities to model and execute the behaviour of the Digital Twin, for example, by conducting a precise but time-consuming simulation and parallel run a machine learning script to gather less detailed but therefore quick recommendations towards the current state of the system and required actions, for example, maintenance intervals. This scenario shows a case in which the benefit of the utilization is achieved within a later stage of the life cycle, but the effort has to be invested already during the development. This is a significant shift of effort and benefit and requires a thorough consideration within the modelling approach, as existing approaches do not yet involve the utilization and the preparation of utilized system models. In addition, the whole use case is taking advantage of two classes, showing that a combination is possible and for some cases reasonable.

Table 2. Scenario description of “SysML 4 Digital Twins”

3.2.2. S2: Formalization of integrated variation management and its use in robustness evaluation

A second scenario is introduced by Horber et al. ( Reference Horber, Goetz, Schleich and Wartzack 2022 a ), which proposed an approach to formalize engineering activities in the context of integrated variation management. Motivated by the variety of different methods, tools as well as data in variation management and a lack of their integration into a consistent model, the approach uses activity, requirement and block diagrams (SysML) to model the contents of integrated variation management. Its use enables the consistent reuse of model elements in connected approaches within the variation management domain, see Table 3 . As applied by Horber et al. ( Reference Horber, Goetz, Schleich and Wartzack 2022 a ) to early stage variation management, which focuses on the development of robust product concepts, the created model can be utilized for further approaches, for example, the robustness evaluation (Horber et al. Reference Horber, Li, Grauberger, Schleich, Matthiesen and Wartzack 2022b ). The approach uses qualitative sketches of product concepts, combines them with embodiment-function-relations and transfers the qualitative model into a corresponding SysML model. The contents of the model can then be retrieved for robustness evaluation and build the basis for further development of the selected concept. Other model elements, such as key characteristics, are thereby relevant for even later stages like parameter design or even production.

Table 3. Scenario description for formalization of integrated variation management

3.3. Methodical implications resulting from the classification system

The two described scenarios give a detailed insight into the utilization of SysML system models as an equal purpose for the creation of system models. While the life cycle stages and classes are finite, the use cases allow various application scenarios for the utilization. However, it does not seem that the use cases affect the modelling approach significantly. In contrast, classes and stages of the life cycle have a significant influence on the approach. While the classes are naturally describing separate approaches and aims of the utilization, life cycle stages strongly influence the approach by determining the context of the creation and the use for a not yet or already existing system with significant differences for the modelling.

However, the scenarios have shown that a methodical approach towards harmonizing the modelling procedure is crucial. This is especially the case where the utilization is conducted along a value chain and where harmonized modelling strategies between stakeholders are necessary. Currently, some system modelling tools support interfaces with other tools and by that enable a model transformation for a further utilization. Thus, many of the other defined classes are not yet sufficiently integrated into systems model toolchains, as this is not only relevant for the system modelling tool, but also for discipline and domain-specific tools that integrate the system model.

Many use cases are utilizing the XMI structure of the modelling language to further utilize the models. The XML files contain the whole model as well as the visualization. This leads to immense and confusing model sizes. Hence, individual packages within the model must be created, which leads to scenarios in which a model is just created for its further utilization but not used again. Thus, the overall benefit of the approach is diminished. A possible solution for this could be the introduction of a utilization view which enables the reduction of a model towards necessary aspects for the utilization without affecting the model itself. Views are a common tool in SysML to enable different degrees of detail on a model as well as different perspectives depending on the discipline and domain (Alt Reference Alt 2012 ; Albers et al. Reference Albers, Bursac, Scherer, Birk, Powelske and Muschik 2019 ). With this, it can be ensured that the model itself can still be used for other purposes, while the utilized version is reduced. However, this reduction cannot be generalized. In fact, while a model transformation reasonably should transfer the visual aspects of a diagram, other utilization classes might not use the diagram at all.

The need for such a methodical approach, that integrates the aimed utilization into the methodical model creation, and its effect on the benefit was shown by Fu et al. ( Reference Fu, Liu and Wang 2021 ) for the reuse of system models. Mendieta et al. ( Reference Mendieta, La Vara, Llorens and Álvarez-Rodríguez 2017 ) already discussed the tool-specific obstacles coming along with XML and SysML model recycling. Nevertheless, the presented classification system offers a first basic structure for modelling guidelines and recommendations. Especially with SysML v2, which is no longer based on UML and offers new potentials, for example, through changing the restriction of the XML exchange format by an accessible API.

The different classes distinguish fundamentally different approaches for the further reuse of SysML models with different aims and necessary efforts. While model transformations might aim to transfer individual diagrams into domain-specific models, such as shown in the transfer of electrical system designs between the Siemens tools Capital Systems Capture and System Modelling Workbench (Hick et al. Reference Hick, Sanladerer, Trautner, Ryan, Piguet, Wilking, Horber, Faustmann, Kranabitl, Kolleger, Bajzek, Schleich and Wartzack 2023 ), others will require the collection of information from the whole model. Especially the transfer of the visual layout of a diagram into domain-specific tools will cause tremendous effort to develop certain modelling guidelines to ensure a transfer without information loss and, most importantly, a benefit with the domain or discipline-specific model. Furthermore, a synchronization requires a harmonized modelling approach throughout the whole model. This shows that the creation of the methodical approach depends on the aim and the corresponding class that was assessed for the utilization of the system model.

This basis of considerations for the methodical approach is enriched by viewing the life cycle stages. Engineers who want to utilize system models will be confronted with several challenges regarding the life cycle of the system. The model maturity significantly differs throughout the development until the instantiation of the system. Therefore, the actual point in time for making use of and benefit from the utilized model might differ from the point in which the system model is created. This leads to two aspects coming along with a life cycle related consideration for the utilization. First, the life cycle stage itself in which the model is utilized. A utilized system model for a concept phase will be based on a low-matured model foundation. Furthermore, it has different aims. Models, which are being used in later stages, will be confronted with a different level of detail regarding the system model or the connected domain-specific models. The second important aspect is considering the difference between utilization and creation of the model. This is a crucial consideration for models that decay over time and do not represent, for example, the behaviour of a system throughout the period of use. This decay needs to be considered. Approaches must integrate this dynamic to represent the stage of a system, in which the system model is utilized and not only the state in which the model is created. Summing up, methodical approaches have to consider these two differentiations for the utilization.

An additional aspect for considerations towards methodical approaches is the differentiation of use cases within the utilization. For example, utilization approaches for the Digital Twin use case are mainly focused on the operation phase of the system predefined by the use case that is describing a predominantly maintenance- and usage-oriented concept. In fact, specific approaches, such as deriving specified documents, for example, a FMEA can help to define a very specific guideline for the utilization, such as seen in several publications (Girard et al. Reference Girard, Baeriswyl, Hendriks, Scherwey, Müller, Hönig, Lunde, Baraldi, Di Maio and Zio 2020 ; Hecht et al. Reference Hecht, Chuidian, Tanaka and Raymond 2020 ). Some of the specific approaches can be clustered together with the classes of the utilization. However, as SysML application throughout enterprises and sectors can be affected by the usage of own profiles and adaptions. Specific guidelines for these might be an inspiring example. But they do not necessarily provide a generic approach for supporting enterprises in the attempt of utilizing their system models.

4. Limitations

The results of this systematic literature are analysed aiming towards the identification of relevant literature in the context of utilization. As a limitation, the derived search string only shows use cases within the language of SysML as shown in Figure 5 . As a widely spread language for modelling systems, this limitation is reasonable but can cause the lack of use cases connected to other languages. An example for this is Capella with its ARCADIA method (Voirin et al. Reference Voirin, Bonnet, Normand, Exertier, Auvray, Bocquet, Bonjour and Krob 2016 ) and a possible PLM integration for system models. From a methodical point of view, other languages might result in other implications when it comes to modelling processes. But the general aim of system modelling and the utilization is similar across the languages and therefore not language neutral.

The literature review was conducted along with the PRISMA guidelines, but the results are limited due to the used search string. To ensure reliable results, the initial understanding of utilization was derived from the literature review of Wolny et al. ( Reference Wolny, Mazak, Carpella, Geist and Wimmer 2020 ) and the search was conducted in a broader way by taking author keywords besides titles into account. Hence, it cannot be finally excluded whether all papers were identified but the methodical approach reduces this risk.

In addition, the presented classification system is an open one. While the classes and the life cycle stages are fixed, the differentiation between use cases is not yet limited and standardized, as more and more papers are published, which add new application scenarios for the utilization of system models. However, at one point these use cases must be clustered to enrich the classification system and differentiate more precise towards specific modelling guidelines and recommendations for methodical approaches.

5. Discussion and conclusions

The high number of publications, which are focussing the utilization of SysML models, shows that the models and their purpose have reached a next step of their evolution. The model transformation is the dominant research direction for the utilization, immediately compensating effort by transforming existing system models into discipline or domain-specific models. As discussed, this could be a first step towards an authoritative source of truth within MBSE and particularly in system modelling. But it reveals that there are still lots of undiscovered potentials within this research field, such as data availability. However, model synchronization is not necessarily needed at any point throughout development but is required at specific milestones to achieve consistency for the next stage of the development process. Especially for complex systems, this consistency is difficult to achieve along the used models. Change scenarios reveal missing model links in case of missing synchronization, thus preventing traceability. These two classes, model transformation and synchronization, require a significant effort for a methodical approach, modelling guidelines and a sufficient tool integration, considering the different aspects of them (Khandoker et al. Reference Khandoker, Sint, Gessl, Zeman, Jungreitmayr, Wahl, Wenigwieser and Kretschmer 2022 ; Saqui-Sannes et al. Reference Saqui-Sannes, Vingerhoeds, Garion and Thirioux 2022 ). This methodical approach is highly relevant to integrate the utilization into existing modelling activities and achieve the holistic aim of compensating effort. The other classes are promising towards a quicker integration and directly benefiting from applying the underlying use cases of the classes, for example, through document derivation.

Enterprises might naturally benefit from using system models, while non-classical sectors that recently just began integrating MBSE methods and models into their development approaches seek for more benefits. Their products might be on the edge of feasible complexity for integrating MBSE. Justifying systems engineering and MBSE through quantifiable performance indicators is still an ongoing research topic (Honour Reference Honour 2013 ; Henderson et al. Reference Henderson, McDermott, van Aken and Salado 2023 ). Compared to current approaches, the utilization of system models offers a quantifiable comparability, for example, through saved lines of code or reduced modelling time.

With this contribution, by giving a literature-based definition and classes, combined with scenarios, the diverse field of SysML utilization has been clustered into different research directions. The definition offers a basis for further research, such as building up modelling strategies and introducing the utilization of system models into methodical approaches. The definition, in combination with the bibliographical characteristics of the term, will help other researchers to identify the intersection of their research with the utilization. It also reveals that a generic recommendation for a methodical approach to conduct this utilization is not feasible, but can be characterized through the presented classification system. This demands further research on specific approaches and the realization of sophisticated utilizations of system models. Especially the realization of the six presented classes includes further potential for modelling activities, use cases and software integration. Based on the results, the defined research questions of this contribution can be answered as follows:

▪ RQ1: The developed search string depicted in Figure 2 shows the bibliographical characteristic of the term utilization. Engineers and researchers are enabled to use this string to identify relevant research in this field and conduct further analysis towards new research questions. It was revealed that a further adaption of the search string, for example, by splitting reuse into “use”, leads to a significant enlargement of the results, which, however, does not increase the amount of relevant paper. This is because most approaches simply describe the use of SysML, which does not refer to utilization (see Figure 5 ). With this contribution and the Supplementary Material , a collection of relevant material for engineers, who are trying to implement the utilization of system models into their work, is presented. In addition, with the presented classes and their interaction with the system model, depicted in Figure 4 , similar approaches can be identified and be extracted from the Supplementary Material to reduce time-consuming research.

▪ RQ2: Derived from the literature review, a holistic definition of the term “Utilization” was proposed and summarized in Figure 5 . Based on that, a possible classification system was provided in Figure 6 to support future attempts for specific methodical approaches that are designed to address a specific class and use case of the utilization. By that, researcher and practitioners can take recourse to existing knowledge in the form of similar approaches when conceptualizing system models for utilization.

▪ RQ3: Along with general methodical implications in Section 3.3, two scenarios were mapped to the given definition and classification system to show scenario-specific implications. Nevertheless, further integration of utilization requires tool support and the current possibilities can be limited by the tool capabilities. In addition, other scenarios might lead to more and different methodical implications. Generally, with this contribution the following three initial methodical implications for engineers and further research were identified:

○ Specific modelling rules or guidelines are needed, especially for taking advantage of the full possible utilization, for example, to harmonize the created system models.

○ While it is crucial to aim the utilization of existing system models, the isolation of these might be useful to narrow them purposefully. This requires the introduction of a new view that aims the utilization of these models and combine the initial system model with the adapted version. For example, this new view can be an additional layer for diagrams of a system model, specifying their reuse in terms of utilization.

○ Model maturity can have a significant impact on the possibilities and implementation of the system model utilization. An example for this is the availability of information at specific milestones of a project and how this information can be provided for specific use cases of utilization. This must be considered for the creation of these models and could be a future research topic to enhance system modelling languages to further meet this requirement.

In summation, a system-model-centred approach is a promising direction for future system development (Bajaj, Zwemer & Cole Reference Bajaj, Zwemer and Cole 2016 ). The utilization of system models will be the key for further realization of this idea. SysML v2 shows a promising approach for this further integration (Bajaj, Friedenthal & Seidewitz Reference Bajaj, Friedenthal and Seidewitz 2022 ), for example, through the accessible API. The definition and classification system were shown for SysML models, but it can be stated that the definition and classification are not limited to this modelling language, as bridges might exist between languages (Badache & Roques Reference Badache and Roques 2018 ). Therefore, principles of the definition as well as the classification are directly translatable towards the other languages or potential enrichments of the languages such as shown with SysML v2.

The upcoming advanced systems require the further integration of systems engineering into the development approaches of enterprises. Advanced engineering methods, such as from digital engineering, can help to enable development strategies for this future of advanced systems. The utilization of system models promises to be a useful approach and addition to the pool of Advanced Engineering methods. It enables the compensation of system modelling effort and a partial evaluation of it. Furthermore, it integrates disciplines into the systems engineering, which are three of the main goals of ASE (Albers et al. Reference Albers, Dumitrescu, Gausemeier, Riedel and Stark 2018 ).

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/dsj.2024.3 .

Acknowledgements

This project is supported by the Federal Ministry for Economic Affairs and Climate Action (BMWK) on the basis of a decision by the German Bundestag. It is part of the IGF Project 22467 BG (FVA 889 II Digital Twin) in collaboration with the Forschungsvereinigung Antriebstechnik (FVA) e.V.

Figure 1. The three main aspects of ASE regarding Dumitrescu et al. (2021) and the contribution of system model reuse.

Figure 6. Proposed classification system for the utilization with integration of scenarios S1.1 and S1.2 (see Table 2) with upstream modelling activities and scenario S2 (see Table 3) into the classification system; visual design based on VDI/VDE Society Measurement and Automatic Control (2015).

Wilking et al. supplementary material

No CrossRef data available.

View all Google Scholar citations for this article.

Save article to Kindle

To save this article to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle .

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

• Fabian Wilking (a1) , Dennis Horber (a1) , Stefan Goetz (a1) and Sandro Wartzack (a1)
• DOI: https://doi.org/10.1017/dsj.2024.3

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox .

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive .

Reply to: Submit a response

- No HTML tags allowed - Web page URLs will display as text only - Lines and paragraphs break automatically - Attachments, images or tables are not permitted

Your details

Your email address will be used in order to notify you when your comment has been reviewed by the moderator and in case the author(s) of the article or the moderator need to contact you directly.

You have entered the maximum number of contributors

Conflicting interests.

Please list any fees and grants from, employment by, consultancy for, shared ownership in or any close relationship with, at any time over the preceding 36 months, any organisation whose interests may be affected by the publication of the response. Please also list any non-financial associations or interests (personal, professional, political, institutional, religious or other) that a reasonable reader would want to know about in relation to the submitted work. This pertains to all the authors of the piece, their spouses or partners.

• Frontiers in Molecular Biosciences
• Biological Modeling and Simulation
• Research Topics

Systems Modeling: Approaches and Applications

Total Downloads

Total Views and Downloads

About this Research Topic

The development of modeling tools has permitted an increased understanding of how components in different systems interact and behave. Thus, systems modeling has led to advances in several areas, such as medicine, biotechnology and engineering. Applications include the study of ecological models, diseases and ...

Keywords : systems modeling, simulation, bio-systems

Important Note : All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.

Topic Editors

Topic coordinators, recent articles, submission deadlines.

Submission closed.

Participating Journals

Total views.

• Demographics

No records found

total views article views downloads topic views

Top countries

Top referring sites, about frontiers research topics.

With their unique mixes of varied contributions from Original Research to Review Articles, Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author.

Overview of System Development Life Cycle Models

24 Pages Posted: 13 Sep 2019

Kwadwo Kyeremeh

Sunyani Technical University - Department of Accountancy

Date Written: September 5, 2019

During the time half of the twentieth century, the utilization of Programmed computers has become huge. As an outcome, software programming has turned out to be increasingly differing and complex. Also, there are expanding requests on software programming – it must be less expensive, have more usefulness, be conveyed speedier, and be of higher quality than already. In the constantly changing environment and society of programming advancement, clearly the procedures and strategies utilized when growing little projects are not adequate while developing extensive frameworks. As one response to this, distinctive improvement lifecycle models have been characterized. This paper portrays the three fundamental sorts of systems Development lifecycle models, from the successive models by means of incremental models to transformative models. The iterative advancement technique is additionally examined, and we additionally intricate the association of advancement lifecycle models to two rising fields in programming designing: programming design and part based programming advancement.

Keywords: framework, system developement, lifecycle, software and programming

Suggested Citation: Suggested Citation

Kwadwo Kyeremeh (Contact Author)

Sunyani technical university - department of accountancy ( email ).

P.O BOX 206 SUNYANI, Brong Ahafo 00233 Ghana 244489758 (Phone)

Do you have a job opening that you would like to promote on SSRN?

Paper statistics, related ejournals, econometrics: computer programs & software ejournal.

Subscribe to this fee journal for more curated articles on this topic

Accounting Technology & Information Systems eJournal

Microeconomics: life cycle models & behavioral life cycle models ejournal, software engineering ejournal, electrical engineering ejournal, anthropology of science & technology studies ejournal.

• Research article
• Open access
• Published: 19 November 2019

Mathematical modelling for health systems research: a systematic review of system dynamics and agent-based models

• Rachel Cassidy   ORCID: orcid.org/0000-0002-4824-0260 1 ,
• Neha S. Singh 1 ,
• Pierre-Raphaël Schiratti 2 , 3 ,
• Agnes Semwanga 4 ,
• Peter Binyaruka 5 ,
• Nkenda Sachingongu 6 ,
• Chitalu Miriam Chama-Chiliba 7 ,
• Zaid Chalabi 8 ,
• Josephine Borghi 1 &
• Karl Blanchet 1  

BMC Health Services Research volume  19 , Article number:  845 ( 2019 ) Cite this article

22k Accesses

46 Citations

5 Altmetric

Metrics details

Mathematical modelling has been a vital research tool for exploring complex systems, most recently to aid understanding of health system functioning and optimisation. System dynamics models (SDM) and agent-based models (ABM) are two popular complementary methods, used to simulate macro- and micro-level health system behaviour. This systematic review aims to collate, compare and summarise the application of both methods in this field and to identify common healthcare settings and problems that have been modelled using SDM and ABM.

We searched MEDLINE, EMBASE, Cochrane Library, MathSciNet, ACM Digital Library, HMIC, Econlit and Global Health databases to identify literature for this review. We described papers meeting the inclusion criteria using descriptive statistics and narrative synthesis, and made comparisons between the identified SDM and ABM literature.

We identified 28 papers using SDM methods and 11 papers using ABM methods, one of which used hybrid SDM-ABM to simulate health system behaviour. The majority of SDM, ABM and hybrid modelling papers simulated health systems based in high income countries. Emergency and acute care, and elderly care and long-term care services were the most frequently simulated health system settings, modelling the impact of health policies and interventions such as those targeting stretched and under resourced healthcare services, patient length of stay in healthcare facilities and undesirable patient outcomes.

Conclusions

Future work should now turn to modelling health systems in low- and middle-income countries to aid our understanding of health system functioning in these settings and allow stakeholders and researchers to assess the impact of policies or interventions before implementation. Hybrid modelling of health systems is still relatively novel but with increasing software developments and a growing demand to account for both complex system feedback and heterogeneous behaviour exhibited by those who access or deliver healthcare, we expect a boost in their use to model health systems.

Peer Review reports

Introduction

Health systems are complex adaptive systems [ 1 ]. As such, they are characterised by extraordinary complexity in relationships among highly heterogeneous groups of stakeholders and the processes they create [ 2 ]. Systems phenomena of massive interdependencies, self-organising and emergent behaviour, non-linearity, time lags, feedback loops, path dependence and tipping points make health system behaviour difficult and sometimes impossible to predict or manage [ 3 ]. Conventional reductionist approaches using epidemiological and implementation research methods are inadequate for tackling the problems health systems pose [ 4 ]. It is increasingly recognised that health systems and policy research need a special set of approaches, methods and tools that derive from systems thinking perspectives [ 5 ]. Health systems encompass a many tiered system providing services to local, district and national populations, from community health centres to tertiary hospitals. Attempting to evaluate the performance of such a multi-faceted organisation presents a daunting task. Mathematical modelling, capable of simulating the behaviour of complex systems, is therefore a vital research tool to aid our understanding of health system functioning and optimisation.

System dynamics model (SDM)

System dynamics models (SDM) and agent-based models (ABM) are the two most popular mathematical modelling methods for evaluating complex systems; while SDM are used to study macro-level system behaviour such as the movement of resources or quantities in a system over time, ABM capture micro-level system behaviour, such as human decision-making and heterogeneous interactions between humans.

While use of SDM began in business management [ 6 , 7 ] it now has wide spread application from engineering to economics, from environmental science to waste and recycling research [ 8 , 9 , 10 , 11 , 12 , 13 ]. A SDM simulates the movement of entities in a system, using differential equations to model over time changes to system state variables. A stock and flow diagram can be used to provide a visual representation of a SDM, describing the relationships between system variables using stocks, rates and influencing factors. The diagram can be interpreted as mimicking the flow of water in and out of a bath tub [ 7 ]; the rates control how much ‘water’ (some quantifiable entity, resource) can leave or enter a ‘bath tub’ (a stock, system variable) which changes over time depending on what constraints or conditions (e.g. environmental or operational) are placed on the system. Often before the formulation of a stock and flow diagram, a causal loop diagram is constructed which can be thought of as a ‘mental model’ of the system [ 14 ], representing key dynamic hypotheses.

Agent-based model (ABM)

Unlike SDM, ABM is a ground-up representation of a system, simulating the changing states of individual ‘agents’ in a system rather than the broad entities or aggregate behaviour modelled in SDM. Aggregate system behaviour can however be inferred from ABM. Use of ABM to model system behaviour has been trans-disciplinary, with application in economics to ecology, from social sciences to engineering [ 15 , 16 , 17 , 18 , 19 ]. There can be multiple types of agent modelled, each assigned their own characteristics and pattern of behaviour [ 20 , 21 ]. Agents can learn from their own experiences, make decisions and perform actions based on set rules (e.g. heuristics), informed by their interactions with other agents, their own assigned attributes or based on their interaction with the modelled environment [ 22 ]. The interactions between agents can result in three levels of communication between agents; one-to-one communication between agents, one-to-many communication between agents and one-to-location communication where an agent can influence other agents contained in a particular location [ 22 ].

Why use SDM and ABM to model health systems?

ABM and SDM, with their ability to simulate micro- and macro-level behaviour, are complementary instruments for examining the mechanisms in complex systems and are being recognised as crucial tools for exploratory analysis. Their use in mapping health systems, for example, has steadily risen over the last three decades. ABM is well-suited to explore systems with dynamic patient or health worker activity, a limitation of other differential equation or event-based simulation tools [ 23 , 24 , 25 ]. Unlike discrete-event simulation (DES) for example, which simulates a queue of events and agent attributes over time [ 26 ], the agents modelled in ABM are decision makers rather than passive individuals. Closer to the true system modelled, ABM can also incorporate ongoing learning from events whereby patients can be influenced by their interactions with other patients or health workers and by their own personal experience with the health system [ 21 ]. SDM has also been identified as a useful tool for simulating feedback and activity across the care continuum [ 27 , 28 , 29 , 30 ] and is highly adept at capturing changes to the system over time [ 31 ]. This is not possible with certain ‘snapshot in time’ modelling approaches such as DES [ 32 ]. SDM is best implemented where the aim of the simulation is to examine aggregate flows, trends and sub-system behaviour as opposed to intricate individual flows of activity which are more suited to ABM or DES [ 33 ].

There are also models that can accommodate two or more types of simulation, known as hybrid models. Hybrid models produce results closer to true system behaviour by drawing on the strengths of one or more modelling methods while reducing the limitations associated with using a single simulation type [ 27 ]. The activity captured in such models emulates the individual variability of patients and health professionals while retaining the complex, aggregate behaviour exhibited in health systems.

Health scientists and policy makers alike have recognised the potential of using SDM and ABM to model all aspects of health systems in support of decision making from emergency department (ED) optimisation [ 34 ] to policies that support prevention or health promotion [ 35 ]. Before implementing or evaluating costly health policy interventions or health service re-structuring in the real world, modelling provides a relatively risk-free and low budget method of examining the likely impact of potential health system policy changes. They allow the simulation of ‘what if’ scenarios to optimise an intervention [ 36 ]. They can help identify sensitive parameters in the system that can impede the success of initiatives and point to possible spill-over effects of these initiatives to other departments, health workers or patients. Perhaps most important of all, these modelling methods allow researchers to produce simulations, results and a graphical-user interface in relation to alternative policy options that are communicable to stakeholders in the health system [ 37 ], those responsible for implementing system-wide initiatives and changes.

Study aim and objectives

Given the increasing amount of literature in this field, the main aim of the study was to examine and describe the use of SDM and ABM to model health systems. The specific objectives were as follows: (1) Determine the geographical, and healthcare settings in which these methods have been used (2) Identify the purpose of the research, particularly the health policies or interventions tested (3) Evaluate the limitations of these methods and study validation, and (4) Compare the use of SDM and ABM in health system research.

Although microsimulation, DES and Markov models have been widely used in disease health modelling and health economic evaluation, our aim in this study was to review the literature on mathematical methods which are used to model complex dynamic systems, SDM and ABM. These models represent two tenants of modelling: macroscopic (top-level) and microscopic (individual-level) approaches. Although microsimulation and DES are individual-based models like ABM, individuals in ABM are “active agents” i.e. decision-makers rather than “passive agents” which are the norm in microsimulation and DES models. Unlike Markov models which are essentially one-dimensional, unidirectional and linear, SDM are multi-dimensional, nonlinear with feedback mechanisms. We have therefore focussed our review on SDM and ABM because they are better suited to characterise the complexity of health systems. This study reviews the literature on the use of SDM and ABM in modelling health systems, and identifies and compares the key characteristics of both modelling approaches in unwrapping the complexity of health systems. In identifying and summarising this literature, this review will shed light on the types of health system research questions that these methods can be used to explore, and what they add to more traditional methods of health system research. By providing an over overview of how these models can be used within health system research, this paper is also expected to encourage wider use and uptake of these methods by health system researchers and policy makers.

The review was conducted in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement [ 38 ].

Search strategy and information sources

The literature on ABM and SDM of health systems has not been confined to a single research discipline, making it necessary to widen the systematic review to capture peer-reviewed articles found in mathematical, computing, medicine and health databases. Accordingly, we searched MEDLINE, EMBASE, Cochrane Library, MathSciNet, ACM Digital Library, HMIC, Econlit and Global Health databases for literature. The search of health system literature was narrowed to identify articles that were concerned with modelling facility-based healthcare, services and related healthcare financing agreements which had been excluded or were not the focus of previous reviews [ 34 , 35 , 39 , 40 , 41 ]. The search criteria used for MEDLINE was as follows, with full search terms for each database and search terms used to locate SDM and ABM literature found in Additional file  1 :

(health system* OR health care OR healthcare OR health service* OR health polic* OR health facil* OR primary care OR secondary care OR tertiary care OR hospital*).ab,ti. AND (agent-based OR agent based).ab,ti. AND (model*).ab,ti.

In addition, the reference list of papers retained in the final stage of the screening process, and systematic reviews identified in the search, were reviewed for relevant literature.

Data extraction and synthesis

The screening process for the review is given in Fig.  1 (adapted from [ 38 ]). All search results were uploaded to Mendeley reference software where duplicate entries were removed. The remaining records were screened using their titles and abstracts, removing entries based on eligibility criteria given in Table  1 . Post-abstract review, the full text of remaining articles was screened. Papers retained in final stage of screening were scrutinised, with data imported to Excel based on the following categories; publication date, geographical and healthcare setting modelled, purpose of research in addition to any policies or interventions tested, rationale for modelling method and software platform, validation and limitations of model. The results were synthesised using descriptive statistics and analysis of paper content that were used to answer the objectives.

a Flow-chart for systematic review of SDMs and b ABMs of health systems (Database research discipline is identified by colour; mathematical and computing (red), medicine (blue) and health (green) databases). Adapted from PRISMA [ 38 ]

The studies were first described by three characteristics: publication date, geographical setting, and what aspect of the health system was modelled and why. These characteristics were chosen for the following reasons. Publication date (Fig.  2 ) allows us to examine the quantity of SDM and ABM studies over time. Geographical settings (Fig. 2 , top) allows us to see which health systems have been studied, as health systems in LMIC are very different from those in developed countries. Studies are classified as modelling health systems in high, upper middle, lower middle and low income countries as classified by The World Bank based on economy, July 2018 [ 42 ]. Finally, we examined which aspects of the health system have been modelled and the types of research/policy questions that the models were designed to address, to shed light on the range of potential applications of these models, and also potential gaps in their application to date.

Number of articles in the final review by year of publication and economic classification

The analysis of paper content was split into three sections; SDM use in health system research (including hybrid SDM-DES), ABM use in health system research (including hybrid ABM-DES) and hybrid SDM-ABM use in health system research. The quality of selected studies will not be presented as our aim was to compare and summarise the application of SDM and ABM in modelling health systems rather than a quality appraisal of studies.

Study selection

The search initially yielded 535 citations for ABM and 996 citations for SDM of facility-based healthcare and services (see Fig. 1 ). Post-full text screening 11 ABM and 28 SDM papers were retained for analysis, six of which utilised hybrid modelling methods. Three of the hybrid modelling papers integrated SDM with DES [ 43 , 44 , 45 ], two integrated ABM with DES [ 24 , 46 ] and one integrated SDM with ABM [ 47 ]. A summary table of selected papers is given in Table  2 .

Descriptive statistics

Publication date.

The first SDM paper to model health systems was published in 1998 [ 56 ] whilst the first publication [ 66 ] utilising ABM came almost a decade later (Fig. 2 ). We found an increasing trend in publications for both modelling approaches, with 90.9% (10/11) and 71.4% (20/28) of all ABM and SDM articles, respectively, having been published in the last decade. The first hybrid modelling article was published in 2010 [ 43 ], using SDM and DES to model the impact of an intervention to aid access to social care services for elderly patients in Hampshire, England.

Geographical setting

The proportion of papers that modelled health systems in high, upper middle, lower middle and low income countries is presented in Fig. 2 . Eighteen (18/28) papers that employed SDM simulated health systems in high income countries including England [ 33 , 36 , 43 , 45 , 50 , 54 , 56 , 57 ] and Canada [ 28 , 51 , 62 ]. Four SDM papers simulated upper middle income country health systems, including Turkey [ 52 , 59 ] and China [ 64 ], with a nominal number of papers (5/28) focussing on lower middle or low income countries (West Bank and Gaza [ 48 , 55 ], Indonesia [ 37 ], Afghanistan [ 30 ] and Uganda [ 60 ]). Almost all ABM papers (9/11) modelled a high income country health system, including the US [ 20 , 23 , 25 ] and Austria [ 65 ]. Two (2/11) ABM papers described an upper-middle income based health system (Brazil [ 22 , 67 ]). All six articles that implemented a hybrid SDM or ABM simulated health systems based in high income countries, including Germany [ 44 ] and Poland [ 47 ].

Healthcare setting and purpose of research

The healthcare settings modelled in the SDM, ABM and hybrid simulation papers are presented in Fig.  3 . Healthcare settings modelled using SDM included systems that were concerned with delivering emergency or acute care (11/28) [ 28 , 31 , 36 , 45 , 47 , 50 , 56 , 57 , 58 , 61 , 62 ], elderly or long-term care services (LTC)(12/28) [ 28 , 31 , 36 , 43 , 44 , 45 , 49 , 50 , 51 , 54 , 61 , 62 ] and hospital waste management (4/28) [ 37 , 48 , 52 , 55 ]. Twenty of the SDM papers selected in this review assessed the impact of health policy or interventions on the modelled system. Common policy targets included finding robust methods to relieve stretched healthcare services, ward occupancy and patient length of stay [ 28 , 31 , 36 , 43 , 49 , 50 , 54 , 58 , 62 ], reducing the time to patient admission [ 33 , 53 , 61 ], targeting undesirable patient health outcomes [ 47 , 58 , 60 , 63 ], optimising performance-based incentive health system policies [ 30 , 59 ] and reducing the total cost of care [ 33 , 54 , 61 ]. The remaining eight papers explored factors leading to undesirable emergency care system behaviour [ 56 , 57 ], simulating hospital waste management systems and predicting future waste generation [ 37 , 48 , 55 ], estimating future demand for cardiac care [ 44 ], exploring the impact of patient admission on health professionals stress level in an integrated care system [ 45 ], and variation in physician decision-making [ 32 ].

The health system sector locations modelled in the SDM, ABM and hybrid modelling literature. Long-term care (LTC); Accountable care organisation (ACO); Maternal, newborn and child health (MNCH)

ABM papers modelled systems focussed on delivering emergency or acute care (4/11) [ 21 , 22 , 47 , 67 ] and accountable care organisations (ACO) or health insurance reimbursement schemes (3/11) [ 23 , 25 , 65 ]. Nine of the ABM papers assessed the impact of health policy or interventions on the modelled system. Common policy targets included decreasing the time agents spent performing tasks, waiting for a service or residing in parts of the system [ 20 , 22 , 24 , 67 ], reducing undesirable patient outcomes [ 23 , 25 , 47 , 67 ], reducing the number of patients who left a health facility without being seen by a physician [ 22 , 67 ] and optimising resource utility (beds and healthcare staff) [ 46 , 66 , 67 ]. The remaining two papers described simulation tools capable of comparing health insurance reimbursement schemes [ 65 ] and assessing risk, allocation of resources and identifying weaknesses in emergency care services [ 21 ].

Papers that utilised hybrid simulation, combining the strengths of two modelling approaches to capture detailed individual variability, agent-decision making and patient flow, modelled systems focussed on delivering elderly care or LTC services [ 43 , 44 , 45 ] and emergency or acute care [ 45 , 47 ]. Four of the hybrid simulation papers assessed the impact of policy or intervention on the modelled system. Policy targets included improving access to social support and care services [ 43 ], reducing undesirable patient outcomes [ 47 ], decreasing patient waiting time to be seen by a physician [ 24 ] and improving patient flow through the system by optimising resource allocation [ 46 ]. The remaining two papers used hybrid simulation to estimate the future demand for health care from patients with cardiac disease [ 44 ] and model patient flow through an integrated care system to estimate impact of patient admission on health care professionals wellbeing [ 45 ].

SDM use in health systems research (including hybrid SDM-DES)

Rationale for using model.

Gaining a holistic system perspective to facilitate the investigation of delays and bottlenecks in health facility processes, exploring counter-intuitive behaviour and monitoring inter-connected processes between sub-systems was cited frequently as reasons for using SDM to model health systems [ 28 , 36 , 37 , 48 , 56 ]. SDM was also described as a useful tool for predicting future health system behaviour and demand for care services, essential for health resource and capacity planning [ 48 , 60 ]. Configuration of the model was not limited by data availability [ 28 , 52 , 64 ] and could integrate data from various sources when required [ 51 ].

SDM was described as a tool for health policy exploration and optimising system interventions [ 33 , 36 , 51 , 54 , 58 , 64 ], useful for establishing clinical and financial ramifications on multiple groups (such as patients and health care providers) [ 63 ], identifying policy resistance or unintended system consequences [ 59 , 61 ] and quantifying the impact of change to the health system before real world implementation [ 62 ]. The modelling platform also provided health professionals, stakeholders and decision makers with an accessible visual learning environment that enabled engagement with experts necessary for model conception and validation [ 48 , 50 , 55 , 57 ]. The model interface could be utilised by decision makers to develop and test alternative policies in a ‘real-world’ framework that strengthened their understanding of system-wide policy impact [ 31 , 49 , 58 , 61 ].

SDM-DES hybrid models enabled retention of deterministic and stochastic system variability and preservation of unique and valuable features of both methods [ 44 ], capable of describing the flow of entities through a system and rapid insight without the need for large data collection [ 43 ], while simulating individual variability and detailed interactions that influence system behaviour [ 43 ]. SDM-DES offered dual model functionality [ 44 ] vital for simulating human-centric activity [ 45 ], reducing the practical limitations that come with using either SDM or DES to model health systems such as attempting to use SDM to model elements which have non-aggregated values (e.g. patient arrival time) [ 45 ] which is better suited for DES.

Healthcare setting

Sixteen papers that utilised SDM modelled systems that were concerned with the delivery of emergency or acute care, or elderly care or LTC services.

Ten of the reviewed papers primarily modelled sectors of the health system that delivered emergency or acute care Footnote 1 , Footnote 2 . Brailsford et al. [ 50 ], Lane et al. [ 56 ], Lane et al. [ 57 ] and Lattimer et al. [ 36 ] simulated the delivery of emergency care in English cities, specifically in Nottingham and London. Brailsford et al. [ 50 ] and Lattimer et al. [ 36 ] created models that replicated the entire emergency care system for the city of Nottingham, from primary care (i.e. General Practice surgeries) to secondary care (i.e. hospital admissions wards), to aid understanding of how emergency care was delivered and how the system would need to adapt to increasing demand. Lane et al. [ 56 ] and Lane et al. [ 57 ] modelled the behaviour of an ED in an inner-London teaching hospital, exploring the knock on effects of ED performance to hospital ward occupancy and elective admissions. Esensoy et al. [ 28 ] and Wong et al. [ 62 ] both modelled emergency care in Canada, Esensoy et al. [ 28 ] focussing on six sectors of the Ontario health system that cared for stroke patients while Wong et al. [ 62 ] simulated the impact of delayed transfer of General Internal Medicine patients on ED occupancy. Rashwan et al. [ 31 ], Walker et al. [ 61 ] and Mahmoudian-Dehkordi et al. [ 58 ] modelled patient flow through a generic emergency care facility with six possible discharge locations in Ireland, a sub-acute extended care hospital with patient flow from feeder facilities in Australia and an intensive care unit, ED and general wards in a generic facility.

Five of the SDM papers primarily simulated the behaviour of LTC facilities or care services for elderly patients Footnote 3 . Ansah et al. [ 49 ] modelled the demand and supply of general LTC services in Singapore with specific focus on the need for LTC and acute health care professionals. Desai et al. [ 54 ] developed a SDM that investigated future demand of care services for older people in Hampshire, England which simulated patient flow through adult social care services offering 13 different care packages. In modelling complex care service demand, Cepoiu-Martin et al. [ 51 ] explored patient flow within the Alberta continuing care system in Canada which offered supportive living and LTC services for patients with dementia. Brailsford et al. [ 43 ] used a hybrid SDM-DES model to investigate how local authorities could improve access to services and support for older people, in particular the long term impact of a new contact centre for patients. The SDM replicated the whole system for long term care, simulating the future demography and demand for care services and the nested DES model simulated the operational issues and staffing of the call centre in anticipation of growing demand for services. Zulkepli et al. [ 45 ] also used SDM-DES to model the behaviour of an integrated care system in the UK, modelling patient flow (DES) and intangible variables (SDM) related to health professionals such as motivation and stress levels.

Policy impact evaluation/testing

Twenty papers that utilised SDM tested the impact of policy or interventions on key health system performance or service indicators. The intended target of these policies ranged from relieving strained and under resourced healthcare services, decreasing healthcare costs to reducing patient mortality rates.

Ansah et al. [ 49 ], Brailsford et al. [ 50 ] and Desai et al. [ 54 ] aimed to reduce occupancy in acute or emergency care departments through policies that targeted elderly utilisation of these services. While demand for LTC services is expected to exponentially increase in Singapore, focus has been placed on expanding the acute care sector. Ansah et al. [ 49 ] simulated various LTC service expansion policies (static ‘current’ policy, slow adjustment, quick adjustment, proactive adjustment) and identified that proactive expansion of LTC services stemmed the number of acute care visits by elderly patients over time and required only a modest increase in the number of health professionals when compared with other policies. In Brailsford et al. [ 50 ] simulation of the entire emergency care system for Nottingham, England, policy testing indicated that while the emergency care system is operating near full capacity, yearly total occupancy of hospital beds could be reduced by re-directing emergency admissions from patients over 60 years of age (who make up around half of all admissions) to more appropriate services, such as those offered by community care facilities. To explore challenges that accompany providing care for an ageing population subject to budget restraints, Desai et al. [ 54 ] simulated the delivery and demand for social care services in Hampshire over a projected 5 year period. In offering care packages to only critical need clients and encouraging extra care services at home rather than offering residential care, the number of patients accessing acute care services reduced over the observed period.

Desai et al. [ 54 ], in addition to Taylor et al. [ 33 ] and Walker et al. [ 61 ], also examined policies that could reduce the total cost of care. Increasing the proportion of hired unqualified care workers (over qualified care workers who are employed at a higher cost rate) resulted in savings which could be fed back into care funding, although Desai et al. [ 54 ] remarked on the legal and practical limitations to this policy. Taylor et al. [ 33 ] examined the impact of shifting cardiac catheterization services from tertiary to secondary level hospitals for low risk investigations and explored how improvements could be made to services. Significant and stable improvements in service, including reduced waiting list and overall cost of service, were achieved with the implementation of strict (appropriate referral) guidelines for admitting patients. Walker et al. [ 61 ] modelled patient flow from feeder hospitals to a single sub-acute extended care facility in Victoria, Australia, to assess the impact of local rules used by the medical registrar for admission. The local admission policy which prioritised admissions from patients under the care of private doctors pushed the total cost of care over the facility budget by 6% whereas employing no prioritisation rule reduced the total cost of care to 3% under budget.

Semwanga et al. [ 60 ], Mahmoudian-Dehkordi et al. [ 58 ] and Worni et al. [ 63 ] evaluated the impact of health policy on undesirable patient outcomes (mortality and post-treatment complication rates). Semwanga et al. [ 60 ] tested the effectiveness of policies designed to promote maternal and neonatal care in Uganda, established from the literature. Policies that enabled service uptake, such as community health education, free delivery kits and motorcycle coupons were significant in reducing neonatal death over the simulated period. Mahmoudian-Dehkordi et al. [ 58 ] explored the intended and unintended consequences of intensive care unit resource and bed management policies on system performance indicators, including patient mortality. During a simulated crisis scenario, prioritising intensive care unit patient admission to general wards over emergency admissions was found to be the most effective policy in reducing total hospital mortality. Worni et al. [ 63 ] estimated the impact of a policy to reduce venous thromboembolism rates post-total knee arthroplasty surgery and identified unintentional consequences of the strategy. The policy prevented the reimbursement of patient care fees in the event that a patient was not taking the recommended prophylaxis medication and consequently develops venous thromboembolism. Simulation results indicated a positive 3-fold decrease in venous thromboembolism rates but an unintended 6-fold increase in the number of patients who develop bleeding complications as a result of compulsory prophylaxis treatment.

Validation (including sensitivity analysis)

Statistically-based models are usually used in quantitative data rich environments where model parameters are estimated through maximum likelihood or least-squares estimation methods. Bayesian methods can also be used to compare alternative statistical model structures. SDMs and ABMs on the other hand are not fitted to data observations in the traditional statistical sense. The data are used to inform model development. Both quantitative data and qualitative data (e.g. from interviews) can be used to inform the structure of the model and the parameters of the model. Furthermore, model structure and parameter values can also be elicited from expert opinion. This means that the nature of validation of ABMs and SDMs requires more scrutiny than that of other types of models.

With increasing complexity of such models, and to strengthen confidence in their use particularly for decision support, models are often subjected to sensitivity analysis and validation tests. Twenty-two papers that utilised SDM undertook model validation, the majority having performed behavioural validity tests (see Additional file  2 for details of validation methods for each model). Key model output such as bed occupancy [ 36 , 50 ], department length of stay [ 62 ] and number of department discharges [ 31 ] were compared with real system performance data from hospitals [ 32 , 33 , 36 , 48 , 50 , 54 , 58 , 59 , 61 , 62 ], local councils [ 54 ], nationally reported figs [ 31 , 64 ]. as well being reviewed by experts [ 57 , 60 ] as realistic. Others performed more structure orientated validity tests. Model conception [ 28 , 60 ], development [ 30 , 36 , 50 , 53 , 54 , 57 , 62 ] and formulation [ 54 , 56 , 59 ] were validated by a variety of experts including health professionals [ 47 , 53 , 54 , 57 , 59 , 62 ], community groups [ 56 ] and leaders [ 60 ], steering committees [ 36 ], hospital and care representatives [ 50 , 56 , 59 ], patient groups [ 60 ] and healthcare policy makers [ 60 ]. Further tests for structural validity included checking model behaviour when subjected to extreme conditions or extreme values of parameters [ 30 , 31 , 52 , 57 , 59 , 60 , 64 ], model dimensional consistency [ 31 , 52 , 57 , 59 , 60 ], model boundary adequacy [ 31 ] and mass balance [ 54 ] and integration error checks [ 31 , 52 ]. Sensitivity analysis was performed to assess how sensitive model output was to changes in key parameters [ 49 , 51 , 57 , 60 , 64 ], to test the impact of parameters that had been based on expert opinion on model output [ 28 ] and varying key system parameters to test the robustness and effectiveness of policies [ 28 , 30 , 52 , 53 , 58 ] (on the assumption of imperfect policy implementation [ 28 ]).

Limitations of research

Most of the model limitations reported were concerned with missing parameters, feedback or inability to simulate all possible future health system innovations. Mielczarek et al. [ 44 ], Cepoiu-Martin et al. [ 51 ], Ansah et al. [ 49 ] and Rashwan et al. [ 31 ] did not take into account how future improvements in technology or service delivery may have impacted results, such as the possibility of new treatment improving patient health outcomes [ 51 ] and how this could impact the future utilisation of acute care services [ 49 ]. Walker et al. [ 61 ] and Alonge et al. [ 30 ] described how the models may not simulate all possible actions or interactions that occurred in the real system, such as all proactive actions taken by hospital managers to achieve budget targets [ 61 ] or all unintended consequences of a policy on the system [ 30 ]. De Andrade et al. [ 53 ] and Rashwan et al. [ 31 ] discussed the reality of model boundaries, that SDMs cannot encapsulate all health sub-sector behaviour and spill-over effects. Although these have been listed here as limitations, not accounting for possible future improvements in healthcare service or not simulating all possible actions in the modelled system did not prevent authors from fulfilling study objectives. When developing a SDM, it is not possible to account for all possible spill-over effects to other healthcare departments and this should not be attempted; model boundaries are set to only include variables and feedback that are pertinent to exploring the defined problem.

Simplification of model parameters was another common limitation. Wong et al. [ 62 ] stated that this would result in some model behaviour not holding in the real system, such as using weekly hospital admission and discharge averages in place of hourly rates due to the hospital recording aggregated data. This aggregation of model parameters may not have reflected real system complexity; Eleyan et al. [ 55 ] did not differentiate between service level and type of hospital when modelling health care waste production (described as future work) and Worni et al. [ 63 ] refrained from stratifying post-surgery complications by severity, potentially combining lethal and less harmful complications within the same stock (although this did not detract from the study conclusion that the rate of complications would increase as a result of the tested policy).

Data availability, lack of costing analysis and short time horizons were also considered credible limitations. Models that had been calibrated with real data were at risk of using datasets that contained measurement errors or incomplete datasets lacking information required to inform model structure or feedback [ 32 ]. Routine facility data required for model conception and formulation was unavailable which restricted the replication of facility behaviour in the model [ 36 ] and restricted validation of model behaviour [ 59 ], although it should be noted that this is only one method among many for SDM validation and the author was able to use other sources of data for this purpose. Lack of costing or cost effectiveness analysis when testing policies [ 60 ], particularly policies that required significant investment or capacity expansion [ 58 ], limited discussion on their feasibility in the real system. Models that simulated events over short time scales did not evaluate long term patient outcomes [ 33 ] or the long term effects of facility policies on certain groups of patient [ 57 ].

ABM use in health system research (including hybrid ABM-DES)

The model’s ability to closely replicate human behaviour that exists in the real system was frequently cited [ 20 , 21 , 22 , 25 , 66 ], providing a deeper understanding of multiple agent decision-making [ 23 , 67 ], agent networks [ 25 ] and interactions [ 21 , 22 ]. The modelling method was described as providing a flexible framework capable of conveying intricate system structures [ 20 ], where simulations captured agent capacity for learning and adaptive behaviour [ 20 , 25 ] and could incorporate stochastic processes that mimicked agent transition between states [ 25 ]. ABM took advantage of key individual level agent data [ 25 ] and integrated information from various sources including demographic, epidemiological and health service data [ 65 ]. The visualisation of systems and interface available with ABM software packages facilitated stakeholder understanding of how tested policies could impact financial and patient health outcomes [ 23 ], particularly those experts in the health industry with minimal modelling experience [ 67 ].

Integrating DES and ABM within a single model ensured an intelligent and flexible approach for simulating complex systems, such as the outpatient clinic described in Kittipittayakorn et al. [ 24 ]. The hybrid model captured both orthopaedic patient flow and agent decision-making that enabled identification of health care bottlenecks and optimum resource allocation [ 24 ].

Seven papers that utilised ABM modelled systems that were either concerned with delivering emergency or acute care 2 , ACOs or health insurance reimbursement schemes.

Liu et al. [ 21 ] and Yousefi et al. [ 22 ] modelled behaviour in EDs in Spanish and Brazilian tertiary hospitals. Liu et al. [ 21 ] simulated the behaviour of eleven key agents in the ED including patients, admission staff, doctors, triage nurses and auxiliary staff. Patients were admitted to the ED and triaged before tests were requested and a diagnosis issued. Over time, agent states changed based on their interaction with other agents such as when a doctor decided upon a course of action for a patient (sending the patient home, to another ward, or continue with diagnosis and treatment). For further details of agent type and model rules for each paper, see Additional file  3 .

Yousefi et al. [ 22 ] modelled the activities of patients, doctors, nurses and receptionists in a ED. Agents could communicate with each other, to a group of other agents or could send a message to an area of the ED where other agents reside. They made decisions based on these interactions and the information available to them at the time. The main focus of the simulation was on patients who left the ED without being seen by a physician; patients decided whether to leave the ED based on a ‘tolerance’ time extracted from the literature, which changed based on their interaction with other agents. In an additional paper, Yousefi et al. [ 67 ] simulated decision-making by patients, doctors, nurses and lab technicians within a generic ED informed from the literature. Group decision-making was employed, whereby facility staff could interact with each other and reach a common solution for improving the efficacy of the department such as re-allocating staff where needed. Yousefi et al. [ 67 ], Yousefi et al. [ 22 ] and Liu et al. [ 21 ] each used a finite state machine (a computational model which describes an entity that can be in one of a finite number of states) to model interactions between agents and their states.

Liu et al. [ 25 ] and Alibrahim et al. [ 23 ] modelled the behaviour of patients, health providers and payers using series of conditional probabilities, where health providers had participated in an ACO in the United States. Liu et al. [ 25 ] presented a model where health providers within an ACO network worked together to reduce congestive heart failure patient healthcare costs and were consequently rewarded a portion of the savings from the payer agent (hypothetically, the Centers for Medicare and Medicaid Services). Patients were Medicare beneficiaries over the age of 65 who developed diabetes, hypertension and/or congestive heart failure and sought care within the network of health providers formed of three hospitals and 15 primary care physician clinics. Alibrahim et al. [ 23 ] adapted Liu et al. [ 25 ] ACO network model to allow patients to bypass their nearest medical provider in favour of an alternative provider. The decision for a patient to bypass their nearest health centre was influenced by patient characteristics, provider characteristics and the geographical distance between health providers. Providers were also given a choice on whether to participate in an ACO network, where they would then need to implement a comprehensive congestive heart failure disease management programme.

Einzinger et al. [ 65 ] created a tool that could be used to compare different health insurance reimbursement schemes in the Austrian health sector. The ABM utilised anonymous routine data from practically all persons with health insurance in Austria, pertaining to medical services accessed in the outpatient sector. In the simulation, patients developed a chronic medical issue (such as coronary heart disease) that required medical care and led to the patient conducting a search of medical providers through the health market. The patient then accessed care at their chosen provider where the reimbursement system, notified of the event via a generic interface, reimbursed the medical provider for patients care.

Nine papers tested the impact of policy on key health system performance or service indicators. The intended target of these policies ranged from decreasing patient length of stay, to reducing the number of patients who leave without being seen by a physician to reducing patient mortality and hospitalisation rates.

Huynh et al. [ 20 ], Yousefi et al. [ 22 ], Yousefi et al. [ 67 ] and Kittipittayakorn et al. [ 24 ] tested policies to reduce the time agents spent performing tasks, waiting for a service or residing in parts of the system. Huynh et al. [ 20 ] modelled the medication administration workflow for registered nurses at an anonymous medical centre in the United States and simulated changes to the workflow to improve medication administration safety. Two policies were tested; establishing a rigid order for tasks to be performed and for registered nurses to perform tasks in the most frequently observed order (observed in a real medical centre) to see if this improved the average amount of time spent on tasks. Yousefi et al. [ 67 ] modelled the effects of group decision-making in ED compared with the standard approach for resource allocation (where a single supervisor allocates resources) to assess which policy resulted in improved ED performance. Turning ‘on’ group decision-making and starting the simulation with a higher number of triage staff and receptionists resulted in the largest reduction of average patient length of stay and number of patients who left without being seen. This last performance indicator was the subject of an additional paper [ 22 ], with focus on patient-to-patient interactions and how this impacted their decision to leave the ED before being seen by a physician. Four policies adapted from case studies were simulated to reduce the number of patients leaving the ED without being seen and average patient length of stay. The policy of fast-tracking patients who were not acutely unwell during triage performed well as opposed to baseline, where acutely ill patients were always given priority. Kittipittayakorn et al. [ 24 ] used ABM-DES to identify optimal scheduling for appointments in an orthopaedic outpatient clinic, with average patient waiting time falling by 32% under the tested policy.

Liu et al. [ 25 ], Alibrahim et al. [ 23 ] and Yousefi et al. [ 67 ] tested the impact of health policy on undesirable patient outcomes (patient mortality and hospitalisation rates). Liu et al. [ 25 ] modelled health care providers who operated within an ACO network and outside of the network and compared patient outcomes. Providers who operated within the ACO network worked together to reduce congestive heart failure patient healthcare costs and were then rewarded with a portion of the savings. As part of their membership, providers implemented evidence-based interventions for patients, including comprehensive discharge planning with post-discharge follow-up; this intervention was identified in the literature as key to reducing congestive heart failure patient hospitalisation and mortality, leading to a reduction in patient care fees without compromising the quality of care. The ACO network performed well, with a 10% reduction observed in hospitalisation compared with the standard care network. In another study [ 23 ] six scenarios were simulated with combinations of patient bypass capability (turned “on” or “off”) and provider participation in the ACO network (no ACO present, optional participation in ACO or compulsory participation in ACO). Provider participation in the ACO, in agreement with Liu et al. [ 25 ], led to reduced mortality and congestive heart failure patient hospitalisation, with patient bypass capability marginally increasing provider ACO participation. Yousefi et al. [ 67 ] also modelled the impact of group decision-making in ED on the number of patient deaths and number of wrong discharges i.e. patients sent to the wrong sector for care after triage and are then discharged before receiving correct treatment.

Nine of the 11 papers that utilised ABM undertook model validation, consisting almost exclusively of behavioural validity tests. Model output, such as patient length of stay and mortality rates, was reviewed by health professionals [ 46 , 66 ] and compared with data extracted from pilot studies [ 20 ], health facilities (historical) [ 22 , 24 , 46 , 65 , 66 ], national health surveys [ 65 ] and relevant literature [ 23 , 25 ]. Papers presented the results of tests to determine the equivalence of variance [ 20 ] and difference in mean [ 20 , 24 ] between model output and real data. Structural validity tests included extreme condition testing [ 23 , 46 ] and engaging health care experts to ensure the accuracy of model framework [ 22 , 47 ]. Sensitivity analysis was performed to determine how variations or uncertainty in key parameters (particularly where they had not been derived from historical or care data [ 65 ]) affected model outcomes [ 23 , 25 ].

The majority of model limitations reported were concerned the use or availability of real system or case data. Huynh et al. [ 20 ], Yousefi et al. [ 67 ] and Liu et al. [ 25 ] formulated their models using data that was obtainable, such as limited sample data extracted from a pilot study [ 20 ], national average trends [ 25 ] and data from previous studies [ 67 ]. Yousefi et al. [ 22 ] case study dataset did not contain key system feedback, such as the tolerance time of patients waiting to be seen by a physician in the ED, although authors were able to extract this data from a comparable study identified in the literature.

Missing model feedback or parameters, strict model boundaries and simplification of system elements were also considered limitations. Huynh et al. [ 20 ], Hutzschenreuter et al. [ 66 ] and Einzinger et al. [ 65 ] did not model all the realistic complexities of their system, such as all possible interruptions to tasks that occur in patient care units [ 20 ], patient satisfaction of admission processes [ 66 ] (which will be addressed in future work), how treatment influences the course of disease or that morbid patients are at higher risk of developing co-morbidity than healthier patients, which would affect the service needs and consumption needs of the patient [ 65 ]. To improve the accuracy of the model, Huynh et al. stated that further research is taking place to obtain real, clinical data (as opposed to clinical simulation lab results) to assess the impact of interruptions on workflow. Liu et al.’s [ 21 ] model boundary did not include other hospital units that may have been affected by ED behaviour and they identify this as future work, for example to include hospital wards that are affected by ED behaviour. Alibrahim et al. [ 23 ] and Einzinger et al. [ 65 ] made simplifications to the health providers and networks that were modelled, such as assuming equal geographical distances and identical care services between health providers in observed networks [ 23 ], limiting the number of factors that influenced a patients decision to bypass their nearest health provider [ 65 ] and not simulating changes to health provider behaviour based on service utilisation or reimbursement scheme in place [ 23 ]. Alibrahim et al. [ 23 ] noted that although the model was constrained by such assumptions, the focus of future work would be to improve the capability of the model to accurately study the impact of patient choice on economic, health and health provider outcomes.

SDM-ABM use in health system research

A single paper used hybrid SDM-ABM to model health system behaviour. Djanatliev et al. [ 47 ] developed a tool that could be used to assess the impact of new health technology on performance indicators such as patient health and projected cost of care. A modelling method that could reproduce detailed, high granularity system elements in addition to abstract, aggregate health system variables was sought and a hybrid SDM-ABM was selected. The tool nested an agent-based human decision-making module (regarding healthcare choices) within a system dynamics environment, simulating macro-level behaviour such as health care financing and population dynamics. A case study was presented to show the potential impact of Mobile Stroke Units (MSU) on patient morbidity in Berlin, where stroke diagnosis and therapy could be initiated quickly as opposed to standard care. The model structure was deemed credible after evaluation by experts, including doctors and health economists.

Comparison of SDM and ABM papers

The similarities and differences among the SDM and ABM body of literature are described in this section and shown in Table  3 . A high proportion of papers across both modelling methods simulated systems that were concerned with emergency or acute care. A high number of SDM papers (11/28) simulated patient flow and pathways through emergency care [ 28 , 31 , 36 , 45 , 47 , 50 , 56 , 57 , 58 , 61 , 62 ] with a subset evaluating the impact of policies that relieved pressure on at capacity ED’s [ 28 , 36 , 50 , 58 , 62 ]. ABM papers simulated micro-level behaviour associated with emergency care, such as health professional and patient behaviour in EDs and what impact agent interactions have on actions taken over time [ 21 , 22 , 47 , 67 ]. ACOs and health insurance reimbursement schemes, a common modelled healthcare setting among the ABM papers [ 23 , 25 , 65 ] was the focus of a single SDM paper [ 63 ] while health care waste management, a popular healthcare setting for SDM application [ 37 , 48 , 52 , 55 ] was entirely absent among the selected ABM literature. SDM and ABM were both used to test the impact of policy on undesirable patient outcomes, including patient mortality [ 23 , 25 , 58 , 60 , 67 ] and hospitalisation rates [ 23 , 25 ]. Interventions for reducing patient waiting time for services [ 24 , 33 , 53 , 61 , 67 ] and patient length of stay [ 22 , 31 , 67 ] were also tested using these methods, while policy exploration to reduce the total cost of care was more frequent among SDM studies [ 33 , 54 , 61 ].

SDM and ABM software platforms provide accessible, user-friendly visualisations of systems that enable engagement with health experts necessary for model validation [ 48 , 50 , 55 , 57 ] and facilitate stakeholder understanding of how alternative policies can impact health system performance under a range conditions [ 31 , 49 , 58 , 61 ]. The ability to integrate information and data from various sources was also cited as rationale for using SDM and ABM [ 51 ]. Reasons for using SDM to model health systems, as opposed to other methods, included gaining a whole-system perspective crucial for investigating undesirable or counter-intuitive system behaviour across sub-systems [ 28 , 36 , 37 , 48 , 56 ] and identifying unintended consequences or policy resistance with tested health policies [ 59 , 61 ]. The ability to replicate human behaviour [ 20 , 21 , 22 , 25 , 66 ] and capacity for learning and adaptive behaviour [ 20 , 25 ] was frequently cited as rationale for using ABM to simulate health systems.

Validation of SDMs and ABMs consisted mostly of behavioural validity tests where model output was reviewed by experts and compared to real system performance data or to relevant literature. Structural validity tests were uncommon among ABM papers while expert consultation on model development [ 30 , 36 , 50 , 53 , 54 , 57 , 62 , 63 ], extreme condition [ 30 , 31 , 52 , 57 , 59 , 60 , 64 ] and dimensional consistency tests [ 31 , 52 , 57 , 59 , 60 ] were frequently reported in the SDM literature. The inability to simulate all actions or interactions that occur in the real system [ 20 , 30 , 61 , 65 , 66 ] and simplification of model parameters [ 23 , 55 , 62 , 63 , 65 ] were described as limitations in both SDM and ABM papers. Data availability for model conception and formulation [ 20 , 22 , 25 , 32 , 36 , 67 ] and the impact of model boundaries (restricting exploration of interconnected sub-system behaviour [ 21 , 31 , 53 ]) were also cited limitations common to both sets of literature. Lack of costing analysis [ 58 , 60 ], short time horizons [ 33 , 57 ] and an inability to model future improvements in technology or service delivery [ 31 , 44 , 49 , 51 ] were additionally cited among the SDM papers.

Statement of principal findings

Our review has confirmed that there is a growing body of research demonstrating the use of SDM and ABM to model health care systems to inform policy in a range of settings. While the application of SDM has been more widespread (with 28 papers identified) there are also a growing number of ABM being used (11), just over half of which used hybrid simulation. A single paper used hybrid SDM-ABM to model health system behaviour. To our knowledge this is the first review to identify and compare the application of both SDM and ABM to model health systems. The first ABM article identified in this review was published almost a decade after the first SDM paper; this reflects to a certain extent the increasing availability of SDM and ABM dedicated software tools with the developments in ABM software lagging behind their SDM modelling counterparts.

Emergency and acute care, and elderly care and LTC services were the most frequently simulated health system setting. Both sets of services are facing exponential increases in demand with constraints on resources, presenting complex issues ideal for evaluation through simulation. Models were used to explore the impact and potential spill over effects of alternative policy options, prior to implementation, on patient outcomes, service use and efficiency under various structural and financial constraints.

Strengths and weaknesses of the study

To ensure key papers were identified, eight databases across four research areas were screened for relevant literature. Unlike other reviews in the field [ 39 , 40 ], there was no restriction placed on publication date. The framework for this review was built to provide a general overview of the SDM and ABM of healthcare literature, capturing papers excluded in other published reviews as a result of strict inclusion criteria. These include reviews that have focussed specifically on compiling examples of modelled health policy application in the literature [ 35 ] or have searched for papers with a particular health system setting, such as those that solely simulate the behaviour of emergency departments [ 34 ]. One particularly comprehensive review of the literature had excluded papers that simulated hospital systems, which we have explicitly included as part of our search framework [ 39 ].

The papers presented in this review, with selection restricted by search criteria, provide a broad picture of the current health system modelling landscape. The focus of this review was to identify models of facility-based healthcare, purposely excluding literature where the primary focus is on modelling disease progression, disease transmission or physiological disorders which can be found in other reviews such as Chang et al. [ 39 ] and Long et al. [ 41 ]. The data sources or details of how data was used to conceptualise and formulate models are not presented in this paper; this could on its own be the focus of another study and we hope to publish these results as future work. This information would be useful for researchers who want to gain an understanding of the type and format of data used to model health systems and best practice for developing and validating such models.

Literature that was not reported in English was excluded from the review which may have resulted in a small proportion of relevant papers being missed. Papers that described DES models, the other popular modelling method for simulating health system processes, were not included in this review (unless DES methods are presented as part of a hybrid model integrated with SDM or ABM) but have been compiled elsewhere [ 68 , 69 , 70 ]. Finally, the quality of the papers was not assessed.

Implications for future research

A nominal number of SDM papers (9/28), an even lower proportion of ABM papers (2/11) and none of the hybrid methods papers simulated health systems based in low- or middle-income countries (LMICs). The lower number of counterpart models in LMICs can be attributed to a lack of capacity in modelling methods and perhaps the perceived scarcity of suitable data; however, the rich quantitative and qualitative primary data collected in these countries for other types of evaluation could be used to develop such models. Building capacity for using these modelling methods in LMICs should be a priority and generating knowledge of how and which secondary data to use in these settings for this purpose. In this review, we observed that it is feasible to use SDM to model low-income country health systems, including those in Uganda [ 60 ] and Afghanistan [ 30 ]. The need to increase the use of these methods within LMICs is paramount; even in cases where there is an absence of sufficient data, models can be formulated for LMICs and used to inform on key data requirements through sensitivity analysis, considering the resource and healthcare delivery constraints experienced by facilities in these settings. This research is vital for our understanding of health system functioning in LMICs, and given the greater resource constraints, to allow stakeholders and researchers to assess the likely impact of policies or interventions before their costly implementation, and to shed light on optimised programme design.

Health system professionals can learn greatly from using modelling tools, such as ABM, SDM and hybrid models, developed originally in non-health disciplines to understand complex dynamic systems. Understanding the complexity of health systems therefore require collaboration between health scientists and scientists from other disciplines such as engineering, mathematics and computer science. Discussion and application of hybrid models is not a new phenomenon in other fields but their utilisation in exploring health systems is still novel; the earliest article documenting their use in this review was published in 2010 [ 43 ]. Five of the six hybrid modelling papers [ 43 , 44 , 45 , 46 , 47 ] were published as conference proceedings (the exception Kittipittayakorn et al. [ 24 ]), demonstrating the need to include conference articles in systematic reviews of the literature in order to capture new and evolving applications of modelling for health systems research.

The configuration and extent to which two distinct types of models are combined has been described in the literature [ 71 , 72 , 73 , 74 , 75 ]. The hybrid modelling papers selected in this review follow what is described as ‘hierarchical’ or ‘process environment’ model structures, the former where two distinct models pass information to each other and the latter where one model simulates system processes within the environment of another model [ 72 ]. Truly ‘integrated’ models, considered the ‘holy grail’ [ 43 ] of hybrid simulation, where elements of the system are simulated by both methods of modelling with no clear distinction, were not identified in this review and in the wider literature remain an elusive target. In a recent review of hybrid modelling in operational research only four papers were identified to have implemented truly integrated hybrid simulation and all used bespoke software, unrestricted by the current hybrid modelling environments [ 76 ].

Of the six hybrid modelling papers, only Djanatliev et al. [ 47 ] presented a model capable of both ABM and SDM simulation. The crucial macro- and micro- level activity captured in such models represent feedback in the wider, complex system while retaining the variable behaviour exhibited by those who access or deliver healthcare. With increasing software innovation and growing demand for multi-method modelling in not only in healthcare research but in the wider research community, we need to increase their application to modelling health systems and progress towards the ‘holy grail’ of hybrid modelling.

We identified 28 papers using SDM methods and 11 papers using ABM methods to model health system behaviour, six of which implemented hybrid model structures with only a single paper using SDM-ABM. Emergency and acute care, and elderly care and LTC services were the most frequently simulated health system settings, modelling the impact of health policies and interventions targeting at-capacity healthcare services, patient length of stay in healthcare facilities and undesirable patient outcomes. A high proportion of articles modelled health systems in high income countries; future work should now turn to modelling healthcare settings in LMIC to support policy makers and health system researchers alike. The utilisation of hybrid models in healthcare is still relatively new but with an increasing demand to develop models that can simulate the macro- and micro-level activity exhibited by health systems, we will see an increase in their use in the future.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.

One of the elderly or LTC services papers also modelled emergency or acute care but it was not the primary focus and is therefore not discussed here.

The single SDM-ABM paper that modelled the delivery of emergency or acute care is discussed in section ‘SDM-ABM use in health system research’.

Six of the emergency or acute care review papers and one of the cardiology care papers also modelled elderly or LTC services but it was not the primary focus and are therefore not discussed here.

Abbreviations

Accountable care organisation

Agent-based model

Discrete-event simulation

Emergency Department

Long-term care

Low- and middle-income countries

System dynamics model

Kitson A, Brook A, Harvey G, Jordan Z, Marshall R, O’Shea R, et al. Using Complexity and Network Concepts to Inform Healthcare Knowledge Translation. Int J Heal Policy Manag. 2017;7:231–43. https://doi.org/10.15171/ijhpm.2017.79 .

Article   Google Scholar  

Paina L, Peters DH. Understanding pathways for scaling up health services through the lens of complex adaptive systems. Health Policy Plan. 2012;27:365–73. https://doi.org/10.1093/heapol/czr054 .

Article   PubMed   Google Scholar  

Lipsitz LA. Understanding health care as a complex system: the foundation for unintended consequences. JAMA. 2012;308:243–4. https://doi.org/10.1001/jama.2012.7551 .

Article   PubMed   PubMed Central   CAS   Google Scholar  

Adam T, de Savigny D. Systems thinking for strengthening health systems in LMICs: need for a paradigm shift. Health Policy Plan. 2012;27:iv1–3. https://doi.org/10.1093/heapol/czs084 .

de Savigny D, Blanchet K, Adam T. Applied systems thinking for health systems research : a methodological handbook: McGraw-Hill Education; 2017.

Shepherd SP. A review of system dynamics models applied in transportation. Transp B Transp Dyn. 2014;2:83–105. https://doi.org/10.1080/21680566.2014.916236 .

Sterman JD. Business dynamics: systems thinking and modeling for a complex world: McGraw-Hill Companies Inc; 2000.

Kunc M, Mortenson MJ, Vidgen R. A computational literature review of the field of system dynamics from 1974 to 2017. J Simul. 2018;12:115–27. https://doi.org/10.1080/17477778.2018.1468950 .

System Dynamics for Engineering Students. Elsevier; 2018. doi: https://doi.org/10.1016/C2011-0-05346-2 .

Radzicki MJ. System Dynamics and Its Contribution to Economics and Economic Modeling. Encycl. Complex. Syst. Sci. New York: Springer New York; 2009. p. 8990–9000. https://doi.org/10.1007/978-0-387-30440-3_539 .

Book   Google Scholar  

Ford A. Global warming and system dynamics. Int Conf Syst Dyn Soc. 2007.

Fiddaman TS. Exploring policy options with a behavioral climate-economy model. Syst Dyn Rev. 2002;18:243–67. https://doi.org/10.1002/sdr.241 .

Popli K, Sudibya GL, Kim S. A review of solid waste management using system dynamics modeling. J Environ Sci Int. 2017;26:1185–200. https://doi.org/10.5322/JESI.2017.26.10.1185 .

Groesser SN, Schaffernicht M. Mental models of dynamic systems: taking stock and looking ahead. Syst Dyn Rev. 2012;28:46–68. https://doi.org/10.1002/sdr.476 .

Epstein JM. Generative social science: studies in agent-based computational modeling: STU-Stud. Princeton University Press; 2006.

Wilensky U, Rand W. An Introduction to Agent-Based Modeling: Mit Press; 2015.

Miller JH, Page SE. Complex Adaptive Systems: An Introduction to Computational Models of Social Life: STU-Stud. Princeton University Press; 2007.

Railsback SF, Grimm V. Agent-Based and Individual-Based Modeling: Princeton University Press; 2011.

Abar S, Theodoropoulos G, Lemarinier P, O’Hare G. Agent based Modelling and simulation tools: a review of the state-of-art software. Comput Sci Rev. 2017;24:13–33. https://doi.org/10.1016/j.cosrev.2017.03.001 .

Huynh N, Snyder R, Vidal J, Tavakoli A, Cai B. Application of computer simulation modeling to medication administration process redesign. J Healthc Eng. 2012;3:649–62. https://doi.org/10.1260/2040-2295.3.4.649 .

Liu Z, Cabrera E, Rexachs D, Luque E. A generalized agent-based model to simulate emergency departments. Sixth Int Conf Adv Syst Simul. 2014:65–70.

Yousefi M, Yousefi M, Fogliatto FS, Ferreira RPM, Kim JH. Simulating the behavior of patients who leave a public hospital emergency department without being seen by a physician: a cellular automaton and agent-based framework. Brazilian J Med Biol Res. 2018;51:e6961. https://doi.org/10.1590/1414-431X20176961 .

Alibrahim A, Wu S. An agent-based simulation model of patient choice of health care providers in accountable care organizations. Health Care Manag Sci. 2018;21:131–43. https://doi.org/10.1007/s10729-016-9383-1 .

Kittipittayakorn C, Ying K-C. Using the integration of discrete event and agent-based simulation to enhance outpatient service quality in an orthopedic department. J Healthc Eng. 2016;2016:4189206. https://doi.org/10.1155/2016/4189206 .

Article   PubMed Central   Google Scholar  

Liu P, Wu S. An agent-based simulation model to study accountable care organizations. Health Care Manag Sci. 2016;19:89–101. https://doi.org/10.1007/s10729-014-9279-x .

Allen M, Spencer A, Gibson A, Matthews J, Allwood A, Prosser S, et al. Right cot, right place, right time: improving the design and organisation of neonatal care networks – a computer simulation study. Heal Serv Deliv Res. 2015;3:9. https://doi.org/10.3310/hsdr03200 .

Brailsford SC. Tutorial: Advances and challenges in healthcare simulation modeling: 2007 Winter Simul. Conf., IEEE; 2007. p. 1436–48. https://doi.org/10.1109/WSC.2007.4419754 .

Esensoy AV, Carter MW. High-Fidelity whole-system patient flow modeling to assess health care transformation policies. Eur J Oper Res. 2018;266:221–37. https://doi.org/10.1016/j.ejor.2017.09.019 .

Homer JB, Hirsch GB. System dynamics modeling for public health: background and opportunities. Am J Public Health. 2006;96:452–8. https://doi.org/10.2105/AJPH.2005.062059 .

Article   PubMed   PubMed Central   Google Scholar  

Alonge O, Lin S, Igusa T, Peters DH. Improving health systems performance in low- and middle-income countries: a system dynamics model of the pay-for-performance initiative in Afghanistan. Health Policy Plan. 2017;32:1417–26. https://doi.org/10.1093/heapol/czx122 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Rashwan W, Abo-Hamad W, Arisha A. A system dynamics view of the acute bed blockage problem in the Irish healthcare system. Eur J Oper Res. 2015;247:276–93. https://doi.org/10.1016/j.ejor.2015.05.043 .

Ghaffarzadegan N, Epstein AJ, Martin EG. Practice variation, bias, and experiential learning in cesarean delivery: A data-based system dynamics approach. Health Serv Res. 2013;48:713–34. https://doi.org/10.1111/1475-6773.12040 .

Taylor K, Dangerfield B. Modelling the feedback effects of reconfiguring health services. J Oper Res Soc. 2005;56:659–75. https://doi.org/10.1057/palgrave.jors.2601862 .

Mohiuddin S, Busby J, Savovic J, Richards A, Northstone K, Hollingworth W, et al. Patient flow within UK emergency departments: a systematic review of the use of computer simulation modelling methods. BMJ Open. 2017;7:e015007. https://doi.org/10.1136/bmjopen-2016-015007 .

Atkinson J-A, Wells R, Page A, Dominello A, Haines M, Wilson A. Applications of system dynamics modelling to support health policy. Public Heal Res Pract. 2015;25. https://doi.org/10.17061/phrp2531531 .

Lattimer V, Brailsford S, Turnbull J, Tarnaras P, Smith H, George S, et al. Reviewing emergency care systems I: insights from system dynamics modelling. Emerg Med J. 2004;21:685–91. https://doi.org/10.1136/emj.2002.003673 .

Chaerul M, Tanaka M, Shekdar AV. A system dynamics approach for hospital waste management. Waste Manag. 2008;28:442–9. https://doi.org/10.1016/j.wasman.2007.01.007 .

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535. https://doi.org/10.1136/bmj.b2535 .

Chang AY, Ogbuoji O, Atun R, Verguet S. Dynamic modeling approaches to characterize the functioning of health systems: A systematic review of the literature. Soc Sci Med. 2017;194:160–7. https://doi.org/10.1016/j.socscimed.2017.09.005 .

Rusoja E, Haynie D, Sievers J, Mustafee N, Nelson F, Reynolds M, et al. Thinking about complexity in health: a systematic review of the key systems thinking and complexity ideas in health. J Eval Clin Pract. 2018;24:600–6. https://doi.org/10.1111/jep.12856 .

Long KM, Meadows GN. Simulation modelling in mental health: a systematic review. J Simul. 2017. https://doi.org/10.1057/s41273-017-0062-0 .

Banco Mundial. World Bank: country and lending groups - DataBank. World Bank Gr. 2018.

Brailsford SC, Desai SM, Viana J. Towards the holy grail: Combining system dynamics and discrete-event simulation in healthcare: Proc. 2010 Winter Simul. Conf., IEEE; 2010. p. 2293–303. https://doi.org/10.1109/WSC.2010.5678927 .

Mielczarek B, Zabawa J. Modeling Healthcare Demand Using a Hybrid Simulation Approach, vol. 2016: Proc. 2016 Winter Simul. Conf., IEEE Press. p. 1535–46.

Zulkepli J. Hybrid simulation for modelling large systems: an example of integrated care model: Proc. 2012 Winter Simul. Conf; 2012.

Viana J, Simonsen TB, Dahl FA, Flo K. A Hybrid Discrete Event Agent Based Overdue Pregnancy Outpatient Clinic Simulation Model. Proc. 2018 Winter Simul. Conf. Piscataway: IEEE Press; 2018. p. 1488–99.

Google Scholar  

Djanatliev A, German R, Kolominsky-Rabas P, Hofmann BM. Hybrid simulation with loosely coupled system dynamics and agent-based models for prospective health technology assessments: Proc. Winter Simul. Conf., Winter Simulation Conference; 2012. p. 69:1–69:12.

Al-Khatib IA, Eleyan D, Garfield J. A system dynamics approach for hospital waste management in a city in a developing country: the case of Nablus, Palestine. Environ Monit Assess. 2016;188:503. https://doi.org/10.1007/s10661-016-5487-9 .

Ansah JP, Eberlein RL, Love SR, Bautista MA, Thompson JP, Malhotra R, et al. Implications of long-term care capacity response policies for an aging population: a simulation analysis. Health Policy. 2014;116:105–13. https://doi.org/10.1016/j.healthpol.2014.01.006 .

Brailsford SC. Emergency and on-demand health care: modelling a large complex system. J Oper Res Soc. 2004;55.

Cepoiu-Martin M, Bischak DP. Policy choices in dementia care - an exploratory analysis of the Alberta Continuing Care System (ACCS) using system dynamics. Spec Issue Complex Forum “real” World Heal Syst Implic Complex. Theor Sci. 2018;24:278–84.

Ciplak N, Barton JR. A system dynamics approach for healthcare waste management: A case study in Istanbul Metropolitan City, Turkey. Waste Manag Res. 2012;30:576–86. https://doi.org/10.1177/0734242X12443405 .

de Andrade L, Lynch C, Carvalho E, Rodrigues CG, Vissoci JRN, Passos GF, et al. System dynamics modeling in the evaluation of delays of care in ST-segment elevation myocardial infarction patients within a tiered health system. PLoS One. 2014;9:e103577. https://doi.org/10.1371/journal.pone.0103577 .

Desai MS, Penn ML, Brailsford S, Chipulu M. Modelling of Hampshire adult services—gearing up for future demands. Health Care Manag Sci. 2008;11:167–76. https://doi.org/10.1007/s10729-007-9049-0 .

Eleyan D, Al-Khatib IA, Garfield J. System dynamics model for hospital waste characterization and generation in developing countries. Waste Manag Res. 2013;31:986–95. https://doi.org/10.1177/0734242X13490981 .

Lane D. Emergency but no accident: a system dynamics study of casualty waiting times in the British NHS. Eurohealth (Lond). 1998;4.

Lane DC. Looking in the wrong place for healthcare improvements: a system dynamics study of an accident and emergency department. J Oper Res Soc. 2000;51.

Mahmoudian-Dehkordi A, Sadat S. Sustaining critical care: using evidence-based simulation to evaluate ICU management policies. Health Care Manag Sci. 2017;20:532–47. https://doi.org/10.1007/s10729-016-9369-z .

Meker T, Barlas Y. Dynamic consequences of performance-based payment Systems in Public Hospitals. Syst Res Behav Sci. 2015;32:459–80. https://doi.org/10.1002/sres.2338 .

Semwanga AR, Nakubulwa S, Adam T. Applying a system dynamics modelling approach to explore policy options for improving neonatal health in Uganda. Heal Res Policy Syst. 2016;14:35. https://doi.org/10.1186/s12961-016-0101-8 .

Walker BC, Haslett T. The dynamics of local rules in hospital admission processes. Aust Health Rev. 2003;26:98–106.

Wong HJ, Wu RC, Caesar M, Abrams H, Morra D. Smoothing inpatient discharges decreases emergency department congestion: a system dynamics simulation model. Emerg Med J. 2010;27:593–8. https://doi.org/10.1136/emj.2009.078543 .

Worni M, Pietrobon R, Zammar GR, Shah J, Yoo B, Maldonato M, et al. System dynamics to model the unintended consequences of denying payment for venous thromboembolism after total knee arthroplasty. PLoS One. 2012;7:e30578. https://doi.org/10.1371/journal.pone.0030578 .

Yu W, Li M, Ge Y, Li L, Zhang Y, Liu Y, et al. Transformation of potential medical demand in China: A system dynamics simulation model. J Biomed Inform. 2015;57:399–414. https://doi.org/10.1016/j.jbi.2015.08.015 .

Einzinger P, Popper N, Breitenecker F, Pfeffer N, Jung R, Endel G. The GAP-DRG Model: Simulation of Outpatient Care for Comparison of Different Reimbursement Schemes. Proc. 2013 Winter Simul. Conf. Simul. Mak. Decis. A complex world. Piscataway: IEEE Press; 2013. p. 2299–308.

Hutzschenreuter AK, Bosman PAN, Blonk-Altena I, van Aarle J, La Poutré H. Agent-based patient admission scheduling in hospitals. Belgian/Netherlands Artif Intell Conf. 2008:315–6. https://doi.org/10.1007/3-540-32062-8 .

Yousefi M, Ferreira R. An agent-based simulation combined with group decision-making technique for improving the performance of an emergency department. Brazilian J Med Biol Res. 2017;50:1–10. https://doi.org/10.1590/1414-431X20175955 .

Günal MM, Pidd M. Discrete event simulation for performance modelling in health care: a review of the literature. J Simul. 2010;4:42–51. https://doi.org/10.1057/jos.2009.25 .

Salleh S, Thokala P, Brennan A, Hughes R, Dixon S. Discrete event simulation-based resource Modelling in health technology assessment. Pharmacoeconomics. 2017;35:989–1006. https://doi.org/10.1007/s40273-017-0533-1 .

Zhang X. Application of discrete event simulation in health care: a systematic review. BMC Health Serv Res. 2018;18:687. https://doi.org/10.1186/s12913-018-3456-4 .

Bennett PG. On linking approaches to decision-aiding: issues and prospects. J Oper Res Soc. 1985;36:659. https://doi.org/10.2307/2582261 .

Chahal K, Eldabi T. Applicability of hybrid simulation to different modes of governance in UK healthcare: 2008 Winter Simul. Conf., IEEE; 2008. p. 1469–77. https://doi.org/10.1109/WSC.2008.4736226 .

Lättilä L, Hilletofth P, Lin B. Hybrid simulation models – when, why, how? Expert Syst Appl. 2010;37:7969–75. https://doi.org/10.1016/j.eswa.2010.04.039 .

Brailsford SC. Hybrid simulation in healthcare: New concepts and new tools: 2015 Winter Simul. Conf., IEEE; 2015. p. 1645–53. https://doi.org/10.1109/WSC.2015.7408284 .

Morgan JS, Howick S, Belton V. A toolkit of designs for mixing discrete event simulation and system dynamics. Eur J Oper Res. 2017;257:907–18. https://doi.org/10.1016/j.ejor.2016.08.016 .

Brailsford SC, Eldabi T, Kunc M, Mustafee N, Osorio AF. Hybrid simulation modelling in operational research: a state-of-the-art review. Eur J Oper Res. 2018. https://doi.org/10.1016/j.ejor.2018.10.025 .

Download references

Acknowledgements

Not applicable.

The work described in this paper was funded by the Health Systems Research Initiative (HSRI). MRC Grant Reference Number: MR/R013454/1.

Author information

Authors and affiliations.

Department of Global Health and Development, London School of Hygiene and Tropical Medicine, 15-17 Tavistock Place, London, WC1H 9SH, UK

Rachel Cassidy, Neha S. Singh, Josephine Borghi & Karl Blanchet

Department of Mathematics, University College London, London, UK

Pierre-Raphaël Schiratti

Sia Partners UK, London, UK

Information Systems Department, College of Computing and Information Sciences, Makerere University, P.O. Box 7062, Kampala, Uganda

Agnes Semwanga

Ifakara Health Institute, PO Box 78373, Dar es Salaam, Tanzania

Peter Binyaruka

Department of Gender Studies, School of Humanities and Social Sciences, University of Zambia, 10101, Lusaka, Zambia

Nkenda Sachingongu

Economic and Business Research Programme, University of Zambia, Institute of Economic and Social Research, P O Box 30900, 10101, Lusaka, Zambia

Chitalu Miriam Chama-Chiliba

Department of Public Health, Environments and Society, London School of Hygiene and Tropical, London, UK

Zaid Chalabi

You can also search for this author in PubMed   Google Scholar

Contributions

RC, KB, ZC specified the search criteria and selection of databases. RC screened titles, evaluated full text articles and was responsible for full text extraction. RC is lead author, NSS, KB, ZC, PS, JB, AS, PB, CC, NS provided guidance on the draft and final manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Rachel Cassidy .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1..

Search criteria used for each database.

Additional file 2.

Descriptive table of validation methods used in SDM and ABM literature.

Additional file 3.

Descriptive table of ABM model rules.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Cassidy, R., Singh, N.S., Schiratti, PR. et al. Mathematical modelling for health systems research: a systematic review of system dynamics and agent-based models. BMC Health Serv Res 19 , 845 (2019). https://doi.org/10.1186/s12913-019-4627-7

Download citation

Received : 14 June 2019

Accepted : 11 October 2019

Published : 19 November 2019

DOI : https://doi.org/10.1186/s12913-019-4627-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• System dynamics
• Agent-based
• Health systems
• Systematic review

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

AI-Based Modeling: Techniques, Applications and Research Issues Towards Automation, Intelligent and Smart Systems

Iqbal h. sarker.

1 Swinburne University of Technology, Melbourne, VIC 3122 Australia

2 Department of Computer Science and Engineering, Chittagong University of Engineering & Technology, Chittagong, 4349 Bangladesh

Artificial intelligence (AI) is a leading technology of the current age of the Fourth Industrial Revolution (Industry 4.0 or 4IR), with the capability of incorporating human behavior and intelligence into machines or systems. Thus, AI-based modeling is the key to build automated, intelligent, and smart systems according to today’s needs. To solve real-world issues, various types of AI such as analytical, functional, interactive, textual, and visual AI can be applied to enhance the intelligence and capabilities of an application. However, developing an effective AI model is a challenging task due to the dynamic nature and variation in real-world problems and data. In this paper, we present a comprehensive view on “AI-based Modeling” with the principles and capabilities of potential AI techniques that can play an important role in developing intelligent and smart systems in various real-world application areas including business, finance, healthcare, agriculture, smart cities, cybersecurity and many more. We also emphasize and highlight the research issues within the scope of our study. Overall, the goal of this paper is to provide a broad overview of AI-based modeling that can be used as a reference guide by academics and industry people as well as decision-makers in various real-world scenarios and application domains.

Introduction

Nowadays, we live in a technological age, the Fourth Industrial Revolution, known as Industry 4.0 or 4IR [ 59 , 91 ], which envisions fast change in technology, industries, societal patterns, and processes as a consequence of enhanced interconnectivity and smart automation. This revolution is impacting almost every industry in every country and causing a tremendous change in a non-linear manner at an unprecedented rate, with implications for all disciplines, industries, and economies. Three key terms Automation , i.e., reducing human interaction in operations, Intelligent , i.e., ability to extract insights or usable knowledge from data, and Smart computing , i.e., self-monitoring, analyzing, and reporting, known as self-awareness, have become fundamental criteria in designing today’s applications and systems in every sector of our lives since the current world is more reliant on technology than ever before. The use of modern smart technologies enables making smarter, faster decisions regarding the business process, ultimately increasing the productivity and profitability of the overall operation, where Artificial Intelligence (AI) is known as a leading technology in the area. The AI revolution, like earlier industrial revolutions that launched massive economic activity in manufacturing, commerce, transportation, and other areas, has the potential to lead the way of progress. As a result, the impact of AI on the fourth industrial revolution motivates us to focus briefly on “ AI-based modeling ” in this paper.

Artificial intelligence (AI) is a broad field of computer science concerned with building smart machines capable of performing tasks that typically require human intelligence. In other words, we can say that it aims is to make computers smart and intelligent by giving them the ability to think and learn using computer programs or machines, i.e., can think and function in the same way that people do. From a philosophical perspective, AI has the potential to help people live more meaningful lives without having to work as hard, as well as manage the massive network of interconnected individuals, businesses, states, and nations in a way that benefits everyone. Thus, the primary goal of AI is to enable computers and machines to perform cognitive functions such as problem-solving, decision making, perception, and comprehension of human communication. Therefore, AI-based modeling is the key to building automated, intelligent and smart systems according to today’s needs, which has emerged as the next major technological milestone, influencing the future of practically every business by making every process better, faster, and more precise.

While today’s Fourth Industrial Revolution is typically focusing on technology-driven “automation, intelligent and smart systems”, AI technology has become one of the core technologies to achieve the goal. However, developing an effective AI model is a challenging task due to the dynamic nature and variation in real-world problems and data. Thus, we take into account several AI categories: The first one is “ Analytical AI ” with the capability of extracting insights from data to ultimately produce recommendations and thus contributing to data-driven decision-making; the Second one is “ Functional AI ” which is similar to analytical AI; however, instead of giving recommendations, it takes actions; the Third one is “ Interactive AI ” that typically allows businesses to automate communication without compromising on interactivity like smart personal assistants or chatbots; the Fourth one is “ Textual AI ” that covers textual analytics or natural language processing through which business can enjoy text recognition, speech-to-text conversion, machine translation, and content generation capabilities; and finally the Fifth one is “ Visual AI ” that covers computer vision or augmented reality fields, discussed briefly in “Why artificial intelligence in today’s research and applications?”.

Although the area of “artificial intelligence” is huge, we mainly focus on potential techniques towards solving real-world issues, where the results are used to build automated, intelligent, and smart systems in various application areas. To build AI-based models, we classify various AI techniques into ten categories: (1) machine learning; (2) neural networks and deep learning; (3) data mining, knowledge discovery and advanced analytics; (4) rule-based modeling and decision-making; (5) fuzzy logic-based approach; (6) knowledge representation, uncertainty reasoning, and expert system modeling; (7) case-based reasoning; (8) text mining and natural language processing; (9) visual analytics, computer vision and pattern recognition; (10) hybridization, searching and Optimization. These techniques can play an important role in developing intelligent and smart systems in various real-world application areas that include business, finance, healthcare, agriculture, smart cities, cybersecurity, and many more, depending on the nature of the problem and target solution. Thus, it is important to comprehend the concepts of these techniques mentioned above, as well as their relevance in a variety of real-world scenarios, discussed briefly in “ Potential AI techniques ”.

Based on the importance and capabilities of AI techniques, in this paper, we give a comprehensive view on “AI-based modeling” that can play a key role towards automation, intelligent and smart systems according to today’s needs. Thus, the key focus is to explain the principles of various AI techniques and their applicability to the advancement of computing and decision-making to meet the requirements of the Fourth Industrial Revolution. Therefore the purpose of this paper is to provide a fundamental guide for those academics and industry professionals who want to study, research, and develop automated, intelligent, and smart systems based on artificial intelligence techniques in relevant application domains.

The main contributions of this paper are therefore listed as follows:

• To define the scope of our study in terms of automation, intelligent and smart computing, and decision-making in the context of today’s real-world needs.
• To explore various types of AI that includes analytical, functional, interactive, textual, and visual AI, to understand the theme of the power of artificial intelligence in computing and decision-making while solving various problems in today’s Fourth Industrial Revolution.
• To provide a comprehensive view on AI techniques that can be applied to build an AI-based model to enhance the intelligence and capabilities of a real-world application.
• To discuss the applicability of AI-based solutions in various real-world application domains to assist developers as well as researchers in broadening their perspectives on AI techniques.
• To highlight and summarize the potential research issues within the scope of our study for conducting future research, system development and improvement.

The rest of the paper is organized as follows. The next section provides a background highlighting why artificial intelligence is in today’s research and application. In the subsequent section, we discuss and summarize how various AI techniques can be used for intelligence modeling in various application areas. Next, we summarize various real-world application areas, where AI techniques can be employed to build automated, intelligent, and smart systems. The impact and future aspect of AI highlighting research issues have been presented in the penultimate section, and the final section concludes this paper.

Why Artificial Intelligence in Today’s Research and Applications?

In this section, our goal is to motivate the study of various AI techniques that can be applied in various application areas in today’s interconnected world. For this, we explore Industry 4.0 and the revolution of AI, types of AI techniques, as well as the relation with the most prominent machine and deep learning techniques. Hence, the scope of our study in terms of research and applications is also explored through our discussion.

Industry 4.0 and the Revolution of AI

We are now in the age of the 4th Industrial Revolution, referred to as Industry 4.0 [ 59 , 91 ], which represents a new era of innovation in technology, particularly, AI-driven technology. After the Internet and mobile Internet sparked the 3rd Industrial Revolution, AI technologies, fueled by data, are now creating an atmosphere of Industry 4.0. The term “Industry 4.0” typically refers to the present trend of leveraging modern technology to automate processes and exchange information. In a broad sense, Industry 4.0 has been defined as “A term used to describe the present trend of industrial technology automation and data exchange, which includes cyber-physical systems, the Internet of Things, cloud computing, and cognitive computing, as well as the development of the smart factory”. The digital revolution to Industry 4.0 begins with data collection, followed by artificial intelligence to interpret the data. Thus, the term “Intelligence Revolution” can be considered in the context of computing and services as the world is being reshaped by AI that incorporates human behavior and intelligence into machines or systems.

AI is the buzzword these days as it is going to impact businesses of all shapes and sizes, across all industries. Existing products or services can be enhanced by industrial AI to make them more effective, reliable, and safe. For example, computer vision is used in the automotive industry to avoid collisions and allow vehicles to stay in their lane, making driving safer. The world’s most powerful nations are hurrying to jump on the AI bandwagon and are increasing their investments in the field. Similarly, the largest and most powerful corporations are working hard to build ground-breaking AI solutions that will put them ahead of the competition. As a result, its impact may be observed in practically every area including homes, businesses, hospitals, cities, and the virtual world, as summarized in “ Real-World Applications of AI ”.

Understanding Various Types of Artificial Intelligence

Artificial intelligence (AI) is primarily concerned with comprehending and carrying out intelligent tasks such as thinking, acquiring new abilities, and adapting to new contexts and challenges. AI is thus considered a branch of science and engineering that focuses on simulating a wide range of issues and functions in the field of human intellect. However, due to the dynamic nature and diversity of real-world situations and data, building an effective AI model is a challenging task. Thus, to solve various issues in today’s Fourth Industrial Revolution, we explore various types of AI that include analytical, functional, interactive, textual, and visual, to understand the theme of the power of AI, as shown in Fig.  1 . In the following, we define the scope of each category in terms of computing and real-world services.

• Analytical AI: Analytics typically refers to the process of identifying, interpreting, and communicating meaningful patterns of data. Thus, Analytical AI aims to discover new insights, patterns, and relationships or dependencies in data and to assist in data-driven decision-making. Therefore, in the domain of today’s business intelligence, it becomes a core part of AI that can provide insights to an enterprise and generate suggestions or recommendations through its analytical processing capability. Various machine learning [ 81 ] and deep learning [ 80 ] techniques can be used to build an analytical AI model to solve a particular real-world problem. For instance, to assess business risk, a data-driven analytical model can be used.
• Functional AI: Functional AI works similarly to analytical AI because it also explores massive quantities of data for patterns and dependencies. Functional AI, on the other hand, executes actions rather than making recommendations. For instance, a functional AI model could be useful in robotics and IoT applications to take immediate actions.
• Interactive AI: Interactive AI typically enables efficient and interactive communication automation, which is well established in many aspects of our daily lives, particularly in the commercial sphere. For instance, to build chatbots and smart personal assistants an interactive AI model could be useful. While building an interactive AI model, a variety of techniques such as machine learning, frequent pattern mining, reasoning, AI heuristic search can be employed.
• Textual AI: Textual AI typically covers textual analytics or natural language processing through which businesses can enjoy text recognition, speech-to-text conversion, machine translation as well as content generation capabilities. For instance, an enterprise may use textual AI to support an internal corporate knowledge repository to provide relevant services, e.g., answering consumers’ queries.
• Visual AI: Visual AI is typically capable to recognize, classify, and sorting items, as well as converting images and videos into insights. Thus, visual AI can be considered as a branch of computer science that trains machines to learn images and visual data in the same manner that humans do. This sort of AI is often used in fields such as computer vision and augmented reality.

As discussed above, each of the AI types has the potential to provide solutions to various real-world problems. However, to provide solutions by taking into account the target applications, various AI techniques and their combinations that include machine learning, deep learning, advanced analytics, knowledge discovery, reasoning, searching, and relevant others can be used, discussed briefly in “ Potential AI techniques ”. As most of the real-world issues need advanced analytics [ 79 ] to provide an intelligent and smart solution according to today’s needs, analytical AI that uses machine learning (ML) and deep learning (DL) techniques can play a key role in the area of AI-powered computing and system.

Various types of artificial intelligence (AI) considering the variations of real-world issues

The Relation of AI with ML and DL

Artificial intelligence (AI), machine learning (ML), and deep learning (DL) are three prominent terminologies used interchangeably nowadays to represent intelligent systems or software. The position of machine learning and deep learning within the artificial intelligence field is depicted in Fig.  2 . According to Fig.  2 , DL is a subset of ML which is also a subset of AI. In general, AI [ 77 ] combines human behavior and intelligence into machines or systems, whereas ML is a way of learning from data or experience [ 81 ], which automates analytical model building. Deep learning [ 80 ] also refers to data-driven learning approaches that use multi-layer neural networks and processing to compute. In the deep learning approach, the term “Deep” refers to the concept of numerous levels or stages through which data is processed to develop a data-driven model.

An illustration of the position of machine learning (ML) and deep Learning (DL) within the area of artificial intelligence (AI)

Thus, both ML and DL can be considered as essential AI technologies, as well as a frontier for AI that can be used to develop intelligent systems and automate processes. It also takes AI to a new level, termed “Smarter AI” with data-driven learning. There is a significant relationship with “Data Science” [ 79 ] as well because both ML and DL can learn from data. These learning methods can also play a crucial role in advanced analytics and intelligent decision-making in data science, which typically refers to the complete process of extracting insights in data in a certain problem domain. Overall, we can conclude that both ML and DL technologies have the potential to transform the current world, particularly in terms of a powerful computational engine, and to contribute to technology-driven automation, smart and intelligent systems. In addition to these learning techniques, several others can play the role in the development of AI-based models in various real-world application areas, depending on the nature of the problem and the target solution, discussed briefly in “  Potential AI techniques ”.

Potential AI Techniques

In this section, we briefly discuss the principles and capabilities of potential AI techniques that can be used in developing intelligent and smart systems in various real-world application areas. For this we divide AI techniques into ten potential categories by taking into account various types of AI, mentioned in earlier “ Why artificial intelligence in today’s research and applications? ”. Followings are the ten categories of AI techniques that can play a key role in automation, intelligent, and smart computer systems, depending on the nature of the problem.

Machine Learning

Machine learning (ML) is known as one of the most promising AI technologies, which is typically the study of computer algorithms that automate analytical model building [ 81 ]. ML models are often made up of a set of rules, procedures, or sophisticated “transfer functions” that can be used to discover interesting data patterns or anticipate behavior [ 23 ]. Machine learning is also known as predictive analytics that makes predictions about certain unknowns in the future through the use of data and is used to solve many real-world business issues, e.g., business risk prediction. In Fig.  3 , a general framework of a machine learning-based predictive model is depicted, where the model is trained from historical data in phase 1 and the outcome is generated for new test data in phase 2. For modeling in a particular problem domain, different types of machine learning techniques can be used according to their learning principles and capabilities, as discussed below.

A general structure of a machine learning based predictive model considering both the training and testing phase

• Supervised learning This is performed when particular goals are specified to be achieved from a set of inputs, i.e., a ‘task-driven strategy’ that uses labeled data to train algorithms to classify data or forecast outcomes, for example—detecting spam-like emails. The two most common supervised learning tasks are classification (predicting a label) and regression (predicting a quantity) analysis, discussed briefly in our earlier paper Sarker et al. [ 81 ]. Navies Bayes [ 42 ], K-nearest neighbors [ 4 ], Support vector machines [ 46 ], Decision Trees - ID3 [ 71 ], C4.5 [ 72 ], CART [ 15 ], BehavDT [ 84 ], IntrudTree [ 82 ], Ensemble learning, Random Forest [ 14 ], Linear regression [ 36 ], Support vector regression [ 46 ], etc. [ 81 ] are the popular techniques that can be used to solve various supervised learning tasks, according to the nature of the given data in a particular problem domain. For instance, to detect various types of cyber-attacks the classification models could be useful, while cyber-crime trend analysis or estimating the financial loss in the domain of cybersecurity, a regression model could be useful, which enables enterprises to assess and manage their cyber-risk.
• Unsupervised learning This is referred to as a ‘data-driven method’, in which the primary goal is to uncover patterns, structures, or knowledge from unlabeled data. Clustering, visualization, dimensionality reduction, finding association rules, and anomaly detection are some of the most common unsupervised tasks, discussed briefly in our earlier paper Sarker et al. [ 81 ]. The popular techniques for solving unsupervised learning tasks are clustering algorithms such as K-means [ 55 ], K-Mediods [ 64 ], CLARA [ 45 ], DBSCAN [ 27 ], hierarchical clustering, single linkage [ 92 ] or complete linkage [ 93 ], BOTS [ 86 ], association learning algorithms such as AIS [ 2 ], Apriori [ 3 ], Apriori-TID and Apriori-Hybrid [ 3 ], FP-Tree [ 37 ], and RARM [ 18 ], Eclat [ 105 ], ABC-RuleMiner [ 88 ] as well as feature selection and extracting techniques like Pearson Correlation [ 81 ], principal component analysis [ 40 , 66 ], etc. that can be used to solve various unsupervised learning-related tasks, according to the nature of the data. An unsupervised clustering model, for example, could be useful in customer segmentation or identifying different consumer groups around which to build marketing or other business strategies.
• Other learning techniques In addition to particular supervised and unsupervised tasks, semi-supervised learning can be regarded as a hybridization of both techniques explained above, as it uses both labeled and unlabeled data to train a model. It could be effective for improving model performance when data must be labeled automatically without human interaction. For instance, classifying Internet content or texts, a semi-supervised learning model could be useful. Reinforcement learning is another type of machine learning training strategy that rewards desired behaviors while punishing unwanted ones. A reinforcement learning agent, in general, is capable of perceiving and interpreting its surroundings, taking actions, and learning through trial and error, i.e., an environment-driven approach, in which the environment is typically modeled as a Markov decision process and decisions are made using a reward function [ 10 ]. Monte Carlo learning, Q-learning, Deep Q Networks, are the most common reinforcement learning algorithms [ 43 ]. Trajectory optimization, motion planning, dynamic pathing, and scenario-based learning policies for highways are some of the autonomous driving activities where reinforcement learning could be used.

Overall, machine learning modeling [ 81 ] has been employed in practically every aspect of our lives, including healthcare, cybersecurity, business, education, virtual assistance, recommendation systems, smart cities, and many more. Blumenstock et al. [ 12 ], for example, provides a machine learning strategy for getting COVID-19 assistance to people who need it the most. Sarker et al. highlight numerous sorts of cyber anomalies and attacks that can be detected using machine learning approaches in the domain of cybersecurity [ 78 , 89 ]. Saharan et al. [ 76 ] describe a machine-learning-based strategy to develop an effective smart parking pricing system for smart city environments. In our earlier paper, Sarker et al. [ 81 ] we briefly discussed various types of machine learning techniques including clustering, feature learning, classification, regression, association analysis, etc. highlighting their working principles, learning capabilities, and real-world applications. In Table  1 , we have outlined the above-mentioned machine learning techniques, emphasizing model building procedures and tasks. Overall, machine learning algorithms can build a model based on training data of a particular problem domain, to make predictions or decisions without having to be explicitly programmed to do so. Thus, we can conclude that machine learning approaches can play a crucial part in the development of useful models in a variety of application areas, based on their learning capabilities and the nature of the data, and the desired outcome.

Various types of machine learning techniques with examples

Neural Networks and Deep Learning

Deep learning (DL) [ 80 ] is known as another popular AI technique, which is based on artificial neural networks (ANN). Nowadays, DL has become a hot topic in the computing world due to its layer-wise learning capability from data. Multiple hidden layers, including input and output layers, make up a typical deep neural network. Figure  4 shows a general structure of a deep neural network ( h i d d e n l a y e r = N and N ≥ 2) comparing with a shallow network ( h i d d e n l a y e r = 1 ). DL techniques can be divided into three major categories, highlighted in our earlier paper Sarker et al. [ 80 ]. These are as below:

• Deep networks for supervised or discriminative learning In supervised or classification applications, this type of DL approach is used to provide a discriminative function. Discriminative deep architectures are often designed to provide pattern categorization discrimination by characterizing the posterior distributions of classes conditioned on observable data [ 20 ]. Multi-layer perceptron (MLP) [ 67 ], Convolutional neural networks (CNN or ConvNet) [ 53 ], Recurrent neural networks (RNN) [ 24 , 57 ], and their variants can be used to build the deep discriminative learning models to solve the relevant real-world issues.
• Deep networks for unsupervised or generative learning This category of deep learning approaches is commonly used to identify high-order correlation qualities or features for pattern analysis or synthesis, as well as the joint statistical distributions of visible data and their associated classes [ 20 ]. The key notion of generative deep architectures is that specific supervisory information, such as target class labels, is unimportant throughout the learning process. Techniques in this category are mostly employed for unsupervised learning, as they are commonly used for feature learning or data generation and representation [ 19 , 20 ]. Thus, generative modeling can also be utilized as a preprocessing step for supervised learning tasks, ensuring discriminative model accuracy. The Generative Adversarial Network (GAN) [ 32 ], Autoencoder (AE) [ 31 ], Restricted Boltzmann Machine (RBM) [ 58 ], Self-Organizing Map (SOM) [ 50 ], and Deep Belief Network (DBN) [ 39 ], as well as their variants, can be used to build the deep generative learning models to solve the relevant real-world issues.
• Deep networks for hybrid learning Generative models are versatile, learning from both labeled and unlabeled data. In contrast, discriminative models are unable to learn from unlabeled data yet outperform their generative versions in supervised tasks. Hybrid networks are motivated by a paradigm for simultaneously training deep generative and discriminative models. Multiple (two or more) deep basic learning models make up hybrid deep learning models, with the basic model being the discriminative or generative deep learning model outlined previously. For instance, a generative model followed by a discriminative model, or an integration of a generative or discriminative model followed by a non-deep learning classifier, may be effective for tackling real-world problems.

Figure  5 shows a taxonomy of these DL techniques that can be employed in many application areas including healthcare, cybersecurity, business, virtual help, smart cities, visual analytics, and many more. For example, Aslan et al. [ 9 ] offer a CNN-based transfer learning strategy for COVID-19 infection detection. Islam et al. [ 41 ] describes a combined deep CNN-LSTM network for the identification of novel coronavirus (COVID-19) using X-ray images. Using transferable generative adversarial networks built on deep autoencoders, Kim et al. [ 48 ] propose a method for detecting zero-day malware. Anuradha et al. [ 8 ] propose a deep CNN-based stock trend prediction utilizing a reinforcement-LSTM model based on large data. Wang et al. [ 100 ] offer a real-time collision prediction technique for intelligent transportation systems based on deep learning. Dhyani et al. [ 22 ] proposed an intelligent Chatbot utilizing deep learning with Bidirectional RNN and attention model. Overall, deep learning approaches can play a crucial role in the development of effective AI models in a variety of application areas, based on their learning capabilities and the nature of the data, and the target outcome.

A general architecture of a a shallow network with one hidden layer and b a deep neural network with multiple hidden layers

A taxonomy of DL techniques [ 80 ], broadly divided into three major categories (1) deep networks for supervised or discriminative learning, (2) deep networks for unsupervised or generative learning, and (3) deep networks for hybrid learning and relevant others

Data Mining, Knowledge Discovery and Advanced Analytics

Over the last decade, data mining has been a common word that is interchangeable with terms like knowledge mining from data, knowledge extraction, knowledge discovery from data (KDD), data or pattern analysis, etc. [ 79 ]. Figure  6 shows a general procedure of the knowledge discovery process. According to Han et al. [ 36 ], the term “knowledge mining from data” should have been used instead. Data mining is described as the process of extracting useful patterns and knowledge from huge volumes of data [ 36 ], which is related to another popular term “Data Science” [ 79 ]. Data science is typically defined as a concept that unites statistics, data analysis, and related methodologies to analyze and investigate realities through data.

A general procedure of the knowledge discovery process

In the area of data analytics, several key questions such as “What happened?”, “Why did it happen?”, “What will happen in the future?”, “What action should be taken?” are common and important [ 79 ]. Based on these questions, four types of analytics such as descriptive, diagnostic, predictive, and prescriptive analytics are highlighted below, which can be used to build the corresponding data-driven models.

• Descriptive analytics It is the analysis of historical data to have a better understanding of how a business has changed. Thus, descriptive analytics answers the question, “What happened in the past?” by describing historical data such as sales and operations statistics, marketing tactics, social media usage, etc.
• Diagnostic analytics It is a type of sophisticated analytics that explores data or content to figure out “Why did it happen?” The purpose of diagnostic analytics is to assist in the discovery of the problem’s root cause.
• Predictive analytics This type of advanced analytics typically explores data to answer the question, “What will happen in the future?” Thus, the primary purpose of predictive analytics is to identify and, in most cases, answer this question with a high degree of confidence.
• Prescriptive analytics This focuses on advising the optimal course of action based on data to maximize the total outcomes and profitability, answering the question, “What action should be taken?”

To summarize, both descriptive and diagnostic analytics examine the past to determine what happened and why it happened. Predictive and prescriptive analytics employ historical data to foresee what will happen in the future and what actions should be made to mitigate such impacts. For a clear understanding, Table  2 shows a summary of these analytics that are applied in various application areas. For example, Hamed et al. [ 35 ] build decision support systems in Arabic higher education institutions using data mining and business intelligence. Alazab et al. [ 5 ] provide a data mining strategy to maximize the competitive advantage on E-business websites. From logs to stories, Afzaliseresht et al. [ 1 ] provide human-centered data mining for cyber threat information. Poort et al. [ 70 ] have described an automated diagnostic analytics workflow for the detection of production events-application to mature gas fields. Srinivas et al. [ 94 ] provide a prescriptive analytics framework for optimizing outpatient appointment systems using machine learning algorithms and scheduling rules. Thus, we can conclude data mining and analytics can play a crucial part to build AI models through the extracted insights from the data.

Various types of analytical methods with examples

Rule-Based Modeling and Decision-Making

Typically, a rule-based system is used to store and modify knowledge to understand data in a meaningful way. A rule base is a sort of knowledge base that has a list of rules. In most cases, rules are written as IF-THEN statements of the form:

IF < a n t e c e d e n t > THEN < c o n s e q u e n t >

Such an IF-THEN rule-based expert system model can have the decision-making ability of a human expert in an intelligent system designed to solve complex problems and knowledge reasoning [ 85 ]. The reason is that the rules in professional frameworks are easily understood by humans and are capable of representing relevant knowledge clearly and effectively. Furthermore, rule-based models may be quickly improved according to the demands by adding, deleting, or updating rules based on domain expert information, or recency, i.e. based on recent trends [ 83 ].

Previously, the term “rule-based system” was used to describe systems that used rule sets that were handcrafted or created by humans. However, rule-based machine learning approaches could be more effective in terms of automation and intelligence, which include mainly classification and association rule learning techniques [ 85 ]. Several popular classification techniques such as decision trees [ 72 ], IntrudTree [ 82 ], BehavDT [ 84 ], Ripple Down Rule learner (RIDOR) [ 101 ], Repeated Incremental Pruning to Produce Error Reduction (RIPPER) [ 102 ], etc. exist with the ability of rule generation. Based on support and confidence value, association rules are built by searching for frequent IF-THEN pattern data. Common association rule learning techniques such as AIS [ 2 ], Apriori [ 3 ], FP-Tree [ 37 ], RARM [ 18 ], Eclat [ 105 ], ABC-RuleMiner [ 88 ], and others can be used to build a rule-based model utilizing a given data set. Sarker et al. [ 88 ], for example, provide a rule-based machine learning strategy for context-aware intelligent and adaptive mobile services. Borah et al. [ 13 ] propose a method for employing dynamic rare association rule mining to find risk variables for unfavorable illnesses. Using case-based clustering and weighted association rule mining, Bhavithra et al. [ 11 ] offer a personalized web page suggestion. Xu et al. [ 103 ] introduced a risk prediction and early warning system for air traffic controllers’ risky behaviors utilizing association rule mining and random forest. Thus, we can conclude that rule-based modeling can play a significant role to build AI models as well as intelligent decision-making in various application areas to solve real-world issues.

Fuzzy Logic-Based Approach

Fuzzy logic is a precise logic of imprecision and approximate reasoning [ 104 ]. This is a natural generalization of standard logic in which a concept’s degree of truth, also known as membership value or degree of membership, can range from 0.0 to 1.0. Standard logic only applies to concepts that are either completely true, i.e., degree of truth 1.0, or completely false, i.e., degree of truth 0.0. Fuzzy logic, on the other hand, has been used to deal with the concept of partial truth, in which the truth value may range from completely true to completely false, such as 0.9 or 0.5. For instance, “if x is very large, do y; if x is not very large, do z”. Here the boundaries of very big and not too big may overlap, i.e. fuzzy. As a result, fuzzy logic-based models can recognize, represent, manipulate, understand, and use data and information that are vague and uncertain [ 104 ]. Figure  7 shows a general architecture of a fuzzy logic system. It typically has four parts as below:

• Fuzzification It transforms inputs, i.e. crisp numbers into fuzzy sets.
• Knowledge-base It contains the set of rules and the IF-THEN conditions provided by the experts to govern the decision-making system, based on linguistic information.
• Inference engine It determines the matching degree of the current fuzzy input concerning each rule and decides which rules are to be fired according to the input field. Next, the fired rules are combined to form the control actions.
• Defuzzification It transforms the fuzzy sets obtained by the inference engine into a crisp value.

Although machine learning models are capable of differentiating between two (or more) object classes based on their ability to learn from data, the fuzzy logic approach is preferred when distinguishing features are vaguely defined and rely on human expertise and knowledge. Thus, the system may work with any type of input data, including imprecise, distorted, or noisy data, as well as with limited data. It is a suitable strategy to use in scenarios with real, continuous-valued elements because it uses data acquired in surroundings with such properties [ 34 ]. Fuzzy logic-based models are used to tackle problems in a variety of fields. Reddy et al. [ 74 ], for example, use a fuzzy logic classifier for heart disease detection, with the derived rules from fuzzy classifiers being optimized using an adaptive genetic algorithm. Krishnan et al. [ 51 ] describes a fuzzy logic-based smart irrigation system using IoT, which sends out periodic acknowledgment messages on task statuses such as soil humidity and temperature. Hamamoto et al. [ 34 ] describe a network anomaly detection method based on fuzzy logic for determining whether or not a given instance is anomalous. Kang et al. [ 44 ] proposed a fuzzy weighted association rule mining approach for developing a customer satisfaction product form. Overall, we can infer that fuzzy logic can make reasonable conclusions in a world of imprecision, uncertainty, and partial data, and thus might be useful in such scenarios while building a model.

A general architecture of fuzzy logic systems

Knowledge Representation, Uncertainty Reasoning, and Expert System Modeling

Knowledge representation is the study of how an intelligent agent’s beliefs, intents, and judgments may be expressed appropriately for automated reasoning, and it has emerged as one of the most promising topics of Artificial Intelligence. Reasoning is the process of using existing knowledge to conclude, make predictions, or construct explanations. Many types of knowledge can be used in various application domains include descriptive knowledge, structural knowledge, procedural knowledge, meta-knowledge, and heuristic knowledge [ 87 ]. Knowledge representation is more than just storing data in a database; it also allows an intelligent machine to learn from its knowledge and experiences to act intelligently as a human. As a result, in designing an intelligent system, an effective method of knowledge representation is required. Several knowledge representation approaches exist in the fields that can be utilized to develop a knowledge-based conceptual model, including logical, semantic network, frame, and production rules [ 95 ]. In the following, we summarize the potential knowledge representation strategies taking real-world issues into account.

An example of ontology components for the entity University [ 26 ]

• Rule-base It typically consists of pairs of the condition, and corresponding action, which means, “IF < c o n d i t i o n > THEN < a c t i o n > ” [ 85 ]. As a result, an agent checks the condition first, and if the condition is satisfied, the related rule fires. The key benefit of a rule-based system like this is that the “condition” part can select which rule is appropriate to use for a given scenario. The “action” portion, on the other hand, is responsible for implementing the problem’s solutions. Furthermore, in a rule-based system, we can easily insert, delete, or update rules as needed.
• Uncertainty and probabilistic reasoning Probabilistic reasoning is a method of knowledge representation in which the concept of probability is used to signify the uncertainty in knowledge, and where probability theory and logic are combined to address the uncertainty [ 65 ]. Probability is the numerical measure of the possibility of an event occurring, and it can be defined as the chance that an uncertain event will occur. To deal with uncertainty in a model, probabilistic models, fuzzy logic, Bayesian belief networks, etc. can be employed.

A knowledge-based system, such as an expert system for decision-making, relies on these representations of knowledge. The inference engine and the knowledge base are two subsystems of the expert system, as represented in Fig.  9 . The information in the knowledge base is organized according to the knowledge representation discussed above. The inference engine looks for knowledge-based information and linkages and, like a human expert, provides answers, predictions, and recommendations. Such a knowledge-based system can be found in many application areas. For instance, Goel et al. [ 29 ] present an ontology-driven context-aware framework for smart traffic monitoring. Chukkapalli et al. [ 16 ] present ontology-driven AI and access control systems for smart fisheries. Kiran et al. [ 49 ] present enhanced security-aware technique and ontology data access control in cloud computing. Syed et al. [ 97 ] present a conceptual ontology and cyber intelligence alert system for cybersecurity vulnerability management. An ontology-based cyber security policy implementation in Saudi Arabia has been presented in Talib et al. [ 98 ]. Recently, Sarker et al. [ 90 ] explores an expert system modeling for personalized decision-making in mobile apps. Thus, knowledge representation and modeling are important to build AI models as well as intelligent decision-making in various application areas to solve real-world issues.

A general architecture of an expert system

Case-Based Reasoning

Case-based reasoning (CBR) is a cognitive science and AI paradigm that represents reasoning as primarily memory-based. CBR is concerned with the “smart” reuse of knowledge from previously solved problems (“cases”) and its adaption to new and unsolved problems. The inference is a problem-solving strategy based on the similarity of the current situation to previously solved problems recorded in a repository. Its premise is that the more similar the two issues are, the more similar their solutions will be. Thus, case-based reasoners handle new problems by obtaining previously stored ’cases’ that describe similar earlier problem-solving experiences and customizing their solutions to meet new requirements. For example, patient case histories and treatments are utilized in medical education to assist diagnose and treating new patients. Figure  10 shows a general architecture of case-based reasoning. CBR research looks at the CBR process as a model of human cognition as well as a method for developing intelligent systems.

A general architecture of case-based reasoning

CBR is utilized in a variety of applications. Lamy et al. [ 52 ], for example, provide a visual case-based reasoning strategy for explainable artificial intelligence for breast cancer. Gonzalez et al. [ 30 ] provide a case-based reasoning-based energy optimization technique. Khosravani et al. [ 47 ] offers a case-based reasoning application in a defect detection system for dripper manufacturing. Corrales et al. [ 17 ] provide a case-based reasoning system for data cleaning algorithm recommendation in classification and regression problems. As the number of stored cases grows, CBR becomes more intelligent and thus might be useful in such scenarios while building a model. However, as the time required to find and process relevant cases increases, the system’s efficiency will decline.

Text Mining and Natural Language Processing

Text mining [ 7 ], also known as text data mining, similar to text analytics, is the process of extracting meaningful information from a variety of text or written resources, such as websites, books, emails, reviews, docs, comments, articles, and so on. Information retrieval, lexical analysis to investigate word frequency distributions, pattern recognition, tagging or annotation, information extraction, and data mining techniques such as link and association analysis, visualization, and predictive analytics are all part of text analysis. Text mining achieves this by employing several analysis techniques, such as natural language processing (NLP). NLP is a text analysis technique that allows machines to interpret human speech. NLP tasks include speech recognition, also known as speech-to-text, word segmentation or tokenization, lemmatization and stemming, part of speech tagging, parsing, word sense disambiguation, named entity recognition, sentiment analysis, topic segmentation and recognition, and natural language generation, which is the task of converting structured data into human language [ 21 ]. Fake news identification, spam detection, machine translation, question answering, social media sentiment analysis, text summarization, virtual agents and chatbots, and other real-world applications use NLP techniques.

Although many language-processing systems were built in the early days using symbolic approaches, such as hand-coding a set of rules and looking them up in a dictionary, NLP now blends computational linguistics with statistical, machine learning, and deep learning models [ 80 , 81 ]. These technologies, when used together, allow computers to process human language in the form of text or speech data and comprehend its full meaning, including the speaker’s or writer’s intent and sentiment. Many works have been done in this area. For example, using the feature ensemble model, Phan et al. [ 68 ] propose a method for improving the performance of sentiment analysis of tweets with a fuzzy sentiment. Using weighted word embeddings and deep neural networks, Onan et al. [ 62 ] provide sentiment analysis on product reviews. Subramaniyaswamy et al. [ 96 ] present sentiment analysis of tweets for estimating event criticality and security. In [ 60 ], the efficacy of social media data in healthcare communication is discussed. Typically, learning techniques rather than static analysis is more effective in terms of automation and intelligence in textual modeling or NLP systems. In addition to standard machine learning algorithms [ 81 ], deep learning models and techniques, particularly, based on convolutional neural networks (CNNs) and recurrent neural networks (RNNs) enable such systems to learn as they go and extract progressively accurate meaning from large amounts of unstructured, unlabeled text and speech input. Thus various deep learning techniques including generative and discriminative models can be used to build powerful textual or NLP model according to their learning capabilities from data, discussed briefly in our earlier paper Sarker et al. [ 80 ], which could also be a significant research direction in the area. Overall, we can conclude that by combining machine and deep learning techniques with natural language processing, computers can intelligently analyze, understand, and infer meaning from human speech or text, and thus could be useful for building textual AI models.

Visual Analytics, Computer Vision and Pattern Recognition

Computer vision [ 99 ] is also a branch of AI that allows computers and systems to extract useful information from digital images, videos, and other visual inputs and act or make recommendations based on that data. From an engineering standpoint, it aims to comprehend and automate operations that the human visual system is capable of. As a result, this is concerned with the automated extraction, analysis, and comprehension of relevant information from a single image or a series of images. In terms of technology, it entails the creation of a theoretical and algorithmic foundation for achieving autonomous visual understanding by processing an image at the pixel level. Typical tasks in the field of visual analytics and computer vision include object recognition or classification, detection, tracking, picture restoration, feature matching, image segmentation, scene reconstruction, video motion analysis, and so on.

Pattern recognition, which is the automated recognition of patterns and regularities in data, is the basis for today’s computer vision algorithms. Pattern recognition often involves the categorization (supervised learning) and grouping (unsupervised learning) of patterns [ 81 ]. Although pattern recognition has its roots in statistics and engineering, due to the greater availability of huge data and a new wealth of processing power, some recent techniques to pattern recognition include the use of machines and deep learning. Convolutional neural networks (CNN or ConvNet) [ 53 , 80 ] have recently demonstrated considerable promise in a variety of computer vision tasks, including classification, object detection, and scene analysis. The general architecture of a convolution neural network is depicted in Figure  11 . Large datasets of thousands or millions of labeled training samples are typically used to train these algorithms. However, the lack of appropriate data limits the applications that can be developed. While enormous volumes of data can be obtained fast, supervised learning also necessitates data that has been labeled. Unfortunately, data labeling takes a long time and costs a lot of money. In this area, a lot of work has been done. Elakkiya et al. [ 25 ] develop a cervical cancer diagnostics healthcare system utilizing hybrid object detection adversarial networks in their paper. Harrou et al. [ 38 ] present an integrated vision-based technique for detecting human falls in a residential setting. Pan et al. [ 63 ] demonstrated a visual recognition based on deep learning for navigation mark classification. Typically, learning techniques rather than static analysis is more effective in terms of automation and intelligence in such visual analytics. In addition to standard machine learning algorithms [ 81 ], various deep learning techniques including generative and discriminative models can be used to build powerful visual model according to their learning capabilities from data, discussed briefly in our earlier paper Sarker et al. [ 80 ], which could also be a significant research direction in the area. Thus, this is important to build effective visual AI models in various application areas to solve real-world issues in the current age of the Fourth Industrial Revolution or Industry 4.0, according to the goal of this paper.

A general architecture of a convolutional neural network (CNN or ConvNet)

Hybrid Approach, Searching, and Optimization

A “hybrid approach” is a blend of multiple approaches or systems to design a new and superior model. As a result, a hybrid strategy integrates the necessary approaches outlined above depending on the demands. For instance, in our earlier publication, Sarker et al. [ 85 ], we have used a hybridization of machine learning and knowledge-base expert system to build an effective context-aware model for intelligent mobile services. In this hybrid context-aware model, context-aware rules are discovered using machine learning techniques, which are used as the knowledge base of an expert system rather than traditional handcrafted static rules to make computing and decision-making processes more actionable and intelligent. Similarly, in another hybrid approach [ 68 ], the concepts of fuzzy logic, deep learning, and natural language processing were integrated to improve Twitter sentiment analysis accuracy. The authors in [ 33 ] present a deep convolutional neural network-based automated and robust object recognition in X-ray baggage inspection, where deep learning is integrated with computer vision analysis. Kang et al. [ 44 ] proposed a fuzzy weighted association rule mining strategy to produce a customer satisfaction product form. Moreover, Sarker et al. discussed various machine learning [ 81 ] and deep learning [ 80 ] techniques and their hybridization that can be used to solve a variety of real-world problems in many application areas such as business, finance, healthcare, smart cities, cybersecurity, etc. Thus, hybridization of multiple techniques could play a key role to build an effective AI model in the area.

Moreover, many AI problems can be solved theoretically by searching through a large number of possible solutions, and the reasoning process may be reduced down to a simple search. Thus, search strategies, also known as universal problem-solving approaches in AI, can also play a significant role to solve real-world issues such as gaming, ranking web pages, video, and other content in search results, etc., due to the properties of its completeness, optimality, time complexity, and space complexity. Depending on the nature of the problems, search algorithms can be uninformed search (a.k.a. blind, brute-force) or informed search (a.k.a. heuristic search). Uninformed search [ 75 ] refers to a group of general-purpose search algorithms that generate search trees without relying on domain information, such as breadth-first, depth-first, uniform cost search, etc. Informed search [ 75 ] algorithms, on the other hand, use additional or problem-specific knowledge in the search process, such as greedy search, A* search, graph search, etc. For example, when searching on Google Maps, one needs to provide information such as a position from the current location to precisely traverse the distance, time traveled, and real-time traffic updates on that specific route. Informed search can solve a variety of complicated problems that cannot be handled any other way. Furthermore, evolutionary computation employs an optimization search technique, such as genetic algorithms, which has a great potential to solve real-world issues. For instance, in the domain of cybersecurity, a genetic algorithm is used for effective feature selection to detect anomalies in fog computing environment [ 61 ]. In [ 28 ], genetic algorithm is used for optimized feature selection to detect Android malware using machine learning techniques. With AI-powered search, the platform learns from the data to provide the most accurate and relevant search results automatically. Thus, searching as well as optimization techniques can be used as a part of hybridization while building AI models to solve real-world problems.

Overall, we can conclude that the above explored ten potential AI techniques can play a significant role while building various AI models such as analytical, functional, interactive, textual, and visual models, depending on the nature of the problem and target application. In the next section, we summarize various real-world application areas, where these AI techniques are employed in today’s interconnected world towards automation, intelligent and smart systems.

Real-World Applications of AI

AI approaches have been effectively applied to a variety of issues in a variety of application areas throughout the last several years. Healthcare, cybersecurity, business, social media, virtual reality and assistance, robotics, and many other application areas are common nowadays. We have outlined some potential real-world AI application areas in Fig.  12 . Various AI techniques, such as machine learning, deep learning, knowledge discovery, reasoning, natural language processing, expert system modeling, and many others, as detailed above in “ Potential AI techniques ” are used in these application domains. We have also listed several AI tasks and techniques that are utilized to solve in several real-world application areas in Table  3 . Overall, we can conclude from Fig.  12 and Table  3 that the future prospects of AI modeling in real-world application domains are vast and there are several opportunities to work and conduct research. In the following section, we discuss the future aspect of AI as well as research issues towards automation, intelligent and smart systems.

Several potential real-world application areas of artificial intelligence (AI)

A summary of AI tasks and methods in several popular real-world applications areas

Future Aspect and Research Issues

Artificial intelligence is influencing the future of almost every sector and every person on the planet. AI has acted as the driving force behind developing technologies for industrial automation, medical applications, agriculture, IoT applications, cybersecurity services, etc. summarized in “ Future Aspect and Research Issues ”, and it will continue to do so for the foreseeable future. This interdisciplinary science comes with numerous advancements and approaches that are possible with the help of deep learning, machine learning algorithms, knowledge-base expert systems, natural language processing, visual recognition, etc. discussed briefly in “ Potential AI techniques ”. Thus, by taking into account the capabilities of AI technologies, we illustrate three essential terms, mentioned in “ Introduction ” within the scope of our study. These are

• Automation One of the main themes of today’s applications is automation, which encompasses a wide range of technologies that reduce human interaction in operations. A program, a script, or batch processing are commonly used in computing to automate tasks. AI-based automation takes the insights gained through computational analytics to the next level, allowing for automated decision-making. As a result, we can describe automation as the development and implementation of technology to manufacture and deliver products and services to increase the efficiency, dependability, and/or speed of various jobs traditionally handled by humans. In customer service, for example, virtual assistants can lower expenses while empowering both customers and human agents, resulting in a better customer experience. Artificial intelligence technology has the potential to automate almost any industry and every person on the planet.
• Intelligent computing It is also known as computational intelligence, and it refers to a computer’s or system’s ability to extract insights or usable knowledge from data or experimental observation, or to learn a specific task. Intelligent computing methodologies include information processing, data mining, and knowledge discovery, as well as machine learning, pattern recognition, signal processing, natural language processing, fuzzy systems, knowledge representation, and reasoning. Transportation, industry, health, agriculture, business, finance, security, and other fields could all benefit from intelligent systems. Thus, the above-mentioned AI techniques, discussed in “ Potential AI techniques ” are the main drivers for performing intelligent computing as well as decision-making.
• Smart computing The word “Smart” can be described as self-monitoring, analyzing, and reporting technology in smart computing, and the word “Computing” can be defined as computational analysis. As a result, it can be thought of as the next generation of computing, which is used to create something self-aware, that is, something that can sense the activities of its environment, massage the gathered data, perform some analytics, and provide the best decisions while also predicting future risks and challenges. In other words, it is a significant multidisciplinary area in which AI-based computational methods and technologies, as explained in “ Potential AI techniques ”, are integrated with engineering approaches to produce systems, applications, and new services that suit societal demands. Overall, it strives to construct a smart system by monitoring, analyzing, and reporting data in a faster and smarter manner, with AI-based modeling playing a vital part in system intelligence and decision-making.

The above terms are also the key focus of the current fourth industrial revolution (Industry 4.0). Business, health care, energy, transportation systems, environment, security, surveillance, industrial systems, information retrieval and publication, entertainment and creativity, and social activities can all benefit from automation, intelligence, and smart computer systems. For example, chatbots, consumer personalization, image-based targeting advertising, and warehouse and inventory automation are all examples of how AI will continue to drive e-commerce. The potential benefits of using AI in medicine are now being investigated. The medical industry has a wealth of data that may be used to develop healthcare-related predictive models. Manufacturing, notably the automobile industry, will be significantly impacted by AI. AI will have an impact on sales operations in a range of industries. Marketing tactics, such as business models, sales procedures, and customer service options, as well as customer behavior, are predicted to be significantly influenced by AI. AI and machine learning will be key technologies in cybersecurity for identifying and forecasting threats [ 77 , 89 ]. AI will be a vital tool for financial security because of its ability to analyze large amounts of data, foresee fraud, and identify it. In the near future, interacting with AI will surely become commonplace. Artificial intelligence can be used to solve incredibly difficult problems and find solutions that are vital to human well-being. These developments have enormous economic and societal implications. Thus, we can say, AI’s potential is limitless and its future will be shaped by our decisions and actions. While our discussion has established a solid foundation on AI-based systems and applications, hence we outline the below ten research issues.

• Several potential AI techniques exist in the area with the capability of solving problems, discussed in “ Potential AI techniques ”. To understand the nature of the problem and an in-depth analysis is important to find a suitable solution, i.e., detecting cyber-anomalies or attacks [ 78 ]. Thus, the challenge is “Which AI technique is most suited to solving a specific real-world problem, taking into account the problem’s nature?”
• One promising research direction for AI-based solutions is to develop a general framework that can handle the issues involved. A well-designed framework and experimental evaluation are both a crucial direction and a significant challenge. Thus, the question is “How can we design an effective AI-based framework to achieve the target outcome by taking into account the issues involved?”
• The digital world contains a wealth of data in this age of the Fourth Industrial Revolution (Industry 4.0 or 4IR), including IoT data, corporate data, health data, cellular data, urban data, cybersecurity data, and many more [ 79 ]. Extracting insights using various analytical methods is important for smart decision-making in a particular system. Thus, the question is “How to extract useful insights or knowledge from real-world raw data to build an automated and intelligent system for a particular business problem?
• Nowadays, data are considered as the most valuable resource in the world and various machine learning [ 81 ] and deep learning [ 80 ] techniques are used to learn from data or past experience, which automates analytical model building. The increase in data and such data-driven analytical modeling have made AI the highest growth in history. Thus, it’s important to do some data pre-processing tasks to feed into the ultimate machine learning model, so the data behaves nicely for the model. Therefore, the question is “How to effectively feed data to a machine or deep learning model to solve a particular real-world problem?”
• The traditional machine learning [ 81 ] and deep learning [ 80 ] techniques may not be directly applicable for the expected outcome in many cases. Thus, designing new techniques or their variants by taking into account model optimization, accuracy, and applicability, according to the nature of the data and target real-world application, could be a novel contribution in the area. Therefore the question is—“How to design an effective learning algorithm or model allowing the application to learn automatically from the patterns or features in the data?”
• In the domain of today’s smart computing, the term ‘context-awareness’ typically refers to a system’s capacity to gather information about its surroundings at any given time and adapt its behavior accordingly. Thus, the concept of context-aware machine learning can play a key role to build an intelligent context-aware application, highlighted in our book Sarker et al. [ 85 ]. Thus, the question is “How to effectively incorporate context-awareness in an AI-based smart system that can sense from the surrounding environment and make intelligent decisions accordingly?”
• Decision rules, represented as IF-THEN statement, can play an important role in the area of AI. Expert systems, a core part of AI, are typically used to solve many real-world complex problems by reasoning through knowledge, which are mostly represented by such IF-THEN rules rather than traditional procedural code [ 85 ]. Thus, a rule-based system can manipulate knowledge and interpret information in a useful way.  Therefore, the question is “How can we design an automated rule-based system emulating the decision-making ability of a human expert through discovering a concise set of IF-THEN rules from the data?
• A decision support system is a type of information system that aids in the decision-making process of a business or organization. AI techniques discussed in “ Potential AI techniques ” can play a key role to provide intelligent decisions across a wide range of sectors (e.g., business, education, healthcare, etc.) rather than the traditional system, according to the nature of the problem. Thus, the challenge is “How can we design an AI-assisted decision-support system that aids a team or organization in making better decisions?”
• Uncertainty refers to an event’s lack of confidence or certainty, e.g., information occurred from unreliable sources. Several strategies, such as the probability-based model or fuzzy logic, discussed in “ Potential AI techniques ” allow for the processing of uncertain and imprecise knowledge while also providing a sophisticated reasoning framework. The ability of AI to identify and handle uncertainty and risk is essential for applying AI to decision-making challenges. Thus, the question is “How to manage uncertainty in AI-enabled decision-making applications”.
• With the widespread availability of various IoT services, Internet of things (IoT) devices are becoming more common in mobile networks. It is essential nowadays to have a lightweight solution that promises high-performing artificial intelligence applications for mobile and IoT devices. Thus, the question is “How to design AI-enabled lightweight model for intelligent decision-making through IoT and mobile devices”.

To summarize, AI is a relatively open topic to which academics can contribute by inventing new methods or refining existing methods to address the issues raised above and solve real-world problems in a range of application areas. AI will be employed in any context where large amounts of data are needed to be handled fast and accurately, and cost savings are required. AI will affect the planet more than anything else in human history. One important thing is that AI-powered automation does not pose a threat to jobs in the workplace for individuals, businesses, or countries with the appropriate skills. AI-certified professionals have access to a wide range of job prospects. AI Engineer, Artificial Intelligence Programmer, AI System Developer, Data Scientist, Machine Learning Engineer, Data Analyst, AI Architect, Deep Learning Engineer, AI Software Engineer, and many other employment opportunities are available to these professionals.

Overall, AI technologies are driving a new wave of economic progress, resolving some of the world’s most challenging issues and delivering solutions to some of humanity’s most significant challenges. Many industries, including information technology, telecommunications, transportation, traffic management, health care, education, criminal justice, defense, banking, and agriculture, have the potential to be transformed by artificial intelligence. Without compromising the significant characteristics that identify mankind, we can assure that AI systems are deliberate, intelligent, and flexible with adequate security. Governments and decision-makers of a country need to focus public policies that promote AI innovation while minimizing unexpected societal consequences to realize its full potential in real-world scenarios.

Concluding Remarks

In this article, we have provided a comprehensive view of AI-based modeling which is considered a key component of the fourth industrial revolution (Industry 4.0). It begins with research motivation and proceeds to AI techniques and breakthroughs in many application domains. Then in numerous dimensions, the important techniques in this area are explored. We take into account ten categories of popular AI techniques in this thorough analysis, including machine learning, deep learning, natural language processing, knowledge discovery, expert system modeling, etc., which can be applied in a variety of applications depending on current demands. In terms of machine intelligence, complex learning algorithms should be trained using data and knowledge from the target application before the system can help with intelligent decision-making.

Overall, AI techniques have proven to be beneficial in a variety of applications and research fields, including business intelligence, finance, healthcare, visual recognition, smart cities, IoT, cybersecurity, and many more, as explored in the paper. Finally, we explored the future aspects of AI towards automation, intelligence, and smart computing systems, highlighting several research issues within the scope of our study. This can also aid researchers in conducting more in-depth analyses, resulting in a more reliable and realistic outcome. Overall, we feel that our study and discussion on AI-based modeling points in the right direction and can be used as a reference guide for future research and development in relevant application domains by academics as well as industry professionals.

Open Access funding enabled and organized by CAUL and its Member Institutions.

Declarations

The author declares no conflict of interest.

This article is part of the topical collection “Advances in Computational Approaches for Artificial Intelligence, Image Processing, IoT and Cloud Applications” guest-edited by Bhanu Prakash K N and M. Shivakumar.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Perspective
• Published: 07 January 2019

Taking climate model evaluation to the next level

• Veronika Eyring   ORCID: orcid.org/0000-0002-6887-4885 1 , 2 ,
• Peter M. Cox   ORCID: orcid.org/0000-0002-0679-2219 3 ,
• Gregory M. Flato 4 ,
• Peter J. Gleckler 5 ,
• Gab Abramowitz   ORCID: orcid.org/0000-0002-4205-001X 6 ,
• Peter Caldwell 5 ,
• William D. Collins   ORCID: orcid.org/0000-0002-4463-9848 7 , 8 ,
• Bettina K. Gier 1 , 2 ,
• Alex D. Hall 9 ,
• Forrest M. Hoffman   ORCID: orcid.org/0000-0001-5802-4134 10 , 11 ,
• George C. Hurtt 12 ,
• Alexandra Jahn   ORCID: orcid.org/0000-0002-6580-2579 13 ,
• Chris D. Jones   ORCID: orcid.org/0000-0002-7141-9285 14 ,
• Stephen A. Klein   ORCID: orcid.org/0000-0002-5476-858X 5 ,
• John P. Krasting 15 ,
• Lester Kwiatkowski   ORCID: orcid.org/0000-0002-6769-5957 16 ,
• Ruth Lorenz   ORCID: orcid.org/0000-0002-3986-1268 17 ,
• Eric Maloney   ORCID: orcid.org/0000-0002-2660-2611 18 ,
• Gerald A. Meehl 19 ,
• Angeline G. Pendergrass   ORCID: orcid.org/0000-0003-2542-1461 19 ,
• Robert Pincus   ORCID: orcid.org/0000-0002-0016-3470 18 ,
• Alex C. Ruane 20 ,
• Joellen L. Russell   ORCID: orcid.org/0000-0001-9937-6056 21 ,
• Benjamin M. Sanderson   ORCID: orcid.org/0000-0001-8635-4624 19 ,
• Benjamin D. Santer 5 ,
• Steven C. Sherwood   ORCID: orcid.org/0000-0001-7420-8216 6 ,
• Isla R. Simpson 19 ,
• Ronald J. Stouffer   ORCID: orcid.org/0000-0002-7900-6290 21 &
• Mark S. Williamson 3  

Nature Climate Change volume  9 ,  pages 102–110 ( 2019 ) Cite this article

16k Accesses

403 Citations

234 Altmetric

Metrics details

• Climate and Earth system modelling
• Environmental sciences
• Projection and prediction
• Scientific community

Earth system models are complex and represent a large number of processes, resulting in a persistent spread across climate projections for a given future scenario. Owing to different model performances against observations and the lack of independence among models, there is now evidence that giving equal weight to each available model projection is suboptimal. This Perspective discusses newly developed tools that facilitate a more rapid and comprehensive evaluation of model simulations with observations, process-based emergent constraints that are a promising way to focus evaluation on the observations most relevant to climate projections, and advanced methods for model weighting. These approaches are needed to distil the most credible information on regional climate changes, impacts, and risks for stakeholders and policy-makers.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 print issues and online access

195,33 € per year

only 16,28 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Global impacts of heat and water stress on food production and severe food insecurity

Increasing frequency and intensity of the most extreme wildfires on Earth

The economic commitment of climate change

IPCC Climate Change 2013: The Physical Science Basis (eds Stocker, T. F. et al.) (Cambridge Univ. Press, 2013).

Meehl, G. A., Boer, G. J., Covey, C., Latif, M. & Stouffer, R. J. The Coupled Model Intercomparison Project (CMIP). Bull. Am. Meteorol. Soc. 81 , 313–318 (2000).

Google Scholar  

Meehl, G. A. et al. THE WCRP CMIP3 multimodel dataset: a new era in climate change research. Bull. Am. Meteorol. Soc. 88 , 1383–1394 (2007).

Taylor, K. E., Stouffer, R. J. & Meehl, G. A. An overview of CMIP5 and the experiment design. Bull. Am. Meteorol. Soc. 93 , 485–498 (2012).

Eyring, V. et al. Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci. Model Dev. 9 , 1937–1958 (2016).

Stouffer, R. J. et al. CMIP5 scientific gaps and recommendations for CMIP6. Bull. Am. Meteorol. Soc. 98 , 95–105 (2017).

Flato, G. et al. in Climate Change 2013: The Physical Science Basis (eds Stocker, T. F. et al.) Ch. 9 (IPCC, Cambridge Univ. Press, 2013).

Maraun, D. et al. Towards process-informed bias correction of climate change simulations. Nat. Clim. Change 7 , 764–773 (2017).

Collins, M. et al. in Climate Change 2013: The Physical Science Basis (eds Stocker, T. F. et al.) Ch. 12 (IPCC, Cambridge Univ. Press, 2013).

Knutti, R. et al. A climate model projection weighting scheme accounting for performance and interdependence. Geophys. Res. Lett. 44 , 1909–1918 (2017).

Wenzel, S., Eyring, V., Gerber, E. P. & Karpechko, A. Y. Constraining future summer austral jet stream positions in the CMIP5 ensemble by process-oriented multiple diagnostic regression. J. Clim. 29 , 673–687 (2016).

Sanderson, B. M., Wehner, M. & Knutti, R. Skill and independence weighting for multi-model assessments. Geosci. Model Dev. 10 , 2379–2395 (2017).

Sanderson, B. M., Knutti, R. & Caldwell, P. Addressing interdependency in a multimodel ensemble by interpolation of model properties. J. Clim. 28 , 5150–5170 (2015).

Bishop, C. H. & Abramowitz, G. Climate model dependence and the replicate Earth paradigm. Clim. Dynam. 41 , 885–900 (2013).

Abramowitz, G. & Bishop, C. H. Climate model dependence and the ensemble dependence transformation of CMIP projections. J. Clim. 28 , 2332–2348 (2015).

Alexander, K. & Easterbrook, S. M. The software architecture of climate models: a graphical comparison of CMIP5 and EMICAR5 configurations. Geosci. Model Dev. 8 , 1221–1232 (2015).

Knutti, R. The end of model democracy? Climatic Change 102 , 395–404 (2010).

Eyring, V. et al. ESMValTool (v1.0) — a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP. Geosci. Model Dev. 9 , 1747–1802 (2016).

CAS   Google Scholar  

Gleckler, P. et al. A more powerful reality test for climate models. Eos https://doi.org/10.1029/2016eo051663 (2016).

Lauer, A. et al. Process-level improvements in CMIP5 models and their impact on tropical variability, the Southern Ocean, and monsoons. Earth Syst. Dynam. 9 , 33–67 (2018).

Ma, C.-C., Mechoso, C. R., Robertson, A. W. & Arakawa, A. Peruvian stratus clouds and the tropical pacific circulation: a coupled ocean-atmosphere GCM study. J. Clim. 9 , 1635–1645 (1996).

Hourdin, F. et al. Parameterization of convective transport in the boundary layer and its impact on the representation of the diurnal cycle of wind and dust emissions. Atmos. Chem. Phys. 15 , 6775–6788 (2015).

Richter, I. Climate model biases in the eastern tropical oceans: causes, impacts and ways forward. WIREs Clim. Change 6 , 345–358 (2015).

Wang, C. Z., Zhang, L. P., Lee, S. K., Wu, L. X. & Mechoso, C. R. A global perspective on CMIP5 climate model biases. Nat. Clim. Change 4 , 201–205 (2014).

Fyfe, J. C., Gillett, N. P. & Zwiers, F. W. Overestimated global warming over the past 20 years. Nat. Clim. Change 3 , 767–769 (2013).

Meehl, G. A., Hu, A., Arblaster, J. M., Fasullo, J. & Trenberth, K. E. Externally forced and internally generated decadal climate variability associated with the Interdecadal Pacific Oscillation. J. Clim. 26 , 7298–7310 (2013).

Fyfe, J. C. et al. Making sense of the early-2000s warming slowdown. Nat. Clim. Change 6 , 224–228 (2016).

Santer, B. D. et al. Volcanic contribution to decadal changes in tropospheric temperature. Nat. Geosci. 7 , 185–189 (2014).

Meehl, G. A., Teng, H. & Arblaster, J. M. Climate model simulations of the observed early-2000s hiatus of global warming. Nat. Clim. Change 4 , 898–902 (2014).

Thoma, M., Greatbatch, R. J., Kadow, C. & Gerdes, R. Decadal hindcasts initialized using observed surface wind stress: evaluation and prediction out to 2024. Geophys. Res. Lett. 42 , 6454–6461 (2015).

Meehl, G. A., Hu, A. & Teng, H. Initialized decadal prediction for transition to positive phase of the Interdecadal Pacific Oscillation. Nat. Commun. 7 , 11718 (2016).

Mears, C. A., Santer, B. D., Wentz, F. J., Taylor, K. E. & Wehner, M. F. Relationship between temperature and precipitable water changes over tropical oceans. Geophys. Res. Lett. 34 , L24709 (2007).

Karl, T. R. et al. Possible artifacts of data biases in the recent global surface warming hiatus. Science 348 , 1469–1472 (2015).

Mears, C. A. & Wentz, F. J. The effect of diurnal correction on satellite-derived lower tropospheric temperature. Science 309 , 1548–1551 (2005).

Mauritsen, T. et al. Tuning the climate of a global model. J. Adv. Model. Earth Syst. 4 , M00A01 (2012).

Hourdin, F. et al. The art and science of climate model tuning. Bull. Am. Meteorol. Soc. 98 , 589–602 (2017).

Bodas-Salcedo, A. et al. COSP Satellite simulation software for model assessment. Bull. Am. Meteorol. Soc. 92 , 1023–1043 (2011).

Eyring, V. et al. Towards improved and more routine Earth system model evaluation in CMIP. Earth Syst. Dynam. 7 , 813–830 (2016).

Phillips, A. S., Deser, C. & Fasullo, J. Evaluating modes of variability in climate models. Eos 95 , 453–455 (2014).

Luo, Y. Q. et al. A framework for benchmarking land models. Biogeosciences 9 , 3857–3874 (2012).

Prabhat. et al. TECA: a parallel toolkit for extreme climate analysis. Proc. Comput. Sci. 9 , 866–876 (2012).

Ferraro, R., Waliser, D. E., Gleckler, P., Taylor, K. E. & Eyring, V. Evolving Obs4MIPs to support phase 6 of the Coupled Model Intercomparison Project (CMIP6). Bull. Am. Meteorol. Soc. 96 , ES131–ES133 (2015).

Hall, A. & Qu, X. Using the current seasonal cycle to constrain snow albedo feedback in future climate change. Geophys. Res. Lett. 33 , L03502 (2006).

Allen, M. R. & Ingram, W. J. Constraints on future changes in climate and the hydrologic cycle. Nature 419 , 224–232 (2002).

Massonnet, F. et al. Constraining projections of summer Arctic sea ice. Cryosphere 6 , 1383–1394 (2012).

O’Gorman, P. A. Sensitivity of tropical precipitation extremes to climate change. Nat. Geosci. 5 , 697–700 (2012).

Cox, P. M. et al. Sensitivity of tropical carbon to climate change constrained by carbon dioxide variability. Nature 494 , 341–344 (2013).

Kidston, J. & Gerber, E. P. Intermodel variability of the poleward shift of the austral jet stream in the CMIP3 integrations linked to biases in 20th century climatology. Geophys. Res. Lett. 37 , L09708 (2010).

Tsushima, Y. et al. Robustness, uncertainties, and emergent constraints in the radiative responses of stratocumulus cloud regimes to future warming. Clim. Dynam. 46 , 3025–3039 (2016).

Brient, F. & Bony, S. Interpretation of the positive low-cloud feedback predicted by a climate model under global warming. Clim. Dynam. 40 , 2415–2431 (2013).

Brient, F. & Schneider, T. Constraints on climate sensitivity from space-based measurements of low-cloud reflection. J. Clim. 29 , 5821–5835 (2016).

Lipat, B. R., Tselioudis, G., Grise, K. M. & Polvani, L. M. CMIP5 models’ shortwave cloud radiative response and climate sensitivity linked to the climatological Hadley cell extent. Geophys. Res. Lett. 44 , 5739–5748 (2017).

Sherwood, S. C., Bony, S. & Dufresne, J. L. Spread in model climate sensitivity traced to atmospheric convective mixing. Nature 505 , 37–42 (2014).

Tian, B. Spread of model climate sensitivity linked to double-Intertropical Convergence Zone bias. Geophys. Res. Lett. 42 , 4133–4141 (2015).

Cox, P. M., Huntingford, C. & Williamson, M. S. Emergent constraint on equilibrium climate sensitivity from global temperature variability. Nature 553 , 319–322 (2018).

Klein, S. A. & Hall, A. Emergent constraints for cloud feedbacks. Curr. Clim. Change Rep. 1 , 276–287 (2015).

Dessler, A. E. & Forster, P. M. An estimate of equilibrium climate sensitivity from interannual variability. J. Geophys. Res. Atmos. 123 , 8634–8645 (2018).

DeAngelis, A. M., Qu, X., Zelinka, M. D. & Hall, A. An observational radiative constraint on hydrologic cycle intensification. Nature 528 , 249–253 (2015).

Li, G., Xie, S.-P., He, C. & Chen, Z. Western Pacific emergent constraint lowers projected increase in Indian summer monsoon rainfall. Nat. Clim. Change 7 , 708–712 (2017).

Wenzel, S., Cox, P. M., Eyring, V. & Friedlingstein, P. Emergent constraints on climate-carbon cycle feedbacks in the CMIP5 Earth system models. J. Geophys. Res. Biogeosci. 119 , 794–807 (2014).

Wenzel, S., Cox, P. M., Eyring, V. & Friedlingstein, P. Projected land photosynthesis constrained by changes in the seasonal cycle of atmospheric CO 2 . Nature 538 , 499–501 (2016).

Kwiatkowski, L. et al. Emergent constraints on projections of declining primary production in the tropical oceans. Nat. Clim. Change 7 , 355–358 (2017).

Chadburn, S. E. et al. An observation-based constraint on permafrost loss as a function of global warming. Nat. Clim. Change 7 , 340–344 (2017).

Hoffman, F. M. et al. Causes and implications of persistent atmospheric carbon dioxide biases in Earth system models. J. Geophys. Res. Biogeosci. 119 , 141–162 (2014).

Donat, M. G., Pitman, A. J. & Angélil, O. Understanding and reducing future uncertainty in mid-latitude daily heat extremes via land surface feedback constraints. Geophys. Res. Lett. 45 , 10627–10636 (2018).

Qu, X. & Hall, A. What controls the strength of snow-albedo feedback? J. Clim. 20 , 3971–3981 (2007).

Waugh, D. W. & Eyring, V. Quantitative performance metrics for stratospheric-resolving chemistry-climate models. Atmos. Chem. Phys. 8 , 5699–5713 (2008).

Karpechko, A. Y., Maraun, D. & Eyring, V. Improving Antarctic total ozone projections by a process-oriented multiple diagnostic ensemble regression. J. Atmos. Sci. 70 , 3959–3976 (2013).

Räisänen, J., Ruokolainen, L. & Ylhäisi, J. Weighting of model results for improving best estimates of climate change. Clim. Dynam. 35 , 407–422 (2010).

Lorenz, R. et al. Prospects and caveats of weighting climate models for summer maximum temperature projections over North America. J. Geophys. Res. Atmos. 123 , 4509–4526 (2018).

Abramowitz, G. et al. Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing. Earth Syst. Dynam. Discuss. https://doi.org/10.5194/esd-2018-51 (2018).

Herger, N. et al. Selecting a climate model subset to optimise key ensemble properties. Earth Syst. Dynam. 9 , 135–151 (2018).

Herger, N. et al. Calibrating climate model ensembles for assessing extremes in a changing climate. J. Geophys. Res. Atmos. 123 , 5988–6004 (2018).

Santer, B. D. et al. Incorporating model quality information in climate change detection and attribution studies. Proc. Natl Acad. Sci. USA 106 , 14778–14783 (2009).

Caldwell, P. M. et al. Statistical significance of climate sensitivity predictors obtained by data mining. Geophys. Res. Lett. 41 , 1803–1808 (2014).

Masson, D. & Knutti, R. Climate model genealogy. Geophys. Res. Lett. 38 , L08703 (2011).

Pennell, C. & Reichler, T. On the effective number of climate models. J. Clim. 24 , 2358–2367 (2011).

Sunyer, M. A., Madsen, H., Rosbjerg, D. & Arnbjerg-Nielsen, K. A Bayesian approach for uncertainty quantification of extreme precipitation projections including climate model interdependency and nonstationary bias. J. Clim. 27 , 7113–7132 (2014).

Sanderson, B. M., Knutti, R. & Caldwell, P. A representative democracy to reduce interdependency in a multimodel ensemble. J. Clim. 28 , 5171–5194 (2015).

Zhang, X. et al. Indices for monitoring changes in extremes based on daily temperature and precipitation data. WIREs Clim. Change 2 , 851–870 (2011).

Sillmann, J., Kharin, V. V., Zhang, X., Zwiers, F. W. & Bronaugh, D. Climate extremes indices in the CMIP5 multimodel ensemble. Part 1. Model evaluation in the present climate. J. Geophys. Res. Atmos. 118 , 1716–1733 (2013).

Pendergrass, A. G. & Hartmann, D. L. The atmospheric energy constraint on global-mean precipitation change. J. Clim. 27 , 757–768 (2014).

Zscheischler, J. et al. Future climate risk from compound events. Nat. Clim. Change 8 , 469–477 (2018).

Gutowski, W. J. Jr et al. WCRP COordinated Regional Downscaling EXperiment (CORDEX): a diagnostic MIP for CMIP6. Geosci. Model Dev. 9 , 4087–4095 (2016).

van Oldenborgh, G. J., Otto, F. E. L., Haustein, K. & Cullen, H. Climate change increases the probability of heavy rains like those of storm Desmond in the UK—an event attribution study in near-real time. Hydrol. Earth Syst. Sci. Discuss. 12 , 13197–13216 (2015).

Ruane, A. C., Goldberg, R. & Chryssanthacopoulos, J. Climate forcing datasets for agricultural modeling: merged products for gap-filling and historical climate series estimation. Agric. For. Meteorol. 200 , 233–248 (2015).

Elliott, J. et al. The Global Gridded Crop Model Intercomparison: data and modeling protocols for Phase 1 (v1.0). Geosci. Model Dev. 8 , 261–277 (2015).

Ruane, A. C. et al. The Vulnerability, Impacts, Adaptation and Climate Services Advisory Board (VIACS AB v1.0) contribution to CMIP6. Geosci. Model Dev. 9 , 3493–3515 (2016).

Warszawski, L. et al. The Inter-Sectoral Impact Model Intercomparison Project (ISI–MIP): project framework. Proc. Natl Acad. Sci. USA 111 , 3228–3232 (2014).

Ruane, A. C. & McDermid, S. P. Selection of a representative subset of global climate models that captures the profile of regional changes for integrated climate impacts assessment. Earth Perspect. 4 , 1 (2017).

Stevens, B. & Bony, S. What are climate models missing? Science 340 , 1053–1054 (2013).

Webb, M. J. et al. The Cloud Feedback Model Intercomparison Project (CFMIP) contribution to CMIP6. Geosci. Model Dev. 10 , 359–384 (2017).

Mears, C. A., Wentz, F. J., Thorne, P. & Bernie, D. Assessing uncertainty in estimates of atmospheric temperature changes from MSU and AMSU using a Monte-Carlo estimation technique. J. Geophys. Res. Atmos. 116 , D08112 (2011).

Argo Argo Float Data and Metadata from Global Data Assembly Centre (Argo GDAC) (SEANOE, 2000); https://doi.org/10.17882/42182

Malavelle, F. F. et al. Strong constraints on aerosol–cloud interactions from volcanic eruptions. Nature 546 , 485–491 (2017).

Reichstein, M. et al. Deep learning and process understanding for data-driven Earth system science. Nature (in the press).

Fountalis, I., Bracco, A. & Dovrolis, C. ENSO in CMIP5 simulations: network connectivity from the recent past to the twenty-third century. Clim. Dynam. 45 , 511–538 (2015).

Barz, B., Rodner, E., Guanche Garcia, Y. & Denzler, J. Detecting regions of maximal divergence for spatio-temporal anomaly detection. IEEE Trans. Pattern Anal. Mach. Intell. https://doi.org/10.1109/TPAMI.2018.2823766 (2018).

Runge, J. et al. Identifying causal gateways and mediators in complex spatio-temporal systems. Nat. Commun. 6 , 8502 (2015).

Kageyama, M. et al. The PMIP4 contribution to CMIP6 – Part 1: Overview and over-arching analysis plan. Geosci. Model Dev. 11 , 1033–1057 (2018).

Haarsma, R. J. et al. High Resolution Model Intercomparison Project (HighResMIP v1.0) for CMIP6. Geosci. Model Dev. 9 , 4185–4208 (2016).

IPCC Special Report on Global Warming of 1.5 °C (Cambridge Univ. Press, 2018).

Download references

Acknowledgements

The authors acknowledge the Aspen Global Change Institute (AGCI) for hosting a workshop on Earth System Model Evaluation to Improve Process Understanding in August 2017 as part of its traditionally landmark summer interdisciplinary sessions ( http://www.agci.org/event/17s2 ). NASA, the Heising-Simons Foundation, Horizon 2020 European Union’s Framework Programme for Research and Innovation under Grant Agreement No 641816, the Coordinated Research in Earth Systems and Climate: Experiments, kNowledge, Dissemination and Outreach (CRESCENDO) project, the ESA Climate Change Initiative (CCI) Climate Model User Group (CMUG), WCRP and the Department of Energy (DOE) all provided support for the workshop. The viewpoint presented here substantially draws on conclusions from that workshop. Portions of this study were supported by the Regional and Global Model Analysis (RGMA) component of the Earth and Environmental System Modeling Program of the US DOE Office of Biological & Environmental Research (BER) Cooperative Agreement DE-FC02-97ER62402 and Contract No. DE-AC05-00OR22725 and the National Science Foundation. NCAR is sponsored by the National Science Foundation.

Author information

Authors and affiliations.

Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany

Veronika Eyring & Bettina K. Gier

University of Bremen, Institute of Environmental Physics (IUP), Bremen, Germany

College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK

Peter M. Cox & Mark S. Williamson

Canadian Centre for Climate Modelling and Analysis, Environment and Climate Change Canada, Victoria, British Columbia, Canada

Gregory M. Flato

Program for Climate Model Diagnosis and Intercomparison (PCMDI), Lawrence Livermore National Laboratory, Livermore, CA, USA

Peter J. Gleckler, Peter Caldwell, Stephen A. Klein & Benjamin D. Santer

Climate Change Research Centre and ARC Centre of Excellence for Climate Extremes, University of New South Wales, Sydney, New South Wales, Australia

Gab Abramowitz & Steven C. Sherwood

Climate and Ecosystem Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA, USA

William D. Collins

Department of Earth and Planetary Science, University of California, Berkeley, CA, USA

University of California, Los Angeles, CA, USA

Alex D. Hall

Computational Earth Sciences Group and Climate Change Science Institute, Oak Ridge National Laboratory, Oak Ridge, TN, USA

Forrest M. Hoffman

Department of Civil and Environmental Engineering, University of Tennessee, Knoxville, TN, USA

Department of Geographical Sciences, University of Maryland, College Park, MD, USA

George C. Hurtt

Department of Atmospheric and Oceanic Sciences and Institute of Arctic and Alpine Research, University of Colorado, Boulder, CO, USA

Alexandra Jahn

Met Office Hadley Centre, Exeter, UK

Chris D. Jones

Geophysical Fluid Dynamics Laboratory, NOAA, Princeton, NJ, USA

John P. Krasting

Laboratoire de Météorologie Dynamique (LMD), IPSL, Paris, France

Lester Kwiatkowski

Institute for Atmospheric and Climate Science, ETH Zurich, Zurich, Switzerland

Ruth Lorenz

Cooperative Institute for Research in Environmental Sciences, University of Colorado, Boulder, CO, USA

Eric Maloney & Robert Pincus

National Center for Atmospheric Research (NCAR), Boulder, CO, USA

Gerald A. Meehl, Angeline G. Pendergrass, Benjamin M. Sanderson & Isla R. Simpson

NASA Goddard Institute for Space Studies, New York, NY, USA

Alex C. Ruane

University of Arizona, Tucson, AZ, USA

Joellen L. Russell & Ronald J. Stouffer

You can also search for this author in PubMed   Google Scholar

Contributions

V.E., P.M.C., G.M.F. and P.J.G. were the co-chairs of the AGCI workshop and led the writing of the paper. All authors participated in the AGCI workshop and contributed to discussions and writing of the text.

Corresponding author

Correspondence to Veronika Eyring .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Eyring, V., Cox, P.M., Flato, G.M. et al. Taking climate model evaluation to the next level. Nature Clim Change 9 , 102–110 (2019). https://doi.org/10.1038/s41558-018-0355-y

Download citation

Received : 25 February 2018

Accepted : 08 November 2018

Published : 07 January 2019

Issue Date : February 2019

DOI : https://doi.org/10.1038/s41558-018-0355-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Assessment of current and future trends in water resources in the gambia river basin in a context of climate change.

• Serigne Mory Khouma Séne
• Cheikh Faye
• Chaitanya B. Pande

Environmental Sciences Europe (2024)

Mapping the global distribution of C4 vegetation using observations and optimality theory

• Xiangzhong Luo
• Haoran Zhou
• Christopher J. Still

Nature Communications (2024)

Future risk of decadal megadrought events over eastern China based on IPO-constrained precipitation

Climate Dynamics (2024)

Revising Alpine summer temperatures since 881 CE

• Eileen Kuhl
• Claudia Hartl

Assessing future changes in daily precipitation tails over India: insights from multimodel assessment of CMIP6 GCMs

• Sagar Rohidas Chavan

Theoretical and Applied Climatology (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

•  We're Hiring!
•  Help Center

System modeling

• Most Cited Papers
• Most Downloaded Papers
• Newest Papers
• Last »
• Model Based Design Follow Following
• SCADA Follow Following
• Philosophy of Nursing Follow Following
• Public Health Nursing Follow Following
• Holistic Health Follow Following
• Contingency Management Follow Following
• Fuzzy Logic Follow Following
• E-Learning Materials Development Follow Following
• Unified Modeling Language Follow Following
• Dynamic Model of WSN Follow Following

Enter the email address you signed up with and we'll email you a reset link.

• Academia.edu Publishing
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Research on the smart broad bean harvesting system and the self-adaptive control method based on cps technologies.

1. Introduction

2. overall framework, 3. key technologies, 3.1. intelligent perception environment configuration of the sbhs, 3.2. digital twin model construction for the sbhs.

• Deduplication involves filtering out spatially repetitive data generated by multiple sensors covering the same area, ensuring that the same event is represented by a single information entry.
• Noise reduction is the process of directly filtering out uninteresting data to retain only those that are useful or of interest to the harvesting operator.
• Heterogeneous value standardization aims to unify data from different sensors with different structures using a standardized descriptive language, such as XML, to facilitate the easy reading of information.
• Missing value fill is carried out by filling in missing numerical values through methods such as default value filling, mean value filling, mode filling, KNN imputation, and predicting missing values as a new label using a model, to ensure the continuity and effectiveness of the data.
• Association analysis is conducted by using correlation mining techniques on the data from the harvesting site to identify frequent patterns, associations, correlations, or causal structures between collections or sets of harvesting resources.
• Combination involves grouping similar schemes together by assigning unified tags, which facilitates subsequent effective analysis and management of similar resources.
• Integration refers to the consolidation of different types of harvesting resources, enabling the overall broad bean harvesting system to be uniformly scheduled and managed in a coordinated manner.
• Mapping relationships ensure that there is synchronization between the physical and virtual spaces, guaranteeing that the digital twin system model can accurately depict the operational conditions within the physical space.

3.3. Colored Petri Net-Based Self-Adaptive Analysis and Optimization of the Harvesting Process

• Obtain the overall structure of the harvesting system and transform all the high-level resources into CPN elements to create a main CPN model in the cloud platform. In general, a whole harvesting system can be divided into five kinds of resources, i.e., harvester, broad bean, growth environment, bean collector and deliverer. Each kind of resource can be seen as an abstract element in the overall CPN model, and each element can be replaced by a detailed sub-model.
• Further analyze the activities of the high-level resources and create a detailed sub-CPN model for each resource or sub-systems. These sub-CPN models can be deployed on the distributed resource side and just set a connection with the high-level elements in the cloud platform.
• Repeat the second procedure until any element is described in detail. There are many elements participating in the harvesting system, and many performance indicators can be monitored, construct a CPN model that considers all the status or elements in the real-life system is impossible and useless. Thus, this article mainly considers the harvesting efficiency, broad bean loss rate, and a hierarchical structure is set, where the elements in higher-level CPN model can be replaced by a detailed sub-model.
• Set arcs, guard functions, and token delivery rules between different places and transitions, so that the CPN model can reflect the practical workflow of the harvesting system.
• Set communication elements between the virtual CPN models and the practical sensors in real-life system, and bind the intelligent tokens with real-life resources so that the virtual CPN models can change their status with the real harvesting system.
• Simulate the CPN models to find the potential deadlocks or other shut-down problems, and then set measures to prevent them. For example, if two harvesters call one broad bean collector, one deadlock happens, then one priority can be given to one harvester according to distance, and the deadlock can be removed. After all the potential problems are solved, the CPN models can work with liveness.

4. Case Study

4.1. case scenerio.

• The intelligent broad bean sensing environment was constructed by configuring different sensors, and key information can be captured, including soil moisture, weather, the maturity of broad beans, and the height of bean pods.
• The smart broad bean harvesting machines were equipped with embedded edge–cloud control devices. Mainly three kinds of machines are considered, the harvesters for harvesting the broad bean in field, collectors for collecting beans from different harvesters, and deliverer for move the beans to the warehouse. On the one hand, the real-time status of the itself can be monitored and analyzed, such as the height of cutting table, the location, and the speed. On the other hand, adaptive adjustment of the harvesting cutting table height and autonomous planning of the harvesting path can be achieved by themselves.
• The edge control devices can sense the raw status of the harvesters, pre-integrate the raw sensed data, so that the data volume can be largely reduced and only key information can be uploaded to cloud control center. At the same time, cloud control directives can be converted to the command that can be executed by the harvester.
• The cloud control center is capable of providing optimal harvesting time information based on the key information provided by the intelligent broad bean sensing environment. It formulates optimized decision making for the harvesting path of broad beans based on the geographical information of the bean growth environment.

4.2. CPN Model-Based Self-Adaptive Analysis and Control for the SBHS

4.3. the management and control platform for the sbhs, 5. conclusions, author contributions, data availability statement, conflicts of interest.

• Akilan, T.; Baalamurugan, K.M. Automated Weather Forecasting and Field Monitoring Using GRU-CNN Model along with IoT to Support Precision Agriculture. Expert Syst. Appl. 2024 , 249 , 123468. [ Google Scholar ] [ CrossRef ]
• Guo, Z.; Zhang, Y.; Zhao, X.; Song, X. CPS-Based Self-Adaptive Collaborative Control for Smart Production-Logistics Systems. IEEE Trans. Cybern. 2021 , 51 , 188–198. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Sharma, V.; Tripathi, A.K.; Mittal, H. Technological Revolutions in Smart Farming: Current Trends, Challenges & Future Directions. Comput. Electron. Agric. 2022 , 201 , 107217. [ Google Scholar ]
• Liu, B.; Zhang, Y.; Lv, J.; Majeed, A.; Chen, C.H.; Zhang, D. A Cost-Effective Manufacturing Process Recognition Approach Based on Deep Transfer Learning for CPS Enabled Shop-Floor. Robot. Comput. Integr. Manuf. 2021 , 70 , 102128. [ Google Scholar ] [ CrossRef ]
• Ma, S.; Huang, Y.; Liu, Y.; Kong, X.; Yin, L.; Chen, G. Edge-Cloud Cooperation-Driven Smart and Sustainable Production for Energy Intensive Manufacturing Industries. Appl. Energy 2023 , 337 , 120843. [ Google Scholar ] [ CrossRef ]
• Xu, G.; Feng, Z.; Guo, E.; Cai, C.; Ding, H. Multi-Objective Optimization of Hydraulic Shovel Using Evolutionary Algorithm. Autom. Constr. 2022 , 142 , 104486. [ Google Scholar ] [ CrossRef ]
• Pagano, A.; Croce, D.; Tinnirello, I.; Vitale, G. A Survey on LoRa for Smart Agriculture: Current Trends and Future Perspectives. IEEE Internet Things J. 2023 , 10 , 3664–3679. [ Google Scholar ] [ CrossRef ]
• Hu, T.; Wang, W.; Gu, J.; Xia, Z.; Zhang, J.; Wang, B. Research on Apple Object Detection and Localization Method Based on Improved YOLOX and RGB-D Images. Agronomy 2023 , 13 , 1816. [ Google Scholar ] [ CrossRef ]
• Wang, H.; Gu, J.; Wang, M. A Review on the Application of Computer Vision and Machine Learning in the Tea Industry. Front. Sustain. Food Syst. 2023 , 7 , 1172543. [ Google Scholar ] [ CrossRef ]
• Yang, N.; Chang, K.; Dong, S.; Tang, J.; Wang, A.; Huang, R.; Jia, Y. Rapid Image Detection and Recognition of Rice False Smut Based on Mobile Smart Devices with Anti-Light Features from Cloud Database. Biosyst. Eng. 2022 , 218 , 229–244. [ Google Scholar ] [ CrossRef ]
• Xia, X.; Yang, G.; Chen, Q.; Song, Z.; Mei, S.; Jin, Y. Analysis on Loss Reduction Technology and Strategy for Combined Harvesting Process of Food Legumes. J. Chin. Agric. Mech. 2022 , 43 , 13. [ Google Scholar ] [ CrossRef ]
• Wang, W.; Zhang, Y.; Gu, J.; Wang, J. A Proactive Manufacturing Resources Assignment Method Based on Production Performance Prediction for the Smart Factory. IEEE Trans. Industr Inform. 2022 , 18 , 46–55. [ Google Scholar ] [ CrossRef ]
• Wang, W.; Zhang, Y.; Zhong, R.Y. A Proactive Material Handling Method for CPS Enabled Shop-Floor. Robot. Comput. Integr. Manuf. 2020 , 61 , 101849. [ Google Scholar ] [ CrossRef ]
• Zhou, M.; Fakayode, O.A.; Ren, M.; Li, H.; Liang, J.; Zhou, C. Green and Sustainable Extraction of Lignin by Deep Eutectic Solvent, Its Antioxidant Activity, and Applications in the Food Industry. Crit. Rev. Food Sci. Nutr. 2023 , 1–27. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Rudrakar, S.; Rughani, P. IoT Based Agriculture (Ag-IoT): A Detailed Study on Architecture, Security and Forensics. Inf. Process. Agric. 2023 . [ Google Scholar ] [ CrossRef ]
• Ruan, J.; Wang, Y.; Chan, F.T.S.; Hu, X.; Zhao, M.; Zhu, F.; Shi, B.; Shi, Y.; Lin, F. A Life Cycle Framework of Green IoT-Based Agriculture and Its Finance, Operation, and Management Issues. IEEE Commun. Mag. 2019 , 57 , 90–96. [ Google Scholar ] [ CrossRef ]
• Sheng, Y.; Zhang, G.; Zhang, Y.; Luo, M.; Pang, Y.; Wang, Q. A Multimodal Data Sensing and Feature Learning-Based Self-Adaptive Hybrid Approach for Machining Quality Prediction. Adv. Eng. Inform. 2024 , 59 , 102324. [ Google Scholar ] [ CrossRef ]
• Kang, H.; Chen, C. Fruit Detection, Segmentation and 3D Visualisation of Environments in Apple Orchards. Comput. Electron. Agric. 2020 , 171 , 105302. [ Google Scholar ] [ CrossRef ]
• Tang, Y.; Zhou, H.; Wang, H.; Zhang, Y. Fruit Detection and Positioning Technology for a Camellia oleifera C. Abel Orchard Based on Improved YOLOv4-Tiny Model and Binocular Stereo Vision. Expert Syst. Appl. 2023 , 211 , 118573. [ Google Scholar ] [ CrossRef ]
• Lei, X.; Wu, M.; Li, Y.; Liu, A.; Tang, Z.; Chen, S.; Xiang, Y. Detection and Positioning of Camellia oleifera Fruit Based on LBP Image Texture Matching and Binocular Stereo Vision. Agronomy 2023 , 13 , 2153. [ Google Scholar ] [ CrossRef ]
• Rad, C.-R.; Hancu, O.; Takacs, I.-A.; Olteanu, G. Smart Monitoring of Potato Crop: A Cyber-Physical System Architecture Model in the Field of Precision Agriculture. Agric. Agric. Sci. Procedia 2015 , 6 , 73–79. [ Google Scholar ] [ CrossRef ]
• Ouafiq, E.M.; Saadane, R.; Chehri, A.; Jeon, S. AI-Based Modeling and Data-Driven Evaluation for Smart Farming-Oriented Big Data Architecture Using IoT with Energy Harvesting Capabilities. Sustain. Energy Technol. Assess. 2022 , 52 , 102093. [ Google Scholar ] [ CrossRef ]
• Ma, S.; Ding, W.; Liu, Y.; Ren, S.; Yang, H. Digital Twin and Big Data-Driven Sustainable Smart Manufacturing Based on Information Management Systems for Energy-Intensive Industries. Appl. Energy 2022 , 326 , 119986. [ Google Scholar ] [ CrossRef ]
• Mitsanis, C.; Hurst, W.; Tekinerdogan, B. A 3D Functional Plant Modelling Framework for Agricultural Digital Twins. Comput. Electron. Agric. 2024 , 218 , 108733. [ Google Scholar ] [ CrossRef ]
• Kalyani, Y.; Bermeo, N.V.; Collier, R. Digital Twin Deployment for Smart Agriculture in Cloud-Fog-Edge Infrastructure. Int. J. Parallel Emergent Distrib. Syst. 2023 , 38 , 461–476. [ Google Scholar ] [ CrossRef ]
• Alves, R.G.; Maia, R.F.; Lima, F. Development of a Digital Twin for Smart Farming: Irrigation Management System for Water Saving. J. Clean. Prod. 2023 , 388 , 135920. [ Google Scholar ] [ CrossRef ]
• Manocha, A.; Sood, S.K.; Bhatia, M. IoT-Digital Twin-Inspired Smart Irrigation Approach for Optimal Water Utilization. Sustain. Comput. Inform. Syst. 2024 , 41 , 100947. [ Google Scholar ] [ CrossRef ]
• Huang, J.; Zhang, F.; Song, J.; Li, W. An Agent-Based Simulation Model of Wheat Market Operation: The Benefit of Support Price. J. Syst. Sci. Syst. Eng. 2022 , 31 , 437–456. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lin, Q.; Zheng, P.; Zhang, Y.; Xia, L.; Zhang, Z.; Liang, J. Establishing a Dynamic and Static Knowledge Model of the Manufacturing Cell Management System: An Active Push Approach. Expert Syst. Appl. 2024 , 243 , 122713. [ Google Scholar ] [ CrossRef ]
• Zhang, H.; Li, X.; Wang, L.; Liu, D.; Wang, S. Construction and Optimization of a Collaborative Harvesting System for Multiple Robotic Arms and an End-Picker in a Trellised Pear Orchard Environment. Agronomy 2024 , 14 , 80. [ Google Scholar ] [ CrossRef ]
• Ren, G.; Wu, T.; Lin, T.; Yang, L.; Chowdhary, G.; Ting, K.C.; Ying, Y. Mobile Robotics Platform for Strawberry Sensing and Harvesting within Precision Indoor Farming Systems. J. Field Robot. 2023 , 1–19. [ Google Scholar ] [ CrossRef ]
• Zamora-Izquierdo, M.A.; Santa, J.; Martínez, J.A.; Martínez, V.; Skarmeta, A.F. Smart Farming IoT Platform Based on Edge and Cloud Computing. Biosyst. Eng. 2019 , 177 , 4–17. [ Google Scholar ] [ CrossRef ]
• Zhang, S.; Wei, X.; Deng, Y.; Zang, C.; Ji, X.; Wang, A. Design and Experiments of the Whole Field Path Tracking Algorithm for a Track-Based Harvester. Nongye Gongcheng Xuebao/Trans. Chin. Soc. Agric. Eng. 2023 , 39 , 36–45. [ Google Scholar ] [ CrossRef ]
• Dai, L.L.; Pan, Q.K.; Miao, Z.H.; Suganthan, P.N.; Gao, K.Z. Multi-Objective Multi-Picking-Robot Task Allocation: Mathematical Model and Discrete Artificial Bee Colony Algorithm. IEEE Trans. Intell. Transp. Syst. 2023 , 25 , 6061–6073. [ Google Scholar ] [ CrossRef ]
• Zhang, Y.; Wang, W.; Wu, N.; Qian, C. IoT-enabled Real-time Production Performance Analysis and Exception Diagnosis Model. IEEE Trans. Autom. Eng. 2016 , 13 , 1318–1332. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Wang, W.; Yang, S.; Zhang, X.; Xia, X. Research on the Smart Broad Bean Harvesting System and the Self-Adaptive Control Method Based on CPS Technologies. Agronomy 2024 , 14 , 1405. https://doi.org/10.3390/agronomy14071405

Wang W, Yang S, Zhang X, Xia X. Research on the Smart Broad Bean Harvesting System and the Self-Adaptive Control Method Based on CPS Technologies. Agronomy . 2024; 14(7):1405. https://doi.org/10.3390/agronomy14071405

Wang, Wenbo, Shaojun Yang, Xinzhou Zhang, and Xianfei Xia. 2024. "Research on the Smart Broad Bean Harvesting System and the Self-Adaptive Control Method Based on CPS Technologies" Agronomy 14, no. 7: 1405. https://doi.org/10.3390/agronomy14071405

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• {{subColumn.name}}

AIMS Mathematics

• {{newsColumn.name}}
• Share facebook twitter google linkedin

Research on investment evaluation of highway projects based on system dynamics model

• Yonghua Liu 1 , 
• Hao Deng 1 , 
• Hanqi Gao 1 ,  ,  , 
• 1. Faculty of Transportation Engineering, Kunming University of Science and Technology, Yunnan Kunming 650504, China
• 2. Yunnan Transportation Research Institute Co., Kunming 650011, China
• Received: 24 March 2024 Revised: 02 May 2024 Accepted: 27 May 2024 Published: 21 June 2024

MSC : 91-10

• Full Text(HTML)
• Download PDF

Aiming at the deficiencies presented by the traditional methods of highway project investment evaluation, the proposed highway investment evaluation method was based on system dynamics. First, we constructed an evaluation index system from profitability, solvency, and risk resistance and clarified the positive and negative causality within the investment evaluation system of highway projects; second, we determined the boundaries of the system dynamics model and divided it into six sub-systems, namely, income, cash flow, investment evaluation, profit, cost, investment and financing, and liabilities; and then, we established the system dynamics model of highway investment evaluation based on the sub-systems. The model made up for the limitations of the traditional discounted cash flow method; finally, taking the China's Yunnan Province an Expressway project as an example, using VENSIM software simulation, we get the evaluation results of the system dynamics model and make a comparative analysis with the discounted cash flow method, which showed that the calculation inaccuracies of the NPV and other financial indicators were in a reasonable range, and the evaluation method had strong operability and practicability. The system dynamics investment evaluation model provided a systematic, intuitive, whole-process investment evaluation method, which provided a theoretical basis for the analysis and decision-making of the investment effect of highway projects.

• transportation economics ,
• investment appraisal ,
• system dynamics ,
• discounted cash flow approach

Citation: Yonghua Liu, Hao Deng, Hanqi Gao, Wei Ni. Research on investment evaluation of highway projects based on system dynamics model[J]. AIMS Mathematics, 2024, 9(8): 20326-20349. doi: 10.3934/math.2024989

Related Papers:

• This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/ -->

Supplements

Access History

• Corresponding author: Email: [email protected] ; 

Reader Comments

• © 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/4.0 )

通讯作者: 陈斌, [email protected]

沈阳化工大学材料科学与工程学院 沈阳 110142

Article views( 133 ) PDF downloads( 44 ) Cited by( 0 )

Figures and Tables

Figures( 22 )  /  Tables( 7 )

Associated material

Other articles by authors.

• Yonghua Liu

Related pages

• on Google Scholar
• Email to a friend
• Order reprints

Export File

• Figure 2.1. Causal feedback loops for major factors
• Figure 2.2. Flow chart of the investment appraisal system for highway projects
• Figure 3.1. Project toll revenue simulation
• Figure 3.2. Simulation of total project income
• Figure 3.3. Simulation of project operating costs
• Figure 3.4. Simulation of project financial costs
• Figure 3.5. Simulation of total project costs and expenses
• Figure 3.6. Total project costs and expenses vs. total revenue
• Figure 3.7. Simulation of project net profit
• Figure 3.8. Simulation of project cash inflows and outflows
• Figure 3.9. Project net cash flow after tax
• Figure 3.10. Simulation of the cumulative after-tax net present value of the project
• Figure 3.11. Simulation of project ROA, ROI, and ROE
• Figure 3.12. Project ICR simulation
• Figure 3.13. Project DSCR simulation
• Figure 3.14. Project LOAR simulation
• Figure 3.15. Sensitivity analysis of toll revenue
• Figure 3.16. Operating cost sensitivity analysis
• Figure 3.17. Mixed-factor sensitivity analysis
• Figure 3.18. Cumulative NPV simulation for extended toll periods
• Figure 3.19. After-tax net cash flow inaccuracy analysis
• Figure 3.20. Cumulative after-tax NPV inaccuracy analysis

Suggestions or feedback?

MIT News | Massachusetts Institute of Technology

• Machine learning
• Social justice
• Black holes
• Classes and programs

Departments

• Aeronautics and Astronautics
• Brain and Cognitive Sciences
• Architecture
• Political Science
• Mechanical Engineering

Centers, Labs, & Programs

• Abdul Latif Jameel Poverty Action Lab (J-PAL)
• Picower Institute for Learning and Memory
• Lincoln Laboratory
• School of Architecture + Planning
• School of Engineering
• School of Humanities, Arts, and Social Sciences
• Sloan School of Management
• School of Science
• MIT Schwarzman College of Computing

Technique improves the reasoning capabilities of large language models

Press contact :, media download.

*Terms of Use:

Images for download on the MIT News office website are made available to non-commercial entities, press and the general public under a Creative Commons Attribution Non-Commercial No Derivatives license . You may not alter the images provided, other than to crop them to size. A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT."

Previous image Next image

Large language models like those that power ChatGPT have shown impressive performance on tasks like drafting legal briefs, analyzing the sentiment of customer reviews, or translating documents into different languages.

These machine-learning models typically use only natural language to process information and answer queries, which can make it difficult for them to perform tasks that require numerical or symbolic reasoning.

For instance, a large language model might be able to memorize and recite a list of recent U.S. presidents and their birthdays, but that same model could fail if asked the question “Which U.S. presidents elected after 1950 were born on a Wednesday?” (The answer is Jimmy Carter.)

Researchers from MIT and elsewhere have proposed a new technique that enables large language models to solve natural language, math and data analysis, and symbolic reasoning tasks by generating programs.

Their approach, called natural language embedded programs (NLEPs), involves prompting a language model to create and execute a Python program to solve a user’s query, and then output the solution as natural language.

They found that NLEPs enabled large language models to achieve higher accuracy on a wide range of reasoning tasks. The approach is also generalizable, which means one NLEP prompt can be reused for multiple tasks.

NLEPs also improve transparency, since a user could check the program to see exactly how the model reasoned about the query and fix the program if the model gave a wrong answer.

“We want AI to perform complex reasoning in a way that is transparent and trustworthy. There is still a long way to go, but we have shown that combining the capabilities of programming and natural language in large language models is a very good potential first step toward a future where people can fully understand and trust what is going on inside their AI model,” says Hongyin Luo PhD ’22, an MIT postdoc and co-lead author of a paper on NLEPs .

Luo is joined on the paper by co-lead authors Tianhua Zhang, a graduate student at the Chinese University of Hong Kong; and Jiaxin Ge, an undergraduate at Peking University; Yoon Kim, an assistant professor in MIT’s Department of Electrical Engineering and Computer Science and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); senior author James Glass, senior research scientist and head of the Spoken Language Systems Group in CSAIL; and others. The research will be presented at the Annual Conference of the North American Chapter of the Association for Computational Linguistics.

Problem-solving with programs

Many popular large language models work by predicting the next word, or token, given some natural language input. While models like GPT-4 can be used to write programs, they embed those programs within natural language, which can lead to errors in the program reasoning or results.

With NLEPs, the MIT researchers took the opposite approach. They prompt the model to generate a step-by-step program entirely in Python code, and then embed the necessary natural language inside the program.

An NLEP is a problem-solving template with four steps. First, the model calls the necessary packages, or functions, it will need to solve the task. Step two involves importing natural language representations of the knowledge the task requires (like a list of U.S. presidents’ birthdays). For step three, the model implements a function that calculates the answer. And for the final step, the model outputs the result as a line of natural language with an automatic data visualization, if needed.

“It is like a digital calculator that always gives you the correct computation result as long as the program is correct,” Luo says.

The user can easily investigate the program and fix any errors in the code directly rather than needing to rerun the entire model to troubleshoot.

The approach also offers greater efficiency than some other methods. If a user has many similar questions, they can generate one core program and then replace certain variables without needing to run the model repeatedly.

To prompt the model to generate an NLEP, the researchers give it an overall instruction to write a Python program, provide two NLEP examples (one with math and one with natural language), and one test question.

“Usually, when people do this kind of few-shot prompting, they still have to design prompts for every task. We found that we can have one prompt for many tasks because it is not a prompt that teaches LLMs to solve one problem, but a prompt that teaches LLMs to solve many problems by writing a program,” says Luo.

“Having language models reason with code unlocks many opportunities for tool use, output validation, more structured understanding into model's capabilities and way of thinking, and more,” says Leonid Karlinsky, principal scientist at the MIT-IBM Watson AI Lab.

“No magic here”

NLEPs achieved greater than 90 percent accuracy when prompting GPT-4 to solve a range of symbolic reasoning tasks, like tracking shuffled objects or playing a game of 24, as well as instruction-following and text classification tasks. The researchers found that NLEPs even exhibited 30 percent greater accuracy than task-specific prompting methods. The method also showed improvements over open-source LLMs. 

Along with boosting the accuracy of large language models, NLEPs could also improve data privacy. Since NLEP programs are run locally, sensitive user data do not need to be sent to a company like OpenAI or Google to be processed by a model.

In addition, NLEPs can enable small language models to perform better without the need to retrain a model for a certain task, which can be a costly process.

“There is no magic here. We do not have a more expensive or fancy language model. All we do is use program generation instead of natural language generation, and we can make it perform significantly better,” Luo says.

However, an NLEP relies on the program generation capability of the model, so the technique does not work as well for smaller models which have been trained on limited datasets. In the future, the researchers plan to study methods that could make smaller language models generate more effective NLEPs. In addition, they want to investigate the impact of prompt variations on NLEPs to enhance the robustness of the model’s reasoning processes.

This research was supported, in part, by the Center for Perceptual and Interactive Intelligence of Hong Kong. 

Share this news article on:

Related links.

• Hongyin Luo
• Computer Science and Artificial Intelligence Laboratory (CSAIL)
• Department of Electrical Engineering and Computer Science

Related Topics

• Computer science and technology
• Artificial intelligence
• Programming
• Electrical Engineering & Computer Science (eecs)

Related Articles

Natural language boosts LLM performance in coding, planning, and robotics

Large language models use a surprisingly simple mechanism to retrieve some stored knowledge

Explained: Generative AI

Learning to grow machine-learning models

Previous item Next item

More MIT News

Faces of MIT: Anthony Hallee-Farrell '13

Read full story →

From group stretches to “Hitting Roman,” MIT Motorsports traditions live on

Creating the crossroads

Study reveals why AI models that analyze medical images can be biased

Leaning into the immune system’s complexity

Scientists use computational modeling to guide a difficult chemical synthesis

• More news on MIT News homepage →

Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA

• Map (opens in new window)
• Events (opens in new window)
• People (opens in new window)
• Careers (opens in new window)
• Accessibility
• Social Media Hub
• MIT on Facebook
• MIT on YouTube
• MIT on Instagram

Development of a human cognition inspired condition management system for equipment

• ORIGINAL ARTICLE
• Open access
• Published: 28 June 2024

Cite this article

You have full access to this open access article

• Maneesh Singh   ORCID: orcid.org/0009-0008-3496-092X 1 ,
• Knut Øvsthus 1 ,
• Anne-Lena Kampen 1 &
• Hariom Dhungana 1  

Biomimicry is an approach for solving industrial challenges by studying similar cases in nature and emulating bio-organisms’ responses. Thus, it helps to solve modern day technological problems using the solutions that bio-organisms have successfully used over the course of millions of years. In an ongoing research project, investigations are being carried out to explore the use of biomimicry approach for developing a framework for a human-centric condition management system . This framework is inspired by the knowledge of human cognition. It is expected that the system will be able to utilize various data and integrate it with analytical models and knowledge-based systems to help an equipment diagnose and recommend optimised operation and maintenance programs. This paper describes the proposed framework for this human-centric condition management system .

Avoid common mistakes on your manuscript.

1 Introduction

Biomimicry is an approach for solving industrial challenges by studying similar cases in nature and emulating bio-organisms’ responses. Thus, the solutions that have been successfully developed and tested by bio-organism over the course of millions of years are used as inspiration to solve modern day technological problems.

In the last few years, biomimicry has generated considerable research interests. The concepts developed from these studies have helped in improving efficiency of various equipment and structures. Some successful examples of application of biomimicry include (Institute 2023 ):

Shikansen Bullet Train (Japan) – Modelling of the frontend of the train after the beak of kingfishers to reduce air drag and noise.

Eastgate Building (Harare, Zimbabwe) – Modelling of internal climate control system after termite mounds to allow natural draft of air.

“Painless” needles – Modelling of injection needles after mosquito’s proboscis (mouth) to make easy insertion needles.

Underwater communication – Modelling after dolphins’ underwater communication method to develop reliable multi-frequency data transmission.

An important aspect of survival of bio-organisms is their ability to regulate their processes with changing environment and to protect-maintain themselves against internal and external attacks. Learning cognitive process by which bio-organisms carry out these tasks can help in developing a cognitive layer for a condition management system . Such a system can help in self-preservation by enabling an equipment to diagnose and recommend optimised operation and maintenance programs.

This paper presents the ideas behind an ongoing project that intends to develop a condition management system (diagnosing and recommending optimised operation and maintenance programs) inspired by human cognitive system, for assets for manufacturing, process and infrastructure industries.

2 Proposed framework

Human body is continuously subjected to numerous attacks; and to protect and maintain itself it follows a number of routines. It does so by adopting protective and maintenance measures, like:

Identification of threats (who, where, how) using sight, hearing, smell;

Identification of attacks (who, where, how) using touch and taste;

Analysis of damage caused by understanding pain, blood flow, etc.;

Identification of failures to protect itself from threat by understanding blood flow, sickness, etc.;

Repair and replacement by generating antibodies, blood clotting, repair of tissues;

Condition assessment to check the progress of repair and replacement.

A close study of human body shows that is has distinct similarities to cyber-physical systems. Both have:

Physical body (human body vs. equipment / structure)

Sensors for collecting data (sense organs vs. sensors)

Data transfer mechanism (nervous system vs. digital data transportation and storage)

Memory and analytics section (brain vs. data analytics)

Thus, it may be possible to learn how the human body protects and maintains itself against external attacks and apply those concepts for developing a condition management system for assets (equipment or structures).

The condition management system for equipment / structure can possibly be developed by systematically:

Connecting physical process (replicating body), monitoring systems (replicating sense organs), data transmission networks (replicating neural network) and decision support system (replicating brain).

Considering (a) equipment; (b) operating conditions; (c) vulnerabilities; (d) potential threats-attacks-damages; (e) condition monitoring; (f) failure profile; and (g) inspection-maintenance planning.

Providing decision support (classical and AI based) by considering various aspects like:

prioritization of reliable sensor data under different operating conditions;

rejection of irrelevant or irrational data;

analysis (processing-reasoning) of data;

rationalizing-interpretation-elaboration of data;

learning from history for continuous improvement.

Figure  1 shows the schematic representation of the framework that is inspired by the working of human brain.

Framework for biologically inspired condition management of industrial processes

A brain consists primarily of two parts – “old brain” and “new brain” (neocortex or “new outer layer”). All animals have the old brain that is made up of many structurally different components or organs performing specific tasks, for example, premeditation and impulsive aggression (amygdala), basic movements (spinal cord), and digestion and breathing (brain stem). In addition to the old brain, mammals have an additional component, often referred as the “new brain” or neocortex. Unlike old brain, neocortex is composed of one large structure that looks similar throughout, but different regions of the neocortex perform different functions related to vision, hearing, touch, speech, taste, thought, etc. (Hawkins 2021 ).

In the proposed framework, inspired by the working of human brain, there are two distinct components – Primary Component (corresponding to the “old brain”) and the Secondary Component (corresponding to the “new brain” or neocortex).

The study of human cognition is extremely vast and complicated. Over the decades, multiple levels of analysis covering various aspects of biological, neurological, sociological and functional human behaviours have been carried out to understand it. While we are aware that a number of parallel and conflicting theories have been proposed to explain different aspects of human cognition system, we have often taken simplified versions of these theories and at times adopted a mix and match approach while using them. Thus, while appreciating the complexity of the subject, we have taken inspirations from human cognition for developing a framework for condition management system . This work is still under development and many important features are still missing.

2.1 Structural attributes

Every physical object, biological or equipment, has a unique structure that is determined by its genes or construction / manufacturing. Structure of an equipment can be characterized based on its physical dimensions, materials of construction, protections (example, coatings, and linings), insulations, lubrications, etc. These structures also come equipped with sense organs / sensors that help the organism / equipment experience its health conditions. For example, in offshore wind turbine parameters like generator bearing temperature, hydraulic oil temperature and gearbox oil temperature can help to diagnose components’ health.

2.2 Environmental attributes

Every physical object, biological or equipment, operates in an environment that not only influences its performance, but also subjects it to a number of several environmental attacks by degrading mechanisms, like corrosion and erosion. These degrading attacks may damage and significantly reduce the structural integrity of process equipment or structure. For example, operating environment parameters like wind speed, ambient temperature, pressure, and humidity can influence the degradation characteristics of an offshore wind turbine.

2.3 Operational attributes

During their operational lifetime, degrading mechanisms, like wear, tear, and deformation, subjects a bio-organism or an equipment to a number of operational attacks. These attacks target the vulnerabilities of the equipment resulting in damages. For example, in an offshore wind turbine some of the operating parameters are rotor pitch angle, rotor RPM and generator RPM.

In cognitive science, memory is classified into different types based on the duration and function of the stored information [Atkinson and Shiffrin, 1971]. Three main types of memory commonly studied are sensory memory, working memory, and long-term memory. It is seen as the storehouse of experiences, and the number of models within memory correlates with the depth of those experiences. More profound experiences lead to a greater number of models, which in turn contribute to information production, knowledge generation, and decision-making processes. Relationships between different pieces of information are referred to as knowledge. Each knowledge is linked to different tasks within memory. Every individual piece of knowledge is stored as a task and encoded with a memory model. Specifically, in quick decision making from old brain we use implicit memory [Schacter, 1987]. In Procedural Decision Making, memories are in various forms, including, threshold values for fault detection, patterns for signature recognition for fault identification and rules for decision making.

2.5 Continuous learning

Neocortex continuously learns new models by registering environmental changes and making or updating the models to reflect on the changes. Neocortex also learns from its wrong predictions and corrects its models. For every change in an event, neocortex makes a prediction based on the model saved in the memory. If the prediction does not match the inputs, the incorrect prediction triggers an alert requesting for the resolution to the difference(s) and for updating of the models. Similarly, the Condition Management System needs to account for changes in input data and imperfections of models by having an architecture that allows update and addition of new models. In order to generate and store these rules and reasoning, there have to be schemas for expert system for learning, organizing and suggesting tasks associated with the recognized features in equipment-environment-corporate system. Inefficient schemas will severely restrict the efficiency and efficacy of the expert system.

2.6 Primary component (“old brain”)

The old brain is hard-wired to control the basic behaviour and functions performed by an organism. This includes the innate behavioural styles and patterns of eating, sleeping, reflex, feelings, emotions, desires, etc. These basic skills are coded in its genes. Most of these activities can take place without conscience decisions from the new brain (Hawkins 2021 ).

Just as a non-mammal can function without neocortex, an important safety feature for the proposed condition management system is the robust and safe functioning of the critical elements of the condition management system even in absence of the inputs from the Secondary Component.

2.6.1 Goals, motivations and strategy

In an animal, the old brain is the seat of needs, goals and motivations. According to cognitive theories, the contents of these needs, goals and motivations exist as knowledge (cognitive) representations in memories.

Relevance, hence utilization and success, of any product or service depends upon the core idea: To what extent does it satisfy existing needs? This puts needs at the centre of any activity—it fuels the need-fulfilment aspiration that drives the motivation system of all activities.

According to Abraham Maslow’s Theory of Needs, a person has five basic needs that have to be satisfied (Fig.  2 ). Even though he himself never represented this theory in form of a hierarchical triangle, the theory is best known in that format (Bridgman et al. 2019 ).

Maslow’s Theory of Needs

After necessary modifications, Maslow’s hierarchy of needs can also be used for explaining the motivation of carrying out condition management of equipment. First in the hierarchy is the physiological need that are related to the survival of the equipment. The next level is the need for safety, protection and security that ensure the proper health of the equipment. The next two layers comprise of the psychological needs. Last layer is that of self-fulfilment needs.

We have made the following changes to the application of the Maslow’s Theory of Needs:

Basic Needs are physical and not psychological

Esteem Needs are not relevant

Belongingness Needs are physical, not psychological and refer to the needs of working as a part of a system (network of equipment)

Self-fulfilment Needs reflect the purpose of having the equipment

The needs are not organized in a hierarchy, rather all the four needs (Physiological, Safety / Security, Belongingness and Self-fulfilment) have to be satisfied to some extent.

The optimum degree of satisfaction for individual needs is not static, it depends upon the corporation strategy and environment.

On the other hand, a process equipment, like a pump or a pipe, is just a physical functional structure whose identity is defined by the role it plays in conjunction with other equipment in the process network. On its own, it has no inherent purpose, need, goal, or motivation; rather it derives them from the usage associated to it by its manufacturers or users. A digital twin, being the “brain” of the equipment, can similarly be assigned needs, goals and motivations that will regulate and optimize its decision making process for the equipment. Thus, a process or an equipment can be assigned two types of needs:

Basic Needs —They refer to all needs related to the survival and well-being of the equipment. These include issues related to Reliability, Availability, Maintainability, Safety and Security.

Functional Needs —They refer to the needs related to working as a part of a system ( Belongingness Needs ) or reflect the purpose of having the equipment ( Self-fulfilment Needs ). It matches the requirements of the Asset Integrity of the equipment, where the Asset Integrity is defined as the ability of an asset to perform its required function effectively and efficiently while protecting health, safety and the environment (Health and Safety Executive 2007 )

All these needs have to be satisfied to some extent and at any particular time, there is an optimum degree of satisfaction for individual needs (Bridgman et al. 2019 ). Since, the desired level of satisfaction varies dynamically according to time and condition, an object (organism or equipment) can have multiple desired endpoints for its needs. These endpoints of the needs are the goals (Fishbach and Ferguson 2007 ). The Basic Needs dictate that the object should take decision with an objective to support self-protection (identifying threats, avoiding attacks, reducing damage) and self-preservation (resorting to repair and replacement). The Functional Needs dictates that the decision making should also have a goal to perform its required function effectively and efficiently without causing any accident. Strategic goals for digital twins may be fixed or adaptable, depending on their purpose and corporate strategies. Some goals may also be subject to modification based on ongoing learning.

Thus, the Basic Needs and the Functional Needs of an equipment, may be interpreted as a need to perform its required function effectively and efficiently without adversely affecting health, safety and environment.

According to Fishbach & Ferguson a goal is a cognitive representation of a desired endpoint that impacts evaluation, emotions and behaviours (Fishbach and Ferguson 2007 ). It directs an organism’s thoughts, feelings, decisions, and behaviours.

An understanding of “cognitive representation” of goals can help in understanding various aspects of the goal settings. Figure  3 shows an example of hierarchical structure of goals. Every living organism has two existential goals – short-term goal (survival) and long-term goal (passing on genes to next generation). To meet the goal of survival, the organism in turn has two sub-goals – self-generation (growing of self from birth until death) and self-maintenance. Self-maintenance entails self-protection (identifying threats, avoiding attacks, reducing damage) and self-preservation (resorting to repair and replacement).

Goals of living organisms

Analogous to it, the Primary Component has modules that contain goals of decision-making aspect for operation (corresponding to self-generation in an organism) and maintenance (corresponding to self-preservation in an organism) of process equipment.

Simpson and Balsam define motivation as the energizing of behaviour in pursuit of a goal (Simpson and Balsam 2016 ). It is fundamental to our interaction with the environment around us. For example, cues regarding the availability of food, a requirement for the goal of survival, may energize (motivate) an organism to take food-seeking actions. A number of factors, like physiological condition, environmental condition and experiences, influences the degree of motivation. The final decision taken by a person is an outcome of cost–benefit analysis involving all the factors and processes that can potentially influence the pursuit of goal. Similarly, in condition management of an equipment cost–benefit analysis is central for any optimisation of operation or maintenance activities.

Depending upon the goals and motivations animals develop a strategy to achieve the goals. Snakes have been observed to develop and execute basic strategies for foraging (hunting), demonstrating that reptilian brain (old brain) has the capacity for developing basic strategies. In mammals, advanced strategies are developed by the new brain.

A process equipment, like a pump or a pipe, is just a physical functional structure whose purpose and identity is defined by its role in the process network to which it is connected. It has no inherent purpose, need, goal, or motivation, but derives them from the ideas associated by its manufacturers or users. Thus, it reflects the intents of the usage and is devoid of any inherent goal, needs, motivations or strategy.

Development of human inspired cognitive condition management system entails a proper understanding of the relationship between humans, equipment and environment. Any pair wise (humans-equipment, equipment-environment and humans-environment) study will only provide a partial understanding of the requirements and offer limited solutions.

An addition of a cognitive human–machine platform could provide it with limited capabilities to autonomously condition-manage itself.

2.6.2 Safety vs. security issues

The main objective of the framework is to reduce issues related to safety and security of an equipment. To be able to meet this objective it is essential to understand the difference between the two and then create strategies to handle them.

In context of this framework, the major distinguishing features for safety and security issues are the identities of subject (entity that performs an action) and object (entity on which the action is performed). Thus (Firesmith 2003 ; Bartnes 2006 ):

In safety issues equipment is the subject and environment (natural environment, system, humans, corporate, etc.) are the object; failure of equipment can adversely affect the environment.

In security issues environment is the subject and equipment is the object; attack by environment can adversely affect the equipment.

Figure  4 illustrates the differences between the safety and security issues in condition management.

Safety versus security issues

Every bio-organism or equipment needs to overcome its vulnerabilities in order to survive. An understanding of these vulnerabilities and the associated safety and security issues can help to reduce the risks associated with the failure.

Similar to humans, the industrial assets (equipment or structure) need to be aware of both – safety and security – issues. Understanding of the potential vulnerabilities and risk of safety and security breaches can help to develop (a) effective and reliable operations, and (b) strategies to minimize the associated risks and prevent failures. [Bartnes, 2006; Firesmith, 2003].

For its security , an equipment needs to detect, identify and mitigate threats even before the attack takes place. If the attacker manages to breach security and exploit the system’s vulnerability, there is a need to detect, identify, and limit the damage caused by the attack.

For its safety , an equipment needs to detect, identify and mitigate degradations mechanisms caused by its own operation.

As in bio-organisms, in the proposed framework all the safety-security related decisions are taken by the Primary Component . Like a bio-organism that can function without input from neocortex, an important safety feature of the framework is the generation of decisions even in absence of the inputs from the Secondary Component , on safe functioning of equipment. In a situation where Awareness and Attention module identifies critical changes in the structure, environmental conditions and operating conditions, the Procedural Decision Making module generates Procedural Process Control Decisions , based on predetermined rules. These rules, while potentially not optimal, should be good enough to take preventive measures for safe operation. This means that on the detection of an unwarranted change, safety measures are implemented thereby ensuring secure operation even if there is insufficient data or information to process and analyze in the Secondary Component , resulting in a suboptimal performance (Fig.  5 ).

Failure profile of humans and equipment

2.6.3 Memories for procedural decision making

Old brain carries a cache of “best practices” that it has either inherited or acquired over long period (Hawkins 2021 ; Redish et al. 2016 ). Similarly, Primary Component, which is inspired by the old brain, has Memories Region containing predetermined rules and reasoning. These rules, while not being optimal, are good enough to take the first decision regarding the safety of the operation. In order to generate and store these rules and reasoning, there have to be schemas for expert system for learning, organising and suggesting tasks associated with the recognised features in equipment-environment-corporate system. Inefficient schemas will severely restrict the efficiency and efficacy of the expert system.

2.6.4 Awareness and attention

Real-time data from sensors first enter the Awareness and Attention Region of the Primary Component. Here the data is pre-processed, cleaned and then compared against baseline values to detect any anomaly (analogous to catching attention). An anomalous data results in activation of the Procedural Decision Making Region.

2.6.5 Procedural decision making

When an animal encounters an incidence or accident, it takes procedural decisions to reduce reaction time. These decision are often characterised as “reflexive actions”. These decisions are based on a combination of recognition processes and their associated decisions (International Organization for Standardization (ISO) 2019 ).

Similarly, Procedural Decision Making Region uses schemas present in the Memories Region to identify failure modes and type of damage. This knowledge is used for estimating safe working conditions for the damaged equipment.

2.7 Secondary component (“new brain”)

“New Brain” or neocortex with its knowledge and understanding of the world, while actively involved, facilitate the old brain in pursuit of its survival goals. Thus, neocortex is only an enabler for motivations that arise in the old brain. It can also set its own goals and motivations based on the needs.

Additionally, since the neocortex is not connected directly to the sensors and cannot directly control the movement, it relies on the old brain for signals from the sensors and for control over the actions (Hawkins 2021 ).

Reflecting the working of the neocortex, on detection of anomaly, the Secondary Component carries out detailed analysis. The main purpose of the Secondary Component is to carry out detailed analysis and recommend optimize operation control and inspection-maintenance schedule.

2.7.1 Memories for deliberative decision making

At birth, neocortex does not have any knowledge about the environment in which the host person lives. As the time passes, the signals that come from sensors via the old brain are not stored in neocortex as a library of facts; rather this learning is in the form of predictive models in the neocortex. Thus, mammals are not born with models in their neocortex, but have the ability to create them by learning. As the animal explores the changing world around them, it makes new models and updates the old ones. This way the brain makes, remembers and reuses hundreds of thousand models to give a “feel” of the real world. According to Hawkins: This leads to the strange truth that what you and I perceive, moment to moment, is a simulation of the world, not the real world . Thus, animals operate not on the basis of the actual environment, but on the simulation ( Neural Twin ) of the environment. These models are saved as memories recalled when needed. These memories can be temporary or long-lasting depending on the utility of the model (Institute 2023 ).

The models present in the neocortex are constantly making multiple simultaneous feed-forward predictions regarding the surroundings. Thus, they are continuously predicting what the following sensory input would be based on the expected or “normal” property and behavior of the environment or object. If the input received from the environment or the object matches the prediction, the brain accepts the model to be accurate and does not register, hence we are not aware of majority of predictions. On the other hand, when there is a mismatch between the prediction and the input, the animal’s attention is drawn. By attending to the differences between expectation and input, the differences are resolved, and the models updated.

Just as the neocortex has several models to represent the environment, the Memories Region of the Secondary Component has a number of models to represent the process equipment. These models analyse the data coming from sensors to interpret the real-time condition of the process equipment, predict its behaviour and control it.

Depending on the requirements of the process equipment, different types of models are used, including computational fluid dynamics (CFD) models for predicting flow of fluids, finite element analysis (FEA) for structural analysis, degradation models for loss of integrity, etc. These models can complement each other so that the condition of the equipment is interpreted not using a single source of data and model, but from a number of heterogeneous sensors and complementary models.

2.7.2 Failure profile analysis

Failure Profile Analysis examines the details of different types of failures that an equipment might experience during the operational life. This provides an understanding of cause, mechanisms, modes and type of failure. It can be performed using several methods and tools such as: Failure Mode and Effects Analysis (FMEA), Fault Tree Analysis (FTA), Root Cause Analysis (RCA), etc. (International Organization for Standardization (ISO) 2019 ). Each of these approaches has its own capabilities and limitations and the choice of approach depends on the system needs and objectives.

2.7.3 Risk assessment

According to ISO 31000, Risk Assessment is the overall process of risk identification, risk analysis and risk evaluation (International Organization for Standardization (ISO) 2009 ). Thus, it is carried out in three steps:

Risk Identification

Risk Analysis

Risk Evaluation

The first step, Risk Identification, is carried out in the Primary Component under Damage Identification .

For carrying out Risk Analysis , Likelihood of Failure is evaluated using necessary data from the real-time sensors, inspection reports, etc. and appropriate models stored in Memories of the Secondary Component . Consequences of Failure for safety, economic and environmental aspects are mostly evaluated using rules, incidental memories, etc. that are present in the Memories of the Primary Component.

Finally, the results from the previous step, Risk Analysis, and Risk Acceptance Criteria stored in the Memories of the Secondary Component are used for Risk Evaluation .

2.7.4 Deliberative decision making

Deliberative decision-making is a slow and computationally intensive process. It requires sufficient data and computational power. Hence, Deliberative Decision Making Region takes only the critical decisions and gives out detailed output regarding failure prognostics, optimized operation control and recommends inspection-maintenance schedule.

Humans carry out deliberation by means of “episodic future thinking” that takes place in two steps: (a) imagination to create a representation of a future scenario; and (b) evaluation of that scenario. The consequences of each possible future scenario are then compared and the best option is selected (Redish et al. 2016 ).

Similarly, for a particular goal, Deliberative Decision Making Region considers various possible scenarios and evaluates corresponding risks and outcomes in terms of safety, cost–benefit, etc. These results are then compared and the best option is used to optimise and control process of the damaged equipment and recommend inspection-maintenance schedule.

2.8 Arbitrator component

The old brain and the new brain are not entirely separate organs; they coordinate and work as a team. At times, decisions made by the two may come in conflict and need to be resolved. For example, conflict between new brain’s goal to hold breath for long and old brain’s need to provide oxygen to the body. More often old brain’s decision prevails over the new brain’s decision (Hawkins 2021 ).

In the proposed framework, an Arbitrator Component works to resolve decisions based on conflicting goals. This component receives decisions from the Primary Component and Secondary Component. At times, because of the optimisation among multiple goals, the Secondary Component may generate multiple decisions. If the differences between the decisions exceed some pre-set thresholds, then arbitration among the decisions is carried out using Arbitration Methodology (Fridman et al. 2023 ).

3 Conclusions

Application of biomimicry for solving technical challenges is relatively new. To the best of our knowledge, use of biomimicry for optimising inspection and maintenance schedule of an equipment has not been explored.

This paper describes a framework, inspired by human cognition system, for developing a condition management system framework. It is expected that the system will be able to utilize real-time data and integrate it with a knowledge-based system so that it can diagnose faults, and recommend optimised operation and maintenance programs of a damaged equipment.

It is expected that the framework can help to:

explicitly address setting of different goals and objectives of the operation and maintenance tasks so as to facilitate decision making under multiple and often conflicting requirements;

handle different types (numerical, pseudo-numerical, linguistic, etc.) of data, information and knowledge for decision making in a systematic and structured manner;

provide access to various mathematical and statistical tools where needed;

take decision in a structured way by integrating

procedural decision making that recommends decision based on limited data / information but weighted towards safety and security of the process,

deliberative decision making that recommends decision based on detailed analysis and weighted towards optimisation of process,

argumentative decision making that takes into account multiple and at times conflicting goals or recommendations (procedural and deliberative).

Bartnes M (2006) Safety vs. Security? In: Proceedings of the 8th International Conference on Probabilistic Safety Assessment and Management, May 14–18, 2006, New Orleans, Louisiana, USA

Bridgman T, Cummings S, Ballard J (2019) Who built Maslow’s pyramid? A history of the creation of management studies’ most famous symbol and its implications for management education. Academy Manag Learn Educ 18(1):81–98

Article   Google Scholar  

Firesmith DG (2003) Common Concepts Underlying Safety, Security, and Survivability Engineering. Technical Note CMU/SEI-2003-TN-033, Carnegie Mellon University

Fishbach A, Ferguson MJ (2007) The Goal Construct in Social Psychology. Social Psychology: Handbook of Basic Principles, pp 490–515

Fridman L, Ding L, Jenik B, Reimer B (2018) Arguing Machines: Human Supervision of Black Box AI Systems That Make Life-Critical Decisions”, available at: https://arxiv.org/ /, accessed: 27 Feb 2023

Hawkins J (2021) A: Thousand Brains – A New Theory of Intelligence. Basic Books, New York

Google Scholar  

Health and Safety Executive (2007). “Key Programme 3 - Asset Integrity Programme”. A report by the Offshore Division of HSE’s Hazardous Installations Directorate HSE 2007 www.hse.gov.uk/offshore/kp3.pdf .

Biomimicry Institute. https://biomimicry.org/ , last accessed 2023/02/10.

International Organization for Standardization (ISO) (2019) IEC 31010:2019, Risk management – Risk assessment techniques.

International Organization for Standardization (ISO) (2009) ISO 31000:2009, Risk management – Principles and guidelines.

Redish AD, Schultheiss NW, Carter EC (2016) The computational complexity of valuation and motivational forces in decision-making processes. Curr Top Behav Neurosci 27:313–333

Simpson EH, Balsam PD (2016) The behavioral neuroscience of motivation: an overview of concepts, measures, and translational applications. Curr Top Behav Neurosci 27:1–12

Download references

Open access funding provided by Western Norway University Of Applied Sciences.

Author information

Authors and affiliations.

Western Norway University of Applied Sciences, Inndalsveien 28, 5063, Bergen, Norway

Maneesh Singh, Knut Øvsthus, Anne-Lena Kampen & Hariom Dhungana

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Maneesh Singh .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Singh, M., Øvsthus, K., Kampen, AL. et al. Development of a human cognition inspired condition management system for equipment. Int J Syst Assur Eng Manag (2024). https://doi.org/10.1007/s13198-024-02391-y

Download citation

Received : 03 December 2023

Revised : 25 February 2024

Accepted : 13 June 2024

Published : 28 June 2024

DOI : https://doi.org/10.1007/s13198-024-02391-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Condition monitoring
• Predictive maintenance
• Find a journal
• Publish with us
• Track your research