research on learning resources

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 04 June 2024

Research on the influencing factors of adult learners' intent to use online education platforms based on expectation confirmation theory

• Guoqiang Pan   ORCID: orcid.org/0000-0003-1355-2077 1 ,
• Yu Mao   ORCID: orcid.org/0009-0002-6508-5458 2 ,
• Ziyuan Song   ORCID: orcid.org/0009-0004-7904-8871 3 &
• Hui Nie   ORCID: orcid.org/0000-0002-4529-5397 4  

Scientific Reports volume  14 , Article number:  12762 ( 2024 ) Cite this article

357 Accesses

1 Altmetric

Metrics details

• Human behaviour

This study addresses the understanding gap concerning the factors that influence the continuous learning intention of adult learners on online education platforms. The uniqueness and significance of this study stem from its dual focus on both platform features, such as service quality, and course features, including perceived interactivity and added value, aspects often overlooked in previous research. Rooted in Expectation Confirmation Theory, the study constructs a comprehensive model to shed light on the complex interplay of these factors. Empirical evidence collected from a survey of 1592 adult learners robustly validates the effectiveness of this model. The findings of the study reveal that platform service quality, perceived interactivity, and perceived added value significantly amplify adult learners' expectation confirmation and perceived usefulness. These elements subsequently enhance learner satisfaction, fostering their ongoing intention to use online education platforms. These insights offer practical guidance for online education providers, emphasizing the necessity to enhance platform service quality and course features to meet adult learners' expectations and perceived usefulness. The study provides valuable perspectives for devising strategies to boost user satisfaction and stimulate continuous usage intention among adult learners in the intensely competitive online education market. This study enriches the literature by uncovering the relationships among platform features, course features, expectation confirmation, perceived usefulness, and continuous usage intention. By proposing a comprehensive model, this study provides a novel theoretical basis for understanding how platform and course features impact adult learners' ongoing intention to use online education platforms, thereby aiding the evolution and refinement of relevant theories.

Similar content being viewed by others

Hybrid working from home improves retention without damaging performance

Misunderstanding the harms of online misinformation

An overview of clinical decision support systems: benefits, risks, and strategies for success

Introduction.

In the beginning, adult higher education primarily relied on face-to-face instruction, utilizing weekends and evenings for classes. However, frequent issues arising from the pandemic, pronounced contradictions in adult learning, and uneven distribution of time and energy led to subpar teaching quality and low efficiency 1 , 2 . Consequently, the introduction of online education platforms for blended learning became necessary.

Adult learners exhibit a range of learning goals and motivations 3 while concurrently grappling with the constraints of time and space 4 , 5 . These variables significantly impact their inclination towards utilizing online education platforms. Unlike traditional students, adult learners often have a wider spectrum of learning objectives 6 , encompassing facets such as career progression, personal interests, and lifelong learning 7 , 8 , 9 . They also demonstrate stronger learning motivation and autonomy, underscoring the importance of personalized learning experiences and resources 10 . The learning outcomes for adult learners on online education platforms are influenced by factors such as course design, the richness of learning resources, and the selection of learning paths 11 , 12 .

For online education platforms, the associated costs of retaining existing users generally surpass those of acquiring new ones 13 . Since online education platforms function as a type of information system, researchers typically probe user behavior within these systems using technology acceptance models and expectancy confirmation models. When investigating users' continued usage intentions, most scholars extend these models based on the expectancy confirmation model. However, the front-end influencing factors for the key variables of expectancy confirmation and perceived usefulness have not been extensively studied. Currently, the factors influencing perceived usefulness are mainly examined from viewpoints such as perceived ease of use, content-driven factors, social influence, subjective norms, and autonomy 14 , 15 , 16 , 17 , 18 . Meanwhile, the factors affecting expectancy confirmation are explored from perspectives such as perceived playfulness, information quality, and service quality 19 , 20 , 21 . Nonetheless, there is a dearth of comprehensive investigation into the influencing factors of perceived usefulness and expectancy confirmation from the broad perspective of online education platform constituents, namely considering technological platform features and online course features. This gap in systematic analysis impedes the ability of online education platforms to adopt effective service improvement measures aligned with their unique characteristics.

In light of the distinct attributes of adult learners participating in online education, this research conducts a systematic review and a thorough consolidation of the determinants impacting perceived usefulness and expectation confirmation 8 . This is done from the standpoint of both technological platform characteristics and the attributes of online courses 9 . By adopting such a comprehensive approach, we can more effectively aid online education platforms in identifying and implementing service improvement measures that resonate with their unique traits.

This study concentrates on online education platforms and adult learners, examining the potential influence of platform features (such as platform service quality) and course attributes (like perceived interactivity and the perceived added value of the course) on adult learners' perceived usefulness and expectation confirmation when engaging with online education platforms. Furthermore, this study delves into the effects of these factors on adult learners' satisfaction and their continued intention to use these platforms.

In the context of online education platforms, the diverse learning objectives and motivations of adult learners 2 , coupled with the constraints they encounter in terms of time and space 5 , play a crucial role. These elements shape their willingness to engage with online education platforms. In contrast to traditional students, adult learners may have a broader range of learning goals, encompassing career advancement, personal interests, and lifelong learning 12 . They exhibit a heightened sense of learning motivation and autonomy, placing a premium on personalized learning experiences and resources 9 . Furthermore, online education platforms can impact the learning outcomes of adult learners through factors such as course design, the abundance of learning resources, and the selection of learning paths 8 , 22 . Consequently, our research will concentrate on how these factors affect the learning outcomes of adult learners on online education platforms, aiding us in better comprehending and catering to their learning requirements.

Theoretical foundation and research hypotheses

Expectation confirmation theory.

In 1980, Oliver introduced the Expectancy Disconfirmation Theory (EDT). This theory posits that users, prior to purchasing a product or service, hold certain expectations. After the actual use of the product or service, users perceive the performance differential between their expectations and the realized experience, termed as expectancy disconfirmation 23 .

The Expectancy Confirmation Theory (ECT) has evolved from the Expectancy Disconfirmation Theory (EDT) and serves as a crucial foundation for studying user continuance. Patterson et al. were among the pioneers to apply the Expectancy Confirmation Theory in the field of information systems 24 . Bhattacherjee proposed the Expectation Confirmation Model (ECM-ISC), which incorporates four main variables: expectation confirmation, perceived usefulness, satisfaction, and continuance intention 25 . Following the introduction of the Expectation Confirmation Model, studies by Larsen on mobile commerce 26 , Tang and others on blogs 27 , Doong on knowledge sharing 28 , and Kim on mobile data services 29 have all affirmed the effectiveness of the Expectation Confirmation Model.

Research hypotheses

In the online education environment for adult learners, platform features refer to the key factors influencing satisfaction and exert a significant impact on adult learners' continued usage intentions. Among these features, platform service quality is crucial and measures the effectiveness and promptness of services provided by service providers. Bhattacherjee defines expectancy confirmation as the degree to which information system users confirm their expectations before and after using the system, where lower expectations and higher actual experiences enhance expectancy confirmation 25 . Researchers such as Dahan et al. have validated through data the positive impact of service quality on expectancy confirmation and satisfaction 30 . Delone et al. through the Information Success Model they constructed, identified service quality as a critical factor influencing satisfaction and usefulness 31 . Moreover, existing research has confirmed the significant influence of service quality on perceived usefulness and expectancy confirmation 27 , 32 , 33 .

Adult learners in online education place a greater emphasis on personalized learning experiences and resources, making the service quality of the platform crucial to their actual learning experiences 13 , 34 . Based on this, we hypothesize that the higher the platform's service quality, the better the actual learning experience for adult learners. Based on this, the following research hypotheses are proposed:

H1: Platform service quality positively influences the expectancy confirmation of adult learners.

H2: Platform service quality positively influences the perceived usefulness of adult learners.

In online education platforms, in addition to platform features, course characteristics are considered the most crucial factors for adult learners' attention 35 . This study measures course features through perceived interactivity and perceived added value. The more satisfied adult learners are with course features, the stronger their intention to continue using the platform. In traditional educational settings, interactions with teachers and peers positively influence students, and similar positive effects are expected in online education. Emphasizing platform interactivity facilitates communication among adult learners, timely issue resolution, and enhances their expectations 36 . If the platform lacks strong interactivity, it may negatively impact the learning experience 37 . Therefore, platforms need to enhance interactivity to cultivate positive online learning habits among adult learners 38 . The stronger the perceived course features, the higher the adult learners' overall satisfaction with the course experience. Yang's study on MOOC users found that interactivity significantly influences expectancy confirmation 20 . Perceived added value refers to additional or value-added services provided beyond basic services 39 , 40 . As an additional benefit, it further enhances users' perceived value of the course, leading to a more positive evaluation of expectancy confirmation. Online education platforms should focus on improving course features, enhancing interactivity, and providing perceived added value to better meet the needs of adult learners 41 . Based on the characteristics of adult learners in online education, this study proposes the following hypotheses:

H3: Course characteristics significantly influence the expectancy confirmation of adult learners.

H3-1: Perceived interactivity has a significant positive impact on the expectancy confirmation of adult learners.

H3-2: Perceived added value has a significant positive impact on the expectancy confirmation of adult learners.

Based on data from the SPOC platform, Guo et al. found that classroom interaction significantly and positively influences perceived usefulness and perceived ease of use in SPOC learning 42 . A research of Wu et al. indicated that interactivity has an impact on perceived usefulness and expectancy confirmation 43 . The study of Qian et al. revealed that perceived interactivity has a positive effect on perceived usefulness 15 . Additionally, Yang found a positive influence of interactivity on perceived usefulness in research involving MOOC users 20 . In the field of mobile communication services, Liu and Chen examined the impact of added value on perceived usefulness and discovered that value-added services have a positive effect on the perceived usefulness and satisfaction of communication customers 44 . This finding aligns with the personalized learning experience needs of adult learners. Based on these observations, the following research hypotheses are proposed:

H4: Course features have a significant impact on the perceived usefulness of adult learners.

H4-1: Perceived interactivity has a significantly positive influence on the perceived usefulness of adult learners.

H4-2: Perceived added value has a significantly positive impact on the perceived usefulness of adult learners.

Bhattacherjee validated the impact of expectancy confirmation on perceived usefulness in the Expectation Confirmation Model 25 . Perceived usefulness refers to the improvement in learning efficiency and the degree of learning outcomes when users utilize online education platforms. Perceived usefulness not only influences adult learners' initial acceptance 45 but also has a significant impact on adult learners' satisfaction and the intention to continue using the platform 25 . Yang, in his study on MOOC users, confirmed the positive effect of expectancy confirmation on perceived usefulness 20 . Qian's research on online learning users also affirmed the positive influence of expectancy confirmation on perceived usefulness 15 . Studies by Hayashi and Lin further supported the impact of expectancy confirmation on perceived usefulness 46 , 47 . Based on this, the following research hypothesis is proposed:

H5: Expectation confirmation positively influences the perceived usefulness of adult learners.

When adult learners' expectations are met or exceeded by the online education platform, signifying a higher level of expectancy confirmation, it leads to increased satisfaction with the platform. This hypothesis is supported by research done on individual users in Social Networking Sites (SNS) as well as studies conducted by Liu and colleagues on short video users 48 . Further, the findings of Chiu et al. and Wang et al. reinforce the influence of expectancy confirmation on satisfaction 49 , 50 . Moreover, this relationship has been substantiated in various digital contexts, such as e-commerce 51 , and e-learning 52 , suggesting that expectancy confirmation is a significant predictor of user satisfaction across different digital platforms. To further expand on this, it's worth noting that expectancy confirmation can also influence other aspects of user experience. For instance, when users' expectations are confirmed, they may perceive the platform as more useful, which can further enhance their satisfaction 53 . Additionally, expectancy confirmation can also impact users' trust in the platform 9 . When users' expectations are met, they may develop a higher level of trust in the platform, which can also contribute to increased satisfaction. Based on this, the following research hypothesis is proposed:

H6: Expectancy confirmation positively influences user satisfaction with the online education platform.

Wang et al. affirmed the positive influence of perceived usefulness on satisfaction in their study of users of Virtual Reality (VR) library services 54 . In a similar vein, Yin et al.'s research on WeChat users in university libraries corroborated the positive effect of perceived usefulness on satisfaction 55 . The connection between perceived usefulness and satisfaction was further explored and substantiated in studies by Bhattacherjee and Lin et al. 25 , 47 . These findings collectively underscore the importance of perceived usefulness in driving user satisfaction across a variety of digital platforms. Building on these insights, we can argue that perceived usefulness is not just an antecedent of satisfaction, but may also play a role in shaping other user attitudes and behaviors 56 . For example, perceived usefulness could influence users' continued intention to use a platform 52 , their trust in the platform 57 , and their willingness to recommend the platform to others 58 . Based on these findings, the following research hypothesis is proposed:

H7: Perceived usefulness positively influences the satisfaction of adult learners.

This study focuses on the intention to continue use, which refers to adult learners' willingness to continue using online education platforms. Satisfaction is the experiential feeling and overall evaluation that adult learners have after using online education platforms. Cao et al., focusing on WeChat Moments experience, found that satisfaction significantly influences users' intention to continue using 59 . The research of Zhang and Yao on mobile government apps also confirmed the impact of satisfaction on the intention to continue use 60 . Scholars such as Bhattacherjee et al., have all verified the positive influence of satisfaction on the intention to continue use 20 , 25 , 46 . Based on this, the following research hypothesis is proposed:

H8: Satisfaction positively influences the intention to continue use of adult learners.

The extent of users' intention to continue using online education platforms reflects their loyalty to the selected platform. The study of Gao and Hu on users of knowledge community services found that service quality has a positive impact on continued usage 61 . Lin, through research on consumer behaviors using the ABC attitude theory, discovered that service quality positively influences shopping attitudes 62 . Zhou et al., in their study of users in the shopping domain, identified service quality as a significant influencing factor on intention to continue usage 63 .

The influence of platform service quality on the intention to continue use among adult learners may be subject to the mediating effects of other variables. Wang's study on the continued use intention of mobile libraries found that service quality affects users' perceived usefulness 64 . Guo and Ming concluded that service quality positively influences users' expectation confirmation and perceived usefulness 65 . Hsu and Lin discovered in their study of mobile client user behavior that service quality initially affects user satisfaction 66 . Yang also identified service quality as a significant factor influencing user satisfaction in mobile reading 67 . Alali and Salim, in their study on a health forum, found a significant impact of service quality on user satisfaction 68 . In the field of information systems, scholars have confirmed the positive impact of expectation confirmation on user satisfaction 46 , 47 , 69 , 70 . This implies that when users have high expectation confirmation, indicating their expectations and usage experience are satisfied, it can enhance their perceived usefulness and increase satisfaction with the platform. Liu's research on video website users validated the effect of perceived usefulness on satisfaction 71 . Yang's study on e-book users also confirmed the positive impact of perceived usefulness on satisfaction 72 . Concurrently, within the realm of consumer behavior, a multitude of empirical investigations have corroborated the affirmative promotional influence of satisfaction on users' proclivity to sustain usage 15 , 20 , 25 , 67 . This suggests that expectation confirmation has an impact on adult learners' satisfaction, and learner satisfaction may further influence their intention to continue use. Combining the positive influence relationship of platform service quality on adult learners' expectation confirmation and perceived usefulness proposed in H1 and H2, this study posits that platform service quality has a positive impact on the intention to continue use among adult learners. Moreover, this positive influence occurs through multiple mediating effects of perceived usefulness, expectation confirmation, and satisfaction among adult learners. Based on this, the following research hypothesis is proposed:

H9: Platform service quality has a positive impact on the intention to continue use of adult learners.

H9-1: In the process of the impact of platform service quality on the intention to continue use of adult learners, expectation confirmation and satisfaction play a chain-mediating role.

H9-2: In the process of the impact of platform service quality on the intention to continue use of adult learners, perceived usefulness and satisfaction play a chain-mediating role.

In terms of the impact of course features on adult learners' intention to use, researchers have made some important findings. The study of Joo et al. discovered that high-quality interactions can stimulate positive evaluations of educational platforms by users, thereby increasing their intention to continue using 73 . This indicates that high-quality interactions, such as timely answering of questions and sharing ideas, can enhance adult learners' loyalty to the platform. Chow et al. investigated the impact of interactions on users' intention to continue using from the dimensions of teacher interaction and peer interaction 74 . The research of Hoffman and Novak found that the higher the user's interactivity, the better their overall experience 75 . In addition, Zhang and Wu pointed out that perceived interactivity can lead to positive emotional changes in users 76 . Perceived added value is the unexpected gain that adult learners feel, which can effectively increase their favorability towards the platform and satisfaction with the usage process 42 , thereby reinforcing their intention to continue using.

Van Noort et al. found that the higher users' perceived interactivity on a website, the higher their satisfaction and willingness to use the website 77 . Qian validated the positive impact of perceived interactivity on satisfaction and continued intention to use 15 . Gefen et al. argued that user interaction with a website can influence their trust attitudes 78 . Kim's study on travel websites revealed that interactivity can enhance users' trust in the website 79 . Park et al. focused on the impact of perceived interactivity on satisfaction and examined the mediating role of perceived usefulness/perceived value 80 . Song and Zinkhan found that website response speed is a crucial factor influencing user satisfaction 81 . Meanwhile, in the field of information systems, numerous scholars have confirmed the influence of expectation confirmation and perceived usefulness on satisfaction, as well as the impact of satisfaction on intention to continue using 15 , 20 , 25 , 67 . Combining with the proposed positive relationships in H3 and H4 regarding course features (perceived interactivity and perceived added value) and adult learners' expectation confirmation and perceived usefulness, this paper suggests that online education course features have a positive impact on adult learners' intention to continue using. Moreover, this positive influence occurs through multiple mediating pathways involving adult learners' perceived usefulness, expectation confirmation, and satisfaction. Based on this, the following research hypotheses are proposed:

H10: Course features significantly impact adult learners' intention to continue using.

H10-1: We posit that perceived interactivity has a positive influence on the continued use intention of adult learners. In the process of how perceived interactivity influences the intention to continue use, we propose that both expectation confirmation and satisfaction, as well as perceived usefulness and satisfaction, play a chain-mediating role. This suggests that when the perceived interactivity of the course meets or exceeds the expectations of adult learners, it confirms their expectations, enhances their satisfaction, and simultaneously elevates the perceived usefulness of the course, ultimately influencing their intention to continue using the platform.

H10-2: We suggest that perceived added value has a positive influence on the continued use intention of adult learners. In the process of how perceived added value influences the intention to continue use, we propose that both expectation confirmation and satisfaction, as well as perceived usefulness and satisfaction, play a chain-mediating role. This implies that when the perceived added value of the course meets or exceeds the expectations of adult learners, it confirms their expectations, enhances their satisfaction, and simultaneously increases the perceived usefulness of the course, ultimately influencing their intention to continue using the platform.

In summary, the research model of factors influencing the intention to continue using online education platforms in this study is illustrated in Fig.  1 .

Research model framework.

Measurement tools

The design of the questionnaire drew inspiration from mature scales used globally to measure the intention to use information systems. Additionally, references were taken from relevant studies on online education platforms both domestically and internationally, with subsequent modifications made to the questionnaire. The questionnaire comprises two parts: the first part gathers adult learners demographic information, while the second part measures the factors influencing the continuous usage behaviour of online education platform adult learners (refer to Table 1 ). The Likert five-point scale method was employed in the questionnaire, where 1 represents strongly disagree , 2 represents disagree , 3 represents neutral , 4 represents agree , a nd 5 represents strongly agree .

Participants

This study selected adult education students from a university in Shanghai as research subjects. By employing a stratified sampling method, we selected participants based on 10% of the total adult student population. This sampling process was carried out stratifying by profession and grade. The total number of participants amounted to 1592, the detailed information of which can be found in Table 2 . Prior to participants answering the questionnaire, there was an introductory statement informing them of the purpose of the survey, emphasizing the confidentiality, anonymity, and voluntary nature of their participation in the research.

Data processing methodology

We used SPSS 23.0 software to first test for common method bias in the data and analyse the correlations between variables. AMOS 24.0 software was employed to test the discriminant validity among variables. Additionally, SPSS 23.0 software and the PROCESS 2.16 macro program with the Bootstrap test method (setting the sample extraction size to 5000 times and the confidence interval to 95%) were used to test for the chained mediation effects.

Informed consent statement

We confirm that informed consent has been obtained from all subjects. Each survey will provide an Informed Consent Form, which will be indicated in the instruction section of the questionnaire.

Common method bias

To avoid the potential issue of substantial common method bias influencing the spurious prediction of independent variables on dependent variables, this study employed two methods (procedural control and statistical control) for control and examination 91 .

Firstly, before distributing the survey questionnaire, this study implemented effective randomization of the various scales used and made a commitment to participants to protect the privacy of their data. Secondly, the study employed the Harman's single-factor test for statistical control. Through exploratory factor analysis conducted on the obtained data without factor rotation, the first factor's variance explained 39.26% (below the critical threshold of 40% variance explained) 92 . Therefore, these methods and results suggest that there is no severe common method bias in the collected data for this study.

Reliability and validity analysis

Through the application of SPSS 23.0, an examination of the reliability and validity of the questionnaire was conducted. The Cronbach's alpha value for the questionnaire was found to be 0.896. Furthermore, the Cronbach's alpha values for each variable were all above 0.689 (refer to Table 3 ), indicating a good level of internal consistency for the variables. Hence, the reliability of the questionnaire is deemed acceptable. The Kaiser–Meyer–Olkin (KMO) value for the questionnaire was 0.914, with individual variable KMO values exceeding 0.5. The overall interpretability is high, justifying the application of principal component analysis. The cumulative variance explanation rate was determined to be 67.779%. Moreover, all factor loading values were above 0.5, signifying good validity of the sample. These results affirm the reliability and validity of the questionnaire, ensuring the robustness of the data analysis and interpretation 93 , 94 , 95 , 96 , 97 .

Fit test analysis

The purpose of model fit testing is to measure the degree of fit between the hypothetical model and the observed data. As shown in Table 4 , overall, the research model exhibits good fit.

Mediation effect testing

Firstly, a preliminary examination of the mediation effect was conducted using the linear hierarchical regression method. The variables showed a correlation, with coefficients between 0.4 and 0.76, and all Composite Reliability (CR) above 0.675, and all Average Variance Extracted (AVE) above 0.5 indicating no severe collinearity among the variables. The correlation results are presented in Table 5 94 , 95 , 96 , 97 .

Furthermore, the Bootstrap method, as implemented in the Process macro program with 5000 resamples and a confidence interval set at 95%, was employed to conduct a more in-depth examination of the chain mediation 46 .

From Fig.  2 and Table 6 , it can be observed that platform service quality significantly influences adult learners' perceived usefulness ( β  = 0.378, p  < 0.0001) and expectation confirmation ( β  = 0.432, p  < 0.0001), supporting hypotheses H1 and H2 . Adult learners' perceived interactivity ( β  = 0.282, p  < 0.0001) and perceived additional value ( β  = 0.353, p  < 0.0001) significantly positively impact their expectation confirmation, supporting hypothesis H3 . Adult learners' perceived interactivity ( β  = 0.333, p  < 0.0001) and perceived additional value ( β  = 0.374, p  < 0.0001) significantly positively influence their perceived usefulness, supporting hypothesis H4 . Adult learners' expectation confirmation positively influences their perceived usefulness ( β  = 0.755, p  < 0.0001), supporting hypothesis H5 . Adult learners' expectation confirmation positively influences their satisfaction ( β  = 0.374, p  < 0.0001), supporting hypothesis H6 . Adult learners' perceived usefulness positively influences their satisfaction ( β  = 0.493, p  < 0.0001), supporting hypothesis H7 . Adult learners' satisfaction positively influences their intention to continue using the platform ( β  = 0.587, p  < 0.0001), supporting hypothesis H8 . Platform service quality significantly influences adult learners' intention to continue using ( β  = 0.374, p  < 0.0001), supporting hypothesis H9 . Adult learners' perceived interactivity ( β  = 0.243, p  < 0.0001) and perceived additional value ( β  = 0.305, p  < 0.0001) positively influence their intention to continue using, supporting hypothesis H10 . All ten research hypotheses derived from the Expectation Confirmation Model are supported. To separately test the mediating effects of perceived usefulness, expectation confirmation, and satisfaction in the relationships between course characteristics, platform features, and continued usage intention, this study employed the bias-corrected nonparametric percentile Bootstrap method, and the results are presented in Table 7 .

Results of the regression analysis on the continued usage intention of adult learners.

Results indicate that the mediating effects of course characteristics, platform features, and continued usage intention are significant. In the mediation path PSQ → EC → SA → CI , the effect value is 0.0874, with a 95% Bootstrap confidence interval ranging from 0.0515 to 0.1436, excluding 0. This implies that expectation confirmation and satisfaction play a significant mediating role in the relationship between platform service quality and continued usage intention, supporting H9-1 . In the mediation path PSQ → PU → SA → CI , the effect value is 0.0742, with a 95% Bootstrap confidence interval ranging from 0.0395 to 0.1104, excluding 0. This suggests that perceived usefulness and satisfaction significantly mediate the relationship between platform service quality and continued usage intention, supporting H9-2 . In the mediation path PI → EC → SA → CI , the effect value is 0.0983, with a 95% Bootstrap confidence interval ranging from 0.0604 to 0.1548, excluding 0. This indicates that expectation confirmation and satisfaction play a significant mediating role in the relationship between perceived interaction and continued usage intention, supporting H10-1–1 . In the mediation path PI → PU → SA → CI , the effect value is 0.0774, with a 95% Bootstrap confidence interval ranging from 0.0453 to 0.1238, excluding 0. This shows that perceived usefulness and satisfaction significantly mediate the relationship between perceived interaction and continued usage intention, supporting H10-1 . In the mediation path PAV → EC → SA → CI , the effect value is 0.0963, with a 95% Bootstrap confidence interval ranging from 0.0612 to 0.1485, excluding 0. This indicates that expectation confirmation and satisfaction play a significant mediating role in the relationship between perceived added value and continued usage intention, supporting H10-2 . In the mediation path PAV → PU → SA → CI , the effect value is 0.0724, with a 95% Bootstrap confidence interval ranging from 0.0433 to 0.1154, excluding 0. This suggests that perceived usefulness and satisfaction significantly mediate the relationship between perceived added value and continued usage intention, supporting H10 .

This study delves into the factors influencing the continued usage intention of adult learners on online education platforms, focusing on both platform features and course characteristics 56 . Beginning with platform features, we examined the impact of platform service quality on user experience. The research revealed that service quality has a direct relationship with users' expectation confirmation and perceived usefulness of the platform 1 , 98 . This aligns with previous research findings, emphasizing adult learners' expectations for high-quality services, including effectiveness and promptness, when using online education platforms. It also corroborates the Information Success Model constructed by Delone et al. 31 , where service quality is identified as a crucial factor significantly affecting satisfaction and usefulness 41 . The study results indicate that enhancing platform service quality will directly strengthen adult learners' expectation confirmation and perceived usefulness, fostering a more positive online learning habit among adult learners.

Furthermore, concerning course characteristics, we gauged the appeal of courses through perceived interactivity and perceived added value 99 . The research results demonstrate a significant positive relationship between adult learners' positive perceptions of course characteristics and their expectation confirmation, perceived usefulness, satisfaction, and ultimately, their intention to continue using the platform 99 . This validates the importance of interactivity and added value in previous research, particularly in the context of online education 34 . Adult learners anticipate courses to have robust interactivity, fostering positive interactions among students and addressing issues promptly, thereby enhancing adult learners' learning expectations 44 . Simultaneously, adult learners positively evaluate the added value provided by the courses, further reinforcing their perception of the course's value and subsequently increasing expectation confirmation and satisfaction.

Adult learners' intention for continuous usage is a complex process influenced by multiple factors related to platform and course characteristics 57 , 100 , 101 . Improvements in platform service quality and course features have a positive impact on adult learners' expectation confirmation, perceived usefulness, and satisfaction, ultimately encouraging adult learners to actively choose to continue using the online education platform 58 . Furthermore, through the analysis of mediating effects, we have identified that expectation confirmation and satisfaction play a chain-mediated role between platform service quality and adult learners' intention for continuous usage 44 . This suggests that enhancing platform service quality will strengthen users' intention for continuous usage by elevating expectation confirmation and satisfaction 57 . Therefore, educational platforms, in enhancing adult learners experience, should not only focus on improving platform service quality but also prioritize optimizing course characteristics to comprehensively enhance adult learners' online learning experience and loyalty.

This study reveals that learners' expectation confirmation and online learning engagement significantly influence learning satisfaction and perceived outcomes across different educational contexts. In formal education, enhancing teaching quality and learner engagement can boost learning outcomes 102 . In corporate training, practical, work-related content and high-quality teaching can heighten satisfaction and perceived outcomes 103 . In informal learning, catering to individual needs and providing a flexible learning environment can significantly enhance satisfaction and learning outcomes 58 . These insights offer valuable strategies for boosting learning satisfaction and perceived outcomes in diverse learning environments.

In the discussion section, we can further explore how emerging technologies, particularly Artificial Intelligence (AI), could potentially influence the relationships outlined in our hypotheses 104 , 105 . The advent of AI offers new possibilities for online education, especially in enhancing the quality of teaching services 105 , 106 . For instance, AI can be used to develop intelligent tutoring systems that provide personalized learning experiences based on learners' styles and needs, potentially enhancing their expectation confirmation and, consequently, their learning satisfaction and perceived learning outcomes 107 . However, the application of AI in online education also presents challenges, such as ensuring the fairness and transparency of AI systems and protecting learners' privacy.

Implications

Implications for theory.

In terms of theoretical implications, this study offers fresh insights into the learning experiences of adult learners on online education platforms. The findings reveal that the confirmation of adult learners' expectations positively impacts both their perceived usefulness and satisfaction. This underscores the pivotal role of user expectation confirmation in the online learning experience, highlighting the correlation between expectations and experiences 100 . Simultaneously, the research results provide profound insights into the psychological gap in the adult learners' experience of online education platforms, offering a theoretical basis for understanding the relationship between adult learners' expectations and actual experiences.

However, to further enhance the value of the research, it is suggested to integrate new context-specific structures and novel theories to improve the parsimony and novelty of the research. For instance, exploring other potential factors in the online education environment, such as community atmosphere and interaction quality, and how they influence adult learners' expectation confirmation and satisfaction could be beneficial 99 . Additionally, adopting novel theoretical perspectives, such as self-determination theory, can provide a new viewpoint for understanding the motivations and behaviors of adult learners in online learning 101 .

Implications for practice

This research offer practical guidance for online education providers, emphasizing the need to enhance platform service quality and course features to meet adult learners' expectations and perceived usefulness. The study provides valuable insights for formulating strategies to improve user satisfaction and foster continuous usage intention among adult learners in the competitive online education market.

A truthful promotional strategy

The research results suggest that online education platforms should adhere to a truthful approach in their promotion and avoid exaggerated claims. This provides practical guidance for operations, helping the platform establish an authentic and trustworthy image in advertising and marketing 52 . This approach reduces the potential psychological gaps that users might experience after use, thereby enhancing overall user satisfaction. This strategy is universally applicable, not only in a formal education environment but also in corporate training and informal learning settings.

Emphasizing the utility in promotion

Online education platforms should highlight their strengths, features, and content quality to help users profoundly understand the platform's utility 44 . This provides direction for the platform in advertising and promotion, aiding users in gaining a more comprehensive understanding of the platform's value, thereby increasing perceived utility and overall satisfaction 12 . This approach is applicable across various educational settings, benefiting formal education, corporate training, and informal learning alike.

Enhancing service quality

The study reveals a significant positive impact of platform service quality on user expectation confirmation and perceived utility 57 . Therefore, platforms should invest in professional training for customer service to improve service quality, aiming to enhance adult learners' expectations and subsequently elevate perceived utility 52 . This practical recommendation provides online education platforms with actionable insights to improve adult learners' experience and contributes to establishing a solid foundation for user satisfaction 5 . Improving service quality is crucial in all educational environments, whether it's formal education, corporate training, or informal learning.

Optimizing course experience

By emphasizing perceived added value and interactivity, platforms can enhance adult learners' satisfaction with course quality 53 . To achieve this, platforms can offer additional services during adult learners' engagement and establish communication channels, enabling adult learners to better experience the utility of the online education platform 103 . This provides a practical and feasible approach for platforms to optimize the course experience and increase user willingness to continue using the platform 58 . This approach has application value in different educational settings and can help improve the course experience in formal education, corporate training, and informal learning.

Limitations and prospects

This study has made notable discoveries about adult learners' sustained intention to utilize online education platforms, but it has limitations. Firstly, the research mainly hinged on adult education students from a single university, potentially limiting the results' broad applicability due to sample specificity. Future research could enhance the findings' external validity by expanding the sample size and incorporating more diverse user groups. Secondly, despite a comprehensive research model, additional latent variables influencing adult learners' continued intention to use online platforms might have been overlooked. Future studies could explore other potential factors for a more comprehensive understanding of the decision-making process when adult learners opt to persist with online education platforms.

Additionally, this study predominantly employs quantitative research methods, leveraging survey data collection. Future research could contemplate integrating more qualitative research methods, like in-depth interviews or observations, to attain a more comprehensive grasp of adult learners' behavioral motivations and experiences 108 . Future research should also advocate for the use of mixed methods or longitudinal studies to empirically substantiate the proposed hypotheses across various types of online education platforms and diverse adult learner populations.

Subsequent research could incorporate negative outcomes, as current research mainly focuses on positive ones. Considering the potential negative impact of high expectations or poor service quality on user satisfaction and continuance intention can provide a more comprehensive understanding of the online learning experience. The rapid development of online education technology necessitates future research to include factors associated with technological advancement (for example, personalized learning driven by artificial intelligence) and their impact on the learning experience.

The upcoming research has the potential to introduce negative outcomes, given that current studies primarily concentrate on positive results. Taking into account the potential negative influence of high expectations or subpar service quality on user satisfaction and sustained intention can offer a more comprehensive comprehension of the online learning experience. Future research should consider the speedy evolution of online education technology, integrating factors related to technological progress (for instance, AI-driven personalized education) and their repercussions on the learning experience can be highly valuable.

Looking ahead, researchers can dedicate efforts to further deepen the investigation into adult learners' continuous intention to use online education platforms, overcoming current study limitations, and continually enhancing the understanding of the mechanisms behind adult learners behaviors.

Conclusions

Based on the Expectation Confirmation Theory and considering the characteristics of online education platforms, this study constructs a research model by focusing on two crucial variables—expectation confirmation and perceived usefulness—and their influencing factors: platform service quality and course service quality. The results indicate the following key findings: (1) Satisfaction is a critical factor influencing adult learners' continued usage of the platform. (2) Adult learners' perceived usefulness affects satisfaction, and the degree of expectation confirmation significantly influences both perceived usefulness and satisfaction. (3) Platform service quality impacts expectation confirmation and plays an essential role in perceived usefulness. (4) Perceived added value and perceived interactivity of course service quality significantly influence expectation confirmation and also play a crucial role in perceived usefulness. (5) Perceived usefulness, expectation confirmation, and satisfaction serve as significant mediators in the relationship between platform features and the intention to continue usage. (6) Perceived usefulness, expectation confirmation, and satisfaction act as significant mediators in the relationship between course features and the intention to continue usage. In summary, these findings shed light on the factors influencing users' continued usage of online education platforms, providing valuable insights for platform operators to enhance user experience and satisfaction.

Data availability

The datasets generated and analysed during the current study are available in the Zenodo repository, https://doi.org/10.5281/zenodo.10584056 .

Li, L. & Zhang, P. Research on the impact of online education platforms on adult learners’ learning outcomes. Res. Electrif. Educ. 38 , 4–10. https://doi.org/10.15885/j.cej.2022.06.004 (2022).

Article   Google Scholar  

Arabacioğlu, T. E-learning system usage continuance intention of adult learners: A data mining approach. Int. J. N. Trends Educ. Implic. (IJONTE). 14 , 187–201 (2023).

Google Scholar  

Moore, J. L., Dickson-Deane, C. & Galyen, K. e-Learning, online learning, and distance learning environments: Are they the same?. Internet High. Educ. 14 , 129–135. https://doi.org/10.1016/j.iheduc.2010.10.001 (2011).

Sarker, S. & Wells, J. D. Understanding self-regulated learning in a fifth-grade mathematics classroom through a cultural-historical activity theory lens. Educ. Stud. Math. 54 , 211–236. https://doi.org/10.1023/A:1027311524535 (2003).

Cheng, X. et al. Investigating students’ satisfaction with online collaborative learning during the COVID-19 period: An expectation-confirmation model. Group Decis. Negot. 32 , 749–778. https://doi.org/10.1007/s10726-023-09829-x (2023).

Article   PubMed   PubMed Central   Google Scholar  

Xie, Y. & Suh, K. Understanding adult learners’ satisfaction and continuance intention in hybrid learning environments: An integrative review. Educ. Technol. Soc. 22 , 177–192. https://doi.org/10.2307/jeductechsoci.22.4.177 (2019).

Broadbent, J. & Poon, W. L. Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. Internet High. Educ. 27 , 1–13. https://doi.org/10.1016/j.iheduc.2015.06.003 (2015).

Hossain, M. N. et al. Investigating the factors driving adult learners’ continuous intention to use M-learning application: A fuzzy-set analysis. J. Res. Innov. Teach. Learn. 14 , 245–270 (2020).

Mailloux, M. L. Adult online learners with multiple roles: A qualitative study of emotional experiences. (2023).

Artino, A. R. J. & Stephens, J. M. Academic motivation and self-regulation: A comparative analysis of undergraduate and graduate students learning online. Internet High. Educ. 12 , 146–151. https://doi.org/10.1016/j.iheduc.2009.03.001 (2009).

Chen, K. C. & Jang, S. J. Motivation in online learning: Testing a model of self-determination theory. Comput. Hum. Behav. 26 , 741–752. https://doi.org/10.1016/j.chb.2010.01.014 (2010).

Lee, J., Song, H. & Kim, Y. Quality factors that influence the continuance intention to use MOOCs an expectation-confirmation perspective. Eur. J. Psychol. Open. 82 , 109–119. https://doi.org/10.1024/2673-8627/a000047 (2023).

Chen, L. & Wu, H. The mediating role of service quality in the relationship between platform features and adult learners’ continuance intention in online education. Educ. Technol. Res. Dev. 20 , 78–91. https://doi.org/10.1007/s11423-022-10058-z (2023).

Lu, J., Yao, J. E. & Yu, C. S. Personal innovativeness, social influences, and adoption of wireless internet services via mobile technology. J. Strateg. Inf. Syst. 14 , 245–268. https://doi.org/10.1016/j.jsis.2005.07.003 (2005).

Qian, Y. Research on the influencing factors of continuous use behavior of online learners—From the perspective of social network environment and learning situation positioning. Mod. Inf. 35 , 50–56 (2015).

Wu, B. & Zhang, C. Empirical study on continuance intentions towards e-learning 2.0 systems. Behav. Inf. Technol. 33 , 1027–1038. https://doi.org/10.1080/0144929X.2013.872200 (2014).

Yoon, C. & Rolland, E. Understanding continuance use in social networking services. J. Comput.-Mediat. Commun. 20 , 1–8. https://doi.org/10.1111/jcc4.12099 (2015).

Kwapong, O. A. T. F. Online learning experiences of adult applicants to a university in Ghana during the Covid-19 outbreak. E-Learn. Digit. M. 20 , 598–617. https://doi.org/10.1177/20427530221125858 (2023).

Cheng, Y. M. Why do users intend to continue using the digital library? An integrated perspective. Aslib. J. Inf. Manag. 66 , 1–25. https://doi.org/10.1108/AJIM-03-2014-0033 (2014).

Yang, G. F. Influence factors of MOOC user’s continuous behavior. Open Educ. Res. 22 , 100–111. https://doi.org/10.13966/j.cnki.kfjyyj.2016.01.012 (2016).

Yin, M. & Li, Q. Research on the continuance usage intention of mobile APP based on the theory of integrating ECT and IS success: Taking the health APP as an example. J. Dalian Univ. Technol. Soc. Sci. Edn. 38 , 81–87 (2017).

Tang, W., Zhang, X. & Tian, Y. Investigating lifelong learners’ continuing learning intention moderated by affective support in online learning. Sustain. Basel. 15 , 1901. https://doi.org/10.3390/su15031901 (2023).

Oliver, R. L. A cognitive model of the antecedents and consequences of satisfaction decisions. J. Market. Res. 17 , 460–469. https://doi.org/10.2307/3150499 (1980).

Patterson, P. G., Johnson, L. W. & Spreng, R. A. Modeling the determinants of customer satisfaction for business-to-business professional services. J. Acad. Market. Sci. 24 , 4–17. https://doi.org/10.1177/009207039602400101 (1996).

Bhattacherjee, A. Understanding information systems continuance: An expectation-confirmation model. Mis. Quart. 25 , 351–370. https://doi.org/10.2307/3250921 (2001).

Larsen, T. J., Sørebø, A. M. & Sørebø, Ø. The role of task-technology fit as users’ motivation to continue information system use. Comput. Hum. Behav. 25 , 778–784. https://doi.org/10.1016/j.chb.2009.02.004 (2009).

Tan, C. H., Yi, Y. & Li, L. Research on the influencing factors of the continuous use intention of users of academic WeChat public accounts. Mod. Inf. 41 , 50–136 (2021).

Doong, H. S. & Lai, H. Exploring usage continuance of e-negotiation systems: Expectation and disconfirmation approach. Group Decis. Negot. 17 , 111–126. https://doi.org/10.1007/s10726-007-9077-8 (2008).

Kim, B. An empirical investigation of mobile data service continuance: Incorporating the theory of planned behavior into the expectation-confirmation model. Expert Syst. Appl. 37 , 7033–7039. https://doi.org/10.1016/j.eswa.2010.03.028 (2010).

Dağhan, G. & Akkoyunlu, B. Modeling the continuance usage intention of online learning environments. Comput. Hum. Behav. 60 , 198–211. https://doi.org/10.1016/j.chb.2016.02.095 (2016).

DeLone, W. H. & McLean, E. R. The DeLone and McLean model of information systems success: A ten-year update. J. Manag. Inf. Syst. 19 , 9–30. https://doi.org/10.1080/07421222.2003.11045748 (2003).

Guo, W. Q. Research on the influencing factors for the continuous use intention of online education paid users. Master’s thesis Type, Jilin University (2017).

Guo, X. D. Research on the influencing factors of online education paid users' continuance usage intention. Master’s thesis Type, Lanzhou University of Finance and Economics (2018).

Liu, C. & Wang, D. Understanding the influence of service quality on adult learners’ satisfaction and continuance intention in online education. Int. J. Educ. Technol. 15 , 112–125. https://doi.org/10.1504/IJET.2022.129783 (2022).

Li, S. & Wang, Y. Effects of course features on sustained usage intention: A study of adult learners. J. Distance Educ. 35 , 112–125. https://doi.org/10.5325/jode.35.1.112 (2023).

Liu, G. Q. Analysis of the characteristics of American primary and secondary school education websites and their implications for China. Prim. Second. Sch. Electron. Educ. 2010 , 43–45 (2010).

Mou, S. Interactive research in the network teaching environment. Silicon Valley . 58 (2011).

Ma, L., Xing, Y., Wu, X. Research on influencing factors of intention to use course websites. China Educ. Technol . 72–78 (2013).

De Chernatony, L., Harris, F. & Riley, F. D. Added value: Its nature, roles and sustainability. Eur. J. Market. 30 , 39–56. https://doi.org/10.1108/03090569610105723 (1996).

Xie, S., Hou, M. Empirical research on purchase intentions for air passenger add-on services based on customer perception. Price Mon . (2016). https://doi.org/10.14076/j.issn.1006-2025.2016.08.14 .

Huang, L. & Li, W. Enhancing online learning experience through perceived added value. Int. J. E-Learn. 28 , 78–91. https://doi.org/10.1891/2696-7895.28.2.7 (2022).

Guo, X. Y., Guo, Z. & Li, M. The mechanism of classroom interaction in SPOC on student participation. Exp. Technol. Manag. 36 , 244–273. https://doi.org/10.16791/j.cnki.sjg.2019.06.058 (2019).

Wu, A. Research on the continuous use intention of online education platform users: Based on the verification of expectation confirmation theory model. J. Harbin Inst. 39 , 117–122 (2018).

Liu, Y. & Chen, H. The impact of value-added services on perceived usefulness in mobile communication: A study of adult consumers. J. Mob. Serv. 30 , 46–58. https://doi.org/10.4067/S2236-74392023000200046 (2023).

Davis, F. D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13 , 319–340. https://doi.org/10.2307/249008 (1989).

Hayashi, A. et al. The role of social presence and moderating role of computer self efficacy in predicting the continuance usage of e-learning systems. J. Inf. Syst. Educ. 31 , 5–10 (2020).

Lin, C. S., Wu, S. & Tsai, R. J. Integrating perceived playfulness into expectation-confirmation model for web portal continuance. Inf. Manag.-Amster. 42 , 683–693. https://doi.org/10.1016/j.im.2004.04.003 (2005).

Liu, R. & Chai, J. Study on the influencing factors of continuous usage behavior of personal users in SNS social network. Soft Sci. 27 , 132–175 (2013).

ADS   Google Scholar  

Chiu, C. M., Hsu, M. H. & Lai, H. Re-examining the influence of trust on online repeat purchase intention: The moderating role of habit and its antecedents. Decis. Support Syst. 53 , 835–845. https://doi.org/10.1016/j.dss.2012.05.014 (2012).

Wang, L., Zhao, W. & Sun, X. Modeling of causes of Sina Weibo continuance intention with mediation of gender effects. Front. Psychol. 7 , 1–11. https://doi.org/10.3389/fpsyg.2016.01716 (2016).

Dastane, O. & Haba, H. F. What drives mobile MOOC’s continuous intention? A theory of perceived value perspective. Int. J. Inf. Learn. Technol. 40 , 148–163. https://doi.org/10.1108/IJILT-04-2022-0087 (2023).

Luo, Y. & Wang, F. A qualitative study of the factors influencing the quality of online learning in higher education. Internet High. Educ. 54 , 100834. https://doi.org/10.1016/j.iheduc.2023.100834 (2023).

Wang, F. & Li, M. Research on adult learners’ motivation to learn on online education platforms. Chin. Adult Educ. 33 , 11–16. https://doi.org/10.19387/j.cnki.cn44-1144/g4.2023.02.011 (2023).

Wang, D., Tao, B. X. & Zheng, G. M. Research on the influencing factors of the continuous usage behavior of VR library service users based on expectation confirmation theory. Mod. Inf. 40 , 111–120 (2020).

Yin, M. Z., Yu, W. P., Zhou, T. M. An empirical study on the influence factors of university library WeChat users' continuous use willingness. Libr. Theory Pract . 98–101 (2017).

Wang, X. & Liu, Y. Research on the quality evaluation of online education platforms for adult learners. J. Contin. Educ. 21 , 12–18. https://doi.org/10.15933/j.cnki.cn44-1277/g4.2021.01.012 (2021).

Zhao, L. & Sun, Y. Research on adult learners’ learning behaviors on online education platforms. Res. Mod. Distance Educ. 25 , 1–8. https://doi.org/10.16060/j.cnki.rmde.2023.01.001 (2023).

Jawaid, M. & Haleem, A. A review of online learning platforms: Features, benefits, and challenges. Int. J. Emerg. Technol. 17 , 19–45. https://doi.org/10.3991/ijet.v17i02.28197 (2022).

Cao, Y. Y., Li, J. J. & Qin, X. H. Post-adoption behavior of SNS users: An integrative model of emotional attachment and ECM-IS. Mod. Inf. 36 , 81–88 (2016).

Zhang, H. & Yao, R. H. Research on the influencing factors of the continuous use intention of mobile government APP users from the perspective of ECM-IS. J. Chongqing Univ. Posts Telecommun. (Soc. Sci. Edn). 32 , 92–101. https://doi.org/10.3969/j.issn.1673-5188.2020.01.14 (2020).

Gao, L. & Hu, C. Analysis of the influencing factors on user’s continuous usage behavior in online knowledge community service. Mod. Inf. 34 , 14–17 (2014).

Lin, L. Research on the Influencing Factors of Consumers' Mobile Shopping Behavior in App . (Beijing Institute of Technology, 2015).

Zhou, P., Fu, S. Y. & Zhao, Y. C. Empirical research on the influencing factors of shopping app users’ continuous use. J. Nanjing Norm. Univ. (Nat. Sci. Edn). 43 , 140–148. https://doi.org/10.3969/j.issn.1001-4616.2020.02.15 (2020).

Wang, F. Research on the influencing factors of the continuous use intention of mobile library users. J. Libr. Work Study. 2017 , 50–56. https://doi.org/10.16384/j.cnki.lwas.2017.07.009 (2017).

Guo, C. & Ming, J. Integrated model and empirical research on the continuous use intention of mobile library users. Mod. Inf. 40 , 79–89 (2020).

Hsu, C. L. & Lin, J. C. What drives purchase intention for paid mobile apps?—An expectation confirmation model with perceived value. Electron. Commer. R A. 14 , 46–57. https://doi.org/10.1016/j.elerap.2014.11.002 (2015).

Yang, G. F. Research on the influencing factors of mobile reading user satisfaction and continuous use intention: Taking content aggregation APP as an example. Mod. Inf. 35 , 57–63 (2015).

CAS   Google Scholar  

Alali, H. & Salim, J. Virtual communities of practice success model to support knowledge sharing behaviour in healthcare sector. Proc. Technol. 9 , 176–183. https://doi.org/10.1016/j.protcy.2013.12.019 (2013).

Hung, M. C., Hwang, H. G. & Hsieh, T. C. An exploratory study on the continuance of mobile commerce: An extension-confirmation model of information system use. Int. J. Mob. Commun. 5 , 409–422 (2007).

Thong, J. Y. L., Hong, S. J. & Tam, K. Y. The effects of post-adoption beliefs on the expectation-confirmation model for technology continuance. Int. J. Hum.-Comput. Stud. 64 , 799–810. https://doi.org/10.1016/j.ijhcs.2006.05.001 (2006).

Liu, H., Pei, L. & Sun, J. An empirical analysis of the continuous usage of video website users based on expectation confirmation model. Inf. Knowl. 2014 , 94–103. https://doi.org/10.13366/j.dik.2014.03.094 (2014).

Yang, T. User continuous use behavior of electronic books: Extension of the expected confirmation model. J. Libr. Sci. Res. 2016 , 76–83. https://doi.org/10.15941/j.cnki.issn1001-0424.2016.22.014 (2016).

Joo, S. & Choi, N. Understanding users’ continuance intention to use online library resources based on an extended expectation-confirmation model. Electron. Libr. 34 , 554–571. https://doi.org/10.1108/EL-02-2015-0025 (2016).

Chow, W. S. & Shi, S. Investigating students’ satisfaction and continuance intention toward e-learning: An extension of the expectation-confirmation model. Proc. Soc. Behav. Sci. 141 , 1145–1149. https://doi.org/10.1016/j.sbspro.2014.05.197 (2014).

Hoffman, D. L. & Novak, T. P. Marketing in hypermedia computer-mediated environments: Conceptual foundations. J. Market. 60 , 50–68 (1996).

Zhang, C. B. & Wu, B. Review of website perceived interactivity research. China Circ. Econ. 30 , 117–127. https://doi.org/10.14089/j.cnki.cn11-3664/f.2016.06.017 (2016).

Van Noort, G., Voorveld, H. A. & Van Reijmersdal, E. A. Interactivity in brand websites: Cognitive, affective, and behavioral responses explained by consumers’ online flow experience. J. Interact. Mark. 26 , 223–234. https://doi.org/10.1016/j.intmar.2011.11.002 (2012).

Gefen, D., Karahanna, E. & Straub, D. W. Trust and TAM in online shopping: An integrated model. MIS Q. 27 , 51–90. https://doi.org/10.2307/30036519 (2003).

Kim, J., Spielmann, N. & McMillan, S. J. Experience effects on interactivity: Functions, processes, and perception in online shopping. J. Bus. Res. 65 , 1543–1550. https://doi.org/10.1016/j.jbusres.2011.10.013 (2012).

Park, M. & Park, J. K. Exploring the influences of perceived interactivity on consumers’ e-shopping effectiveness. J. Custom. Behav. 8 , 361–379. https://doi.org/10.1362/147539209X480990 (2009).

Song, J. H. & Zinkhan, G. M. Determinants of perceived website interactivity. J. Market. 72 , 99–113. https://doi.org/10.1509/jmkg.72.2.99 (2008).

Lee, M. C. Explaining and predicting users’ continuance intention toward e-learning: An extension of the expectation-confirmation model. Comput. Educ. 54 (2), 506–516 (2010).

Venkatesh, R. Computation of elastic moduli and dispersion in (s-d) interactive atomic liquids. The example of liquid Pt and Pd metals. Phys. Status Solidi (b). 176 (1), 91–99 (1993).

Article   ADS   CAS   Google Scholar  

Wang, Y. S. & Liao, Y. W. Assessing e-government systems success: A validation of the Delone and McLean model of information systems success. Gov. Inf. Q. 25 (4), 717–733 (2008).

Roca, J. C., Chiu, C. M. & Martínez, F. J. Understanding e-learning continuance intention: An extension of the technology acceptance model. Int. J. Hum.-Comput. Stud. 64 (8), 683–696 (2006).

Kettanurak, V. N., Ramamurthy, K. & Haseman, W. D. User attitude as a mediator of learning performance improvement in an interactive multimedia environment: An empirical investigation of the degree of interactivity and learning styles. Int. J. Hum. Comput. Stud. 54 (4), 541–583 (2001).

Yang, G. F. Research on influencing factors of continuance usage intention of MOOC users. Open Educ. Res. 22 (1), 100–111 (2016).

Zhao, L. & Lu, Y. Enhancing perceived interactivity through network externalities: An empirical study on micro-blogging service satisfaction and continuance intention. Decis. Support Syst. 53 (4), 825–834 (2012).

Grönroos, C. Keynote paper from marketing mix to relationship marketing—Towards a paradigm shift in marketing. Manag. Decis. 35 (4), 322–399 (1997).

Rodríguez-Ardura, I. & Meseguer-Artola, A. What leads people to keep on e-learning? An empirical analysis of learners’ experiences and their effects on continuance intention. Interact. Learn. Environ. 24 (6), 1030–1053 (2016).

Podsakoff, P. M. et al. Common method biases in behavioral research: A critical review of the literature and recommended remedies. J. Appl. Psychol. 88 , 879–950. https://doi.org/10.1037/0021-9010.88.5.879 (2003).

Article   PubMed   Google Scholar  

Jordan, P. J. & Troth, A. C. Common method bias in applied settings: The dilemma of researching in organizations. Aust. J. Manag. 45 , 3–14. https://doi.org/10.1177/0312896219871976 (2020).

Brunner, M. & Suess, H. How precisely can cognitive abilities be measured? The distinction between composite and construct reliabilities in intelligence assessment exemplified with the Berlin Intelligence Structure Test. Diagnostica. 53 , 184–193. https://doi.org/10.1026/0012-1924.53.4.184 (2007).

Peterson, R. A. & Kim, Y. On the relationship between coefficient alpha and composite reliability. J. Appl. Psychol. 98 , 194–198. https://doi.org/10.1037/a0030767 (2013).

Aguirre-Urreta, M. I., Ronkko, M. & McIntosh, C. N. A cautionary note on the finite sample behavior of maximal reliability. Psychol. Methods. 24 , 236–252. https://doi.org/10.1037/met0000176 (2019).

Koran, J. Indicators per factor in confirmatory factor analysis: More is not always better. Struct. Equ. Model. Multidiscip. J. 27 , 765–772. https://doi.org/10.1080/10705511.2019.1706527 (2020).

Article   MathSciNet   Google Scholar  

Fu, Y., Wen, Z. & Wang, Y. A comparison of reliability estimation based on confirmatory factor analysis and exploratory structural equation models. Educ. Psychol. Meas. 82 , 205–224. https://doi.org/10.1177/00131644211008953 (2022).

Dhami, S. K. & Al-Emran, M. A meta-analysis of the factors affecting the quality of online learning. Int. J. Educ. Technol. H. 19 , 1–22. https://doi.org/10.1186/s41239-022-00403-3 (2022).

Kim, S. Y. & Frick, T. W. Factors influencing adult learners’ satisfaction with online learning: A meta-analysis. Internet High. Educ. 53 , 100801. https://doi.org/10.1016/j.iheduc.2022.100801 (2022).

Oluwatayo, A. E. & Adepoju, T. O. The challenges and opportunities of online learning for adult learners: A systematic review. Int. J. Educ. Technol. H. 20 , 1–27. https://doi.org/10.1186/s41239-022-00422-0 (2023).

Alzahrani, A. I. & Aljohani, N. R. A systematic review of online learning platforms: Features, affordances, and limitations. J. Comput. Assist. Learn. 37 , 559–588. https://doi.org/10.1111/jcal.12527 (2021).

Sun, C. Teaching reform in classrooms based on college students' course experience and modern educational technology in China. (2021).

Zhang, H. & Zhao, G. Research on adult learners’ satisfaction with online education platforms. J. Distance Educ. 35 , 10–16. https://doi.org/10.16009/j.cnki.cn44-1162/g4.2023.03.010 (2023).

Lee, K. M. et al. Autonomic machine learning platform. Int. J. Inf. Manag. 49 , 491–501. https://doi.org/10.1016/j.ijinfomgt.2019.07.003 (2019).

Lanubile, F., Martinez-Fernandez, S. & Quaranta, L. Training future machine learning engineers: A project-based course on MLOps. IEEE Softw. 41 , 60–67. https://doi.org/10.1109/MS.2023.3310768 (2024).

Alhothali, A. et al. Predicting student outcomes in online courses using machine learning techniques: A review. Sustainability Basel https://doi.org/10.3390/su14106199 (2022).

Chen, Y. Association analysis of online learning behaviour in interactive education based on an intelligent concept machine. Int. J. Contin. Eng. Educ. 30 , 161–175 (2020).

Alzahrani, A. I. & Aljohani, N. R. Factors affecting the quality of online learning: A systematic review of the literature. Educ. Technol. Soc. 24 , 28–48. https://doi.org/10.14751/et&s.2021.423 (2021).

Download references

Acknowledgements

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Human Participants Ethics Committee Shanghai Normal University (No. 202345). We confirm that we have obtained informed consent from all participants.

This research received no external funding.

Author information

Authors and affiliations.

College of Continuing Education, Shanghai Normal University, Shanghai, China

Guoqiang Pan

School of Humanities, Shanghai Normal University, Shanghai, China

School of Education, Shanghai Normal University, Shanghai, China

Ziyuan Song

School of Foreign Language Studies, Shanghai University of Engineering Science, Shanghai, China

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, G.P. and H.N.; methodology, G.P. and H.N.; validation, Z.S; investigation, G.P. and Y.M.; data curation, Y.M. and Z.Y.; writing—original draft preparation, G.P. and H.N.; writing—review and editing, G.P.; supervision, H.N.; project administration, G.P. All authors have read and agreed to the published version of the manuscript.

Corresponding authors

Correspondence to Guoqiang Pan or Hui Nie .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Pan, G., Mao, Y., Song, Z. et al. Research on the influencing factors of adult learners' intent to use online education platforms based on expectation confirmation theory. Sci Rep 14 , 12762 (2024). https://doi.org/10.1038/s41598-024-63747-9

Download citation

Received : 12 January 2024

Accepted : 31 May 2024

Published : 04 June 2024

DOI : https://doi.org/10.1038/s41598-024-63747-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Adult learners
• Online education platform
• Expectation confirmation model
• Expectation confirmation
• Continuance intention

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Elsevier - PMC COVID-19 Collection

A systematic review of research on online teaching and learning from 2009 to 2018

Associated data.

Systematic reviews were conducted in the nineties and early 2000's on online learning research. However, there is no review examining the broader aspect of research themes in online learning in the last decade. This systematic review addresses this gap by examining 619 research articles on online learning published in twelve journals in the last decade. These studies were examined for publication trends and patterns, research themes, research methods, and research settings and compared with the research themes from the previous decades. While there has been a slight decrease in the number of studies on online learning in 2015 and 2016, it has then continued to increase in 2017 and 2018. The majority of the studies were quantitative in nature and were examined in higher education. Online learning research was categorized into twelve themes and a framework across learner, course and instructor, and organizational levels was developed. Online learner characteristics and online engagement were examined in a high number of studies and were consistent with three of the prior systematic reviews. However, there is still a need for more research on organization level topics such as leadership, policy, and management and access, culture, equity, inclusion, and ethics and also on online instructor characteristics.

• • Twelve online learning research themes were identified in 2009–2018.
• • A framework with learner, course and instructor, and organizational levels was used.
• • Online learner characteristics and engagement were the mostly examined themes.
• • The majority of the studies used quantitative research methods and in higher education.
• • There is a need for more research on organization level topics.

1. Introduction

Online learning has been on the increase in the last two decades. In the United States, though higher education enrollment has declined, online learning enrollment in public institutions has continued to increase ( Allen & Seaman, 2017 ), and so has the research on online learning. There have been review studies conducted on specific areas on online learning such as innovations in online learning strategies ( Davis et al., 2018 ), empirical MOOC literature ( Liyanagunawardena et al., 2013 ; Veletsianos & Shepherdson, 2016 ; Zhu et al., 2018 ), quality in online education ( Esfijani, 2018 ), accessibility in online higher education ( Lee, 2017 ), synchronous online learning ( Martin et al., 2017 ), K-12 preparation for online teaching ( Moore-Adams et al., 2016 ), polychronicity in online learning ( Capdeferro et al., 2014 ), meaningful learning research in elearning and online learning environments ( Tsai, Shen, & Chiang, 2013 ), problem-based learning in elearning and online learning environments ( Tsai & Chiang, 2013 ), asynchronous online discussions ( Thomas, 2013 ), self-regulated learning in online learning environments ( Tsai, Shen, & Fan, 2013 ), game-based learning in online learning environments ( Tsai & Fan, 2013 ), and online course dropout ( Lee & Choi, 2011 ). While there have been review studies conducted on specific online learning topics, very few studies have been conducted on the broader aspect of online learning examining research themes.

2. Systematic Reviews of Distance Education and Online Learning Research

Distance education has evolved from offline to online settings with the access to internet and COVID-19 has made online learning the common delivery method across the world. Tallent-Runnels et al. (2006) reviewed research late 1990's to early 2000's, Berge and Mrozowski (2001) reviewed research 1990 to 1999, and Zawacki-Richter et al. (2009) reviewed research in 2000–2008 on distance education and online learning. Table 1 shows the research themes from previous systematic reviews on online learning research. There are some themes that re-occur in the various reviews, and there are also new themes that emerge. Though there have been reviews conducted in the nineties and early 2000's, there is no review examining the broader aspect of research themes in online learning in the last decade. Hence, the need for this systematic review which informs the research themes in online learning from 2009 to 2018. In the following sections, we review these systematic review studies in detail.

Comparison of online learning research themes from previous studies.

2.1. Distance education research themes, 1990 to 1999 ( Berge & Mrozowski, 2001 )

Berge and Mrozowski (2001) reviewed 890 research articles and dissertation abstracts on distance education from 1990 to 1999. The four distance education journals chosen by the authors to represent distance education included, American Journal of Distance Education, Distance Education, Open Learning, and the Journal of Distance Education. This review overlapped in the dates of the Tallent-Runnels et al. (2006) study. Berge and Mrozowski (2001) categorized the articles according to Sherry's (1996) ten themes of research issues in distance education: redefining roles of instructor and students, technologies used, issues of design, strategies to stimulate learning, learner characteristics and support, issues related to operating and policies and administration, access and equity, and costs and benefits.

In the Berge and Mrozowski (2001) study, more than 100 studies focused on each of the three themes: (1) design issues, (2) learner characteristics, and (3) strategies to increase interactivity and active learning. By design issues, the authors focused on instructional systems design and focused on topics such as content requirement, technical constraints, interactivity, and feedback. The next theme, strategies to increase interactivity and active learning, were closely related to design issues and focused on students’ modes of learning. Learner characteristics focused on accommodating various learning styles through customized instructional theory. Less than 50 studies focused on the three least examined themes: (1) cost-benefit tradeoffs, (2) equity and accessibility, and (3) learner support. Cost-benefit trade-offs focused on the implementation costs of distance education based on school characteristics. Equity and accessibility focused on the equity of access to distance education systems. Learner support included topics such as teacher to teacher support as well as teacher to student support.

2.2. Online learning research themes, 1993 to 2004 ( Tallent-Runnels et al., 2006 )

Tallent-Runnels et al. (2006) reviewed research on online instruction from 1993 to 2004. They reviewed 76 articles focused on online learning by searching five databases, ERIC, PsycINFO, ContentFirst, Education Abstracts, and WilsonSelect. Tallent-Runnels et al. (2006) categorized research into four themes, (1) course environment, (2) learners' outcomes, (3) learners’ characteristics, and (4) institutional and administrative factors. The first theme that the authors describe as course environment ( n  = 41, 53.9%) is an overarching theme that includes classroom culture, structural assistance, success factors, online interaction, and evaluation.

Tallent-Runnels et al. (2006) for their second theme found that studies focused on questions involving the process of teaching and learning and methods to explore cognitive and affective learner outcomes ( n  = 29, 38.2%). The authors stated that they found the research designs flawed and lacked rigor. However, the literature comparing traditional and online classrooms found both delivery systems to be adequate. Another research theme focused on learners’ characteristics ( n  = 12, 15.8%) and the synergy of learners, design of the online course, and system of delivery. Research findings revealed that online learners were mainly non-traditional, Caucasian, had different learning styles, and were highly motivated to learn. The final theme that they reported was institutional and administrative factors (n  = 13, 17.1%) on online learning. Their findings revealed that there was a lack of scholarly research in this area and most institutions did not have formal policies in place for course development as well as faculty and student support in training and evaluation. Their research confirmed that when universities offered online courses, it improved student enrollment numbers.

2.3. Distance education research themes 2000 to 2008 ( Zawacki-Richter et al., 2009 )

Zawacki-Richter et al. (2009) reviewed 695 articles on distance education from 2000 to 2008 using the Delphi method for consensus in identifying areas and classified the literature from five prominent journals. The five journals selected due to their wide scope in research in distance education included Open Learning, Distance Education, American Journal of Distance Education, the Journal of Distance Education, and the International Review of Research in Open and Distributed Learning. The reviewers examined the main focus of research and identified gaps in distance education research in this review.

Zawacki-Richter et al. (2009) classified the studies into macro, meso and micro levels focusing on 15 areas of research. The five areas of the macro-level addressed: (1) access, equity and ethics to deliver distance education for developing nations and the role of various technologies to narrow the digital divide, (2) teaching and learning drivers, markets, and professional development in the global context, (3) distance delivery systems and institutional partnerships and programs and impact of hybrid modes of delivery, (4) theoretical frameworks and models for instruction, knowledge building, and learner interactions in distance education practice, and (5) the types of preferred research methodologies. The meso-level focused on seven areas that involve: (1) management and organization for sustaining distance education programs, (2) examining financial aspects of developing and implementing online programs, (3) the challenges and benefits of new technologies for teaching and learning, (4) incentives to innovate, (5) professional development and support for faculty, (6) learner support services, and (7) issues involving quality standards and the impact on student enrollment and retention. The micro-level focused on three areas: (1) instructional design and pedagogical approaches, (2) culturally appropriate materials, interaction, communication, and collaboration among a community of learners, and (3) focus on characteristics of adult learners, socio-economic backgrounds, learning preferences, and dispositions.

The top three research themes in this review by Zawacki-Richter et al. (2009) were interaction and communities of learning ( n  = 122, 17.6%), instructional design ( n  = 121, 17.4%) and learner characteristics ( n  = 113, 16.3%). The lowest number of studies (less than 3%) were found in studies examining the following research themes, management and organization ( n  = 18), research methods in DE and knowledge transfer ( n  = 13), globalization of education and cross-cultural aspects ( n  = 13), innovation and change ( n  = 13), and costs and benefits ( n  = 12).

2.4. Online learning research themes

These three systematic reviews provide a broad understanding of distance education and online learning research themes from 1990 to 2008. However, there is an increase in the number of research studies on online learning in this decade and there is a need to identify recent research themes examined. Based on the previous systematic reviews ( Berge & Mrozowski, 2001 ; Hung, 2012 ; Tallent-Runnels et al., 2006 ; Zawacki-Richter et al., 2009 ), online learning research in this study is grouped into twelve different research themes which include Learner characteristics, Instructor characteristics, Course or program design and development, Course Facilitation, Engagement, Course Assessment, Course Technologies, Access, Culture, Equity, Inclusion, and Ethics, Leadership, Policy and Management, Instructor and Learner Support, and Learner Outcomes. Table 2 below describes each of the research themes and using these themes, a framework is derived in Fig. 1 .

Research themes in online learning.

Online learning research themes framework.

The collection of research themes is presented as a framework in Fig. 1 . The themes are organized by domain or level to underscore the nested relationship that exists. As evidenced by the assortment of themes, research can focus on any domain of delivery or associated context. The “Learner” domain captures characteristics and outcomes related to learners and their interaction within the courses. The “Course and Instructor” domain captures elements about the broader design of the course and facilitation by the instructor, and the “Organizational” domain acknowledges the contextual influences on the course. It is important to note as well that due to the nesting, research themes can cross domains. For example, the broader cultural context may be studied as it pertains to course design and development, and institutional support can include both learner support and instructor support. Likewise, engagement research can involve instructors as well as learners.

In this introduction section, we have reviewed three systematic reviews on online learning research ( Berge & Mrozowski, 2001 ; Tallent-Runnels et al., 2006 ; Zawacki-Richter et al., 2009 ). Based on these reviews and other research, we have derived twelve themes to develop an online learning research framework which is nested in three levels: learner, course and instructor, and organization.

2.5. Purpose of this research

In two out of the three previous reviews, design, learner characteristics and interaction were examined in the highest number of studies. On the other hand, cost-benefit tradeoffs, equity and accessibility, institutional and administrative factors, and globalization and cross-cultural aspects were examined in the least number of studies. One explanation for this may be that it is a function of nesting, noting that studies falling in the Organizational and Course levels may encompass several courses or many more participants within courses. However, while some research themes re-occur, there are also variations in some themes across time, suggesting the importance of research themes rise and fall over time. Thus, a critical examination of the trends in themes is helpful for understanding where research is needed most. Also, since there is no recent study examining online learning research themes in the last decade, this study strives to address that gap by focusing on recent research themes found in the literature, and also reviewing research methods and settings. Notably, one goal is to also compare findings from this decade to the previous review studies. Overall, the purpose of this study is to examine publication trends in online learning research taking place during the last ten years and compare it with the previous themes identified in other review studies. Due to the continued growth of online learning research into new contexts and among new researchers, we also examine the research methods and settings found in the studies of this review.

The following research questions are addressed in this study.

• 1. What percentage of the population of articles published in the journals reviewed from 2009 to 2018 were related to online learning and empirical?
• 2. What is the frequency of online learning research themes in the empirical online learning articles of journals reviewed from 2009 to 2018?
• 3. What is the frequency of research methods and settings that researchers employed in the empirical online learning articles of the journals reviewed from 2009 to 2018?

This five-step systematic review process described in the U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse Procedures and Standards Handbook, Version 4.0 ( 2017 ) was used in this systematic review: (a) developing the review protocol, (b) identifying relevant literature, (c) screening studies, (d) reviewing articles, and (e) reporting findings.

3.1. Data sources and search strategies

The Education Research Complete database was searched using the keywords below for published articles between the years 2009 and 2018 using both the Title and Keyword function for the following search terms.

“online learning" OR "online teaching" OR "online program" OR "online course" OR “online education”

3.2. Inclusion/exclusion criteria

The initial search of online learning research among journals in the database resulted in more than 3000 possible articles. Therefore, we limited our search to select journals that focus on publishing peer-reviewed online learning and educational research. Our aim was to capture the journals that published the most articles in online learning. However, we also wanted to incorporate the concept of rigor, so we used expert perception to identify 12 peer-reviewed journals that publish high-quality online learning research. Dissertations and conference proceedings were excluded. To be included in this systematic review, each study had to meet the screening criteria as described in Table 3 . A research study was excluded if it did not meet all of the criteria to be included.

Inclusion/Exclusion criteria.

3.3. Process flow selection of articles

Fig. 2 shows the process flow involved in the selection of articles. The search in the database Education Research Complete yielded an initial sample of 3332 articles. Targeting the 12 journals removed 2579 articles. After reviewing the abstracts, we removed 134 articles based on the inclusion/exclusion criteria. The final sample, consisting of 619 articles, was entered into the computer software MAXQDA ( VERBI Software, 2019 ) for coding.

Flowchart of online learning research selection.

3.4. Developing review protocol

A review protocol was designed as a codebook in MAXQDA ( VERBI Software, 2019 ) by the three researchers. The codebook was developed based on findings from the previous review studies and from the initial screening of the articles in this review. The codebook included 12 research themes listed earlier in Table 2 (Learner characteristics, Instructor characteristics, Course or program design and development, Course Facilitation, Engagement, Course Assessment, Course Technologies, Access, Culture, Equity, Inclusion, and Ethics, Leadership, Policy and Management, Instructor and Learner Support, and Learner Outcomes), four research settings (higher education, continuing education, K-12, corporate/military), and three research designs (quantitative, qualitative and mixed methods). Fig. 3 below is a screenshot of MAXQDA used for the coding process.

Codebook from MAXQDA.

3.5. Data coding

Research articles were coded by two researchers in MAXQDA. Two researchers independently coded 10% of the articles and then discussed and updated the coding framework. The second author who was a doctoral student coded the remaining studies. The researchers met bi-weekly to address coding questions that emerged. After the first phase of coding, we found that more than 100 studies fell into each of the categories of Learner Characteristics or Engagement, so we decided to pursue a second phase of coding and reexamine the two themes. Learner Characteristics were classified into the subthemes of Academic, Affective, Motivational, Self-regulation, Cognitive, and Demographic Characteristics. Engagement was classified into the subthemes of Collaborating, Communication, Community, Involvement, Interaction, Participation, and Presence.

3.6. Data analysis

Frequency tables were generated for each of the variables so that outliers could be examined and narrative data could be collapsed into categories. Once cleaned and collapsed into a reasonable number of categories, descriptive statistics were used to describe each of the coded elements. We first present the frequencies of publications related to online learning in the 12 journals. The total number of articles for each journal (collectively, the population) was hand-counted from journal websites, excluding editorials and book reviews. The publication trend of online learning research was also depicted from 2009 to 2018. Then, the descriptive information of the 12 themes, including the subthemes of Learner Characteristics and Engagement were provided. Finally, research themes by research settings and methodology were elaborated.

4.1. Publication trends on online learning

Publication patterns of the 619 articles reviewed from the 12 journals are presented in Table 4 . International Review of Research in Open and Distributed Learning had the highest number of publications in this review. Overall, about 8% of the articles appearing in these twelve journals consisted of online learning publications; however, several journals had concentrations of online learning articles totaling more than 20%.

Empirical online learning research articles by journal, 2009–2018.

Note . Journal's Total Article count excludes reviews and editorials.

The publication trend of online learning research is depicted in Fig. 4 . When disaggregated by year, the total frequency of publications shows an increasing trend. Online learning articles increased throughout the decade and hit a relative maximum in 2014. The greatest number of online learning articles ( n  = 86) occurred most recently, in 2018.

Online learning publication trends by year.

4.2. Online learning research themes that appeared in the selected articles

The publications were categorized into the twelve research themes identified in Fig. 1 . The frequency counts and percentages of the research themes are provided in Table 5 below. A majority of the research is categorized into the Learner domain. The fewest number of articles appears in the Organization domain.

Research themes in the online learning publications from 2009 to 2018.

The specific themes of Engagement ( n  = 179, 28.92%) and Learner Characteristics ( n  = 134, 21.65%) were most often examined in publications. These two themes were further coded to identify sub-themes, which are described in the next two sections. Publications focusing on Instructor Characteristics ( n  = 21, 3.39%) were least common in the dataset.

4.2.1. Research on engagement

The largest number of studies was on engagement in online learning, which in the online learning literature is referred to and examined through different terms. Hence, we explore this category in more detail. In this review, we categorized the articles into seven different sub-themes as examined through different lenses including presence, interaction, community, participation, collaboration, involvement, and communication. We use the term “involvement” as one of the terms since researchers sometimes broadly used the term engagement to describe their work without further description. Table 6 below provides the description, frequency, and percentages of the various studies related to engagement.

Research sub-themes on engagement.

In the sections below, we provide several examples of the different engagement sub-themes that were studied within the larger engagement theme.

Presence. This sub-theme was the most researched in engagement. With the development of the community of inquiry framework most of the studies in this subtheme examined social presence ( Akcaoglu & Lee, 2016 ; Phirangee & Malec, 2017 ; Wei et al., 2012 ), teaching presence ( Orcutt & Dringus, 2017 ; Preisman, 2014 ; Wisneski et al., 2015 ) and cognitive presence ( Archibald, 2010 ; Olesova et al., 2016 ).

Interaction . This was the second most studied theme under engagement. Researchers examined increasing interpersonal interactions ( Cung et al., 2018 ), learner-learner interactions ( Phirangee, 2016 ; Shackelford & Maxwell, 2012 ; Tawfik et al., 2018 ), peer-peer interaction ( Comer et al., 2014 ), learner-instructor interaction ( Kuo et al., 2014 ), learner-content interaction ( Zimmerman, 2012 ), interaction through peer mentoring ( Ruane & Koku, 2014 ), interaction and community building ( Thormann & Fidalgo, 2014 ), and interaction in discussions ( Ruane & Lee, 2016 ; Tibi, 2018 ).

Community. Researchers examined building community in online courses ( Berry, 2017 ), supporting a sense of community ( Jiang, 2017 ), building an online learning community of practice ( Cho, 2016 ), building an academic community ( Glazer & Wanstreet, 2011 ; Nye, 2015 ; Overbaugh & Nickel, 2011 ), and examining connectedness and rapport in an online community ( Bolliger & Inan, 2012 ; Murphy & Rodríguez-Manzanares, 2012 ; Slagter van Tryon & Bishop, 2012 ).

Participation. Researchers examined engagement through participation in a number of studies. Some of the topics include, participation patterns in online discussion ( Marbouti & Wise, 2016 ; Wise et al., 2012 ), participation in MOOCs ( Ahn et al., 2013 ; Saadatmand & Kumpulainen, 2014 ), features that influence students’ online participation ( Rye & Støkken, 2012 ) and active participation.

Collaboration. Researchers examined engagement through collaborative learning. Specific studies focused on cross-cultural collaboration ( Kumi-Yeboah, 2018 ; Yang et al., 2014 ), how virtual teams collaborate ( Verstegen et al., 2018 ), types of collaboration teams ( Wicks et al., 2015 ), tools for collaboration ( Boling et al., 2014 ), and support for collaboration ( Kopp et al., 2012 ).

Involvement. Researchers examined engaging learners through involvement in various learning activities ( Cundell & Sheepy, 2018 ), student engagement through various measures ( Dixson, 2015 ), how instructors included engagement to involve students in learning ( O'Shea et al., 2015 ), different strategies to engage the learner ( Amador & Mederer, 2013 ), and designed emotionally engaging online environments ( Koseoglu & Doering, 2011 ).

Communication. Researchers examined communication in online learning in studies using social network analysis ( Ergün & Usluel, 2016 ), using informal communication tools such as Facebook for class discussion ( Kent, 2013 ), and using various modes of communication ( Cunningham et al., 2010 ; Rowe, 2016 ). Studies have also focused on both asynchronous and synchronous aspects of communication ( Swaggerty & Broemmel, 2017 ; Yamagata-Lynch, 2014 ).

4.2.2. Research on learner characteristics

The second largest theme was learner characteristics. In this review, we explore this further to identify several aspects of learner characteristics. In this review, we categorized the learner characteristics into self-regulation characteristics, motivational characteristics, academic characteristics, affective characteristics, cognitive characteristics, and demographic characteristics. Table 7 provides the number of studies and percentages examining the various learner characteristics.

Research sub-themes on learner characteristics.

Online learning has elements that are different from the traditional face-to-face classroom and so the characteristics of the online learners are also different. Yukselturk and Top (2013) categorized online learner profile into ten aspects: gender, age, work status, self-efficacy, online readiness, self-regulation, participation in discussion list, participation in chat sessions, satisfaction, and achievement. Their categorization shows that there are differences in online learner characteristics in these aspects when compared to learners in other settings. Some of the other aspects such as participation and achievement as discussed by Yukselturk and Top (2013) are discussed in different research themes in this study. The sections below provide examples of the learner characteristics sub-themes that were studied.

Self-regulation. Several researchers have examined self-regulation in online learning. They found that successful online learners are academically motivated ( Artino & Stephens, 2009 ), have academic self-efficacy ( Cho & Shen, 2013 ), have grit and intention to succeed ( Wang & Baker, 2018 ), have time management and elaboration strategies ( Broadbent, 2017 ), set goals and revisit course content ( Kizilcec et al., 2017 ), and persist ( Glazer & Murphy, 2015 ). Researchers found a positive relationship between learner's self-regulation and interaction ( Delen et al., 2014 ) and self-regulation and communication and collaboration ( Barnard et al., 2009 ).

Motivation. Researchers focused on motivation of online learners including different motivation levels of online learners ( Li & Tsai, 2017 ), what motivated online learners ( Chaiprasurt & Esichaikul, 2013 ), differences in motivation of online learners ( Hartnett et al., 2011 ), and motivation when compared to face to face learners ( Paechter & Maier, 2010 ). Harnett et al. (2011) found that online learner motivation was complex, multifaceted, and sensitive to situational conditions.

Academic. Several researchers have focused on academic aspects for online learner characteristics. Readiness for online learning has been examined as an academic factor by several researchers ( Buzdar et al., 2016 ; Dray et al., 2011 ; Wladis & Samuels, 2016 ; Yu, 2018 ) specifically focusing on creating and validating measures to examine online learner readiness including examining students emotional intelligence as a measure of student readiness for online learning. Researchers have also examined other academic factors such as academic standing ( Bradford & Wyatt, 2010 ), course level factors ( Wladis et al., 2014 ) and academic skills in online courses ( Shea & Bidjerano, 2014 ).

Affective. Anderson and Bourke (2013) describe affective characteristics through which learners express feelings or emotions. Several research studies focused on the affective characteristics of online learners. Learner satisfaction for online learning has been examined by several researchers ( Cole et al., 2014 ; Dziuban et al., 2015 ; Kuo et al., 2013 ; Lee, 2014a ) along with examining student emotions towards online assessment ( Kim et al., 2014 ).

Cognitive. Researchers have also examined cognitive aspects of learner characteristics including meta-cognitive skills, cognitive variables, higher-order thinking, cognitive density, and critical thinking ( Chen & Wu, 2012 ; Lee, 2014b ). Lee (2014b) examined the relationship between cognitive presence density and higher-order thinking skills. Chen and Wu (2012) examined the relationship between cognitive and motivational variables in an online system for secondary physical education.

Demographic. Researchers have examined various demographic factors in online learning. Several researchers have examined gender differences in online learning ( Bayeck et al., 2018 ; Lowes et al., 2016 ; Yukselturk & Bulut, 2009 ), ethnicity, age ( Ke & Kwak, 2013 ), and minority status ( Yeboah & Smith, 2016 ) of online learners.

4.2.3. Less frequently studied research themes

While engagement and learner characteristics were studied the most, other themes were less often studied in the literature and are presented here, according to size, with general descriptions of the types of research examined for each.

Evaluation and Quality Assurance. There were 38 studies (6.14%) published in the theme of evaluation and quality assurance. Some of the studies in this theme focused on course quality standards, using quality matters to evaluate quality, using the CIPP model for evaluation, online learning system evaluation, and course and program evaluations.

Course Technologies. There were 35 studies (5.65%) published in the course technologies theme. Some of the studies examined specific technologies such as Edmodo, YouTube, Web 2.0 tools, wikis, Twitter, WebCT, Screencasts, and Web conferencing systems in the online learning context.

Course Facilitation. There were 34 studies (5.49%) published in the course facilitation theme. Some of the studies in this theme examined facilitation strategies and methods, experiences of online facilitators, and online teaching methods.

Institutional Support. There were 33 studies (5.33%) published in the institutional support theme which included support for both the instructor and learner. Some of the studies on instructor support focused on training new online instructors, mentoring programs for faculty, professional development resources for faculty, online adjunct faculty training, and institutional support for online instructors. Studies on learner support focused on learning resources for online students, cognitive and social support for online learners, and help systems for online learner support.

Learner Outcome. There were 32 studies (5.17%) published in the learner outcome theme. Some of the studies that were examined in this theme focused on online learner enrollment, completion, learner dropout, retention, and learner success.

Course Assessment. There were 30 studies (4.85%) published in the course assessment theme. Some of the studies in the course assessment theme examined online exams, peer assessment and peer feedback, proctoring in online exams, and alternative assessments such as eportfolio.

Access, Culture, Equity, Inclusion, and Ethics. There were 29 studies (4.68%) published in the access, culture, equity, inclusion, and ethics theme. Some of the studies in this theme examined online learning across cultures, multi-cultural effectiveness, multi-access, and cultural diversity in online learning.

Leadership, Policy, and Management. There were 27 studies (4.36%) published in the leadership, policy, and management theme. Some of the studies on leadership, policy, and management focused on online learning leaders, stakeholders, strategies for online learning leadership, resource requirements, university policies for online course policies, governance, course ownership, and faculty incentives for online teaching.

Course Design and Development. There were 27 studies (4.36%) published in the course design and development theme. Some of the studies examined in this theme focused on design elements, design issues, design process, design competencies, design considerations, and instructional design in online courses.

Instructor Characteristics. There were 21 studies (3.39%) published in the instructor characteristics theme. Some of the studies in this theme were on motivation and experiences of online instructors, ability to perform online teaching duties, roles of online instructors, and adjunct versus full-time online instructors.

4.3. Research settings and methodology used in the studies

The research methods used in the studies were classified into quantitative, qualitative, and mixed methods ( Harwell, 2012 , pp. 147–163). The research setting was categorized into higher education, continuing education, K-12, and corporate/military. As shown in Table A in the appendix, the vast majority of the publications used higher education as the research setting ( n  = 509, 67.6%). Table B in the appendix shows that approximately half of the studies adopted the quantitative method ( n  = 324, 43.03%), followed by the qualitative method ( n  = 200, 26.56%). Mixed methods account for the smallest portion ( n  = 95, 12.62%).

Table A shows that the patterns of the four research settings were approximately consistent across the 12 themes except for the theme of Leaner Outcome and Institutional Support. Continuing education had a higher relative frequency in Learner Outcome (0.28) and K-12 had a higher relative frequency in Institutional Support (0.33) compared to the frequencies they had in the total themes (0.09 and 0.08 respectively). Table B in the appendix shows that the distribution of the three methods were not consistent across the 12 themes. While quantitative studies and qualitative studies were roughly evenly distributed in Engagement, they had a large discrepancy in Learner Characteristics. There were 100 quantitative studies; however, only 18 qualitative studies published in the theme of Learner Characteristics.

In summary, around 8% of the articles published in the 12 journals focus on online learning. Online learning publications showed a tendency of increase on the whole in the past decade, albeit fluctuated, with the greatest number occurring in 2018. Among the 12 research themes related to online learning, the themes of Engagement and Learner Characteristics were studied the most and the theme of Instructor Characteristics was studied the least. Most studies were conducted in the higher education setting and approximately half of the studies used the quantitative method. Looking at the 12 themes by setting and method, we found that the patterns of the themes by setting or by method were not consistent across the 12 themes.

The quality of our findings was ensured by scientific and thorough searches and coding consistency. The selection of the 12 journals provides evidence of the representativeness and quality of primary studies. In the coding process, any difficulties and questions were resolved by consultations with the research team at bi-weekly meetings, which ensures the intra-rater and interrater reliability of coding. All these approaches guarantee the transparency and replicability of the process and the quality of our results.

5. Discussion

This review enabled us to identify the online learning research themes examined from 2009 to 2018. In the section below, we review the most studied research themes, engagement and learner characteristics along with implications, limitations, and directions for future research.

5.1. Most studied research themes

Three out of the four systematic reviews informing the design of the present study found that online learner characteristics and online engagement were examined in a high number of studies. In this review, about half of the studies reviewed (50.57%) focused on online learner characteristics or online engagement. This shows the continued importance of these two themes. In the Tallent-Runnels et al.’s (2006) study, the learner characteristics theme was identified as least studied for which they state that researchers are beginning to investigate learner characteristics in the early days of online learning.

One of the differences found in this review is that course design and development was examined in the least number of studies in this review compared to two prior systematic reviews ( Berge & Mrozowski, 2001 ; Zawacki-Richter et al., 2009 ). Zawacki-Richter et al. did not use a keyword search but reviewed all the articles in five different distance education journals. Berge and Mrozowski (2001) included a research theme called design issues to include all aspects of instructional systems design in distance education journals. In our study, in addition to course design and development, we also had focused themes on learner outcomes, course facilitation, course assessment and course evaluation. These are all instructional design focused topics and since we had multiple themes focusing on instructional design topics, the course design and development category might have resulted in fewer studies. There is still a need for more studies to focus on online course design and development.

5.2. Least frequently studied research themes

Three out of the four systematic reviews discussed in the opening of this study found management and organization factors to be least studied. In this review, Leadership, Policy, and Management was studied among 4.36% of the studies and Access, Culture, Equity, Inclusion, and Ethics was studied among 4.68% of the studies in the organizational level. The theme on Equity and accessibility was also found to be the least studied theme in the Berge and Mrozowski (2001) study. In addition, instructor characteristics was the least examined research theme among the twelve themes studied in this review. Only 3.39% of the studies were on instructor characteristics. While there were some studies examining instructor motivation and experiences, instructor ability to teach online, online instructor roles, and adjunct versus full-time online instructors, there is still a need to examine topics focused on instructors and online teaching. This theme was not included in the prior reviews as the focus was more on the learner and the course but not on the instructor. While it is helpful to see research evolving on instructor focused topics, there is still a need for more research on the online instructor.

5.3. Comparing research themes from current study to previous studies

The research themes from this review were compared with research themes from previous systematic reviews, which targeted prior decades. Table 8 shows the comparison.

Comparison of most and least studied online learning research themes from current to previous reviews.

L = Learner, C=Course O=Organization.

5.4. Need for more studies on organizational level themes of online learning

In this review there is a greater concentration of studies focused on Learner domain topics, and reduced attention to broader more encompassing research themes that fall into the Course and Organization domains. There is a need for organizational level topics such as Access, Culture, Equity, Inclusion and Ethics, and Leadership, Policy and Management to be researched on within the context of online learning. Examination of access, culture, equity, inclusion and ethics is very important to support diverse online learners, particularly with the rapid expansion of online learning across all educational levels. This was also least studied based on Berge and Mrozowski (2001) systematic review.

The topics on leadership, policy and management were least studied both in this review and also in the Tallent-Runnels et al. (2006) and Zawacki-Richter et al. (2009) study. Tallent-Runnels categorized institutional and administrative aspects into institutional policies, institutional support, and enrollment effects. While we included support as a separate category, in this study leadership, policy and management were combined. There is still a need for research on leadership of those who manage online learning, policies for online education, and managing online programs. In the Zawacki-Richter et al. (2009) study, only a few studies examined management and organization focused topics. They also found management and organization to be strongly correlated with costs and benefits. In our study, costs and benefits were collectively included as an aspect of management and organization and not as a theme by itself. These studies will provide research-based evidence for online education administrators.

6. Limitations

As with any systematic review, there are limitations to the scope of the review. The search is limited to twelve journals in the field that typically include research on online learning. These manuscripts were identified by searching the Education Research Complete database which focuses on education students, professionals, and policymakers. Other discipline-specific journals as well as dissertations and proceedings were not included due to the volume of articles. Also, the search was performed using five search terms “online learning" OR "online teaching" OR "online program" OR "online course" OR “online education” in title and keyword. If authors did not include these terms, their respective work may have been excluded from this review even if it focused on online learning. While these terms are commonly used in North America, it may not be commonly used in other parts of the world. Additional studies may exist outside this scope.

The search strategy also affected how we presented results and introduced limitations regarding generalization. We identified that only 8% of the articles published in these journals were related to online learning; however, given the use of search terms to identify articles within select journals it was not feasible to identify the total number of research-based articles in the population. Furthermore, our review focused on the topics and general methods of research and did not systematically consider the quality of the published research. Lastly, some journals may have preferences for publishing studies on a particular topic or that use a particular method (e.g., quantitative methods), which introduces possible selection and publication biases which may skew the interpretation of results due to over/under representation. Future studies are recommended to include more journals to minimize the selection bias and obtain a more representative sample.

Certain limitations can be attributed to the coding process. Overall, the coding process for this review worked well for most articles, as each tended to have an individual or dominant focus as described in the abstracts, though several did mention other categories which likely were simultaneously considered to a lesser degree. However, in some cases, a dominant theme was not as apparent and an effort to create mutually exclusive groups for clearer interpretation the coders were occasionally forced to choose between two categories. To facilitate this coding, the full-texts were used to identify a study focus through a consensus seeking discussion among all authors. Likewise, some studies focused on topics that we have associated with a particular domain, but the design of the study may have promoted an aggregated examination or integrated factors from multiple domains (e.g., engagement). Due to our reliance on author descriptions, the impact of construct validity is likely a concern that requires additional exploration. Our final grouping of codes may not have aligned with the original author's description in the abstract. Additionally, coding of broader constructs which disproportionately occur in the Learner domain, such as learner outcomes, learner characteristics, and engagement, likely introduced bias towards these codes when considering studies that involved multiple domains. Additional refinement to explore the intersection of domains within studies is needed.

7. Implications and future research

One of the strengths of this review is the research categories we have identified. We hope these categories will support future researchers and identify areas and levels of need for future research. Overall, there is some agreement on research themes on online learning research among previous reviews and this one, at the same time there are some contradicting findings. We hope the most-researched themes and least-researched themes provide authors a direction on the importance of research and areas of need to focus on.

The leading themes found in this review is online engagement research. However, presentation of this research was inconsistent, and often lacked specificity. This is not unique to online environments, but the nuances of defining engagement in an online environment are unique and therefore need further investigation and clarification. This review points to seven distinct classifications of online engagement. Further research on engagement should indicate which type of engagement is sought. This level of specificity is necessary to establish instruments for measuring engagement and ultimately testing frameworks for classifying engagement and promoting it in online environments. Also, it might be of importance to examine the relationship between these seven sub-themes of engagement.

Additionally, this review highlights growing attention to learner characteristics, which constitutes a shift in focus away from instructional characteristics and course design. Although this is consistent with the focus on engagement, the role of the instructor, and course design with respect to these outcomes remains important. Results of the learner characteristics and engagement research paired with course design will have important ramifications for the use of teaching and learning professionals who support instruction. Additionally, the review also points to a concentration of research in the area of higher education. With an immediate and growing emphasis on online learning in K-12 and corporate settings, there is a critical need for further investigation in these settings.

Lastly, because the present review did not focus on the overall effect of interventions, opportunities exist for dedicated meta-analyses. Particular attention to research on engagement and learner characteristics as well as how these vary by study design and outcomes would be logical additions to the research literature.

8. Conclusion

This systematic review builds upon three previous reviews which tackled the topic of online learning between 1990 and 2010 by extending the timeframe to consider the most recent set of published research. Covering the most recent decade, our review of 619 articles from 12 leading online learning journal points to a more concentrated focus on the learner domain including engagement and learner characteristics, with more limited attention to topics pertaining to the classroom or organizational level. The review highlights an opportunity for the field to clarify terminology concerning online learning research, particularly in the areas of learner outcomes where there is a tendency to classify research more generally (e.g., engagement). Using this sample of published literature, we provide a possible taxonomy for categorizing this research using subcategories. The field could benefit from a broader conversation about how these categories can shape a comprehensive framework for online learning research. Such efforts will enable the field to effectively prioritize research aims over time and synthesize effects.

Credit author statement

Florence Martin: Conceptualization; Writing - original draft, Writing - review & editing Preparation, Supervision, Project administration. Ting Sun: Methodology, Formal analysis, Writing - original draft, Writing - review & editing. Carl Westine: Methodology, Formal analysis, Writing - original draft, Writing - review & editing, Supervision

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

1 Includes articles that are cited in this manuscript and also included in the systematic review. The entire list of 619 articles used in the systematic review can be obtained by emailing the authors.*

Appendix B Supplementary data to this article can be found online at https://doi.org/10.1016/j.compedu.2020.104009 .

Appendix A. 

Research Themes by the Settings in the Online Learning Publications

Research Themes by the Methodology in the Online Learning Publications

Appendix B. Supplementary data

The following are the Supplementary data to this article:

References 1

• Ahn J., Butler B.S., Alam A., Webster S.A. Learner participation and engagement in open online courses: Insights from the Peer 2 Peer University. MERLOT Journal of Online Learning and Teaching. 2013; 9 (2):160–171. * [ Google Scholar ]
• Akcaoglu M., Lee E. Increasing social presence in online learning through small group discussions. International Review of Research in Open and Distance Learning. 2016; 17 (3) * [ Google Scholar ]
• Allen I.E., Seaman J. Babson survey research group; 2017. Digital compass learning: Distance education enrollment Report 2017. [ Google Scholar ]
• Amador J.A., Mederer H. Migrating successful student engagement strategies online: Opportunities and challenges using jigsaw groups and problem-based learning. Journal of Online Learning and Teaching. 2013; 9 (1):89. * [ Google Scholar ]
• Anderson L.W., Bourke S.F. Routledge; 2013. Assessing affective characteristics in the schools. [ Google Scholar ]
• Archibald D. Fostering the development of cognitive presence: Initial findings using the community of inquiry survey instrument. The Internet and Higher Education. 2010; 13 (1–2):73–74. * [ Google Scholar ]
• Artino A.R., Jr., Stephens J.M. Academic motivation and self-regulation: A comparative analysis of undergraduate and graduate students learning online. The Internet and Higher Education. 2009; 12 (3–4):146–151. [ Google Scholar ]
• Barnard L., Lan W.Y., To Y.M., Paton V.O., Lai S.L. Measuring self-regulation in online and blended learning environments. Internet and Higher Education. 2009; 12 (1):1–6. * [ Google Scholar ]
• Bayeck R.Y., Hristova A., Jablokow K.W., Bonafini F. Exploring the relevance of single‐gender group formation: What we learn from a massive open online course (MOOC) British Journal of Educational Technology. 2018; 49 (1):88–100. * [ Google Scholar ]
• Berge Z., Mrozowski S. Review of research in distance education, 1990 to 1999. American Journal of Distance Education. 2001; 15 (3):5–19. doi: 10.1080/08923640109527090. [ CrossRef ] [ Google Scholar ]
• Berry S. Building community in online doctoral classrooms: Instructor practices that support community. Online Learning. 2017; 21 (2):n2. * [ Google Scholar ]
• Boling E.C., Holan E., Horbatt B., Hough M., Jean-Louis J., Khurana C., Spiezio C. Using online tools for communication and collaboration: Understanding educators' experiences in an online course. The Internet and Higher Education. 2014; 23 :48–55. * [ Google Scholar ]
• Bolliger D.U., Inan F.A. Development and validation of the online student connectedness survey (OSCS) International Review of Research in Open and Distance Learning. 2012; 13 (3):41–65. * [ Google Scholar ]
• Bradford G., Wyatt S. Online learning and student satisfaction: Academic standing, ethnicity and their influence on facilitated learning, engagement, and information fluency. The Internet and Higher Education. 2010; 13 (3):108–114. * [ Google Scholar ]
• Broadbent J. Comparing online and blended learner's self-regulated learning strategies and academic performance. The Internet and Higher Education. 2017; 33 :24–32. [ Google Scholar ]
• Buzdar M., Ali A., Tariq R. Emotional intelligence as a determinant of readiness for online learning. International Review of Research in Open and Distance Learning. 2016; 17 (1) * [ Google Scholar ]
• Capdeferro N., Romero M., Barberà E. Polychronicity: Review of the literature and a new configuration for the study of this hidden dimension of online learning. Distance Education. 2014; 35 (3):294–310. [ Google Scholar ]
• Chaiprasurt C., Esichaikul V. Enhancing motivation in online courses with mobile communication tool support: A comparative study. International Review of Research in Open and Distance Learning. 2013; 14 (3):377–401. [ Google Scholar ]
• Chen C.H., Wu I.C. The interplay between cognitive and motivational variables in a supportive online learning system for secondary physical education. Computers & Education. 2012; 58 (1):542–550. * [ Google Scholar ]
• Cho H. Under co-construction: An online community of practice for bilingual pre-service teachers. Computers & Education. 2016; 92 :76–89. * [ Google Scholar ]
• Cho M.H., Shen D. Self-regulation in online learning. Distance Education. 2013; 34 (3):290–301. [ Google Scholar ]
• Cole M.T., Shelley D.J., Swartz L.B. Online instruction, e-learning, and student satisfaction: A three-year study. International Review of Research in Open and Distance Learning. 2014; 15 (6) * [ Google Scholar ]
• Comer D.K., Clark C.R., Canelas D.A. Writing to learn and learning to write across the disciplines: Peer-to-peer writing in introductory-level MOOCs. International Review of Research in Open and Distance Learning. 2014; 15 (5):26–82. * [ Google Scholar ]
• Cundell A., Sheepy E. Student perceptions of the most effective and engaging online learning activities in a blended graduate seminar. Online Learning. 2018; 22 (3):87–102. * [ Google Scholar ]
• Cung B., Xu D., Eichhorn S. Increasing interpersonal interactions in an online course: Does increased instructor email activity and voluntary meeting time in a physical classroom facilitate student learning? Online Learning. 2018; 22 (3):193–215. [ Google Scholar ]
• Cunningham U.M., Fägersten K.B., Holmsten E. Can you hear me, Hanoi?" Compensatory mechanisms employed in synchronous net-based English language learning. International Review of Research in Open and Distance Learning. 2010; 11 (1):161–177. [ Google Scholar ]
• Davis D., Chen G., Hauff C., Houben G.J. Activating learning at scale: A review of innovations in online learning strategies. Computers & Education. 2018; 125 :327–344. [ Google Scholar ]
• Delen E., Liew J., Willson V. Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments. Computers & Education. 2014; 78 :312–320. [ Google Scholar ]
• Dixson M.D. Measuring student engagement in the online course: The Online Student Engagement scale (OSE) Online Learning. 2015; 19 (4):n4. * [ Google Scholar ]
• Dray B.J., Lowenthal P.R., Miszkiewicz M.J., Ruiz‐Primo M.A., Marczynski K. Developing an instrument to assess student readiness for online learning: A validation study. Distance Education. 2011; 32 (1):29–47. * [ Google Scholar ]
• Dziuban C., Moskal P., Thompson J., Kramer L., DeCantis G., Hermsdorfer A. Student satisfaction with online learning: Is it a psychological contract? Online Learning. 2015; 19 (2):n2. * [ Google Scholar ]
• Ergün E., Usluel Y.K. An analysis of density and degree-centrality according to the social networking structure formed in an online learning environment. Journal of Educational Technology & Society. 2016; 19 (4):34–46. * [ Google Scholar ]
• Esfijani A. Measuring quality in online education: A meta-synthesis. American Journal of Distance Education. 2018; 32 (1):57–73. [ Google Scholar ]
• Glazer H.R., Murphy J.A. Optimizing success: A model for persistence in online education. American Journal of Distance Education. 2015; 29 (2):135–144. [ Google Scholar ]
• Glazer H.R., Wanstreet C.E. Connection to the academic community: Perceptions of students in online education. Quarterly Review of Distance Education. 2011; 12 (1):55. * [ Google Scholar ]
• Hartnett M., George A.S., Dron J. Examining motivation in online distance learning environments: Complex, multifaceted and situation-dependent. International Review of Research in Open and Distance Learning. 2011; 12 (6):20–38. [ Google Scholar ]
• Harwell M.R. 2012. Research design in qualitative/quantitative/mixed methods. Section III. Opportunities and challenges in designing and conducting inquiry. [ Google Scholar ]
• Hung J.L. Trends of e‐learning research from 2000 to 2008: Use of text mining and bibliometrics. British Journal of Educational Technology. 2012; 43 (1):5–16. [ Google Scholar ]
• Jiang W. Interdependence of roles, role rotation, and sense of community in an online course. Distance Education. 2017; 38 (1):84–105. [ Google Scholar ]
• Ke F., Kwak D. Online learning across ethnicity and age: A study on learning interaction participation, perception, and learning satisfaction. Computers & Education. 2013; 61 :43–51. [ Google Scholar ]
• Kent M. Changing the conversation: Facebook as a venue for online class discussion in higher education. MERLOT Journal of Online Learning and Teaching. 2013; 9 (4):546–565. * [ Google Scholar ]
• Kim C., Park S.W., Cozart J. Affective and motivational factors of learning in online mathematics courses. British Journal of Educational Technology. 2014; 45 (1):171–185. [ Google Scholar ]
• Kizilcec R.F., Pérez-Sanagustín M., Maldonado J.J. Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education. 2017; 104 :18–33. [ Google Scholar ]
• Kopp B., Matteucci M.C., Tomasetto C. E-tutorial support for collaborative online learning: An explorative study on experienced and inexperienced e-tutors. Computers & Education. 2012; 58 (1):12–20. [ Google Scholar ]
• Koseoglu S., Doering A. Understanding complex ecologies: An investigation of student experiences in adventure learning programs. Distance Education. 2011; 32 (3):339–355. * [ Google Scholar ]
• Kumi-Yeboah A. Designing a cross-cultural collaborative online learning framework for online instructors. Online Learning. 2018; 22 (4):181–201. * [ Google Scholar ]
• Kuo Y.C., Walker A.E., Belland B.R., Schroder K.E. A predictive study of student satisfaction in online education programs. International Review of Research in Open and Distance Learning. 2013; 14 (1):16–39. * [ Google Scholar ]
• Kuo Y.C., Walker A.E., Schroder K.E., Belland B.R. Interaction, Internet self-efficacy, and self-regulated learning as predictors of student satisfaction in online education courses. Internet and Higher Education. 2014; 20 :35–50. * [ Google Scholar ]
• Lee J. An exploratory study of effective online learning: Assessing satisfaction levels of graduate students of mathematics education associated with human and design factors of an online course. International Review of Research in Open and Distance Learning. 2014; 15 (1) [ Google Scholar ]
• Lee S.M. The relationships between higher order thinking skills, cognitive density, and social presence in online learning. The Internet and Higher Education. 2014; 21 :41–52. * [ Google Scholar ]
• Lee K. Rethinking the accessibility of online higher education: A historical review. The Internet and Higher Education. 2017; 33 :15–23. [ Google Scholar ]
• Lee Y., Choi J. A review of online course dropout research: Implications for practice and future research. Educational Technology Research & Development. 2011; 59 (5):593–618. [ Google Scholar ]
• Li L.Y., Tsai C.C. Accessing online learning material: Quantitative behavior patterns and their effects on motivation and learning performance. Computers & Education. 2017; 114 :286–297. [ Google Scholar ]
• Liyanagunawardena T., Adams A., Williams S. MOOCs: A systematic study of the published literature 2008-2012. International Review of Research in Open and Distance Learning. 2013; 14 (3):202–227. [ Google Scholar ]
• Lowes S., Lin P., Kinghorn B.R. Gender differences in online high school courses. Online Learning. 2016; 20 (4):100–117. [ Google Scholar ]
• Marbouti F., Wise A.F. Starburst: A new graphical interface to support purposeful attention to others' posts in online discussions. Educational Technology Research & Development. 2016; 64 (1):87–113. * [ Google Scholar ]
• Martin F., Ahlgrim-Delzell L., Budhrani K. Systematic review of two decades (1995 to 2014) of research on synchronous online learning. American Journal of Distance Education. 2017; 31 (1):3–19. [ Google Scholar ]
• Moore-Adams B.L., Jones W.M., Cohen J. Learning to teach online: A systematic review of the literature on K-12 teacher preparation for teaching online. Distance Education. 2016; 37 (3):333–348. [ Google Scholar ]
• Murphy E., Rodríguez-Manzanares M.A. Rapport in distance education. International Review of Research in Open and Distance Learning. 2012; 13 (1):167–190. * [ Google Scholar ]
• Nye A. Building an online academic learning community among undergraduate students. Distance Education. 2015; 36 (1):115–128. * [ Google Scholar ]
• Olesova L., Slavin M., Lim J. Exploring the effect of scripted roles on cognitive presence in asynchronous online discussions. Online Learning. 2016; 20 (4):34–53. * [ Google Scholar ]
• Orcutt J.M., Dringus L.P. Beyond being there: Practices that establish presence, engage students and influence intellectual curiosity in a structured online learning environment. Online Learning. 2017; 21 (3):15–35. * [ Google Scholar ]
• Overbaugh R.C., Nickel C.E. A comparison of student satisfaction and value of academic community between blended and online sections of a university-level educational foundations course. The Internet and Higher Education. 2011; 14 (3):164–174. * [ Google Scholar ]
• O'Shea S., Stone C., Delahunty J. “I ‘feel’like I am at university even though I am online.” Exploring how students narrate their engagement with higher education institutions in an online learning environment. Distance Education. 2015; 36 (1):41–58. * [ Google Scholar ]
• Paechter M., Maier B. Online or face-to-face? Students' experiences and preferences in e-learning. Internet and Higher Education. 2010; 13 (4):292–297. [ Google Scholar ]
• Phirangee K. Students' perceptions of learner-learner interactions that weaken a sense of community in an online learning environment. Online Learning. 2016; 20 (4):13–33. * [ Google Scholar ]
• Phirangee K., Malec A. Othering in online learning: An examination of social presence, identity, and sense of community. Distance Education. 2017; 38 (2):160–172. * [ Google Scholar ]
• Preisman K.A. Teaching presence in online education: From the instructor's point of view. Online Learning. 2014; 18 (3):n3. * [ Google Scholar ]
• Rowe M. Developing graduate attributes in an open online course. British Journal of Educational Technology. 2016; 47 (5):873–882. * [ Google Scholar ]
• Ruane R., Koku E.F. Social network analysis of undergraduate education student interaction in online peer mentoring settings. Journal of Online Learning and Teaching. 2014; 10 (4):577–589. * [ Google Scholar ]
• Ruane R., Lee V.J. Analysis of discussion board interaction in an online peer mentoring site. Online Learning. 2016; 20 (4):79–99. * [ Google Scholar ]
• Rye S.A., Støkken A.M. The implications of the local context in global virtual education. International Review of Research in Open and Distance Learning. 2012; 13 (1):191–206. * [ Google Scholar ]
• Saadatmand M., Kumpulainen K. Participants' perceptions of learning and networking in connectivist MOOCs. Journal of Online Learning and Teaching. 2014; 10 (1):16. * [ Google Scholar ]
• Shackelford J.L., Maxwell M. Sense of community in graduate online education: Contribution of learner to learner interaction. International Review of Research in Open and Distance Learning. 2012; 13 (4):228–249. * [ Google Scholar ]
• Shea P., Bidjerano T. Does online learning impede degree completion? A national study of community college students. Computers & Education. 2014; 75 :103–111. * [ Google Scholar ]
• Sherry L. Issues in distance learning. International Journal of Educational Telecommunications. 1996; 1 (4):337–365. [ Google Scholar ]
• Slagter van Tryon P.J., Bishop M.J. Evaluating social connectedness online: The design and development of the social perceptions in learning contexts instrument. Distance Education. 2012; 33 (3):347–364. * [ Google Scholar ]
• Swaggerty E.A., Broemmel A.D. Authenticity, relevance, and connectedness: Graduate students' learning preferences and experiences in an online reading education course. The Internet and Higher Education. 2017; 32 :80–86. * [ Google Scholar ]
• Tallent-Runnels M.K., Thomas J.A., Lan W.Y., Cooper S., Ahern T.C., Shaw S.M., Liu X. Teaching courses online: A review of the research. Review of Educational Research. 2006; 76 (1):93–135. doi: 10.3102/00346543076001093. [ CrossRef ] [ Google Scholar ]
• Tawfik A.A., Giabbanelli P.J., Hogan M., Msilu F., Gill A., York C.S. Effects of success v failure cases on learner-learner interaction. Computers & Education. 2018; 118 :120–132. [ Google Scholar ]
• Thomas J. Exploring the use of asynchronous online discussion in health care education: A literature review. Computers & Education. 2013; 69 :199–215. [ Google Scholar ]
• Thormann J., Fidalgo P. Guidelines for online course moderation and community building from a student's perspective. Journal of Online Learning and Teaching. 2014; 10 (3):374–388. * [ Google Scholar ]
• Tibi M.H. Computer science students' attitudes towards the use of structured and unstructured discussion forums in fully online courses. Online Learning. 2018; 22 (1):93–106. * [ Google Scholar ]
• Tsai C.W., Chiang Y.C. Research trends in problem‐based learning (pbl) research in e‐learning and online education environments: A review of publications in SSCI‐indexed journals from 2004 to 2012. British Journal of Educational Technology. 2013; 44 (6):E185–E190. [ Google Scholar ]
• Tsai C.W., Fan Y.T. Research trends in game‐based learning research in online learning environments: A review of studies published in SSCI‐indexed journals from 2003 to 2012. British Journal of Educational Technology. 2013; 44 (5):E115–E119. [ Google Scholar ]
• Tsai C.W., Shen P.D., Chiang Y.C. Research trends in meaningful learning research on e‐learning and online education environments: A review of studies published in SSCI‐indexed journals from 2003 to 2012. British Journal of Educational Technology. 2013; 44 (6):E179–E184. [ Google Scholar ]
• Tsai C.W., Shen P.D., Fan Y.T. Research trends in self‐regulated learning research in online learning environments: A review of studies published in selected journals from 2003 to 2012. British Journal of Educational Technology. 2013; 44 (5):E107–E110. [ Google Scholar ]
• U.S. Department of Education, Institute of Education Sciences . InstituteofEducationSciences; Washington,DC: 2017. What Works Clearinghouse procedures and standards handbook, version3.0. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_v3_0_standards_handbook.pdf Retrievedfrom. [ Google Scholar ]
• Veletsianos G., Shepherdson P. A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. International Review of Research in Open and Distance Learning. 2016; 17 (2) [ Google Scholar ]
• VERBI Software . 2019. MAXQDA 2020 online manual. Retrieved from maxqda. Com/help-max20/welcome [ Google Scholar ]
• Verstegen D., Dailey-Hebert A., Fonteijn H., Clarebout G., Spruijt A. How do virtual teams collaborate in online learning tasks in a MOOC? International Review of Research in Open and Distance Learning. 2018; 19 (4) * [ Google Scholar ]
• Wang Y., Baker R. Grit and intention: Why do learners complete MOOCs? International Review of Research in Open and Distance Learning. 2018; 19 (3) * [ Google Scholar ]
• Wei C.W., Chen N.S., Kinshuk A model for social presence in online classrooms. Educational Technology Research & Development. 2012; 60 (3):529–545. * [ Google Scholar ]
• Wicks D., Craft B.B., Lee D., Lumpe A., Henrikson R., Baliram N., Wicks K. An evaluation of low versus high collaboration in online learning. Online Learning. 2015; 19 (4):n4. * [ Google Scholar ]
• Wise A.F., Perera N., Hsiao Y.T., Speer J., Marbouti F. Microanalytic case studies of individual participation patterns in an asynchronous online discussion in an undergraduate blended course. The Internet and Higher Education. 2012; 15 (2):108–117. * [ Google Scholar ]
• Wisneski J.E., Ozogul G., Bichelmeyer B.A. Does teaching presence transfer between MBA teaching environments? A comparative investigation of instructional design practices associated with teaching presence. The Internet and Higher Education. 2015; 25 :18–27. * [ Google Scholar ]
• Wladis C., Hachey A.C., Conway K. An investigation of course-level factors as predictors of online STEM course outcomes. Computers & Education. 2014; 77 :145–150. * [ Google Scholar ]
• Wladis C., Samuels J. Do online readiness surveys do what they claim? Validity, reliability, and subsequent student enrollment decisions. Computers & Education. 2016; 98 :39–56. [ Google Scholar ]
• Yamagata-Lynch L.C. Blending online asynchronous and synchronous learning. International Review of Research in Open and Distance Learning. 2014; 15 (2) * [ Google Scholar ]
• Yang J., Kinshuk, Yu H., Chen S.J., Huang R. Strategies for smooth and effective cross-cultural online collaborative learning. Journal of Educational Technology & Society. 2014; 17 (3):208–221. * [ Google Scholar ]
• Yeboah A.K., Smith P. Relationships between minority students online learning experiences and academic performance. Online Learning. 2016; 20 (4):n4. * [ Google Scholar ]
• Yu T. Examining construct validity of the student online learning readiness (SOLR) instrument using confirmatory factor analysis. Online Learning. 2018; 22 (4):277–288. * [ Google Scholar ]
• Yukselturk E., Bulut S. Gender differences in self-regulated online learning environment. Educational Technology & Society. 2009; 12 (3):12–22. [ Google Scholar ]
• Yukselturk E., Top E. Exploring the link among entry characteristics, participation behaviors and course outcomes of online learners: An examination of learner profile using cluster analysis. British Journal of Educational Technology. 2013; 44 (5):716–728. [ Google Scholar ]
• Zawacki-Richter O., Backer E., Vogt S. Review of distance education research (2000 to 2008): Analysis of research areas, methods, and authorship patterns. International Review of Research in Open and Distance Learning. 2009; 10 (6):30. doi: 10.19173/irrodl.v10i6.741. [ CrossRef ] [ Google Scholar ]
• Zhu M., Sari A., Lee M.M. A systematic review of research methods and topics of the empirical MOOC literature (2014–2016) The Internet and Higher Education. 2018; 37 :31–39. [ Google Scholar ]
• Zimmerman T.D. Exploring learner to content interaction as a success factor in online courses. International Review of Research in Open and Distance Learning. 2012; 13 (4):152–165. [ Google Scholar ]

• Research article
• Open access
• Published: 26 February 2019

Promoting open educational resources-based blended learning

• Thanuja Chandani Sandanayake   ORCID: orcid.org/0000-0001-5430-6070 1  

International Journal of Educational Technology in Higher Education volume  16 , Article number:  3 ( 2019 ) Cite this article

26k Accesses

20 Citations

32 Altmetric

Metrics details

The OER movement has empowered researchers and educators to become more innovative in their teaching and learning, through the openness and flexibility. The use and adaptation of OER have been recommended as a very cost-effective investment in quality teaching-learning. In conventional teaching practices, teachers mostly spend time developing learning materials, reviewing lecture notes, anticipating questions and formulating answers, preparing for examinations. This method is no longer appropriate with the learner’s current association with the technology. This research aims on promoting OER-based blended learning for the undergraduate learners. Action research has been conducted in order to identify the learner adaptation to the new culture of OER-based blended learning. This research has evaluated the learner perceptions on OER-based blended learning. The learner performance records were also evaluated as a measure of quality of learning. The study has focused on how the OER materials to be incorporated in the online course development in undergraduate learning. At the same time, research provides feedback on the use of OER- based blended learning methods. The study further elaborates on effective assessment activities which need to be used in OER-based blended learning. Learners were quite positive on these effective assessment activities. Moreover, the study specifies the importance of incorporating OER in undergraduate online learning.

Introduction

Blended learning is one of the most accepted learning modes where the learners get the opportunity to learn using online digital media as well as the traditional classroom methods (Bonk & Graham, 2013 ). The use of online learning methods in blended learning helps the course designers in using learning materials as their preference. Open Educational Resources (OERs) are the types of educational materials that are used in the public domain or introduced with an open license (UNESCO, 2002 ). Open Educational Resources (OERs) are becoming popular among such online course designers since OER are the types of educational materials that are in the public domain or introduced with an open license (UNESCO, 2014 ). The Open Educational Resources (OER) movement has empowered educators to become more innovative in their pedagogical practices, through the openness and flexibility in educational resource use permitted by open licensing of materials (Karunanayake, Naidu, & Mohan, 2016 ). Similar research has been done by many other researches and some can be found in Dhanarajan & Porter, 2013 ; Jhangiani, Pitt, Hendricks, Key, & Lalonde, 2016 ; Glenda & Trotter, 2017 ; Clements & Pawlowski, 2012 and Pete, Mulder, & Neto, 2017 . The use and adaptation of OER has been recommended as a very cost-effective investment in curriculum development and quality teaching-learning material development (Dhanarajan & Porter, 2013 ). 5R Concept of OER (i.e: Retain, Reuse, Revise, Remix and Redistribute) gives the opportunity for the course designers to modify the course as per the course objectives and requirements (Wiley, 2014 ). The best explanation of 5Rs of openness is applicable in describing all possible ways of how OER being integrated; Retain, Reuse, Revise, Remix and Redistribute (Wiley, 2014 ).

Enabling user engagement in novel methods of using resources to move beyond more access to them shows the best practices of Open Educational Resources (Karunanayake et al., 2016 ). Wiley has described the following rights to access materials: retain (the right to make, to own and control copies of content), reuse (in many ways), revise (adaptation, making adjustments, modifications and change), remix (combinations with the original or revised content with other open content, thereby making something innovative such as a mash up), and finally, re-distribute (sharing the new content with others) (Wiley, 2014 ). In many local universities, the undergraduate degree programmes offer full-time face to face classroom basis learning. This learning environment is conventional since the students are familiar with the existing delivery mechanism (Mason, Shuman, & Cook, 2013 ). Apart from that, the academics or course developers are reluctant to use novel approaches in the course delivery since both learners and teachers do not like a drastic changes in teaching and learning (Tallvid, 2016 ).

This research has been carried out at the Faculty of Information Technology of University of Moratuwa, Sri Lanka. The University of Moratuwa offers most of the undergraduate courses face to face while some teachers offer the courses in blended mode with the help of Moodle Learning Management System (LMS) (Moodle reference - https://moodle.itfac.mrt.ac.lk/login/index.php ). These teachers or academics use the online platforms only to conduct and upload the assignments. In such courses teaching happens through off line mode. The Faculty of Information Technology uses face to face mode as the main delivery method. The Faculty of IT has been using Moodle LMS for the teaching and learning process. Most of the time, Moodle LMS is used to upload assignments, upload lecture or lesson learning materials which are used during lectures. The learners are given the required training on how to use the official LMS for academic purposes at the beginning of the first year.

The Faculty of Information Technology has been offering blended mode courses to the learners over 10 years. However, OER-based blended learning is new since it has never been used in previous courses or as a delivery mode. This course was the first OER-integrated online course offered by the Faculty of Information Technology. Since this is a new delivery experience, the research team has conducted the study through an Action research. The main aim of the research was to promote OER-based learning for the undergraduates. Therefore this research study was conducted based on the undergraduate learners to promote OER-based blended learning in conventional universities.

Aim and objectives

The aim of the research study is to conduct an action research on promoting OER-based learning in a blended learning approach. Following are the major objectives of the research study:

Design an intervention to promote OER-based learning

Conduct an intervention with the undergraduate learners in a blended learning model.

Evaluate the learner performance in OER-based course

Identify the learner observations on OER-based learning

Identify possible ways to promote OER-based learning for undergraduate courses.

Literature review

Conventional teaching and learning method is no longer appropriate for the learner’s current association with the technology. Therefore technology-enabled learning (TEL) plays a vital role in contemporary education structures. The literature further says that the basic and fundamental problem of the traditional teaching process is that the faculty members often equate their learning process to their students’ (Liyoshi & Vijay-Kumar, 2008 ). As mentioned in the introduction, OER is teaching and learning materials that are freely available online for everyone to use. Larsen and Vincent-Lancrin ( 2005 ) OER has further defined that, “The open sharing of one’s educational resources implies that knowledge is made freely available on non-commercial terms,”. At the same time, Hylen ( 2005 ) defines OER initiatives as “open courseware and content; open software tools (e.g. learning management systems); open material for e-learning capacity building of faculty staff; repositories of learning objects; and free educational courses.”

The OER-based online learning is the latest method of learning since the learners and the teachers get the freedom of using the copyright free materials for the academic work (Karunanayaka et al., 2016 ). OERs help enhance the teaching and learning across the globe immensely. Mostly, OER learning materials are available at “free and open” concept which provides a great advantage for developing countries where many learners may not be able to afford textbooks, where access to classrooms may be limited, and where teacher-training programs may be lacking (UNESCO, 2014 ).

They are also important in developed and industrialized countries since OER-integrated learning offers significant cost savings. In adult education contexts, most of OER materials are offered free to the learners where the learners get the benefit of accessing the world’s best courses and even degree programs. This is cost effective since learners do not have to spend a lot for textbooks and learning materials. Moreover, OER provides free and legal access to some of the world’s best courses for teachers which can lead to great innovations. For the students who have financial difficulties in buying textbooks, OER integrated learning is valuable. At the same time, learners are given freedom to learn anytime and anywhere they want. This introduction to OER integrated learning was thus conducted as Action Research study to deeply analyze the area. Open learning approach removes unnecessary barriers to learning especially for adult learning (Marina, 2011 ). At the same time, it aims to develop and make the learners engage in education and training opportunities which open up doors for different areas of learning. As explained in the book ‘A Basic Guide to OER’ (Butcher, 2015 ), OER-integrated learning incorporates and highlights key principles from which many stakeholders will be benefited such as;

Promotes lifelong learning opportunities and encompasses education and training

Encourage independent and critical thinking through learner-centered learning process.

Encourage flexible learning – allow learners to make their own decision on where, when, what and how they learn

Prior learning, prior experience, and demonstrated competencies

Learners should be able to gather knowledge from different learning contexts;

Providers should create the conditions for a fair chance of learner success.

OER-based learning makes the concept of resource-based learning of particular interest. A significant number of researchers have discussed the matter of the quality of OER as a learning resource (Butcher, 2015 ; Wiley, 2014 ). In open and distance learning concept, openness and resource-based learning are widely used. Resource-based learning creates a better platform to transform a culture of open learning and teaching across many educational systems to offer a better quality to the significant number of learners (Jarvis, 2012 ; Marina, 2011 ).

There are numerous types of resources available in OER offer for online learners (Karunanayake et al., 2016 ). Such materials are hosted as e-resources, blogs, materials in LMS, software tools, open courseware content, free educational courses, open materials for e-learning, wiki s, online learning repositories. The format of each category differs from each other and different facilities are also available in such resources. As described earlier, OER is valuable and this technological learning will focus on the open provision and use of course elements and learning materials or open content for only courses. At the same time, OER courses explore a very wide variety of projects where it leads to develop and provide complete learning programs, to institutions that publish the materials they use in their own teaching. Such programs publish the syllabi, lecture notes, reading lists, assessments, forums, projects facilitation and much other learning-related useful information (Bang, Dalsgaard, Kjær, & O’Donovan, 2016 ). OER includes the resources such as lecture notes, publications of staff, online courseware content, different educational programmes hosted internally and externally (OECD, 2014 ).

Research methodology

As mentioned in the introduction, the aim of the research is to promote OER-based learning among the undergraduate learners. Hence the study has been conducted as an action research study to achieve the main aim of the research. The action research was conducted according to the five-stage model of action research (Mills, 2011 ) which maps with the research objectives. The methodology adopted in this research study was questionnaire-based evaluation, course peer review and learner performance records in assessment activities. The learners’ insights towards OER-based learning were evaluated at the end of the course.

The selected course was offered for the students who follow B.Sc (Hons) in Information Technology and B.Sc (Hons) in Information Technology and Management. The course title is IS 4310 – Business Studies. There were 106 students registered for the course. This course is an elective course module which was offered in the level 4 of both degree programmes. The course was conducted within 14 calendar weeks. Blended teaching and learning method were used. The learner’s performance was evaluated through course assignments, quizzes, and discussion forums.

Research design in action research

Action research is a process of systematic inquiry that seeks to improve social issues affecting the daily/ everyday life (Nolen & Putten, 2007 ; Stringer, 2008 ). Many educational researchers of educational action research refer to a wide variety of evaluative, investigative, and analytical research methods designed to identify and solve issues of academics, or educational institutes help to develop practical solutions to solve them efficiently (Kemmis & McTaggart, 1988 ).

Educational action research is applied to educational programs or educational techniques that do not sometimes enounce or experience any problem or an issue, but education researchers simply want to learn new techniques, methods, and phenomena and improve (Ferrance, 2000 ). Today many educational programmers and educators are involved in education action research in order to build a better learning experience for potential learners and teachers. The action research study was carried out in five phases. Each phase has its unique features which need high attention in order to proceed further. Therefore the research design has aligned the study with the five phases of action research as stated in the below Table  1 . Adapting to new learning cultures is difficult for the adult learners (Ruey, 2010 ). The OER-based learning has made the adult learners’ life attuned into a new direction.

The learners may face problems due to the new change but the learners’ learning abilities and capabilities will be enhanced. The OER-based intervention course is planned according to the steps of the action research. Each and every step of the process reaches out to its aim and objectives by providing. The target groups of learners are not used to OER environment and the researcher analyzed their learning pattern before developing the intervention course.

The lesson plan should be outlined in a way it reflects the delivery of lesson content with a proper schedule. The time allotted for preparation, presentation and evaluation activities should be appropriate and adequate. The Business Studies course has also structured its lesson and sub-lessons according to the given semester plan. Each lesson consists of 2 h of teaching hours and 1 h of tutorial sessions which is being identified as direct contact hours of students. The estimated learning time of the course is as follows. Total hours allocated for the course to conduct face to face sessions are 42. This includes 2 h of lectures and 1 h of tutorial sessions run through the 12 weeks of a semester. Apart from this schedule, the research has developed some more learning activities, forums, assessments to be accessed via Moodle LMS. The course has been developed in a way described in Table  1 and the average time a student had to spend on the subject per week is equal to 6 h. This includes face to face session and online sessions. Further, the research has evaluated that the actual average time spent on the lesson is 6 h. The following Table  2 , shows the course structure according to time allocated for course activities. Figure  1 shows the screenshot of the Moodle interface.

Moodle interface of the OER-based online course

Questionnaire design of action research

The research methodology of this action research was adopted by considering how to gather information from multiple sources. The most important method was the questionnaire based evaluation. There were two types of questionnaires. One was to get the feedback from the student and the other was to get the feedback from peers. The learners’ feedback was collected at the end of the course module but the peer evaluation was conducted in the middle of the course by the course appointed examination moderator.

The student’s feedback questionnaire consisted of both structured and unstructured questions which were open-ended. The student’s feedback evaluation has used the 5 points Likert scale (Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, Strongly Disagree) structure in getting the feedback. The questions of the questionnaire will be discussed in the data analysis and the results section. The questionnaire was distributed among 106 respondents (students) and all of them responded to the survey. The peer evaluation was conducted by the moderator of the course appointed by the Senate of the university. The moderator has evaluated all the course materials, assignments, quizzes lesson outlines and course guided which was made available for the students. Apart from that, the moderator has observed one face to face session to evaluate the physical conduct of the lecture.

The questionnaire of the research was developed based on three research studies. The first study was conducted by Elango, Gudep, and Selvam ( 2008 ) Quality of e-Learning: An Analysis Based on e-Learners’ Perception of e-Learning. An attempt has been made to investigate the issues related to the quality dimensions of e-learning. This study aims at analyzing the perception of e-learners on various dimensions of quality such as Relevance of courses, Effectiveness of delivery mode, Course Compliance and Confidence.

The second study which was considered when developing the questionnaire is Students’ perceptions on incorporating e-learning into teaching and learning at the University of Ghana done by Michael Tagoe in 2012. This study, which was developed based on the Technology Acceptance Model (TAM), examines students’ perceptions on incorporating e-learning into teaching and learning (Tagoe, 2012 ). Tagoe’s study has mainly focused on the categories such as access to computers, prior experience, and perceived ease of use, perceive usefulness, attitude towards e-learning and behavioral intention to use e-learning.

Buzzetto-More has also conducted an analysis on student’s perception on Various E-Learning Components in 2008. The research describes the learning perceptions and preferences of students. Buzzetto-More argues that regardless of the delivery method, there are numerous tools and features at the disposal of students and instructors, and it is important for the e-learning community to examine both preferences and usage of these features (Buzzetto-More, 2008 ). Further the survey was designed to assess students’ technology access, skills, and usage; prior experiences with e-learning, course delivery preferences, perceived satisfaction with e-learning, and perceptions of, and preferences towards, various e-learning components (Buzzetto-More, 2008 ).

The survey was conducted during the course and the learner feedback was collected. Apart from that learners’ performance records were monitored too. Both qualitative and qualitative analyses have been conducted. Qualitative data has been reviewed using descriptive statistics with the help of SPSS (SPSS Inc. Released 2007. SPSS for Windows, Version 16.0. Chicago, SPSS Inc.) Statistical software and quantitative data has been revised using Nvivo (Nvivo 11 for Windows) software. Results of the action research study are explained in the results section.

Data analysis and results

The data collection was conducted at the end of the course. Questionnaire survey was conducted online for all the learners. Here the learner perceptions and learner performance records were evaluated to observe whether the research objectives were fulfilled.

Learner perception results

According to the methodology, the first evaluation was received by the learners and it was about their familiarity and usage of such courses. The question was “Have you experienced OER-based learning before?”. The analysis of the results was given in Fig.  2 .

Usage of OER integrated online courses

The Fig.  2 shows that nearly 30% of earners have attempted and accessed only OER integrated online course before. A majority of learners, which is 70% of them, have not accessed the OER integrated online course before. This interprets that courses of this nature are new to the majority of the learners. According to JISC study 2012 , there are numerous advantages for new users such as: enhanced quality and flexibility of resources, freedom of access (and enhanced opportunities for learning and support for learner-centered, self-directed, peer-to-peer and social/informal learning. Further, the learners are able to develop their skills and they have an opportunity to test course materials before getting enrolled and compare it with other courses. Anyhow this set of IT undergraduates also had new and fresh experience of learning through OER-based blended learning.

According to the results analysis, the next evaluation is about the time spent on learning activities per week. The course was designed in such a way that the learners are given numerous activities to complete within a specific time limit. These time limits vary with the amount of activities to be performed by the learners. According to Table  2 the average time to be spent on the lessons per week is 6 h. Figure  3 shows the students feedback on how they have spent their time on lesson activities per week. The majority of learners, about 41%, have spent 4–5 h per week. There are 34% of learners were spent 3–4 h per week. Therefore average learning time has been calculated as 3.5–4 h per week.

Time spent on lessons per week

The next analysis is based on the evaluation done on different categories of the course variables as mentioned above. The categories were Course Structure, Lesson format, Assessment Activities, User guide and Overall Performance. The students were evaluated through a questionnaire and each category has been evaluated with the use of two or more questions. Descriptive statistics were used to analyse the questions on the different elements of the course. The elements which were evaluated are course structure, lesson formats, assessment activities, user guide and overall satisfaction. Different questions were used to evaluate the elements and they were measured using five points Likert scale from Strongly Agree to Strongly Disagree format. The following Fig.  4 shows the analysis of average mean values of each category of the questionnaire.

Average mean values of the research feedback

As per the Fig.  4 the average mean values are more than 4 which show that the learners have positive attitude towards each and every activity of the course. Especially the learners have positive feelings and beliefs about the assessments which show 4.5 average mean value as the assessment scores. Overall satisfaction of the course equals to 4.3, which implies that the students are satisfied with the overall course elements.

When critically analyzing the mean values of the questionnaire, the lowest mean values were found for the questions “The course structure is clear and user-friendly” (X = 3.95082) and “I believe the lessons were properly explained and elaborated.” (X = 3.557377). This implies that learners have certain negative feelings on the structure and the lesson instructions.

The above results illustrate that the learners show a positive attitude towards incorporating OER elements into other courses of their degree programme. Especially the assessments of the courses can be well guided through clear instructions. The learners have mentioned their views as given below.

The next analysis is based on the leaner’s’ comments made for the question “What do you think about the structure of OER-based course, presentation of lesson OER materials and lesson formats?”. The comments were analyzed and categorized according to its positivity and negativity. The positive comments can be summarized as follows:

OER materials are clear and understandable

The course was very user-friendly

Many of practical examples were given

Well organized course format

Clear and readable lessons

The flow of the OER materials are good

Easy to understand

Helpful in developing knowledge

Enhance the ability of thinking

New things learned from OER

Learners can interact each other

Easy to access and well organized

The relationship among the lessons was good

Negative feedback was as follows:

Some materials were a bit lengthy, need to reduce the length

Please explain the technical terms more, we are not familiar with educational terms

Prefer the lesson structure, but the lessons should be more interactive.

Some lessons were difficult to understand, they are more philosophical. Please make them simple as possible

The majority of the students has provided positive feedback for this question. Students believe that OER integrated course was interesting and they had the chance to enhance their knowledge through that. The structure for the lessons was well formed and they have found it easy to access the system. Apart from that, the learners have commented that the lesson structure was helpful for them to understand the course activities in advance clearly.

The next question of the research is “Do you think OER-based online courses are good teaching and learning methods in general? Pleases state your views.” This is to review the ideas and views of the students about how important the OER-based online learning courses are. It is impressing to see both positive and negative perceptions towards addressing this question. The positive and negative feedback is listed as follows in Table  3 .

The evaluation of the question of “What do you think about the assessment activities used in the OER-based course?” is also interesting since the learners have shown their motivation to get the online assessments in this particular evaluation. According to the feedback received, it is identified that learners enjoy a lot of online assessments/work. The OER course comprises of both formative and summative assessments. The assessments started with individual and gradually moved into group activities. The learners were given proper help using OER. Students have commented that they need more feedback on the assessments offered online. Apart from that, the learners have commented that they need more time in completing the assessments due to their busy schedule. Results were given in Table  4 .

The next question that the research has reviewed is related to institutional capability and capacity to conduct OER-based learning. It is important to review the feedback of current learners on the future conducts of such courses in their degree programmes. There were good reviews and proposals made by the current learners listed in Table  5 below.

Further, this research looked at the learner advice and suggestions based on their own experience. It is quite obvious that every education researcher should critically evaluate the learner feedback on the activities which are offered or practiced.

The undergraduate learners, who have attempted and completed the course, have provided very effective ideas on the further development of OER-based blended learning in the technology-oriented faculty. The familiarity and the fluency in technology will be helpful for them to be in touch with these types of courses in their undergraduate level.

The solutions suggested by the students for the aforementioned question/s are quite interesting. The students have provided the following suggestions:

Need to add teleconferencing

Brainstorming sessions

Warm up sessions for learners

High interactivity with the learners and teachers

Cost-effective and sustainable models

LMS needs to be more interactive

More live discussion forums where learners can share thoughts

Provide quick feedback on the assessment activities of the course

Learner performance

This was the first time that the course module was offered to the learners as an OER-integrated online course. In the previous year the course was offered as a traditional face to face lecture mode without the use of OER. Therefore the assessment marks were evaluated in order to evaluate the learner performance. The learner performance records were evaluated based on the marks that the learners have received for the three graded assessments conducted in the course. Apart from that, the self-graded quizzes are also conducted in order to evaluate the learner performance. The learners were given three different formative assessment activities. The very first assessment was an individual activity and the second and third assessments were group activities based on the outcome of the first assignment. The researcher has done a comparison between the average assessment marks between the two courses conducted with and without the use of OER. The average assessment marks comparison between with and without use of OER is given in the Fig.  5 .

Assessment Marks of the students

As displayed in the Fig.  5 , marks of the three formative assessments are distributed in a skewed normal curve. The average mark of assignment 1 was 74, assignment 2 is 81 and assignment 3 is 80. The marks of the learners for the three assignments lie between 34 to 96. According to the assessment activities given, the only assessment which was not considered in grading was the discussion forum.

Due to the number of students who attempted the discussion forum, it was difficult to evaluate their level of engagement with the forum based on a specific scenario. Only the feedback was given for the discussion forums. This is one of the biggest problems found in online discussion forums because it is difficult to rationalize the evaluating criteria of the discussion. Therefore the learners were not given marks for the discussion forums of the course.

Discussion and conclusions

This research was conducted to promote OER-based learning among undergraduates. An action research methodology was adopted in this study to achieve the research objectives. The major aim was to introduce the OER integrated online learning to the undergraduates and observe differences of their results. The OER integration in the course was done using an intervention course. The intervention course consisted of OER learning materials covering the research objectives. As presented in the results, OER-based blended learning was quite new to the learners. These undergraduates faced challenges in finding relevant free and open materials. OER materials were helpful for them. In the blended learning environment, the course facilitators need to facilitate the learners using multiple learning methods in order to make the learning journey more effective and successful. This study has proven the importance of using novel methods in blended learning mode through an action research study. What is lacking in most of the online course are the social interactions between learners.

The designers of OER-based blended course should create the social interactions though innovative interactions such as peer facilitated discussion forums, video based learning materials to deliver course content, group based assessment activities which lead the learners to apply analytical skills, hands-on practical experience for the learners (specially for IT undergraduates), learning activities to improve the learner’s self learning abilities.

The OER-based online courses enhance the quality of learning experience by enabling flexibility of recourses while applying knowledge in a wider context. According to the learner perceptions, the OER-based blended learning concept is highly preferred by them. Moreover the learner-centered, self-directed, peer-to-peer and social/informal learning approaches are promoted in this experiment while the learners get better facilitation, enhance self-learning skills. Further, the researcher observed that learners were sharing their knowledge and skills. The researcher has identified the importance of flexible learning. Hence the 5R feature of OER allows the course designers to ‘Retain’, ‘Reuse’, ‘Revise’, ‘Remix’, and ‘Redistribute’ the learning content (Wiley, 2014 ). One of the main benefits of OER integration is that the materials can be revised and remixed; they can be customized to fit according to the learning requirements or the learning objectives of undergraduates.

The research further highlights the sustainability of OER-based online courses in undergraduate degrees. Hence the teachers are able to use the courses in multiple instances by improving the quality day by day. For example, video materials can accompany text and this is one way of presenting course content in multiple formats which helps the undergraduates to learn more easily. The use of OER materials instead of traditional textbooks can substantially reduce the cost of course materials for both teachers and undergraduate students. Therefore this is a sustainable model for the undergraduate learning and it can improve the quality of teaching and learning process. The next stage of this research study is to identify all the required instructional design features to be incorporated in the OER-integrated online learning and design an intervention to evaluate all the ID features. At the same time though this research study focused only on a limited group of student future the research will be focused on handling a larger set of learners.

Bang, J., Dalsgaard, C., Kjær, A., & O’Donovan, M. (2016). M.MOOCs in higher education - opportunities and threats or how small can a learning unit be in university degree programmes? In Proceedings of Enhancing European Higher Education “Opportunities and impact of new modes of teaching” , (pp. 543–553).

Google Scholar  

Bonk C.J., Graham C.R. (2013). The handbook of blended learning: Globle perspectives, Local Designs, https://books.google.lk/books?hl=en&lr=&id=2u2TxK06PwUC&oi=fnd&pg=PT14&dq=what+is+blended+learning&ots=a1BVA76Ecg&sig=bMkRdmroz21v9oaFnZHAMXNHPTM&redir_esc=y#v=onepage&q=what%20is%20blended%20learning&f=false

Butcher, N. (2015). A basic guide to open educational resources (OER). Commonwealth of learning & NESCO Retrieved from: http://unesdoc.unesco.org/images/0021/002158/215804e.pdf , CC BY SA.

Buzzetto-More, N. A. (2008). Student perceptions of various E-learning components. Interdisciplinary Journal of E-Learning and Learning Objects , 4 (1), 114-135

Clements, K., & Pawlowski, J. (2012). User-oriented quality for OER: Understanding teachers’ views on re-use, quality, and trust. Journal of Computer Assisted Learning , 28 (1), 4–14 https://doi.org/10.1111/j.1365-2729.2011.00450.x .

Article   Google Scholar  

Dhanarajan, G., & Porter, D. (2013). Open educational resources: An Asian perspective, (vol. 1). Commonwealth of Learning and OER Asia., Retrieved from: https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/pub_PS_OER_Asia_web.pdf .

Elango, R., Gudep, V. K., & Selvam, M. (2008). Quality of e-learning: An analysis based on e-learners’ perception of e- learning. The Electronic Journal of e-Learning , 6 (1), 31–44.

Ferrance, E., (2000). Action research, Northeast and Islands Regional Educational Laboratory At Brown University, Retrieved from: https://www.brown.edu/academics/education-alliance/sites/brown.edu.academics.education-alliance/files/publications/act_research.pdf

Glenda, J. C., & Trotter, H. (2017). An OER framework, heuristic and lens: Tools for understanding lecturers’ adoption of OER. Open Praxis Open Education Consortium Global Conference , 9 (2), 151–171.

Hylen, J. (2005). Open educational resources: Opportunities and challenges. Retrieved from: http://www.oecd.org/dataoecd/1/49/35733548.doc .

Jarvis, P. (2012). Adult learning in the social context . UK: Routledge.

Jhangiani, R., Pitt, R., Hendricks, C., Key, J., & Lalonde, C. (2016). Exploring Faculty Use of Open Educational Resources at British Columbia Post-Secondary Institutions. BCcampus Research Report . Victoria: BCcampus Retrieved from http://bccampus.ca/files/2016/01/BCFacultyUseOfOER_final.pdf .

JISC, (2012). Open Education Resources: Stakeholders and Benefits, Retrieved from: https://openeducationalresources.pbworks.com/w/page/24838012/Stakeholders%20and%20benefits

Karunanayaka, S.P., Naidu S, Kugamoorthy, T.D.T.L. Dhanapala A, Ariyaratne LR, Gonsalkorala M, Rajini, W.M.S, Weerakoon, W.M.S, Wanasinghe S, Karunanayake M.L, Sudarshana N.M.R.K Nawaratne, M.N.C. Fernando and K. Gnaneratnam, (2016). Towards open educational practices with Reflective Practice, Open University Research Sessions.

Karunanayake S.P., Naidu S., & Mohan M. (2016). Transformational Change at the Intersections of Technology, Education and Design at the Open University of Sri Lanka, pan-commonwealth forum 8 (PCF8)

Kemmis, S., & McTaggart, R. (1988). The action research planner . Australia: Deakin University Press.

Larsen, K. & Vincent-Lancrin, S. (2005). The impact of ICT on tertiary education: Advances and promises. http://advancingknowledge.com/drafts/Larsen-The%20impact%20of%20ICT%20on%20tertiary%20education%20-%20AKKE.doc

Liyoshi, T., & Vijay-Kumar, M. S. (2008). Opening Up Education-The Collective Advancement of Education through Open Technology, Open Content, and Open Knowledge. MIT press, London The Carnegie Foundation for the Advancement of Teaching. Retreived from: http://archive.carnegiefoundation.org/pdfs/elibrary/elibrary_pdf_788.pdf .

Marina, F. (2011). Barriers to adult learning: Bridging the gap. Australian Journal of Adult Learning , 5 (3), 583–590.

Mason, G. S., Shuman, T. R., & Cook, K. E. (2013). Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course. IEEE Transactions on Education , 56 (4), Retrived from: https://moodle.ruhr-uni-bochum.de/m/pluginfile.php/278896/mod_resource/content/2/Mason-Shuman_Cook_Comparing%20classical%20and%20Inverted%20Classroom.pdf

Mills, G. E. (2011). Action research: A guide for the teacher researcher , (4th ed., ). Boston: Pearson.

Nolen, A. L., & Putten, J. V. (2007). Action research in education: Addressing gaps in ethical principles and practices. Journal of Educational Researcher , 36 (7), 401–407.

Organisation for Economic Co-operation and Development (OECD), (2014). Giving knowledge for free, Retrieved from: http://www.oecd.org/education/ceri/38654317.pdf

Pete, J., Mulder, F., & Neto, J. D. O. (2017). Differentiation in access to, and the use and sharing of (open) educational resources among students and lecturers at Kenyan universities. Open Praxis Open Education Consortium Global Conference , 9 (2), 173–194.

Ruey S, (2010). A case study of constructivist instructional strategies for adult online learning, https://doi.org/10.1111/j.1467-8535.2009.00965.x

Book   Google Scholar  

Sagor, R. D. (2010). The action research guidebook: A four-stage process for educators and school teams . India: Sage Publications.

Stringer, E. T. (2008). Action research in education , (2nd ed., ). New Jersey: Pearson.

Tagoe, M. (2012). Students’ perceptions on incorporating e-learning into teaching and learning at the University of Ghana. International Journal of Education and Development using Information and Communication Technology (IJEDICT) , 8 (1), 91–103.

Tallvid, M. (2016). Understanding teachers’ reluctance to the pedagogical use of ICT in the 1: 1 classroom. Journal of Education and Information Technologies , 21 (3), 503-519

UNESCO. 2002. Forum on the impact of open courseware for higher education in developing countries: Final report Retrieved November 3, 2008, from www.wcet.info/resources/publications/unescofinalreport.pdf . [Google Scholar]

UNESCO, (2014). Communication and Information; http://www.unesco.org/new/en/communication-and-information/access-to-knowledge/open-educational-resources/why-should-i-care-about-oers/

Wiley, D., (2014). The access compromise and the 5th R. Weblog. Retrieved from http://opencontent.org/blog/archives/3

Download references

Acknowledgements

Not applicable.

Availability of data and materials

Raw data is available of necessary.

Author information

Authors and affiliations.

Faculty of Information Technology, University of Moratuwa, Moratuwa, Sri Lanka

Thanuja Chandani Sandanayake

You can also search for this author in PubMed   Google Scholar

Contributions

One hundred percent contribution by the corresponding author (TCS). The author read and approved the final manuscript.

Corresponding author

Correspondence to Thanuja Chandani Sandanayake .

Ethics declarations

Competing interests.

The author declares that she has no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Sandanayake, T.C. Promoting open educational resources-based blended learning. Int J Educ Technol High Educ 16 , 3 (2019). https://doi.org/10.1186/s41239-019-0133-6

Download citation

Received : 13 June 2018

Accepted : 29 January 2019

Published : 26 February 2019

DOI : https://doi.org/10.1186/s41239-019-0133-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Blended learning
• Action research
• Learning environment

The Role of Learning Resources, School Environment, and Climate in Transforming Schools from Buildings to Learning Communities

• Open Access
• First Online: 11 November 2021

Cite this chapter

You have full access to this open access chapter

• Ines Elezović 22 ,
• Beti Lameva 23 &
• Falk Brese 24  

Part of the book series: IEA Research for Education ((IEAR,volume 13))

4695 Accesses

1 Citations

International large-scale assessments can play a critical role in identifying factors that have an effect on student learning and achievement. IEA’s Trends in International Mathematics and Science Study (TIMSS), as the only international study of primary level mathematics and science education, is increasingly important in supporting continuous improvement in the quality of education and education systems. TIMSS also collects background information about the material and non-material factors that potentially affect teaching and learning processes, and the 2019 cycle of TIMSS provided a unique opportunity to analyze the role these factors play in education across the Dinaric region. Previous research has suggested that there are two especially important sets of socioeconomic background variables that need to be taken into consideration when analyzing possible factors related to student achievement and their attitudes toward teaching and learning at school. These are, firstly, personal student characteristics and their home resources and, secondly, school climate and material resources. Modeling of the TIMSS 2019 data for the Dinaric education systems indicated that material, environmental, and school climate factors were only weakly associated with student achievement across the region, explaining less than 12% of the variance in student achievement in science and less than 11% of the variance in mathematics achievement. These results indicate that education authorities in the region should not automatically assume that the material characteristics of the school environment, as well as elements of school climate, are the best or only areas for potential improvement. Access to home learning resources, parental support, and students’ and teachers’ attitudes toward learning and teaching seem to be more important factors in explaining differences in student achievement across the Dinaric region than previously perceived.

You have full access to this open access chapter,  Download chapter PDF

Similar content being viewed by others

Classroom management, school staff relations, school climate, and academic achievement: testing a model with urban high schools.

Eastern versus Western gaps in whole child education: how school climate works for academic achievement and well-being across China, Korea, Finland, and the United States

Trend analyses of TIMSS 2015 and 2019: school factors related to declining performance in mathematics

• School climate
• School environment
• School material resources
• Trends in International Mathematics and Science Study (TIMSS)

1 Introduction

Around the world, education authorities are interested in identifying factors that have an effect on their students’ achievement, instigating educational reforms that enhance positive elements of their systems and diminish any negative effects. International large-scale assessments (ILSAs) are viewed as increasingly important in supporting continuous improvement in the quality of education and education systems. Such worldwide assessments, like those conducted by the International Association for the Evaluation of Educational Achievement (IEA) and the Organisation for Economic Cooperation and Development (OECD), as well as others emerging from European Union (EU) initiatives, report the influence of material and non-material factors on teaching and learning processes. IEA’s Trends in International Mathematics and Science Study (TIMSS) is especially important for science, technology, engineering, and mathematics (STEM) because it is the only international study of those subjects at the primary school level.

In general, TIMSS has shown that student achievement around the globe has improved since the study began collecting data and measuring trends in 1995, with many national systems showing increasing achievement at both grades four and eight for the mathematics and science subjects. As Mullis said in 2016: “The positive trends indicate education is improving worldwide, and it’s not at the expense of equity between high and low achieving students” (TIMSS PIRLS International Study Center, 2016 ). With this in mind, we investigated school resources and characteristics of the school environment across the Dinaric region; our aim was to understand what underlying factors promote schools as good, successful, and open places for teaching and learning.

Seven participants from the Dinaric region took part in TIMSS 2019, namely Albania, Bosnia and Herzegovina, Croatia, Kosovo, Footnote 1 Montenegro, North Macedonia, and Serbia. Croatia and Serbia both also participated in TIMSS 2015 (where they tested grade four students). Footnote 2 Both achieved results above the international average in TIMSS 2015 for grade four science, with Serbia also scoring above the TIMSS international average in grade four mathematics (Croatian student achievement for grade four mathematics was around the TIMSS international average). Both education systems also reported an increase in student achievement in mathematics and science between TIMSS 2011 and TIMSS 2015, mirroring the global trend of improvement in student achievement in the subjects assessed by TIMSS. However, while this improvement continued for mathematics in the 2019 cycle of TIMSS in Croatia, a decline in science achievement was noted (despite still scoring above the TIMSS international average). Meanwhile, in Serbia, both assessment areas showed a decline in student achievement between TIMSS 2015 and TIMSS 2019, and the student mathematics achievement score declined below their TIMSS 2011 score (although still remaining above the TIMSS international average). All other Dinaric systems represented in this report recorded grade four student achievement scores below the TIMSS 2019 international average; among this group, Albania’s results were closest to the TIMSS 2019 international average and Kosovo’s results furthest from the TIMSS 2019 international average for both the mathematics and science assessment areas.

We were interested in whether available school resources, the school environment, and school climate could be linked to student achievement in the Dinaric region. Prior research (Kutsyuruba et al., 2015 ) has indicated that these factors may play an important part in developing successful schools and students, but, given that cultural factors may also be involved, the data collected by TIMSS 2019 provides the first opportunity to establish the interacting associations between these factors and student achievement across the Dinaric region.

For the Dinaric participants that were involved in TIMSS 2015 and earlier cycles, there has already been an initial exploration of these concepts and their potential effect. School principals reported that almost three-quarters of all students participating in TIMSS 2015 were “affected” or “affected a lot” by the shortage of resources for mathematics and science instruction. In the Dinaric region at that time, 18% of Croatian schools reported “more than 25% students coming from economically disadvantaged homes (and not more than 25% from economically affluent homes);” in Slovenia this figure was 23%, and it was 44% in Serbia. For all three participants, better achievement results were noted for students in schools where “more than 25% of the student body comes from economical affluent homes (and not more than 25% from economically disadvantaged homes)” than for students in schools in that fell into the other two groups, which contained proportionally more students from disadvantaged homes (Martin et al., 2016 ; Mullis et al., 2016a ). Almost a fifth of primary schools in Croatia, a quarter of primary schools in Slovenia, and half of the primary schools in Serbia contained students from homes with (relatively) harsh socioeconomic conditions, and it is perhaps not unexpected that this would have a negative effect on learning and teaching in these schools. Many ILSAs, such as OECD’s Programme for International Student Assessment PISA and IEA’s TIMSS and Progress in International Reading Literacy Study (PIRLS), have highlighted the importance of home environment in supporting student success (Martin et al., 2016 ; Mullis et al., 2016a , 2017 , 2020 ; OECD, 2019c ).

Similarly, teachers surveyed in TIMSS 2015 reported having “moderate to severe problems” with school conditions and resources for 17% of students in Slovenia, 23% of students in Croatia and 35% of students in Serbia. It is interesting to note that, in all three participants, students from schools that teachers had identified as strongly affected by such problems nevertheless tended to record higher average achievement in mathematics and science than less affected students. The TIMSS 2015 international results indicated that, generally, students with teachers who reported that their school had no problems with resources had the highest achievement, and students with teachers who reported that their school was “affected a lot” by problems with conditions and resources had the lowest average achievement among their peers (Martin et al., 2016 ; Mullis et al., 2016a ), which seems more in line with expectations. To explain the apparent deviation in the relationship between material resources at school and achievement in the Dinaric region, some research has suggested that, in conditions when material resources are lacking, teachers (and other staff) tend to give more attention to students’ learning and are more available and willing to help as a form of compensation (OECD ( 2019a ).

Another general conclusion from TIMSS 2015 was that according to parents, principals, and teachers, as well as students themselves, the majority of grade four students were attending good schools. On average, across all TIMSS 2015 participants, 58% of parents were reportedly very satisfied with students’ school performance, 52% of teachers were very satisfied with their jobs, more than half the teachers and principals reported that their school achieved a high level of academic success or that there was very strong emphasis on academic success in their school (>60%), and the majority of students (66%) reported a strong sense of school belonging. In the Dinaric region, the patterns found followed these general conclusions (Martin et al., 2016 ; Mullis et al., 2016a ).

In TIMSS 2015, school climate was represented by a composite TIMSS “Safe and Orderly School” scale (Martin et al., 2016 ; Mullis et al., 2016a ). In general, TIMSS 2015 found that the majority of grade four students were in safe school environments (56%, according to teachers) and, according to principals, 59% of schools had “hardly any discipline problems.” Conversely, 16% of all students reported that they were bullied about once a week in their schools, which perhaps challenges teachers’ and principals’ generally positive perceptions of school safety and school climate. The percentage of students that reported being bullied in TIMSS 2015 was close to the TIMSS international average in Slovenia (14%), but below the TIMSS international average in Croatia and Serbia (8%). In TIMSS 2015, 76% of students in Croatia attended schools where hardly any discipline problems were reported by their principals. Principals in Serbia and Slovenia were more critical than principals in Croatia about the state in their schools (they reported that while around 50% of students were in schools with “hardly any problems”, more than one third of them were in schools with “minor problems”). When teachers were asked to assess safety and order in their schools, they were more cautious than principals in their assessment, with around half reporting that students were in “very safe and orderly schools” in Croatia (48%) and Serbia (52%), while Slovenian teachers were more critical in their assessment (around 29%). In Serbia and Slovenia, students belonging to the schools that teachers reported as being very safe and orderly also tended to achieve the highest scores in mathematics in science. In Croatia, there was no significant difference in the achievement between the groups.

Almost half of the students in Serbia (49%), and more than a half of the students in Croatia (61%) and Slovenia (82%) had teachers reporting that teaching mathematics and science was somewhat or very limited by student needs (Martin et al., 2016 ; Mullis et al., 2016a ). In Croatia and Slovenia, students whose teachers reported that teaching was not at all limited achieved the best scores in mathematics. This was also true for the science achievement results for Slovenia, but it was interesting that students whose teachers reported that teaching was somewhat or very limited by students needs only scored a few points less on the TIMSS achievement scale. Serbia’s results were quite different, and students whose teachers reported that teaching was very limited by students needs tended to achieve the best scores in both mathematics and science. The TIMSS 2019 data showed similar patterns for Croatia and Serbia (Mullis et al., 2020 ).

While it is important to assess conditions in schools, as a source of material and environmental support to promote student learning, a student’s home resources for learning (both in terms of material assets and cultural capital) are well-proven indicators of student success in school (Matković et al., 2019 ; Meinck et al., 2018 ). In TIMSS 2015, students whose parents reported many home resources for learning had much higher achievement than students whose parents reported some or few resources. The difference in achievement between the students with many home resources (17–18%) and those with few resources (8–9%) was 142 points for mathematics and 141 for science. A similarly massive difference was reported by PIRLS 2016, and, in both TIMSS 2015 and PIRLS 2016, students whose parents reported often spending time with their children on early literacy and numeracy learning activities had a higher achievement than students whose parents did so only sometimes or almost never (Mullis et al., 2017 ).

The conceptual model of effective schools within the PIRLS and TIMSS studies was also put to test. An effective school was perceived as safe and orderly, had adequate facilities and equipment and well-resourced classrooms, was staffed with well-prepared teachers, it supported academic success, and provided effective instruction. Martin and Mullis ( 2013 , p. 8) concluded, “After controlling for home background, of the school environment variables, Schools Are Safe and Orderly was related to higher achievement in at least one subject in 15 countries, and Schools Support Academic Success in 10 countries. Students Engaged in Reading, Mathematics, and Science Lessons was the most powerful school instruction variable, related to higher achievement in at least one subject in 15 countries, again after controlling for home background. All in all, a school that was safe and orderly, promoted academic excellence, and provided engaging instruction, could be considered to have several important characteristics for effectiveness.”

Resources for education are generally focused on physical conditions for schooling, such as having enough space for classes, and ensuring basic utilities and perhaps specialized classrooms are available. More recent discussion on material resources in schools often refers exclusively to the availability of information and communication technologies (ICT) in schools, namely whether students have access to equipment such as laptops, tablets, broadband internet, interactive classrooms, and e-libraries. Both of these aspects are addressed in the TIMSS background questionnaires (TIMSS & PIRLS International Study Center 2018 ). Digital skills have been noted as being increasingly important in almost all aspects of teaching and learning, in acknowledgment of the need to prepare today’s students to function as tomorrow’s digital workers (Fraillon et al., 2020 ). Footnote 3 The integration of ICT is brings some new innovative forms of teaching in classrooms all over the world, having both advantages and disadvantages (Eickelmann, 2011 ).

OECD’s PISA also researches the relation between student achievement and material investments in education, and has repeatedly concluded that investing in the school system initially has positive effects on achievement, but a point is eventually reached when additional investments have a more modest effect on student results and other factors become more important. Essentially, when everything material has been resolved, less tangible elements of the quality of processes of teaching and learning will still need to be tackled to achieve more advanced results. Nevertheless, there are always exceptions, as OECD ( 2019a , p. 56) noted, “While an inadequately resourced education system cannot deliver good results, Estonia, with a level of expenditure on education that is about 30% lower than the OECD average, is nevertheless one of the top-performing OECD countries in reading, mathematics and science.”

When international large-scale assessments deliver their results, additional research on available data is performed in almost every country around the world. In Croatia, PISA 2006 data showed that home socioeconomic indicators, along with the region of residence, explained 24% of the variance in students’ science achievement and confirmed how important these factors are for student achievement (Gregurović & Kuti, 2010 ). As PISA only tests students aged 15, more information is needed at other school levels to make informed decisions about schooling. Reflecting on the results from international data prompts at least two questions about the relationship between material resources available to students and their success measured in terms of knowledge attainment in important learning areas. First, can provision of resources in school overcome the lack of resources at the individual (student, home) level? Secondly, can school characteristics, such as open school climate or a positive school culture oriented towards achievement and academic belonging, overcome a lack of material resources both on the individual and school level?

In general, previous studies have established more indicative connections between student achievement and school environments and school climate (Bear et al., 2014 ; OECD, 2019b ; Schulz et al., 2010 ), than between student achievement and school material resources. For instance, TIMSS 2015 results have shown that, for almost all grade four students, a positive sense of school belonging was related to higher average mathematics and science achievement (Martin et al., 2016 ; Mullis et al., 2016a ).

Having in mind that one of the most important goals of every teaching process is to help students become future prosperous adults by putting emphasis on both cognitive outcomes and affective dimensions (attitudes, values, and beliefs), educational systems that aim to be successful should go beyond procurement of material resources. Investing in the continuous professional development of teachers and principals is commonly recommended as a means of ensuring quality education, but other recommendations include investment in developing transversal (lifelong learning) skills or widening use of ICT in school (Drigas & Vasiliki, 2015 ; OECD, 2019d ; UNESCO [United Nations Educational, Scientific and Cultural Organization], 2014 ; Webb & Cox, 2004 ).

2 Methodology and Research Questions

We aimed to investigate whether a particular set of contextual factors was related to achievement, and if and to what extent these factors represented important elements of school life. Our research was designed to address the relative importance of two factors that previous research has suggested may be associated with student achievement. Firstly, how important were school material resources and the school physical environment (in terms of general wealth or plurality of school possessions, i.e., important school equipment and spaces or lack of thereof), school location, and principals’ perceptions of the affluence of the families from which enrolled students come from. Secondly, how important was the overall school climate? The elements of school climate here include the social determinants of everyday school life, such as student issues that affect teaching, safe and orderly school environments (as reported by teachers), and bullying among students (as reported by students).

From this we distil three critical research questions:

How well equipped with material resources for learning are schools across the Dinaric region?

What can TIMSS tell us about the learning environment in schools across the Dinaric region?

How comparable are important aspects of school climate across the Dinaric region?

We used data collected by TIMSS 2019 from seven educational systems across the Dinaric region in our analyses. These included students’ achievement results at grade four in mathematics and science, and contextual information derived from responses to the students’, teachers’ and principals’ questionnaires. For more information about samples, methods, procedures, and data that we used, see Sect. 5 and the TIMSS 2019 technical report (Martin et al., 2020 ).

2.1 Indicators and Variables Used

We identified several variables and scales in the TIMSS 2019 international reports as being of potential interest for our research (Table 1 ). We investigated one of the main aspects of schooling by creating two indexes to assess the availability of material resources in schools, one for mathematics and one for science. These indexes combined teachers’ and principals’ responses to questions about whether the school possessed a number of specific items (such as computers or a library) and the prevalence of different conditions posing obstacles for teaching into a simple summative “Index of School Material Resources” (see Table 1 and Tables S.8 and S.9 in the supplementary materials available for download at  www.iea.nl/publications/RfEVol13 ).

The Index of School Material Resources combines information collected by TIMSS 2019 on the availability of computers during mathematics/science lessons, existence and size of the school library, existence of classroom libraries, provision of digital learning resources, and instruction being affected by mathematics/science lessons resource shortages. The Index of School Material Resources for teaching science comprised one additional variable about the availability of a dedicated science laboratory in the school. For both mathematics and science, we split the derived index into three categories: (1) few resources available, (2) some resources available, and (3) many resources available in the school (see Table S.10 in the supplementary materials available for download at  www.iea.nl/publications/RfEVol13 ).

Among the contextual data TIMSS collects, there are several indicators regarding the school environment. In the school questionnaire, principals were asked whether the school is situated in an urban or rural settlement and about student composition in their school (if more students come from disadvantaged homes or more students come from affluent home backgrounds). We analyzed the relationship between student achievement and the factors creating the school environment (whether the school was located in an urban or rural environment and the school principal’s assessment of the school composition). These demographic determinants have been of interest to researchers for decades, in their attempts to define what conditions underlie student achievement; higher student achievement has been linked to urban and/or wealthier environments (see chapter “ Scaffolding the Learning in Rural and Urban Schools: Similarities and Differences ” for more information on this topic).

The third factor that we addressed was school climate, which we reduced to the aspect of perceptions of safety and order within school. Defining school climate is complex, despite often being cited as an important explanatory factor for many student outcomes (Brand et al., 2008 ; Cohen et al., 2009 ; Hoy et al., 1991 ). TIMSS reports have consistently shown a positive relationship between student achievement and teachers’ and principals’ reports that the school is safe and orderly (Martin et al., 2016 ; Mullis et al., 2016a , 2020 ). The TIMSS scale on student bullying in school, reported by students themselves, is also important element of assessing the overall safety and state of interrelations within the school and thus included into this analysis (Martin et al., 2020 ). In TIMSS frameworks bullying is defined as “repeated aggressive behavior that is intended to harm students who are physically or psychologically less strong, and takes a variety of forms ranging from name calling to inflicting mental and physical harm” (Mullis and Martin 2017 , p. 68). For some, this may be perceived as narrowed perspective of the concept of school climate, which is why we chose to analyze both the physical and social dimensions of school life in an attempt to provide a multidimensional approach. We thus undertook a comparative analysis of teachers’ perspectives on safety and order at school and students’ reports on bullying (aggregated at school level). As many national authorities around the world are aware, and the TIMSS 2019 international report reconfirmed (Mullis et al., 2020 ), the question of school safety (i.e., student bullying) remains an important problem in education. The teacher Safe and Orderly School scale encompasses of eight statements: one asking about conditions outside of the school (i.e., safety in the neighborhood), three about teachers’ subjective feeling of safety and order within the school, and another three about students’ adherence to school discipline (respecting the rules, teachers and property). We categorized students as being in “very safe and orderly schools” if, on average, their teachers agreed a lot with four of the eight statements and agreed a little with the other four statements.

Another indicator that we used to assess school climate was the TIMSS 2019 scale named “Classroom Teaching Limited by Students Not Ready for Instruction,” which is composed of eight variables collected by the TIMSS teacher questionnaire. These questions assess teachers’ perceptions of the severity of different limitations that negatively affect their classes. Teachers were asked whether their students lacked prerequisite knowledge or skills, suffered from lack of basic nutrition or not enough sleep, were absent from class, disruptive or disinterested, had to deal impairments (either mental, emotional or psychological), or did not understand the language of instruction.

We used these variables as predictors in regression analyses that investigated whether those elements of school life were related to student achievement.

3.1 Material Resources for Learning in Schools

3.1.1 index of school material resources.

As explained in Sect. 2.1 , we created two indexes to explore the effects of school material resources, one for mathematics and one for science; the science material resources index contained one additional variable (availability of a science laboratory in the school). Not having a science laboratory in school was related to lower achievement results in science only in one system (Albania). In Montenegro, there was no difference in science achievement among the students in schools with or without a science laboratory, and in Bosnia and Herzegovina, Croatia, and Kosovo, the difference was small and insignificant. In Montenegro and Serbia, students from schools without science laboratories achieved higher scores in science (more than 10 points higher on average) than those in schools with a science laboratory.

We further examined principals’ reports about conditions for teaching related to shortage of resources. Across the Dinaric region, relatively few students were affected either “somewhat” or “a lot” by shortages of resources for mathematics and science instruction, with the lowest percentages reported in Kosovo, Albania, and North Macedonia (<8%), and the highest percentage of affected students in Serbia (20%).

These results seem to differ from teachers’ reports; this may be because principals are either less aware of the resource problems reported by their teachers or less willing to admit classroom resource issues. The distribution of material resources for mathematics lessons varies significantly across the region (Fig.  1 ). Data from Bosnia and Herzegovina, Croatia, Montenegro, North Macedonia, and Serbia were quite consistent, with more than half to two-thirds of students belonging to the intermediate category that enjoys “some resources” (from 57% in North Macedonia to 76% in Croatia). In Albania and Kosovo, however, almost two-thirds of all students attended schools where principals indicated that their school was equipped with comparatively few resources. Only six percent of students in Albania were reported as having “many resources” and, in Kosovo, no students fell into this category. In interpreting these statistics, it is important to note that our school material resources scale and/or constructed index was comprised of physical objects and spaces, while, in the TIMSS 2019 schools questionnaire, principals responded to questions on shortages directly aimed at identifying specific issues, such as providing contents and tools that assisted teaching, along with questions about the availability of specialized staff (teachers); this last question was of particular interest to STEM education in the Dinaric systems.

Index of School Material Resources for Mathematics. Percentage of students in schools with different amounts of resources for mathematics lessons. Note In Kosovo and Serbia, the national defined population covers 90–95% of the national target population

The distribution of material resources for science lessons was very similar (Fig.  2 ). In Montenegro, North Macedonia, and Serbia, principals’ reports indicated that around half the students belonged to the intermediate category of “some resources”. Around two-thirds of students were in this category in Bosnia and Herzegovina (64%) and Croatia (72%), and around a third in Albania (31%) and Kosovo (36%). Croatia and Serbia had the smallest number of students in the category with “few resources” (5% and 8%, respectively), while Albania and Kosovo had the smallest number of students in the category of “many resources” (10% and 7%, respectively).

Index of School Material Resources for Science. Percentage of students at schools with different amounts of resources for science lessons. Note In Kosovo and Serbia, the national defined population covers 90–95% of the national target population

Using the Index of School Material Resources, we found that, in three of the Dinaric participants, differences in mathematics achievement among students at schools were related to the amount of resources. In Albania, on average, students at schools with some resources scored 40 points more than students at schools with only few resources, and students at schools with many resources scored, on average, 73 points more than their peers at schools with few resources. In Croatia, students at schools with few resources, on average, scored 25 points less on the mathematics scale than students at schools with some or many resources. In Serbia, there was a 35 point achievement gap between students at schools with low resources and those at schools with many resources. However, we found no significant similar achievement gaps in Bosnia and Herzegovina, Kosovo, Montenegro, and North Macedonia. Regarding science achievement, we found similarly that students at schools with more resources on average tended to score higher on the TIMSS assessment, except in Montenegro; however, the achievement gap was only significant in Albania (Fig.  3 ).

Difference in a mean mathematics achievement and b mean science achievement between TIMSS achievement scores for students at schools with many resources and students at schools with few resources. Notes * The difference is statistically significant ( p  < 0.05). In Kosovo and Serbia, the national defined population covers 90–95% of the national target population

3.1.2 Information and Communication Technology Resources

We found that, on average across the Dinaric region, most students were in schools that were equipped with computers for class use, with the highest percentages in Croatia (97% both for mathematics and science lessons) and the lowest in Kosovo (54% for mathematics lessons) and Macedonia (63% for science lessons) (Fig.  4 ). When teachers were asked whether each student had a computer to use in mathematics and/or science classes, the situation differed; the highest percentages were in Bosnia and Herzegovina (36% for mathematics and 30% for science), and lowest in Kosovo and Serbia (≤3%). The computer-student ratio ranged widely across the region, from 0.14 in Albania and Kosovo, 0.22 in Serbia, 0.24 in Croatia, 0.25 in Montenegro, and 0.41 in Bosnia and Herzegovina, to 0.77 in North Macedonia.

Student access to computers in school for mathematics and science lessons: a percentage of students in classes where each student has a computer; b percentage of students in classes that have computers for students to share; and c percentage of students in schools that have computers for class use. Note In Kosovo and Serbia the national defined population covers 90–95% of the national target population.

As well as providing hardware, there is a more sophisticated aspect to ICT in schools, reflected by the construction of online networks through interactive tools and the publication of online content for teaching and learning, such as providing digital learning resources. The progress toward full integration of ICT into teaching and learning has been largely gradual up until 2020, when the COVID-19 pandemic threw education systems around the world into “overnight” digitalization, whether they were prepared for it or not. Across the Dinaric region, TIMSS 2019 data indicated that the provision of “online learning management systems” differed substantially (Table 2 ). Principals reported that students’ access to digital learning resources was good (Table 2 ).

3.2 School Environments Across Dinaric Countries

In terms of school location, more than half of the students are located in urban areas in all seven participants, with the highest percentage in Montenegro (85%) and the lowest percentages in Croatia and Kosovo (57%). In general, more students attend urban schools, and more disadvantaged students tend to attend schools situated in rural areas (see chapter “ Scaffolding the Learning in Rural and Urban Schools: Similarities and Differences ” for a more detailed analysis of this topic).

According to their principals, the percentage of students at more disadvantaged schools ranged from 13% in Croatia to 42% in Albania. Principals in North Macedonia reported that 66% of students were in more affluent schools; this was the highest perceived percentage for that category in the Dinaric region.

Previous research (Mullis et al., 2016a ; OECD, 2019a ) has shown that student achievement in mathematics is related to student home socioeconomic status or school principals’ perceptions of family affluence. Our analysis of the TIMSS 2019 results confirmed these findings. The students from more affluent schools tended to achieve the best TIMSS mathematics scores in every system in the Dinaric region except Kosovo. In five participants, the mathematics achievement of students at more affluent schools was higher than that of students from more disadvantaged schools, with the biggest achievement gaps in North Macedonia (44 points) and Albania (39 points). In Bosnia and Herzegovina and Kosovo, there was no statistically significant difference between these groups.

As with mathematics, students from more affluent schools tended to achieve the best TIMSS science scores in every system in the Dinaric region except Kosovo. In six participants, the science achievement of students at more affluent schools was higher than that of students from more disadvantaged schools, with the biggest achievement gaps in North Macedonia (50 points) and Albania (42 points). In Kosovo, there was no statistically significant differences between these groups.

We also assessed results related to the TIMSS scale “Teaching Somewhat or Very Limited by Students not Ready for Instruction” (Mullis et al., 2020 , exhibits 10.10 and 10.11). Teachers generally reported that relatively few limitations were created by students who were not yet ready for instruction, at least in comparison with other TIMSS participants. In Albania, 71% of students attended schools that were affected “very little” by students not ready for instruction; in Kosovo 63% of students attended schools that were affected “very little” and, in North Macedonia, this was 60%. In the other participants, less than half of the students had teachers who reported facing few issues (49% in Croatia and Serbia, 46% in Montenegro, and 45% in Bosnia and Herzegovina). At least a third of students in the region had teachers that reported experiencing “some” or “a lot” of limitations due to students not ready for instruction.

3.3 School Climate: Safety and Order at Schools

When we assessed perceptions of safety and order in schools, we found that teachers’ perceptions of this dimension of school climate differed quite considerably across the region (Table 3 ).

Teachers of almost all students in Albania perceived their schools as very safe and orderly places, but only about half of the students in Croatia had teachers who thought their schools were very safe and orderly. In general, across the Dinaric region, only small percentages of students attended schools perceived by their teachers as “less then safe and orderly” (≤3%), and, in most participating systems, except Croatia, there were also fairly low percentages of students in schools that teachers perceived as “somewhat safe and orderly” (Table 3 ).

According to students, student bullying was present and relatively widespread in the Dinaric region. The percentages of students who reported frequent (monthly or weekly) bullying ranged from 15% of students in Albania to 32% of students in North Macedonia. Numerous national and international reports have reported findings on school violence in the Dinaric region. For example, when looking at adolescent experiences, the United Nations Children’s Fund ( 2017 ) reported that a quarter of students in Albania and North Macedonia experienced bullying in schools. Dinaric educational systems have strongly promoted zero violence policies in schools in response to this problem, and prevention programs have also been developed to tackle internet and cyber-bullying.

We analyzed the TIMSS 2019 data on bullying at school level in relation to school material resources, for both mathematics and science, and identified no significant differences between the schools belonging to the groups with few and many resources for learning (Figs. 5 and 6 ).

Percentages of students being bullied monthly or weekly in schools versus school resources for learning mathematics. Note In Kosovo and Serbia, the national defined population covers 90–95% of the national target population

Percentages of students being bullied monthly or weekly in schools versus school resources for learning science. Note In Kosovo and Serbia, the national defined population covers 90–95% of the national target population

In general, we note that the education systems that scored higher on the Indexes of School Material Resources were not experiencing lower levels of bullying in schools. A focus on developing more intangible elements, such as a supportive school climate, a culture of achievement, and trust in school as an institution, may result in better environments for teaching and learning within schools.

3.4 Impact of the Schools’ Material Resources, Environment, Composition and Climate on the Achievement Results (Regression Analysis)

Having investigated the effects of several school-related factors on achievement, we undertook multivariate regression modeling to obtain a more comprehensive picture how all these factors were interrelated with achievement. The regression analyses revealed that the importance and significance of the factors varied across the region. We found that the Index of School Material Resources, and the school environment and climate factors explained only two percent of variance in student achievement in mathematics in Bosnia and Herzegovina, Croatia, and Montenegro, six percent of variance in Serbia, seven percent in Kosovo, and up to 11% of variance in Albania and North Macedonia (Table 4 ). The regression models also only explained two percent of variance in student achievement in science in Bosnia and Herzegovina, Croatia, and Montenegro, seven percent of variance in Kosovo and Serbia, and up to 12% of variance in Albania and North Macedonia (Table 5 ). The low power of variables related to school resources, school environment, and school climate in explaining student achievement strongly suggests that factors related to students’ home resources and the personal characteristics of students (interests, motivation, beliefs), and teachers’ and teaching characteristics together play a much greater part in supporting student achievement, as other chapters in this book confirm.

4 Conclusions

Around the world, education authorities are interested in supporting better learning for all and international large-scale assessments play a critical role in identifying and supporting solutions that affect student achievements (Mihaljević Kosor et al. 2019 ). Although ILSA results sometimes lead researchers and policymakers to suggest that student achievement can be improved simply by something as obvious as investing in material resources, our research reveals that the answers are much more complex. Looking at the Dinaric region alone, factors related to material resources, school environment, and school climate did not show uniform or particularly strong effects on student achievement, although there were some interesting patterns that were aligned with the wider international results. In the TIMSS 2019 international results, higher average achievement in mathematics and science at grade four was associated with fewer school resource shortages and higher school emphasis on academic success (Mullis et al., 2020 ). Regarding some elements of school climate, higher average achievement in mathematics and science, at both grade four and grade eight, was associated with students having a greater sense of school belonging and experiencing little or no bullying. At the system level, the results of PISA 2018 for 15-year-olds also indicated “that instruction hindered by a lack of educational materials was associated with lower reading scores in all participating countries and economies. School systems that showed more equity in the allocation of material resources tended to score higher in reading” (OECD, 2020 , p. 196).

We found that the amount of material resources in schools was related to grade four students’ mathematics and science achievement in four of the Dinaric participants (Albania, North Macedonia, and Serbia), and related only to their mathematics achievement in Croatia. We found that schools with more students coming from affluent backgrounds tended to have the highest achievement in every participating system except Kosovo. Other research found that that there was a stronger emphasis on academic success in schools that are better equipped (see chapter “ Characteristics of Principals and Schools in the Dinaric Region ”). According to their teachers’ perceptions, almost all students in Albania to about half of the students in Croatia were taught in very safe and orderly schools. There was not a high prevalence of bullying in the Dinaric region, although around a third of students in Croatia and North Macedonia reported that they were bullied monthly or weekly; this is a worrying level of bullying, and educational professionals in the region should devote more attention to finding solutions to tackle this issue.

Although many education systems in the Dinaric region still have much to improve in terms of equipping schools with better material resources, our study highlights the importance of effective practice, and developing a supportive school climate and culture of achievement. “Ensuring that all schools have adequate and high-quality material resources, and the appropriate support, is key if students from all backgrounds are to be given equal opportunities to learn and succeed at school” (OECD, 2020 , p. 16). As the definition of school material resources has broadened to include ICT skills and the associated digital tools and resources, school systems face a whole new level of procurement.

Our study has confirmed that, beside the physical environment and material resources that support learning in schools, there are additional, less tangible dimensions of school life, which are equally important for the successful achievement of educational goals. The most important task of educational systems and school authorities is still to set and maintain both material factors (resources) and social factors of school functioning (such as safety, order, support, and emphasis on achievement goals), and often the core aim is to improve student achievement. But, ideally, schools should provide equal opportunities for students that come from challenging or deprived environments; it is important that schools are not just buildings but also active catalysts of change through learning processes. Theory and ILSA results suggest that a good physical environment and sufficient material resources, together with supportive teachers, the existence of peer practices (for teachers and students), innovative methods, an open climate for discussion, and willingness to cooperate with parents and/or the wider community, establishes a productive setting for better learning outcomes. Where schools do not have a shortage of material resources (such as space, equipment, or staffing), a critical factor for success is supporting healthy social relationships and fostering an open school climate, providing a school environment free from bullying and other stress factors. Our analyses showed that school-level variables only explained low levels of variance across the Dinaric region; consequently we conclude that home resources, the sociocultural capital of parents/guardians, and their willingness to participate in their child’s schooling must play a major role in student achievement, together with students’ attitudes toward the subject matter and their schools. While upgrading the material aspects of the educational environment is something that schools can influence and work hard on improving, good results can also be obtained by creating strong and healthy teaching and learning communities.

This designation is without prejudice to positions on status, and is in line with UNSCR 1244/1999 (United Nations 1999 ) and the International Court of Justice (ICJ) Opinion on the Kosovo declaration of independence (ICJ 2010 ).

Slovenia also participated in TIMSS 2015 (and achieved above the TIMSS international average results in mathematics and science), but did not participate in TIMSS 2019 survey and thus could not take part in this comparative analysis of the Dinaric region.

Here the term “digital” does not simply refer to digital machines and processes, but to the entire political, social, and economic context and infrastructure within which they have emerged. We now live in a “digital age” (Burston et al., 2010 , p. 215).

Bear, G., Yang, C., Pell, M., & Gaskins, C. (2014). Validation of a brief measure of teachers’ perceptions of school climate: Relations to student achievement and suspensions. Learning Environments Research, 17 , 339–354. https://doi.org/10.1007/s10984-014-9162-1

Article   Google Scholar  

Brand, S., Felner, R. D., Seitsinger, A., Burns, A., & Bolton, N. (2008). A large scale study of the assessment of the social environment of middle and secondary schools: The validity and utility of teachers’ ratings of school climate, cultural pluralism, and safety problems for understanding school effects and school improvement. Journal of School Psychology, 46 , 507–535. https://doi.org/10.1016/j.jsp.2007.12.001

Burston, J., Dyer-Witheford, N., & Hearn, A. (2010). Digital labour: Workers, authors, citizens. Ephemera, 10 (3), 214–221. http://www.ephemerajournal.org/sites/default/files/pdfs/10-3ephemeranov10.pdf

Cohen, J., McCabe, L., Michelli, N. M., & Pickeral, T. (2009). School climate: Research, policy, practice, and teacher education. The Teachers College Record, 111, 180–213. https://eric.ed.gov/?id=EJ826002

Drigas, A., & Vasiliki, T. (2015). Lifelong learning and ICTs: A review. International Journal of Recent Contributions from Engineering Science & IT, 3 (2), 15–20. https://doi.org/10.3991/ijes.v3i2.4353

Eickelmann, B. (2011). Supportive and hindering factors to a sustainable implementation of ICT in schools. Journal for Educational Research Online/ Journal für Bildungsforschung Online, 3 (1), 75–103. https://www.pedocs.de/volltexte/2011/4683/pdf/JERO_2011_1_Eickelmann_Sup

Fishbein, B., Foy, P., & Yin, L. (2021). TIMSS 2019 user guide for the international database. TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timss2019/international-database/

Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020). Preparing for life in a digital world: IEA international computer and information literacy study 2018 international report. Springer International Publishing. https://www.iea.nl/publications/study-reports/preparing-life-digital-world

Gregurović, M., & Kuti, S. (2010). Učinak socioekonomskog statusa na obrazovno postignuće učenika: Primjer PISA istraživanja, Hrvatska 2006 [The effect of socioeconomic status on students’ educational achievement: Example of PISA research, Croatia 2006]. Revija Za Socijalnu Politiku, 17 (2), 179–196. https://doi.org/10.3935/rsp.v17i2.918

Hoy, W. K., Tarter, C. J., & Kottkamp, R. B. (1991). Open schools/healthy schools: Measuring organizational climate . Sage.

Google Scholar  

ICJ. (2010). Accordance with international law of the unilateral declaration of independence in respect of Kosovo, Advisory Opinion, I.C.J. reports 2010 . International Court of Justice. https://www.icj-cij.org/public/files/case-related/141/141-20100722-ADV-01-00-EN.pdf

Kutsyuruba, B., Klinger, D., & Hussain, A. (2015). Relationships among school climate, school safety, and student achievement and well-being: A review of the literature. Review of Education, 3 , 103–135. https://doi.org/10.1002/rev3.3043

Matković T., Dobrotić, I., & Baran J. (2019). Što vrtić ima s tim? Pristup ranom i predškolskom odgoju i obrazovanju i reprodukcija društvenih nejednakosti u redovnom školovanju: analiza podataka PISA i TIMSS istraživanja [What’s kindergarten got to do with it? Access to early childhood education and care and reproduction of social inequalities in regular education: Analysis of PISA and TIMSS Ddata]. Revija za sociologiju , 49 (1), 7–35. https://doi.org/10.5613/rzs.49.1.1

Martin, M. O., & Mullis, I. V. S. (Eds.). (2013). TIMSS and PIRLS 2011: Relationships among reading, mathematics, and science achievement at the fourth grade—Implications for early learning. TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timsspirls2011/downloads/TP11_Intro.pdf

Martin, M. O., Mullis, I. V. S., Foy, P., & Hooper, M. (2016). TIMSS 2015 International Results in Science. Retrieved from Boston College, TIMSS & PIRLS International Study Center. http://timssandpirls.bc.edu/timss2015/international-results/

Meinck, S., Stancel-Piatak, A., & Verdisco, A. (2018). Preparing the ground: The importance of early learning activities at home for fourth grade student achievement. IEA Compass: Briefs in Education 3. International Association for the Evaluation of Educational Achievement. https://www.iea.nl/publications/series-journals/iea-compass-briefs-education-series/september-2018-preparing-ground

Mihaljević Kosor M., Malešević Perović L., & Golem S. (2019). The role of international benchmarking in shaping educational policy in small European countries. In H. A. Ingþórsson, N. Alfirević, J. Pavičić, & D. Vican (Eds.), Educational leadership in policy: Challenges and implementation within Europe (pp. 27–40). Palgrave Macmillan. https://doi.org/10.1007/978-3-319-99677-6_3

Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2017). PIRLS 2016 international results in reading. TIMSS & PIRLS International Study Center, Boston College. http://timssandpirls.bc.edu/pirls2016/international-results/

Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016a). TIMSS 2015 international results in mathematics. TIMSS & PIRLS International Study Center, Boston College. http://timssandpirls.bc.edu/timss2015/international-results/

Mullis, I. V. S., Martin, M. O., & Loveless, T. (2016b). 20 years of TIMSS: International trends in mathematics and science achievement, curriculum, and instruction. TIMSS & PIRLS International Study Center, Boston College. http://timssandpirls.bc.edu/timss2015/international-results/timss2015/wp-content/uploads/2016/T15-20-years-of-TIMSS.pdf

Mullis, I. V. S., Martin, M. O., Foy, P., Kelly, D. L., & Fishbein, B. (2020). TIMSS 2019 international results in mathematics and science. TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timss2019/international-results/

Martin, M. O., von Davier, M., & Mullis, I. V. S. (Eds.). (2020). Methods and procedures: TIMSS 2019 technical report. TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timss2019/methods

OECD. (2019a). PISA 2018 results (Volume I): What students know and can do . OECD Publishing. https://www.oecd.org/pisa/publications/pisa-2018-results-volume-i-5f07c754-en.htm

OECD. (2019b). PISA 2018 results (Volume II): Where all students can succeed . OECD Publishing. https://www.oecd.org/pisa/publications/pisa-2018-results-volume-ii-b5fd1b8f-en.htm

OECD. (2019c). PISA 2018 results (Volume III): What school life means for students’ lives . OECD Publishing. https://www.oecd.org/pisa/publications/pisa-2018-results-volume-iii-acd78851-en.htm

OECD. (2019d). TALIS 2018 results (Volume I): Teachers and school leaders as lifelong learners . OECD Publishing. http://www.oecd.org/education/talis-2018-results-volume-i-1d0bc92a-en.htm

OECD. (2020). Education in the Western Balkans: Findings from PISA. OECD Publishing. https://www.oecd-ilibrary.org/education/education-in-the-western-balkans_764847ff-en

Schulz, W., Ainley, J., Fraillon, J., Kerr, D., & Losito, B. (2010). ICCS 2009 international report: Civic knowledge, attitudes and engagement among lower secondary school students in thirty-eight countries . International Association for the Evaluation of Educational Achievement. https://www.iea.nl/publications/study-reports/international-reports-iea-studies/iccs-2009-international-report

TIMSS & PIRLS International Study Center. (2016). Press release: As global study TIMSS turns 20, new results show East Asian students continue to outperform peers in mathematics . TIMSS & PIRLS International Study Center, Boston College. http://timss2015.org/wp-content/uploads/2016/T15-Press-Release-FINAL-11-29.pdf

TIMSS & PIRLS International Study Center. (2018). TIMSS 2019 context questionnaires. TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timss2019/questionnaires/index.html

UNESCO. (2014). Harnessing the potential of ICTs for literacy teaching and learning. UNESCO Institute for Lifelong Learning. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000229517

United Nations. (1999). Resolution 1244 (1999). Adopted by the Security Council at its 4011th meeting, on 10 June 1999. United Nations Security Council. https://digitallibrary.un.org/record/274488?ln=en

United Nations Children’s Fund. (2017). A familiar face: Violence in the lives of children and adolescents . UNICEF. https://data.unicef.org/resources/a-familiar-face/

Webb, M., & Cox, M. (2004). A review of pedagogy related to information and communications technology. Technology, Pedagogy and Education, 13 (3), 235–286.

Yin, L., & Fishbein, B. (2020). Creating and interpreting the TIMSS 2019 context questionnaire scales. In M. O. Martin, M. von Davier, & I. V. S. Mullis (Eds.), Methods and procedures: TIMSS 2019 technical report (pp. 16.1–16.331). TIMSS & PIRLS International Study Center, Boston College.  https://timssandpirls.bc.edu/timss2019/methods/chapter-16.html

Download references

Author information

Authors and affiliations.

National Centre for External Evaluation of Education, Zagreb, Croatia

Ines Elezović

National Examination Centre, Skopje, Republic of Macedonia

Beti Lameva

International Association for the Evaluation of Educational Achievement (IEA), Hamburg, Germany

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ines Elezović .

Editor information

Editors and affiliations.

Educational Research Institute, Ljubljana, Slovenia

Barbara Japelj Pavešić

International Association for the Evaluation of Educational Achievement (IEA), Amsterdam, The Netherlands

Paulína Koršňáková

Sabine Meinck

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License ( http://creativecommons.org/licenses/by-nc/4.0/ ), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2022 International Association for the Evaluation of Educational Achievement (IEA)

About this chapter

Elezović, I., Lameva, B., Brese, F. (2022). The Role of Learning Resources, School Environment, and Climate in Transforming Schools from Buildings to Learning Communities. In: Japelj Pavešić, B., Koršňáková, P., Meinck, S. (eds) Dinaric Perspectives on TIMSS 2019. IEA Research for Education, vol 13. Springer, Cham. https://doi.org/10.1007/978-3-030-85802-5_6

Download citation

DOI : https://doi.org/10.1007/978-3-030-85802-5_6

Published : 11 November 2021

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-85801-8

Online ISBN : 978-3-030-85802-5

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Research on learning resource recommendation based on knowledge graph and collaborative filtering, 1. introduction, 2.1. collaborative filtering algorithm, 2.2. the knowledge graph-based recommendation algorithm, 3. combining knowledge graph and collaborative filtering recommendation algorithm, 3.1. knowledge graph similarity calculation, 3.1.1. construction of knowledge graph, 3.1.2. knowledge graph representation learning, 3.2. collaborative filtering similarity calculation, 3.2.1. implicit feedback model, 3.2.2. item based collaborative filtering, 3.3. similarity fusion, 4. analysis of experimental data and results, 4.1. experimental dataset, 4.2. evaluating indicator, 4.3. fusion proportion experiment and result analysis, 4.4. algorithm comparison, 4.5. preference comparison experiment and result analysis, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Dahdouh, K.; Dakkak, A.; Oughdir, L.; Ibriz, A. Large-scale e-learning recommender system based on Spark and Hadoop. J. Big Data 2019 , 6 , 2. [ Google Scholar ] [ CrossRef ]
• Fu, Q.K.; Lin, C.J.; Hwang, G.J.; Zhang, L. Impacts of a mind mapping-based contextual gaming approach on EFL students’ writing performance, learning perceptions and generative uses in an English course. Comput. Educ. 2019 , 137 , 59–77. [ Google Scholar ] [ CrossRef ]
• Cui, L.Z.; Guo, F.L.; Liang, Y.J. Research overview of educational recommender systems. In Proceedings of the 2nd International Conference on Computer Science and Application Engineering, Hohhot, China, 22–24 October 2018; pp. 1–7. [ Google Scholar ]
• Lee, Y.; Choi, J. A review of online course dropout research: Implications for practice and future research. Educ. Technol. Res. Dev. 2011 , 59 , 593–618. [ Google Scholar ] [ CrossRef ]
• Kong, L.; Ma, H.; Peng, Y.; Tang, W. Hybrid Recommendation of Personalized MOOC Resources: A User Context-aware Approach. In Proceedings of the 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Hangzhou, China, 4–6 May 2022; pp. 537–542. [ Google Scholar ]
• Chiappe-Laverde, A.; Hine, N.; Martínez-Silva, J.A. Literature and Practice: A Critical Review of MOOCs. Comunicar 2015 , 22 , 9–17. [ Google Scholar ] [ CrossRef ]
• Martín-Monje, E.; Castrillo, M.D.; Mañana-Rodríguez, J. Understanding online interaction in language MOOCs through learning analytics. Comput. Assist. Lang. Learn. 2018 , 31 , 251–272. [ Google Scholar ] [ CrossRef ]
• Feng, W.; Tang, J.; Liu, T.X. Understanding dropouts in MOOCs. Proc. AAAI Conf. Artif. Intell. 2019 , 33 , 517–524. [ Google Scholar ] [ CrossRef ]
• Lu, J.; Wu, D.; Mao, M.; Wang, W.; Zhang, G. Recommender system application developments: A survey. Decis. Support Syst. 2015 , 74 , 12–32. [ Google Scholar ] [ CrossRef ]
• Pecori, R. A virtual learning architecture enhanced by fog computing and big data streams. Future Internet 2018 , 10 , 4. [ Google Scholar ] [ CrossRef ]
• Buder, J.; Schwind, C. Learning with personalized recommender systems: A psychological view. Comput. Hum. Behav. 2012 , 28 , 207–216. [ Google Scholar ] [ CrossRef ]
• Joy, J.; Raj, N.S. An ontology model for content recommendation in personalized learning environment. In Proceedings of the Second International Conference on Data Science, E-Learning and Information Systems, Dubai, United Arab Emirates, 2–5 December 2019; pp. 1–6. [ Google Scholar ]
• Liu, Z.; Cheng, M.; Li, Z.; Liu, Q.; Chen, E. One Person, One Model—Learning Compound Router for Sequential Recommendation. In Proceedings of the 2022 IEEE International Conference on Data Mining (ICDM), Orlando, FL, USA, 28 November–1 December 2022; pp. 289–298. [ Google Scholar ]
• Huang, L.; Jiang, B.; Lv, S.; Liu, Y.; Li, D. A Review of Recommender Systems Based on Deep Learning. J. Comput. Sci. 2018 , 7 , 1619–1647. [ Google Scholar ]
• Bollen, D.; Knijnenburg, B.P.; Willemsen, M.C.; Graus, M. Understanding choice overload in recommender systems. In Proceedings of the Fourth ACM Conference on Recommender Systems, Barcelona, Spain, 26–30 September 2010; pp. 63–70. [ Google Scholar ]
• Wang, C.; Zhu, H.; Zhu, C.; Zhang, X.; Chen, E.; Xiong, H. Personalized employee training course recommendation with career development awareness. In Proceedings of the Web Conference 2020, Taipei, Taiwan, 20–24 April 2020; pp. 1648–1659. [ Google Scholar ]
• Lv, P.; Wang, X.; Xu, J.; Wang, J. Intelligent personalised exercise recommendation: A weighted knowledge graph-based approach. Comput. Appl. Eng. Educ. 2021 , 29 , 1403–1419. [ Google Scholar ] [ CrossRef ]
• Wan, S.; Niu, Z. A hybrid e-learning recommendation approach based on learners’ influence propagation. IEEE Trans. Knowl. Data Eng. 2019 , 32 , 827–840. [ Google Scholar ] [ CrossRef ]
• Zhu, Y.; Lu, H.; Qiu, P.; Shi, K.; Chambua, J.; Niu, Z. Heterogeneous teaching evaluation network based offline course recommendation with graph learning and tensor factorization. Neurocomputing 2020 , 84–95. [ Google Scholar ] [ CrossRef ]
• Zhu, Y.; Lin, Q.; Lu, H.; Shi, K.; Liu, D.; Chambua, J.; Wan, S.; Niu, Z. Recommending learning objects through attentive heterogeneous graph convolution and operation-aware neural network. IEEE Trans. Knowl. Data Eng. 2021 . [ Google Scholar ] [ CrossRef ]
• Pereira, M.; Etemad, E.; Paulovich, F. Iterative learning to rank from explicit relevance feedback. In Proceedings of the 35th Annual ACM Symposium on Applied Computing, Brno, Czech Republic, 30 March–3 April 2020; pp. 698–705. [ Google Scholar ]
• Wen, H.; Yang, L.; Estrin, D. Leveraging post-click feedback for content recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems, Copenhagen, Denmark, 16–20 September 2019; pp. 278–286. [ Google Scholar ]
• Himeur, Y.; Alsalemi, A.; Al-Kababji, A.; Bensaali, F.; Amira, A.; Sardianos, C.; Dimitrakopoulos, G.; Varlamis, I. A survey of recommender systems for energy efficiency in buildings: Principles, challenges and prospects. Inf. Fusion 2021 , 72 , 1–21. [ Google Scholar ] [ CrossRef ]
• Chen, J.; Zhang, H.; He, X.; Nie, L.; Liu, W.; Chua, T.S. Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, Japan, 7–11 August 2017; pp. 335–344. [ Google Scholar ]
• Zhou, M.; Ding, Z.; Tang, J.; Yin, D. Micro behaviors: A new perspective in e-commerce recommender systems. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, Marina Del Rey, CA, USA, 5–9 February 2018; pp. 727–735. [ Google Scholar ]
• Xu, B.; Huang, Z.; Liu, J.; Shen, S.; Liu, Q.; Chen, E.; Wu, J.; Wang, S. Learning Behavior-oriented Knowledge Tracing. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach, CA, USA, 6–10 August 2023; pp. 2789–2800. [ Google Scholar ]
• Sharif, N.; Afzal, M.T. Recommendation approaches for e-learners: A survey. In Proceedings of the 7th International Conference on Management of computational and collective intElligence in Digital EcoSystems, Caraguatatuba, Brazil, 25–29 October 2015; pp. 137–141. [ Google Scholar ]
• Klašnja-Milićević, A.; Ivanović, M.; Vesin, B.; Budimac, Z. Enhancing e-learning systems with personalized recommendation based on collaborative tagging techniques. Appl. Intell. 2018 , 48 , 1519–1535. [ Google Scholar ] [ CrossRef ]
• Xian, Y.; Fu, Z.; Muthukrishnan, S.; De Melo, G.; Zhang, Y. Reinforcement knowledge graph reasoning for explainable recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France, 21–25 July 2019; pp. 285–294. [ Google Scholar ]
• Gu, Y.; Zhang, Y.; Yang, H. Movie recommendation algorithm based on movie attributes and interaction information. J. Nanjing Univ. Sci. Technol. 2022 , 46 , 177–184. [ Google Scholar ]
• Feng, X.; Yang, Q. Collaborative filtering algorithm with information enhancement based on knowledge graph. J. Jiangxi Norm. Univ. Nat. Sci. 2022 , 04 , 386–393. [ Google Scholar ]
• Li, Z.; Gao, D.; Liu, X.; Wu, J. Knowledge Graph-Based Ocean Numerical Forecast Data Recommendation Algorithm. Comput. Eng. Des. 2023 , 05 , 1385–1391. [ Google Scholar ]
• Essalmi, F.; Ayed LJ, B.; Jemni, M.; Graf, S. A fully personalization strategy of E-learning scenarios. Comput. Hum. Behav. 2010 , 26 , 581–591. [ Google Scholar ] [ CrossRef ]
• Dwivedi, P.; Bharadwaj, K.K. e-Learning recommender system for a group of learners based on the unified learner profile approach. Expert Syst. 2015 , 32 , 264–276. [ Google Scholar ] [ CrossRef ]
• Alharbe, N.; Rakrouki, M.A.; Aljohani, A. A collaborative filtering recommendation algorithm based on embedding representation. Expert Syst. Appl. 2023 , 215 , 119380. [ Google Scholar ] [ CrossRef ]
• Aljunid, M.F.; Dh, M. An efficient deep learning approach for collaborative filtering recommender system. Procedia Comput. Sci. 2020 , 171 , 829–836. [ Google Scholar ] [ CrossRef ]
• Mandal, S.; Maiti, A. Explicit feedback meet with implicit feedback in GPMF: A generalized probabilistic matrix factorization model for recommendation. Appl. Intell. 2020 , 50 , 1955–1978. [ Google Scholar ] [ CrossRef ]
• Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; Yakhnenko, O. Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Process. Syst. 2013 , 26 , 2787–2795. [ Google Scholar ]
• Wang, H.; Lun, B.; Zhang, X.; Sun, X. Multi-modal entity alignment based on joint knowledge representation learning. Control. Decis. 2020 , 35 , 2855–2864. [ Google Scholar ]
• Wang, Z.; Zhang, J.; Feng, J.; Chen, Z. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAl Conference on Artificial Intelligence, Québec City, QC, Canada, 27–31 July 2014; Volume 28. [ Google Scholar ]
• Lin, Y.; Liu, Z.; Sun, M.; Liu, Y.; Zhu, X. Learning entity and relation embeddings for knowledge graphcompletion. In Proceedings of the AAAl Conference on Artificial Intelligence, Austin, TX, USA, 25–30 January 2015; Volume 29. [ Google Scholar ]
• Ji, G.; He, S.; Xu, L.; Liu, K.; Zhao, J. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long papers); Association for Computational Linguistics: Beijing, China, 2015; pp. 687–696. [ Google Scholar ]
• Bellogín, A.; Cantador, I.; Díez, F.; Castells, P.; Chavarriaga, E. An empirical comparison of social, collaborative filtering, and hybrid recommenders. ACM Trans. Intell. Syst. Technol. TIST 2013 , 4 , 14. [ Google Scholar ] [ CrossRef ]
• Zhang, F.; Yuan, N.J.; Lian, D.; Xie, X.; Ma, W.Y. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 353–362. [ Google Scholar ]
• Wang, X.; He, X.; Cao, Y.; Liu, M.; Chua, T.S. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2019; pp. 950–958. [ Google Scholar ]
• Yu, J.; Luo, G.; Xiao, T.; Zhong, Q.; Wang, Y.; Feng, W.; Luo, J.; Wang, C.; Hou, L.; Li, J.; et al. MOOCCube: A large-scale data repository for NLP applications in MOOCs. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics; Association for Computational Linguistics: Beijing, China, 2020; pp. 3135–3142. [ Google Scholar ]

Click here to enlarge figure

Share and Cite

Niu, Y.; Lin, R.; Xue, H. Research on Learning Resource Recommendation Based on Knowledge Graph and Collaborative Filtering. Appl. Sci. 2023 , 13 , 10933. https://doi.org/10.3390/app131910933

Niu Y, Lin R, Xue H. Research on Learning Resource Recommendation Based on Knowledge Graph and Collaborative Filtering. Applied Sciences . 2023; 13(19):10933. https://doi.org/10.3390/app131910933

Niu, Yanmin, Ran Lin, and Han Xue. 2023. "Research on Learning Resource Recommendation Based on Knowledge Graph and Collaborative Filtering" Applied Sciences 13, no. 19: 10933. https://doi.org/10.3390/app131910933

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Explore. Create. Collaborate.

Support a vibrant, educator-focused commons.

The tens of thousands of open resources on OER Commons are free - and they will be forever - but building communities to support them, developing new collections, and creating infrastructure to grow the open community isn’t. Grassroots donations from people like you can help us transform teaching and learning.

Make a Donation Today!

Introducing Open Author

Create OER with Open Author

Open Author helps you build and publish Open Educational Resources for you and for the benefit of educators and learners everywhere.

Build. Save. Collaborate.

Groups provides a flexible environment to organize, create, share, and discuss resources with others in your network. Collaborate with group members, tag and add resources to shared folders, create your own collections, all within a public or private group.

Extensive Library, Powerful Findability

Harness the power of the Commons

Dedicated to professional learning.

Our award-winning OER Professional Learning programs support instructors and curriculum specialists to gain the necessary skills required to find, adapt, and evaluate high quality open materials.

In-person and virtual OER workshops help focus instructors to develop a "commons" mindset, to experience the benefits of open processes using the platform's tools, collaborative features, and workflows. This can include using Open Author for creating, remixing, and publishing shared curriculum.

Become an OER Commons pro

How to Search OER Commons

How to Use Groups

How to Create a Profile

• Learning how to find, evaluate, and use resources to explore a topic in depth

Series Editor : Michael Theall, Youngstown State University Authors : Gail MacKay, Indiana University Kokomo; Barbara Millis, University of Nevada-Reno; Rebecca Brent, Education Designs, Inc.

At institutions of higher education across the U.S., information literacy (IL) is being integrated into general education curricula as a specific learning objective. The Association of College and Research Libraries (ACRL) (1) defines information literate students as those who “recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information.” As the world moves toward a knowledge-based economy, information literacy becomes a crucial component of preparing students for the lifelong learning that current and future job markets demand.

IDEA Research Report #1 (2) states that, “…It is important to recognize that much of the subject matter content which students learn today will be outdated in 5-10 years after they graduate.” Thus, an emphasis on lifelong learning seems imperative. Canja (3), for example, suggests that “… Lifelong learning has become an economic necessity for national and global productivity. With the decline in birth rates in major developed countries, persons—still active, still healthy—must continue in the workforce, trained and retrained” (p. 27). Ironically, IDEA Research Report #1 also finds that the objectives identified as emphasizing lifelong learning (Learning to find and use resources, and Gave tests/projects that covered most important points) were identified as “Important” or “Essential” in only about 30% of the classes using IDEA. The ACRL (1) notes, “Information literacy forms the basis for lifelong learning. It is common to all disciplines, to all learning environments, and to all levels of education. It enables learners to master content and extend their investigations, become more self-directed, and assume greater control over their own learning.” However, information literacy does not concern itself only with technical resources. Successful students and workers must also be able to affiliate with others and to seek and find expertise among the human resources that are available (4).

Seeking out information resources and then using them to address a question or a problem are engaging activities, and there are several attached benefits. First is recognition of the value of the resources. Next is application of the new information and the construction of new knowledge. Intrinsic motivation results from the realization that learning is taking place and ultimately, these practical and motivational effects promote continued use of the resources, lifelong learning, and facilitates deep learning.

For example, here are key components that characterize a deep, rather than a surface approach to learning. Rhem (5) summarizes them as follows:

Motivational context: We learn best what we feel a need to know. Intrinsic motivation remains inextricably bound to some level of choice and control. Courses that remove these take away the sense of ownership and kill one of the strongest elements in lasting learning.

Learner activity: Deep learning and “doing” travel together. Doing in itself isn’t enough. Faculty must connect activity to the abstract conceptions that make sense of it, but passive mental postures lead to superficial learning.

Interaction with others: As Noel Entwistle put it in a recent email message, “The teacher is not the only source of instruction or inspiration.”

A well-structured knowledge base: This does not just mean presenting new material in an organized way. It also means engaging and reshaping, when necessary, the concepts students bring with them. Deep approaches and learning for understanding are integrative processes. The more fully new concepts can be connected with students’ prior experience and existing knowledge, the more it is they will be impatient with inert facts and eager to achieve their own synthesis (p. 4).

If instructors are to motivate students to acquire the skills of information literacy that will help them to remain lifelong learners, then they need to design research projects and assignments that get students into the knowledge base and engage them in critical thinking activities through active learning and interaction with one another. Through such sequenced assignments, students can learn how to answer relevant questions and to solve challenging problems.

Keep in mind that an important component of finding and using resources to explore topics is evaluating the quality of those resources. In an information-rich world, students must be able to determine if a resource is reliable and valid enough to use in their work. These information literacy skills (and even quantitative literacy skills–see the Teaching Note, “Learning appropriate methods for collecting, analyzing, and interpreting numerical information”) must be taught. See the Teaching Note, “Encouraged students to use multiple resources (e.g. Internet, library holdings, outside experts) to improve understanding,” for more ideas.

Teaching This Objective

The most relevant IDEA instructional method is “encouraged students to use multiple resources to improve understanding.” This Learning Note complements Baron’s with some general guidelines that focus on developing good research projects or assignments to assist with “learning how to find and use resources for answering questions or solving problems” and attempts to help instructors provide students with effective and feasible assignments. With today’s information overload, students need guidance in locating and using appropriate resources for answering questions and solving problems. Students must hone these skills throughout their lives. Academic librarians can serve as an instructor’s best ally.

Other IDEA instructional methods that are important to Objective #9 include items #2 Finding ways to help students answer their own questions, #8 Stimulating intellectual effort, #15 Inspiring students to set and achieve goals, #18 Asking students to help each other understand ideas or concepts, and #19 Assigning work that requires original or creative thinking. These relationships are logical because the nature of investigative activity requires intellectual effort, focused exploration, and creativity, and the connections between problem solving and gathering information and evidence have been well-documented (6). These methods support many of the specific hints described below.

Motivation as a starting point. Locating information for its own sake provides practice, but it fails to engage motivated students in productive work linked to an understood outcome. Feldman suggests that student achievement remains strongly correlated to the perceived outcomes of instruction (7). The relevance of assigned work is also critical to student’s active engagement (8) and a major predictor of student ratings of their teachers (9). Thus, skill development becomes much more productive when there is a clearly understood link between the assigned work and specific learning goals or tangible products. The real-world analog is obvious: people do not search for information unless they have a reason to do so. Because in many teaching-learning situations, teachers expect students to explore issues and topics that may not intrinsically interest them, demonstrating relevance and utility become critical first steps in getting students engaged (See “Related course material to real-life situations” and “Introduced stimulating ideas about the subject”). Allowing students some choice of topic or project can motivate them to take a deeper approach to learning (10).

Sequence the research project or assignment. If instructors want students to learn to find and use resources to tackle stimulating questions and challenging problems through research, they need to design sequenced activities that motivate students and get them into the knowledge base. This can often be accomplished through the individual work that students do either as discrete homework assignments or as smaller parts of an extended research project. What becomes of these assignments or project components is critical for deep learning. Instructors should design in-class exercises where learners are actively engaged with the material they prepared individually and with each other (11).

A. Planning

• Arrange for library instruction. Even students who have achieved some level of proficiency with library research will benefit from the reinforcement and enhancement of their skills. Require attendance. Attend yourself, asking questions as a learner.
• Bring the class to the library or ask a librarian to come to your classroom when they are ready to begin their project, not in advance. Students learn best when there is an immediate and applicable need.
• Send a copy of the assignment to the instruction librarian at your campus. Ask for input before finalizing the assignment. Librarians, for example, are highly skeptical of the academic value of commonly assigned “Library Scavenger Hunts.”
• Include homegrown resource guides, sometimes called “pathfinders,” in your initial quest for student library sources. Often campus instruction and reference librarians develop these guides for various fields or disciplines. If your field is not included, ask the library to develop a resource guide for your area. These subject guides provide students with suggestions for “where to start” their research. Included in the guides are both print and electronic sources such as subject encyclopedias; specialized periodical indexes such as Applied Science and Technology Index ; PsycINFO ; or Sociological Abstracts ; also included are reference works or standards in the field such as The Physician’s Desk Reference (PDR) ; CRC Handbook of Chemistry and Physics ; or the Statistical Abstract of the United States .
• Consider alternatives to the conventional research paper. Excellent assignment ideas reside on the Web, often at other campus library sites (12, 13, 14, 15, 16).

B. Designing

• Provide your expectations for the assignment in writing to your students. Let them know what the assignment involves and what you expect them to learn from the experience. “I don’t know what s/he wants” is a student lament transcending the ages. Make their day; tell them. To help students fully understand these expectations in practice, consider providing strong and weak samples of typical segments of the project or assignment to discuss and critique in class.
• Specify how the assignment fits with the goals or objectives of the course to show relevance. Be as explicit as possible. Share this information, also, with instruction librarians to help them determine appropriate sources.
• Provide students with the grading criteria in writing for the project or assignment.
• Offer a variety of flexible topics, encouraging students to choose ones that interest them.
• Review the student-selected topics to see that they are appropriate and achievable. Avoid very current or local topics if students need scholarly sources as scholarly peer reviewed journals take time to reach publication.
• Place materials on “Reserve,” if necessary, to avoid having 30 students compete for six books.
• Discuss the role of attribution and documentation in a community of scholars. Include a policy on plagiarism in the syllabus. Emphasize the ethical use of information and of the avoidance of plagiarism. Aside from ethics, there are also copyright laws, both national and international, to consider. Specifically discuss appropriate and inappropriate use of online material, a gray area for many students.
• Announce which style manual you expect students to use. Be very specific about documentation for online sources. Many style manuals are difficult to interpret.

Provide opportunities to engage in deep learning. As noted in the background section, the key components characterizing deep learning are motivation context, learner activity, interaction with others, and a well-structured knowledge base (5). As an example, faculty members can ask students, as part of a larger research project, to prepare paired annotations based on the double entry journal recommended by writing across the curriculum and classroom assessment experts (17). The teacher or the students identify a pool of articles on the question or problem at hand. Each student, working individually out-of-class, prepares a reflective commentary on one of the articles or chapters. They do so using a double column format (a Microsoft Word table works beautifully) where they cite the key points of the research article on the left-hand side and reactions, questions, commentary, and connections with other readings on the right, aligning the key point with the reflective commentary. The entries in these columns will not be the same length. When students come to class, the teacher randomly pairs them with another student who has read and analyzed the same research article. The two partners now read one another’s reflective commentaries, comparing both the key points they have identified and their specific responses to them. They discuss their reasons for the choices they made. Then working together, they prepare a composite annotation summarizing the article (See IDEA Paper No. 38).

This activity should be repeated several times during the semester, pairing different students. It enables students to reflect on their own thinking skills (metacognition) and to compare their thinking with those of other students. The more paired annotations they complete, the more skilled students become at identifying key points in an article and “using resources for answering questions or solving problems.” This structure thus enables teachers to sequence learning in meaningful ways. It builds critical thinking and writing skills by having students analyze and then compare their responses to the same piece of writing. It has the additional virtue of being relevant to virtually any discipline. Over the course of the semester, students build a repertoire of annotated research articles they can bring to bear on the given question or problem.

A note about technology. A thorough discussion of the ways in which new technologies can support and supplement students’ efforts to find and use resources is beyond the scope of this Note. However, we should mention at minimum, that the bounty that awaits students who explore web-based resources comes with a price: the equally large amount of inaccurate, incomplete, and sometimes distorted information that can be found in any web search. The critical issue for teachers is to construct assignments that require specific information known to exist and is accessible with minimum interference from useless, irrelevant, or biased data . Your resource librarian can be a tremendous asset in saving you hours of work (e.g., training students on effective and efficient search strategies and helping everyone to avoid wasting time and effort on valueless information). All disciplines and courses deal with electronic information and we cannot ignore its potential value. What is important to remember in constructing assignments is that the work must have a meaningful relationship to a clearly stated outcome. There has to be a tangible “payoff” in terms of students being able to connect the work to an understood and desired result.

Assessing This Objective

• Develop a rubric (or a form) to assess the announced grading criteria. For example, assign a certain number of points for each component of a project or assignment (see 2 below). What percentage of the total will the final paper and bibliography be? Note what happens if any of the required items are a day late; two days late; etc. What percentage will mechanics—spelling, punctuation, grammar—contribute to the final grade?
• Sequence parts of the project or assignment by establishing intermittent deadlines along the way. This practice not only helps prevent procrastination, but also helps to deter plagiarism. For example, have due dates for the overall topic and the thesis statement, due dates for a preliminary bibliography of “X” number of sources, an outline, a first draft, oral presentation, written or in-class peer reviews, etc.
• Require critical thinking. If students are using Web sites, for example, ask for the background or credentials of the author; ask for the date of last revision if currency is important; ask if students found any bias on the site; and ask why they selected this site from among all the others.
• Make use of peer reviewing throughout the research project or assignment to provide an additional source of feedback and add to the active learning and student interactions essential for deep learning. Have students exchange drafts and apply the rubric or checklist that will be used to assess the assignment. The opportunity for critical review of another draft and seeing comments from a peer will help them more fully understand the expectations, leading to better final products.
• Review respected resources such as the Tutorial for Developing and Evaluating Library Assignments at the University of Maryland University College, Adelphi, MD (18) and the Scoring Criteria for Development/Resource-Based Learning Project at Delta College, University Center, MI (19).
• Association of College and Research Libraries (ACRL). (2006). Information literacy competency standards for higher education. American Library Association. Retrieved September 27, 2006 from http://www.ala.org/ala/acrl/acrlstandards/standardsguidelines.htm
• Hoyt, D. P., & Perera, S. (2000). Teaching approach, instructional objectives, and learning: IDEA research report #1 . Manhattan, KS: IDEA Center, Kansas State University.
• Canja, E. T. (2002). Lifelong learning: Challenges and opportunities. CAEL Forum and News , 26-29.
• Klemp, G. O. (1977). Three factors of success. In D. W. Vermilye (Eds.) Relating work and education: Current issues in higher education 1977 . San Francisco: Jossey-Bass.
• Rhem, J. (1995). Close-up: Going deep. The National Teaching and Learning Forum, 5 (1), 4.
• See the Problem Based Learning website at: http://www.samford.edu/pbl/ for many resources and references. Retrieved September 27, 2006.
• Feldman, K. A. (1989). The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies. Research in Higher Education, 30 , 583-645.
• Theall, M. (1999). What have we learned? A synthesis and some guidelines for effective motivation in higher education. In M. Theall (Eds.) “Motivation from within: Encouraging faculty and students to excel.” New Directions for Teaching and Learning, 78 . San Francisco: Jossey-Bass.
• Franklin, J. L., & Theall, M. (1995). The relationship of disciplinary differences and the value of class preparation time to student ratings of instruction. In N. Hativa & M. Marincovich (Eds.) “Disciplinary differences in teaching and learning: Implications for practice.” New Directions for Teaching and Learning, 64 . San Francisco: Jossey-Bass.
• Felder, R.M., & Brent, R. (2005). Understanding student differences. Journal of Engineering Education, 94 (1), 57-72. Retrieved September 27, 2006 from http://www.ncsu.edu/effective_teaching/Papers/Understanding_Differences.pdf
• Millis, B. J. (2006). Helping faculty learn to teach better and “smarter” through sequenced activities. In S. Chadwick-Blossy & D.R. Robertson (Eds.). To Improve the Academy , Vol 24. (pp. 216-230). Bolton, MA: POD Network and Anker Publications.
• Designing assignments , University of Washington Libraries.
• Effective assignments using library and Internet resources . (2004). Teaching Library, University of California, Berkeley. Retrieved September 27, 2006 from http://www.lib.berkeley.edu/TeachingLib/assignments.html
• Fister, B., & Fuhr, S. (2001). Suggestions for assignments. Enhancing developmental research skills in the undergraduate curriculum . Folke Bernadotte Memorial Library, Gustavus Adolphus College. Retrieved September 27, 2006 from http://www.gustavus.edu/oncampus/academics/library/IMLS/assignmentsuggestions.html
• Recommendations for creating effective library assignments. (2005). Mitchell Memorial Library, Library Instructional Services, Mississippi State University. Retrieved September 27, 2006 from http://library.msstate.edu/content/templates/?a=323&z=74
• Creative assignments using information competency and writing. (2006). Ohio University, Athens OH. Retrieved September 27, 2006 from http://www.library.ohiou.edu/inst/creative.html
• Millis, B., & Cottell, P. (1998). Cooperative learning for higher education faculty. Greenwood Press: American Council on Education, Oryx Press.
• Kelley, K., & McDonald, R. (2005). Section 4: Designing assignments that contain writing and research. In Information literacy and writing assessment project: Tutorial for developing and evaluating assignments . Information and Library Services, University of Maryland University College.
• Examples of good assessments . (2006). Delta College Library, Delta College. Retrieved September 27, 2006 from http://www.delta.edu/library/assessments.html
• IDEA Paper No. 38: Enhancing Learning – and More! – Through Cooperative Learning , Millis
• IDEA Paper No. 41: Student Goal Orientation, Motivation, and Learning , Svinicki
• from http://ejournals.library.gatech.edu/ijsotel/index.php/ijsotel/article/view/19/18
• Gaining A Basic Understanding of the Subject
• Developing knowledge and understanding of diverse perspectives, global awareness, or other cultures
• Learning to apply course material
• Developing specific skills, competencies, and points of view needed by professionals in the field most closely related to this course
• Acquiring skills in working with others as a member of a team
• Developing creative capacities
• Gaining a broader understanding and appreciation of intellectual/cultural activity
• Developing skill in expressing myself orally or in writing
• Developing ethical reasoning and/or ethical decision making
• Learning to analyze and critically evaluate ideas, arguments, and points of view
• Learning to apply knowledge and skills to benefit others or serve the public good
• Learning appropriate methods for collecting, analyzing, and interpreting numerical information

Solution Tree Blog

Research Based Learning: a Lifelong Learning Necessity

  “Give a person a fish and he will eat for a day; teach a person to fish and she will eat for a lifetime.” – Adapted from a saying by an unknown author

What is Research-Based Learning? Research-based learning (RBL) consists of a framework that helps to prepare students to be lifelong inquirers and learners. The term “research,” which often conjures up a picture of students writing research reports, is here defined as a way of thinking about teaching and learning, a perspective, a paradigm. It is a specific approach to classroom teaching that places less emphasis on teacher-centered learning of content and facts and greater emphasis on students as active researchers.

In a research-based learning approach, students actively search for and then use multiple resources, materials, and texts in order to explore important, relevant, and interesting questions and challenges. They find, process, organize and evaluate information and ideas as they build reading skills and vocabulary. They learn how to read for understanding, form interpretations, develop and evaluate hypotheses, and think critically and creatively. They learn how to solve problems, challenges, and dilemmas. Finally, they develop communication skills through writing and discussion.

In the five stages of research-based learning, students:

a. Identify and clarify issues, questions, challenges, and puzzles. A key component of research-based learning is the identification and clarification of issues, problems, challenges and questions for discussion and exploration. The learner is able to seek relevancy in the work they are doing and to become deeply involved in the learning process. b. Find and process information. Students are tasked with searching for, finding, closely reading, processing, and using information related to the identified issue and question from one or more sources. As they seek out resources and read information, and then organize, classify, categorize, define, and conceptualize data. In the process, they become better readers. c. Think critically and creatively. Students are provided with the opportunity to use their researched information to compare and contrast, interpret, apply, infer, analyze, synthesize, and think creatively. d. Apply knowledge and ideas and draw conclusions. Students use what they have learned to draw conclusions, complete an authentic task, summarize results, solve problems, make decisions, or answer key questions. e. Communicate results. Students communicate results of their research activities in a number of possible ways, such as through a written research report, a persuasive essay, a book designed to teach younger students, a math problem solution, a plan of action, or a slide presentation to members of the community.

The Teacher’s Role Teachers play a key role in the success of research-based instruction by engaging and involving students in information gathering and processing. While teachers might occasionally provide information through lectures, and textbooks are used as a source of information, there is an emphasis placed on students learning how to seek out and process resources themselves. A teacher provides a climate that supports student curiosity and questioning . Teachers enable students to ask questions and pose problems. Students are invited to ask and answer questions. The classroom climate is conducive to using higher-order thinking and problem-solving skills to apply knowledge to solve problems. Teachers attempt to build ways for students to take ownership of their learning, to create a value and a purpose for learning.

In a research-based learning classroom, teachers often act more like a coach, guiding students as they develop questions and problems, helping students to find, read, sort, and evaluate information, giving students the opportunity to draw their own conclusions, and providing the time and the opportunity for students to communicate results.

Finally, one of the most important components of a successful research-based learning program is the ability to help students understand and apply this approach consistently, by providing them with research-based opportunities for learning. Thus students are encouraged to bring in additional materials and resources to help the class understand a topic, choose and complete projects and performance tasks as part of their units of study, and discuss issues using evidence from sources of information. The classroom climate and environment continually encourage students to express their opinions, problem solve, and think at higher levels.

Student Outcomes Significant outcomes occur when this approach is utilized over time. Learning how to search for and find reliable information and resources is a skill that is important for a lifetime of learning, Reading many different kinds of texts strengthens reading skills and builds vocabulary. Thinking skills are developed as students classify, organize, and synthesize information. “Habits of mind,” such as perseverance and resilience are strengthened through long-term projects. Writing skills are developed through note-taking, reflection activities, and many different types of writing tasks.

In addition, students feel greater ownership for their learning and the learning process and thus develop greater self-esteem with regard to learning. There is greater interest in and curiosity about learning and a willingness to work harder to learn. Students are more likely to retain information longer because it is more meaningful to them and organized in a more interesting fashion.

Finally, students are able to learn the difference between reliable and unreliable information, ideas, and resources, a key need in today’s world with so much misleading and erroneous information.

Summary The stages of research-based learning, key activities, and student outcomes are summarized in chart one, below. This framework also fits nicely with the four-phase model of instruction examined in my book Teaching for Lifelong Learning: How to Prepare Students for a Changing World (Solution Tree Press, 2021) and in a previous Solution Tree blog post: Using a Four-Phase Instructional Model to Plan and Teach for Lifelong Learning .

Teachers who provide a structure for research-based learning as part of their regular teaching routine should experience greater interest and involvement on the part of their students, and help students develop both skills and a fundamental knowledge base that are important for a lifetime of learning.

Zorfass, Judith and Copel. Harriet. The I-Search: Guiding Students Toward Relevant Research. In Educational Leadership, Volume 53, Number 1, September 1995, pp. 48-51.

Zorfass, Judith and Copel, Harriet (1998) Teaching Middle School Students to be Active Researchers. Alexandria, VA: Association for Supervision and Curriculum Development.

Solution Tree

Here's some awesome bio info about me! Short codes are not allowed, but perhaps we can work something else out.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Welcome to a collaborative resource that offers practical guidance and inspiring stories from authors and expert practitioners.

Subscribe to Our Blog

3 Key Learning Outcomes and How to Question Them

Getting to Know Differentiation in the Classroom

3 Books to Fire Up Your Imagination This Summer

How to Foster Civil Mindsets in Classroom Communities

We are currently working through process changes as part of the new checkout experience. If you need any support accessing your account, placing an order, signing up for a new account or have any other questions, please reach out to our  Customer Experience Team . We value your business and appreciate your patience.

Gale makes it easier for you to help learners thrive.

Learn more >>  

The Latest at Gale

Explore Our Products

Support your researchers with up-to-date, full-text magazines, newspapers, journals, periodicals, videos, and podcasts.

Enhance your institution’s ability to do deep research with rare primary source materials that are now at your fingertips digitally.

The Gale eBook platform brings our extensive eBook collection to libraries—from foundational classics to new topical titles.

Bestselling titles and authors in large print, across fiction (like mystery and westerns) and nonfiction (like history and lifestyle).

Learning solutions that support or provide full class curriculums and help to teach a wide range of disciplines.

Explore over 5,000 reference, nonfiction, and fiction titles across multiple formats, including large print.

Are you a student or a researcher?

Gale offers thousands of academic tools like databases, eBooks, primary source content, eLearning solutions, and more. See what's available at your library!

Access Through Your Library >>  

The Gale Blog

Get ready for pride month: explore lgbtq history.

From the Stonewall Riots to contemporary milestones, discover the struggles and triumphs of the LGBTQ community. With resources including archival documents and scholarly articles, delve into diverse narratives shaping LGBTQ identity and rights.

Gale vs. Google: See Who Wins in Your Classroom

Evaluate the ease of use, reliability, and quality of content between Google and Gale In Context , aiming to instill critical thinking and research skills while promoting the use of academic databases for school assignments.

New ReadSpeaker Enhancements for Improved Accessibility

With improved features, including customizable reading options and multilingual support, the ReadSpeaker tool offers a more inclusive reading experience. Whether accessing educational resources or research materials, users can now navigate content more seamlessly, fostering greater engagement and understanding for all.

• Advanced Search

Research on Learning Resource Recommendation Based on Learner Model

Nantong health college of Jiangsu province, China

Nantong Vocational University, China

New Citation Alert added!

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

• Publisher Site

ICETM '22: Proceedings of the 2022 5th International Conference on Education Technology Management

In the era of education big data, personalized learning has become the new normal of digital learning. As an important application direction of personalized learning system, learning resource recommendation is used to solve the problems of "information overload" and "information maze" caused by massive learning resources. This paper first constructs learner profile data based on learners' learning behavior, and uses GA-K-means algorithm to cluster learners according to the characteristic data of learner model, which effectively solves the cold start problem caused by untimely resource scoring. Finally, a learning resource recommendation method is designed from the three dimensions of consolidation, promotion and expansion, and N resources with the highest degree of fit are recommended to learners. The experimental results show that GA-K-means algorithm is significantly better than the traditional K-means clustering algorithm in stability and effectiveness, and the classification of learner groups is also in line with the actual situation, which can recommend personalized learning resources that meet the cognitive level for students.

Index Terms

Applied computing

Recommendations

Research of personalized learning resource recommendation based on learner's fdi.

The personalized learning resource recommendation, as one higher level of the requirements for the learning based on resources of people in the Internet age, has been paid more attention by researchers. Some existing researches can provide personalized ...

Recommendation of Learning Resources and Users Using an Aggregation-Based Approach

The present paper proposes our recommendation approach for the actors of e-learning. It is based on the collaborative filtering approach and some characteristics of e-learning, namely: the roles and interests of actors as well as the representation of ...

Sharing learner information through a web services-based learning architecture

This paper introduces the architecture developed for the exchange of learners model information among e-learning systems in the AdaptWeb Project. This Web-learning environment offers an adaptive content associated with a particular student's profile. ...

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

• Information
• Contributors

Published in

Copyright © 2022 ACM

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected] .

In-Cooperation

Association for Computing Machinery

New York, NY, United States

Publication History

• Published: 5 May 2023

Permissions

Request permissions about this article.

Check for updates

Author tags.

• learner model
• learning resource
• recommendation algorithm
• research-article
• Refereed limited

Funding Sources

Other metrics.

• Bibliometrics
• Citations 0

Article Metrics

• 0 Total Citations View Citations
• 44 Total Downloads
• Downloads (Last 12 months) 42
• Downloads (Last 6 weeks) 2

This publication has not been cited yet

View or Download as a PDF file.

View online with eReader.

Digital Edition

View this article in digital edition.

HTML Format

View this article in HTML Format .

Share this Publication link

https://dl.acm.org/doi/10.1145/3582580.3582589

Share on Social Media

• 0 References

Export Citations

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

• U.S. Locations
• UMGC Europe
• Learn Online
• Find Answers
• 855-655-8682
• Current Students

Online Guide to Writing and Research

The research process, explore more of umgc.

• Online Guide to Writing

Research Resources

What are research resources.

Research resources are usually thought of as primary sources and secondary sources. Click on the tabs to learn more about both.

Primary sources  can be firsthand accounts of actual events written by an eyewitness or original literary or artistic works. They may be letters, official records, interviews, survey results, or unanalyzed statistical data. These sources contain raw data and information, such as the original work of art or immediate impressions. 

Secondary sources , on the other hand, are usually discussions, evaluations, syntheses, and analyses of primary and secondary source information.  

You will no doubt use both primary and secondary sources throughout your academic career. When you use them, and in what combination, usually depends on what you are researching and the discipline for which you are writing. If you are unclear about which sources to use, ask your professor for guidance.

Types of Research

Your research question and the kind of research you do will guide the types of resources you will need to complete your research. Students’ access to information is greater than ever before. To be a good researcher, you must be able to locate, organize, evaluate, and communicate information.

Common Places to Find Research

Research resources are found in various places, both within and outside the traditional library. Your research resources can come from your personal experiences; print media such as books, brochures, journals, magazines, and newspapers; and electronic sources found on the Internet. They may also come from interviews and surveys you or someone else conduct. 

Your Library

Libraries are a main resource for conducting academic research. Learning how to use them and their resources effectively is important to understanding the research process.  Libraries provide access to information through online research databases and library catalogs, ebooks and ejournals, and Internet resources, as well as traditional print resources. Understanding how to select and use the appropriate resources for specific information needs is the key to successful research. To become adept at locating and using information for research, you must know about the many different resources that are available to you.

The following links provide information about the resources available to you as a UMGC student through the UMGC library:

  About the Library

  Ask a Librarian

  Library Resources

Key Takeaways

Primary sources include firsthand accounts, raw data, and other original material.

Secondary sources include material that interprets and analyzes primary sources.

Mailing Address: 3501 University Blvd. East, Adelphi, MD 20783 This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . © 2022 UMGC. All links to external sites were verified at the time of publication. UMGC is not responsible for the validity or integrity of information located at external sites.

Table of Contents: Online Guide to Writing

Chapter 1: College Writing

How Does College Writing Differ from Workplace Writing?

What Is College Writing?

Why So Much Emphasis on Writing?

Chapter 2: The Writing Process

Doing Exploratory Research

Getting from Notes to Your Draft

Introduction

Prewriting - Techniques to Get Started - Mining Your Intuition

Prewriting: Targeting Your Audience

Prewriting: Techniques to Get Started

Prewriting: Understanding Your Assignment

Rewriting: Being Your Own Critic

Rewriting: Creating a Revision Strategy

Rewriting: Getting Feedback

Rewriting: The Final Draft

Techniques to Get Started - Outlining

Techniques to Get Started - Using Systematic Techniques

Thesis Statement and Controlling Idea

Writing: Getting from Notes to Your Draft - Freewriting

Writing: Getting from Notes to Your Draft - Summarizing Your Ideas

Writing: Outlining What You Will Write

Chapter 3: Thinking Strategies

A Word About Style, Voice, and Tone

A Word About Style, Voice, and Tone: Style Through Vocabulary and Diction

Critical Strategies and Writing

Critical Strategies and Writing: Analysis

Critical Strategies and Writing: Evaluation

Critical Strategies and Writing: Persuasion

Critical Strategies and Writing: Synthesis

Developing a Paper Using Strategies

Kinds of Assignments You Will Write

Patterns for Presenting Information

Patterns for Presenting Information: Critiques

Patterns for Presenting Information: Discussing Raw Data

Patterns for Presenting Information: General-to-Specific Pattern

Patterns for Presenting Information: Problem-Cause-Solution Pattern

Patterns for Presenting Information: Specific-to-General Pattern

Patterns for Presenting Information: Summaries and Abstracts

Supporting with Research and Examples

Writing Essay Examinations

Writing Essay Examinations: Make Your Answer Relevant and Complete

Writing Essay Examinations: Organize Thinking Before Writing

Writing Essay Examinations: Read and Understand the Question

Chapter 4: The Research Process

Planning and Writing a Research Paper

Planning and Writing a Research Paper: Ask a Research Question

Planning and Writing a Research Paper: Cite Sources

Planning and Writing a Research Paper: Collect Evidence

Planning and Writing a Research Paper: Decide Your Point of View, or Role, for Your Research

Planning and Writing a Research Paper: Draw Conclusions

Planning and Writing a Research Paper: Find a Topic and Get an Overview

Planning and Writing a Research Paper: Manage Your Resources

Planning and Writing a Research Paper: Outline

Planning and Writing a Research Paper: Survey the Literature

Planning and Writing a Research Paper: Work Your Sources into Your Research Writing

Research Resources: Where Are Research Resources Found? - Human Resources

Research Resources: What Are Research Resources?

Research Resources: Where Are Research Resources Found?

Research Resources: Where Are Research Resources Found? - Electronic Resources

Research Resources: Where Are Research Resources Found? - Print Resources

Structuring the Research Paper: Formal Research Structure

Structuring the Research Paper: Informal Research Structure

The Nature of Research

The Research Assignment: How Should Research Sources Be Evaluated?

The Research Assignment: When Is Research Needed?

The Research Assignment: Why Perform Research?

Chapter 5: Academic Integrity

Academic Integrity

Giving Credit to Sources

Giving Credit to Sources: Copyright Laws

Giving Credit to Sources: Documentation

Giving Credit to Sources: Style Guides

Integrating Sources

Practicing Academic Integrity

Practicing Academic Integrity: Keeping Accurate Records

Practicing Academic Integrity: Managing Source Material

Practicing Academic Integrity: Managing Source Material - Paraphrasing Your Source

Practicing Academic Integrity: Managing Source Material - Quoting Your Source

Practicing Academic Integrity: Managing Source Material - Summarizing Your Sources

Types of Documentation

Types of Documentation: Bibliographies and Source Lists

Types of Documentation: Citing World Wide Web Sources

Types of Documentation: In-Text or Parenthetical Citations

Types of Documentation: In-Text or Parenthetical Citations - APA Style

Types of Documentation: In-Text or Parenthetical Citations - CSE/CBE Style

Types of Documentation: In-Text or Parenthetical Citations - Chicago Style

Types of Documentation: In-Text or Parenthetical Citations - MLA Style

Types of Documentation: Note Citations

Chapter 6: Using Library Resources

Finding Library Resources

Chapter 7: Assessing Your Writing

How Is Writing Graded?

How Is Writing Graded?: A General Assessment Tool

The Draft Stage

The Draft Stage: The First Draft

The Draft Stage: The Revision Process and the Final Draft

The Draft Stage: Using Feedback

The Research Stage

Using Assessment to Improve Your Writing

Chapter 8: Other Frequently Assigned Papers

Reviews and Reaction Papers: Article and Book Reviews

Reviews and Reaction Papers: Reaction Papers

Writing Arguments

Writing Arguments: Adapting the Argument Structure

Writing Arguments: Purposes of Argument

Writing Arguments: References to Consult for Writing Arguments

Writing Arguments: Steps to Writing an Argument - Anticipate Active Opposition

Writing Arguments: Steps to Writing an Argument - Determine Your Organization

Writing Arguments: Steps to Writing an Argument - Develop Your Argument

Writing Arguments: Steps to Writing an Argument - Introduce Your Argument

Writing Arguments: Steps to Writing an Argument - State Your Thesis or Proposition

Writing Arguments: Steps to Writing an Argument - Write Your Conclusion

Writing Arguments: Types of Argument

Appendix A: Books to Help Improve Your Writing

Dictionaries

General Style Manuals

Researching on the Internet

Special Style Manuals

Writing Handbooks

Appendix B: Collaborative Writing and Peer Reviewing

Collaborative Writing: Assignments to Accompany the Group Project

Collaborative Writing: Informal Progress Report

Collaborative Writing: Issues to Resolve

Collaborative Writing: Methodology

Collaborative Writing: Peer Evaluation

Collaborative Writing: Tasks of Collaborative Writing Group Members

Collaborative Writing: Writing Plan

General Introduction

Peer Reviewing

Appendix C: Developing an Improvement Plan

Working with Your Instructor’s Comments and Grades

Appendix D: Writing Plan and Project Schedule

Devising a Writing Project Plan and Schedule

Reviewing Your Plan with Others

By using our website you agree to our use of cookies. Learn more about how we use cookies by reading our  Privacy Policy .

Science of Reading: From Research to Practice for All Teachers

We all want what’s best for students with respect to reading instruction. Wanting what’s best for students is the easy part. Knowing what’s best and how to effectively orchestrate that in the classroom is considerably more complicated.

There’s a lot of information available about the science of reading, but connecting to the research is one of the most effective ways to guarantee you’re bringing the right ideas into the classroom.

To ensure you’re getting the most accurate information, steeped in evidence, Lexia® has collaborated with Dr. Dana Robertson to gain his perspective about the research surrounding science of reading instruction in classrooms.

In this Q&A, we delve into the literature and illuminate the critical elements science of reading experts have identified for fostering effective literacy learning.

What does it look like to have effectively implemented the science of reading in a classroom?

To effectively implement the science of reading in a way that maximizes instructional time, schools need a rigorous and coherent curriculum 1 within and across grades that systematically ensures foundational skills are sufficiently developed. This curriculum should consistently prioritize :

• Higher-order thinking and meaning-focused work
• Integration of reading into disciplinary areas to build content and conceptual knowledge
• Many opportunities for discussing content and ideas, and for reading and writing connected text (extended texts such as stories, informational texts, websites, etc.)

Rapid and accurate decoding is one means to this end, but an over-emphasis on code-focused skills is not what the science of reading instruction tells us.

This curriculum—which is not necessarily a published program—needs to progress incrementally to ensure each grade level builds on and extends the work of a previous grade rather than repeating lessons or jumping too far ahead. This requires educators to collaborate on careful vertical  (from grade to grade) and horizontal (within grades) alignment of the curriculum (which is not the same as standards) that also allows for differentiation of instruction responsive to the student’s needs.

For example, do teachers within a school have a coherent vision of what it means to develop argumentative skills—a critical aspect of at least Western schooling—such that they can identify how students would develop those skills, as readers and writers, incrementally from kindergarten onward?

What are some practical strategies teachers can use to support literacy learning for all students, regardless of subject area?

When considering what can be done across subject areas, we know these practices matter:

Developing enough proficiency in single-syllable and multisyllabic decoding to be able to read widely matters.

• Engage in word study that integrates phonics, phonological awareness, spelling, and morphology together instead of separate lessons on these skills. Research has not definitely identified a single best approach (synthetic vs. analytic vs. analogy).

The volume of wide reading practice matters.

• Provide ample opportunity for students to practice their word reading skills across a range of text types and develop fluency instead of a reading diet of only “decodable” texts that are tightly controlled to match phonics lessons. Students need highly decodable texts for a window of time while consolidating word reading abilities, but at the same time , they need broader text access.

Giving students tools to access text but maintaining the overall focus on topic knowledge matters.

• Teach students to strategically read texts while they discuss and reason about the text’s content instead of spending a lot of time teaching individual comprehension strategies.

Providing many opportunities for productive use of new words matters.

• Build vocabulary knowledge while students are reading the text, and then continually prompt students to use these new words in their speaking and writing instead of only pre-teaching vocabulary and having students define the words. Definitional understanding of words is important, but vocabulary instruction involves so much more.

Helping students grapple with more sophisticated and complex language structures and text matters.

• Teach sentence comprehension and composing to develop students’ abilities to understand the cohesion of ideas while also having students read and write connected texts. This sentence-level work scaffolds text reading.

Teaching students to write well and see the connections to reading and knowledge development matters.

• Finally, teach students to write for varied purposes and in varied forms (i.e., graphic organizers and annotations, summaries, extended writing such as exposition, poetry, stories, and arguments) while also developing word reading and comprehension.

What are some common challenges teachers face when trying to apply science of reading principles in their classrooms, and how can they overcome them?

Beyond the sheer amount of information available (some useful and some not) to teachers, time is always a challenge. Efficient yet effective instructional pacing is always a balancing act between the curricular demands and the needs of students. One of the main priorities should be to maximize instructional time.

Some districts may also be asking teachers to implement a particular curricular program with fidelity, yet sometimes “fidelity” prompts a narrowed focus on what instruction can and should look like. Instead, I would look for teachers to implement instructional materials with integrity by adapting lessons to address high-leverage instructional practices, yet also be responsive to the needs of students. Schoolwide curricular materials provide a great way to establish a consistent baseline of instruction and Scope and Sequence across grades, but they often contain too many lesson components to be feasibly done well during instructional time and many fall short of accounting for the linguistic, social, and cultural diversities students bring to the classroom.

Other challenges stem from what it takes for teachers and administration to come together as a school and work toward the collective efficacy of literacy instruction. Oftentimes, implementation of evidence-based reading instruction is challenging because of various teacher, leadership, and schoolwide structures that are not aligned. This is not easy to overcome, but it is essential for school personnel to collaboratively align the school’s infrastructure on:

• The nature and enactment of curriculum.
• The professional learning (with ongoing coaching support and teacher collaboration) needed to realize that enactment.
• Leadership that propels and sustains what it takes to enact schoolwide evidence-based instruction. 2

What determines an effective literacy professional learning experience for teachers?

Meeting the challenge of implementing high-quality reading instruction and meaningfully improving students’ literacy learning requires teachers’ professional learning opportunities that include:

• Ongoing inquiry-focused approach that promotes meaningful engagement with content and colleagues.
• A focus on knowledge creation rather than transmission of the prescribed curriculum.
• Networks and partnerships of both inside teacher expertise and outside expertise.
• A focus on specific vexing problems teachers encounter with their students related to the curriculum that also values teachers’ experiences and knowledge.

It’s not a policy or program or test that matters in sustainably improving students’ reading lives. As Lieberman and Miller 3 note, teachers get better at their craft by engaging in ongoing professional learning that is inclusive, broad-based, and grounded in the day-to-day realities of their jobs.

What are important considerations for schools and districts when providing ongoing support and professional development for teachers in literacy education? What should teachers be petitioning for?

First, it is important to consider these teacher factors 4 :

• Have teachers (whether in-service or pre-service 5 ) had the professional learning opportunities to understand and enact the instructional practice? Just like students, teachers bring different knowledge and experiences to their environment and these differences need to be accounted for when learning new practices.
• Do teachers see a contextual fit between the new practices and what they perceive as their student and curricular needs?
• Do teachers have a sense of agency in how evidence-based practices are applied in their classroom? Teachers are professionals who bring valued knowledge and experiences to conversations of curriculum and instruction.

Second, it is important to consider the nature of collaborative opportunities, time, and resources available 6 :

• Is there a growing professional capital among the people in the school?
• Are there increased opportunities for teachers to collaborate and reflect on instruction and student learning?
• Is there access to the resources, both material and human, teachers need to advance their understanding of evidence-based reading instruction?

Third, it is important to consider the schoolwide culture and structures so collaborations can occur in ways that improve teachers’ collective efficacy.

• Is there a balance between innovation and socialization? Schools need structures that provide ample opportunity for teachers and administrators to explore instructional practices, while also bringing teachers together with supportive socialization “pressure” to generate, share, and use those innovative practices.
• Is there a balance between cohesion and diversity of thought? Schools need structures where social interactions occur within and across teachers and administrators with a high level of cohesion, yet also with interactions that express diverse attitudes. Teachers are less likely to grow in an ongoing fashion when they all hold the same attitudes and share the same approaches to problems of practice.
• Is there a balance between divergence and convergence that characterizes teachers’ shared understandings? Schools need structures where teachers and administrators come together about goals and approaches while also having trust and space for asking critical questions and encouraging new ways of thinking. It is the merging of understanding that brings individuals into the collective.

If these are out of balance, professional learning outcomes may not be fully realized or sustained.

Moving forward, how can teachers ensure their literacy instruction is evidence-based and aligned with science of reading principles?

There is so much information available to teachers through curricular programs and resources, books and articles, websites, and social media. It can be dizzying to follow. Add to this the issue of access to research that is often published in journals behind paywalls and the sometimes incomprehensible nature of educational research filled with jargon.

First, consider: What does “evidence-based” mean, and for whom and in which contexts? This is where we need to remember the applied research across varied research methodologies mentioned previously. What is working in actual classrooms for whom and how? My goal is always to use the approaches that get students interested in and reading text as much as possible with increasing independence.

As a starting point, draw on resources that are vetted and freely accessible such as the What Works Clearinghouse (WWC) Practice Guides 7 published by the Institute of Education Sciences (IES). Since they only report on experimental and quasiexperimental research available to date when the reports are published, they are not perfect (no research or program provides a silver bullet); yet they can provide a baseline for evaluating instructional materials and programs and understanding instructional approaches that have been validated in at least some school-based research contexts. The site even provides a rubric and describes a process school divisions can use to evaluate their instructional materials 8 .

Second, use resources such as those provided by IES to be a critical consumer of programs and resources. Focus on the aspects of instruction that are highly leveraged for making a difference in students’ reading lives and make adaptations in response to students’ strengths and needs. Remember also the experiential knowledge of teachers critically matters when determining “what works” for groups of students, and research continually points to the importance of the teacher when considering the effectiveness of instructional programs and approaches. While all educators should strive to be critical consumers, establishing the “baseline” curricular approaches is likely best suited for a school English language arts leadership team composed of grade-level or content-area teacher representatives and specialists.

Third, establish and continually refine the school's rigorous and coherent curriculum. In doing so, engage in ongoing professional inquiry as a network of professionals—including those in the school and outside collaborators—pool their collective knowledge to discuss and refine their understanding of what is working and what might need to be enhanced or refined.

• Au, K. H., & Raphael, T. E. (2021). What matters. Reading Research Quarterly , 56, S65-S67. https://doi.org/10.1002/rrq.403
• Woulfin, S., & Gabriel, R. E. (2020). Interconnected infrastructure for improving reading instruction. Reading Research Quarterly , 55, S109-S117. https://doi.org/10.1002/rrq.339
• Lieberman, A., & Miller, L. (2014). Teachers as professionals: Evolving definitions of staff development. In L. E. Martin, S. Kragler, D. J. Quatroche, & K. L Bauserman (Eds.), Handbook of professional development in education: Successful models and practices, PreK-12 (pp. 3-21). Guilford.
• McChesney, K., & Aldridge, J. M. (2021). What gets in the way? A new conceptual model for the trajectory from teacher professional development to impact. Professional Development in Education , 47, 834-852. https://doi.org/10.1080/19415257.2019.1667412
• Hindman, A. H., Morrison, F. J., Connor, C. M., & Connor, J. A. (2020). Bringing the science of reading to preservice elementary teachers: Tools that bridge research and practice. Reading Research Quarterly , 55, S197-S206. https://www.proquest.com/scholarly-journals/bringing-science-reading-preservice-elementary/docview/2625004130/se-2
• Cirkony, C., Rickinson, M., Walsh, L., Gleeson, J., Salisbury, M., Cutler, B., Berry, M., & Smith, K. (2024). Beyond Effective Approaches: A Rapid Review Response to Designing Professional Learning. Professional Development in Education , 50(1), 24-45. https://doi.org/10.1080/19415257.2021.1973075
• https://ies.ed.gov/ncee/wwc/
• https://ies.ed.gov/ncee/rel/Products/Region/southeast/Publication/3814

You Might Also Like

Why Every Educator Needs to Understand the Science of Reading

Why is the science of reading so important? Because it gives teachers the background knowledge they need to help their students successfully achieve literacy. Read on to learn about the impact the science of reading can have on your students.

Evidence-Based Science of Reading Strategies for Instruction

It is critical teachers and students have the tools to thrive when learning how to read. That’s why Lexia® has put together this comprehensive guide for the evidence behind teaching reading. Join us in exploring these effective science of reading strategies.

Science of Reading: Summer Reading for School Administrators

Lexia’s Science of Reading Week 2024 offered valuable insight from education experts about the transformational power of science-based literacy instruction. Check out this curated list of recommended resources to help drive effectiveness and engagement in your literacy program.

A Cambium Learning® Group Brand