higher education case study technology

How technology is shaping learning in higher education

About the authors.

This article is a collaborative effort by Claudio Brasca, Charag Krishnan , Varun Marya , Katie Owen, Joshua Sirois, and Shyla Ziade, representing views from McKinsey’s Education Practice.

The COVID-19 pandemic forced a shift to remote learning overnight for most higher-education students, starting in the spring of 2020. To complement video lectures and engage students in the virtual classroom, educators adopted technologies that enabled more interactivity and hybrid models of online and in-person activities. These tools changed learning, teaching, and assessment in ways that may persist after the pandemic. Investors have taken note. Edtech start-ups raised record amounts of venture capital in 2020 and 2021, and market valuations for bigger players soared.

A study conducted by McKinsey in 2021 found that to engage most effectively with students, higher-education institutions can focus on eight dimensions  of the learning experience. In this article, we describe the findings of a study of the learning technologies that can enable aspects of several of those eight dimensions (see sidebar “Eight dimensions of the online learning experience”).

Eight dimensions of the online learning experience

Leading online higher-education institutions focus on eight key dimensions of the learning experience across three overarching principles.

Seamless journey

Clear education road map: “My online program provides a road map to achieve my life goals and helps me structure my day to day to achieve steady progress.”

Seamless connections: “I have one-click access to classes and learning resources in the virtual learning platform through my laptop or my phone.”

Engaging teaching approach

Range of learning formats: “My program offers a menu of engaging courses with both self-guided and real-time classes, and lots of interaction with instructors and peers.”

Captivating experiences: “I learn from the best professors and experts. My classes are high quality, with up-to-date content.”

Adaptive learning: “I access a personalized platform that helps me practice exercises and exams and gives immediate feedback without having to wait for the course teacher.”

Real-world skills application: “My online program helps me get hands-on practice using exciting virtual tools to solve real-world problems.”

Caring network

Timely support: “I am not alone in my learning journey and have adequate 24/7 support for academic and nonacademic issues.”

Strong community: “I feel part of an academic community and I’m able to make friends online.”

In November 2021, McKinsey surveyed 600 faculty members and 800 students from public and private nonprofit colleges and universities in the United States, including minority-serving institutions, about the use and impact of eight different classroom learning technologies (Exhibit 1). (For more on the learning technologies analyzed in this research, see sidebar “Descriptions of the eight learning technologies.”) To supplement the survey, we interviewed industry experts and higher-education professionals who make decisions about classroom technology use. We discovered which learning tools and approaches have seen the highest uptake, how students and educators view them, the barriers to higher adoption, how institutions have successfully adopted innovative technologies, and the notable impacts on learning (for details about our methodology, see sidebar “About the research”).

Double-digit growth in adoption and positive perceptions

Descriptions of the eight learning technologies.

• Classroom interactions: These are software platforms that allow students to ask questions, make comments, respond to polls, and attend breakout discussions in real time, among other features. They are downloadable and accessible from phones, computers, and tablets, relevant to all subject areas, and useful for remote and in-person learning.
• Classroom exercises: These platforms gamify learning with fun, low-stakes competitions, pose problems to solve during online classes, allow students to challenge peers to quizzes, and promote engagement with badges and awards. They are relevant to all subject areas.
• Connectivity and community building: A broad range of informal, opt-in tools, these allow students to engage with one another and instructors and participate in the learning community. They also include apps that give students 24/7 asynchronous access to lectures, expanded course materials, and notes with enhanced search and retrieval functionality.
• Group work: These tools let students collaborate in and out of class via breakout/study rooms, group preparation for exams and quizzes, and streamlined file sharing.
• Augmented reality/virtual reality (AR/VR): Interactive simulations immerse learners in course content, such as advanced lab simulations for hard sciences, medical simulations for nursing, and virtual exhibit tours for the liberal arts. AR can be offered with proprietary software on most mobile or laptop devices. VR requires special headsets, proprietary software, and adequate classroom space for simultaneous use.
• AI adaptive course delivery: Cloud-based, AI-powered software adapts course content to a student’s knowledge level and abilities. These are fully customizable by instructors and available in many subject areas, including business, humanities, and sciences.
• Machine learning–powered teaching assistants: Also known as chatbot programs, machine learning–powered teaching assistants answer student questions and explain course content outside of class. These can auto-create, deliver, and grade assignments and exams, saving instructors’ time; they are downloadable from mobile app stores and can be accessed on personal devices.
• Student progress monitoring: These tools let instructors monitor academic progress, content mastery, and engagement. Custom alerts and reports identify at-risk learners and help instructors tailor the content or their teaching style for greater effectiveness. This capability is often included with subscriptions to adaptive learning platforms.

Survey respondents reported a 19 percent average increase in overall use of these learning technologies since the start of the COVID-19 pandemic. Technologies that enable connectivity and community building, such as social media–inspired discussion platforms and virtual study groups, saw the biggest uptick in use—49 percent—followed by group work tools, which grew by 29 percent (Exhibit 2). These technologies likely fill the void left by the lack of in-person experiences more effectively than individual-focused learning tools such as augmented reality and virtual reality (AR/VR). Classroom interaction technologies such as real-time chatting, polling, and breakout room discussions were the most widely used tools before the pandemic and remain so; 67 percent of survey respondents said they currently use these tools in the classroom.

About the research

In November 2021, McKinsey surveyed 634 faculty members and 818 students from public, private, and minority-serving colleges and universities over a ten-day period. The survey included only students and faculty who had some remote- or online-learning experience with any of the eight featured technologies. Respondents were 63 percent female, 35 percent male, and 2 percent other gender identities; 69 percent White, 18 percent Black or African American, 8 percent Asian, and 4 percent other ethnicities; and represented every US region. The survey asked respondents about their:

• experiences with technology in the classroom pre-COVID-19;
• experiences with technology in the classroom since the start of the COVID-19 pandemic; and
• desire for future learning experiences in relation to technology.

The shift to more interactive and diverse learning models will likely continue. One industry expert told us, “The pandemic pushed the need for a new learning experience online. It recentered institutions to think about how they’ll teach moving forward and has brought synchronous and hybrid learning into focus.” Consequently, many US colleges and universities are actively investing to scale up their online and hybrid program offerings .

Differences in adoption by type of institution observed in the research

• Historically Black colleges and universities (HBCUs) and tribal colleges and universities made the most use of classroom interactions and group work tools (55 percent) and the least use of tools for monitoring student progress (15 percent).
• Private institutions used classroom interaction technologies (84 percent) more than public institutions (63 percent).
• Public institutions, often associated with larger student populations and course sizes, employed group work and connectivity and community-building tools more often than private institutions.
• The use of AI teaching-assistant technologies increased significantly more at public institutions (30 percent) than at private institutions (9 percent), though overall usage remained comparatively higher at private institutions.
• The use of tools for monitoring student progress increased by 14 percent at private institutions, versus no growth at public institutions.

Some technologies lag behind in adoption. Tools enabling student progress monitoring, AR/VR, machine learning–powered teaching assistants (TAs), AI adaptive course delivery, and classroom exercises are currently used by less than half of survey respondents. Anecdotal evidence suggests that technologies such as AR/VR require a substantial investment in equipment and may be difficult to use at scale in classes with high enrollment. Our survey also revealed utilization disparities based on size. Small public institutions use machine learning–powered TAs, AR/VR, and technologies for monitoring student progress at double or more the rates of medium and large public institutions, perhaps because smaller, specialized schools can make more targeted and cost-effective investments. We also found that medium and large public institutions made greater use of connectivity and community-building tools than small public institutions (57 to 59 percent compared with 45 percent, respectively). Although the uptake of AI-powered tools was slower, higher-education experts we interviewed predict their use will increase; they allow faculty to tailor courses to each student’s progress, reduce their workload, and improve student engagement at scale (see sidebar “Differences in adoption by type of institution observed in the research”).

While many colleges and universities are interested in using more technologies to support student learning, the top three barriers indicated are lack of awareness, inadequate deployment capabilities, and cost (Exhibit 3).

Students want entertaining and efficient tools

More than 60 percent of students said that all the classroom learning technologies they’ve used since COVID-19 began had improved their learning and grades (Exhibit 4). However, two technologies earned higher marks than the rest for boosting academic performance: 80 percent of students cited classroom exercises, and 71 percent cited machine learning–powered teaching assistants.

Although AR/VR is not yet widely used, 37 percent of students said they are “most excited” about its potential in the classroom. While 88 percent of students believe AR/VR will make learning more entertaining, just 5 percent said they think it will improve their ability to learn or master content (Exhibit 5). Industry experts confirmed that while there is significant enthusiasm for AR/VR, its ability to improve learning outcomes is uncertain. Some data look promising. For example, in a recent pilot study, 1 “Immersive biology in the Alien Zoo: A Dreamscape Learn software product,” Dreamscape Learn, accessed October 2021. students who used a VR tool to complete coursework for an introductory biology class improved their subject mastery by an average of two letter grades.

Faculty embrace new tools but would benefit from more technical support and training

Faculty gave learning tools even higher marks than students did, for ease of use, engagement, access to course resources, and instructor connectivity. They also expressed greater excitement than students did for the future use of technologies. For example, while more than 30 percent of students expressed excitement for AR/VR and classroom interactions, more than 60 percent of faculty were excited about those, as well as machine learning–powered teaching assistants and AI adaptive technology.

Eighty-one percent or more of faculty said they feel the eight learning technology tools are a good investment of time and effort relative to the value they provide (Exhibit 6). Expert interviews suggest that employing learning technologies can be a strain on faculty members, but those we surveyed said this strain is worthwhile.

While faculty surveyed were enthusiastic about new technologies, experts we interviewed stressed some underlying challenges. For example, digital-literacy gaps have been more pronounced since the pandemic because it forced the near-universal adoption of some technology solutions, deepening a divide that was unnoticed when adoption was sporadic. More tech-savvy instructors are comfortable with interaction-engagement-focused solutions, while staff who are less familiar with these tools prefer content display and delivery-focused technologies.

According to experts we interviewed, learning new tools and features can bring on general fatigue. An associate vice president of e-learning at one university told us that faculty there found designing and executing a pilot study of VR for a computer science class difficult. “It’s a completely new way of instruction. . . . I imagine that the faculty using it now will not use it again in the spring.” Technical support and training help. A chief academic officer of e-learning who oversaw the introduction of virtual simulations for nursing and radiography students said that faculty holdouts were permitted to opt out but not to delay the program. “We structured it in a ‘we’re doing this together’ way. People who didn’t want to do it left, but we got a lot of support from vendors and training, which made it easy to implement simulations.”

Reimagining higher education in the United States

Takeaways from our research.

Despite the growing pains of digitizing the classroom learning experience, faculty and students believe there is a lot more they can gain. Faculty members are optimistic about the benefits, and students expect learning to stay entertaining and efficient. While adoption levels saw double-digit growth during the pandemic, many classrooms have yet to experience all the technologies. For institutions considering the investment, or those that have already started, there are several takeaways to keep in mind.

• It’s important for administration leaders, IT, and faculty to agree on what they want to accomplish by using a particular learning technology. Case studies and expert interviews suggest institutions that seek alignment from all their stakeholders before implementing new technologies are more successful. Is the primary objective student engagement and motivation? Better academic performance? Faculty satisfaction and retention? Once objectives are set, IT staff and faculty can collaborate more effectively in choosing the best technology and initiating programs.
• Factor in student access to technology before deployment. As education technology use grows, the digital divide for students puts access to education at risk. While all the institution types we surveyed use learning technologies in the classroom, they do so to varying degrees. For example, 55 percent of respondents from historically Black colleges and universities and tribal colleges and universities use classroom interaction tools. This is lower than public institutions’ overall utilization rate of 64 percent and private institutions’ utilization rate of 84 percent. Similarly, 15 percent of respondents from historically Black colleges and universities and tribal colleges and universities use tools for monitoring student progress, while the overall utilization rate for both public and private institutions is 25 percent.
• High-quality support eases adoption for students and faculty. Institutions that have successfully deployed new learning technologies provided technical support and training for students and guidance for faculty on how to adapt their course content and delivery. For example, institutions could include self-service resources, standardize tools for adoption, or provide stipend opportunities for faculty who attend technical training courses. One chief academic officer told us, “The adoption of platforms at the individual faculty level can be very difficult. Ease of use is still very dependent upon your IT support representative and how they will go to bat to support you.”
• Agree on impact metrics and start measuring in advance of deployment. Higher-education institutions often don’t have the means to measure the impact of their investment in learning technologies, yet it’s essential for maximizing returns. Attributing student outcomes to a specific technology can be complex due to the number of variables involved in academic performance. However, prior to investing in learning technologies, the institution and its faculty members can align on a core set of metrics to quantify and measure their impact. One approach is to measure a broad set of success indicators, such as tool usage, user satisfaction, letter grades, and DFW rates (the percentage of students who receive a D, F, or Withdraw) each term. The success indicators can then be correlated by modality—online versus hybrid versus in-class—to determine the impact of specific tools. Some universities have offered faculty grants of up to $20,000 for running pilot programs that assess whether tools are achieving high-priority objectives. “If implemented properly, at the right place, and with the right buy-in, education technology solutions are absolutely valuable and have a clear ROI,” a senior vice president of academic affairs and chief technology officer told us.

In an earlier article , we looked at the broader changes in higher education that have been prompted by the pandemic. But perhaps none has advanced as quickly as the adoption of digital learning tools. Faculty and students see substantial benefits, and adoption rates are a long way from saturation, so we can expect uptake to continue. Institutions that want to know how they stand in learning tech adoption can measure their rates and benchmark them against the averages in this article and use those comparisons to help them decide where they want to catch up or get ahead.

Claudio Brasca is a partner in McKinsey’s Bay Area office, where Varun Marya is a senior partner; Charag Krishnan is a partner in the New Jersey office; Katie Owen is an associate partner in the St. Louis office, where Joshua Sirois is a consultant; and Shyla Ziade is a consultant in the Denver office.

The authors wish to thank Paul Kim, chief technology officer and associate dean at Stanford School of Education, and Ryan Golden for their contributions to this article.

Explore a career with us

Related articles.

Setting a new bar for online higher education

How to transform higher-education institutions for the long term

Scaling online education: Five lessons for colleges

Digital Transformation in Higher Education: 7 Areas for Enhancing Digital Learning

This article reflects on current practices and directions for digital transformation through a framework that supports the strategic responses and structural changes that higher education institutions could implement to enhance digital teaching and learning.

Higher education is in the era of digital transformation (Dx). Learning technologies and digital platforms are no longer an afterthought; they are critical for teaching and learning. The COVID-19 pandemic served as a catalyst for Dx, forcing colleges, universities, instructors, and students to shift online rapidly. Some instructors and students were prepared for the shift; those who were unprepared had to catch up quickly. Footnote 1 This article reflects on current practices and directions for Dx through a framework that supports the strategic responses and structural changes that higher education institutions could implement to enhance digital teaching and learning.

Defining Digital Transformation

As experts in learning design, instruction, and educational technology, we define Dx for digital learning in the higher education context as leveraging digital technologies to enable major educational improvements, enhance learner and instructor experiences, and create new instructional models through policies, planning, partnerships, and support . Our definition builds on existing research and Gregory Vial's 2019 definition of Dx. It also aligns with the EDUCAUSE definition of Dx: "a series of deep and coordinated culture, workforce, and technology shifts that enable new educational and operating models and transform an institution's operations, strategic directions, and value proposition." Footnote 2

Dx is driven by and built on digital technologies. It changes the educational landscape significantly. Keeping up with Dx helps higher education institutions operate effectively, stay competitive in an increasingly digital world, and prepare learners for the digital workplace.

Building a Dx Framework for Digital Learning in Higher Education

We have witnessed Dx in higher education institutions through our work as university professors and educational technology researchers. In this article, we propose a framework focused on integrating digital technologies to cause Dx in higher education settings. According to Vial, structural changes in four areas are critical for Dx: organizational structure, organizational culture, leadership, employee roles, and skills. Footnote 3 Our Dx framework for digital learning in higher education discusses seven aspects within each of those four areas: digital learning technologies, instructional modality, personnel and support services, organizational policies and planning, instructor development, learner development, and partnerships (see figure 1). Some colleges and universities might already be in the middle of Dx, and others might just be getting started.

• Learning management systems (LMS). An LMS is used to house all course materials, modules, and activities. The instructor can send announcements, engage in discussions, develop and grade assignments, and maintain an online grade book in the LMS.
• Synchronous technologies. Synchronous technologies are used to conduct real-time online meetings. Synchronous technologies include various functionalities, such as audio and video, text/chat, screen sharing, polls, whiteboards, and breakout rooms for small group discussions. These functionalities can help instructors maintain interactivity in online classrooms. Footnote 5
• Multimedia applications. Multimedia can engage learners and includes audio, video, and other interactive elements. Footnote 6 Multimedia software can be used to record microlectures, demonstrations, orientations, etc. Some multimedia software is open access. More robust applications must be purchased. Some multimedia applications can also be embedded within the LMS for easy access and use.
• Collaborative applications. Web-based or cloud-based word processing, presentation, social participation, and whiteboard applications allow students to collaborate online with their peers and instructors.
• Cloud-based technologies. Colleges and universities rely on various cloud-based applications. Some faculty use cloud-based applications to store files so they can access them from anywhere in the world and aren't restricted to their office computers.
• Emerging technologies. Artificial intelligence (AI), extended reality (XR), augmented reality (AR), virtual reality (VR), analytics, and other emerging technologies can enable more innovative and engaging teaching methods and learning experiences. Footnote 7

This is not an exhaustive list of the technologies that can be used for digital teaching and learning. Technology leaders need to evaluate the outcomes of each technology and consider its quality and cost before purchasing it for their campuses. Technology leaders should also examine their technology infrastructure to ensure it is adequate for digital teaching and learning. Footnote 8

• On-campus technology-enhanced. In this modality, teaching and learning occur in person, and technology is used to enhance instruction.
• Hybrid/blended. This modality blends in-person and online instruction to provide students with the flexibility of on-campus and online learning.
• Asynchronous online. In this modality, teaching and learning occur online without any real-time meetings.
• Synchronous online. In this modality, teaching and learning occur online in real time.
• Bichronous online. This modality blends asynchronous and synchronous online teaching and learning. Students participate in the asynchronous classes at the time and location of their choice, and they participate in the synchronous classes in real time. Footnote 9
• HyFlex. This modality offers the most flexibility. It combines in-person and online students in the same classroom. Footnote 10 HyFlex learning is similar to hybrid/blended learning, but it allows students to choose their modality based on their needs and daily circumstances.

All of these modalities have digital teaching and learning elements, though technology-enhanced on-campus courses have minimal technology integration. The other five modalities rely heavily on digital teaching and learning.

With more institutions and programs offering online courses, students have more options. Students can now choose to complete courses and programs at any time and from any place. Higher education leaders, instructors, and learners have tested the efficiency and effectiveness of digital learning. Today, more institutions are open to these models of teaching and learning, though they might still be emerging in some contexts.

• Instructional designers. Higher education institutions are hiring more instructional designers and technology specialists than in past years. Administrators and instructors have a better understanding and appreciation for instructional designers' expertise in digital learning design. Footnote 12 Instructional designers partner with instructors to design effective courses for various modalities.
• Technology support specialists. More staff are needed to maintain networks and technology if an institution increases its digital teaching and learning offerings. Though technology support is already available on most campuses, the increase in digital teaching and learning has resulted in a need for 24/7 technology support for students and instructors. Research indicates that faculty are interested in receiving multifaceted support, including one-to-one and just-in-time support. Footnote 13
• Academic and student support services. Academic support is needed so students can access library resources and writing centers. Student support services (registration, academic advising, study strategy consultations, etc.) are also needed for digital teaching and learning. Likewise, students with needs should have access to services that can assist them with digital learning.
• Incentives and recognition. Faculty need to be recognized and offered incentives and awards for being innovators in digital teaching. Footnote 14 Financial incentives and course release time give faculty opportunities and time to explore and integrate digital innovations into their courses.

Support, services, incentives, and recognition motivate instructors to adopt innovative digital teaching methods.

• Policies and standards. Institutional policies and standards need to be set up for digital teaching and learning. Administrators need to consider a range of policies, such as teaching load, enrollment criteria, and performance and evaluation standards. For example, new course evaluation instruments should be created or adapted to evaluate digital teaching.
• Strategic planning. Strategic planning is "the process of defining a strategy as well as deciding on the resources that are allocated to pursue a strategy in order to achieve firms' goals." Footnote 15 Administrators must integrate Dx into their strategic planning and get faculty buy-in.
• Funding models. Administrators should examine funding models for different modalities. Online courses provide opportunities for campuses to offer differential tuition rates since students do not have to be on campus or pay fees for campus-based resources.
• Equitable learning opportunities. Inequities in student access to technology were brought to light during the pandemic. Institutions should ensure that students have the hardware, software, and internet access they need to participate in online courses. Courses should also be accessible to students with cognitive and physical disabilities. Hence, policies and planning to reduce the digital divide are essential.

Overall, more policies are needed to support digital teaching and learning. Instructional leaders must also rethink any possible inequities related to digital teaching and learning, including funding, personnel, technology, and existing policies.

• Pedagogical and technological skills. Faculty should be given opportunities to develop their pedagogical and technological skills and learn how to integrate content. Faculty development professionals should continue to offer varied training on digital teaching and learning.
• Faculty beliefs. Faculty attitudes toward digital teaching and learning are evolving from reluctance to willingness. Professional development opportunities can support this evolution by focusing on teaching instructors how to establish positive value beliefs toward technology and digital teaching and how to align their teaching philosophies with digital teaching practices. Footnote 16
• Accessibility . Accessible courses can benefit students with physical and cognitive disabilities. Faculty must be prepared to make their digital courses accessible. This takes additional time and effort and requires backing from administrators, technology support (e.g., closed captioning services), and instructional design support. Footnote 17
• Intellectual property rights and copyright. During digital teaching, faculty need resources and support to become more familiar with intellectual property rights and the copyright of electronic materials.

When faculty switched to digital teaching and learning during the pandemic, many did not have adequate time to apply online teaching principles. Taking the time to rethink and apply pedagogical best practices will enhance the quality of online courses.

• Computers and internet access. Students should have access to computers and the internet to be successful digital learners. Though many students have access to these tools, a digital divide still exists. Instructors and administrators must consider students' digital access before they engage in digital teaching.
• Time management and self-regulation. Digital learning comes with flexibility. This flexibility, however, puts greater reliance on self-regulated learning. For example, students have to learn to manage their time well, reduce distractions, and avoid procrastination during digital learning. Footnote 18
• Instructional content and people. Students must be able to learn from a variety of content formats (text, audio, video) when instructors post lectures, podcasts, and discussions. They also must learn to engage with their instructor, peers, and content in a digital environment.
• Help. In a digital learning environment, students might be separated by distance, and they need to be able to reach out when they need help. Assistance could be provided by a technology help desk or an instructor.
• Community building. Students need opportunities to develop social communities and platforms for social interactions (e.g., online orientations, online social time for students to meet each other, etc.). Students will rely on their community to stay connected and engaged in digital learning. Footnote 19

Technology resources, time management and self-regulation, engagement and help-seeking strategies, and community building help digital learners succeed.

• Collaboration with other universities. Colleges and universities that already provided digital learning offered training and workshops to support instructors at other institutions. Expanding collaborations like these globally could strengthen digital teaching and learning worldwide.
• Collaboration with professional organizations. Professional organizations that are leaders in digital learning supported higher education institutions by offering training, workshops, and resources.
• Collaboration with industry. In some countries, industries outside of higher education partnered with institutions to provide access to the internet and electronic devices. Industry partnerships bring digital innovations to higher education institutions more quickly.

Partnerships with colleges and universities, professional organizations, and outside industries strengthen digital teaching and learning initiatives by capitalizing on the knowledge of experts in the space.

While our framework highlights seven distinct areas, achieving Dx for digital learning is an iterative process. As advanced digital technologies evolve, Dx initiatives will become commonplace for higher education institutions. Dx for digital learning brings flexibility and accessibility to students and prepares them to solve problems in the digital world. Dx efforts will continue to shape the norms and practices within higher education so that it adapts and evolves in parallel with society.

• Wahab Ali, "Online and Remote Learning in Higher Education Institutes: A Necessity in Light of COVID-19 Pandemic," Higher Education Studies 10, no. 3 (2020): 16–25; Florence Martin, Kui Xie, and Doris U. Bolliger, "Engaging Learners in the Emergency Transition to Online Learning during the COVID-19 Pandemic," supplement, Journal of Research on Technology in Education 54, S1 (2022): S1–S13; Ramona Maile Cutri, Juanjo Mena, and Erin Feinauer Whiting, "Faculty Readiness for Online Crisis Teaching: Transitioning to Online Teaching during the COVID-19 Pandemic," European Journal of Teacher Education 43, no. 4 (2020): 523–541. Jump back to footnote 1 in the text. ↩
• Gregory Vial defined digital transformation as a process in which "digital technologies create disruptions triggering strategic responses from organizations that seek to alter their value creation paths while managing the structural changes and organizational barriers that affect the positive and negative outcomes of this process." See Gregory Vial, "Understanding Digital Transformation: A Review and a Research Agenda," The Journal of Strategic Information Systems 28, no. 2 (June 2019): 118; Susan Grajek and Betsy Reinitz, "Getting Ready for Digital Transformation: Change Your Culture, Workforce, and Technology,"   EDUCAUSE Review, July 8, 2019. Jump back to footnote 2 in the text. ↩
• Ibid. Jump back to footnote 3 in the text. ↩
• Kui Xie and N. Hawk, "Technology's Role and Place in Student Learning: What We Have Learned from Research and Theories," in Technology in School Classrooms: How It Can Transform Teaching and Student Learning Today, eds. J.G., Cibulka, & B.S. Cooper (Lanham, MD: Rowman & Littlefield, 2017), 1–17. Jump back to footnote 4 in the text. ↩
• Alice Gruber and Elwira Bauer, "Fostering Interaction in Synchronous Online Class Sessions with Foreign Language Learners," in Teaching, Technology, and Teacher Education during the COVID-19 Pandemic: Stories from the Field, eds. R.E. Ferdig, E. Baumgartner, R. Hartshorne, R. Kaplan-Rakowski, and C. Mouza (Waynesville, NC: Association for the Advancement of Computing in Education, 2020), 175–178. Jump back to footnote 5 in the text. ↩
• Florence Martin and Anthony Karl Betrus, Digital Media for Learning , (Cham, Switzerland: Springer, 2019). Jump back to footnote 6 in the text. ↩
• Tanya Joosten, Kate Lee-McCarthy, Lindsey Harness, and Ryan Paulus, Digital Learning Innovation Trends, research report, (Boston, MA: Online Learning Consortium, February 2020). Jump back to footnote 7 in the text. ↩
• Christopher Hill and William Lawton, "Universities, the Digital Divide and Global Inequality," Journal of Higher Education Policy and Management 40, no. 6 (October 2018): 598–610; Kui Xie, Min Kyu Kim, Sheng-Lun Cheng, and Nicole C. Luthy, "Teacher Professional Development through Digital Content Evaluation," Educational Technology Research and Development 65, no. 4 (August 2017): 1067–1103; Kui Xie, Gennaro Di Tosto, Sheng-Bo Chen, and Vanessa W. Vongkulluksn, "A Systematic Review of Design and Technology Components of Educational Digital Resources," Computers & Education 127 (December 2018): 90–106. Jump back to footnote 8 in the text. ↩
• Florence Martin, Drew Polly, and Albert Ritzhaupt, "Bichronous Online Learning: Blending Asynchronous and Synchronous Online Learning,"   EDUCAUSE Review, September 8, 2020. Jump back to footnote 9 in the text. ↩
• Brian Beatty, "Hybrid Courses with Flexible Participation: The HyFlex Course Design," in Practical Applications and Experiences in K-20 Blended Learning Environments, eds. L. Kyei-Blankson and E. Ntuli (Hershey, PA: IGI Global, 2014), 153–177. Jump back to footnote 10 in the text. ↩
• Swapna Kumar, Albert Ritzhaupt, and Neuza Sofia Pedro, "Development and validation of the Online Instructor Support Survey (OISS)," Online Learning 26, no. 1 (2022). Jump back to footnote 11 in the text. ↩
• Yuan Chen and Saul Carliner, "A Special SME: An Integrative Literature Review of the Relationship Between Instructional Designers and Faculty in the Design of Online Courses for Higher Education," Performance Improvement Quarterly 33, no. 4 (2021): 471–495. Jump back to footnote 12 in the text. ↩
• Drew Polly, Florence Martin, and T. Christa Guilbaud, "Examining Barriers and Desired Supports to Increase Faculty Members' Use of Digital Technologies: Perspectives of Faculty, Staff and Administrators," Journal of Computing in Higher Education 33, no. 1 (2021): 135–156. Jump back to footnote 13 in the text. ↩
• Ibid. Jump back to footnote 14 in the text. ↩
• Christian Matt, Thomas Hess, and Alexander Benlian, "Digital Transformation Strategies," Business & Information Systems Engineering 57, no. 5 (2015): 339–343. Jump back to footnote 15 in the text. ↩
• Vanessa W. Vongkulluksn, Kui Xie, and Margaret A. Bowman, "The Role of Value on Teachers' Internalization of External Barriers and Externalization of Personal Beliefs for Classroom Technology Integration," Computers & Education 118 (2018): 70–81. Jump back to footnote 16 in the text. ↩
• Thelma C. Guilbaud, Florence Martin, and Xiaoxia Newton, "Faculty Perceptions on Accessibility in Online Learning: Knowledge, Practice and Professional Development," Online Learning 25, no. 2 (2021): 6–35. Jump back to footnote 17 in the text. ↩
• Sheng-Lun Cheng and Kui Xie, "Why College Students Procrastinate in Online Courses: A Self-Regulated Learning Perspective," The Internet and Higher Education 50 (2021): 100807. Jump back to footnote 18 in the text. ↩
• Beith Oyarzun and Florence Martin, "A Case Study on Multi-modal Course Delivery and Social Learning Opportunities," Bulletin of the IEEE Technical Committee on Learning Technology 15, no. 1 (2013): 25–28. Jump back to footnote 19 in the text. ↩

Florence Martin is a Professor of Learning, Design, and Technology at North Carolina State University.

Kui Xie is a Ted and Lois Cyphert Distinguished Professor and a Professor of Educational Psychology and Learning Technologies at The Ohio State University.

© 2022 Florence Martin and Kui Xie. The text of this work is licensed under a Creative Commons BY-SA 4.0 International License.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Managing digital transformation: a case study in a higher education institution.

1. Introduction

2. literature review, 3. methodology, 4.1. improving the teaching process, 4.2. internal communication, 4.3. cultural change, 4.4. data-driven management, 4.5. new management models, 5. discussion and conclusions, 6. research limitations, 7. future lines of research, author contributions, data availability statement, conflicts of interest, abbreviations.

• Romero, M.; Guédria, W.; Panetto, H.; Barafort, B. Towards a characterisation of smart systems: A systematic literature review. Comput. Ind. 2020 , 120 , 103224. [ Google Scholar ] [ CrossRef ]
• Schwarzmüller, T.; Brosi, P.; Duman, D.; Welpe, I. How does the digital transformation affect organizations? Key themes of change in work design and leadership. Manag. Rev. 2018 , 29 , 114–138. [ Google Scholar ] [ CrossRef ]
• Pucciarelli, F.; Kaplan, A. Competition and strategy in higher education: Managing complexity and uncertainty. Bus. Horiz. 2016 , 59 , 311–320. [ Google Scholar ] [ CrossRef ]
• Kane, G.C.; Phillips, A.N.; Copulsky, J.R.; Andrus, G.R. The Technology Fallacy—How People Are the Real Key to Digital Transformation ; The MIT Press: London, UK, 2019. [ Google Scholar ]
• Constantinides, P.; Henfridsson, O.; Parker, G.G. Introduction—Platforms and infrastructures in the digital age. Inf. Syst. Res. 2018 , 29 , 381–400. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Jackson, N. Managing for competency with innovation change in higher education: Examining the pitfalls and pivots of digital transformation. Bus. Horiz. 2019 , 62 , 761–772. [ Google Scholar ] [ CrossRef ]
• Oreg, S. Personality, context, and resistance to organizational change. Eur. J. Work Organ. Psychol. 2006 , 15 , 73–101. [ Google Scholar ] [ CrossRef ]
• Brunetti, F.; Matt, D.T.; Bonfanti, A.; De Longhi, A.; Pedrini, G.; Orzes, G. Digital transformation challenges: Strategies emerging from a multi-stakeholder approach. TQM J. 2020 , 32 , 697–724. [ Google Scholar ] [ CrossRef ]
• Elrod, P.D.; Tippett, D.D. The “death valley” of change. J. Organ. Chang. Manag. 2002 , 15 , 273–291. [ Google Scholar ] [ CrossRef ]
• Benavides, L.M.C.; Tamayo Arias, J.A.; Arango Serna, M.D.; Branch Bedoya, J.W.; Burgos, D. Digital transformation in higher education institutions: A systematic literature review. Sensors 2020 , 20 , 3291. [ Google Scholar ] [ CrossRef ]
• Matt, C.; Hess, T.; Belian, A. Digital Transformation Strategies. Bus. Inf. Syst. Eng. 2015 , 57 , 339–343. [ Google Scholar ] [ CrossRef ]
• Tirabeni, L.; Soderquist, K. Connecting the dots: Framing employee-driven innovation in open innovation contexts. Int. J. Innov. Technol. Manag. 2019 , 16 , 1950031. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Chesbrough, H.; Bogers, M. Explicating Open Innovation: Clarifying an Emerging Paradigm for Understanding Innovation. New Frontiers in Open Innovation ; Chesbrough, W.V.H., Ed.; Oxford University Press: Oxford, UK, 2014. [ Google Scholar ]
• Sandhu, G. The role of academic libraries in the digital transformation of the universities. In Proceedings of the 2018 5th International Symposium on Emergencing Trends Technologies in Libraries and Information Services (ETTLIS), Noida, India, 21–23 February 2018; pp. 292–296. [ Google Scholar ]
• Guinan, P.; Parise, S.; Langowitz, N. Creating an innovative digital project team: Levers to enable digital transformation. Bus. Horiz. 2019 , 62 , 717–727. [ Google Scholar ] [ CrossRef ]
• Bharadwaj, A.; El Sawy, O.A.; Pavlou, P.; Venkatraman, N. Digital Business Strategy: Toward a Next Generation of Insights. MIS Q. 2013 , 37 , 471–482. [ Google Scholar ] [ CrossRef ]
• Schmidt, E.; Rosenberg, J.; Eagle, A. How Google Works ; Grand Central Publishing: New York, NY, USA, 2014. [ Google Scholar ]
• WEF. The Future of Jobs Reports 2016. Research Report, 12. Available online: https://es.weforum.org/reports/the-future-of-jobs-report-2018/ (accessed on 17 September 2018).
• Schwab, K. The Fourth Industrial Revolution ; Currency; Edipro: São Paulo, Brazil, 2017. [ Google Scholar ]
• Perides, M.; Vasconcellos, E.; Vasconcellos, L. Change Management In Digital Transformation Projects: Case Study In A Financial Organization. Rev. De Gestão E Proj. 2020 , 11 , 54. [ Google Scholar ] [ CrossRef ]
• Kozanoglu, D.; Abedin, B. Understanding the role of employees in digital transformation: Conceptualization of digital literacy of employees as a multi-dimensional organizational affordance. J. Enterp. Inf. Manag. 2021 , 34 , 1649–1672. [ Google Scholar ] [ CrossRef ]
• Cueva Gaibor, D. Digital Transformation in Today´s University. Rev. Conrado 2020 , 16 , 483–489. [ Google Scholar ]
• Chaudhuri, S.; Ghosh, R. Reverse mentoring: A social exchange tool for keeping the boomers engaged and millennials committed. Human. Hum. Resour. Dev. Rev. 2012 , 11 , 55–76. [ Google Scholar ] [ CrossRef ]
• Abad-Segura, E.; González-Zamar, M.D.; Infante-Moro, J.C.; Ruipérez García, G. Sustainable management of digital transformation in higher education: Global research trends. Sustainability 2020 , 12 , 2107. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Endozo, A.N.; Oluyinka, S.; Daenos, R.G. Teachers’ experiences towards usage of Learning Management System: CANVAS. In Proceedings of the 2019 11th International Conference on Education Technology and Computers, Amsterdam, The Netherlands, 28–31 October 2019; pp. 91–95. [ Google Scholar ]
• Köhler, T. Del aprendizaje a la práctica. Nuevas estrategias para la gestión del conocimiento en la economía digital. Front. De La Investig. Del Comer. Electrónico 2006 , 4 . [ Google Scholar ]
• Goldie, J.G.S. Connectivism: A knowledge learning theory for the digital age? Med. Teach. 2016 , 38 , 1064–1069. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Lea, S.J.; Stephenson, D.; Troy, J. Higher education students’ attitudes to student-centred learning: Beyond’educational bulimia’? Stud. High. Educ. 2003 , 28 , 321–334. [ Google Scholar ] [ CrossRef ]
• Wright, G.B. Student-centered learning in higher education. Int. J. Teach. Learn. High. Educ. 2011 , 23 , 92–97. [ Google Scholar ]
• Geryk, M. (Ed.) Universities of the future: Universities in transition under the influence of stakeholders’ changing requirements. In Proceedings of the Advances in Human Factors, Business Management and Leadership: Proceedings of the AHFE 2017 International Conferences on Human Factors in Management and Leadership, and Business Management and Society, The Westin Bonaventure Hotel, Los Angeles, CA, USA, 17–21 July 2017 ; Springer International Publishing: Cham, Switzerland, 2018; pp. 116–124. [ Google Scholar ] [ CrossRef ]
• Burrows, J. Going beyond labels: A framework for profiling institutional stakeholders. Contemp. Educ. 1999 , 70 , 5. [ Google Scholar ]
• Eisenhardt, K.M. Building Theories from Case Study Research. Acad. Manag. Rev. 1989 , 14 , 532–550. [ Google Scholar ] [ CrossRef ]
• Yin, R.K. Case study evaluations: A decade of progress? In Evaluation Models. Evaluation in Education and Human Services ; Springer: Dordrecht, The Netherlands, 2000; pp. 185–193. [ Google Scholar ]
• Bustos Díaz, J.; Mellen Vinagre, T.; Nicolas-Sans, R. University Teaching Planning in Times of COVID-19: Analysis of the Catalan Context and Proposal for a Future Model from ESIC Business and Marketing School Experience. Sustainability 2021 , 13 , 5936. [ Google Scholar ] [ CrossRef ]
• Rodríguez-Sánchez, J.L.; González-Torres, T.; Montero-Navarro, A.; Gallego-Losada, R. Investing time and resources for work–life balance: The effect on talent retention. Int. J. Environ. Res. Public Health 2020 , 17 , 1920. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Sein-Echaluce, M.L.; Fidalgo-Blanco, Á.; Alves, G. Technology Behaviors in Education Innovation (No. ART-2017-98966). Comput. Hum. Behav. 2017 , 72 , 596–598. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Fathema, N.; Akanda, M.H. Effects of instructors’ academic disciplines and prior experience with learning management systems: A study about the use of Canvas. Australas. J. Educ. Technol. 2020 , 36 , 113–125. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Blštáková, J.; Joniaková, Z.; Stachová, Q.; Jankelová, N.; Stacho, Z. Reflection of digitalization on business values: The results of examining values of people Management in a Digital age. Sustainability 2020 , 12 , 2502. [ Google Scholar ] [ CrossRef ]
• Ilag, B.N.; Sabale, A.M. Microsoft teams overview. In Troubleshooting Microsoft Teams: Enlisting the Right Approach and Tools in Teams for Mapping and Troubleshooting Issues ; Apress: Berkeley, CA, USA, 2022; pp. 17–74. [ Google Scholar ]
• Schlesinger, W.; Cervera, A.; Calderón, H. The role of trust, image and shared values in the creation of value and loyalty: Application to the graduate-university relationship. Span. Res. Mag. Res.-ESIC 2014 , 18 , 126–139. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Latorre-Cosculluela, C.; Vázquez-Toledo, S.; Rodríguez-Martínez, A.; Liesa-Orús, M. Design Thinking: Creatividad y pensamiento crítico en la universidad. Rev. Electrónica De Investig. Educ. 2020 , 22 , e28. [ Google Scholar ] [ CrossRef ]
• Arango Serna, M.D.; Branch, J.W.; Castro Benavides, L.M.; Burgos, D. Un modelo conceptual de transformación digital. Openergy y el caso de la Universidad Nacional de Colombia. Educ. Khowledge Soc. 2018 , 19 , 95–107. [ Google Scholar ] [ CrossRef ] [ Green Version ]
• Cardenas-Navia, I.; Fitzgerald, B.K. The digital dilemma: Winning and losing strategies in the digital talent race. Ind. High. Educ. 2019 , 33 , 214–217. [ Google Scholar ] [ CrossRef ]
• Yin, R.K. Discovering the future of the case study. Method in evaluation research. Eval. Pract. 1994 , 15 , 283–290. [ Google Scholar ] [ CrossRef ]
• Freeman, R.E.; McVea, J. A stakeholder approach to strategic management. In Handbook of Strategic Management ; Blackwell Publishing: Oxford, UK, 2005; pp. 183–201. [ Google Scholar ]
• Freeman, R.E.; Harrison, J.S.; Wicks, A.C.; Parmar, B.L.; De Colle, S. Stakeholder Theory: The State of the Art ; Cambridge University Press: Cambridge, UK, 2010. [ Google Scholar ]
• Chetty, S. The case study method for research in small and medium sized firms. Int. Small Bus. J. 1996 , 15 , 73–85. [ Google Scholar ] [ CrossRef ]
• Bonache, J. El estudio de casos como estrategia de construcción teórica: Características, críticas y defensas. Cuad. Econ. Y Dir. La Empresa 1999 , 3 , 123–140. [ Google Scholar ]

Click here to enlarge figure

Share and Cite

Díaz-Garcia, V.; Montero-Navarro, A.; Rodríguez-Sánchez, J.-L.; Gallego-Losada, R. Managing Digital Transformation: A Case Study in a Higher Education Institution. Electronics 2023 , 12 , 2522. https://doi.org/10.3390/electronics12112522

Díaz-Garcia V, Montero-Navarro A, Rodríguez-Sánchez J-L, Gallego-Losada R. Managing Digital Transformation: A Case Study in a Higher Education Institution. Electronics . 2023; 12(11):2522. https://doi.org/10.3390/electronics12112522

Díaz-Garcia, Vicente, Antonio Montero-Navarro, José-Luis Rodríguez-Sánchez, and Rocío Gallego-Losada. 2023. "Managing Digital Transformation: A Case Study in a Higher Education Institution" Electronics 12, no. 11: 2522. https://doi.org/10.3390/electronics12112522

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Digital technologies in support of students learning in Higher Education: literature review

• Marta Pinto Faculty of Psychology and Education Science of University of Porto
• Carlinda Leite Faculty of Psychology and Education Science of University of Porto

Aldhafeeri, F., & Male, T. (2016). Investigating the learning challenges presented by digital technologies to the College of Education in Kuwait University. Education and Information Technologies, 21(6), 1509-1519. doi:10.1007/s10639-015-9396-2

Alzahrani, M. G. (2017). The effect of using online discussion forums on students’ learning. Turkish Online Journal of Educational Technology, 16(1), 164-176.

Amador, P., & Amador, J. (2014). Academic advising via Facebook: Examining student help seeking. Internet and Higher Education, 21, 9-16. doi:10.1016/j.iheduc.2013.10.003

Andone, D., & Vasiu, R. (2016) MOOCs in higher education—flipped classroom or a new smart learning model? In. Lecture Notes in Educational Technology (pp. 303-307).

Araujo, C. M., Spritzer, I. M. P. A., & de Souza, C. G. (2012). Technology innovation - Electronic game in the Brazilian higher education. Journal of Technology Management and Innovation, 7(3), 32-43.

Aresta, M., Pedro, L., & Santos, C. (2015). Mobile learning and higher education: A theoretical overview. Journal of Mobile Multimedia, 11(1-2), 147-156.

Arias-Masa, J., Alonso-Díaz, L., Cubo-Delgado, S., Gutiérrez-Esteban, P., & Yuste-Tosina, R. (2014). Assessment of the use of synchronous virtual classrooms in higher education. New Educational Review, 38(4), 223-237.

Armstrong & Franklin, T., J. (2008). A review of current and developing international practice in the use of social networking (Web 2.0) in higher education. In: Franklin Consulting.

Aslam, W. (2017). Videoconferencing and higher education teaching in Politics and International Relations classrooms. Politics, 37(4), 500-519. doi:10.1177/0263395716633708

Attia, M. A. (2014) Postgraduate students' perceptions toward online assessment: The case of the faculty of education, Umm Al-Qura university. In: Vol. 24. International Perspectives on Education and Society (pp. 151-173).

Badenhorst, C., & Mather, C. (2014). Blogging geographies. Journal of Geography in Higher Education, 38(2), 193-207. doi:10.1080/03098265.2014.908276

Balakrishnan, B., & Woods, P. C. (2013). A comparative study on real lab and simulation lab in communication engineering from students' perspectives. European Journal of Engineering Education, 38(2), 159-171. doi:10.1080/03043797.2012.755499

Baranova, A. R., & Ladner, R. A. (2016). Role of a game as a mechanism for maintenance of students' educational motives when learning a foreign language. Social Sciences (Pakistan), 11(14), 3623-3626.

Barrett, M. E., Swan, A. B., Mamikonian, A., Ghajoyan, I., Kramarova, O., & Youmans, R. J. (2014). Technology in note taking and assessment: The effects of congruence on student performance. International Journal of Instruction, 7(1), 49-58.

Bartlett, M. E., & Bartlett, J. E. (2016). Case study on the impact of technology on incivility in higher education. Journal of Educators Online, 13(2), 1-18.

Batista, J., Morais, S., & Ramos, F. (2016). Researching the Use of Communication Technologies in Higher Education Institutions in Portugal. In M. Pinheiro & D. Simões (Eds.), Handbook of Research on Engaging Digital Natives in Higher Education Settings (pp. 280 - 303). Hershey, PA, USA: IGI Global.

Becker, S. A., Cummins, M., Davis, A., Freeman, A., Glesinger Hall, C., & Ananthanarayanan, V. (2017). NMC Horizon Report: 2017 Higher Education Edition. Retrieved from Austin, Texas:

Ben Youssef, A., Dahmani, M., & Omrani, N. (2013). Information technologies, students’ e-skills and diversity of learning process. Education and Information Technologies, 20(1), 141-159. doi:10.1007/s10639-013-9272-x

Bennett, D., Rowley, J., Dunbar-Hall, P., Hitchcock, M., & Blom, D. (2016). Electronic portfolios and learner identity: An eportfolio case study in music and writing. Journal of Further and Higher Education, 40(1), 107-124. doi:10.1080/0309877X.2014.895306

Bennett, S., Bishop, A., Dalgarno, B., Kennedy, G., & Waycott, J. (2012). Implementing web 2.0 technologies in higher education: A collective case study. Computers & Education(0). doi:10.1016/j.compedu.2011.12.022

Bharuthram, S., & Kies, C. (2013). Introducing e-learning in a South African higher education institution: Challenges arising from an intervention and possible responses. British Journal of Educational Technology, 44(3), 410-420. doi:10.1111/j.1467-8535.2012.01307.x

Blair, E., Maharaj, C., & Primus, S. (2016). Performance and perception in the flipped classroom. Education and Information Technologies, 21(6), 1465-1482. doi:10.1007/s10639-015-9393-5

Blanc, S., Benlloch-Dualde, J. V., & Benet, G. (2015). Engaging students in an undergraduate computer technology course: An active-learning approach. International Journal of Engineering Education, 31(2), 610-618.

Blau, I., & Shamir-Inbal, T. (2017). Re-designed flipped learning model in an academic course: The role of co-creation and co-regulation. Computers and Education, 115, 69-81. doi:10.1016/j.compedu.2017.07.014

Borup, J., West, R. E., & Graham, C. R. (2012). Improving online social presence through asynchronous video. Internet and Higher Education, 15(3), 195-203. doi:10.1016/j.iheduc.2011.11.001

Bridges, S., Chang, J. W. W., Chu, C. H., & Gardner, K. (2014). Blended learning in situated contexts: 3-year evaluation of an online peer review project. European Journal of Dental Education, 18(3), 170-179. doi:10.1111/eje.12082

Bullen, M., & Morgan, T. (2015). Digital learners in higher education: Implications for teaching, learning & technology. Teaching and learning in digital worlds: Strategies and issues in higher education, 11-19.

Buraphadeja, V., & Kumar, S. (2012). Content analysis of wiki discussions for knowledge construction: Opportunities and challenges. International Journal of Web-Based Learning and Teaching Technologies, 7(2), 28-42. doi:10.4018/jwltt.2012040103

Carbonell Carrera, C., & Bermejo Asensio, L. A. (2017). Landscape interpretation with augmented reality and maps to improve spatial orientation skill. Journal of Geography in Higher Education, 41(1), 119-133. doi:10.1080/03098265.2016.1260530

Carmen-Ricoy, M., & Da Silva-Couto, M. J. V. (2016). Digital mobile devices and skills to use in the "knowledge society. Convergencia, 23(70), 59-85.

Chang, R. I., Hung, Y. H., & Lin, C. F. (2015). Survey of learning experiences and influence of learning style preferences on user intentions regarding MOOCs. British Journal of Educational Technology, 46(3), 528-541. doi:10.1111/bjet.12275

Chapple, D., Weir, B., & Martin, R. S. (2017). Can the incorporation of quick response codes and smartphones improve field-based science education? International Journal of Innovation in Science and Mathematics Education, 25(2), 49-71.

Chawinga, W. D. (2017). Taking social media to a university classroom: teaching and learning using Twitter and blogs. International Journal of Educational Technology in Higher Education, 14(1). doi:10.1186/s41239-017-0041-6

Cheng, I. N. Y., Chan, J. K. Y., Kong, S. S. Y., & Leung, K. M. Y. (2016). Effectiveness and obstacle of using Facebook as a tool to facilitate student-centred learning in higher education. Asia-Pacific Forum on Science Learning and Teaching, 17(2).

Conole, G., & Alevizou, P. (2010). A literature review of the use of Web 2.0 tools in Higher Education. Walton Hall, Milton Keynes, UK: The Open University, retrieved February, 17, 2011.

Cook-Sather, A. (2017). Virtual forms, actual effects: how amplifying student voice through digital media promotes reflective practice and positions students as pedagogical partners to prospective high school and practicing college teachers. British Journal of Educational Technology, 48(5), 1143-1152. doi:10.1111/bjet.12523

Cooke, M., Watson, B., Blacklock, E., Mansah, M., Howard, M., Johnston, A., . . . Murfield, J. (2012). Lecture capture: First year student nurses' experiences of a web-based lecture technology. Australian Journal of Advanced Nursing, 29(3), 14-21.

Costa, C., Alvelos, H., & Teixeira, L. (2013). The students' motivation to use the wiki technology: A practical study in higher education. Educacao e Pesquisa, 39(3), 775-790. doi:10.1590/S1517_97022013000300014

Costa, C., & Harris, L. (2017). Reconsidering the technologies of intellectual inquiry in curriculum design. Curriculum Journal, 28(4), 559-577. doi:10.1080/09585176.2017.1308260

Crook, A., Mauchline, A., Maw, S., Lawson, C., Drinkwater, R., Lundqvist, K., . . . Park, J. (2012). The use of video technology for providing feedback to students: Can it enhance the feedback experience for staff and students? Computers and Education, 58(1), 386-396. doi:10.1016/j.compedu.2011.08.025

Dalgarno, B., Reupert, A., & Bishop, A. (2015). Blogging while on professional placement: explaining the diversity in student attitudes and engagement. Technology, Pedagogy and Education, 24(2), 189-209. doi:10.1080/1475939X.2013.847481

Deepak, A. C., Wisner, B. L., & Benton, A. D. (2016). Intersections between Technology, Engaged Learning, and Social Capital in Social Work Education. Social Work Education, 35(3), 310-322. doi:10.1080/02615479.2016.1154661

Dickie, V. A., & Meier, H. (2015). The facebook tutor: Networking education. Ubiquitous Learning, 8(2), 1-12.

Dubovi, I., Levy, S. T., & Dagan, E. (2017). Now I know how! The learning process of medication administration among nursing students with non-immersive desktop virtual reality simulation. Computers and Education, 113, 16-27. doi:10.1016/j.compedu.2017.05.009

Edmunds, R., Thorpe, M., & Conole, G. (2012). Student attitudes towards and use of ICT in course study, work and social activity: A technology acceptance model approach. British Journal of Educational Technology, 43(1), 71-84. doi:10.1111/j.1467-8535.2010.01142.x

Estriégana-Valdehita, R., Plata, R. B., & Medina-Merodio, J. A. (2017). Educational technology in flipped course design. International Journal of Engineering Education, 33(4), 1199-1212.

Faizi, R., Chiheb, R., & El Afia, A. (2015). Students' perceptions towards using Web 2.0 technologies in education. International Journal of Emerging Technologies in Learning, 10(6), 32-36. doi:10.3991/ijet.v10i6.4858

Ferrer-Torregrosa, J., Jiménez-Rodríguez, M. Á., Torralba-Estelles, J., Garzón-Farinós, F., Pérez-Bermejo, M., & Fernández-Ehrling, N. (2016). Distance learning ects and flipped classroom in the anatomy learning: Comparative study of the use of augmented reality, video and notes. BMC Medical Education, 16(1). doi:10.1186/s12909-016-0757-3

Fessakis, G., Dimitracopoulou, A., & Palaiodimos, A. (2013). Graphical interaction analysis impact on groups collaborating through blogs. Educational Technology and Society, 16(1), 243-253.

Fuller, I. C., & France, D. (2016). Does digital video enhance student learning in field-based experiments and develop graduate attributes beyond the classroom? Journal of Geography in Higher Education, 40(2), 193-206. doi:10.1080/03098265.2016.1141186

Gachago, D., Livingston, C., & Ivala, E. (2016). Podcasts: A technology for all? British Journal of Educational Technology, 47(5), 859-872. doi:10.1111/bjet.12483

Gallardo Echenique, E., Marqués Molías, L., & Bullen, M. (2015). Students in higher education: Social and academic uses of digital technology. RUSC Universities and Knowledge Society Journal, 12(1), 25-37. doi:10.7238/rusc.v12i1.2078

Gan, C., Li, H., & Liu, Y. (2017). Understanding mobile learning adoption in higher education An empirical investigation in the context of the mobile library. Electronic Library, 35(5), 846-860. doi:10.1108/EL-04-2016-0093

García, F., Martin, D., de la Escalera, A., Armingol, J. M., & Al-Kaff, A. H. (2016). Enhancing engineering learning through SPOC courses. International Journal of Technologies in Learning, 23(3), 15-20.

Garcia-Vera, V. E., & Chiner, E. (2017). Factors influencing graduate students' preference of software tools for building engineering applications. International Journal of Engineering Education, 33(1), 128-137.

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2), 95-105. doi:10.1016/j.iheduc.2004.02.001

González-Tablas, A. I., De Fuentes, J. M., Hernández-Ardieta, J. L., & Ramos, B. (2013). Leveraging quiz-based multiple-prize web tournaments for reinforcing routine mathematical skills. Educational Technology and Society, 16(3), 28-43.

Han, I., & Shin, W. S. (2016). The use of a mobile learning management system and academic achievement of online students. Computers and Education, 102, 79-89. doi:10.1016/j.compedu.2016.07.003

Hedén, L., & Ahlstrom, L. (2016). Individual response technology to promote active learning within the caring sciences: An experimental research study. Nurse Education Today, 36, 202-206. doi:10.1016/j.nedt.2015.10.010

Heiman, T., & Shemesh, D. O. (2012). Students With LD in Higher Education: Use and Contribution of Assistive Technology and Website Courses and Their Correlation to Students' Hope and Well-Being. Journal of Learning Disabilities, 45(4), 308-318. doi:10.1177/0022219410392047

Henderson, M., Selwyn, N., & Aston, R. (2017). What works and why? Student perceptions of ‘useful’ digital technology in university teaching and learning. Studies in Higher Education, 42(8), 1567-1579. doi:10.1080/03075079.2015.1007946

Hennessy, C., & Forrester, G. (2014). Developing a framework for effective audio feedback: A case study. Assessment and Evaluation in Higher Education, 39(7), 777-789. doi:10.1080/02602938.2013.870530

Issa, T., Issa, T., & Chang, V. (2012). Technology and higher education: An Australian case study. International Journal of Learning, 18(3), 223-236.

Jabbour, K. K. (2014). An analysis of the effect of mobile learning on lebanese higher education. Informatics in Education, 13(1), 1-15.

Jang, Y. (2015). Convenience matters: A qualitative study on the impact of use of social media and collaboration technologies on learning experience and performance in higher education. Education for Information, 31(1-2), 73-98. doi:10.3233/EFI-150948

Jiménez-Castillo, D., Sánchez-Fernández, R., & Marín-Carrillo, G. M. (2017). Dream team or odd couple? Examining the combined use of lectures and podcasting in higher education. Innovations in Education and Teaching International, 54(5), 448-457. doi:10.1080/14703297.2016.1148622

Jones, N., Georghiades, P., & Gunson, J. (2012). Student feedback via screen capture digital video: Stimulating student's modified action. Higher Education, 64(5), 593-607. doi:10.1007/s10734-012-9514-7

Josefsson, P., Hrastinski, S., Pargman, D., & Pargman, T. C. (2016). The student, the private and the professional role: Students’ social media use. Education and Information Technologies, 21(6), 1583-1594. doi:10.1007/s10639-015-9403-7

Kara, A., Cagiltay, N. E., & Dalveren, Y. (2014). An enhanced course in digital communications. International Journal of Engineering Education, 30(4), 1048-1059.

Kingston, D. G., Eastwood, W. J., Jones, P. I., Johnson, R., Marshall, S., & Hannah, D. M. (2012). Experiences of using mobile technologies and virtual field tours in Physical Geography: Implications for hydrology education. Hydrology and Earth System Sciences, 16(5), 1281-1286. doi:10.5194/hess-16-1281-2012

Klimova, B., & Poulova, P. (2016). Mobile learning in higher education. Advanced Science Letters, 22(5-6), 1111-1114. doi:10.1166/asl.2016.6673

Lai, K. W., & Hong, K. S. (2015). Technology use and learning characteristics of students in higher education: Do generational differences exist? British Journal of Educational Technology, 46(4), 725-738. doi:10.1111/bjet.12161

Lambert, N., & Watkins, L. (2013). Meet Mohammed: Using simulation and technology to support learning. Journal of Mental Health Training, Education and Practice, 8(2), 66-75. doi:10.1108/JMHTEP-03-2012-0001

Lee, C. Y. (2015). The effects of online peer assessment and family entrepreneurial experience on students’ business planning performance. Turkish Online Journal of Educational Technology, 14(1), 123-132.

Lemus-Zúñiga, L. G., Montañana, J. M., Buendia-García, F., Poza-Luján, J. L., Posadas-Yagüe, J. L., & Benlloch-Dualde, J. V. (2015). Computer-assisted method based on continuous feedback to improve the academic achievements of first-year students on computer engineering. Computer Applications in Engineering Education, 23(4), 610-620. doi:10.1002/cae.21633

Lu, S. C., & Jeng, Y. C. (2012). Mobile learning in higher education: Students' learning effectiveness. ICIC Express Letters, 6(10), 2705-2710.

Maduku, D. K. (2015). Factors of E-book Use Intentions: Perspective of Students in a Developing Country. Perspectives on Global Development and Technology, 14(6), 597-618. doi:10.1163/15691497-12341364

Magogwe, J. M., Ntereke, B., & Phetlhe, K. R. (2015). Facebook and classroom group work: A trial study involving University of Botswana Advanced Oral Presentation students. British Journal of Educational Technology, 46(6), 1312-1323. doi:10.1111/bjet.12204

Mancillas, L. K., & Brusoe, P. W. (2016). Born Digital: Integrating Media Technology in the Political Science Classroom. Journal of Political Science Education, 12(4), 375-386. doi:10.1080/15512169.2015.1096792

Marques, M. A., Viegas, M. C., Costa-Lobo, M. C., Fidalgo, A. V., Alves, G. R., Rocha, J. S., & Gustavsson, I. (2014). How remote labs impact on course outcomes: Various practices using VISIR. IEEE Transactions on Education, 57(3), 151-159. doi:10.1109/TE.2013.2284156

Marstio, T., & Kivelä, S. (2014). Technology enhanced learning in a higher education context: Building bridges by student empowerment and regional development. Nordic Journal of Digital Literacy, 2014(1), 66-87.

Mathews, S., Andrews, L., & Luck, E. (2012). Developing a Second Life virtual field trip for university students: An action research approach. Educational Research, 54(1), 17-38. doi:10.1080/00131881.2012.658197

Mathisen, P. (2012). Video feedback in higher education - a contribution to improving the quality of written feedback. Nordic Journal of Digital Literacy, 2012(2), 97-113.

Mattis, K. V. (2015). Flipped Classroom Versus Traditional Textbook Instruction: Assessing Accuracy and Mental Effort at Different Levels of Mathematical Complexity. Technology, Knowledge and Learning, 20(2), 231-248. doi:10.1007/s10758-014-9238-0

Menkhoff, T., & Bengtsson, M. L. (2012). Engaging students in higher education through mobile learning: Lessons learnt in a Chinese entrepreneurship course. Educational Research for Policy and Practice, 11(3), 225-242. doi:10.1007/s10671-011-9123-8

Merhi, M. I. (2015). Factors influencing higher education students to adopt podcast: An empirical study. Computers and Education, 83, 32-43. doi:10.1016/j.compedu.2014.12.014

Migin, M. W., & Falahat, M. (2016). Effect of social media usage on university students in an emerging country. Turkish Online Journal of Educational Technology, 2016(NovemberSpecialIssue), 803-808.

Mirriahi, N., Liaqat, D., Dawson, S., & Gašević, D. (2016). Uncovering student learning profiles with a video annotation tool: reflective learning with and without instructional norms. Educational Technology Research and Development, 64(6), 1083-1106. doi:10.1007/s11423-016-9449-2

Muñoz-Repiso, A. G. V., & Tejedor, F. J. T. (2012). The incorporation of ICT in higher education. the contribution of ROC curves in the graphic visualization of differences in the analysis of the variables. British Journal of Educational Technology, 43(6), 901-919. doi:10.1111/j.1467-8535.2011.01270.x

Neira, E. A. S., Salinas, J., & Crosetti, B. B. (2017). Emerging Technologies (ETs) in education: A systematic review of the literature published between 2006 and 2016. International Journal of Emerging Technologies in Learning, 12(5), 128-149. doi:10.3991/ijet.v12i05.6939

Norman, H., Din, R., Nordin, N., & Ryberg, T. (2013). A review on the use and perceived effects of mobile blogs on learning in higher educational settings. Asian Social Science, 10(1), 209-222. doi:10.5539/ass.v10n1p209

Novakovich, J., Miah, S., & Shaw, S. (2017). Designing curriculum to shape professional social media skills and identity in virtual communities of practice. Computers and Education, 104, 65-90. doi:10.1016/j.compedu.2016.11.002

Olelewe, C. J., & Agomuo, E. E. (2016). Effects of B-learning and F2F learning environments on students' achievement in QBASIC programming. Computers and Education, 103, 76-86. doi:10.1016/j.compedu.2016.09.012

Parkes, M., & Fletcher, P. (2017). A longitudinal, quantitative study of student attitudes towards audio feedback for assessment. Assessment & Evaluation in Higher Education, 42(7), 1046-1053. doi:10.1080/02602938.2016.1224810

Pedró, F. (2009). New Millenium Learners in Higher Education: Evidence and Policy Implications. In International Conference on 21st Century Competencies, Brussels: OECD/CERI.

Pedro, L., Barbosa, C., & Santos, C. (2018). A critical review of mobile learning integration in formal educational contexts. International Journal of Educational Technology in Higher Education, 15(1), 10.

Pinto, M., Souza, F., Nogueira, F., Balula, A., Pedro, L., Pombo, L., Ramos, F., Moreira, A. (2012). Tracing the use of communication technologies in higher education - a literature review. In CSEDU 2012 4th International Conference on Computer Supported Education Proceedings, Porto,181-186.

Pomerol, J. C., Epelboin, Y., & Thoury, C. (2015). MOOCs and Higher Education. MOOCs, 83-100.

Raes, A., Vanderhoven, E., & Schellens, T. (2015). Increasing anonymity in peer assessment by using classroom response technology within face-to-face higher education. Studies in Higher Education, 40(1), 178-193. doi:10.1080/03075079.2013.823930

Romero, M., Guitert, M., Sangrà, A., & Bullen, M. (2013). Do UOC students fit in the net generation profile? An approach to their habits in ICT use. International Review of Research in Open and Distance Learning, 14(3), 158-181.

Russo, T., Zhang, J., & Fallon, M. (2015). Influence of perceived technology use of university students on academic and social performance in college. International Journal of Interdisciplinary Civic and Political Studies, 10(2), 17-31.

Samir Abou El-Seoud, M., Taj-Eddin, I. A. T. F., Seddiek, N., El-Khouly, M. M., & Nosseir, A. (2014). E-learning and students' motivation: A research study on the effect of e-learning on higher education. International Journal of Emerging Technologies in Learning, 9(4), 20-26. doi:10.3991/ijet.v9i4.3465

Sánchez-Lugo, J., & Domínguez-Flores, N. (2013). The use of mobile technologies by undergraduate students from the university of Puerto Rico system. Ubiquitous Learning, 5(3), 53-66.

Sayımer, İ., Yüksel, A., & Demir, B. (2015). Transformatıon of instructıonal and learnıng paradıgm in dıgıtal age: Socıal networkıng practıces and academıc expectatıons of hıgher educatıon students in Turkey. Turkish Online Journal of Educational Technology, 2015, 782-789.

Seale, J. (2013). When digital capital is not enough: Reconsidering the digital lives of disabled university students. Learning, Media and Technology, 38(3), 256-269. doi:10.1080/17439884.2012.670644

Seale, J., Georgeson, J., Mamas, C., & Swain, J. (2015). Not the right kind of 'digital capital'? An examination of the complex relationship between disabled students, their technologies and higher education institutions. Computers and Education, 82, 118-128. doi:10.1016/j.compedu.2014.11.007

Selwyn, N. (2016). Is technology good for education? : John Wiley & Sons.

Selwyn, N., Henderson, M., Finger, G., Larkin, K., Smart, V., and Chao, S. (2016). What works and Why? Understanding successful technology enabled learning within institutional contexts Part B Appendices. Canberra, ACT: Australian Government Office for Learning and Teaching.

Sheriff, R. E. (2012). An evaluation of students' and lecturers' use of technologies: An engineering case study. Engineering Education, 7(1), 33-46. doi:10.11120/ened.2012.07010033

Shurygin, V. Y., & Krasnova, L. A. (2016). Electronic learning courses as a means to activate students’ independent work in studying physics. International Journal of Environmental and Science Education, 11(8), 1743-1751. doi:10.12973/ijese.2016.551a

Sieche, S., Krey, B., & Bastiaens, T. J. (2013). Investigating students' usage and acceptance of electronic books. Journal of Educational Multimedia and Hypermedia, 22(4), 465-487.

Siemens, & Tittenberger, P. (2009). Handbook of Emerging Technologies for Learning. In. Retrieved from http://umanitoba.ca/learning_technologies/cetl/HETL.pdf

Siemens, G., & Tittenberger, P. (2009). Handbook of Emerging Technologies for Learning. In. Retrieved from http://umanitoba.ca/learning_technologies/cetl/HETL.pdf

Singh, U. G., & Wassermann, J. M. (2016). A Story of a Journey in Implementing an E-Assessment System at a South African University. Africa Education Review, 13(3-4), 1-16. doi:10.1080/18146627.2016.1202549

Sleeman, J., Lang, C., & Lemon, N. (2016). Social Media Challenges and Affordances for International Students: Bridges, Boundaries, and Hybrid Spaces. Journal of Studies in International Education, 20(5), 391-415. doi:10.1177/1028315316662975

Sobaih, A. E. E., & Moustafa, M. A. (2016). Speaking the Same Language: The Value of Social Networking Sites for Hospitality and Tourism Higher Education in Egypt. Journal of Hospitality and Tourism Education, 28(1), 21-31. doi:10.1080/10963758.2015.1127169

Solmaz, O. (2016). #Beyond140: Helping pre-service teachers construct a community of inquiry on twitter. Turkish Online Journal of Educational Technology, 15(4), 1-15.

Sopina, E., & McNeill, R. (2015). Investigating the relationship between quality, format and delivery of feedback for written assignments in higher education. Assessment and Evaluation in Higher Education, 40(5), 666-680. doi:10.1080/02602938.2014.945072

Srifa, P. (2016). Learning and teaching environments by using virtual reality technology. Turkish Online Journal of Educational Technology, 2016(DecemberSpecialIssue), 1-5.

Stevenson, M. E., & Hedberg, J. G. (2017). Mobilizing learning: a thematic review of apps in K-12 and higher education. Interactive Technology and Smart Education, 14(2), 126-137. doi:10.1108/ITSE-02-2017-0017

Stewart, K. D., & Ivala, E. (2017). Silence, voice, and “other languages”: Digital storytelling as a site for resistance and restoration in a South African higher education classroom. British Journal of Educational Technology, 48(5), 1164-1175. doi:10.1111/bjet.12540

Terrion, J. L., & Aceti, V. (2012). Perceptions of the effects of clicker technology on student learning and engagement: A study of freshmen Chemistry students. Research in Learning Technology, 20(2). doi:10.3402/rlt.v20i0.16150

Tolulope, A. E., Adenubi, O. S., & Oluwole, F. C. (2015). Evaluating undergraduates’ attitude towards the use of podcast for learning selected educational technology concepts. Global Media Journal, 2015.

Tri-Agif, I., Noorhidawati, A., & Ghalebandi, S. G. (2016). Continuance intention of using e-book among higher education students. Malaysian Journal of Library and Information Science, 21(1), 19-33.

Umek, L., Keržič, D., Aristovnik, A., & Tomaževič, N. (2017). An assessment of the effectiveness of moodle e-learning system for undergraduate Public Administration education. International Journal of Innovation and Learning, 21(2), 165-177. doi:10.1504/IJIL.2017.081939

Valentín, A., Mateos, P. M., González-Tablas, M. M., Pérez, L., López, E., & García, I. (2013). Motivation and learning strategies in the use of ICTs among university students. Computers and Education, 61(1), 52-58. doi:10.1016/j.compedu.2012.09.008

Vázquez-Cano, E., Meneses, E. L., & Sánchez-Serrano, J. L. S. (2015). Analysis of social worker and educator's areas of intervention through multimedia concept maps and online discussion forums in higher education. Electronic Journal of e-Learning, 13(5), 333-346.

Viberg, O., & Grönlund, Å. (2013). Cross-cultural analysis of users' attitudes toward the use of mobile devices in second and foreign language learning in higher education: A case from Sweden and China. Computers and Education, 69, 169-180. doi:10.1016/j.compedu.2013.07.014

Virtanen, M. A., Kääriäinen, M., Liikanen, E., & Haavisto, E. (2017). The comparison of students’ satisfaction between ubiquitous and web-basedlearning environments. Education and Information Technologies, 22(5), 2565-2581. doi:10.1007/s10639-016-9561-2

Virtanen, P., Myllärniemi, J., & Wallander, H. (2013). Diversifying higher education: Facilitating different ways of learning. Campus-Wide Information Systems, 30(3), 201-211. doi:10.1108/10650741311330384

Waldeck, J. H., & Dougherty, K. (2012). Collaborative communication technologies and learning in college courses: Which are used, for what purposes, and to what ends? Learning, Media and Technology, 37(4), 355-378. doi:10.1080/17439884.2011.592497

Waycott, J., Thompson, C., Sheard, J., & Clerehan, R. (2017). A virtual panopticon in the community of practice: Students' experiences of being visible on social media. Internet and Higher Education, 35, 12-20. doi:10.1016/j.iheduc.2017.07.001

Weller, M. (2007). Virtual learning environments: Using, choosing and developing your VLE: Routledge.

Wesch, M. (2009). From Knowledgable to Knowledge-able: Learning in New Media Environments. The Academic Commons Magazine. Retrieved from http://academiccommons.org/

Wihlborg, M., Friberg, E. E., Rose, K. M., & Eastham, L. (2017). Facilitating learning through an international virtual collaborative practice: A case study. Nurse Education Today, 61, 3-8. doi:10.1016/j.nedt.2017.10.007

Yang, J. L., Coleman, K., Das, M., & Hawkins, N. (2015). Integrated career development learning and ePortfolios: Improving student self-efficacy in employability skills in an undergraduate science course. International Journal of Adult, Community and Professional Learning, 22(1), 1-17.

Yi, Y. J., You, S., & Bae, B. J. (2016). The influence of smartphones on academic performance: The development of the technology-to-performance chain model. Library Hi Tech, 34(3), 480-499. doi:10.1108/LHT-04-2016-0038

Zacharis, N. Z. (2012). Predicting college students' acceptance of podcasting as a learning tool. Interactive Technology and Smart Education, 9(3), 171-183. doi:10.1108/17415651211258281

Zitzelsberger, H., Campbell, K. A., Service, D., & Sanchez, O. (2015). Using wikis to stimulate collaborative learning in two online health sciences courses. Journal of Nursing Education, 54(6), 352-355. doi:10.3928/01484834-20150515-08

The authors who publish in this journal agree to the following terms:

• Authors retain copyright and grant the journal the right of first publication.
• The texts published in Digital Education Review, DER, are under a license Attribution-Noncommercial-No Derivative Works 4,0 Spain, of Creative Commons. All the conditions of use in: Creative Commons ,
• In order to mention the works, you must give credit to the authors and to this Journal.
• Digital Education Review, DER,  does not accept any responsibility for the points of view and statements made by the authors in their work.

Make a Submission

• Advanced Search

The Gap between Higher Education and the Software Industry — A Case Study on Technology Differences

Software Engineering and Education Group, Mid Sweden University, Sweden

New Citation Alert added!

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

• Publisher Site
• View all Formats

ECSEE '23: Proceedings of the 5th European Conference on Software Engineering Education

We see an explosive global labour demand in the Software Industry, and higher education institutions play a crucial role in supplying the industry with professionals with relevant education. Existing literature identifies a gap between what software engineering education teaches students and what the software industry demands. Using our open-sourced Job Market AnalyseR (JMAR) text-analysis tool, we compared keywords from higher education course syllabi and job posts to investigate the knowledge gap from a technology-focused departure point. We present a trend analysis of technology in job posts over the past six years in Sweden. We found that demand for cloud and automation technology such as Kubernetes and Docker is rising in job ads but not that much in higher education syllabi. The language used in higher education syllabi and job ads differs where the former emphasizes concepts and the latter technologies more heavily. We discuss possible remedies to bridge this mismatch to draw further conclusions in future work, including calibrating JMAR to other industry-relevant aspects, including soft skills, software concepts, or new demographics.

Index Terms

Applied computing

Collaborative learning

Social and professional topics

Professional topics

Computing education

Recommendations

How higher-education systems influence software engineering degree programs.

Undergraduate programs that cover knowledge related to software engineering range from software engineering-specific programs to traditional computer science programs, which might teach some subjects related to software engineering. Software engineering-...

A comprehensive software engineering education program for grades 6 to 12 in NYC public schools (abstract only)

How do we best prepare middle and high school students for college and career pathways in software engineering? Current industry and academic trends suggest that students are best prepared for postsecondary success in new and emerging interdisciplinary ...

Computer Science Education in North-Rhine Westphalia, Germany—A Case Study

In North-Rhine Westphalia, the most populated state in Germany, Computer Science (CS) has been taught in secondary schools since the early 1970s. This article provides an overview of the past and current situation of CS education in North-Rhine ...

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

• Information
• Contributors

Published in

OTH Regensburg

Copyright © 2023 Owner/Author

This work is licensed under a Creative Commons Attribution-NoDerivatives International 4.0 License.

In-Cooperation

Association for Computing Machinery

New York, NY, United States

Publication History

• Published: 19 June 2023

Check for updates

Author tags.

• Education Gap
• Industrial Demand
• Software Engineering Education
• Text Analysis
• research-article
• Refereed limited

Acceptance Rates

Funding sources, other metrics.

• Bibliometrics
• Citations 0

Article Metrics

• 0 Total Citations View Citations
• 280 Total Downloads
• Downloads (Last 12 months) 280
• Downloads (Last 6 weeks) 46

This publication has not been cited yet

View or Download as a PDF file.

View online with eReader.

Digital Edition

View this article in digital edition.

HTML Format

View this article in HTML Format .

Share this Publication link

https://dl.acm.org/doi/10.1145/3593663.3593690

Share on Social Media

• 0 References

Export Citations

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Digital Transformation in Higher Education: A Case Study on Strategic Plans

2020, Vysshee obrazovanie v Rossii = Higher Education in Russia

Digital transformation is considered as an inevitable process for higher education systems like all socioeconomic institutions and systems. The digital transformation, which paradoxically incorporates both challenges and conveniences, has to become the focus of corporations' strategies. The aim of this study is to determine the status of digital transformation in universities' strategies. For this purpose, the strategic plans of 18 Turkish universities, which ranked at the top 1000 most often in the world rankings, were evaluated with the content analysis method. Findings indicate that expressions about the components of digital transformation in the strategic plans of the universities were gathered under 4 themes, 14 categories and 35 codes. The expressions of the universities about digital transformation are coded under the category of diversity and flexibility of learning technologies, especially education theme and distance/open learning. It is observed that universities have the least digital transformation strategies concerning research and social service missions. In this sense, it was concluded that universities could not perform digital transformation beyond technological infrastructure renewal into an integrated transformation model and strategic vision. The results of the study were compared with empirical and theoretical studies in the literature. For universities and future studies, it was proposed that Turkish universities are compareable with the universities abroad, which show the successful examples of the digital transformation, and that the quantitative and/or qualitative methods related to the subject can be applied by internal and external stakeholders, especially in the sector's evaluations.

Related Papers

Education and Information Technologies

Issam Tlemsani

IJAR Indexing

Over the past few years Higher Education has felt the influence of different technological and social trends towards digitalization which have led to many teaching innovation projects based on digital technologies. These innovations, far from being isolated and static phenomena, ought to be seen as instances of a genuine digital transformation process of Higher Education. Yet it hasn’t just been Higher Education; many other sectors such as the press, banking, television or the music industry have also been affected by digital transformation processes which in many cases have been disruptive. The main argument of this article is that a study of e-innovation in University teaching should be approached within a complete analysis of the digital transformation taking place at Higher Education Institutions, which are highly complex organizations. It is only through adopting this organizational perspective that one can gain a complete vision and overview of the changes and challenges universities are facing. These days digital transformation is a strategic priority for many business organizations. Is this also the case for universities? In order to try and answer this question the article will take a qualitative approach based on documentary analysis. It will look at the results from the analysis of the case of a Spanish university and shows evidence supporting the argument that universities have a conception of digital technologies that is not strategic enough, merely as tools.

Elinda Kajo Mece

Higher Education Institutions (HEIs) are involved in an evolution to a new model of university called digital university. This model implies not only adopting new technologies but also developing an organizational strategic transformation which includes information, processes, human aspects, and more. Because an organization’s digital maturity correlates with the scope of its digital transformation efforts, this study aims to identify digital transformation initiatives (DTI) taken by HEIs, defining the new processes and technologies used to implement them. The main motivation is to have a real and clear vision of how universities are transforming themselves, discovering the most relevant DTI that they have applied and if they are doing it through an integrated plan aligned with a digital strategy, as recommended by experts. We conducted a Multivocal Literature Review, as methodology research, to include both academic and grey literature in the analysis. Main results show that the DT...

Human, Technologies and Quality of Education, 2022

Galina Robertsone

Digital transformation (DT) is rapidly penetrating all spheres of human life, and higher education is no exception. This process is inevitable and ensures competitive advantage and other benefits for Higher Education Institutions (HEI) in case of success. Due to the COVID-19 pandemic, HEIs worldwide were forced to completely transform their working methods and go digital in a very short period. Some institutions are more successful in this transformation by possessing the ability to overcome DT challenges and combining internal and external success factors. This research aims to identify what drives digital transformation in Higher Education Institutions, what benefits are there for them, what challenges they need to overcome, and what are the success factors of digital transformation in higher education.

ECOLHE (Empower Competences for Onlife Learning in HE) International Conference Proceedings

Emanuela Proietti

Some studies show that most European HE institutions haven't made much progress in changing the courses they offer to a student centred learning model that can take into account developments and opportunities in technology-enhanced education. Challenges posed by digital transformation to universities do not regard only teaching and learning processes. There are different levels of institutional and organizational action which produce effects on these processes. The paper presents some results of a part of the field research of the Erasmus+ Project ECOLHE. Six case studies have been carried out. They have aimed to investigate how the universities involved develop their strategic approaches to digitalisation. The results presented refer to the focus groups conducted in 2021.

Martín Serna 😎

Higher education institutions (HEIs) have been permeated by the technological advancement that the Industrial Revolution 4.0 brings with it, and forces institutions to deal with a digital transformation in all dimensions. Applying the approaches of digital transformation to the HEI domain is an emerging field that has aroused interest during the recent past, as they allow us to describe the complex relationships between actors in a technologically supported education domain. The objective of this paper is to summarize the distinctive characteristics of the digital transformation (DT) implementation process that have taken place in HEIs. The Kitchenham protocol was conducted by authors to answer the research questions and selection criteria to retrieve the eligible papers. Nineteen papers (1980–2019) were identified in the literature as relevant and consequently analyzed in detail. The main findings show that it is indeed an emerging field, none of the found DT in HEI proposals have ...

Development of Digital Transformation in Higher Education Institutions

Shaikh A B D U L Hannan

Technological improvement has impacted Higher education institutions (HEIs) Industrialization 4.0 which has required organizations to embrace changes in the digital age in all aspects. Bringing digitalization concepts to such HEI area is a different concept that has attracted attention these days because it enables us both to elaborate complex relationships among actors in a technologically-supported education sector. This study has aimed to investigate the characteristics of the digitalization (DT) integrated system in HEIs. This is essential to thrive in a competitive environment and to satisfy the needs of electronic service users who do have technological capabilities. An analytical procedure and consent form were employed to collect research data for this study. The primary results demonstrate that it is a developing area, since none of the discovered DT in HEI projects have been established in a whole dimension. This has necessitated more studies on how HEIs can understand DT and handle the new requirements exacted by the fourth industrial transformation.

International Journal of Information and Education Technology

Sri Suning Kusumawardani

For higher education institutions that encourage digital transformation, understanding the barriers are necessary for the digital transformation accomplishment. The purpose of this paper is to present a review of the literature on barriers to digital transformation in higher education. To get a wide overview in identifying the barriers to the implementation of digital transformation, a structured literature review was used to select the relevant studies published. Nine categories were identified based on the literature reviewed: vision, strategy and policy, resources, leadership, digital skill and knowledge, technology, adaptability, resistance to change, and government and economic. Our findings provided a fish-bone diagram that outlines twenty-two barriers to digital transformation in higher education. The main contribution of this study is a synthesis of the state of the art of barriers to digital transformation in higher education. We contribute to provide a common basic underst...

International conference KNOWLEDGE-BASED ORGANIZATION

Cosmin TILEAGA

Digital transformation of higher education undergoes cultural, mental, and technological change as an overall process. The present paper represents a correlation of the current literature that brings to light the key pillars aspects of the digital transformation of higher education. The objective of this research study was to outline a model of digital transformation strategy applicable to the Romanian academic environment. The original contribution of this study lies in establishing the key pillars elements and necessary stages for the implementation of digital transformation strategy for higher education which should be focused on students and effects.

IFAC-PapersOnLine

Ylber Limani

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Asian Journal of Distance Education

Junhong Xiao

Problems and Perspectives in Management

hamza khraim

Imre Petkovics

Education Sciences

Thomais Gkrimpizi

9th International Conference on Higher Education Advances (HEAd'23)

Carina Ström Hylén

Dumitrita Iftode

Jamshed Khalid , Mohamed Soliman

Issam Tlemsani , Mohamed Ashmel Mohamed Hashim

Journal of economics, finance and management studies

samira El ferouali , Said Ouhadi , Abdelali Ezziadi

2018 International Conference on Information and Communication Technology for the Muslim World (ICT4M)

Jamaludin Ibrahim

Sudan Journal of Medical Sciences

Nazik Elmalaika O S A HUSAIN

5th International Scientific Conference: Is it Time for a Total Reset?

nino papachashvili

Sustainability

EMILIO ABAD SEGURA

Lecture notes in networks and systems

Pilar Marquès

Higher Education Studies

chanin tungpantong

European Higher Education Area: Challenges for a New Decade

laura malita

International Journal of Higher Education

Almaz Gapsalamov

EDULEARN proceedings

Martina Tomicic Furjan

International Journal of Innovative Research and Scientific Studies

Chanchira Laorach

Murilo Cardoso

Lecture Notes in Computer Science

Jorge Guadalupe

Proceedings of 6th International Scientific Conference Contemporary Issues in Business, Management and Economics Engineering ‘2019

Jelena Titko

Springer eBooks

cathrine Tømte

Luis Manuel Cerdá Suárez

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Elsevier - PMC COVID-19 Collection

Emergency remote teaching and students’ academic performance in higher education during the COVID-19 pandemic: A case study

Santiago iglesias-pradas.

a Departamento de Ingeniería de Organización, Administración de Empresas y Estadística, ETSI de Telecomunicación, Universidad Politécnica de Madrid, Spain

Ángel Hernández-García

Julián chaparro-peláez, josé luis prieto.

b Department of Electronic Physics, Electric Engineering and Applied Physics, ETSI de Telecomunicación, Universidad Politécnica de Madrid, Spain

The COVID-19 pandemic has caused a massive disruption in the way traditional higher education institutions deliver their courses. Unlike transitions from face-to-face teaching to blended, online or flipped classroom in the past, changes in emergency remote teaching –a temporary shift of instructional delivery to an alternate remote delivery mode due to crisis circumstances– happen suddenly and in an unplanned way. This study analyzes the move to emergency remote teaching at the School of Telecommunication Engineering (Universidad Politécnica de Madrid), and the impact of organizational aspects related to unplanned change, instruction-related variables –class size, synchronous/asynchronous delivery– and use of digital supporting technologies, on students' academic performance. Using quantitative data of academic records across all (N = 43) courses of a bachelor's degree programme in Telecommunication Engineering and qualitative data from a questionnaire delivered to all (N = 43) course coordinators, the research also compares the academic results of students during the COVID-19 pandemic with those of previous years. The results of this case study show an increase in students' academic performance in emergency remote teaching, and support the idea that organizational factors may contribute to successful implementation of emergency remote teaching; the analysis does not find differences across courses with different class sizes or delivery modes. The study further explores possible explanations for the results of the analysis, considering organizational, individual and instruction-related aspects.

1. Introduction

In 2020, the impact of COVID-19 has been noted in practically all areas of activity, but its effect has been particularly strong in teaching and learning. The pandemic has shaken up the landscape of higher education worldwide, with responses to the pandemic from higher education institutions generally falling into three categories ( Hodges, Moore, Lockee, Trust, & Bond, 2020 ; Smalley, 2020 ): maintaining in-class teaching with social distancing, creating hybrid models (blended learning, limitation of students in campus) or moving to online instruction.

In Spain, where the predominant teaching modality across universities is face-to-face learning –only 15 percent of Bachelor's Degree students are enrolled in public or private distance education universities ( EDUCAbase, 2020 )–, the declaration of the state of alarm in the nation and the enforcement of total lockdown by national authorities ( Real Decreto 463/2020, 2020 ) in an attempt to control the virus spread forced all face-to-face universities to move to online instruction, which required changing the teaching methods and resources to adapt them for distance education.

Three days before –March 11, 2020, coinciding with the declaration of the COVID-19 as a pandemic ( World Health Organization, 2020 )–, all face-to-face education activities were suspended in the Autonomous Community of Madrid ( Order 338/2020, 2020 ). This suspension occurred barely four weeks into the second semester. With over 98 percent of undergraduate students attending face-to-face learning in Madrid ( EDUCAbase, 2020 ), the impact of the pandemic was even more dramatic. Mostly unprepared, higher education instructors had to make the necessary changes and adjustments overnight. Of course, this situation has not been exclusive to Spain, as Crawford et al. (2020) show in their review of responses to the COVID-19 situation across 20 different countries, noting that the move to online teaching has occurred only in some cases –mostly European countries ( Crawford et al., 2020 ).

Existing literature on the transition from face-to-face teaching to blended, online or flipped classroom learning study these changes under the premise that the instructional changes are carefully planned by the instructors. Most often, the move to online teaching is carried out voluntarily by the teaching staff with help from support personnel. It is a process that takes both resources –human, intellectual, technical– and time: it is estimated that adapting a typical course to online teaching (including planning, preparation and development) takes between six and nine months ( Hodges et al., 2020 ). However, research on how to make these sudden transitions rapidly, and the potential effects of the decisions taken by institutions and instructors regarding the use of different instructional methods or supporting technologies, means (or at least meant before COVID-19) venturing into uncharted territory.

When sudden transitions as a response to a crisis occur, coordinated measures would take too long to put into place, especially when the rigidity of bureaucracy constrains the ability to change by establishing rigid rules ( Haveman, 1992 ). Therefore, the decision on which adaptation strategy to make is left to individuals; in this case, instructors who have to swiftly select among multiple digital tools with different capabilities to support teaching while balancing their workload. More particularly, some of these decisions include the support of asynchronous –e.g. content management systems, message boards, e-mail, pre-recorded videos of class sessions– or synchronous –e.g. chat, videoconferencing or real-time collaboration systems, instant messaging– tools and may even involve changes in the assessment activities or assessment criteria. This research investigates the impact of such choices in academic performance to identify successful transitioning strategies.

As a word of caution, it has been argued by experts that the teaching modality offered as a result of the transition to digital spaces caused by COVID-19 cannot be labelled as ‘online learning’, and thus a new concept has emerged to define the new situation: ‘emergency remote teaching’ ( Hodges et al., 2020 ; Milman, 2020 ; Rapanta, Botturi, Goodyear, Guàrdia, & Koole, 2020 ; UoPeople, 2020 ). Hodges et al. (2020) indicate that the main difference between online learning and emergency remote teaching lies in that online learning results from careful instructional design and planning, requiring an investment in a whole ecosystem of learner supports that takes time to build, whereas emergency remote teaching emerges as a response to a crisis and entails a temporary shift of instructional delivery to an alternate delivery mode that involves the use of fully remote solutions for instruction that would otherwise be delivered using face-to face, blended or hybrid courses. The key term then is 'temporary', as emergency remote teaching assumes that teaching will return to the original format once the crisis ends.

If we observe the changes in teaching and learning caused by COVID-19 under the lens of emergency remote teaching, it could be argued that most, or at least part, of the findings from existing research on online learning might not be applicable to this situation. However, it is also true that the decisions that instructors have had to make to deliver their courses are not that different from the intervention features ( Means, Bakia, & Murphy, 2014 ) or learning design options ( Hodges et al., 2020 ) they have to choose when they plan, design and implement an online course. Admittedly, some of these options, such as breadth (whole program, course, portion of course, brief episode) or modality (blended, semi-blended) are imposed by the pandemic situation and cannot be really chosen, but instructors do have some degree of freedom in their decision about the remaining ones (e.g. online communication synchrony, student-instructor ratio, role of summative assessments, etc.).

With these nuances under consideration, this research study aims to answer three main research questions:

• RQ1: How have instructors adapted their teaching of graduate courses to emergency remote teaching under the COVID-19 pandemic in a specific context (the School of Telecommunication Engineering at Universidad Politécnica de Madrid)?
• RQ2: Are there any differences in students' academic performance between the courses delivered in emergency remote teaching and traditional face-to-face courses?
• RQ3: Are there any differences in students' academic performance depending on the different instructional decisions made by the instructors in emergency remote teaching?

To answer these research questions, the study describes the experience of the changes undergone by the School of Telecommunication Engineering at Universidad Politécnica de Madrid as a result of the COVID-19 pandemic, explores the potential impact of different instructional decisions (online communication synchrony, number of students, digital technologies used) in the academic results of students enrolled in 43 Bachelor's Degree courses, and confronts these results with those from the two previous academic years.

The remainder of this document is structured as follows: Section 2 reviews the literature on organizational aspects that might affect the outcomes in cases of sudden, unplanned changes, as well as literature on differences in academic performance between online and face-to-face learning, and describes the main supporting technologies for emergency remote teaching. Section 3 frames the concepts developed in the previous section in the context of this research study. Section 4 details the method used in the empirical research, which is followed by the presentation of the data analysis and results in Section 5. Section 6 discusses the main findings from the research and aims to explain the results of the analysis, and Section 7 outlines the main limitations of the study.

2. Literature review

2.1. unplanned change: organizational aspects.

Unplanned change is the response to a need for action precipitated by unanticipated events or crises ( Knowles & Saxberg, 1988 ). To study unplanned change, we turn to the organizational ecology theory ( Hannan & Freeman, 1977 ), based on the theory of punctuated equilibria ( Eldredge & Gould, 1972 ) in evolutionary biology. The main idea of the ecological model of organizations is that organizations are subject to structural inertia, which limits their ability to rapidly adapt to changes ( Haveman, 1992 ). Under both the biological and organizational theories, sudden transformations are denominated punctuational changes, which operate under the structural reordering of environmental conditions.

From an organizational view, we may then ask: what factors may affect success when an organization faces punctuational changes? First, organizational change may be beneficial if two conditions are met: that the changes occur in response to dramatic environmental shifts (punctuational change), and that they build on established routines and competences ( Haveman, 1992 ). In the case of higher education institutions and the COVID-19 outbreak, the first condition is met, whereas the second depends on the decisions made by the instructors. These decisions depend on the familiarity of instructors with online learning and supporting technologies, as well as the compatibility of the new situation with the instructional methods that they have been using over the years. Haveman (1992) further emphasizes that under sudden environmental changes simply reproducing previous ways of doing things may lead to failure. Additionally, the degree of diversification in how the organization reacts –changes across three dimensions: the clients it serves, the goods and services it produces, and the technologies it employs– affects performance in a way that the more closely the new activities are related to the previous ones, the greater the probability of success.

The structure of the organization also has an impact on the potential success of the change. Literature on organizational change agrees on identifying bureaucracy as one of the primary barriers to the implementation of changes. While bureaucratic systems are effective in implementing planned change ( Knowles & Saxberg, 1988 ), they are generally not efficient in responding to unplanned or punctuational changes because of the numerous rules under which they operate and their natural tendency to resist to innovation ( Haveman, 1992 ; Knowles & Saxberg, 1988 ). A final important aspect to consider is the ability of an organization to foster the existence of an organic organization, based on informal relations and where communication is diffused among members in a non-centralized way, as such kind of organizations have higher capabilities to cope with changes ( Knowles & Saxberg, 1988 ).

2.2. Academic performance in face-to-face versus online learning

Keeping in mind the difference between emergency remote teaching and online learning described in the introductory section, but also the similar elements of decision when moving teaching to the online space, this subsection explores the differences found in prior research related to academic performance between face-to-face and online learning.

The study of the differences in student achievement –measured as student final grades– between face-to-face, blended, and online learning has been a central topic in educational research for decades. The results of these analyses vary and seem to be extremely dependent on the type of analysis and the sample of the study. For instance, the results from the analysis of single courses may offer interesting but anecdotal evidence of these differences due to many different potential confounding variables –e.g. Urtel's (2008) finding that students perform better in face-to-face instruction. As the number of courses under analysis increases, however, the results seem to confirm that students obtain higher grades in online learning compared to those in face-to-face instruction, even though the difference is negligible –e.g. Ladyshewsky's (2004) analysis of 9 course units or Cavanaugh and Jacquemin's (2015) study on 5000 courses.

An alternative view is offered by meta-analyses. Existing meta-analyses tend to support the idea that either the academic performance of students –as final course grades– in online learning is higher than in face-to-face courses, or there are no significant differences between both. For instance, Shachar and Neumann (2003) found that distance education outperforms face-to-face learning; Zhao, Lei, Yan, Lai, and Tan (2005) found no significant differences in student outcomes but also warned that a large number of factors vary from one study to another; Jahng, Krug, and Zhang (2007) found no significant differences for aggregated undergraduate and graduate courses, but also that graduate courses were significantly less effective in online learning than in face-to-face modes, while the opposite occurred in undergraduate courses; and Means, Toyama, Murphy, and Baki (2013) found that students performed modestly better in online learning conditions when considering both ‘pure online’ and blended learning, but the differences were only sustained when comparing blended and face-to-face learning –in other words, there were no significant differences between ‘pure online’ and face-to-face learning.

More recently, the focus has shifted to the comparison of blended and face-to-face learning, with results supporting higher achievement in blended learning ( Bernard, Borokhovski, Schmid, Tamim, & Abrami, 2014 ; Vo, Zhu, & Diep, 2017 ) and a moderating effect of different variables, such as the kind of computer support used, interaction treatments or whether the courses belong to STEM or non-STEM disciplines –higher effect is found in STEM disciplines ( Vo et al., 2017 ). From the above, we might expect either no significant differences between courses from previous years and the same courses delivered in emergency remote teaching, or a slight increase in performance in emergency remote teaching. A different result would probably indicate a failed implementation of emergency remote teaching and emphasize the need for careful planning when moving a course online.

2.3. Variables affecting academic performance in emergency remote teaching implementations

In order to find the most relevant variables for analysis, this research turns to the different options available for the design of online courses. Means et al. (2014) and Hodges et al. (2020) define nine dimensions that must be considered in online learning design. Some of these dimensions might not be applicable to emergency remote teaching when campuses are closed –e.g. instruction modality– whereas others cannot be decided upon by the instructors due to the need to comply with pre-existing learning guides –e.g. pacing– or are related to the instructional design and are difficult to be rapidly modified when the course has already started –e.g. role of online assessments, pedagogy or student role online. From the remaining four (student-to instructor ratio, synchrony, instructor role online and source of feedback), this research focuses on the former two, understanding student-to-instructor ratio as class size.

2.3.1. Class size

There is a reasonable agreement in what can be considered small, medium, or large class sizes in online learning. For instance, Hoyt and Lee (2002) and Benton, Li, Brown, Guo, and Sullivan (2015) differentiate between small (10–14 students), medium (15–34 students), large (35–49 students) and very large (over 50 students) classes, while Means et al. (2014) and Hodges et al. (2020) propose four different levels (fewer than 35 students, between 36 and 99, between 100 and 999, and over 1000). Therefore, there seems to be a consensus in that there is a difference between courses with 35 students of fewer, and courses with over 35 students.

In their analysis of a Technology and Education course, Tomei (2004 ; 2006) found that the ideal class size of the course was smaller in online learning than in face-to-face settings −12 students and 17 students, respectively–, and that online learning demanded at least 20 percent more instructor time and workload than traditional instruction. Later on, based on their review of 20 studies on class size in online learning, Taft, Perkowski, and Martin (2011) argued that class size may depend on the educational framework –constructivist-objectivist continuum, community of inquiry model and Bloom's taxonomy–, but in most stances the optimal number lies in the range from 15 to 30 students; the authors also suggest that classes larger than 30 students would resemble the characteristics of one-way student-instructor communication in traditional settings.

Even though the results of the different studies under analysis do not indicate differences in academic performance, they do agree in that online learning increases the instructor's workload, a factor that must be accounted for in emergency remote teaching because instructors generally require additional time to adapt not only to the shift in their instruction, but also to the characteristics of a new workplace, very likely far less adequate than their offices and classrooms. Finally, Burch (2019) adds that class size may be related to a student's outcomes due to the positive relationship between student participation and outcomes, even though these results must be taken with some caution, as they were only observed in medium and small class sizes –under 30 students– ( Parks-Stamm, Zafonte, & Palenque, 2017 ) and they were also contingent on instructor participation. Based on the above, we would expect to observe worse average academic performance in larger courses.

2.3.2. Synchrony

There are two essential modes of instruction when considering synchrony: synchronous and asynchronous teaching, even though a mix of both may be possible –e.g. Yamagata-Lynch (2014) . The difference between the two modes lies in that in synchronous online teaching the instructor and students are physically separated but communicate in real time, whereas in asynchronous online teaching the separation is both spatial and temporal ( Roblyer, Freeman, Donaldson, & Maddox, 2007 ). Examples of the former include the use of videoconferencing software or chats, whereas the latter includes the use of tools such as message boards or pre-recorded videos and presentations, with varying degrees of interactivity. In a way, synchronous courses have a higher resemblance to face-to-face classroom instruction in that students and instructors meet in the same place at the same time, which is not generally the case of asynchronous online learning ( Bernard et al., 2004 ). Asynchronous courses, on the other hand, have other benefits, such as allowing students to have a self-paced approach to the course.

Research on the influence of synchrony in distance or online learning does not seem to agree on which one is more effective when considering student outcomes. Bernard et al. (2004) conducted a meta-review of 232 studies and concluded that, when compared to traditional teaching, asynchronous distance education rendered substantial better outcomes, even though at an expense of higher drop-out rates. However, subsequent studies showed no differences between synchronous and asynchronous teaching delivery modes ( Roblyer et al., 2007 ), pointing out that both types are effective in delivering online teaching, even though students show preference for synchronous course sessions ( Skylar, 2009 ). More recent studies observe differences in higher order thinking skills developed via student social constructivism, in favor of asynchronous teaching ( Brierton, Wilson, Kistler, Flowers, & Jones, 2016 ). The reason for this difference may lie in that asynchronous learning offers more flexibility –self-organization, more time for reflection–, which is why it may yield better results in adult learners (perhaps in combination with optional synchronous sessions), while younger students may benefit more from a structure of required synchronous sessions ( Hodges et al., 2020 ).

2.4. Supporting technologies in emergency remote teaching

The choices in the digital tools available for emergency remote teaching are as varied as the number of possible pedagogical approaches and learning contexts and applications –see Hernandez-de-Menendez and Morales-Menendez (2019) for an overview of current software tools to support educational processes. For every specific aspect of learning, it is very likely that diverse applications are available –as an example, Chaparro-Peláez, Iglesias-Pradas, Rodríguez-Sedano, and Acquila-Natale (2020) evaluate nine different software applications for peer assessment. Before implementing any tool in their pedagogical practice, instructors typically take their time in evaluating the functionalities, operation, installation, and usability of the different range of technologies that may be most effective in their courses. However, in the case of emergency remote teaching, there is little room for testing due to the urgency of moving online in a very short span of time, and therefore instructors tend to turn to what they already know and the tools they have in place before the crisis ( Dill, Fischer, McMurtrie, & Supiano, 2020 ).

In his review of the response to COVID-19 by US and South African universities, Chaka (2020) finds that there are two main types of online tools and resources that have been widely adopted across all institutions: learning management systems (LMS) and video conferencing platforms. Among LMS, Canvas and Blackboard were the most used online tools in the US, and Moodle was predominant in South Africa, which suggests that instructors –and the university at a higher level– chose to resort to their directly available digital platform to support educational processes in the first place. Regarding video conferencing tools, Zoom stands out as the most used tool, followed by other options with collaborative approaches, such as Blackboard Collaborate, Microsoft Teams or WebEx. Despite the prominence gained by these tools during the pandemic, their use in educational settings is not new; for example, McCoy (2015) reports the use of Zoom by doctoral students, and Macaulay and Dyer (2010) detail their experience with the implementation of WebEx in a pilot program to introduce interactive web conferencing in courses at Towson University. Microsoft Teams was launched at the end of 2016, but recent research has already addressed its use in educational settings –e.g., Poston, Apostel, and Richardson (2020) .

Other digital tools such as cloud-based file repositories –e.g., Google Drive, Dropbox, Microsoft OneDrive–, messaging platforms –e.g., WhatsApp, Telegram– or social networking sites seem to have had lower adoption rates –even though it is very likely that their use has been commonplace among instructors who were already using them in their teaching and among universities that rely on cloud-based technological infrastructure and applications provided by companies like Microsoft and Google. Based on this evidence, this study analyzes whether the instructors’ choices of digital tools may have any relationship with academic outcomes.

3. Research setting

This study focuses on the changes implemented in the bachelor's degree in Telecommunication Engineering at the School of Telecommunication Engineering (Universidad Politécnica de Madrid). In order to provide contextual background to the reader, this section presents an overview of both the organizational structure of the institution and the degree program to establish a correspondence with the different aspects detailed in the literature review that helps explain the results of the analysis. This section also details the timeline of events to contextualize the decisions made by the instructors during the pandemic and provide further insight about the impact of the implementation of emergency remote teaching at the School of Telecommunication Engineering.

3.1. Organizational structure and overview of the program

Universidad Politécnica de Madrid is a technical-oriented higher education institution that focuses on the different fields of engineering; the only degree programs offered besides engineering bachelor's and master's degree programs are Physical Activity and Sports, and Fashion Design. The university had more than 37,000 graduate and undergraduate students enrolled in official degrees in 2019 ( Servicio de Biblioteca, 2020 ) and is organized around 17 schools and faculties and 10 research centers and institutes across four different campuses. Universidad Politécnica de Madrid operates under a semi-decentralized structure, with all high-level decisions made by the rectorate, while giving a great degree of autonomy to the different schools –an exception to this would be the institution's financial management, which is mostly centralized.

Schools, and more specifically the School of Telecommunication Engineering, are divided into departments; some departments are divided into units spread across different schools. The relationship between departments and schools is similar to that of the university and schools; therefore, departments have some degree of operational autonomy, especially regarding instructional decisions. Bureaucratic tasks for instructors are mostly limited to quality assurance processes. The School of Telecommunication Engineering currently offers three undergraduate and nine graduate degree programs. Each programme establishes coordination mechanisms with regular meetings of course coordinators at programme and year levels, which facilitates the flow of information regarding instructional practices both formally and informally. Business practices are a second source of informal knowledge: a distinctive characteristic of the School, especially when considered at the national level, is the proximity of faculty to leading business companies in the information technology sector, be it for educational collaboration purposes, research projects or supervision of student internships. This proximity facilitates the acquisition of information about how companies develop collaborative and training practices, as well as what software applications instructors can incorporate into their teaching practice.

Technical infrastructure and digital supporting tools for education are provided by the university's Distance Education Bureau, which offers services such as the campus wide LMS (Moodle, which was already used prior to the COVID-19 crisis in the large majority of courses), production of multimedia resources, online learning consulting and virtual labs Gabinete de Tele-Educación, 2020 . Other relevant software tools available for all students and instructors include the Microsoft's Office 360 suite and Blackboard Collaborate. In response to the crisis and to prevent system and network overload, an additional instance of Moodle was created only for examination purposes, and Zoom licenses were acquired for the different departments. However, and given the decentralized nature of the university, instructors could choose to use any other technology they deemed convenient for their courses, at the cost of not receiving official support.

The program under analysis in this study is the bachelor's degree in Telecommunication Engineering. It is a four-year degree where the first three years include core courses that are common for all the students; in the fourth year the students specialize in one out of four disciplines. Additionally, elective courses are offered to obtain the necessary credits to complete the studies. All courses are delivered as face-to-face courses. Each academic year 300 new students are admitted in the program, and at any time during the academic year the degree hosts over 1500 students (2017–18: 1528; 2018–19: 1533; 2019–20: 1536). Around 10 percent of the students abandon the program; dropout tends to occur mainly during the first year, and the average time to complete the program is between 5 and 6 years. Due to the technology-intensive nature of the degree, most –if not all– students are proficient in the use of digital technologies, own laptops/tablets and smartphones, and have wireless connection at home, which limits the potential impact of the COVID-19 due to socio-economic differences and the digital divide.

3.2. Timeline of events

In order to get a more nuanced picture of the responses to the pandemic, it is necessary to explain the conditions under which the emergency remote teaching was implemented. In the second semester, courses started on the week of January 29, 2020. By that date, no infection cases had been reported in Spain yet. The first reported case of COVID-19 occurred in La Gomera, Canary Islands, on January 31, 2020 ( Linde, 2020 ); it was not until February 25, 2020, that the first positive case was reported in Madrid ( Ministerio de Sanidad, 2020 ), which prompted the COVID-19 protocol in the region. During the following days, the School faculty started informal discussions about how to better react to a potential pandemic outbreak; at this point, a complete move to online learning had been discarded.

On March 6, 2020, and considering the increasing number of cases in the region, the School Board of Directors sent an e-mail with a notification for an emergency meeting to all course coordinators and student delegates. In the meeting, celebrated on March 9, 2020, different scenarios were considered; a discussion followed about potential courses of action in case of total suspension of face-to-face instruction.

Only one day later, the Government of the Autonomous Community of Madrid published Order 338/2020 (2020) , which effectively declared the suspension of face-to-face instruction starting March 11, 2020. In March 14, 2020, the state of alarm was declared in the whole national territory ( Real Decreto 463/2020, 2020 ), resulting in total lockdown of the general population. A partial suppression of the lockdown during the next months, or de-scalation plan, was structured in four different phases ( Ministerio de Sanidad - Gobierno de España, 2020 ). Phase 0 could still be considered an effective lockdown, as the mobility of citizens was heavily limited. Fig. 1 depicts the chronology of events. From the figure, it is straightforward to note the very limited time for reaction that the pandemic left to students, instructors and course coordinators; it is also worth noting that faculty and students of the School of Telecommunication Engineering experienced effective lockdown for the whole duration of the course after the declaration of the state of alarm.

Timeline of events (above the horizontal axis, events relative to teaching and learning; below the horizontal axis, events related to regulatory aspects).

The study uses a sample of all (N = 43) the courses of the Telecommunication Engineering Bachelor's Degree at the School of Telecommunication Engineering (Universidad Politécnica de Madrid). Two different data sources are used in the study: instructional decisions made on the transition from face-to-face to online learning were collected from an open survey to course coordinators (open and close dates are indicated in Fig. 1 ); the second data source contains course-level aggregated student grades from the last three academic years (2017-18 to 2019-20). After inspection of the data sets, one of the courses had to be removed because all the teaching and grading was concentrated in the first month of the semester, and therefore no emergency remote teaching was implemented as the classes had already finished by March 14.

The qualitative survey asked course coordinators the following: (a) teaching methods used during in-class and off-class hours, (b) digital tools used to teach the course sessions, (c) number of students regularly following the course, (d) type of assessment activities, (e) tools used for student tracking, (f) likelihood to change the continuous assessment to final-exam only assessment, (g) whether the assessment criteria and/or system were changed, (h) student attendance (class size) during emergency remote teaching, and (i) general comments about emergency remote teaching and main problems encountered. All course coordinators answered the questionnaire.

The statistical methods used to test the differences include one-way repeated measures ANOVA to test for differences in academic performance across the past three years and independent t-tests to test for differences in final grades between courses in the second semester of the academic year 2019–2020 across the following variables from the questionnaire sent to course coordinators: class size, synchrony, and digital tools used by the instructors. For the study, we established two different class sizes (small and medium, under 35 students; large, 36 students or more). A course was considered to be synchronous when it required students to be present and connected at a given time on a given platform for the course session –videoconferencing systems, chats–, and asynchronous when the instructors provided the materials –course documentation, external links, pre-recorded sessions– for learners to study at their own pace.

Finally, the grading data set contains the number of students who achieved a given mark in a course. Because the courses may differ greatly in size, and in order to make comparisons possible, we analyze the relative rates (passing rates and percentage of students achieving a specific grade) when comparing data from different academic years, and yearly variation of passing grades between the previous academic year (2018–19) and the results under emergency remote teaching (2019–20) to observe the effect of the choices in digital tools used as support during emergency remote teaching.

5. Data analysis

Firstly, we use R software (version 4.0.2) to plot the percentage of students passing each course versus the number of students participating in emergency remote teaching, differentiating between elective and non-elective courses ( Fig. 2 ). From Fig. 2 , every elective course except for two achieve a 100 percent pass rate; another characteristic of elective courses is that they typically may be considered small to medium regarding class size. Therefore, and to better explain the results of the analysis and gain useful insight, the analysis of the global data set will be complemented by separate analyses of the groups of elective and non-elective courses.

Number of students vs. percentage of students who passed the course (dot colors represent elective and non-elective courses).

Fig. 3 shows the number of courses according to class size and synchrony delivery type. From the figure, there is a balance between small-medium and large courses, and a slight difference in the delivery mode, with more instructors choosing to adopt synchronous teaching –which, in this case, could be considered as a direct translation of face-to-face content delivery in a virtual space. When electiveness is considered ( Fig. 3 , right-bottom), the difference in synchrony increases, with almost two thirds of the courses being taught synchronously. However, and as Fig. 4 shows, class size did not seem to determine whether the course was given synchronously or asynchronously, regardless of electiveness.

Number of courses by class size and synchrony delivery type (top), refined by course electiveness (bottom).

Number of students per synchrony delivery type across elective and non-elective courses.

For the test of differences, we used the R package ggstatsplot ( Patil, 2018 ), which provides support for repeated measures one-way ANOVA and independent t-tests, with the most common options for the analysis –e.g. parametric, non-parametric, adjustment type, report of the results of the analysis, etc.– and combines it with a graphical output. First, we performed the repeated measures ANOVA to test differences across the percentage of students who passed the second semester courses in the past three academic years (sphericity problems were discarded after observation of Mauchly's test results). The result ( Fig. 5 ) shows a significant increase (between 7 and 10 percentual points) in the percentage of the students passing the course under emergency remote teaching when compared to the previous two years (no significant differences were found between the 2017–18 and 2018–19 academic years).

Differences in percentage of students passing the course in the past three academic years (second semester courses).

When electiveness is accounted for ( Fig. 6 ), the analysis reveals no significant differences across elective courses, while the differences are sustained across non-elective (core) courses, which shows that the improvement in academic performance was caused by an overall increase in student outcomes across core courses.

Differences in percentage of students passing the course in the past three academic years across core/non-elective (left) and elective (right) courses (second semester courses).

Fig. 7 , Fig. 8 further explore the data on a per-grade basis across all courses and non-elective courses, respectively. From the figures, the number of students that had slightly above average or outstanding performance did not change significatively, but the number of students that had a very good performance (from 7 to 10 points out of 10, excluding outstanding students) did vary significantly. The cause of this result might be attributed to an overall shift in individual grades that would cause the usual normal distribution to move toward higher marks.

Differences in percentage of students in different grade ranges in the past three academic years (second semester courses).

Differences in percentage of students in different grade ranges in the past three academic years in core/non-elective, courses (second semester courses).

To discard potential confounding effects, we also explored the differences across first semester courses in the past three years, all of which were delivered as face-to-face courses. The analysis ( Fig. 9 ) shows stability in passing rates (around 80 percent) and no statistical differences across all three years, which suggests that the improvement of passing rates in the second semester could possibly be attributed to the effect of emergency remote teaching.

Differences in percentage of students passing the course in the past three academic years (first semester courses).

Regarding the influence of class size and synchrony delivery mode, the independent t-tests ( Fig. 10 , Fig. 11 ) show no significant differences in the variation of passing grades between emergency remote teaching and face-to-face remote teaching in the previous academic year, and therefore we cannot confirm a possible influence of these variables on academic performance in emergency remote teaching.

Differences in the percentage of students passing the course based on class size (left) and synchrony (right) across all courses (top) and only core, non-elective courses (bottom). The percentage reflects the variation, in percentage, of students who passed the course from the 2018–19 to the 2019-20 academic year.

Differences in the percentage of students passing the course based on the videoconferencing platform used across all second semester courses.

When considering the different digital tools used as support for emergency remote teaching, all the courses used the institutional LMS (Moodle) as support for content delivery and assignments; two courses did not use any tools other than Moodle. This result emphasizes the importance of having at least some minimum technical infrastructure to support digital remote teaching, especially as a pre-requisite for successful response in emergency remote teaching. Interestingly, and despite its integration with the institutional Moodle, no respondents used Blackboard Collaborate; the introduction of Blackboard Collaborate was relatively new to the university, and therefore it is highly possible that instructors have turned to videoconferencing tools with which they were more familiar, such as Zoom, Skype, Teams or Webex. The results of the analyses suggest that differences in variations of passing grades cannot be attributed to the choice of one software application or another, except in the case of Webex, where the differences are significant. However, this finding should be taken with caution, as only three courses incorporated Webex.

We also explored the data set in search of other potential variables of influence ( Fig. 12 ). Passing grade variation across the different years in the degree was considered of special interest, given that we could expect better academic performance among second-year to fourth-year students than among freshmen due to higher experience in the use of the institutional LMS, as well as better organization skills and better communication with instructors. Fig. 12 (left) shows a relatively improved performance associated with more advanced courses; however, the analysis finds that this improvement is not significant. Finally, Fig. 12 (right) shows the variation in passing grades in relation to the instructors' perceived students' attitudes; the results suggest that the improvement was higher in courses with worse perceived students’ attitudes, an unexpected result that should be further explored in the future.

Annual variation (from year 2018–19 to year 2019–20) of students passing the course depending on course year (from first to fourth) and students' attitudes (as perceived by the instructors).

6. Discussion

In answering RQ1, we did not observe a special preference for different teaching methods or digital tools, with the exception of the institution's LMS (Moodle, used in all courses) that gave support to course management. Regarding synchrony, nearly 60 percent of coordinators and instructors chose to continue their classes using synchronous teaching, mostly through videoconferencing tools –this percentage rose to two thirds of the courses when only core courses are considered. This result seems to confirm that, when facing punctual changes such as those caused by the pandemic, instructors seem to resort to digital tools that they are most familiar with –i.e., what ‘already works’– and with instructional methods that most easily resemble current practices –i.e., synchronous sessions that mimic face-to-face learning. Of course, time is an important variable to explain these results, as instructors barely had a week to prepare the move to online teaching.

The responses to the open questions in the questionnaire seem to confirm that, even though more than one quarter of the coordinators did not experience the transition as problematic, adaptation time was indeed one of the main problems encountered by the instructors: ten of them complained about the short period of time available to become familiar with the use of new digital tools and the changes in the learning processes. In words of these instructors: “The lack of awareness about all the possibilities and uses of online tools available is being a problem” , or “It is something new and different, and both faculty and students need to adapt. We are just becoming familiar with online teaching tools. Maybe other courses had already worked in this direction, but in our course we still used chalk and blackboard in face-to-face sessions”. Perhaps the most illustrative remark in this regard is that “remote teaching, when properly implemented and planned in advance, may be useful in some instances. In the case at hand, my overall impression is not relevant because there is no choice, we need to adapt. The main problem lies in that the time we had to move from face-to-face to online teaching has been very short” . All these statements emphasize the need for continuous training on the use of digital educational tools and their incorporation to traditional practices as a means to facilitate transition in times of crisis.

Regarding RQ2, the analysis reveals that the overall academic performance of students in emergency remote conditions was significantly better than traditional face-to-face instruction. Our results then seem to confirm, at a larger scale, those of Gonzalez et al. (2020) in a different university in Madrid. However, under that view one could argue that, at least from this experience, emergency remote teaching is a superior form of instruction to traditional face-to-face courses. Of course, this absolute interpretation is probably very far from reality, and it is difficult to think that every higher institution should be in a constant pre-crisis or crisis state to improve their teaching, or that they should just move all their teaching to online spaces.

Further, and answering RQ3, the results would also support that the choice of digital tools, delivery methods or class size does not have any relevance whatsoever in students’ outcomes in remote learning. If so, does it make any difference? What are the underlying causes of the results? What lessons can be learned from this study? While Gonzalez et al. (2020) conclude that the increase in academic performance may derive from an improvement in students learning strategies and self-regulation skills, we aim to go beyond and seek to find alternative explanations from organizational, individual (both of instructors and students) and instruction-related aspects.

6.1. Organizational aspects

The theory revised in our literature review may shed some light in explaining the results of the study from an organizational view. The first thing that must be noted is that the degree of diversification in the activities and processes has been relatively low: the clients served (students) and services delivered (teaching) remained the same, while only the technologies employed changed (and, in some cases, very slightly), which seems to confirm that the more related the activities to the previous ones, the higher the probability of success ( Haveman, 1992 ).

Second, it could be argued that the School of Telecommunication Engineering at Universidad Politécnica de Madrid was relatively well prepared for the crisis in terms of technical infrastructure; a fully functional instance of Moodle has been in place for several years now, and most courses regularly use their Moodle virtual spaces at least as educational content repository and asynchronous communication channel with students via message boards; even though the transition to emergency remote teaching posed some challenges in scaling the system to ensure quality of service for higher number of concurrent users (the LMS gives support to all degrees in the university), additional instances of the LMS were provided to support specific tasks, such as exams.

Third, the existence of formal and informal communication channels facilitated making faster and more informed decisions about the available options, despite the short time available for response. According to Knowles and Saxberg (1988) , these informal channels and an organic structure help successfully coping with changes. In this case, flexibility was further enabled by the School's Board of Directors, which established the necessary informal communication channels in early March in anticipation for the crisis, and therefore helped prepare possible responses. It is interesting to observe that this informal discussion was later transformed into formal communication prior to the moment of crisis. Even though the reaction time was too short –as per the statements of one quarter of the instructors–, the fact is that adjustment mechanisms had already been put in place.

Fourth, the federated or semi-decentralized structure also seems to have favored a rapid response: as instructors felt free to decide on which digital tools and what instructional design they implemented in their courses, no time was spent in bureaucracy and compliance with decisions that had to be made at higher levels. In this sense, the results suggest that flexible structures and rich informal information flows, together with a decent technical infrastructure and staff's technical literacy and innovativeness, may help succeed in facing a moment of crisis such as the COVID-19 pandemic. Interestingly enough, later on during the course a notification was sent by the university asking all instructors to only use officially approved institutional software for examinations; in our opinion, this might have had a negative impact on final grades if tools other than the ones mentioned in this study –already approved by the institution– had been adopted by the instructors, which was not the case.

6.2. Individual aspects

6.2.1. instructors’ digital skills.

Based on the results, we must also look into individual aspects that might help explain the findings from this study. A first aspect worth considering is a particular characteristic of the School: its strong technical orientation; because the School specializes in information and communication technologies and systems, most instructors are technology experts and use synchronous/asynchronous communication tools and learning virtual spaces on a daily basis. While we have not found supporting literature on the relationship between instructors’ digital skills and student achievement, particularly in higher education –most of the research on digital literacy of instructors focuses on the development of digital literacy skills or digital competence among pre-service teachers–, it is reasonable to think that it may have been a contributing factor to an effective and rapid deployment of emergency remote teaching. A good example of this is the adaptation of courses with a high workload in laboratory settings; without students being able to physically access the labs, the faculty teaching those courses opted to rapidly develop ad-hoc virtual simulation environments from scratch, something that would have never been possible without said digital skills.

6.2.2. Students’ digital skills and background

On the students’ side, there is prior evidence of the positive relationship between digital skills and academic performance ( Kim, Hong, & Song, 2019 ; Soleymani, 2014 ). There is an ongoing debate about the fact that being a digital native does not directly equate to being a digital learner –e.g., having developed digital competence in a formal or informal educational setting– ( Gallardo-Echenique, Marqués-Molías, Bullen, & Strijbos, 2015 ), and therefore we cannot make the a priori assumption that students have developed the necessary digital skills to succeed in an e-learning context, or that they have the necessary resources to even follow an online course –the pandemic has unveiled the problems caused by the digital divide in education ( Iivari, Sharma, & Ventä-Olkkonen, 2020 ; Zhong, 2020 ). However, in our case, that assumption seems reasonable because (1) young people with the highest levels of digital competence tend to be on courses involving ICT and are more favorable predisposed to use digital tools ( Sánchez-Caballé, Gisbert-Cervera, & Esteve-Mon, 2021 ); and (2) most, if not all students are proficient in the use of digital technologies and have their own devices –smartphones, tablets, desktop and/or laptop computers– that they already use in face-to-face courses to take notes and complete their assignments 1 . Additionally the degree programme is strongly focused on STEM matters and the majority of students have already specialized in STEM during their secondary education; this focus on STEM matters may have contributed to the improvement in student outcomes, confirming the findings of Vo et al. (2017) in blended learning.

6.2.3. Procrastination and anxiety

Procrastination and anxiety are well-known detrimental variables to academic achievement in online learning, with the former two affecting the latter ( Cormack, Eagle, & Davies, 2020 ; Frazier, Gabriel, Merians, & Lust, 2019 ; Kim & Nembhard, 2019 ; Pascoe, Hetrick, & Parker, 2020 ; Sanchez-Ruiz & El Khoury, 2019 ). The outbreak of the COVID-19 pandemic may surely have amplified their relevance in the academic achievement of students, but in a more nuanced way than it might seem wherever lockdown has been enforced.

There is enough evidence that the stress experienced by students –and, let us not forget, also by instructors– has increased during the pandemic ( Elmer, Mepham, & Stadtfeld, 2020 ; Son, Hegde, Smith, Wang, & Sasangohar, 2020 ). The pandemic lockdown stress is also closely related to anxiety, loneliness and depression ( Misirlis, Zwaan, & Weber, 2020 ), and it is therefore a contributing factor in a potential decrease of students’ academic performance. Additionally, different reports have confirmed important changes in consumption habits of Gen Z-ers during lockdown ( Hawthorne-Castro, 2020 ; Jones, 2020 ), especially in social media, online gaming, and online video and TV/video streaming services, all of which favor procrastination.

We have not tested students’ attitudes and behaviors in this study, and therefore we cannot assess the potential negative impact of these variables, but future research should also consider how the context of the lockdown may have softened their effect. For example, the higher time devoted to digital entertainment may have been compensated by the inexistence of commuting time –on average, between one and two hours in Madrid– and any other social or leisure activities that could not be carried out due to the lockdown. Besides, one of the most usual mechanisms of Gen-Z-ers during the lockdown to cope with stress, anxiety and loneliness has been the use of videoconferencing tools –Zoom being among the most popular, with an increase in use of almost 5000 percent in Spain between March 9 and April 20 ( Cuesta, 2020 )– to stay in touch with friends and family; a side effect of the wide adoption of these tools is that their use in the courses delivered synchronously may have been perceived as a natural extension of the campus life and face-to-face courses, which might have had a positive effect on learning.

6.3. Aspects related to learning instruction

Some factors relating to the (forced) changes in the instructional design of the courses may also help explain the results of the study. Most coordinators stated that they had to make changes in the different assignments that students needed to complete to pass the course. In the degree, continuous assessment –which comprises multiple individual or team graded assignments and/or tests during the course and, optionally, a final exam– is the default type of assessment, unless students opt for final examination-only assessment; due to the pandemic, many of these continuous assessment assignments were either delayed, simplified –likely reducing the difficulty level– or directly removed.

This decision had two important consequences. First, students have had higher flexibility to take self-paced learning (it was not unusual in face-to-face instruction that students put more effort in preparing some courses than others depending on the due date of intermediate assignments in the different courses). Therefore, students have found themselves in a better position to organize their own study time and pace, including adjusting for the mix of synchronous and asynchronous delivery of the sessions.

Second, assessment activities were constrained by technical, time-related, and even regulatory factors. From the comprehensive map of assessment scenarios in emergency remote teaching by García-Peñalvo, Corell, Abella-García, & Grande, 2020 , the available options have been mostly limited to different types of questionnaires, delivery of documents and oral presentations using videoconferencing systems. While many intermediate graded activities may include some of the former, the structure of the typical final exam in the degree 2 is very difficult to translate to an online context ( Keijzer-de Ruijter & Draaijer, 2019 ), unless students are allowed to submit a digitized copy of a hand-written exam on paper, which may add technical complexity and be more time-consuming for both students and faculty. Consequently, some courses have turned to multi-choice question tests. These tests may have been perceived as easier by students, and are better suited to assess knowledge rather than skill ( Hettiarachchi, Balasooriya, Mor, & Huertas, 2016 ), which makes it somewhat problematic to compare the results obtained under lockdown with those of previous years.

Finally, we should also consider the possibility of the existence of cheating behaviors. Despite the effort from instructors in taking measures to prevent cheating when designing their exams, the put in place of an institution-wide code of honor for online examinations and the use of plagiarism detection software (Turnitin), proctoring was restricted due to privacy issues. Previous literature supports the idea that students perceive cheating to be easier and more prevalent in online courses, and that unproctored remote exams include more cheating behaviors than proctored ones ( Clark, Callam, Paul, Stoltzfus, & Turner, 2020 ), and therefore we cannot discard the potential effect of dishonest behaviors among some students, a result also observed by Balderas & Caballero-Hernández, 2020 in online exams in a Computer Science and Engineering course during the pandemic in Spain.

6.4. Additional considerations

For a better understanding of the unplanned move to online teaching, we also summarize the course coordinators’ perceptions about the change to emergency remote teaching, focusing on two different aspects: overall perception of emergency remote teaching and main problems encountered in the change process.

6.4.1. Course coordinators’ perception of emergency remote teaching

In general, the overall impression of coordinators about the move to remote teaching under the pandemic is positive (48.8 percent), albeit nuanced. The coordinators find value in online learning, especially from the students' perspective. For example, one instructor states that “[…] students like it. They suggested the use of [Microsoft] Teams, and I know that they are satisfied and have suggested the same to other instructors” ; another instructor's comment in this line is that “[emergency remote teaching] is at the same level of acceptance [among students] than offline classes” . Instructors also perceive that students find it useful to be able to revise the content of the session at a later moment, which complements their technical notes.

Other coordinators who have a positive impression also find value in how the move to online teaching has made them reflect about their own teaching. As one coordinator says: “[Online teaching] may be a good complementary tool that may help us reflect about its true value in traditional teaching” . Other coordinators have incorporated this reflection during the implementation of the changes as a result of the first days of their teaching; for example, one coordinator questioned that “Maybe we are heavily leaning toward keeping synchronous learning (in class time), when a good planning of the activities (asynchronous, giving some freedom to students) with a correct control, monitoring and feedback (synchronous in online office hours or asynchronous –correcting and marking, where the [student's] work lies–) may work out very well” . This reflection about synchrony was also shared by other instructors, most of which coincidentally opted for asynchronous delivery modes.

Interestingly, 10 coordinators (23.3 percent) expressed their concerns about student participation and engagement in the course. Comments like “So far, low active engagement of students” or “As of now, I have noted a decrease in participation” illustrate this feeling. However, when observing the data, most of these courses are delivered asynchronously, which suggests that students may prefer to engage actively in synchronous sessions. A possible explanation is that it takes less effort for students to verbally participate in a videoconference than to develop their ideas in writing on an e-mail or message board, with the added benefit of instant feedback in the case of the former. Finally, other 10 coordinators stated that moving to online learning takes time to adapt, and 3 coordinators stated that online learning cannot be a replacement for face-to-face instruction.

6.4.2. Main problems encountered

Twelve coordinators (27.9 percent) did not seem to find any important problems with their adaptation to emergency remote teaching. Among the remaining coordinators, four categories of issues were raised: the first one (10 coordinators, 23.3 percent) groups different objections about the short time required to adapt to the new processes and tools that support emergency remote teaching –e.g., “[Online teaching] requires a training that has a long learning curve and great initial effort” –; the second one (10 coordinators, 23.3 percent) is related to technical problems with the different videoconferencing platforms supporting synchronous sessions –e.g., disconnections, high latency– and with the LMS –maximum file size, uptime and service availability–; the third one (6 coordinators, 14.0 percent) focuses on low student participation, engagement and motivation, including poorer immediate feedback due to lack of visual contact and social presence; finally, the fourth category (3 coordinators, 7.0 percent) relates to a loss in the experimental aspects of learning, which has an impact on courses involving a high amount of laboratory sessions.

7. Conclusion

The present study analyzed the move to emergency remote teaching in all the courses in a bachelor's degree in Engineering and its effects on students' academic performance. The study of the effects of the COVID-19 pandemic in higher education and the implementation of emergency remote teaching has gained interest among scholars, as this special section in Computers in Human Behavior and special issues in other journals evidence –e.g., Reynolds and Chu (2020) , and many others under development. Our research study, while arguably limited in scope to one institution and one degree program, has some distinctive characteristics to offer a significant contribution to this new field of knowledge.

From a theoretical approach, the study incorporates organizational aspects, based on the notion of punctuational change in organizational ecology, that may affect successful implementation of emergency remote teaching. The analysis also provides evidence of similar results to those of existing research comparing planned online/blended learning and face-to-face instruction: the findings from this study suggest that class size, the choice of synchronous and asynchronous delivery and the choice of virtual communication tools do not have a significant effect on students’ academic performance.

The study finds that students achieved better results under emergency remote teaching. As mentioned in Section 6 , while counter-intuitive, this result confirms, across a larger number of courses, the findings of Gonzalez et al. (2020) in a very similar albeit smaller context. Given that both studies were conducted in a region with strict lockdown during the pandemic, it would be of utmost interest to compare the results with other implementations of emergency remote teaching in regions or countries with less severe lockdowns, or lack thereof.

From a wider perspective, the study seems to be in support of some aspects of the C ♭-model for both online and offline environments in higher education ( Sailer, Schultz-Pernice, & Fischer, 2020 ). While the C ♭ - model formulates a holistic and comprehensive framework that includes proximal and distal factors affecting students' learning outcomes, and therefore its scope exceeds by far the focus of this study, our conceptual framework, the results of this research and the explanations laid out in the discussion section do address some of the foundational blocks of the C ♭-model. For instance, the positive results found in this study suggest that distal factors –higher education instructors' knowledge, skills, and attitudes toward technology; their qualification; and institutional, organizational, and administrative factors, together with instructors' and students’ equipment and digital skills– do have an effect on student outcomes. 3

The study also offers interesting implications for teaching practice. First, the results suggest that organizational readiness –technical infrastructure and support, flexible structures that facilitate decision-making and empower instructors, the availability of informal communication channels, and development of digital skills of faculty members– have a positive effect when rapidly adapting teaching in the context of a crisis or change of paradigm. Higher education institutions should pay careful attention to these aspects if they seek to be able to quickly respond to environmental changes while sustaining the delivery of high-quality education.

Second, the results highlight that successfully moving to online learning –or, in this case, emergency remote teaching– goes beyond the mere choice of a specific technology. The study did not find significant differences between the different digital tools used in the courses. Currently, the range of software applications to support learning is so wide that instructors might do well valuing compatibility with learners’ –and their own– practices, both in terms of familiarity with the software and its fit with the instructional approach, over other aspects when considering the use of a digital tool.

Third, the choice of delivery mode did not seem to affect students’ academic performance. While this result would suggest that this choice might also be left to instructors, previous literature ( Moallem, 2015 ; Oztok, Zingaro, Brett, & Hewitt, 2013 ; Xie, Liu, Bhairma, & Shim, 2018 ) suggests that a mix of both approaches –or bichronous online learning ( Martin, Polly, & Ritzhaupt, 2020 )– works better because it combines the benefits of both delivery modes –i.e., increased social presence and interactivity in synchronous online learning and self-paced learning and flexibility in asynchronous online learning. In fact, our findings could be the result of the combination of both methods, not within a course but across courses.

Fourth, we found that class size did not have an impact of academic performance. In our literature review, we showed that online learning generally benefits from small- or medium-class sizes; therefore, this result contends previous literature and should be further explored by future research.

7.1. Limitations

This study has certain limitations, of which the specifity of the context stands out as the most notable. As a case study, the results are specific of one institution –and in particular, of one engineering school– and one subject –a bachelor's degree in Telecommunication Engineering-, and the same applies to the conditions experienced during the course –the strictest lockdown among all European countries. The choice of the institution and the subject was made by convenience, and we acknowledge that its effect on organizational aspects –availability of technical infrastructure, organizational structure and processes, students' and instructors' digital skills, equipment and general positive attitudes towards the use of educational technology–, and therefore on the results, is not negligible in the least. In addition, as mentioned in the previous section, the strict lockdown also allows for a very nuanced view of self-regulated learning, especially under emergency remote teaching conditions. The combination of such factors might largely bias the results found in this study; therefore, we do not dare to claim universal validity of our findings, but rather present this case to allow for comparisons with other studies framed in contexts different than the one in this research.

A second limitation has to do with other elements of the C ♭-model ( Sailer et al., 2020 ) that are proximal rather than distal to student outcomes, such as the type of learning activities involved –e.g., we only considered delivery mode and supporting technologies rather than the type of learning activity performed–; even regarding student outcomes we just focused on a single aspect: the development of professional and knowledge skills. A more in-depth analysis of the factors affecting student outcomes should adopt a more holistic view of outcomes, including self-regulation, digital skills and attitudes toward digital technology, as well as a more detailed observation of the different learning activities.

A third limitation of the study is the omission of students’ views and perceptions of the process of moving to emergency remote teaching; such a perspective would offer further insight about the different aspects covered in this study and complement those that were left out of the scope of the research.

7.2. Concluding remarks

The COVID-19 pandemic caught the educational world by surprise, forcing higher education institutions to respond with different solutions overnight in a context of unplanned change. A second wave is coming, or has already arrived in some places 4 ; many higher education institutions will now extend, adapt or fine-tune their digital processes, and consequently instructors will now extend, adapt or fine-tune their instructional design. Shall it still be considered emergency remote teaching? Until when can this situation be sustainable or considered transitory? If the pandemic has proven something is that unplanned change, even when we find relatively positive results such as the ones in this study, should only be the seed of planned change.

Teaching will definitely change when (instead of if , hopefully) the pandemic is over, and the situation has been a wake-up call to higher education institutions about the need to integrate digital technologies into educational processes. It is time to talk about the digital transformation of education for good, because what may (temporarily) work in emergency remote teaching –e.g., offering a digital copy of the course content, replacing an hour of face-to-face class by a synchronous virtual room using videoconferencing systems, simply sending course materials for students to read, etc.– is definitely not the best way to make the most of the possibilities brought by digital educational technologies ( García-Peñalvo et al., 2020 ).

In the same vein, there is a big difference between emergency remote teaching and a real move to online/blended learning, with the key word here being emergency : all studies being conducted during the pandemic reflect a temporary response from instructors and institutions. It is impossible to sustain a constant state of emergency, and therefore the COVID-19 pandemic should be seen not as a fix before returning to the old ways but as an opportunity to improve digital readiness among higher education institutions. The sudden and temporary state of the changes seen in the delivery of instruction during the pandemic forced instructors to rely on readily available digital tools that facilitated fast adaptation, but a true digital transformation calls for integration of Industry 4.0 tools (artificial intelligence, robots, internet of things, educational data analytics) and rethinking of the teaching-learning process itself ( Bonfield, Salter, Longmuir, Benson, & Adachi, 2020 ; Koul & Nayar, 2020 ).

Acknowledgments

The authors would like to thank the Board of Directors of the School of Telecommunication Engineering at Universidad Politécnica de Madrid for their support and their collaboration in providing the aggregated data set of academic records used in this study.

Biographies

Santiago Iglesias-Pradas is MSc in Telecommunication Engineering, MBA and PhD in Information Systems by the Universidad Politécnica de Madrid. Santiago is Professor at the School of Telecommunication Engineering (UPM). He focuses his research on e-commerce, technology acceptance and learning analytics.

Ángel Hernández-García is MSc in Telecommunication Engineering, Master SAP in Integrated Information Systems, and PhD in Information Systems by Universidad Politécnica de Madrid (Spain). He is Associate Professor at the Department of Organization Engineering, Business Administration and Statistics (School of Telecommunication Engineering, Universidad Politécnica de Madrid). He focuses his research on electronic commerce, technology acceptance, social media and learning analytics. He has been guest editor and published research articles in leading international journals.

Julián Chaparro-Peláez is PhD in Telecommunication Engineering by Universidad Politécnica de Madrid (Spain) and Professor at the Department of Organization Engineering, Business Administration and Statistics (Universidad Politécnica de Madrid, Spain). His research interests include management information systems, electronic commerce and digital transformation of organizations.

José Luis Prieto is MSc in Physics by Universidad Complutense de Madrid and PhD by Universidad Politécnica de Madrid. He has been Research Associate at the Department of Materials Science, Old Cavendish, in Cambridge University and Associate Professor at York University. He is currently Associate Professor at the Department of Electronic Physics, Electric Engineering and Applied Physics (School of Telecommunication Engineering, Universidad Politécnica de Madrid), Researcher at the Institute for Optoelectronic Systems and Microtechnology (Universidad Politécnica de Madrid) and Deputy Director for Coordination of the Bachelor's Degree in Telecommunication Technologies and Services.

1 While laptop ownership does not necessarily equate to higher academic performance, Reisdorf, Triwibowo, and Yankelevich (2020) note that ownership could be beneficial to nonowners.

2 This type of exam includes one or more engineering problems presented as cases where students are required to apply all the theoretical concepts, generally as a sequential process. The assessment then includes both the description of the process and the final result.

3 While the C♭-model builds on a multi-faceted view of students' learning outcomes (a composite of professional knowledge and skills, self-regulation, basic digital skills and attitudes toward digital technology), the focus of this study is just a single element, namely professional knowledge and skills, using course final grade as a proxy.

4 A comment by the authors: while this is unfortunate news for human mankind, it also represents an opportunity for ongoing research on sustained impact and short- and mid-term effects of the pandemic on higher education, broadening the knowledge acquired during the first wave. At the time of publication, we are already experiencing the third wave of the pandemic, according to experts.

• Balderas A., Caballero-Hernández J.A. Eighth international conference on technological ecosystems for enhancing multiculturality. 2020. Analysis of learning records to detect student cheating on online exams: Case study during COVID-19 pandemic; pp. 752–757. [ CrossRef ] [ Google Scholar ]
• Benton S.L., Li D., Brown R., Guo M., Sullivan P. Vol. 18. The Individual Development and Educational Assessment Center; 2015. https://ideacontent.blob.core.windows.net/content/sites/2/2020/01/Revising-the-IDEA-Student-Ratings-of-Instruction-System-2002-2011-Data_Technical_report_18.pdf (Revising the IDEA student ratings of instruction system 2002-2011 data). IDEA Technical Report. Retrieved from. [ Google Scholar ]
• Bernard R.M., Abrami P.C., Lou Y., Borokhovski E., Wade A., Wozney L., et al. How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research. 2004; 74 (3):379–439. doi: 10.3102/00346543074003379. [ CrossRef ] [ Google Scholar ]
• Bernard R.M., Borokhovski E., Schmid R.F., Tamim R.M., Abrami P.C. A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education. 2014; 26 (1):87–122. doi: 10.1007/s12528-013-9077-3. [ CrossRef ] [ Google Scholar ]
• Bonfield C.A., Salter M., Longmuir A., Benson M., Adachi C. Transformation or evolution?: Education 4.0, teaching and learning in the digital age. Higher Education Pedagogies. 2020; 5 (1):223–246. doi: 10.1080/23752696.2020.1816847. [ CrossRef ] [ Google Scholar ]
• Brierton S., Wilson E., Kistler M., Flowers J., Jones D. A comparison of higher order thinking skills demonstrated in synchronous and asynchronous online college discussion posts. North American Colleges and Teachers of Agriculture. 2016; 60 (1):14–21. https://www.jstor.org/stable/10.2307/nactajournal.60.1.14 Retrieved from. [ Google Scholar ]
• Burch B. Quality Matters; 2019. The effects of instructor participation and class size on student participation in an online class discussion forum. [ Google Scholar ]
• Cavanaugh J.K., Jacquemin S.J. A large sample comparison of grade based student learning outcomes in online vs. Face-to-Face courses. Journal of Asynchronous Learning Networks. 2015; 19 (2) doi: 10.24059/olj.v19i2.454. [ CrossRef ] [ Google Scholar ]
• Chaka C. 2020. Higher education institutions and the use of online instruction and online tools and resources during the COVID-19 outbreak - an online review of selected U.S . and SA's universities. [ CrossRef ] [ Google Scholar ]
• Chaparro-Peláez J., Iglesias-Pradas S., Rodríguez-Sedano F.J., Acquila-Natale E. Extraction, processing and visualization of peer assessment data in moodle. Applied Sciences. 2020; 10 (1) doi: 10.3390/app10010163. [ CrossRef ] [ Google Scholar ]
• Clark T.M., Callam C.S., Paul N.M., Stoltzfus M.W., Turner D. Testing in the time of COVID-19: A sudden transition to unproctored online exams. Journal of Chemical Education. 2020; 97 (9):3413–3417. doi: 10.1021/acs.jchemed.0c00546. [ CrossRef ] [ Google Scholar ]
• Cormack S.H., Eagle L.A., Davies M.S. A large-scale test of the relationship between procrastination and performance using learning analytics. Assessment & Evaluation in Higher Education. 2020:1–14. doi: 10.1080/02602938.2019.1705244. [ CrossRef ] [ Google Scholar ]
• Crawford J., Butler-Hunderson K., Rudolph J., Malkawi B., Glowatz M., Burton R., et al. COVID-19: 20 countries’ higher education intra-period digital pedagogy responses. Journal of Applied Learning & Teaching‏ 2020; 3 (1):9–28. doi: 10.37074/jalt.2020.3.1.7. [ CrossRef ] [ Google Scholar ]
• Cuesta A. 2020. El uso de Zoom se dispara en España durante el confinamiento. https://www.mobileworldlive.com/spanish/el-uso-de-zoom-se-dispara-en-espana-durante-el-confinamiento Retrieved September 25, 2020, from Mobile World Live website: [ Google Scholar ]
• Dill E., Fischer K., McMurtrie B., Supiano B. As coronavirus spreads, the decision to move classes online is the first step. What comes next? The Chronicle of Higher Education. 2020; 66 (25) https://www.chronicle.com/article/as-coronavirus-spreads-the-decision-to-move-classes-online-is-the-first-step-what-comes-next/ Retrieved from. [ Google Scholar ]
• EDUCAbase . 2020. Matriculados por nivel académico, tipo y modalidad de la universidad, tipo de centro, sexo, grupo de edad y rama de enseñanza. http://estadisticas.mecd.gob.es/EducaJaxiPx/Datos.htm?path=/Universitaria/Alumnado/Nueva_Estructura/GradoCiclo/Matriculados//l0/&file=Mat_GradCiclo_Sex_Edad(1)_Rama_Tot.px&type=pcaxis Retrieved from. [ Google Scholar ]
• Eldredge N., Gould S.J. Punctuated equilibria: An alternative to phyletic gradualism. Models in Paleobiology. 1972; 82 :115. [ Google Scholar ]
• Elmer T., Mepham K., Stadtfeld C. Students under lockdown: Comparisons of students' social networks and mental health before and during the COVID-19 crisis in Switzerland. PloS One. 2020; 15 (7) doi: 10.1371/journal.pone.0236337. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Frazier P., Gabriel A., Merians A., Lust K. Understanding stress as an impediment to academic performance. Journal of American College Health. 2019; 67 (6):562–570. doi: 10.1080/07448481.2018.1499649. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gallardo-Echenique E.E., Marqués-Molías L., Bullen M., Strijbos J.-W. Let's talk about digital learners in the digital era. International Review of Research in Open and Distance Learning. 2015; 16 (3) doi: 10.19173/irrodl.v16i3.2196. [ CrossRef ] [ Google Scholar ]
• Gabinete de Tele-Educación . 2020. Nuestros servicios. http://serviciosgate.upm.es/gate/servicios Retrieved. [ Google Scholar ]
• García-Peñalvo F.J., Corell A., Abella-García V., Grande M. Online Assessment in Higher Education in the Time of COVID-19 - La evaluación online en la educación superior en tiempos de la COVID-19. Education in the Knowledge Society (EKS) 2020; 21 :1–26. doi: 10.14201/eks.23086. [ CrossRef ] [ Google Scholar ]
• Gonzalez T., de la Rubia M.A., Hincz K.P., Comas-Lopez M., Subirats L., Fort S., et al. Influence of COVID-19 confinement in students’ performance in higher education. PLoS One. 2020; 15 (10):1–25. doi: 10.1371/journal.pone.0239490. e0239490. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hannan M.T., Freeman J. The population ecology of organizations. American Journal of Sociology. 1977; 82 (5):929–964. [ Google Scholar ]
• Haveman H.A. Between a rock and a hard place: Organizational change and performance under conditions of fundamental environmental transformation. Administrative Science Quarterly. 1992; 37 (1):48–75. [ Google Scholar ]
• Hawthorne-Castro J. 2020. COVID-19's impact on Millennial and gen Z media habits — and how Marketers should Pivot. https://www.targetmarketingmag.com/post/covid-19s-impact-media-habits-millennial-gen-z/ Retrieved September 25, 2020, from Target Marketing website. [ Google Scholar ]
• Hernandez-de-Menendez M., Morales-Menendez R. Technological innovations and practices in engineering education: A review. International Journal on Interactive Design and Manufacturing. 2019; 13 (2):713–728. doi: 10.1007/s12008-019-00550-1. [ CrossRef ] [ Google Scholar ]
• Hettiarachchi E., Balasooriya I., Mor E., Huertas M.A. Formative assessment, learning data analytics and gamification. 2016. E-assessment for skill acquisition in online engineering education; pp. 49–64. [ CrossRef ] [ Google Scholar ]
• Hodges C., Moore S., Lockee B., Trust T., Bond A. 2020. The difference between emergency remote teaching and online learning. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning Retrieved from Educause Review website: [ Google Scholar ]
• Hoyt D.P., Lee E. 2002. Technical report No. 12 basic data for the revised IDEA system. https://ideacontent.blob.core.windows.net/content/sites/2/2020/01/Basic-Data-Revised-IDEA-System_techreport-12.pdf Retrieved from. [ Google Scholar ]
• Iivari N., Sharma S., Ventä-Olkkonen L. Digital transformation of everyday life – how COVID-19 pandemic transformed the basic education of the young generation and why information management research should care? International Journal of Information Management. 2020:102183. doi: 10.1016/j.ijinfomgt.2020.102183. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jahng N., Krug D., Zhang Z. Student achievement in online distance education compared to face-to-face education. European Journal of Open, Distance and E-Learning. 2007; 10 (1):1–16. http://www.eurodl.org/materials/contrib/2007/Jahng_Krug_Zhang.htm Retrieved from. [ Google Scholar ]
• Jones K. 2020. How COVID-19 has impacted media consumption, by Generation. Retrieved September 25, 2020, from visual Capitalist website: https://www.visualcapitalist.com/media-consumption-covid-19/ [ Google Scholar ]
• Keijzer-de Ruijter M., Draaijer S. 2019. Digital exams in engineering education. [ CrossRef ] [ Google Scholar ]
• Kim H.J., Hong A.J., Song H.-D. The roles of academic engagement and digital readiness in students' achievements in university e-learning environments. International Journal of Educational Technology in Higher Education. 2019; 16 (21):1–18. doi: 10.1186/s41239-019-0152-3. [ CrossRef ] [ Google Scholar ]
• Kim J.-E., Nembhard D.A. The impact of procrastination on engineering students' academic performance. International Journal of Engineering Education. 2019; 35 (4):1008–1017. [ Google Scholar ]
• Knowles H.P., Saxberg B.O. Organizational leadership of planned and unplanned change. A systems approach to organizational viability. Futures. 1988; 20 (3):252–265. doi: 10.1016/0016-3287(88)90081-X. [ CrossRef ] [ Google Scholar ]
• Koul S., Nayar B. The holistic learning educational ecosystem: A classroom 4.0 perspective. Higher Education Quarterly. 2020:1–15. doi: 10.1111/hequ.12271. [ CrossRef ] [ Google Scholar ]
• Ladyshewsky R.K. E-learning compared with face to face: Differences in the academic achievement of postgraduate business students. Australasian Journal of Educational Technology. 2004; 20 (3):316–336. doi: 10.14742/ajet.1350. [ CrossRef ] [ Google Scholar ]
• Linde P. 2020. Sanidad confirma en La Gomera el primer caso de coronavirus en España. https://elpais.com/sociedad/2020/01/31/actualidad/1580509404_469734.html Retrieved September 11, 2020, from El País website: [ Google Scholar ]
• Macaulay L., Dyer L.T. In: Proceedings of E-learn 2010--world conference on E-learning in corporate, government, healthcare, and higher education. Sanchez J., Zhang K., editors. 2010. Best practices and benefits of interactive web conferencing via WebEx; p. 261. https://www.learntechlib.org/p/35941/ Retrieved from. [ Google Scholar ]
• Martin F., Polly D., Ritzhaupt A. 2020. Bichronous online learning: Blending asynchronous and synchronous online learning. EDUCAUSE review, (September) https://er.educause.edu/articles/2020/9/bichronous-online-learning-blending-asynchronous-and-synchronous-online-learning Retrieved from. [ Google Scholar ]
• McCoy K.S. Using Zoom, cloud based video conferencing system: To enhance a distance education course and/or program. Proceedings of Society for Information Technology & Teacher Education International Conference. 2015; 2015 (2009):412–415. [ Google Scholar ]
• Means B., Bakia M., Murphy R. Routledge; New York, NY: 2014. Learning online: What research tells us about whether, when and how. [ Google Scholar ]
• Means B., Toyama Y., Murphy R., Baki M. The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record. 2013; 115 (3) [ Google Scholar ]
• Milman N.B. Education Week; 2020. This is emergency remote teaching, not just online teaching. https://www.edweek.org/ew/articles/2020/03/30/this-is-emergency-remote-teaching-not-just.html Retrieved from. [ Google Scholar ]
• Ministerio de Sanidad . 2020. Activado el protocolo por #coronavirus en la Comunidad de Madrid tras dar positivo una persona en las pruebas. La Consejería ha comunicado el envío de muestras para su confirmación al Centro Nacional de Microbiología [Tweet] https://twitter.com/sanidadgob/status/1232428734499741696 Retrieved from. [ Google Scholar ]
• Ministerio de Sanidad, Gobierno de España . 2020. Plan para la transisción hacia una nueva normalidad. https://www.lamoncloa.gob.es/consejodeministros/resumenes/Documents/2020/PlanTransicionNuevaNormalidad.pdf Retrieved from. [ Google Scholar ]
• Misirlis N., Zwaan M.H., Weber D. 2020. International students' loneliness, depression and stress levels in COVID-19 crisis. The role of social media and the host university. ArXiv:2005.12806. http://arxiv.org/abs/2005.12806 Retrieved from. [ Google Scholar ]
• Moallem M. The impact of synchronous and asynchronous communication tools on learner self-regulation, social presence, immediacy, intimacy and satisfaction in collaborative online learning. The Online Journal of Distance Education and E-Learning. 2015; 3 (3):53–77. http://tojdel.net/pdf/v03i03/v03i03-08.pdf Retrieved from. [ Google Scholar ]
• Order 338/2020 . 2020. Orden 338/2020, de 9 de marzo, de la Consejería de Sanidad, por la que se adoptan medidas preventivas y recomendaciones de salud pública en la Comunidad de Madrid como consecuencia de la situación y evolución del coronavirus (COVID-19) https://www.bocm.es/boletin/CM_Orden_BOCM/2020/03/10/BOCM-20200310-1.PDF Retrieved September 2, 2020, from. [ Google Scholar ]
• Oztok M., Zingaro D., Brett C., Hewitt J. Exploring asynchronous and synchronous tool use in online courses. Computers & Education. 2013; 60 (1):87–94. doi: 10.1016/j.compedu.2012.08.007. [ CrossRef ] [ Google Scholar ]
• Parks-Stamm E.J., Zafonte M., Palenque S.M. The effects of instructor participation and class size on student participation in an online class discussion forum. British Journal of Educational Technology. 2017; 48 (6):1250–1259. doi: 10.1111/bjet.12512. [ CrossRef ] [ Google Scholar ]
• Pascoe M.C., Hetrick S.E., Parker A.G. The impact of stress on students in secondary school and higher education. International Journal of Adolescence and Youth. 2020; 25 (1):104–112. doi: 10.1080/02673843.2019.1596823. [ CrossRef ] [ Google Scholar ]
• Patil I. 2018. Ggstatsplot: “ggplot2” based plots with statistical details. [ CrossRef ] [ Google Scholar ]
• Poston J., Apostel S., Richardson K. Pedagogicon conference proceedings. 2020. Using Microsoft teams to enhance engagement and learning with any Class : It's fun and easy; pp. 1–5. https://encompass.eku.edu/pedagogicon%0APoston Retrieved from. [ Google Scholar ]
• Rapanta C., Botturi L., Goodyear P., Guàrdia L., Koole M. Online university teaching during and after the covid-19 crisis: Refocusing teacher presence and learning activity. Postdigital Science and Education. 2020:1–23. doi: 10.1007/s42438-020-00155-y. [ CrossRef ] [ Google Scholar ]
• Real Decreto 463/2020 Real Decreto 463/2020, de 14 de marzo, por el que se declara el estado de alarma para la gestión de la situación de crisis sanitaria ocasionada por el COVID-19. Boletín Oficial Del Estado. 2020; 67 (I):25390–25400. [ Google Scholar ]
• Reisdorf B.C., Triwibowo W., Yankelevich A. Laptop or bust: How lack of technology affects student achievement. American Behavioral Scientist. 2020; 64 (7):927–949. doi: 10.1177/0002764220919145. [ CrossRef ] [ Google Scholar ]
• Reynolds R., Chu S.K.W. Guest editorial. Information and Learning Sciences. 2020; 121 (5/6):233–239. doi: 10.1108/ILS-05-2020-144. [ CrossRef ] [ Google Scholar ]
• Roblyer M.D., Freeman J., Donaldson M.B., Maddox M. A comparison of outcomes of virtual school courses offered in synchronous and asynchronous formats. Internet and Higher Education. 2007; 10 (4):261–268. doi: 10.1016/j.iheduc.2007.08.003. [ CrossRef ] [ Google Scholar ]
• Sailer M., Schultz-Pernice F., Fischer F. Contextual facilitators for learning activities involving technology in higher education: The C♭-model. Computers in Human Behavior. 2020 [ Google Scholar ]
• Sánchez-Caballé A., Gisbert-Cervera M., Esteve-Mon F. The digital competence of university students: A systematic literature review. Aloma. 2021; 38 (1):63–74. In this issue. [ Google Scholar ]
• Sanchez-Ruiz M.-J., El Khoury J. A model of academic, personality, and emotion-related predictors of university academic performance. Frontiers in Psychology. 2019; 10 (2435):1–7. doi: 10.3389/fpsyg.2019.02435. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Servicio de Biblioteca Universitaria . 2020. Memoria estadística de la Universidad Politécnica de Madrid (Año 2019) http://www.upm.es/sfs/Rectorado/Vicerrectorado%20de%20Tecnologias%20de%20la%20Informacion%20y%20Servicios%20en%20Red/Servicio%20de%20Biblioteca%20Universitaria/Nuestra%20biblioteca/Normativa%20y%20Documentos/Estadísticas/Documentos_Estadisticas/Memoria_Estadistica_SBU_2019.pdf Retrieved. [ Google Scholar ]
• Shachar M., Neumann Y. Differences between traditional and distance education academic performances: A meta-analytic approach. International Review of Research in Open and Distance Learning. 2003; 4 (2):100–104. doi: 10.19173/irrodl.v4i2.153. [ CrossRef ] [ Google Scholar ]
• Skylar A.A. A comparison of asynchronous online text-based lectures and synchronous interactive web conferencing lectures. Issues in Teacher Education. 2009; 18 (2):69–84. [ Google Scholar ]
• Smalley A. 2020. Higher education responses to coronavirus (COVID-19) https://www.ncsl.org/research/education/higher-education-responses-to-coronavirus-covid-19.aspx Retrieved September 9, 2020, from National Conference of State Legislatures website: [ Google Scholar ]
• Soleymani M.R. Investigating the relationship between information literacy and academic performance among students. Journal of Education and Health Promotion. 2014; 3 :95. doi: 10.4103/2277-9531.139677. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Son C., Hegde S., Smith A., Wang X., Sasangohar F. Effects of COVID-19 on college students' mental health in the United States: Interview survey study. Journal of Medical Internet Research. 2020; 22 (9) doi: 10.2196/21279. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Taft S., Perkowski T., Martin L. A framework for evaluating class size in online education. Quarterly Review of Distance Education. 2011; 12 (3):181. [ Google Scholar ]
• Tomei L. The impact of online teaching on faculty load: Computing the ideal class size for traditional, online, and hybrid courses. International Journal of Instructional Technology and Distance Learning. 2004; 1 (1):39–50. doi: 10.4018/IJOPCD.2019070101. [ CrossRef ] [ Google Scholar ]
• Tomei L.A. The impact of online teaching on faculty load: Computing the ideal class size for online courses. Journal of Technology and Teacher Education. 2006; 14 (3):531–541. doi: 10.4018/IJOPCD.2019070101. [ CrossRef ] [ Google Scholar ]
• UoPeople . 2020. Emergency remote teaching Vs. Online learning: A comparison. https://www.uopeople.edu/blog/emergency-remote-teaching-vs-online-learning/ Retrieved September 3, 2020, from. [ Google Scholar ]
• Urtel M.G. Assessing academic performance between traditional and distance education course formats. Educational Technology & Society. 2008; 11 (1):322–330. [ Google Scholar ]
• Vo H.M., Zhu C., Diep N.A. The effect of blended learning on student performance at course-level in higher education: A meta-analysis. Studies In Educational Evaluation. 2017; 53 :17–28. doi: 10.1016/j.stueduc.2017.01.002. [ CrossRef ] [ Google Scholar ]
• World Health Organization . 2020. WHO Director-General’s opening remarks at the media briefing on COVID-19 - 11 March 2020. https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020 Retrieved September 4, 2020, from. [ Google Scholar ]
• Xie H., Liu W., Bhairma J., Shim E. Vol. 3. 2018. Analysis of synchronous and asynchronous E-learning environments; pp. 270–274. (3rd joint international information technology, mechanical and electronic engineering conference (JIMEC 2018) analysis). [ CrossRef ] [ Google Scholar ]
• Yamagata-Lynch L.C. Blending online asynchronous and synchronous learning. International Review of Research in Open and Distance Learning. 2014; 15 (2):189–212. doi: 10.19173/irrodl.v15i2.1778. [ CrossRef ] [ Google Scholar ]
• Zhao Y., Lei J., Yan B., Lai C., Tan H.S. What makes the difference? A practical analysis of research on the effectiveness of distance education. Teachers College Record. 2005; 107 (8):1836–1884. doi: 10.1111/j.1467-9620.2005.00544.x. [ CrossRef ] [ Google Scholar ]
• Zhong R. 2020. The coronavirus Exposes Education's digital divide. https://www.nytimes.com/2020/03/17/technology/china-schools-coronavirus.html Retrieved September 24, 2020, from The New York Times website. [ Google Scholar ]

Advertisement

E-assessment challenges during e-learning in higher education: A case study

• Published: 06 January 2024

Cite this article

• Yazid Meftah Ali Wahas   ORCID: orcid.org/0000-0002-6646-5279 1 &
• Akbar Joseph A. Syed 2  

273 Accesses

Explore all metrics

Technology has become a fundamental means to encourage reliable and more effective assessments. Rapid technological developments have led to the widespread use of digital platforms and devices in all aspects of life. Educational institutions worldwide had to take advantage of this technological leap during pandemics such as COVID-19, which changed the shape of higher education and prompted the entire globe to adopt online learning as a new form of teaching. E-learning has ushered in a revolution in the educational process. E-assessment has become a significant tool of e-learning in many parts of the world, and has served as an alternative to evaluate students’ performance. E-assessment has many advantages such as being reliable, flexible, and accessible through many devices. However, it is unfamiliar to both teachers and students and vulnerable to piracy, cheating, and impersonation. Thus, this study aims to investigate the challenges of e-assessment faced by teachers and students during e-learning at Aligarh Muslim University (AMU), India. The theory of planned behavior (TPB) was applied to test participants’ attitudes toward implementing e-assessment during online learning. The study used a quantitative method, and an online questionnaire was delivered to 120 participants. The survey addressed three domains: (1) technological and technical challenges, (2) teachers’ challenges, and (3) students’ challenges. The findings of the study showed that both teachers and students were unfamiliar with this type of assessment as they used it for the first time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Theoretical model of investigating determinants for a successful Electronic Assessment System (EAS) in higher education

Students’ Perceptions of e-Assessment

The Future of E-assessments in the UAE: Students’ Perspectives

Data availability.

The data used in this study are available from the corresponding author on reasonable request.

Abedi, E. A., Prestridge, S., & Hodge, S. (2023). Teachers’ beliefs about technology integration in Ghana: A qualitative study of teachers’, headteachers’ and education officials’ perceptions. Education and Information Technologies . https://doi.org/10.1007/s10639023-12049-0

Article   Google Scholar  

Adedoyin, O. B., & Soykan, E. (2020). COVID-19 pandemic and online learning: The challenges and opportunities. Interactive Learning Environments , 1–13. https://doi.org/10.1080/10494820.2020.1813180

Adegbija, M. V. (2012). New technologies and the conduct of E-Examinations. A case study of the national . Open University of Nigeria.

Google Scholar  

Ajzen, I. (1991). The theory of planned behavior. Orgnizational Behavior and Human Decision Processes, 50 (2), 179–211.

Ajzen, I. (2005). Attitudes, personality, and behavior . McGraw-Hill International.

Akbari, M., Danesh, M., Moumenihelali, H., & Rezvani, A. (2023). How does identity theory contribute to the continuance use of E-learning: The mediating role of Inertia and moderating role of computer self-efficacy. Education and Information Technologies, 28 (6), 6321–6345.

Al-Anzi, F. M., & Al-Shamrani, A. I. (2022). Challenges of e-assessment during e-learning during Covid-19 according to teachers’ perceptions in Hail Province. Arabic Journal for Specific Education, 23 , 371–394.

Al-Jadee’a, M. Q. (2017). The attitudes of academic staff towards e-assessment and challenges of its implementation at Tabuk University. International Specialized Educational Journal, 6 , 77–87.

Al-Maqbali, A. H., & Raja Hussain, R. (2022). The impact of online assessment challenges on assessment principles during COVID-19 in Oman. Journal of University Teaching & Learning Practice, 19 (2), 73–91. https://doi.org/10.53761/1.19.2.6

Al-Rouqi, A. A. (2017). The degree of natural sciences teachers’ practice of e-assessment methods in the intermediate stage in Riyadh. (Master thesis, Imam Mohammed Bin Saud Islamic University, Riyadh).

Al-Zaid, H. A. (2019). The impact of e-assessment (Kahoot App) as a sample on increasing motivation of Princess Noura Bint Abdurhaman. Faculty of Education Journal for Humanities and Educational Sciences Babil University, 34 , 509–527.

Al-Zain, H. (2017). Program training effectiveness in improving skill of designing and producing e-assessment tools among faculty members and their satisfaction. Journal of Islamic University for Educational and Psychological Studies, 25 , 21–45.

Alruwais, N., Wills, G., & Wald, M. (2018). Advantages and challenges of using e-assessment. International Journal of Information and Education Technology, 8 (1), 34–37.

Alsadoon, H. (2017). Students’ perceptions of E-assessment at Saudi Electronic University. Turkish Online Journal of Educational Technology-TOJET, 16 (1), 147–153.

Aroonsrimarakot, S., Laiphrakpam, M., Chathiphot, P., Saengsai, P., & Prasri, S. (2022). Online learning challenges in Thailand and strategies to overcome the challenges from the students’ perspectives. Education and Information Technologies, 28 , 8153–8170. https://doi.org/10.1007/s10639-022-11530-6

Ataullah, M. I. M. (2016). Attitudes of Teachers and Students towards e-assessment and its challenges at AlMansoura University. Educational and Psychological Studies Journal, 90 , 201–247.

Bao, W. (2020). COVID-19 and online teaching in higher education: A case study of Peking University. Human Behavior and Emerging Technologies, 2 (2), 113–115.

Bdair, I. A. (2021). Nursing students’ and faculty members’ perspectives about online learning during the COVID-19 pandemic: A qualitative study. Teaching and Learning in Nursing, 16 (3), 220–226.

Beaunoyer, E., Dupéré, S., & Guitton, M. J. (2020). COVID-19 and digital inequalities: Reciprocal impacts and mitigation strategies. Computers in Human Behavior, 111 , 106424.

Bousalem, K. (2022). The effectiveness of electronic assessment in distance learning. Ichkalat in Language and Literature Journal.  https://www.asjp.cerist.dz/en/downarticle/238/11/1/181635 . Accessed 20 Apr 2023

Cook, J., & Jenkins, V. (2010). Getting started with E-assessment . University of Bath.

Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). SAGE Publications, Inc.

Crisp, G., Guàrdia, L., & Hillier, M. (2016). Using e-assessment to enhance student learning and evidence learning outcomes. International Journal of Educational Technology in Higher Education, 13 , 18. https://doi.org/10.1186/s41239-016-0020-3

De Villiers, R., Scott-Kennel, J., & Larke, R. (2016). Principles of effective e-assessment: A proposed framework. Journal of International Business Education, 11 , 65–92.

Dhal, P. K. (2020). Emerging Issues and Challenges in Higher Education of India. In Higher Education of India . https://doi.org/10.6084/m9.figshare.12547589

Doğan, N., Kibrislioğlu, N., Kelecioğlu, H., & Hambleton, R. K. (2020). An overview of e-assessment. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 35 (Special Issue), 1–5.

Donovan, J., Mader, C., & Shinsky, J. (2007). Online vs. traditional course evaluation formats: Student perceptions. Journal of Interactive Online Learning , 6(3), 158–180. Retrieved from http://www.ncolr.org/jiol/issues/pdf/6.3.2.pdf .  Accessed 1 Oct 2023

Eljinini, M., & Alsamarai, S. (2012). The impact of e-assessments system on the success of the implementation process. Modern Education and Computer Science, 4 (11), 76–84.

Farooq, F., Rathore, F. A., & Mansoor, S. N. (2020). Challenges of online medical education in Pakistan during COVID-19 pandemic. Journal of the College of Physicians and Surgeons Pakistan, 30 (1), 67–69. https://doi.org/10.29271/jcpsp.2020.supp1.s67

Flanigan, A. E., & Babchuk, W. A. (2022). Digital distraction in the classroom: Exploring instructor perceptions and reactions. Teaching in Higher Education, 27 (3), 352–370.

García-Morales, V. J., Garrido-Moreno, A., & Martín-Rojas, R. (2021). The transformation of higher education after the COVID disruption: Emerging challenges in an online learning scenario. Frontiers in Psychology, 12 , 196. https://doi.org/10.3389/fpsyg.2021.616059

Gilbert, L., Whitelock, D., & Gale, V. (2011). Synthesis report on assessment and feedback with technology enhancement. Electronics and Computer Science EPrints Southampton , available at: http://srafte.ecs.soton.ac.uk .  Accessed 30 Sep 2023

Godhe, A. L. (2023). Teachers’ experience of the breakdown of infrastructures during the pandemic. Education and Information Technologies . https://doi.org/10.1007/s10639023-12027-6

Govindarajan, V., & Srivastava, A. (2020). What the shift to virtual learning could mean for the future of higher ed. Harvard Business Review, 31 (1), 3–8. https://shorturl.at/eEVW7

Hillier, M. (2014). The very idea of e-Exams: student (pre) conceptions. In Proceedings of ASCILITE 2014-Annual Conference of the Australian Society for Computers in Tertiary Education (pp. 77–88).

Holden, O. L., Norris, M. E., & Kuhlmeier, V. A. (2021). Academic integrity in online assessment: A research review. In Frontiers in Education (Vol. 6, p. 639814). Frontiers Media SA.

Huda, S. S. M., Kabir, M., & Siddiq, T. (2020). E-assessment in higher education: Students perspective. International Journal of Education and Development Using Information and Communication Technology, 16 (2), 250–258.

Janke, S., Rudert, S. C., Petersen, Ä., Fritz, T. M., & Daumiller, M. (2021). Cheating in the wake of COVID-19: How dangerous is ad-hoc online testing for academic integrity? Computers and Education Open, 2 , 100055.

Jordan, S., & Mitchell, T. (2009). e-Assessment for learning? The potential of short-answer free-text questions with tailored feedback. British Journal of Educational Technology, 40 (2), 371–385.

Kuikka, M., Kitola, M., & Laakso, M. J. (2014). Challenges when introducing electronic exam. Research in Learning Technology, 22 . https://doi.org/10.3402/rlt.v22.22817

Liao, H. L., Liu, S. H., Pi, S. M., & Chou, Y. J. (2011). Factors affecting lifelong learners’ intention to continue using e-learning website: An empirical study. New Horizons in web-based Learning-ICWL 2010 Workshops: ICWL 2010 Workshops: STEG, CICW, WGLBWS, and IWKDEWL, Shanghai, China, December 7–11, 2010 revised selected papers 9 (pp. 112–119). Springer.

Lin, K. M. (2011). e-Learning continuance intention: Moderating effects of user e-learning experience. Computers & Education, 56 (2), 515–526.

Maatuk, A. M., Elberkawi, E. K., Aljawarneh, S., Rashaideh, H., & Alharbi, H. (2022). The COVID-19 pandemic and E-learning: Challenges and opportunities from the perspective of students and instructors. Journal of Computing in Higher Education, 34 (1), 21–38.

Marriott, P. (2009). Students’ evaluation of the use of online summative assessment on an undergraduate financial accounting module. British Journal of Educational Technology, 40 (2), 237–254.

Mohmmed, A. O., Khidhir, B. A., Nazeer, A., & Vijayan, V. J. (2020). Emergency remote teaching during coronavirus pandemic: The current trend and future directive at Middle East College Oman. Innovative Infrastructure Solutions, 5 (3), 1–11. https://doi.org/10.1007/s41062-020-00326-7

Mouloudj, K., Bouarar, A. C., & Stojczew, K. (2021). Analyzing the students’ intention to use online learning system in the context of COVID-19 pandemic: A theory of planned behavior approach (Vol. 3, p. 9). University of South Florida M3 Center Publishing.

Patil, H., & Undale, S. (2023). Willingness of university students to continue using e-Learning platforms after compelled adoption of technology: Test of an extended UTAUT model. Education and Information Technologies, 28 , 14943–14965. https://doi.org/10.1007/s10639-023-11778-6

Peytcheva-Forsyth, R., Aleksieva, L., & Yovkova, B. (2018). The Impact of prior experience of e-learning and e-assessment on students’ and teachers’ approaches to the use of a student authentication and authorship checking system. EDULEARN18 Proceedings , 2311–2321. https://doi.org/10.21125/edulearn.2018.0626

 Pham, A. T. (2022). University students’ attitudes towards the application of quizizz in learning english as a foreign language. International Journal of Emerging Technologies in Learning, 17 (19). https://doi.org/10.3991/ijet.v17i19.32235

Rahim, A. F. A. (2020). Guidelines for online assessment in emergency remote teaching during the COVID-19 pandemic. Education in Medicine Journal, 12 (2), 59–68. https://doi.org/10.21315/eimj2020.12.2.6

Reju, S. A., & Adesina, A. (2009). Fundamentals of on-line examinations. In Paper presented at a training workshop for academic staff on on-line examination System in National Open University of Nigeria, at the Model Study Centre Computer Laboratory .

Ridgway, J., McCusker, S., & Pead, D. (2004). Literature review of e-assessment. Bristol. Retrieved from http://dro.dur.ac.uk/1929/ . Accessed 2 Oct 2023

Rueda-Gómez, K.L., Rodríguez-Muñiz, L.J. & Muñiz-Rodríguez, L. (2023) Factors that mediate the success of the use of online platforms to support learning: the view of university teachers. Education and Information technologies, 1–24. https://doi.org/10.1007/s10639-023-11916-0

Slimi, Z. (2020). Online learning and teaching during COVID-19: A case study from Oman. International Journal of Information Technology and Language Studies, 4 (2), 44–56.

Sorensen, E. (2013). Implementation and student perceptions of e-assessment in a Chemical Engineering module. European Journal of Engineering Education, 38 (2), 172–185. https://doi.org/10.1080/03043797.2012.760533

Stodberg, U. (2012). A research review of e-assessment. Assessment and Evaluation in Higher Education, 37 (5), 591–604. https://doi.org/10.1080/02602938.2011.557496

Syed, A.J.A, & Wahas, Y.M.A (2020). Challenges and solutions in teaching english through poetry to EFL Students at Hajjah University: A Case Study of William Wordsworth’s Lucy and John Donne’s Death poems. rEFLections , 27 (2), 189–198. https://so05.tci-thaijo.org/index.php/reflections/article/view/248043 . Accessed 30 July 2023

Tuah, N. A. A., & Naing, L. (2021). Is online assessment in higher education institutions during COVID-19 pandemic reliable? Siriraj Medical Journal, 73 (1), 61–68.

Tuparova, D., Goranova, E., Voinohovska, V., Asenova, P., Tuparov, G., & Gyudzhenov, I. (2015). Teachers’ attitudes towards the use of e-assessment–results from a survey in Bulgaria. Procedia Social and Behavioral Sciences, 191 , 2236–2240.

Wahas, Y. M. A. (2023). Challenges of e-learning faced by ESL learners during the Covid-19 Pandemic: A case study. International Journal of Language and Translation Research, 3 (1), 41–58. https://doi.org/10.22034/IJLTR.2023.169033

Way, A. (2012). The use of e-assessments in the Nigerian higher education system. The Turkish Online Journal of Distance Education, 13 (1), 140–152.

Xu, H., & Mahenthiran, S. (2016). Factors that influence online learning assessment an satisfaction: using moodle as a learning management system. International Business Research , 9 (2). https://doi.org/10.5539/ibr.v9n2p1

Zaiton, H. (2005). E-learning: Concept, issues, implementation and assessment . Dar Alsoltiah.

Zhang, W., Wang, Y., Yang, L., & Wang, C. (2020). Suspending classes without stopping learning: China’s education emergency management policy in the COVID-19 outbreak. Journal of Risk and Financial Management, 13 (3), 55.

Download references

Author information

Authors and affiliations.

Hajjah University, Hajjah City, Yemen

Yazid Meftah Ali Wahas

Aligarh Muslim University, Aligarh City, UP, India

Akbar Joseph A. Syed

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Yazid Meftah Ali Wahas .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Wahas, Y.M.A., Syed, A.J.A. E-assessment challenges during e-learning in higher education: A case study. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-023-12421-0

Download citation

Received : 22 May 2023

Accepted : 12 December 2023

Published : 06 January 2024

DOI : https://doi.org/10.1007/s10639-023-12421-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• E-assessment
• Covid-19 pandemic
• Aligarh Muslim University
• Find a journal
• Publish with us
• Track your research

• Artificial Intelligence
• Splunk Life
• Global Impact
• Tips & Tricks
• Subscribe to Splunk Blogs
• Blog Authors

Data Integration for Higher Education: An NJIT and Splunk Case Study

• Share on Facebook
• Share on LinkedIn

One institution prioritizing data integration is the New Jersey Institute of Technology (NJIT), which has revolutionized its capacity to integrate data across its campus infrastructure. With the Splunk platform, NJIT can now combine disparate streams of event, academic and infrastructure data, bolstering physical and digital security across the campus. 

Redefining Student Success

After experiencing substantial growth in recent years, NJIT relies on the Splunk platform to help the institution collect non-obvious data from various facets of the student and campus experience across its community of over 12,000 full-time students. This way, NJIT’s IT teams can effectively monitor student and university progression toward critical objectives, such as expanding on-site attendance. By strategically mining non-obvious data from various aspects of the student and campus experience, NJIT has effectively monitored progress toward critical objectives, such as increasing on-site attendance, resulting in a remarkable 28% jump in enrollment in recent years.

Gaining a Holistic View of Campus Infrastructure

NJIT's IT team leverages Splunk Observability Cloud and Splunk Synthetic Monitoring to observe activity across the campus infrastructure and better prevent, detect and respond to downtime. Through a holistic view of its cloud environment, NJIT can detect issues and update applications ahead of critical university moments, like registration and exams. As a fast-paced institution, NJIT's IT team can proactively eliminate student disruption while addressing an issue before it has a long-term negative impact and improve student experiences with university services on campus or attending remotely.

Training Tomorrow’s Leaders With a Student-Powered SOC

NJIT's vision for a student-powered security operations center (SOC) is another example of how data integration can drive academic and operational success. By partnering with Splunk, NJIT aims to support hundreds of students and train the future cyber workforce over the next decade. This approach showcases NJIT's forward-thinking approach to educational excellence. 

Learn about how another leading institution is working to protect its students, community and state with a student-powered SOC powered by Splunk.

Data integration is critical for educational institutions like NJIT. Using Splunk, NJIT has effectively integrated data across its campus infrastructure, providing meaningful insights that have driven academic and operational success. NJIT's approach to data integration serves as an excellent use case for every university on how comprehensive visibility of its data — from academics to data privacy to space utilization — can impact the broader student experience.

How can the Splunk platform help you build greater digital resilience? Learn more about how organizations across the public sector rely on Splunk to prevent disruptions and deliver secure, enhanced citizen services.

Follow all the conversations coming out of #splunkconf24 !

Follow @splunk

Digital Resilience Pays Off

Download this e-book to learn about the role of Digital Resilience across enterprises.

A purpose-driven, passionate, and disruptive sales executive at the Data to Everything company, Splunk. Mary Lou Prevost holds the title of Group Vice President, managing the State, Local Government and Education (SLED) sales team for the US.

With over 20 years of experience, Prevost is a pioneer in the technology industry. A first executive hire to start and lead the SLED vertical at both CA Technologies (acquired by Broadcom) and Palo Alto Networks.

A perfect blend of enterprise technology and cybersecurity mixed with a deep understanding of the challenges facing SLED leaders, Prevost knows how to build teams, partnerships, and trust to deliver innovative solutions and improve outcomes for all.

Prevost is excited to work in SLED for the market leader known to turn data into doing, committed to helping every customer take astronomical amounts of data and make sense of it all to secure, deliver citizen solutions, migrate to the cloud, and remain wildly innovative in the changing world of education. “Helping drive everyone in our State & local governments, higher education and academic medical centers to thrive in the Data Age.

While not traveling for meetings, Prevost spends time with her family and three children, has a passion for the outdoors, and mentoring Women in STEM.

Related Articles

A unified security and observability platform, meet keith: the first .conf attendee two-years running, security at .conf21 virtual: analytics-fueled, automation-driven and cloud-delivered.

• About Splunk

The Splunk platform removes the barriers between data and action, empowering observability, IT and security teams to ensure their organizations are secure, resilient and innovative.

Founded in 2003, Splunk is a global company  — with over 7,500 employees, Splunkers have received over 1,020 patents to date and availability in 21 regions around the world — and offers an open, extensible data platform that supports shared data across any environment so that all teams in an organization can get end-to-end visibility, with context, for every interaction and business process. Build a strong data foundation with Splunk.

Subscribe to our blog

Get the latest articles from Splunk straight to your inbox.

Connect with Splunk on X

Connect with splunk on instagram.

• How Splunk Compares
• Perspectives by Splunk
• Splunk Policy Positions
• Splunk Protects
• Splunk Ventures
• Supplier Central
• Why Splunk?
• Free Trials & Downloads
• View All Products
• Documentation
• Investor Relations
• Training & Certification
• T-Shirt Store
• View All Resources
• What is SIEM?
• An Introduction to Observability
• Cyber Threat Intelligence (CTI): An Introduction
• The OSI Model in 7 Layers
• Telemetry 101: An Introduction
• What is Site Reliability Engineering?
• DORA Metrics: The Complete Guide
• Network Monitoring: How It Works
• InfoSec: An Introduction to Information Security
• Network Security: A Complete Introduction
• Root Cause Analysis: A Beginner's Guide to RCA
• Contact Sales
• Contact Support
• Gartner Peer Insights™
• TrustRadius

© 2005 - 2024 Splunk Inc. All rights reserved.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Knowledge-building in an environment mediated by digital technology: A case study in higher education

Affiliation.

• 1 Department of Theory and History of Education, Faculty of Education, University of Salamanca, Salamanca, Spain.
• PMID: 36119125
• PMCID: PMC9470513
• DOI: 10.1007/s10639-022-11304-0

The advancement of technology in recent years seems to be prompting a re-ontologising of the world. Digital technology is transforming the educational spaces we inhabit, as well as our way of processing information. Although there are already numerous studies that have addressed this technological reality, only a handful have done so from a theoretical perspective. That is why we present research that seeks to reinforce the latest theoretical contributions for understanding how modern technology may be affecting the way in which knowledge is built. Based on the latest research in social constructivism, this is a qualitative study designed to contribute to the creation of a specific theoretical framework for an onlife world. An ill-structured task and a semi-structured interview were used to observe the use of the thinking skills that enable us to build knowledge and the relationship between them. The results show that the ways of building knowledge are changing, as digital technology fosters the use of higher-order thinking skills that, furthermore, operate in a chaotic, complex, and unpredictable manner. In conclusion, this study upholds the notion that the ways of building knowledge are changing, but we still need more empirical contributions to create a generally accepted theoretical construct for explaining how we build knowledge through digital technology.

Keywords: Education; Higher education; Higher order thinking; Learning theory; Re-ontologising; Technology.

© The Author(s) 2022.

PubMed Disclaimer

Conflict of interest statement

Competing interestThe authors declare that they have no competing interests.

Evolution of Bloom’s taxonomy

Theory of web mediated knowledge…

Theory of web mediated knowledge synthesis

Ill-structured task

Main categories used according to…

Main categories used according to the percentage of units of analysis coded

Rate of use of categories…

Rate of use of categories depending on the type of technology. Note. The…

Graph of the relationships between…

Graph of the relationships between nodes and sub-nodes

Graph of the relationships between nodes

Similar articles

• Evidence Brief: The Effectiveness Of Mandatory Computer-Based Trainings On Government Ethics, Workplace Harassment, Or Privacy And Information Security-Related Topics [Internet]. Peterson K, McCleery E. Peterson K, et al. Washington (DC): Department of Veterans Affairs (US); 2014 May. Washington (DC): Department of Veterans Affairs (US); 2014 May. PMID: 27606391 Free Books & Documents. Review.
• The effectiveness of internet-based e-learning on clinician behavior and patient outcomes: a systematic review protocol. Sinclair P, Kable A, Levett-Jones T. Sinclair P, et al. JBI Database System Rev Implement Rep. 2015 Jan;13(1):52-64. doi: 10.11124/jbisrir-2015-1919. JBI Database System Rev Implement Rep. 2015. PMID: 26447007
• Student and educator experiences of maternal-child simulation-based learning: a systematic review of qualitative evidence protocol. MacKinnon K, Marcellus L, Rivers J, Gordon C, Ryan M, Butcher D. MacKinnon K, et al. JBI Database System Rev Implement Rep. 2015 Jan;13(1):14-26. doi: 10.11124/jbisrir-2015-1694. JBI Database System Rev Implement Rep. 2015. PMID: 26447004
• Macromolecular crowding: chemistry and physics meet biology (Ascona, Switzerland, 10-14 June 2012). Foffi G, Pastore A, Piazza F, Temussi PA. Foffi G, et al. Phys Biol. 2013 Aug;10(4):040301. doi: 10.1088/1478-3975/10/4/040301. Epub 2013 Aug 2. Phys Biol. 2013. PMID: 23912807
• Higher order thinking skills competencies required by outcomes-based education from learners. Chabeli MM. Chabeli MM. Curationis. 2006 Aug;29(3):78-86. doi: 10.4102/curationis.v29i3.1107. Curationis. 2006. PMID: 17131612 Review.
• Afflerbach P, Cho B-Y, Kim J-Y. Conceptualizing and assessing higher-order thinking in reading. Theory into Practice. 2015;54(3):203–212. doi: 10.1080/00405841.2015.1044367. - DOI
• Aguilera-Hermida PA. College students’ use and acceptance of emergency online learning due to COVID-19. International Journal of Educational Research Open. 2020;1:100011. doi: 10.1016/j.ijedro.2020.100011. - DOI - PMC - PubMed
• Anderson LW, Krathwohl DR. A taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman; 2001.
• Behnagh RF, Yasrebi S. An examination of constructivist educational technologies: Key affordances and conditions. British Journal of Educational Technology. 2020;51(6):1907–1919. doi: 10.1111/bjet.13036. - DOI
• Bixler BA, Land SM. Supporting college students’ Ill-structured problem solving in a web-based learning environment. Journal of Educational Technology Systems. 2010;39(1):3–15. doi: 10.2190/ET.39.1.b. - DOI

LinkOut - more resources

Full text sources.

• Europe PubMed Central
• PubMed Central
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

On The Site

Teaching About Technology in Schools Through Technoskeptical Inquiry

June 3, 2024 | victorialynn | Harvard Educational Review Contributors , Voices in Education

By Jacob Pleasants, Daniel G. Krutka, and T. Philip Nichols

New technologies are rapidly transforming our societies, our relationships, and our schools. Look no further than the intense — and often panicked — discourse around generative AI , the metaverse , and the creep of digital media into all facets of civic and social life . How are schools preparing students to think about and respond to these changes?

In various ways, students are taught how to use technologies in school. Most schools teach basic computing skills and many offer elective vocational-technical classes. But outside of occasional conversations around digital citizenship, students rarely wrestle with deeper questions about the effects of technologies on individuals and society.

Decades ago, Neil Postman (1995) argued for a different form of technology education focused on teaching students to critically examine technologies and their psychological and social effects. While Postman’s ideas have arguably never been more relevant, his suggestion to add technology education as a separate subject to a crowded curriculum gained little traction. Alternatively, we argue that technology education could be an interdisciplinary endeavor that occurs across core subject areas. Technology is already a part of English Language Arts (ELA), Science, and Social Studies instruction. What is missing is a coherent vision and common set of practices and principles that educators can use to align their efforts.

To provide a coherent vision, in our recent HER article , we propose “technoskepticism” as an organizing goal for teaching about technology. We define technoskepticism as a critical disposition and practice of investigating the complex relationships between technologies and societies. A technoskeptical person is not necessarily anti-technology, but rather one who deeply examines technological issues from multiple dimensions and perspectives akin to an art critic.

We created the Technoskepticism Iceberg as a framework to support teachers and students in conducting technological inquiries. The metaphor of an iceberg conveys how many important influences of technology lie beneath our conscious awareness. People often perceive technologies as tools (the “visible” layer of the iceberg), but technoskepticism requires that they be seen as parts of systems (with interactions that produce many unintended effects) and embedded with values about what is good and desirable (and for whom). The framework also identifies three dimensions of technology that students can examine. The technical dimension concerns the design and functions of a technology, including how it may work differently for different people. The psychosocial dimension addresses how technologies change our individual cognition and our larger societies. The political dimension considers who makes decisions concerning the terms, rules, or laws that govern technologies.

To illustrate these ideas, how might we use the Technoskeptical Iceberg to interrogate generative AI such as ChatGPT in the core subject areas?

A science/STEM classroom might focus on the technical dimension by investigating how generative AI works and demystifying its ostensibly “intelligent” capabilities. Students could then examine the infrastructures involved in AI systems , such as immense computing power and specialized hardware that in turn have profound environmental consequences. A teacher could ask students to use their values to weigh the costs and potential benefits of ChatGPT.

A social studies class could investigate the psychosocial dimension through the longer histories of informational technologies (e.g., the printing press, telegraph, internet, and now AI) to consider how they shifted people’s lives. They could also explore political questions about what rules or regulations governments should impose on informational systems that include people’s data and intellectual property.

In an ELA classroom, students might begin by investigating the psychosocial dimensions of reading and writing, and the values associated with different literacy practices. Students could consider how the concept of “authorship” shifts when one writes by hand, with word processing software, or using ChatGPT. Or how we are to engage with AI-generated essays, stories, and poetry differently than their human-produced counterparts. Such conversations would highlight how literary values are mediated by technological systems . 

Students who use technoskepticism to explore generative AI technologies should be better equipped to act as citizens seeking to advance just futures in and out of schools. Our questions are, what might it take to establish technoskepticism as an educational goal in schools? What support will educators need? And what might students teach us through technoskeptical inquiries?

Postman, N. (1995). The End of Education: Redefining the Value of School. Vintage Books.

About the Authors

Jacob Pleasants is an assistant professor of science education at the University of Oklahoma. Through his teaching and research, he works to humanize STEM education by helping students engage with issues at the intersection of STEM and society.

Daniel G. Krutka is a dachshund enthusiast, former high school social studies teacher, and associate professor of social studies education at the University of North Texas. His research concerns technology, democracy, and education, and he is the cofounder of the Civics of Technology project ( www.civicsoftechnology.org ).

T. Philip Nichols is an associate professor in the Department of Curriculum and Instruction at Baylor University. He studies the digitalization of public education and the ways science and technology condition the ways we practice, teach, and talk about literacy.

They are the authors of “ What Relationships Do We Want with Technology? Toward Technoskepticism in Schools ” in the Winter 2023 issue of Harvard Educational Review .

• Gift Guides
• Voices in Education

Higher Ed Support Center

Support by product:, additional support resources:.

• FAQ & Popular Support Articles
• Technical Support: Help & Submit a Ticket
• General Help: Orders, Pricing and Shipping

Frequently Asked Questions

• How do I request a refund?
• How can I check the status of my order?
• How can I get an invoice?
• How do I purchase a virtual access code?
• How do I request an instructor sample?
• How can I find my local sales rep?
• What is McGraw Hill’s return policy?

Popular Support Articles

For students.

• Connect - Access Code and Registration Support
• Connect - Resources to Help You Register
• Connect - Temporary Access

For Instructors

• Technical Support Troubleshooting
• Connect - Reviewing Instructor Resources for a Course

Technical Support

FAQs, articles and videos to help you troubleshoot our products.

Contact a Technical Support Representative

Phone:  (800) 331-5094 Chat:  Chat with a Representative

Email & Live Chat: Sunday: 12:00 PM to 12:00 AM EST Monday-Thursday: 24 hours EST Friday: 12:00 AM to 9:00 PM ET Saturday: 10:00 AM to 8:00 PM ET

Phone: Sunday: 12:00 PM to 1:00 AM EST Monday-Thursday: 8:00 AM to 1:00 AM EST Friday: 8:00 AM to 9:00 PM ET Saturday: 10:00 AM to 8:00 PM ET

Holiday Closures: Memorial Day. 4th of July, Thanksgiving, Day after Thanksgiving, Christmas Eve, Christmas Day, and New Year’s Day

General Help

For general questions related to pricing, orders, shipping and returns.

Phone:   (800) 338-3987 Fax:   (800) 953-8691

Mail: McGraw-Hill P.O. Box 182605 Columbus, OH 43218

Hours of Operation:

Monday-Friday:  8:00 AM to 8:00 PM EST

Outside the United States: Contact International Customer Support

• 0 || popularSuggestions?.length===0" :aria-label="'Search for '+ popular?.expression" data-analytics-search="tNav|Search|Popular">
• Sales: 833-390-1896
• Support: 844-211-5308

Helping educators deliver opportunity through connectivity.

SHARE THIS ARTICLE:

THE CUSTOMER

• Western Governors University (WGU)  , the nation’s largest online university

THE CHALLENGE

• Higher-education students cannot complete their studies without reliable internet connectivity
• Many urban and rural communities lack affordable internet options

THE OPPORTUNITY

• Help educators better serve undergraduate and graduate students by providing connectivity, devices, and infrastructure
• Make higher education accessible to everyone who wants it—no matter where they live
• Empower student achievement by creating portals to the 21 st -century internet economy

Delivered via 5G cellular network; speeds vary due to factors affecting cellular networks.  See full terms.

T-Mobile supports WGU’s Nationwide Online Access Initiative.

We provide the coverage, capacity, and devices for students to participate in higher education online.

• Make higher education accessible to all and help educators better serve students through connectivity, devices, and infrastructure

Creating equity in higher ed.

Up to 50 million Americans lack the reliable broadband service essential to access economic opportunities, distance learning, jobs, and even civic engagement in the 21 st  century.

COVID-19 has shined a light on this problem. Approximately 20% of college students in a  Communication Research  study 1  report difficulty maintaining access to technology, including internet connectivity and functioning devices. Students of lower socioeconomic status and students of color disproportionately experienced hardships.

An Ed Trust-West  study 2  found that in California alone, more than 102,000 California college students from lower income households (14%) and 145,000 college students of color (13%) may lack internet access. Also, more than 109,000 students from lower income households (15%) and nearly 134,000 students of color (12%) may lack access to a device to engage in distance learning.

1   https://journals.sagepub.com/doi/10.1177/0093650218796366 2   https://west.edtrust.org/resource/the-digital-divide-in-higher-ed/

The disparities 3  are more pronounced with rural and urban populations than in suburban areas, where reliable broadband access is more publicly available.

Western Governors University (WGU) wrestles with this challenge every day. Founded nearly 25 years ago by the governors of 19 states, the school has grown into the nation’s largest online university, with 130,000 graduate and undergraduate students attending classes virtually and 200,000 alumni.

An alternative to traditional education.

WGU’s founders created the university to offer an alternative to a traditional education that has failed many learners, including adult returning students. At WGU, 70% of students identify with one or more of four underserved populations: first-generation college students, low-income households, people of color, or residents of rural areas.

WGU students are also untraditional because their average age is 37, and most attended—but did not complete—college. Many work full time and have families. All are attracted to WGU because the school offers the promise of an affordable, high-quality education that can change their lives.

WGU’s philosophy and approach may explain why its enrollment has increased by 7% while that of traditional higher education institutions is down by 15%.

3  https://www.crpe.org/thelens/digital-divide-among-students-during-covid-19-who-has-access-who-doesnt

“The internet changes the nature of education. You don’t have to build the campuses and classrooms. High-speed connectivity is one of the easiest ways to provide access to higher education in a way we couldn’t before. Every institution should invest heavily in expanding connectivity and access.”

-Scott Pulsipher,  President, Western Governors University

What’s at stake for WGU.

WGU's mission goes beyond offering affordable college classes. It is committed to providing students with access to opportunities and a path to a better life.

The university built its entire curriculum and reputation on the results it drives for students. For example, WGU proudly reports that graduates see an average salary increase of more than $11,000 within two years of graduation. And survey data showing that 97% of employers say WGU grads meet or exceed their expectations.

To prepare students for successful careers, the school’s curriculum focuses on four high-demand fields: K–12 teaching and education, nursing and healthcare, information technology, and business. WGU’s more than 60 degree programs are designed to fit a high-growth, highly rewarding career path.

The importance of increasing graduation rates.

But the key to fulfilling the school’s mission is finding ways to help students who are busy juggling jobs, family, and other responsibilities to complete their studies.

Every university is concerned about graduation rates. Under ideal conditions, just  59.8% 4  of full-time students attending traditional universities complete their undergraduate studies within six years. WGU’s completion rates are difficult to compare, since the Department of Education only counts first-time, full-time students and excludes the non-traditional, returning students that comprise 95% of WGU’s student body. However, the school projects its six-year graduation rate for undergraduate students will be 52% in 2021, and it has set a goal of 65% by 2025.

To reach this objective, WGU helps students overcome challenges that can interrupt their studies.

In 2020, one challenge was the pandemic, which hit students hard. A significant number of those who had relied on work computers or office internet to attend classes and complete their coursework lost their jobs and broadband access, at the same time. That made it difficult to stay in school.

The Online Access Scholarship.

In response, WGU created a $1 million Online Access Scholarship fund to keep students in school by underwriting the cost of broadband access and devices needed to carry on with classes.

The program, originally designed to help about 1,000 students, has expanded and is available to both existing and prospective students. And Online Access Scholarship awardees may also qualify for a Resiliency Grant, a needs-based tuition scholarship which helps students pay for school.

4   https://nces.ed.gov/programs/digest/d17/tables/dt17_326.10.asp?referer=raceindicators

“We won’t rest until every American who wants access to quality education, available 24/7, gets it.”

–Tonya Drake,  Chancellor, WGU Washington

COVID-19 forced  Eulanda, from St. Louis, Missouri,  to quit her job because she feared getting infected at work or during her daily commute and then infecting her asthmatic son.

The situation threatened to derail her work toward an IT degree that she hoped would change her family’s life. Eulanda was struggling to juggle household bills—including broadband fees.

“No one should have to choose between keeping the lights on or keeping Wi-Fi ,” she said.

That was a choice she didn’t have to make. WGU’s Online Access Scholarship covered the cost of broadband internet in her home.

Solving education equity challenges with connectivity.

WGU sought out T-Mobile for Education after learning of its commitment to supporting K-12 education through its Project 10Million initiative, which offers eligible K-12 households 100GB of data per year and a free mobile hotspot for five years. Project 10Million also allows participating school districts to apply the value of the free program toward additional data plans based on specific students' needs. And while Project 10Million is not a higher-education initiative, WGU felt it demonstrates T-Mobile has a deep commitment to equity in education.

Together, WGU and T-Mobile for Education created a program tailored to WGU students' needs to provide broadband and device resources, bring about systemic change, and bridge the gap between talent and opportunity.

Successfully stretching budgets.

The WGU program is extensive and significantly helps stretch the university’s investment in the scholarship program. The involvement of T-Mobile has already helped WGU double the number of students supported by the Online Access initiative. As part of its commitment, T-Mobile is working with WGU to deliver free hotspot devices and unlimited service to 2,000 WGU students to give them high-speed internet access at home.

T-Mobile is uniquely qualified to help WGU connect its students. With the rapidly expanding capacity on its network, T-Mobile can deploy high-speed internet services in the most rural of communities, where no high-speed internet service existed before.

The T-Mobile network is built from the ground up for the next wave of innovation.

Ashley started college in Austin but was forced to drop out when she moved home to rural Texas to be with her family after the birth of her baby.

The problem: There was little to no internet access where her family lived in Woodville, Texas.

After communicating her situation to WGU, the school awarded Ashley its first Online Access Scholarship. Within minutes of a technician’s arrival at the family’s rural home, Ashley had high-speed internet access and could resume her education.

“The nature of education has forever changed. Yet students still face the challenges of battling for bandwidth at home, sharing limited data plans and even the prospect of intermittent or no internet connectivity. We won’t stop until every student who needs wireless broadband has it.”

–Mike Katz, Executive Vice President, T-Mobile for Business

The  WGU- T-Mobile partnership  offers promise for the future.

The WGU- T-Mobile joint effort has become a true partnership.

T-Mobile for Education is delivering connectivity that will helps students complete their education, acquire better jobs, and change their lives. But the company also benefits from a relationship with a school system that is committed to tailoring its curriculum to meet the emerging needs of companies and industries.

Educating the future T-Mobile workforce.

As WGU works with T-Mobile , both organizations are committed to identifying the kinds of IT and business skills needed in the coming years and collaborating in the development of a new curriculum that aligns with those skills.

Eventually, T-Mobile intends to create paid internship opportunities for WGU students and, ultimately, jobs for some of the school’s graduates.

Stronger together

The opportunities for collaboration only start there.  WGU has become the nation’s largest online university by creating partnerships with private and public sector organizations. As the school’s connectivity partner, T-Mobile expects to play a role in future collaborations that require connectivity, infrastructure expertise and investment.

Why? Because both organizations share a vision and recognize that they are stronger together.

To see what T-Mobile can do for your institution, visit https://www. t-mobile .com/HigherEd  or call our team of Education experts at 1-877-386-4246

About T-Mobile for Higher Education

T-Mobile is committed to Higher Education institutions, from urban campuses to rural colleges, to online universities like WGU, to help them navigate disruptive times and deliver on the future of learning. Our Higher Education program provides access and equity that forges opportunities for the workforce of the future, unlocks innovation with America’s largest and 5G network, and solves connectivity challenges via a mutually beneficial partnership that minimizes cost and complexity.

“ T-Mobile is an innovative employer that sees the nature of the future of work. The greater our partnership, the better our ability to identify skills and develop curriculum that aligns with that skill development. As these initiatives scale, T-Mobile and WGU will be at the forefront of redeveloping roles, skills and knowledge for the future.”

–Scott Pulsipher,  President, Western Governors University

Want even more trends, insights, and success stories?

Fast & Reliable: Based on T-Mobile analysis of eligible customer speed data reflecting consistent broadband speeds. Delivered via 5G cellular network; speeds vary due to factors affecting cellular networks. See T-Mobile .com/OpenInternet for additional details.