artificial intelligence in education policy

Artificial intelligence and the Futures of Learning

The Artificial Intelligence and the Futures of Learning project builds on the  Recommendation on the Ethics of Artificial Intelligence  adopted at the 41st session of the UNESCO General Conference in 2019 and follows up on the recommendations of the UNESCO global report  Reimagining our futures together: a new social contract for education ,  launched in November 2021. It is implemented within the framework of the  Beijing Consensus on Artificial Intelligence and Education  and against the backdrop of the UNESCO Strategy on technological innovation in education (2021-2025) .

The project will address both the human and technological dimensions related to AI and the futures of learning. 

Strands of work

The project consists of three independent but complementary strands: 

• AI and the Future of Learning
• Guidance for Generative AI in education and research
• AI Competency Frameworks for Students and Teachers 

Policy dialogue and consultations

• International Forums on AI and Education: International Forum on AI and Education: Steering AI to Empower Teachers and Transform Teaching (December 2022); International Forum on AI and Education: Ensuring AI as a Common Good to Transform Education (December 2021); International Forum on AI and the Futures of Education: Developing competencies for the AI Era (December 2020); and International Conference on Artificial Intelligence and Education: Planning Education in the AI Era: Lead the leap (May 2019).
• Ministerial Roundtable on Generative AI in Education : A virtual ministerial meeting on Generative AI in education took place on 25 May 2023, which gathered 25 ministers to debate on the urgent need for regulations on generative AI and competencies needed to reap its benefits. More information: https://www.unesco.org/en/articles/ministerial-roundtable-generative-ai-education
• Consultations on AI competency framework for teachers (October 2022) : the consultation was attended by 15 international experts in AI and education and more than 70 participants. More information: https://www.unesco.org/en/articles/unesco-supports-definition-and-development-ai-competencies-teachers

Knowledge production

• Guidance for Generative AI in Education and Research: The Guidance has been drafted and will be launched during Digital Learning Week (4-7 September, 2023).
• Drafting the AI Competency Framework for school students: The first draft of the Framework for consultation that will be presented during Digital Learning Week (4-7 September 2023).
• Drafting the AI Competency Framework for teachers: The first draft of the Framework for consultation that will be presented during Digital Learning Week (4-7 September 2023).
• K-12 AI curricula: a mapping of government-endorsed AI curricula : The report builds on the results of a survey on “AI curricula for school students” which was circulated to all UNESCO Member States in 2022. The report will inform the development of the AI Competency Framework for Students . 
• A survey on the governmental use of AI in education : Completed in early 2023, the survey covers assessments of Member States’ setting up of regulations on ethics of AI and its use in education, strategies for AI in education, and national programmes on developing AI competencies for teachers. The results of the survey informed the AI Competency Framework for teachers. 
• Definition of Algorithm Literacy and Data Literacy : A call for contribution to the definition of Algorithm Literacy and Data Literacy was launched in June 2023. The selected think-pieces will feed inputs for the development of AI Competency Frameworks for students and teachers.
• An in-depth case study on K-12 AI curricula of the United Arab Emirates : In collaboration with the Regional Center for Educational Planning (RCEP) and the Ministry of Education of the United Arab Emirates (UAE), UNESCOcompleted a case study on the UAE’s K-12 AI curriculum and its implementation. The case study is planned for release in 2023.

Capacity building

• Workshop on AI curriculum development for schools in Oman (May 2022) : UNESCO, the Ministry of Education of Oman, RCEP, UNESCO Doha and Ericsson conducted an online workshop to empower 25+ national curriculum developers in integrating AI competencies into K-12 education. More information: https://www.unesco.org/en/articles/oman-embarks-development-k-12-ai-curricula-support-unesco-and-rcep
• Workshop on Coding and AI for teachers in Lebanon (May 2023): As part of the “Teaching Coding and AI for Teachers and K-12 Students” programme in Lebanon, UNESCO, UNESCO Beirut and Ministry of Education of Lebanon conducted a three-day training on Coding and AI, empowering teachers and staff with AI skills and computational thinking. More information: https://www.unesco.org/en/articles/unesco-and-lebanon-join-forces-develop-coding-skills-teachers-and-disadvantaged-students

Publications

• International forum on AI and education: steering AI to empower teachers and transform teaching, 5-6 December 2022; analytical report , UNESCO, 2023
• K-12 AI curricula: a mapping of government-endorsed AI curricula , UNESCO, 2022
• AI and education: Guidance for policy-makers , UNESCO, 2021
• International Forum on AI and Education: Ensuring AI as a Common Good to Transform Education, synthesis report , UNESCO 2021
• International Forum on AI and the Futures of Education, developing competencies for the AI Era, 7-8 December 2020: synthesis report , UNESCO, 2020
• Artificial intelligence in education: Compendium of promising initiatives , UNESCO, 2020
• Beijing Consensus on Artificial Intelligence and Education , UNESCO, 2019
• Artificial intelligence in education: Compendium of promising initiatives , UNESCO, 2019
• International conference on Artificial intelligence and Education, Planning education in the AI Era: Lead the leap: final report , UNESCO 2019

Related events

• Digital Learning Week on “Steering technology for education” , 4 -7 September 2023
• Ministerial Roundtable on Generative AI in Education , 25 May 2023
• International forum on AI and education: steering AI to empower teachers and transform teaching , 5-6 December 2022
• International Forum on AI and Education: Ensuring AI as a Common Good to Transform Education , 7-8 December 2021
• Launch of the AI and the Futures of Learning Project , 30 September 2021 
• International Forum on Artificial Intelligence and the Futures of Education , 7-9 December 2020
• Online Edition of Mobile Learning Week 2020 - Beyond Disruption: Technology Enabled Learning Futures ,12-14 October 2020
• International Conference on Artificial Intelligence and Education , 16-18 May 2019
• Mobile Learning Week 2019 – Artificial intelligence for Sustainable Development ,4-8 March 2019

The project is supported by the Tomorrow Advancing Life Education Group (TAL) of China, a long-term partner of UNESCO and one of the sponsors of the International Conference on Artificial Intelligence and Education.

Related items

• Artificial intelligence

• Become a Member
• Artificial Intelligence
• Computational Thinking
• Digital Citizenship
• Edtech Selection
• Global Collaborations
• STEAM in Education
• Teacher Preparation
• ISTE Certification
• School Partners
• Career Development
• ISTELive 24
• Solutions Summit
• Leadership Exchange
• 2024 ASCD Leadership Summit
• 2025 ASCD Annual Conference
• Edtech Product Database
• Solutions Network
• Sponsorship & Advertising
• Sponsorship & Advertising

Artificial Intelligence in Education

To prepare students to thrive as learners and leaders of the future, educators must become comfortable teaching with and about Artificial Intelligence. Generative AI tools such as ChatGPT , Claude and Midjourney , for example, further the opportunity to rethink and redesign learning. Educators can use these tools to strengthen learning experiences while addressing the ethical considerations of using AI. ISTE is the global leader in supporting schools in thoughtfully, safely and responsibly introducing AI in ways that enhance learning and empower students and teachers.

Interested in learning how to teach AI?

Sign up to learn about ISTE’s AI resources and PD opportunities. 

StretchAI: An AI Coach Just for Educators

ISTE and ASCD are developing the first AI coach specifically for educators. With Stretch AI, educators can get tailored guidance to improve their teaching, from tips on ways to use technology to support learning, to strategies to create more inclusive learning experiences. Answers are based on a carefully validated set of resources and include the citations from source documents used to generate answers. If you are interested in becoming a beta tester for StretchAI, please sign up below.

Leaders' Guide to Artificial Intelligence

School leaders must ensure the use of AI is thoughtful and appropriate, and supports the district’s vision. Download this free guide  (or the UK version ) to get the background you need to guide your district in an AI-infused world.

UPDATED! Free Guides for Engaging Students in AI Creation

ISTE and GM have partnered to create Hands-On AI Projects for the Classroom guides to provide educators with a variety of activities to teach students about AI across various grade levels and subject areas. Each guide includes background information for teachers and student-driven project ideas that relate to subject-area standards. 

The hands-on activities in the guides range from “unplugged” projects to explore the basic concepts of how AI works to creating chatbots and simple video games with AI, allowing students to work directly with innovative AI technologies and demonstrate their learning. 

These updated hands-on guides are available in downloadable PDF format in English, Spanish and Arabic from the list below.

Artificial Intelligence Explorations for Educators unpacks everything educators need to know about bringing AI to the classroom. Sign up for the next course and find out how to earn graduate-level credit for completing the course.

As a co-founder of  TeachAI , ISTE provides guidance to support school leaders and policy makers around leveraging AI for learning.

Dive deeper into AI and learn how to navigate ChatGPT in schools with curated resources and tools  from ASCD and ISTE.

Join our Educator AI Community on Connect

ISTE+ASCD’s free online community brings together educators from around the world to share ideas and best practices for using artificial intelligence to support learning.

Learn More From These Podcasts, Blog Posts, Case Studies and Websites

Partners Code.org, ETS, ISTE and Khan Academy offer engaging sessions with renowned experts to demystify AI, explore responsible implementation, address bias, and showcase how AI-powered learning can revolutionize student outcomes

One of the challenges with bias in AI comes down to who has access to these careers in the first place, and that's the area that Tess Posner, CEO of the nonprofit AI4All, is trying to address.

Featuring in-depth interviews with practitioners, guidelines for classroom teachers and a webinar about the importance of AI in education, this site provides K-12 educators with practical tools for integrating AI and computational thinking across their curricula.

This 15-hour, self-paced introduction to artificial intelligence is designed for students in grades 9-12. Educators and students should create a free account at P-TECH before viewing the course.

Explore More in the Learning Library

Explore more books, articles, and tools about artificial intelligence in the Learning Library.

• artificial intelligence

Become an Insider

Sign up today to receive premium content.

How to Enact an AI Policy in Your K–12 Schools

Bryan Krause is a K–12 Education Strategist for CDW Education. He is a former teacher, coach and district administrator with more than 30 years of experience in Colorado. He was principal of a school that suffered a school shooting and has shared his experience and learnings with numerous school districts and organizations nationally. In addition, he has led response teams in numerous school crisis situations to help in response and recovery.

Wendy Jones is a K–12 Education Strategist Manager for CDW•G.

As K–12 leaders plan for the return to school, many should consider the need for a policy on artificial intelligence. AI conversations are being had in ed tech circles, with experts and thought leaders discussing the technology as it advances. At this year’s Consortium for School Networking conference in March, panelists discussed ChatGPT and its place in schools. At ISTELive 23, presenters urged schools to have an AI policy in place, adding that it’s better to have a policy that can be revised than to have no guidance.

In talking about AI, and looking for advice to create a policy, it’s important to remember that AI and machine learning have been around for years. While ChatGPT accelerated the interest in generative AI, many technologies in schools already rely on AI or ML to some extent. Microsoft and Google both use the tech for features as familiar as autocomplete.

There’s AI in school safety technology as well. Cybersecurity tools such as web filtering and network monitoring rely on the tech, and physical security solutions such as next-generation cameras use AI for license plate recognition and more.

Understanding how AI works and, more importantly, how it doesn’t work is key for modern school leaders.

Click the banner to unlock complimentary resources from CDW for your modern K–12 classroom.

Navigate Common Fears and Misconceptions Around AI in Schools

Many of the common fears about AI are unfounded. Rumors swirl that AI will replace teachers or that students will stop learning how to read and write.

Instead, some of the most innovative teachers have already begun incorporating AI into lesson plans , using the tool to improve student writing and comprehension.

Others fear the technology will cause classes to stray from the curriculum or render it irrelevant. Yet, in the same way that teachers are integrating this tool into individualized lessons, they are using it to support and further the standards to which they need to teach.

One genuine concern is that of digital equity. If schools ban the use of generative AI in classrooms or on school devices, students with devices and access at home will continue to explore the capabilities of AI. Those who rely on school-issued devices, meanwhile, will fall behind their peers when it comes to AI skills, which experts say will be crucial in the workforce in the near future.

LEARN MORE: How ChatGPT is impacting innovation in K-12 education.

Create AI Policies for Schools That Are Fair and Thoughtful

While it’s important to enact a policy, school leaders should make sure they plan it thoughtfully. To do so, they must gather input from all the necessary stakeholders. This includes school staff — administrators, the IT department and educators — as well as students and the community.

Students are the end users of this tech in many circumstances, and they will need AI skills in future jobs, so their perspective is important to the conversation.

Engaging the community is also essential before creating a policy on AI. This ensures community members are informed about how AI is being used in the classroom and how it works. Including the community can lessen or dispel the fears around this technology.

DIVE DEEPER: What are the pros and cons of ChatGPT in education?

Bookmark Resources on AI in K–12 Schools

School leaders should also turn to trusted resources for recommendations on their AI policies. The U.S. Department of Education recently released insights and recommendations on AI for schools. The White House also put out its Blueprint for an AI Bill of Rights . These resources can give admins a starting point when creating a policy.

The team of education strategists at CDW can also help K–12 IT professionals struggling to navigate an AI policy. They work with school districts and boards of education across the country to provide recommendations, guidance and direction on emerging technologies. Interested school leaders can also attend the AI webinar hosted by CDW Education July 27 .

This article is part of the “ ConnectIT: Bridging the Gap Between Education and Technology ” series.

• Digital Transformation
• IT Governance
• Artificial Intelligence
• Machine Learning

Related Articles

Unlock white papers, personalized recommendations and other premium content for an in-depth look at evolving IT

Copyright © 2024 CDW LLC 200 N. Milwaukee Avenue , Vernon Hills, IL 60061 Do Not Sell My Personal Information

• Research article
• Open access
• Published: 24 April 2023

Artificial intelligence in higher education: the state of the field

• Helen Crompton   ORCID: orcid.org/0000-0002-1775-8219 1 , 3 &
• Diane Burke 2  

International Journal of Educational Technology in Higher Education volume  20 , Article number:  22 ( 2023 ) Cite this article

96k Accesses

96 Citations

64 Altmetric

Metrics details

This systematic review provides unique findings with an up-to-date examination of artificial intelligence (AI) in higher education (HE) from 2016 to 2022. Using PRISMA principles and protocol, 138 articles were identified for a full examination. Using a priori, and grounded coding, the data from the 138 articles were extracted, analyzed, and coded. The findings of this study show that in 2021 and 2022, publications rose nearly two to three times the number of previous years. With this rapid rise in the number of AIEd HE publications, new trends have emerged. The findings show that research was conducted in six of the seven continents of the world. The trend has shifted from the US to China leading in the number of publications. Another new trend is in the researcher affiliation as prior studies showed a lack of researchers from departments of education. This has now changed to be the most dominant department. Undergraduate students were the most studied students at 72%. Similar to the findings of other studies, language learning was the most common subject domain. This included writing, reading, and vocabulary acquisition. In examination of who the AIEd was intended for 72% of the studies focused on students, 17% instructors, and 11% managers. In answering the overarching question of how AIEd was used in HE, grounded coding was used. Five usage codes emerged from the data: (1) Assessment/Evaluation, (2) Predicting, (3) AI Assistant, (4) Intelligent Tutoring System (ITS), and (5) Managing Student Learning. This systematic review revealed gaps in the literature to be used as a springboard for future researchers, including new tools, such as Chat GPT.

A systematic review examining AIEd in higher education (HE) up to the end of 2022.

Unique findings in the switch from US to China in the most studies published.

A two to threefold increase in studies published in 2021 and 2022 to prior years.

AIEd was used for: Assessment/Evaluation, Predicting, AI Assistant, Intelligent Tutoring System, and Managing Student Learning.

Introduction

The use of artificial intelligence (AI) in higher education (HE) has risen quickly in the last 5 years (Chu et al., 2022 ), with a concomitant proliferation of new AI tools available. Scholars (viz., Chen et al., 2020 ; Crompton et al., 2020 , 2021 ) report on the affordances of AI to both instructors and students in HE. These benefits include the use of AI in HE to adapt instruction to the needs of different types of learners (Verdú et al., 2017 ), in providing customized prompt feedback (Dever et al., 2020 ), in developing assessments (Baykasoğlu et al., 2018 ), and predict academic success (Çağataylı & Çelebi, 2022 ). These studies help to inform educators about how artificial intelligence in education (AIEd) can be used in higher education.

Nonetheless, a gap has been highlighted by scholars (viz., Hrastinski et al., 2019 ; Zawacki-Richter et al., 2019 ) regarding an understanding of the collective affordances provided through the use of AI in HE. Therefore, the purpose of this study is to examine extant research from 2016 to 2022 to provide an up-to-date systematic review of how AI is being used in the HE context.

Artificial intelligence has become pervasive in the lives of twenty-first century citizens and is being proclaimed as a tool that can be used to enhance and advance all sectors of our lives (Górriz et al., 2020 ). The application of AI has attracted great interest in HE which is highly influenced by the development of information and communication technologies (Alajmi et al., 2020 ). AI is a tool used across subject disciplines, including language education (Liang et al., 2021 ), engineering education (Shukla et al., 2019 ), mathematics education (Hwang & Tu, 2021 ) and medical education (Winkler-Schwartz et al., 2019 ),

Artificial intelligence

The term artificial intelligence is not new. It was coined in 1956 by McCarthy (Cristianini, 2016 ) who followed up on the work of Turing (e.g., Turing, 1937 , 1950 ). Turing described the existence of intelligent reasoning and thinking that could go into intelligent machines. The definition of AI has grown and changed since 1956, as there has been significant advancements in AI capabilities. A current definition of AI is “computing systems that are able to engage in human-like processes such as learning, adapting, synthesizing, self-correction and the use of data for complex processing tasks” (Popenici et al., 2017 , p. 2). The interdisciplinary interest from scholars from linguistics, psychology, education, and neuroscience who connect AI to nomenclature, perceptions and knowledge in their own disciplines could create a challenge when defining AI. This has created the need to create categories of AI within specific disciplinary areas. This paper focuses on the category of AI in Education (AIEd) and how AI is specifically used in higher educational contexts.

As the field of AIEd is growing and changing rapidly, there is a need to increase the academic understanding of AIEd. Scholars (viz., Hrastinski et al., 2019 ; Zawacki-Richter et al., 2019 ) have drawn attention to the need to increase the understanding of the power of AIEd in educational contexts. The following section provides a summary of the previous research regarding AIEd.

Extant systematic reviews

This growing interest in AIEd has led scholars to investigate the research on the use of artificial intelligence in education. Some scholars have conducted systematic reviews to focus on a specific subject domain. For example, Liang et. al. ( 2021 ) conducted a systematic review and bibliographic analysis the roles and research foci of AI in language education. Shukla et. al. ( 2019 ) focused their longitudinal bibliometric analysis on 30 years of using AI in Engineering. Hwang and Tu ( 2021 ) conducted a bibliometric mapping analysis on the roles and trends in the use of AI in mathematics education, and Winkler-Schwartz et. al. ( 2019 ) specifically examined the use of AI in medical education in looking for best practices in the use of machine learning to assess surgical expertise. These studies provide a specific focus on the use of AIEd in HE but do not provide an understanding of AI across HE.

On a broader view of AIEd in HE, Ouyang et. al. ( 2022 ) conducted a systematic review of AIEd in online higher education and investigated the literature regarding the use of AI from 2011 to 2020. The findings show that performance prediction, resource recommendation, automatic assessment, and improvement of learning experiences are the four main functions of AI applications in online higher education. Salas-Pilco and Yang ( 2022 ) focused on AI applications in Latin American higher education. The results revealed that the main AI applications in higher education in Latin America are: (1) predictive modeling, (2) intelligent analytics, (3) assistive technology, (4) automatic content analysis, and (5) image analytics. These studies provide valuable information for the online and Latin American context but not an overarching examination of AIEd in HE.

Studies have been conducted to examine HE. Hinojo-Lucena et. al. ( 2019 ) conducted a bibliometric study on the impact of AIEd in HE. They analyzed the scientific production of AIEd HE publications indexed in Web of Science and Scopus databases from 2007 to 2017. This study revealed that most of the published document types were proceedings papers. The United States had the highest number of publications, and the most cited articles were about implementing virtual tutoring to improve learning. Chu et. al. ( 2022 ) reviewed the top 50 most cited articles on AI in HE from 1996 to 2020, revealing that predictions of students’ learning status were most frequently discussed. AI technology was most frequently applied in engineering courses, and AI technologies most often had a role in profiling and prediction. Finally, Zawacki-Richter et. al. ( 2019 ) analyzed AIEd in HE from 2007 to 2018 to reveal four primary uses of AIEd: (1) profiling and prediction, (2) assessment and evaluation, (3) adaptive systems and personalization, and (4) intelligent tutoring systems. There do not appear to be any studies examining the last 2 years of AIEd in HE, and these authors describe the rapid speed of both AI development and the use of AIEd in HE and call for further research in this area.

Purpose of the study

The purpose of this study is in response to the appeal from scholars (viz., Chu et al., 2022 ; Hinojo-Lucena et al., 2019 ; Zawacki-Richter et al., 2019 ) to research to investigate the benefits and challenges of AIEd within HE settings. As the academic knowledge of AIEd HE finished with studies examining up to 2020, this study provides the most up-to-date analysis examining research through to the end of 2022.

The overarching question for this study is: what are the trends in HE research regarding the use of AIEd? The first two questions provide contextual information, such as where the studies occurred and the disciplines AI was used in. These contextual details are important for presenting the main findings of the third question of how AI is being used in HE.

In what geographical location was the AIEd research conducted, and how has the trend in the number of publications evolved across the years?

What departments were the first authors affiliated with, and what were the academic levels and subject domains in which AIEd research was being conducted?

Who are the intended users of the AI technologies and what are the applications of AI in higher education?

A PRISMA systematic review methodology was used to answer three questions guiding this study. PRISMA principles (Page et al., 2021 ) were used throughout the study. The PRISMA extension Preferred Reporting Items for Systematic Reviews and Meta-Analysis for Protocols (PRISMA-P; Moher et al., 2015 ) were utilized in this study to provide an a priori roadmap to conduct a rigorous systematic review. Furthermore, the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA principles; Page et al., 2021 ) were used to search, identify, and select articles to be included in the research were used for searching, identifying, and selecting articles, then in how to read, extract, and manage the secondary data gathered from those studies (Moher et al., 2015 , PRISMA Statement, 2021 ). This systematic review approach supports an unbiased synthesis of the data in an impartial way (Hemingway & Brereton, 2009 ). Within the systematic review methodology, extracted data were aggregated and presented as whole numbers and percentages. A qualitative deductive and inductive coding methodology was also used to analyze extant data and generate new theories on the use of AI in HE (Gough et al., 2017 ).

The research begins with the search for the research articles to be included in the study. Based on the research question, the study parameters are defined including the search years, quality and types of publications to be included. Next, databases and journals are selected. A Boolean search is created and used for the search of those databases and journals. Once a set of publications are located from those searches, they are then examined against an inclusion and exclusion criteria to determine which studies will be included in the final study. The relevant data to match the research questions is then extracted from the final set of studies and coded. This method section is organized to describe each of these methods with full details to ensure transparency.

Search strategy

Only peer-reviewed journal articles were selected for examination in this systematic review. This ensured a level of confidence in the quality of the studies selected (Gough et al., 2017 ). The search parameters narrowed the search focus to include studies published in 2016 to 2022. This timeframe was selected to ensure the research was up to date, which is especially important with the rapid change in technology and AIEd.

The data retrieval protocol employed an electronic and a hand search. The electronic search included educational databases within EBSCOhost. Then an additional electronic search was conducted of Wiley Online Library, JSTOR, Science Direct, and Web of Science. Within each of these databases a full text search was conducted. Aligned to the research topic and questions, the Boolean search included terms related to AI, higher education, and learning. The Boolean search is listed in Table 1 . In the initial test search, the terms “machine learning” OR “intelligent support” OR “intelligent virtual reality” OR “chatbot” OR “automated tutor” OR “intelligent agent” OR “expert system” OR “neural network” OR “natural language processing” were used. These were removed as they were subcategories of terms found in Part 1 of the search. Furthermore, inclusion of these specific AI terms resulted in a large number of computer science courses that were focused on learning about AI and not the use of AI in learning.

Part 2 of the search ensured that articles involved formal university education. The terms higher education and tertiary were both used to recognize the different terms used in different countries. The final Boolean search was “Artificial intelligence” OR AI OR “smart technologies” OR “intelligent technologies” AND “higher education” OR tertiary OR graduate OR undergraduate. Scholars (viz., Ouyang et al., 2022 ) who conducted a systematic review on AIEd in HE up to 2020 noted that they missed relevant articles from their study, and other relevant journals should intentionally be examined. Therefore, a hand search was also conducted to include an examination of other journals relevant to AIEd that may not be included in the databases. This is important as the field of AIEd is still relatively new, and journals focused on this field may not yet be indexed in databases. The hand search included: The International Journal of Learning Analytics and Artificial Intelligence in Education, the International Journal of Artificial Intelligence in Education, and Computers & Education: Artificial Intelligence.

Electronic and hand searches resulted in 371 articles for possible inclusion. The search parameters within the electronic database search narrowed the search to articles published from 2016 to 2022, per-reviewed journal articles, and duplicates. Further screening was conducted manually, as each of the 138 articles were reviewed in full by two researchers to examine a match against the inclusion and exclusion criteria found in Table 2 .

The inter-rater reliability was calculated by percentage agreement (Belur et al., 2018 ). The researchers reached a 95% agreement for the coding. Further discussion of misaligned articles resulted in a 100% agreement. This screening process against inclusion and exclusion criteria resulted in the exclusion of 237 articles. This included the duplicates and those removed as part of the inclusion and exclusion criteria, see Fig.  1 . Leaving 138 articles for inclusion in this systematic review.

(From: Page et al., 2021 )

PRISMA flow chart of article identification and screening

The 138 articles were then coded to answer each of the research questions using deductive and inductive coding methods. Deductive coding involves examining data using a priori codes. A priori are pre-determined criteria and this process was used to code the countries, years, author affiliations, academic levels, and domains in the respective groups. Author affiliations were coded using the academic department of the first author of the study. First authors were chosen as that person is the primary researcher of the study and this follows past research practice (e.g., Zawacki-Richter et al., 2019 ). Who the AI was intended for was also coded using the a priori codes of Student, Instructor, Manager or Others. The Manager code was used for those who are involved in organizational tasks, e.g., tracking enrollment. Others was used for those not fitting the other three categories.

Inductive coding was used for the overarching question of this study in examining how the AI was being used in HE. Researchers of extant systematic reviews on AIEd in HE (viz., Chu et al., 2022 ; Zawacki-Richter et al., 2019 ) often used an a priori framework as researchers matched the use of AI to pre-existing frameworks. A grounded coding methodology (Strauss & Corbin, 1995 ) was selected for this study to allow findings of the trends on AIEd in HE to emerge from the data. This is important as it allows a direct understanding of how AI is being used rather than how researchers may think it is being used and fitting the data to pre-existing ideas.

Grounded coding process involved extracting how the AI was being used in HE from the articles. “In vivo” (Saldana, 2015 ) coding was also used alongside grounded coding. In vivo codes are when codes use language directly from the article to capture the primary authors’ language and ensure consistency with their findings. The grounded coding design used a constant comparative method. Researchers identified important text from articles related to the use of AI, and through an iterative process, initial codes led to axial codes with a constant comparison of uses of AI with uses of AI, then of uses of AI with codes, and codes with codes. Codes were deemed theoretically saturated when the majority of the data fit with one of the codes. For both the a priori and the grounded coding, two researchers coded and reached an inter-rater percentage agreement of 96%. After discussing misaligned articles, a 100% agreement was achieved.

Findings and discussion

The findings and discussion section are organized by the three questions guiding this study. The first two questions provide contextual information on the AIEd research, and the final question provides a rigorous investigation into how AI is being used in HE.

RQ1. In what geographical location was the AIEd research conducted, and how has the trend in the number of publications evolved across the years?

The 138 studies took place across 31 countries in six of seven continents of the world. Nonetheless, that distribution was not equal across continents. Asia had the largest number of AIEd studies in HE at 41%. Of the seven countries represented in Asia, 42 of the 58 studies were conducted in Taiwan and China. Europe, at 30%, was the second largest continent and had 15 countries ranging from one to eight studies a piece. North America, at 21% of the studies was the continent with the third largest number of studies, with the USA producing 21 of the 29 studies in that continent. The 21 studies from the USA places it second behind China. Only 1% of studies were conducted in South America and 2% in Africa. See Fig.  2 for a visual representation of study distribution across countries. Those continents with high numbers of studies are from high income countries and those with low numbers have a paucity of publications in low-income countries.

Geographical distribution of the AIEd HE studies

Data from Zawacki-Richter et. al.’s ( 2019 ) 2007–2018 systematic review examining countries found that the USA conducted the most studies across the globe at 43 out of 146, and China had the second largest at eleven of the 146 papers. Researchers have noted a rapid trend in Chinese researchers publishing more papers on AI and securing more patents than their US counterparts in a field that was originally led by the US (viz., Li et al., 2021 ). The data from this study corroborate this trend in China leading in the number of AIEd publications.

With the accelerated use of AI in society, gathering data to examine the use of AIEd in HE is useful in providing the scholarly community with specific information on that growth and if it is as prolific as anticipated by scholars (e.g., Chu et al., 2022 ). The analysis of data of the 138 studies shows that the trend towards the use of AIEd in HE has greatly increased. There is a drop in 2019, but then a great rise in 2021 and 2022; see Fig.  3 .

Chronological trend in AIEd in HE

Data on the rise in AIEd in HE is similar to the findings of Chu et. al. ( 2022 ) who noted an increase from 1996 to 2010 and 2011–2020. Nonetheless Chu’s parameters are across decades, and the rise is to be anticipated with a relatively new technology across a longitudinal review. Data from this study show a dramatic rise since 2020 with a 150% increase from the prior 2 years 2020–2019. The rise in 2021 and 2022 in HE could have been caused by the vast increase in HE faculty having to teach with technology during the pandemic lockdown. Faculty worldwide were using technologies, including AI, to explore how they could continue teaching and learning that was often face-to-face prior to lockdown. The disadvantage of this rapid adoption of technology is that there was little time to explore the possibilities of AI to transform learning, and AI may have been used to replicate past teaching practices, without considering new strategies previously inconceivable with the affordances of AI.

However, in a further examination of the research from 2021 to 2022, it appears that there are new strategies being considered. For example, Liu et. al.’s, 2022 study used AIEd to provide information on students’ interactions in an online environment and examine their cognitive effort. In Yao’s study in 2022, he examined the use of AI to determine student emotions while learning.

RQ2. What departments were the first authors affiliated with, and what were the academic levels and subject domains in which AIEd research was being conducted?

Department affiliations

Data from the AIEd HE studies show that of the first authors were most frequently from colleges of education (28%), followed by computer science (20%). Figure  4 presents the 15 academic affiliations of the authors found in the studies. The wide variety of affiliations demonstrate the variety of ways AI can be used in various educational disciplines, and how faculty in diverse areas, including tourism, music, and public affairs were interested in how AI can be used for educational purposes.

Research affiliations

In an extant AIED HE systematic review, Zawacki-Richter et. al.’s ( 2019 ) named their study Systematic review of research on artificial intelligence applications in higher education—where are the educators? In this study, the authors were keen to highlight that of the AIEd studies in HE, only six percent were written by researchers directly connected to the field of education, (i.e., from a college of education). The researchers found a great lack in pedagogical and ethical implications of implementing AI in HE and that there was a need for more educational perspectives on AI developments from educators conducting this work. It appears from our data that educators are now showing greater interest in leading these research endeavors, with the highest affiliated group belonging to education. This may again be due to the pandemic and those in the field of education needing to support faculty in other disciplines, and/or that they themselves needed to explore technologies for their own teaching during the lockdown. This may also be due to uptake in professors in education becoming familiar with AI tools also driven by a societal increased attention. As the focus of much research by education faculty is on teaching and learning, they are in an important position to be able to share their research with faculty in other disciplines regarding the potential affordances of AIEd.

Academic levels

The a priori coding of academic levels show that the majority of studies involved undergraduate students with 99 of the 138 (72%) focused on these students. This was in comparison to the 12 of 138 (9%) for graduate students. Some of the studies used AI for both academic levels: see Fig.  5

Academic level distribution by number of articles

This high percentage of studies focused on the undergraduate population was congruent with an earlier AIED HE systematic review (viz., Zawacki-Richter et al., 2019 ) who also reported student academic levels. This focus on undergraduate students may be due to the variety of affordances offered by AIEd, such as predictive analytics on dropouts and academic performance. These uses of AI may be less required for graduate students who already have a record of performance from their undergraduate years. Another reason for this demographic focus can also be convenience sampling, as researchers in HE typically has a much larger and accessible undergraduate population than graduates. This disparity between undergraduates and graduate populations is a concern, as AIEd has the potential to be valuable in both settings.

Subject domains

The studies were coded into 14 areas in HE; with 13 in a subject domain and one category of AIEd used in HE management of students; See Fig.  6 . There is not a wide difference in the percentages of top subject domains, with language learning at 17%, computer science at 16%, and engineering at 12%. The management of students category appeared third on the list at 14%. Prior studies have also found AIEd often used for language learning (viz., Crompton et al., 2021 ; Zawacki-Richter et al., 2019 ). These results are different, however, from Chu et. al.’s ( 2022 ) findings that show engineering dramatically leading with 20 of the 50 studies, with other subjects, such as language learning, appearing once or twice. This study appears to be an outlier that while the searches were conducted in similar databases, the studies only included 50 studies from 1996 to 2020.

Subject domains of AIEd in HE

Previous scholars primarily focusing on language learning using AI for writing, reading, and vocabulary acquisition used the affordances of natural language processing and intelligent tutoring systems (e.g., Liang et al., 2021 ). This is similar to the findings in studies with AI used for automated feedback of writing in a foreign language (Ayse et al., 2022 ), and AI translation support (Al-Tuwayrish, 2016 ). The large use of AI for managerial activities in this systematic review focused on making predictions (12 studies) and then admissions (three studies). This is positive to see this use of AI to look across multiple databases to see trends emerging from data that may not have been anticipated and cross referenced before (Crompton et al., 2022 ). For example, to examine dropouts, researchers may consider examining class attendance, and may not examine other factors that appear unrelated. AI analysis can examine all factors and may find that dropping out is due to factors beyond class attendance.

RQ3. Who are the intended users of the AI technologies and what are the applications of AI in higher education?

Intended user of AI

Of the 138 articles, the a priori coding shows that 72% of the studies focused on Students, followed by a focus on Instructors at 17%, and Managers at 11%, see Fig.  7 . The studies provided examples of AI being used to provide support to students, such as access to learning materials for inclusive learning (Gupta & Chen, 2022 ), provide immediate answers to student questions, self-testing opportunities (Yao, 2022 ), and instant personalized feedback (Mousavi et al., 2020 ).

Intended user

The data revealed a large emphasis on students in the use of AIEd in HE. This user focus is different from a recent systematic review on AIEd in K-12 that found that AIEd studies in K-12 settings prioritized teachers (Crompton et al., 2022 ). This may appear that HE uses AI to focus more on students than in K-12. However, this large number of student studies in HE may be due to the student population being more easily accessibility to HE researchers who may study their own students. The ethical review process is also typically much shorter in HE than in K-12. Therefore, the data on the intended focus should be reviewed while keeping in mind these other explanations. It was interesting that Managers were the lowest focus in K-12 and also in this study in HE. AI has great potential to collect, cross reference and examine data across large datasets that can allow data to be used for actionable insight. More focus on the use of AI by managers would tap into this potential.

How is AI used in HE

Using grounded coding, the use of AIEd from each of the 138 articles was examined and six major codes emerged from the data. These codes provide insight into how AI was used in HE. The five codes are: (1) Assessment/Evaluation, (2) Predicting, (3) AI Assistant, (4) Intelligent Tutoring System (ITS), and (5) Managing Student Learning. For each of these codes there are also axial codes, which are secondary codes as subcategories from the main category. Each code is delineated below with a figure of the codes with further descriptive information and examples.

Assessment/evaluation

Assessment and Evaluation was the most common use of AIEd in HE. Within this code there were six axial codes broken down into further codes; see Fig.  8 . Automatic assessment was most common, seen in 26 of the studies. It was interesting to see that this involved assessment of academic achievement, but also other factors, such as affect.

Codes and axial codes for assessment and evaluation

Automatic assessment was used to support a variety of learners in HE. As well as reducing the time it takes for instructors to grade (Rutner & Scott, 2022 ), automatic grading showed positive use for a variety of students with diverse needs. For example, Zhang and Xu ( 2022 ) used automatic assessment to improve academic writing skills of Uyghur ethnic minority students living in China. Writing has a variety of cultural nuances and in this study the students were shown to engage with the automatic assessment system behaviorally, cognitively, and affectively. This allowed the students to engage in self-regulated learning while improving their writing.

Feedback was a description often used in the studies, as students were given text and/or images as feedback as a formative evaluation. Mousavi et. al. ( 2020 ) developed a system to provide first year biology students with an automated personalized feedback system tailored to the students’ specific demographics, attributes, and academic status. With the unique feature of AIEd being able to analyze multiple data sets involving a variety of different students, AI was used to assess and provide feedback on students’ group work (viz., Ouatik et al., 2021 ).

AI also supports instructors in generating questions and creating multiple question tests (Yang et al., 2021 ). For example, (Lu et al., 2021 ) used natural language processing to create a system that automatically created tests. Following a Turing type test, researchers found that AI technologies can generate highly realistic short-answer questions. The ability for AI to develop multiple questions is a highly valuable affordance as tests can take a great deal of time to make. However, it would be important for instructors to always confirm questions provided by the AI to ensure they are correct and that they match the learning objectives for the class, especially in high value summative assessments.

The axial code within assessment and evaluation revealed that AI was used to review activities in the online space. This included evaluating student’s reflections, achievement goals, community identity, and higher order thinking (viz., Huang et al., 2021 ). Three studies used AIEd to evaluate educational materials. This included general resources and textbooks (viz., Koć‑Januchta et al., 2022 ). It is interesting to see the use of AI for the assessment of educational products, rather than educational artifacts developed by students. While this process may be very similar in nature, this shows researchers thinking beyond the traditional use of AI for assessment to provide other affordances.

Predicting was a common use of AIEd in HE with 21 studies focused specifically on the use of AI for forecasting trends in data. Ten axial codes emerged on the way AI was used to predict different topics, with nine focused on predictions regarding students and the other on predicting the future of higher education. See Fig.  9 .

Predicting axial codes

Extant systematic reviews on HE highlighted the use of AIEd for prediction (viz., Chu et al., 2022 ; Hinojo-Lucena et al., 2019 ; Ouyang et al., 2022 ; Zawacki-Richter et al., 2019 ). Ten of the articles in this study used AI for predicting academic performance. Many of the axial codes were often overlapping, such as predicting at risk students, and predicting dropouts; however, each provided distinct affordances. An example of this is the study by Qian et. al. ( 2021 ). These researchers examined students taking a MOOC course. MOOCs can be challenging environments to determine information on individual students with the vast number of students taking the course (Krause & Lowe, 2014 ). However, Qian et al., used AIEd to predict students’ future grades by inputting 17 different learning features, including past grades, into an artificial neural network. The findings were able to predict students’ grades and highlight students at risk of dropping out of the course.

In a systematic review on AIEd within the K-12 context (viz., Crompton et al., 2022 ), prediction was less pronounced in the findings. In the K-12 setting, there was a brief mention of the use of AI in predicting student academic performance. One of the studies mentioned students at risk of dropping out, but this was immediately followed by questions about privacy concerns and describing this as “sensitive”. The use of prediction from the data in this HE systematic review cover a wide range of AI predictive affordances. students Sensitivity is still important in a HE setting, but it is positive to see the valuable insight it provides that can be used to avoid students failing in their goals.

AI assistant

The studies evaluated in this review indicated that the AI Assistant used to support learners had a variety of different names. This code included nomenclature such as, virtual assistant, virtual agent, intelligent agent, intelligent tutor, and intelligent helper. Crompton et. al. ( 2022 ), described the difference in the terms to delineate the way that the AI appeared to the user. For example, if there was an anthropomorphic presence to the AI, such as an avatar, or if the AI appeared to support via other means, such as text prompt. The findings of this systematic review align to Crompton et. al.’s ( 2022 ) descriptive differences of the AI Assistant. Furthermore, this code included studies that provide assistance to students, but may not have specifically used the word assistance. These include the use of chatbots for student outreach, answering questions, and providing other assistance. See Fig.  10 for the axial codes for AI Assistant.

AI assistant axial codes

Many of these assistants offered multiple supports to students, such as Alex , the AI described as a virtual change agent in Kim and Bennekin’s ( 2016 ) study. Alex interacted with students in a college mathematics course by asking diagnostic questions and gave support depending on student needs. Alex’s support was organized into four stages: (1) goal initiation (“Want it”), (2) goal formation (“Plan for it”), (3) action control (“Do it”), and (4) emotion control (“Finish it”). Alex provided responses depending on which of these four areas students needed help. These messages supported students with the aim of encouraging persistence in pursuing their studies and degree programs and improving performance.

The role of AI in providing assistance connects back to the seminal work of Vygotsky ( 1978 ) and the Zone of Proximal Development (ZPD). ZPD highlights the degree to which students can rapidly develop when assisted. Vygotsky described this assistance often in the form of a person. However, with technological advancements, the use of AI assistants in these studies are providing that support for students. The affordances of AI can also ensure that the support is timely without waiting for a person to be available. Also, assistance can consider aspects on students’ academic ability, preferences, and best strategies for supporting. These features were evident in Kim and Bennekin’s ( 2016 ) study using Alex.

Intelligent tutoring system

The use of Intelligent Tutoring Systems (ITS) was revealed in the grounded coding. ITS systems are adaptive instructional systems that involve the use of AI techniques and educational methods. An ITS system customizes educational activities and strategies based on student’s characteristics and needs (Mousavinasab et al., 2021 ). While ITS may be an anticipated finding in AIED HE systematic reviews, it was interesting that extant reviews similar to this study did not always describe their use in HE. For example, Ouyang et. al. ( 2022 ), included “intelligent tutoring system” in search terms describing it as a common technique, yet ITS was not mentioned again in the paper. Zawacki-Richter et. al. ( 2019 ) on the other hand noted that ITS was in the four overarching findings of the use of AIEd in HE. Chu et. al. ( 2022 ) then used Zawacki-Richter’s four uses of AIEd for their recent systematic review.

In this systematic review, 18 studies specifically mentioned that they were using an ITS. The ITS code did not necessitate axial codes as they were performing the same type of function in HE, namely, in providing adaptive instruction to the students. For example, de Chiusole et. al. ( 2020 ) developed Stat-Knowlab, an ITS that provides the level of competence and best learning path for each student. Thus Stat-Knowlab personalizes students’ learning and provides only educational activities that the student is ready to learn. This ITS is able to monitor the evolution of the learning process as the student interacts with the system. In another study, Khalfallah and Slama ( 2018 ) built an ITS called LabTutor for engineering students. LabTutor served as an experienced instructor in enabling students to access and perform experiments on laboratory equipment while adapting to the profile of each student.

The student population in university classes can go into the hundreds and with the advent of MOOCS, class sizes can even go into the thousands. Even in small classes of 20 students, the instructor cannot physically provide immediate unique personalize questions to each student. Instructors need time to read and check answers and then take further time to provide feedback before determining what the next question should be. Working with the instructor, AIEd can provide that immediate instruction, guidance, feedback, and following questioning without delay or becoming tired. This appears to be an effective use of AIEd, especially within the HE context.

Managing student learning

Another code that emerged in the grounded coding was focused on the use of AI for managing student learning. AI is accessed to manage student learning by the administrator or instructor to provide information, organization, and data analysis. The axial codes reveal the trends in the use of AI in managing student learning; see Fig.  11 .

Learning analytics was an a priori term often found in studies which describes “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Long & Siemens, 2011 , p. 34). The studies investigated in this systematic review were across grades and subject areas and provided administrators and instructors different types of information to guide their work. One of those studies was conducted by Mavrikis et. al. ( 2019 ) who described learning analytics as teacher assistance tools. In their study, learning analytics were used in an exploratory learning environment with targeted visualizations supporting classroom orchestration. These visualizations, displayed as screenshots in the study, provided information such as the interactions between the students, goals achievements etc. These appear similar to infographics that are brightly colored and draw the eye quickly to pertinent information. AI is also used for other tasks, such as organizing the sequence of curriculum in pacing guides for future groups of students and also designing instruction. Zhang ( 2022 ) described how designing an AI teaching system of talent cultivation and using the digital affordances to establish a quality assurance system for practical teaching, provides new mechanisms for the design of university education systems. In developing such a system, Zhang found that the stability of the instructional design, overcame the drawbacks of traditional manual subjectivity in the instructional design.

Another trend that emerged from the studies was the use of AI to manage student big data to support learning. Ullah and Hafiz ( 2022 ) lament that using traditional methods, including non-AI digital techniques, asking the instructor to pay attention to every student’s learning progress is very difficult and that big data analysis techniques are needed. The ability to look across and within large data sets to inform instruction is a valuable affordance of AIEd in HE. While the use of AIEd to manage student learning emerged from the data, this study uncovered only 19 studies in 7 years (2016–2022) that focused on the use of AIEd to manage student data. This lack of the use was also noted in a recent study in the K-12 space (Crompton et al., 2022 ). In Chu et. al.’s ( 2022 ) study examining the top 50 most cited AIEd articles, they did not report the use of AIEd for managing student data in the top uses of AIEd HE. It would appear that more research should be conducted in this area to fully explore the possibilities of AI.

Gaps and future research

From this systematic review, six gaps emerged in the data providing opportunities for future studies to investigate and provide a fuller understanding of how AIEd can used in HE. (1) The majority of the research was conducted in high income countries revealing a paucity of research in developing countries. More research should be conducted in these developing countries to expand the level of understanding about how AI can enhance learning in under-resourced communities. (2) Almost 50% of the studies were conducted in the areas of language learning, computer science and engineering. Research conducted by members from multiple, different academic departments would help to advance the knowledge of the use of AI in more disciplines. (3) This study revealed that faculty affiliated with schools of education are taking an increasing role in researching the use of AIEd in HE. As this body of knowledge grows, faculty in Schools of Education should share their research regarding the pedagogical affordances of AI so that this knowledge can be applied by faculty across disciplines. (4) The vast majority of the research was conducted at the undergraduate level. More research needs to be done at the graduate student level, as AI provides many opportunities in this environment. (5) Little study was done regarding how AIEd can assist both instructors and managers in their roles in HE. The power of AI to assist both groups further research. (6) Finally, much of the research investigated in this systematic review revealed the use of AIEd in traditional ways that enhance or make more efficient current practices. More research needs to focus on the unexplored affordances of AIEd. As AI becomes more advanced and sophisticated, new opportunities will arise for AIEd. Researchers need to be on the forefront of these possible innovations.

In addition, empirical exploration is needed for new tools, such as ChatGPT that was available for public use at the end of 2022. With the time it takes for a peer review journal article to be published, ChatGPT did not appear in the articles for this study. What is interesting is that it could fit with a variety of the use codes found in this study, with students getting support in writing papers and instructors using Chat GPT to assess students work and with help writing emails or descriptions for students. It would be pertinent for researchers to explore Chat GPT.

Limitations

The findings of this study show a rapid increase in the number of AIEd studies published in HE. However, to ensure a level of credibility, this study only included peer review journal articles. These articles take months to publish. Therefore, conference proceedings and gray literature such as blogs and summaries may reveal further findings not explored in this study. In addition, the articles in this study were all published in English which excluded findings from research published in other languages.

In response to the call by Hinojo-Lucena et. al. ( 2019 ), Chu et. al. ( 2022 ), and Zawacki-Richter et. al. ( 2019 ), this study provides unique findings with an up-to-date examination of the use of AIEd in HE from 2016 to 2022. Past systematic reviews examined the research up to 2020. The findings of this study show that in 2021 and 2022, publications rose nearly two to three times the number of previous years. With this rapid rise in the number of AIEd HE publications, new trends have emerged.

The findings show that of the 138 studies examined, research was conducted in six of the seven continents of the world. In extant systematic reviews showed that the US led by a large margin in the number of studies published. This trend has now shifted to China. Another shift in AIEd HE is that while extant studies lamented the lack of focus on professors of education leading these studies, this systematic review found education to be the most common department affiliation with 28% and computer science coming in second at 20%. Undergraduate students were the most studied students at 72%. Similar to the findings of other studies, language learning was the most common subject domain. This included writing, reading, and vocabulary acquisition. In examination of who the AIEd was intended for, 72% of the studies focused on students, 17% instructors, and 11% managers.

Grounded coding was used to answer the overarching question of how AIEd was used in HE. Five usage codes emerged from the data: (1) Assessment/Evaluation, (2) Predicting, (3) AI Assistant, (4) Intelligent Tutoring System (ITS), and (5) Managing Student Learning. Assessment and evaluation had a wide variety of purposes, including assessing academic progress and student emotions towards learning, individual and group evaluations, and class based online community assessments. Predicting emerged as a code with ten axial codes, as AIEd predicted dropouts and at-risk students, innovative ability, and career decisions. AI Assistants were specific to supporting students in HE. These assistants included those with an anthropomorphic presence, such as virtual agents and persuasive intervention through digital programs. ITS systems were not always noted in extant systematic reviews but were specifically mentioned in 18 of the studies in this review. ITS systems in this study provided customized strategies and approaches to student’s characteristics and needs. The final code in this study highlighted the use of AI in managing student learning, including learning analytics, curriculum sequencing, instructional design, and clustering of students.

The findings of this study provide a springboard for future academics, practitioners, computer scientists, policymakers, and funders in understanding the state of the field in AIEd HE, how AI is used. It also provides actionable items to ameliorate gaps in the current understanding. As the use AIEd will only continue to grow this study can serve as a baseline for further research studies in the use of AIEd in HE.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Alajmi, Q., Al-Sharafi, M. A., & Abuali, A. (2020). Smart learning gateways for Omani HEIs towards educational technology: Benefits, challenges and solutions. International Journal of Information Technology and Language Studies, 4 (1), 12–17.

Google Scholar  

Al-Tuwayrish, R. K. (2016). An evaluative study of machine translation in the EFL scenario of Saudi Arabia. Advances in Language and Literary Studies, 7 (1), 5–10.

Ayse, T., & Nil, G. (2022). Automated feedback and teacher feedback: Writing achievement in learning English as a foreign language at a distance. The Turkish Online Journal of Distance Education, 23 (2), 120–139. https://doi.org/10.7575/aiac.alls.v.7n.1p.5

Article   Google Scholar  

Baykasoğlu, A., Özbel, B. K., Dudaklı, N., Subulan, K., & Şenol, M. E. (2018). Process mining based approach to performance evaluation in computer-aided examinations. Computer Applications in Engineering Education, 26 (5), 1841–1861. https://doi.org/10.1002/cae.21971

Belur, J., Tompson, L., Thornton, A., & Simon, M. (2018). Interrater reliability in systematic review methodology: Exploring variation in coder decision-making. Sociological Methods & Research, 13 (3), 004912411887999. https://doi.org/10.1177/0049124118799372

Çağataylı, M., & Çelebi, E. (2022). Estimating academic success in higher education using big five personality traits, a machine learning approach. Arab Journal Scientific Engineering, 47 , 1289–1298. https://doi.org/10.1007/s13369-021-05873-4

Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8 , 75264–75278. https://doi.org/10.1109/ACCESS.2020.2988510

Chu, H., Tu, Y., & Yang, K. (2022). Roles and research trends of artificial intelligence in higher education: A systematic review of the top 50 most-cited articles. Australasian Journal of Educational Technology, 38 (3), 22–42. https://doi.org/10.14742/ajet.7526

Cristianini, N. (2016). Intelligence reinvented. New Scientist, 232 (3097), 37–41. https://doi.org/10.1016/S0262-4079(16)31992-3

Crompton, H., Bernacki, M. L., & Greene, J. (2020). Psychological foundations of emerging technologies for teaching and learning in higher education. Current Opinion in Psychology, 36 , 101–105. https://doi.org/10.1016/j.copsyc.2020.04.011

Crompton, H., & Burke, D. (2022). Artificial intelligence in K-12 education. SN Social Sciences, 2 , 113. https://doi.org/10.1007/s43545-022-00425-5

Crompton, H., Jones, M., & Burke, D. (2022). Affordances and challenges of artificial intelligence in K-12 education: A systematic review. Journal of Research on Technology in Education . https://doi.org/10.1080/15391523.2022.2121344

Crompton, H., & Song, D. (2021). The potential of artificial intelligence in higher education. Revista Virtual Universidad Católica Del Norte, 62 , 1–4. https://doi.org/10.35575/rvuen.n62a1

de Chiusole, D., Stefanutti, L., Anselmi, P., & Robusto, E. (2020). Stat-Knowlab. Assessment and learning of statistics with competence-based knowledge space theory. International Journal of Artificial Intelligence in Education, 30 , 668–700. https://doi.org/10.1007/s40593-020-00223-1

Dever, D. A., Azevedo, R., Cloude, E. B., & Wiedbusch, M. (2020). The impact of autonomy and types of informational text presentations in game-based environments on learning: Converging multi-channel processes data and learning outcomes. International Journal of Artificial Intelligence in Education, 30 (4), 581–615. https://doi.org/10.1007/s40593-020-00215-1

Górriz, J. M., Ramírez, J., Ortíz, A., Martínez-Murcia, F. J., Segovia, F., Suckling, J., Leming, M., Zhang, Y. D., Álvarez-Sánchez, J. R., Bologna, G., Bonomini, P., Casado, F. E., Charte, D., Charte, F., Contreras, R., Cuesta-Infante, A., Duro, R. J., Fernández-Caballero, A., Fernández-Jover, E., … Ferrández, J. M. (2020). Artificial intelligence within the interplay between natural and artificial computation: Advances in data science, trends and applications. Neurocomputing, 410 , 237–270. https://doi.org/10.1016/j.neucom.2020.05.078

Gough, D., Oliver, S., & Thomas, J. (2017). An introduction to systematic reviews (2nd ed.). Sage.

Gupta, S., & Chen, Y. (2022). Supporting inclusive learning using chatbots? A chatbot-led interview study. Journal of Information Systems Education, 33 (1), 98–108.

Hemingway, P. & Brereton, N. (2009). In Hayward Medical Group (Ed.). What is a systematic review? Retrieved from http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/syst-review.pdf

Hinojo-Lucena, F., Arnaz-Diaz, I., Caceres-Reche, M., & Romero-Rodriguez, J. (2019). A bibliometric study on its impact the scientific literature. Education Science . https://doi.org/10.3390/educsci9010051

Hrastinski, S., Olofsson, A. D., Arkenback, C., Ekström, S., Ericsson, E., Fransson, G., Jaldemark, J., Ryberg, T., Öberg, L.-M., Fuentes, A., Gustafsson, U., Humble, N., Mozelius, P., Sundgren, M., & Utterberg, M. (2019). Critical imaginaries and reflections on artificial intelligence and robots in postdigital K-12 education. Postdigital Science and Education, 1 (2), 427–445. https://doi.org/10.1007/s42438-019-00046-x

Huang, C., Wu, X., Wang, X., He, T., Jiang, F., & Yu, J. (2021). Exploring the relationships between achievement goals, community identification and online collaborative reflection. Educational Technology & Society, 24 (3), 210–223.

Hwang, G. J., & Tu, Y. F. (2021). Roles and research trends of artificial intelligence in mathematics education: A bibliometric mapping analysis and systematic review. Mathematics, 9 (6), 584. https://doi.org/10.3390/math9060584

Khalfallah, J., & Slama, J. B. H. (2018). The effect of emotional analysis on the improvement of experimental e-learning systems. Computer Applications in Engineering Education, 27 (2), 303–318. https://doi.org/10.1002/cae.22075

Kim, C., & Bennekin, K. N. (2016). The effectiveness of volition support (VoS) in promoting students’ effort regulation and performance in an online mathematics course. Instructional Science, 44 , 359–377. https://doi.org/10.1007/s11251-015-9366-5

Koć-Januchta, M. M., Schönborn, K. J., Roehrig, C., Chaudhri, V. K., Tibell, L. A. E., & Heller, C. (2022). “Connecting concepts helps put main ideas together”: Cognitive load and usability in learning biology with an AI-enriched textbook. International Journal of Educational Technology in Higher Education, 19 (11), 11. https://doi.org/10.1186/s41239-021-00317-3

Krause, S. D., & Lowe, C. (2014). Invasion of the MOOCs: The promise and perils of massive open online courses . Parlor Press.

Li, D., Tong, T. W., & Xiao, Y. (2021). Is China emerging as the global leader in AI? Harvard Business Review. https://hbr.org/2021/02/is-china-emerging-as-the-global-leader-in-ai

Liang, J. C., Hwang, G. J., Chen, M. R. A., & Darmawansah, D. (2021). Roles and research foci of artificial intelligence in language education: An integrated bibliographic analysis and systematic review approach. Interactive Learning Environments . https://doi.org/10.1080/10494820.2021.1958348

Liu, S., Hu, T., Chai, H., Su, Z., & Peng, X. (2022). Learners’ interaction patterns in asynchronous online discussions: An integration of the social and cognitive interactions. British Journal of Educational Technology, 53 (1), 23–40. https://doi.org/10.1111/bjet.13147

Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. Educause Review, 46 (5), 31–40.

Lu, O. H. T., Huang, A. Y. Q., Tsai, D. C. L., & Yang, S. J. H. (2021). Expert-authored and machine-generated short-answer questions for assessing students learning performance. Educational Technology & Society, 24 (3), 159–173.

Mavrikis, M., Geraniou, E., Santos, S. G., & Poulovassilis, A. (2019). Intelligent analysis and data visualization for teacher assistance tools: The case of exploratory learning. British Journal of Educational Technology, 50 (6), 2920–2942. https://doi.org/10.1111/bjet.12876

Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4 (1), 1–9. https://doi.org/10.1186/2046-4053-4-1

Mousavi, A., Schmidt, M., Squires, V., & Wilson, K. (2020). Assessing the effectiveness of student advice recommender agent (SARA): The case of automated personalized feedback. International Journal of Artificial Intelligence in Education, 31 (2), 603–621. https://doi.org/10.1007/s40593-020-00210-6

Mousavinasab, E., Zarifsanaiey, N., Kalhori, S. R. N., Rakhshan, M., Keikha, L., & Saeedi, M. G. (2021). Intelligent tutoring systems: A systematic review of characteristics, applications, and evaluation methods. Interactive Learning Environments, 29 (1), 142–163. https://doi.org/10.1080/10494820.2018.1558257

Ouatik, F., Ouatikb, F., Fadlic, H., Elgoraria, A., Mohadabb, M. E. L., Raoufia, M., et al. (2021). E-Learning & decision making system for automate students assessment using remote laboratory and machine learning. Journal of E-Learning and Knowledge Society, 17 (1), 90–100. https://doi.org/10.20368/1971-8829/1135285

Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011–2020. Education and Information Technologies, 27 , 7893–7925. https://doi.org/10.1007/s10639-022-10925-9

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T., Mulrow, C., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. British Medical Journal . https://doi.org/10.1136/bmj.n71

Popenici, S. A. D., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning, 12 (22), 1–13. https://doi.org/10.1186/s41039-017-0062-8

PRISMA Statement. (2021). PRISMA endorsers. PRISMA statement website. http://www.prisma-statement.org/Endorsement/PRISMAEndorsers

Qian, Y., Li, C.-X., Zou, X.-G., Feng, X.-B., Xiao, M.-H., & Ding, Y.-Q. (2022). Research on predicting learning achievement in a flipped classroom based on MOOCs by big data analysis. Computer Applied Applications in Engineering Education, 30 , 222–234. https://doi.org/10.1002/cae.22452

Rutner, S. M., & Scott, R. A. (2022). Use of artificial intelligence to grade student discussion boards: An exploratory study. Information Systems Education Journal, 20 (4), 4–18.

Salas-Pilco, S., & Yang, Y. (2022). Artificial Intelligence application in Latin America higher education: A systematic review. International Journal of Educational Technology in Higher Education, 19 (21), 1–20. https://doi.org/10.1186/S41239-022-00326-w

Saldana, J. (2015). The coding manual for qualitative researchers (3rd ed.). Sage.

Shukla, A. K., Janmaijaya, M., Abraham, A., & Muhuri, P. K. (2019). Engineering applications of artificial intelligence: A bibliometric analysis of 30 years (1988–2018). Engineering Applications of Artificial Intelligence, 85 , 517–532. https://doi.org/10.1016/j.engappai.2019.06.010

Strauss, A., & Corbin, J. (1995). Grounded theory methodology: An overview. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 273–285). Sage.

Turing, A. M. (1937). On computable numbers, with an application to the Entscheidungs problem. Proceedings of the London Mathematical Society, 2 (1), 230–265.

Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59 , 443–460.

MathSciNet   Google Scholar  

Ullah, H., & Hafiz, M. A. (2022). Exploring effective classroom management strategies in secondary schools of Punjab. Journal of the Research Society of Pakistan, 59 (1), 76.

Verdú, E., Regueras, L. M., Gal, E., et al. (2017). Integration of an intelligent tutoring system in a course of computer network design. Educational Technology Research and Development, 65 , 653–677. https://doi.org/10.1007/s11423-016-9503-0

Vygotsky, L. S. (1978). Mind and society: The development of higher psychological processes . Harvard University Press.

Winkler-Schwartz, A., Bissonnette, V., Mirchi, N., Ponnudurai, N., Yilmaz, R., Ledwos, N., Siyar, S., Azarnoush, H., Karlik, B., & Del Maestro, R. F. (2019). Artificial intelligence in medical education: Best practices using machine learning to assess surgical expertise in virtual reality simulation. Journal of Surgical Education, 76 (6), 1681–1690. https://doi.org/10.1016/j.jsurg.2019.05.015

Yang, A. C. M., Chen, I. Y. L., Flanagan, B., & Ogata, H. (2021). Automatic generation of cloze items for repeated testing to improve reading comprehension. Educational Technology & Society, 24 (3), 147–158.

Yao, X. (2022). Design and research of artificial intelligence in multimedia intelligent question answering system and self-test system. Advances in Multimedia . https://doi.org/10.1155/2022/2156111

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education—Where are the educators? International Journal of Educational Technology in Higher Education, 16 (1), 1–27. https://doi.org/10.1186/s41239-019-0171-0

Zhang, F. (2022). Design and application of artificial intelligence technology-driven education and teaching system in universities. Computational and Mathematical Methods in Medicine . https://doi.org/10.1155/2022/8503239

Zhang, Z., & Xu, L. (2022). Student engagement with automated feedback on academic writing: A study on Uyghur ethnic minority students in China. Journal of Multilingual and Multicultural Development . https://doi.org/10.1080/01434632.2022.2102175

Download references

Acknowledgements

The authors would like to thank Mildred Jones, Katherina Nako, Yaser Sendi, and Ricardo Randall for data gathering and organization.

Author information

Authors and affiliations.

Department of Teaching and Learning, Old Dominion University, Norfolk, USA

Helen Crompton

ODUGlobal, Norfolk, USA

Diane Burke

RIDIL, ODUGlobal, Norfolk, USA

You can also search for this author in PubMed   Google Scholar

Contributions

HC: Conceptualization; Data curation; Project administration; Formal analysis; Methodology; Project administration; original draft; and review & editing. DB: Conceptualization; Data curation; Project administration; Formal analysis; Methodology; Project administration; original draft; and review & editing. Both authors read and approved this manuscript.

Corresponding author

Correspondence to Helen Crompton .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Crompton, H., Burke, D. Artificial intelligence in higher education: the state of the field. Int J Educ Technol High Educ 20 , 22 (2023). https://doi.org/10.1186/s41239-023-00392-8

Download citation

Received : 30 January 2023

Accepted : 23 March 2023

Published : 24 April 2023

DOI : https://doi.org/10.1186/s41239-023-00392-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial Intelligence
• Systematic review
• Higher education

Educating in a World of Artificial Intelligence

• Posted February 9, 2023
• By Jill Anderson
• Learning Design and Instruction
• Teachers and Teaching
• Technology and Media

Senior Researcher Chris Dede isn't overly worried about growing concerns over generative artificial intelligence, like ChatGPT, in education. As a longtime researcher on emerging technologies, he's seen many decades where new technologies promised to upend the field. Instead, Dede says artificial intelligence requires educators to get smarter about how they teach in order to truly take advantage of what AI has to offer.“The trick about AI is that to get it, we need to change what we're educating people for because if you educate people for what AI does well, you're just preparing them to lose to AI. But if you educate them for what AI can't do, then you've got IA [Intelligence Augmentation],” he says. Dede, the associate director of research for the National AI Institute for Adult Learning and Online Education , says AI raises the bar and it has the power to significantly impact learning in powerful ways.

In this episode of the Harvard EdCast, Dede talks about how the field of education needs to evolve and get smarter, in order to work with — not against — artificial intelligence. 

ADDITIONAL RESOURCES

• Dede's keynote on Intelligence Augmentation , delivered at an AI and Education conference
• Brief on Intelligence Augmentation, co-authored by Dede for HGSE’s Next Level Lab

Jill Anderson:  I'm Jill Anderson. This is the Harvard EdCast. 

Chris Dede thinks we need to get smarter about using artificial intelligence and education. He has spent decades exploring emerging learning technologies as a Harvard researcher. The recent explosion of generative AI, like ChatGPT, has been met with mixed reactions in education. Some public school districts have banned it. Some colleges and universities have tweaked their teaching and learning already. 

Generative AI raises que

Chris Dede: I've actually been working with AI for more than half a century. Way back when when I was a graduate student, I read the first article on AI in education, which was published in 1970. And the author confidently predicted that we wouldn't need teachers within five or six years because AI was going to do everything. And of course, we still see predictions like that today. 

But having lived through nine hype cycles for AI, I'm both impressed by how much it's advanced, but I'm also wary about elaborate claims for it. And there is a lot of excitement now about generative AI is the term that people are using, which includes programs like ChatGPT. It includes things like Dolly that are capable of creating images. It includes really AI on its own doing performances that we previously would have thought were something that people would have to do. 

But it's interesting to compare ChatGPT to a search engine. And people don't remember this, but there was a time when-- before search engines when people really struggled to find resources, and there was enormous excitement when search engines came out. And search engines are, in fact, AI. They are based on AI at the back end, coming up with lists of things that hopefully match what you typed in. In fact, the problem with the search engine becomes not trying to find anything, but trying to filter everything to decide what's really useful. 

So you can think of ChatGPT as the next step beyond a search engine where instead of getting a list of things and then you decide which might be useful and you examine them, you get an answer that says, this is what I think you want. And that is really more the AI taking charge than it is the AI saying, I can help you. Here's some things that you might look at and decide about. That makes me wary because AI is not at a stage where it really understands what it's saying. 

And so it will make up things when it doesn't know them, kind of a not very good student seeing if they can fake out the teacher. And it will provide answers that are not customized to somebody's culture or to somebody's reading level or to somebody's other characteristics. So it's really quite limited. 

I know that Harvard has sent some wording out that I've now put into my syllabi about students being welcome to use whatever tools they want. But when they present something as their work, it has to be something that they wrote themselves. It can't be something that somebody else wrote, which is classic plagiarism. It can't be something that Chat AI wrote that they're presenting as their work and so on. I think that what Chat AI does is it raises the bar for human performance. 

I know a lot about what people are going through now in terms of job interviews because my older daughter is an HR manager, and my younger daughter just graduated. And she's having a lot of job interviews. And in contrast to earlier times, now, job interviews typically involve a performance. 

If you're going to be hired for a marketing position, they'll say bring in a marketing plan when we do our face-to-face interview on this, and we'll evaluate it. Or in her case, in mechanical engineering, they say when you come in, there's this system that you're going to have a chance to debug, and we'll see how well you do it. Those employers are going to type the same thing into Chat AI. And if someone comes in with something that isn't any better than Chat AI, they're not going to get hired because why hire somebody that can't outcompete a free resource? 

Jill Anderson:  Oh interesting. 

Chris Dede: So it raises the bar for human performance in an interesting way. 

Jill Anderson:  Your research looks at something called intelligence augmentation. I want to know what that means and how that's different from artificial intelligence. 

Chris Dede: Intelligence augmentation is really about the opposite of this sort of negative example I was describing where now you've got to outthink Chat AI if you want to get a job. It says, when is the whole more than the sum of the parts? When do a person and AI working together do things that neither one could do as well on their own? 

And often, people think, well, yeah, I can see a computer programmer, there might be intelligence augmentation because I know that machines can start to do programming. What they don't realize is that it applies to a wide range of jobs, including mine, as a college professor. So I am the associate director for research in a national AI institute funded by the National Science Foundation on adult learning and online education. And one of the things the Institute is building is AI assistants for college faculty. 

So there's question answering assistants to help with student questions, and there's tutoring assistants and library assistants and laboratory assistants. There's even a social assistant that can help students in a large class meet other students who might be good learning partners. So now, as a professor, I'm potentially surrounded by all these assistants who are doing parts of my job, and I can be deskilled by that, which is a bad future. You sort of end up working for the assistant where they say, well, here's a question I can't answer. 

So you have to do it. Or you can upskill because the assistant is taking over routine parts of the job. And in turn, you can focus much more deeply on personalization to individual students, on bringing in cultural dimensions and equity dimensions that AI does not understand and cannot possibly help with. The trick about AI is that to get it, we need to change what we're educating people for because if you educate people for what AI does well, you're just preparing them to lose to AI. But if you educate them for what AI can't do, then you've got IA. 

Jill Anderson:  So that's the goal here. We have to change the way that we're educating young people, even older people at this point. I mean, everybody needs to change the way that they're learning about these things and interacting with them. 

Chris Dede: They do. And we're hampered by our system of assessment because the assessments that we use, including Harvard with the GRE and the SAT and so on, those are what AI does well. AI can score really well on psychometric tests. So we're using the wrong measure, if you will. We need to use performance assessments to measure what people can do to get into places like Harvard or higher education in general because that's emphasizing the skills that are going to be really useful for them. 

Jill Anderson:  You mentioned at the start artificial intelligence isn't really something brand new. This has been around for decades, but we're so slow to adapt and prepare and alter the way that we do things that once it reaches kind of the masses, we're already behind. 

Chris Dede:  Well, we are. And the other part of it is that we keep putting old wine in new bottles. I mean, this is — if I had to write a headline for the entire history of educational technology, it would be old wine in new bottles. But we don't understand what the new bottle really means. 

So let me give you an example of something that I think generative AI could make a big difference, be very powerful, but I'm not seeing it discussed in all the hype about generative AI. And that is evidence-based modeling for local decisions. So let's take climate change. 

One of the problems with climate change is that let's say that you're in Des Moines, Iowa, and you read about all this flooding in California. And you say to yourself, well, I'm not next to an ocean. I don't live in California. And I don't see why I should be that worried about this stuff. 

Now, no one has done a study, I assume, of flooding in Des Moines, Iowa, in 2050 based on mid-level projections about climate change. But with generative AI, we can estimate that now. 

Generative AI can reach out across topographic databases, meteorological databases, and other related databases to come up with here's the parts of Des Moines that are going to go underwater in 2050 and here's how often this is going to happen if these models are correct. That really changes the dialogue about climate change because now you're talking about wait a minute.  You mean that park I take my kids to is going to have a foot of water in it? So I think that kind of evidence-based modeling is not something that people are doing with generative AI right now, but it's perfectly feasible. And that's the new wine that we can put in the new bottle. 

Jill Anderson:  That's really a great way to use that. I mean, and you could even use that in your classroom. Something that you said a long, long time ago was that — and this is paraphrasing — the idea that we often implement new technology, and we make this mistake of focusing on students first rather than teachers.   Chris Dede:  In December, I gave a keynote at a conference called Empowering Learners for the Age of AU that has been held the last few years. And one of the things I talked about was the shift from teaching to learning. Both are important, but teaching is ultimately sort of pouring knowledge into the minds of learners. And learning is much more open ended, and it's essential for the future because every time you need to learn something new, you can't afford to go back and have another master's degree. You need to be able to do self-directed learning. 

And where AI can be helpful with this is that AI can be like an intellectual partner, even when you don't have a teacher that can help you learn in different ways. One of the things that I've been working on with a professor at the Harvard Business School is AI systems that can help you learn negotiation. 

Now, the AI can't be the person you're negotiating with. AI is not good at playing human beings — not yet and not for quite a long time, I think. But what AI can do is to create a situation where a human being can play three people at once. So here you are. You're learning how to negotiate a raise. 

You go into a virtual conference room. There's three virtual people who are three bosses. There's one simulation specialist behind all three, and you negotiate with them. And then at the end, the system gives you some advice on what you did well and not so well. 

And if you have a human mentor, that person gives you advice as well. Ronda Bandy, who was a professor in HGSE until she moved to Hunter College, she and I have published five articles on the work we did for the HGSE's Reach Every Reader Project on using this kind of digital puppeteering to help teachers practice equitable discussion leading. So again, here's something that people aren't talking about where AI on the front end can create rich evocative situations, and AI and machine learning on the back end can find really interesting patterns for improvement. 

Jill Anderson:  You know, Chris, how hard is it to get there for educators? 

Chris Dede: I think, in part, that's what these national AI institutes are about. Our institute, which is really adult learning with a workplace focus, is looking at that part of the spectrum. There's another institute whose focus is middle school and high school and developing AI partners for students where the student and the partner are learning together in a different kind of IA. There's a third Institute that's looking at narrative and storytelling as a powerful form of education and how can AI help with narrative and storytelling. 

You can imagine sitting down. Mom and dad aren't around. You've got a storybook like Goldilocks and the Three Bears, and you've got something like Alexa that can listen to what you're reading and respond. 

And so you begin, and you say, Goldilocks went out of her house one day and went into the woods and got lost. And Alexa says, why do you think Goldilocks went into the woods? Was she a naughty girl? No. Or was she an adventurous girl, or was she deeply concerned about climate change and wanting to study ecosystems? 

I mean, I'm being playful about this, but I think the point is that AI doesn't understand any of the questions that it's asking but it can ask the questions, and then the child can start to think deeper than just regurgitating the story. So there's all sorts of possibilities here that we just have to think of as new wine instead of asking how can AI automate our order thinking about teaching and learning. 

Jill Anderson:  I've been hearing a lot of concern about writing in particular-- writing papers where young people are actually expressing their own ideas, concerns about plagiarism and cheating, which I would say the latter have long existed as challenges in education, aren't really a new one. Does AI really change this? And how might a higher ed or any educator really look at this differently? 

Chris Dede:  So I think where AI changes this is it helps us understand the kind of writing that we should be teaching versus the kind of writing that we are teaching. So I remember preparing my children for the SAT, and it used to have something called the essay section. And you had to write this very formal essay that was a certain number of paragraphs, and the topic sentences each had to do this and so on. 

Nobody in the world writes those kinds of essays in the real world. They're just like an academic exercise. And of course, AI now can do that beautifully. 

But any reporter will tell you that they could never use Chat AI to write their stories because stories is what they write. They write narratives. If you just put in a description, you'll be fired from your reportorial job because no one is interested in descriptions. They want a story. 

So giving students a description and teaching them to turn it into a story or teaching them to turn it into something else that has a human and creative dimension for it, how would you write this for a seventh-grader that doesn't have much experience with the world? How would you write this for somebody in Russia building on the foundation of what AI gives you and taking it in ways that only people can? That's where writing should be going. 

And of course, good writing teachers will tell you, well, that's nothing new. I've been teaching my students how to write descriptive essays. The people who are most qualified to talk about the limits of AI are the ones who teach what the AI is supposedly doing. 

Jill Anderson:  So do you have any helpful tips for educators regardless of what level they're working at on where to kind of begin embracing this technology? 

Chris Dede: What AI can do well is what's called reckoning, which is calculative prediction. And I've given some examples of that with flooding in Des Moines and other kinds of things. And what people do is practical wisdom, if you will, and it involves culture and ethics and what it's like to be embodied and to have the biological things that are part of human nature and so on. 

So when I look at what I'm teaching, I have to ask myself, how much of what I'm teaching is reckoning? So I'm preparing people to lose to AI. And how much of what I'm teaching is practical wisdom? 

So for example, we spend a lot of time in vocational technical education and standard academic education teaching people to factor. How do you factor these complex polynomials? 

There is no workplace anywhere in the world, even in the most primitive possible conditions, where anybody makes a living by factoring. It's an app. It's an app on a phone. Should you know a little bit about factoring so it's not magic? Sure. 

Should you become fluent in factoring? Absolutely not. It's on the wrong side of the equation.  So I think just teachers and curriculum developers and assessors and stakeholders in the outcomes of education need to ask themselves, what is being taught now, and which parts of it are shifting over? And how do we include enough about those parts that AI isn't magic? But how do we change the balance of our focus to be more on the practical wisdom side? 

Jill Anderson:  So final thoughts here — don't be scared but figure out how to use this to your advantage? 

Chris Dede: Yeah, don't be scared. AI is not smart. It really isn't. People would be appalled if they knew how little AI understands what it's telling you, especially given how much people seem to be relying on it. But it is capable of taking over parts of what you do that are routine and predictable and, in turn, freeing up the creative and the innovative and the human parts that are really the rewarding part of both work the life. 

EdCast: Chris Dede is a senior research fellow at the Harvard Graduate School of Education. He is also a co-principal investigator of the National Artificial Intelligence Institute in adult learning and online education. I'm Jill Anderson. This is the Harvard EdCast produced by the Harvard Graduate School of Education. Thanks for listening.  [MUSIC PLAYING] 

Subscribe to the Harvard EdCast.

An education podcast that keeps the focus simple: what makes a difference for learners, educators, parents, and communities

Related Articles

Learning in Digital Worlds

Get on Board with AI

Anant Agarwal discusses how and why educators need to embrace AI

Sal Khan on Innovations in the Classroom

• My Favorites

You have successfully logged in but...

... your login credentials do not authorize you to access this content in the selected format. Access to this content in this format requires a current subscription or a prior purchase. Please select the WEB or READ option instead (if available). Or consider purchasing the publication.

• Disclaimers

Introduction

Smart education technologies: definitions and context, the uses of artificial intelligence in classrooms and educational systems, future potentials, 2. artificial intelligence in education: bringing it all together.

Artificial intelligence has led to a generation of technologies in education – for use in classrooms and by school systems more broadly – with considerable potential to bring education forward. This chapter provides a broad overview of the technologies currently being used, their core applications, and their potential going forward. The chapter also provides definitions of some of the key terms that will be used throughout this book. It concludes with a discussion of the potentials that may be achieved if these technologies are integrated, the shifts in thinking about supporting learners through one-on-one learning experiences to influencing systems more broadly, and other key directions for R&D and policy in the future.

For decades, educators and researchers have looked to computers as having the potential to revolutionise education. Today, much of the use of computers in education still falls short of revolutionary – a lot of learning still involves one instructor teaching many students simultaneously, and considerable computer-based learning takes place using curricula and technologies that replicate traditional practices such as drill and practice. However, the best practices of computers in education appear to go considerably beyond that. Millions of learners now use intelligent tutoring systems as part of their mathematics classes – systems that recognise student knowledge, implement mastery learning where students do not advance until they can demonstrate understanding of a topic, and have hints available on demand (VanLehn, 2011[1]) . Millions of learners around the world watch lectures and complete exercises within massive online open courses, offering the potential to study thousands of topics that would not be available in colleges locally (Milligan and Littlejohn, 2017[2]) . An increasing number of children and adults learn from (and are even assessed within) advanced online interactions such as simulations, games, virtual reality, and augmented reality (De Jong and Van Joolingen, 1998[3] ; Rodrigo et al., 2015[4] ; Shin, 2017[5] ; De Freitas, 2018[6]) . Perhaps none of these systems fully captures the sophistication dreamed of in early accounts of the potential of these systems (Carbonell, 1970[7]) (Stephenson, 1998[8]) . On the other hand, their scale and degree integration into formal educational systems have gone beyond what seemed plausible even as recently as the turn of the century.

Increasingly, computer-based education has been artificially intelligent education. Advances in artificial intelligence (AI) in the 1980s, 1990s, and first decade of the new millennium have translated to new potentials for learning technologies in several areas. The core advances in AI in those decades led to advances in more specialised use of AI in education – the research and practice communities of learning analytics and educational data mining – from around 2004 to today. As the research field advanced, new methods filtered into systems used by learners at scale. AI is today used to recognise what students know (and their engagement and learning strategies) to predict their future trajectories, better assess learners along multiple dimensions, and – ultimately – help both humans and computers decide how to better support students.

As these technologies develop, as they mature, as they scale, it becomes worth asking the question: where are we going? And where could we be going? If we can understand the frontiers and potentials of artificial intelligence in education, we may be able to shape research and development through careful design of policy over the next decade to get there.

In the chapters of the book, the authors use their expertise in specific areas and challenges in artificial intelligence in education to explore the frontiers and potential of this area. What technology and teaching approaches are just becoming available in research classrooms that could soon be available to a much broader range of students? How can artificial intelligence shape educational systems more broadly, from academic advising to credentialing, to make them more adaptive to learner needs? Where could we be in 10 years with the right guidance and support for research and development? Where are the opportunities for incremental but positive impacts on learners? And where are the opportunities for radically transforming education and learner experiences?

In the remainder of this overview, I will clarify some terms and domains relevant to this book. Next, I will situate the chapters of this book in the context of broader trends and opportunities in the field (including some trends and opportunities that were not explicitly covered by the authors). The final section will discuss upcoming opportunities of a broader nature, cutting across types of artificially intelligent learning technologies that can be supported through shifts in policy.

In this section, some definitions and context that are key to understanding smart technologies in education are presented.

Educational technology

Educational technology at its most obvious level refers simply to the use of any technology – any applied machinery or equipment – in education. Throughout the last hundred years, practitioners and researchers have sometimes become overly enthusiastic about finding applications for new technologies in education. See, for instance, reports by Cuban (1986[9]) of instructors teaching students with traditional lecture pedagogies but inside an early-generation airplane.

Today, most discussion of technology in education is focused on computers and digitalisation, though older technologies such as radio and television still play an important role – especially in many middle-income countries during the recent COVID-19 pandemic (OECD, n.d.[10]) . Educational technologies can refer to a range of technologies. I provide a few examples here (others are given in the context of the chapters in this book).

Computer tutors or intelligent tutoring systems provide students with a learning experience where the learning system adapts presentation based on some model or ongoing assessment of the student, a model of the subject area being learned, and a model of how to teach (Wenger, 1987[11]) . Each of these models can be more sophisticated or more basic. Baker (2016[12]) notes that contemporary intelligent tutoring systems tend to be sophisticated in only one area (which differs between systems) and very simple in other areas.

Digital learning games embed learning into a fun activity that resembles a game. The degree of gamification can vary from activities that embed learning into core gameplay and which may not even seem to be a learning activity (see, for instance, SimCity and Civilisation) to more obvious learning activities where the student gets rewards for successful performance (for instance, getting to throw a banana at a monkey after answering a math problem correctly in MathBlaster ). 

Simulations are computerised imitations of a process or activity that would be difficult or costly to do in the real world as an educational activity. Increasing numbers of students today use virtual laboratories to conduct experiments that could be dangerous, expensive, or difficult, and also to receive feedback and learning support while completing these activities.

Virtual reality systems embed learners in 3D depictions of real-world activities. Like simulations, they make it feasible to engage in activities from a home or computer lab that would be expensive, dangerous, or simply impossible to engage in otherwise. Augmented reality systems embed additional information and experiences into real-world activities, ranging from pop-up details that appear and ambient displays (information that is available in the environment without having to focus on it) to overlaying a different world on top of the current one. Both augmented reality and virtual reality often rely upon headsets to present visual information to learners.

Educational robots have a physical presence and interact with students in real-world activities to support their learning. While robots as educational DIY kits have been available since the 1980s, a recent development sees robots take up the role of tutor.

Massive online open courses (MOOCs) provide students with a basic learning experience, typically consisting of videos and quizzes. The innovation around MOOCs is not in the learning experience – it is typically a simplified version of a large lecture class – but, rather, in making materials developed by faculty at world-famous universities, often on highly specialised topics, accessible to learners around the world.

Educational data

Data are, quite simply, facts gathered together. Whereas a few facts gathered together do not enable us to reason about the relationships represented in that information, the accumulation of large quantities of information does and that is the modern power of big data. Educational data used to be dispersed, hard to collect, and small-scale. Individual teachers might keep a gradebook on paper; the school might keep disciplinary records in the basement; and curriculum developers would have a very limited idea of how their materials were being used and what students were struggling with. Today, educational data is gathered at a much larger scale. Gradebooks, disciplinary data, assessment data, absence data and more is stored centrally by local education agencies (or often by national or even trans-national vendors). Curriculum developers often gather extensive data on usage and learning. As of this writing, the regulations around handling, storage, and use of educational data vary considerably between countries, with some countries having very strict practices (particularly on the European continent), and other countries having less restrictive regulations. Each of these sources of data can be used to improve educational quality and support learning, supporting both artificial intelligence/machine learning (next definition) and human refinement of learning content and experiences.

Artificial intelligence and machine learning

Artificial intelligence is the capacity for computers to perform tasks traditionally thought to involve human intelligence or, more recently, tasks beyond the ability of human intelligence. Stemming from relatively simple, general-purpose systems in the 1960s, artificial intelligence today generally involves more specific-purpose systems that complete a specific task involving reasoning about data or the world, and then interaction with the world (more commonly through a phone or a computer interface than actual physical interaction). Machine learning (increasingly called data science, and also called both data mining and analytics) is a sub-area of artificial intelligence, present at a low level since the beginning of the field but becoming a particular emphasis in the 1990s through to today. Machine learning is when a system discovers patterns from data – becoming more effective at doing so when more data is available (and even more so, when more comprehensive or representative data is available). There is a broad range of machine-learning methods, classified mostly into supervised learning (attempting to predict or infer a specific known variable) and unsupervised learning (trying to discover the structure or relationships in a set of variables). There have roughly been two generations of machine learning: a first generation of relatively simple, interpretable methods and a second generation of much more complex, sophisticated, hard-to-interpret methods.

Artificial Intelligence in Education (AIED)

Artificial Intelligence in Education (AIED) arose as an interdisciplinary subfield in the early 1980s with a bi-annual (now annual) conference and peer-reviewed journal, although examples of this research area were present even before that. Much of the early work in artificial intelligence in education involved intelligent tutoring systems but the field has broadened over the years to include all of the types of educational systems/interactions defined above, and has expanded to include several independent conferences and journals. The revolution in machine learning and data mining impacted artificial intelligence in education as well, with a significant shift around 2010 – influenced by the emergence of a separate scientific conference, Educational Data Mining – towards much more use of this type of method. Today, AIED systems incorporate a range of functionality for identifying aspects of the learner, and a range of ways they can interact with and respond to learners.

Learning analytics

Learning Analytics, also referred to as Educational Data Mining, has emerged as a field since 2008 with two major international conferences and peer-reviewed journals. The goal of learning analytics is to use the increasing amounts of data coming from education to better understand and make inferences on learners and the contexts which they learn from. Learning analytics and educational data mining apply the methods of machine learning/data science to education, with methods and problems emerging specific to education. Challenges such as inferring student knowledge in real-time and predicting future school dropout have seen particular interest, but there have been a range of other applications for these methods, from inferring prerequisite relationships in a domain such as mathematics to understanding the factors that lead to student boredom. A taxonomy of methods and applications for learning analytics is given in (Baker and Siemens, 2014[13] ; DeFalco et al., 2017[14]) . Learning analytics models are most frequently deployed in two types of technology: intelligence augmentation systems and personalised learning systems (discussed in the next section).

Intelligence augmentation systems , also called decision support systems, communicate information to stakeholders such as teachers and stakeholders in a way that supports decision-making. While they can simply provide raw data, they often provide information distilled through machine-learning models, predictions, or recommendations. Intelligence augmentation systems often leverage predictive analytics systems, which make predictions about students’ potential future outcomes, and – ideally – also provide understandable reasons for these predictions. Predictive analytics systems are now used at scale to try to understand which students are at risk of dropping out of high school or failing to complete college, with an eye towards providing interventions which get students back on track. Intelligence augmentation systems often communicate information to stakeholders through dashboards , which communicate data through graphs and tables that allow the user to drill down for information about specific learners. Today, personalised learning systems and predictive analytics systems often use dashboards to communicate information to teachers, occasionally make dashboards available for school counselors, academic advisors, and school leaders, and rarely make dashboards available for parents. The quality of the data presented in dashboards can vary considerably from learning system to learning system.

This book focuses on two key areas: 1) New Educational Technologies and Approaches for the Classroom, and 2) New Educational Technologies and Approaches for Educational Systems. These new technologies often but not always involve artificial intelligence. Within this section, I will summarise work in each of these areas, including both the work discussed in the chapters in this report, but going beyond as well.

New educational technologies and approaches for the classroom

As computerised educational technologies become more commonly accessible to teachers and students, there is increasing awareness that the technology does not simply increase convenience for teachers or provide a fun alternative activity for students – it can promote new methods for teaching and learning. 

Personalised Learning. One major trend within learning, driven by these technologies, is the move towards personalising learning to a greater degree. Personalisation of learning did not start with computerised technology – in a sense, it has been available since the first use of one-on-one tutoring, thousands of years ago (if not earlier). However, with the increase in systematised, standardised schooling and teaching over a hundred years ago, awareness increased that many students’ learning needs were being poorly met by one-size-fits-all curriculum. Classroom approaches such as mastery learning (each student works on material until mastery and only then moves on to the next topic) were developed, but proved difficult to scale due to the demands on the teacher. Educational technologies provided a ready solution to this problem – the computer could manage some of the demands of personalising learning, identifying each individual student’s degree of mastery and providing them with learning activities relevant to their current position within the curriculum.

The first dimension that educational technologies became effective at personalising for was a student’s knowledge or state of learning. Molenaar (2021[15]) details efforts to develop better personalisation of learning for learners, providing a framework for the degree of automation in personalised learning systems. Her chapter discusses the shift from teacher-driven systems to computer-based technologies that can take a larger role in immediate decision-making, remaining within guidelines and goals specified by the teacher.

Next, educational technologies became more effective at personalising for differences in students’ self-regulated learning – their ability to make good choices during learning that enhance their learning outcomes and efficiency. This topic is also discussed in Inge Molenaar’s chapter (Molenaar, 2021[15]) . Modern educational technologies in many cases have the ability to recognise when students are using ineffective or inefficient strategies, and to provide them recommendations or nudges to get back onto a more effective trajectory.

A contemporary trend, which is still primarily in research classrooms rather than wide-scale deployment, is the move towards also recognising and adapting to student engagement, affect, and emotion. Discussed by Sidney D’Mello (2021[16]) , these systems recognise these aspects of a student’s experience either from their interaction and behaviour within the system or from physical and physiological sensors. There are now several examples of educational technologies – particularly intelligent tutoring systems and games – which have been able to identify a student who is bored, frustrated, or gaming the system (trying to find strategies to complete materials without needing to learn) and re-engage them productively e.g. (DeFalco et al., 2017[14]) .

Increasing research also looks at trying to personalise to increase broader motivation or interest. This work differs from the work on engagement and affect in terms of time-scale. Whereas engagement and affect often manifests in brief time periods – as short as a few seconds – motivation and interest are more long-term stable aspects of student experience. Work by Kizilcec and colleagues (Kizilcec et al., 2017[17]) , for instance, has tried to connect student learning experiences with their values, leading to greater degrees of completion of online courses. Work by Walkington and colleagues (Walkington, 2013[18] ; Walkington and Bernacki, 2019[19]) has modified the contents of learning systems to match student personal interests, leading students to work faster, become disengaged less often, and learn more.

New Pedagogies. Although the most obvious impact of artificially-intelligent educational technologies is through personalising learning directly, new pedagogies and teacher practices have also emerged. These pedagogies and practices enable teachers to support their students or provide their students with experiences in ways that were generally not feasible prior to the technology being developed.

Perhaps the largest shift has been in the information available to teachers. Dashboards provide teachers with data on a range of aspects of their students’ performance and learning. This has produced a major shift in how homework is used. In the past, homework would need to be brought to class by students. It could be graded by the teacher after that (meaning that feedback and learning support would be delayed), or students could grade it with the teacher in a large group, which is not a very time-efficient approach. In contrast, data from homework technologies today can become available to teachers in real-time. This means that teachers can identify which students are struggling and which materials students struggled on in general before class even starts. This enables strategies where, for instance, teachers identify which students displayed common errors and can identify students who can demonstrate both incorrect and correct problem-solving strategies for whole-class discussion. It also enables teachers to message students who are behind in completing materials (or even in starting to work through materials), helping get the student back on track (Arnold and Pistilli, 2012[20]) .

Similar uses are available for formative assessment systems, which are being increasingly used in contexts where students have high-stakes end-of-year examinations. These systems often go beyond teacher-designed homework in terms of their breadth and comprehensiveness of coverage of key skills and concepts. They are increasingly used by teachers to determine what topics to review with their classes as well as what types of supplemental supports to provide to specific students.

Formative assessment systems are increasingly used in K-12 education worldwide. The most widely used formative assessment systems, such as NWEA MAP (Finnerty, 2018[21]) , present students with traditional multiple-choice items and measure straightforward mathematics and language arts competencies – essentially providing another test to students to complete, but one where their teachers will get useful data linked to competencies that will be seen on future standardised examination. A small number of emerging formative assessment systems assess more complicated constructs and/or embed assessment into more complex activities, such as games (Shute and Kim, 2014[22]) .

Data from formative assessment systems can be used with platforms designed to provide lists of supplemental resources for specific skills, concepts, and topics. Especially post-COVID, both local education agencies, and regional and national governments have worked to develop platforms with supplemental learning resources for students and parents. However, right now, these platforms are generally not connected directly to formative assessment systems so the teacher or parent needs to look up the resources for a student struggling with a specific competency.

One concern about formative assessment systems is that time spent using a formative assessment system is time not spent learning – a loss of instructional time. For this reason, there has been a trend towards embedding formative assessment into personalised learning. Several widely-used personalised learning systems, such as MATHia, Mindspark, Reasoning Mind, and ASSISTments, provide teachers with formative assessment data on which competencies the student is missing (Feng and Heffernan, 2006[23] ; Khachatryan et al., 2014[24]) . This information is distilled from students’ regular learning activity, avoiding a loss of instructional time.

There is also better information available to teachers on what is going on in their classes in real-time, an area discussed in detail by Pierre Dillenbourg (Dillenbourg, 2021[25]) . Classroom analytics can provide the teacher with information on a range of aspects of class performance, from individual students’ difficulties with material in real-time to the relative effectiveness of collaboration by different student groups. A teacher cannot watch every student (or every student group) at all times – better data can help them understand where to focus their efforts, and which students would benefit from a conversation right now.

Beyond just providing better data, it is possible to use technology to give students a range of experiences that were not feasible a generation ago. In their chapter, Tony Belpaeme and Fumihide Tanaka (Belpaeme and Tanaka, 2021[26]) discuss the new potentials of having robots interact with students in classrooms.

Using simulations and games in class can enable teachers to demonstrate complex and hard to understand systems to students. They can also allow students to explore and interact with these systems on their own. There seems to be particular educational benefit to the combination of a rich simulation or game experience that enables a student to develop informal, practical understanding, and then a teacher lecture or explanation that helps a student bridge from that informal understanding to more formal, academic conceptual understanding (Asbell-Clarke et al., 2020[27]) . Modern technologies also offer new potentials for the use of collaborative learning, with systems that can scaffold effective collaboration strategies (Strauß and Rummel, 2020[28]) , and systems that can provide rich experiences to collaborate around, such as interactive tabletops (Martinez Maldonado et al., 2012[29]) . 

Equity. New educational technologies are typically designed with the goal of improving student and teacher experiences and outcomes. However, the designers of these systems do not always consider how the full spectrum of learners are impacted. Often, systems are designed by members of specific demographic groups (typically higher socio-economic status, not identified as having special needs, and members of racial/ethnic/national majority groups) with members of their own groups in mind (not always intentionally). This can lead to lower educational effectiveness for members of other groups.

For example, Judith Good (2021[30]) discusses how there has been little effort to create educational technologies specifically designed for students with disabilities or special needs. She discusses examples of technologies that could support learners with autism, dysgraphia and visual impairment. The lack of attention to individuals with special needs by the scientific community and by developers of artificially intelligent educational technologies is a major source of inequity and a missed opportunity. Designing policies that facilitate developing systems to support learners with special needs (for instance, by developing approaches that improve access to data on disabilities while protecting student privacy) and the creation of incentives to develop for special needs populations may help to address this inequity.

Another key area of inequity is in support of historically underserved and underrepresented populations, including ethnic/racial minorities and linguistic minorities. Most educational technologies are developed by members of historically well-supported populations. They are often first piloted with members of historically well-supported populations. Testing for effectiveness with historically underrepresented populations often occurs only in later stages of development (or in final large-scale evaluations of efficacy) when it is too late to make major design changes. There is increasing evidence that both educational research design findings and algorithms obtained on majority populations can fail to apply or function more poorly for other populations of learners (Ocumpaugh et al., 2014[31] ; Karumbaiah, Ocumpaugh and Baker, 2019[32]) .

REPORT on artificial intelligence in education, culture and the audiovisual sector

19.4.2021 - ( 2020/2017(INI) )

Committee on Culture and Education Rapporteur: Sabine Verheyen Rapporteur for the opinion (*): Ondřej Kovařík, Committee on Civil Liberties, Justice and Home Affairs (*) Associated committee – Rule 57 of the Rules of Procedure

MOTION FOR A EUROPEAN PARLIAMENT RESOLUTION

Explanatory statement.

• OPINION OF THE COMMITTEE ON CIVIL LIBERTIES, JUSTICE AND HOME AFFAIRS
• OPINION OF THE COMMITTEE ON THE INTERNAL MARKET AND CONSUMER PROTECTION
• OPINION OF THE COMMITTEE ON LEGAL AFFAIRS
• OPINION OF THE COMMITTEE ON WOMEN'S RIGHTS AND GENDER EQUALITY

INFORMATION ON ADOPTION IN COMMITTEE RESPONSIBLE

Final vote by roll call in committee responsible.

on artificial intelligence in education, culture and the audiovisual sector

( 2020/2017(INI) )

The European Parliament ,

–   having regard to the Charter of Fundamental Rights of the European Union,

–   having regard to Articles 165, 166 and 167 of the Treaty on the Functioning of the European Union,

–   having regard to the Council conclusions of 9 June 2020 on shaping Europe’s digital future [1] ,

–   having regard to the opinion of the European Economic and Social Committee of 19 September 2018 on the digital gender gap [2] ,

–   having regard to the Commission proposal for a regulation of the European Parliament and of the Council of 6 June 2018 establishing the Digital Europe Programme for the period 2021-2027 ( COM(2018)0434 ),

–   having regard to the Commission communication of 30 September 2020 on the Digital Education Action Plan 2021-2027: Resetting education and training for the digital age ( COM(2020)0624 ),

–   having regard to the Commission communication of 30 September 2020 on achieving the European Education Area by 2025 ( COM(2020)0625 ),

–   having regard to the Commission report of 19 February 2020 on the safety and liability implications of artificial intelligence, the Internet of Things and robotics ( COM(2020)0064 ),

–   having regard to the Commission white paper of 19 February 2020 entitled ‘Artificial Intelligence – A European approach to excellence and trust’ ( COM(2020)0065 ),

–   having regard to the Commission communication of 19 February 2020 on a European strategy for data ( COM(2020)0066 ),

–   having regard to the Commission communication of 25 April 2018 entitled ‘Artificial Intelligence for Europe’ ( COM(2018)0237 ),

–   having regard to the Commission communication of 17 January 2018 on the Digital Education Action Plan ( COM(2018)0022 ),

–   having regard to the report of the Commission High-Level Expert Group on Artificial Intelligence of 8 April 2019 entitled ‘Ethics Guidelines for Trustworthy AI’,

–   having regard to its resolution of 12 February 2019 on a comprehensive European industrial policy on artificial intelligence and robotics [3] ,

–   having regard to its resolution of 11 September 2018 on language equality in the digital age [4] ,

–   having regard to its resolution of 12 June 2018 on modernisation of education in the EU [5] ,

–   having regard to its resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics [6] ,

–   having regard to its resolution of 1 June 2017 on digitising European industry [7] ,

–   having regard to the briefing of its Policy Department for Structural and Cohesion Policies of May 2020 on the use of artificial intelligence in the cultural and creative sectors,

–   having regard to the in-depth analysis of its Policy Department for Structural and Cohesion Policies of May 2020 on the use of artificial intelligence in the audiovisual sector,

–   having regard to the study of its Policy Department for Citizens’ Rights and Constitutional Affairs of April 2020 on the education and employment of women in science, technology and the digital economy, including AI and its influence on gender equality,

–   having regard to Rule 54 of its Rules of Procedure,

–   having regard to the opinions of the Committee on Civil Liberties, Justice and Home Affairs, the Committee on the Internal Market and Consumer Protection, the Committee on Legal Affairs and the Committee on Women’s Rights and Gender Equality,

–   having regard to the report of the Committee on Culture and Education (A9-0127/2021),

A.   whereas artificial intelligence (AI) technologies, which may have a direct impact on our societies, are being developed at a fast pace and are increasingly being used in almost all areas of our lives, including education, culture and the audiovisual sector; whereas ethical AI is likely to help improve labour productivity and help accelerate economic growth;

B.   whereas the development, deployment and use of AI, including the software, algorithms and data used and produced by it, should be guided by the ethical principles of transparency, explainability, fairness, accountability and responsibility;

C.   whereas public investment in AI in the Union has been vastly lagging behind other major economies; whereas underinvestment in AI will be likely to have an impact on the Union’s competitiveness across all sectors;

D.   whereas an integrated approach to AI and the availability, collection and interpretation of high-quality, trustworthy, fair, transparent, reliable, secure and compatible data are essential for the development of ethical AI;

E.   whereas Article 21 of the EU Charter of Fundamental Rights prohibits discrimination on a wide range of grounds; whereas multiple forms of discrimination should not be replicated in the development, deployment and use of AI systems;

F.   whereas gender equality is a core principle of the Union enshrined in the Treaties and should be reflected in all Union policies, including in education, culture and the audiovisual sector, as well as in the development of technologies such as AI;

G.   whereas past experiences, especially in technical fields, have shown that developments and innovations are often based mainly on male data and that women’s needs are not fully reflected; whereas addressing these biases requires greater vigilance, technical solutions and the development of clear requirements of fairness, accountability and transparency;

H.   whereas incomplete and inaccurate data sets, the lack of gender-disaggregated data and incorrect algorithms can distort the processing of an AI system and jeopardise the achievement of gender equality in society; whereas data on disadvantaged groups and intersectional forms of discrimination tends to be incomplete or even absent;

I.   whereas gender inequalities, stereotypes and discrimination can also be created and replicated through the language and images disseminated by the media and AI-powered applications; whereas education, cultural programmes and audiovisual content have considerable influence in shaping people’s beliefs and values and are a fundamental tool for combatting gender stereotypes, decreasing the digital gender gap and establishing strong role models; whereas an ethical and regulatory framework must be in place ahead of implementing automatised solutions for these key areas in society;

J.   whereas science and innovation can bring life-changing benefits, especially for those who are furthest behind, such as women and girls living in remote areas; whereas scientific education is important for obtaining skills, decent work and jobs of the future, as well as for breaking with gender stereotypes that regard these as stereotypically masculine fields; whereas science and scientific thinking are key to democratic culture, which in turn is fundamental for advancing gender equality;

K.   whereas one woman in ten in the Union has already suffered some form of cyber-violence since the age of 15 and whereas cyber-harassment remains a concern in the development of AI, including in education; whereas cyber-violence is often directed at women in public life, such as activists, women politicians and other public figures; whereas AI and other emerging technologies can play an important role in preventing cyber-violence against women and girls and educating people;

L.   whereas the Union and its Member States have a particular responsibility to harness, promote and enhance the added value of AI technologies and to make sure that these technologies are safe and serve the well-being and general interest of Europeans; whereas these technologies can make a huge contribution to achieving our common goal of improving the lives of our citizens and fostering prosperity in the Union by helping to develop better strategies and innovation in a number of areas, namely in education, culture and the audiovisual sector;

M.   whereas most AI is based on open-source software, which means that source codes can be inspected, modified and enhanced;

N.   whereas certain adjustments to specific existing EU legislative instruments may be necessary to reflect the digital transformation and address new challenges posed by the use of AI technologies in education, culture and the audiovisual sector, such as the protection of personal data and privacy, combatting discrimination, promoting gender equality, and respecting intellectual property rights (IPR), environmental protection and consumers’ rights;

O.   whereas it is important to provide the audiovisual sector with access to data from the global platforms and major players in order to ensure a level playing field;

P.   whereas AI and future applications or inventions made with the help of AI can have a dual nature, much like with any other technology; whereas AI and related technologies raise many concerns regarding the ethics and transparency of their development, deployment and use, notably on data collection, use and dissemination; whereas the benefits and risks of AI technologies in education, culture and the audiovisual sector must be carefully assessed and their effects on all aspects of society thoroughly and continuously analysed, without undermining their potential;

Q.   whereas education aims to achieve human potential, creativity and authentic social change, while using data-driven AI systems incorrectly may hinder human and social development;

R.   whereas education and educational opportunities are a fundamental right; whereas the development, deployment and use of AI technologies in the education sector should be classified as high risk and subject to stricter requirements on safety, transparency, fairness and accountability;

S.   whereas high-quality, fast and secure pervasive connectivity, broadband, high-capacity networks, IT expertise, digital skills, digital equipment and infrastructure, as well as societal acceptance and a targeted and accommodating policy framework, are some of the preconditions for the broad and successful deployment of AI in the Union; whereas it is essential that such infrastructure and equipment be deployed equally across the Union in order to tackle the persistent digital gap between its regions and citizens;

T.   whereas addressing the gender gap in science, technology, engineering, arts and maths (STEAM) subjects is an absolute necessity to ensure that the whole of society is equally and fairly represented when developing, deploying and using AI technologies, including the software, algorithms and data used and produced by them;

U.   whereas it is essential to ensure that all people in the Union acquire the necessary skills from an early age in order to better understand the capabilities and limitations of AI, to prepare themselves for the increasing presence of AI and related technologies in all aspects of human activity, and to be able to fully embrace the opportunities that they offer; whereas the widespread acquisition of digital skills across all parts of society in the Union is a precondition for achieving a fair digital transformation beneficial to all;

V.   whereas, with that aim in view, the Member States must invest in digital education and media training, equipping schools with the proper infrastructure and the necessary end devices, and place greater emphasis on the teaching of digital skills and capabilities as part of school curricula;

W.   whereas AI and related technologies can be used to improve learning and teaching methods, notably by helping education systems to use fair data to improve educational equity and quality, while promoting tailor-made curricula and better access to education and improving and automating certain administrative tasks; whereas equal and fair access to digital technologies and high-speed connectivity are required in order to make the use of AI beneficial to the whole of society; whereas it is of the utmost importance to ensure that digital education is accessible to all, including those from disadvantaged backgrounds and people with disabilities; whereas learning outcomes do not depend on technology per se, but on how teachers can use technology in pedagogically meaningful ways;

X.   whereas AI has particular potential to offer solutions for the day-to-day challenges of the education sector, such as the personalisation of learning, monitoring learning difficulties, the automation of subject-specific content/knowledge, providing better professional training and supporting the transition to a digital society;

Y.   whereas AI could have practical applications in terms of reducing the administrative work of educators and educational institutions, freeing up time for their core teaching and learning activities;

Z.   whereas new AI-based applications in education are facilitating progress in a variety of disciplines, such as language learning and maths;

AA.   whereas AI-enabled personalised learning experiences can not only help to increase students’ motivation and enable them to reach their full potential, but also reduce drop-out rates;

AB.   whereas AI can increasingly help make teachers more effective by giving them a better understanding of students’ learning methods and styles and helping them to identify learning difficulties and better assess individual progress;

AC.   whereas the Union’s digital labour market is lacking almost half a million experts in big data sciences and data analysis, who are intrinsic to the development and use of quality and trustworthy AI;

AD.   whereas the application of AI in education raises concerns around the ethical use of data, learners’ rights, data access and protection of personal data, and therefore entails risks to fundamental rights such as the creation of stereotyped models of learners’ profiles and behaviour that could lead to discrimination or risks of doing harm by the scaling-up of bad pedagogical practices;

AE.   whereas culture plays a central role in the use of AI technologies at scale and is emerging as a key discipline for cultural heritage thanks to the development of innovative technologies and tools and their effective application to respond to the needs of the sector;

AF.   whereas AI technologies can be used to promote and protect cultural heritage, including by using digital tools to preserve historical sites and finding innovative ways to make the datasets of cultural artefacts held by cultural institutions across the Union more widely and easily accessible, while allowing users to navigate the vast amount of cultural and creative content; whereas the promotion of interoperability standards and frameworks is key in this regard;

AG.   whereas the use of AI technologies for cultural and creative content, notably media content and tailored content recommendations, raises issues around data protection, discrimination and cultural and linguistic diversity, risks producing discriminatory output based on biased entry data, and could restrict diversity of opinion and media pluralism;

AH.   whereas AI-based personalised content recommendations can often better target individuals’ specific needs, including cultural and linguistic preferences; whereas AI can help to promote linguistic diversity in the Union and contribute to the wider dissemination of European audiovisual works, in particular through automatic subtitling and dubbing of audiovisual content in other languages; whereas making media content across languages is therefore fundamental to support cultural and linguistic diversity;

AI.   whereas AI drives innovation in newsrooms by automating a variety of mundane tasks, interpreting data and even generating news such as weather forecasts and sports results;

AJ.   whereas Europe’s linguistic diversity means that promoting computational linguistics for rights-based AI offers specific potential for innovations which can be used to make global cultural and information exchanges in the digital age democratic and non-discriminatory;

AK.   whereas AI technologies may have the potential to benefit special needs education, as well as the accessibility of cultural and creative content for people with disabilities; whereas AI enables solutions such as speech recognition, virtual assistants and digital representations of physical objects; whereas digital creations are already playing their part in making such content available to people with disabilities;

AL.   whereas AI applications are omnipresent in the audiovisual sector, in particular on audiovisual content platforms;

AM.   whereas AI technologies therefore contribute to the creation, planning, management, production, distribution, localisation and consumption of audiovisual media products;

AN.   whereas while AI can be used to generate fake content, such as ‘deepfakes’, which are growing exponentially and constitute an imminent threat to democracy, it can also be used as an invaluable tool for identifying and immediately combatting such malicious activity, for example through real-time fact checking or labelling of content; whereas most deepfake material is easy to spot; whereas at the same time, AI-powered detection tools are generally successful in flagging and filtering out such content; whereas there is a lack of a legal framework on this issue;

General observations

1.   Underlines the strategic importance of AI and related technologies for the Union; stresses that the approach to AI and its related technologies must be human-centred and anchored in human rights and ethics, so that AI genuinely becomes an instrument that serves people, the common good   and the general interest of citizens;

2.   Underlines that the development, deployment and use of AI in education, culture and the audiovisual sector must fully respect fundamental rights, freedoms and values, including human dignity, privacy, the protection of personal data, non-discrimination and freedom of expression and information, as well as cultural diversity and intellectual property rights, as enshrined in the Union Treaties and the Charter of Fundamental Rights;

3.   Asserts that education, culture and the audiovisual sector are sensitive areas as far as the use of AI and related technologies is concerned, as they have the potential to impact on the cornerstones of the fundamental rights and values of our society; stresses, therefore, that ethical principles should be observed in the development, deployment and use of AI and related technologies in these sectors, including the software, algorithms and data used and produced by them;

4.   Recalls that algorithms and AI should be ‘ethical by design’, with no built-in bias, in a way that guarantees maximum protection of fundamental rights;

5.   Reiterates the importance of developing quality, compatible and inclusive AI and related technologies for use in deep learning which respect and defend the values of the Union, notably gender equality, multilingualism and the conditions necessary for intercultural dialogue, as the use of low-quality, outdated, incomplete or incorrect data may lead to poor predictions and in turn discrimination and bias; highlights that it is essential to develop capabilities at both national and Union level to improve data collection, safety, systematisation and transferability, without harming privacy; takes note of the Commission’s proposal to create a single European data space;

6.   Recalls that AI may give rise to biases and thus to various forms of discrimination based on sex, race, colour, ethnic or social origin, genetic features, language, religion or belief, political or any other opinion, membership of a national minority, property, birth, disability, age or sexual orientation; recalls, in this regard, that the rights of all people must be ensured and that AI and related technologies must not be discriminatory in any form;

7.   Emphasises that such bias and discrimination can arise from already biased datasets that reflect existing discrimination in society; recalls, in this context, that it is essential to involve the relevant stakeholders, including civil society, to prevent gender, social and cultural biases from being inadvertently included in AI algorithms, systems and applications; stresses the need to work on the most efficient way of reducing bias in AI systems in line with ethical and non-discrimination standards; underlines that the datasets used to train AI should be as broad as possible in order to represent society in the best and most relevant way, that the outputs should be reviewed to avoid all forms of stereotypes, discrimination and bias and that, where appropriate, AI should be used to identify and rectify human bias wherever it exists; calls on the Commission to encourage and facilitate the sharing of de-biasing strategies for data;

8.   Calls on the Commission and the Member States to take into account ethical aspects, including from a gender perspective, when developing AI policy and legislation, and, if necessary, to adapt the current legislation, including Union programmes and ethical guidelines for the use of AI;

9.   Calls on the Commission and the Member States to devise measures that fully incorporate the gender dimension, such as awareness-raising campaigns, training and curricula, which should provide information to citizens on how algorithms operate and their impact on their daily lives; further calls on them to nurture gender-equal mindsets and working conditions that lead to the development of more inclusive technology products and work environments; urges the Commission and the Member States to ensure the inclusion of digital skills and AI training in school curricula and to make them accessible to all, as a way to close the digital gender divide;

10.   Stresses the need for training for workers and educators dealing with AI to promote the ability to identify and correct gender-discriminatory practices in the workplace and in education, and for workers developing AI systems and applications to identify and remedy gender-based discrimination in the AI systems and applications they develop; calls for the establishment of clear responsibilities in companies and educational institutions to ensure that there is no gender-based discrimination in the workplace or educational context; highlights that genderless images of AI and robots should be used for educational and cultural purposes, unless gender is a key factor for some reason;

11.   Highlights the importance of the development and deployment of AI applications in education, culture and the audiovisual sector in collecting gender-disaggregated and other equality data, and of applying modern machine learning de-biasing techniques, if needed, to correct gender stereotypes and gender biases which may have negative impacts;

12.   Calls on the Commission to include education in the regulatory framework for high-risk AI applications, given the importance of ensuring that education continues to contribute to the public good, as well as the high sensitivity of data on pupils, students and other learners; emphasises that in the education sector, this deployment should involve educators, learners and the wider society and should take into account the needs of all and the expected benefits in order to ensure that AI is used purposefully and ethically;

13.   Calls on the Commission to encourage the use of Union programmes such as Horizon Europe, Digital Europe and Erasmus+ to promote multidisciplinary research, pilot projects, experiments and the development of tools including training, for the identification of gender biases in AI, as well as awareness-raising campaigns for the general public;

14.   Stresses the need to create diverse teams of developers and engineers to work alongside the main actors in education, culture and the audiovisual sector in order to prevent gender or social bias from being inadvertently included in AI algorithms, systems and applications; stresses the need to consider the variety of different theories through which AI has been developed to date and could be further advanced in the future;

15.   Points out that taking due care to eliminate bias and discrimination against particular groups, including gender stereotypes, should not halt technological progress;

16.   Reiterates the importance of fundamental rights and the overarching supremacy of the legislation of data and privacy protection, which is imperative when dealing with such technologies; recalls that data protection and privacy can be particularly affected by AI, in particular children’s data; underlines that the principles established in the General Data Protection Regulation (GDPR) [8] are binding for the deployment of AI in that regard; recalls, moreover, that all AI applications must fully respect Union data protection law, namely the GDPR and the ePrivacy Directive [9] ; stresses the right to obtain human intervention when AI and related technologies are being used;

17.   Calls on the Commission and the Member States to implement an obligation of transparency and explainability of AI-based automated individual decisions taken within the framework of prerogatives of public power, and to implement penalties to enforce this; calls for the implementation of systems which use human verification and intervention by default, and for due process, including the right of appeal and redress as well as access to remedies;

18.   Notes the potentially negative impact of personalised advertising, in particular micro-targeted and behavioural advertising, and of the assessment of individuals, especially minors, without their consent, by interfering in the private life of individuals, asking questions as to the collection and use of the data used to personalise advertising, and offering products or services or setting prices; calls on the Commission, therefore, to introduce strict limitations on targeted advertising based on the collection of personal data, starting with a ban on cross-platform behavioural advertising, without harming small and medium-sized enterprises (SMEs); recalls that the ePrivacy Directive currently only allows targeted advertising subject to opt-in consent, otherwise making it illegal; calls on the Commission to prohibit the use of discriminatory practices for the provision of services or products;

19.   Stresses the need for media organisations to be informed about the main parameters of algorithm-based AI systems that determine ranking and search results on third-party platforms, and for users to be informed about the use of AI in decision-making services and empowered to set their privacy parameters via transparent and understandable measures;

20.   Stresses that AI can support content creation in education, culture and the audiovisual sector, alongside information and educational platforms, including listings of different kinds of cultural objects and a multitude of data sources; notes the risks of IPR infringement when blending AI and different technologies with a multiplicity of sources (documents, photos, films) to improve how that data is displayed, researched and visualised; calls for AI to be used to ensure a high level of IPR protection within the current legislative framework, such as by alerting individuals and businesses if they are in danger of inadvertently infringing the rules, or by assisting IPR rights holders if the rules are actually infringed; emphasises, therefore, the importance of having an appropriate legal framework at Union level for the protection of IPR in connection with the use of AI;

21.   Stresses the need to strike a balance between, on the one hand, the development of AI systems and their use in education, culture and the audiovisual sector and, on the other, measures to safeguard competition and market competitiveness for AI companies in these sectors; emphasises, in this regard, the need to encourage companies to invest in the innovation of AI systems used in these sectors, while also ensuring that those providing such applications do not obtain a market monopoly; underlines the need for AI to be made widely available to the cultural and creative sectors and industries (CCSI) across Europe in order to maintain a level playing field and fair competition for all stakeholders and actors in Europe; calls on the Commission and the Member States, when taking decisions on competition policy, including mergers, to take greater account of the role played by data and algorithms in the concentration of market power;

22.   Stresses the need to systematically address the social, ethical and legal issues raised by the development, deployment and use of AI such as the transparency and accountability of algorithms, non-discrimination, equal opportunities, freedom and diversity of opinion, media pluralism and the ownership, collection, use and dissemination of data and content; recommends that common European guidelines and standards to protect privacy be devised while making effective use of the data available; calls for transparency in the development and accountability in the use of algorithms;

23.   Calls on the Commission to put forward a comprehensive set of provisions designed to regulate AI applications on a horizontal basis and to supplement them with sector-specific rules, for example for audiovisual media services;

24.   Stresses the need for investment in research and innovation on the development, deployment and use of AI and its applications in education, culture and the audiovisual sector; highlights the importance of public investment in these services and the complementary added value provided by public-private partnerships in order to achieve this objective and deploy the full potential of AI in these sectors, in particular education, in view of the substantial amount of private investment made in recent years; calls on the Commission to find additional funding to promote research and innovation into AI applications in these sectors;

25.   Underlines that algorithmic systems can be an enabler for reducing the digital divide in an accelerated way, but unequal deployment risks creating new divides or accelerating the deepening of existing ones; expresses its concern that knowledge and infrastructure are not developed in a consistent way across the Union, which limits the accessibility of products and services that rely on AI, in particular in sparsely populated and socio‑economically vulnerable areas; calls on the Commission to ensure cohesion in the sharing of the benefits of AI and related technologies;

26.   Calls on the Commission to establish requirements for the procurement and deployment of AI and related technologies by Union public sector bodies in order to ensure compliance with Union law and fundamental rights; highlights the added value of using instruments such as public consultations and impact assessments prior to the procurement or deployment of AI systems, as recommended in the report of the Special Rapporteur to the UN General Assembly on AI and its impact on freedom of opinion and expression [10] ; encourages public authorities to incentivise the development and deployment of AI by public funding and public procurement; stresses the need to strengthen the market by providing SMEs with the opportunity to participate in the procurement of AI applications in order to ensure the involvement of technology companies of all sizes and thus guarantee resilience and competition;

27.   Calls for independent audits to be conducted regularly to examine whether the AI applications being used and the related checks and balances are in accordance with specified criteria, and for those audits to be supervised by independent and sufficient overseeing authorities; calls for specific stress tests to assist and enforce compliance;

28.   Notes the benefits and risks of AI in terms of cybersecurity and its potential in combatting cybercrime, and emphasises the need for any AI solutions to be resilient to cyberattacks while respecting Union fundamental rights, especially the protection of personal data and privacy; stresses the importance of monitoring the safe use of AI and the need for close collaboration between the public and private sectors to counter user vulnerabilities and the dangers arising in this connection; calls on the Commission to evaluate the need for better prevention in terms of cybersecurity and mitigation measures thereof;

29.   Highlights that the COVID-19 pandemic crisis can be considered as a probation period for the development, deployment and use of digital and AI-related technologies in education and culture, as exemplified by the many online schooling platforms and online tools for cultural promotion employed across the Member States; calls on the Commission, therefore, to take stock of those examples when considering a common Union approach to the increased use of such technological solutions;

30.   Recalls the importance of strengthening digital skills and achieving a high standard of media, digital and information literacy at Union level as a prerequisite for the use of AI in education; underlines the need, in this regard, to ensure Union-wide digital and AI literacy, in particular through the development of training opportunities for teachers; insists that the use of AI technologies in schools should help to narrow the social and regional digital gap; welcomes the Commission’s updated Digital Education Action Plan, which addresses the use of AI in education; calls on the Commission, in that regard, to make digital capabilities, media literacy and training and AI-related skills the priorities of this plan, while raising awareness about the potential misuses and malfunctioning of AI; calls on the Commission, in that connection, to place special emphasis on children and young people in precarious situations, as they need particular support in the area of digital education; urges the Commission to duly address AI and robotics initiatives in education in its forthcoming AI legislative proposals; urges the Member States to invest in digital equipment in schools, using Union funds to this end;

31.   Highlights that the use of AI in education systems brings a wide range of possibilities, opportunities and tools for making it more innovative, inclusive, efficient and increasingly effective by introducing new high-quality learning methods that are quick, personalised and student-centric; stresses, however, that as it will impact education and social inclusion, the availability of such tools must be ensured for all social groups by establishing equal access to education and learning and leaving no one behind, especially people with disabilities;

32.   Underlines that in order to engage with AI both critically and effectively, citizens need at least a basic understanding of this technology; calls on the Member States to integrate awareness-raising campaigns about AI in their actions on digital literacy; calls on the Commission and the Member States to promote digital literacy plans and forums for discussion to involve citizens, parents and students in a democratic dialogue with public authorities and stakeholders concerning the development, deployment and use of AI technologies in education systems; stresses the importance of providing educators, trainers and others with the right tools and know-how with regard to AI and related technologies in terms of what they are, how they are used and how to use them properly and in accordance with the law, in order to avoid IPR infringements; highlights, in particular, the importance of digital literacy for people working in the education and training sectors and of improving digital training for the elderly, bearing in mind that the younger generations already have a basic notion of these technologies, having grown up with them;

33.   Stresses that the real objective of AI in education systems should be to make education as individualised as possible, offering students personalised academic paths in line with their strengths and weaknesses and didactic material tailored to their characteristics, while maintaining educational quality and the integrating principle of our education systems;

34.   Recalls the fundamental and multifaceted role that teachers play in education and in making it inclusive, especially in early childhood, where skills are acquired that will enable students to progress throughout their lives, such as in personal relations, study skills, empathy and cooperative work; stresses, therefore, that AI technologies cannot be used to the detriment or at the expense of in-person education, as teachers must not be replaced by any AI or AI-related technologies;

35.   Stresses that the learning benefits of using AI in education will depend not only on AI itself, but on how teachers use AI in the digital learning environment to meet the needs of pupils, students and teachers; points out, therefore, the need for AI programmers to involve teaching communities in the development, deployment and use of AI technologies where possible, creating a nexus environment to form connections and cooperation between AI programmers, developers, companies, schools, teachers and other public and private stakeholders in order to create AI technologies that are suitable for real-life educational environments, reflect the age and developmental readiness of each learner and meet the highest ethical standards; highlights that educational institutions should only deploy trustworthy, ethical, human-centred technologies which are auditable at every stage of their lifecycle by public authorities and civil society; emphasises the advantages of free and open-source solutions in this regard; calls for schools and other educational establishments to be provided with the financial and logistical support as well as the expertise required to introduce solutions for the learning of the future;

36.   Highlights, moreover, the need to continuously train teachers so they can adapt to the realities of AI-powered education and acquire the necessary knowledge and skills to use AI technologies in a pedagogical and meaningful way, enabling them to fully embrace the possibilities offered by AI and to understand its limitations; calls for digital teaching to be part of every teacher’s training in the future and calls for teachers and people working in education and training to be given the opportunity to continue their training in digital teaching throughout their lives; calls, therefore, for the development of training programmes in AI for teachers in all fields and across Europe; highlights, furthermore, the importance of reforming teaching programmes for new generations of teachers allowing them to adapt to the realities of AI-powered education, as well as the importance of drawing up and updating handbooks and guidelines on AI for teachers;

37.   Is concerned about the lack of specific higher education programmes for AI and the lack of public funding for AI across the Member States; believes that this is putting Europe’s future digital ambitions at risk;

38.   Is worried about the fact that few AI researchers are pursuing an academic career as tech firms can offer better pay and less bureaucracy for research; believes that part of the solution would be to direct more public money towards AI research at universities;

39.   Underlines the importance of equipping people with general digital skills from childhood onwards in order to close the qualifications gap and better integrate certain population groups into the digital labour market and digital society; points out that it will become more and more important to train highly skilled professionals from all backgrounds in the field of AI, ensure the mutual recognition of such qualifications throughout the Union, and upskill the existing and future workforce to enable it to cope with the future realities of the labour market; encourages the Member States, therefore, to assess their educational offer and to upgrade it with AI-related skills, where necessary, and to put in place specific curricula for AI developers, while also including AI in traditional curricula; highlights the need to ensure mutual recognition of professional qualifications in AI skills across the Union, as several Member States are upgrading their educational offer with AI-related skills and putting in place specific curricula for AI developers; welcomes the Commission’s efforts to include digital skills as one of the qualifications requirements for certain professions harmonised at Union level under the Professional Qualifications Directive [11] ; stresses the need for these to be in line with the assessment list of the ethical guidelines for trustworthy AI, and welcomes the Commission’s proposal to transform this list into an indicative curriculum for AI developers; recalls the special needs of vocational education and training (VET) with regard to AI and calls for a collaborative approach across Europe to enhance the potential offered by AI in VET; underlines the importance of training highly skilled professionals in this area, including ethical aspects in curricula, and of supporting underrepresented groups in this field, as well as of creating incentives for those professionals to seek work within the Union; recalls that women are underrepresented in AI and that this may create significant gender imbalances in the future labour market;

40.   Stresses the need for governments and educational institutions to rethink, rework and adapt their educational curricula to the needs of the 21st century by devising educational programmes that place greater emphasis on STEAM subjects in order to prepare learners and consumers for the increasing prevalence of AI and facilitate the acquisition of cognitive skills; underlines, in this regard, the importance of diversifying this sector and of encouraging students, especially women and girls, to enrol in STEAM courses, in particular in robotics and AI-related subjects; calls for more financial and scientific resources to motivate skilled people to stay in the Union while attracting those with skills from third countries; notes, furthermore, the considerable number of start-ups working with AI and developing AI technologies; stresses that SMEs will require additional support and AI-related training to comply with digital and AI-related regulation;

41.   Notes that automation and the development of AI may drastically and irreversibly change employment; emphasises that priority should be given to tailoring skills to the needs of the future job market, in particular in education and the CCSI; underlines, in this context, the need to upskill the future workforce; stresses, furthermore, the importance of deploying AI to reskill and upskill the European labour market in the CCSI, in particular in the audiovisual sector, which has already been severely impacted by the COVID-19 crisis;

42.   Calls on the Commission to assess the level of risk of AI deployment in the education sector in order to ascertain whether AI applications in education should be included in the regulatory framework for high risk and subject to stricter requirements on safety, transparency, fairness and accountability, in view of the importance of ensuring that education continues to contribute to the public good and the acute sensitivity of data on pupils, students and other learners; underlines that datasets used to train AI should be reviewed in order to avoid reinforcing certain stereotypes and other kinds of bias;

43.   Calls on the Commission to propose a futureproof legal framework for AI so as to provide legally binding ethical measures and standards to ensure fundamental rights and freedoms and the development of trustworthy, ethical and technically robust AI applications, including integrated digital tools, services and products such as robotics and machine learning, with particular regard to education; calls for the data used and produced by AI applications in education to be accessible, interoperable and of high quality, and to be shared with the relevant public authorities in an accessible way and with respect for copyright and trade secrets legislation; recalls that children constitute a vulnerable group who deserve particular attention and protection; stresses that while AI can benefit education, it is necessary to take into account its technological, regulatory and social aspects, with adequate safeguards and a human-centred approach that ultimately ensures that human beings are always able to control and correct the system’s decisions; points out, in this regard, that teachers must control and supervise any deployment and use of AI technologies in schools and universities when interacting with pupils and students; recalls that AI systems must not take any final decision that could affect educational opportunities, such as students’ final evaluation, without full human supervision; recalls that automated decisions about natural persons based on profiling, where they have legal or similar effects, must be strictly limited and always require the right to human intervention and the right to an explanation under the GDPR; underlines that this should be strictly adhered to, especially in the education system, where decisions about future chances and opportunities are taken;

44.   Expresses serious concern that schools and other education providers are becoming increasingly dependent on educational technology (edtech) services, including AI applications, provided by a few private companies that enjoy a dominant market position; believes that this should be scrutinised through Union competition rules; stresses the importance, in this regard, of supporting the uptake of AI by SMEs in education, culture and the audiovisual sector through the appropriate incentives that create a level playing field; calls, in this context, for investment in European IT companies in order to develop the necessary technologies within the Union, given that the major companies that currently provide AI are based outside the Union; strongly recalls that the data of minors is strictly protected by the GDPR and can only be processed if completely anonymised or consent has been given or authorised by the holder of parental responsibility, in strict compliance with the principles of data minimisation and purpose limitation; calls for more robust protection and safeguards in the education sector where children’s data is concerned and calls on the Commission to take more effective steps in that regard; calls for clear information to be provided to children and their parents about the possible use and processing of children’s data, including through awareness-raising and information campaigns;

45.   Underlines the specific risks in the use of AI automated recognition applications, which are developing at pace; recalls that children are a particularly sensitive group; recommends that the Commission and the Member States ban automated biometric identification, such as facial recognition for educational and cultural purposes, on educational and cultural premises, unless its use is allowed by law;

46.   Stresses the need to increase customer choice to stimulate competition and broaden the range of services offered by AI technologies for educational purposes; encourages public authorities, in this regard, to incentivise the development and deployment of AI technologies through public funding and public procurement; considers that technologies used by public education providers or purchased with public funding should be based on open-source technologies;

47.   Notes that innovation in education is overdue, as highlighted by the COVID-19 pandemic and the ensuing switch to online and distance learning; stresses that AI-driven educational tools such as those for assessing and identifying learning difficulties can improve the quality and effectiveness of online learning;

48.   Stresses that next-generation digital infrastructure and internet coverage are of strategic significance for providing AI-powered education to European citizens; in light of the COVID-19 crisis, calls on the Commission to elaborate a strategy for a European 5G that ensures Europe’s strategic resilience and is not dependent on technologies from states which do not share our values;

49.   Calls for the creation of a pan-European university and research network focused on AI in education, which should bring together institutions and experts from all fields to examine the impact of AI on learning and identify solutions to enhance its potential;

Cultural heritage

50.   Reiterates the importance of access to culture for every citizen throughout the Union; highlights, in this context, the importance of the exchange of best practices among Member States, educational facilities and cultural institutions and similar stakeholders; further considers it of vital importance that the resources available at both Union and national level are used to the maximum of their potential in order to further improve access to culture; stresses that there are a multitude of options to access culture and that all varieties should be explored in order to determine the most appropriate option; highlights the importance of consistency with the Marrakech Treaty;

51.   Stresses that AI technologies can play a significant role in preserving, restoring, documenting, analysing, promoting and managing tangible and intangible cultural heritage, including by monitoring and analysing changes to cultural heritage sites caused by threats such as climate change, natural disasters and armed conflicts;

52.   Stresses that AI technologies can increase the visibility of Europe’s cultural diversity; points out that these technologies provide new opportunities for cultural institutions, such as museums, to produce innovative tools for cataloguing artefacts as well as documenting and making cultural heritage sites more accessible, including through 3D modelling and augmented virtual reality; stresses that AI will also enable museums and art galleries to introduce interactive and personalised services for visitors by providing them with a list of suggested items based on their interests, expressed in person and online;

53.   Stresses that the use of AI will herald new innovative approaches, tools and methodologies allowing cultural workers and researchers to create uniform databases with suitable classification schemes as well as multimedia metadata, enabling them to make connections between different cultural heritage objects and thus increase knowledge and provide a better understanding of cultural heritage;

54.   Stresses that good practices in AI technologies for the protection and accessibility of cultural heritage, in particular for people with disabilities, should be identified and shared between cultural networks across the Union, while encouraging research on the various uses of AI to promote the value, accessibility and preservation of cultural heritage; calls on the Commission and the Member States to promote the opportunities offered by the use of AI in the CCSI;

55.   Stresses that AI technologies can also be used to monitor the illicit trafficking of cultural objects and the destruction of cultural property, while supporting data collection for recovery and reconstruction efforts of both tangible and intangible cultural heritage; notes, in particular, that the development, deployment and use of AI in customs screening procedures may support efforts to prevent the illicit trafficking of cultural heritage, in particular to supplement systems which allow customs authorities to target their efforts and resources on items that pose the greatest risk;

56.   Notes that AI could benefit the research sector, for example through the role that predictive analytics can play in fine-tuning data analysis, for example on the acquisition and movement of cultural objects; stresses that the Union must step up investment and foster partnerships between industry and academia in order to enhance research excellence at European level;

57.   Recalls that AI can be a revolutionary tool for promoting cultural tourism and highlights its considerable potential in predicting tourism flows, which could help cities struggling with over-tourism;

Cultural and creative sectors and industries (CCSI)

58.   Regrets the fact that culture is not among the priorities outlined in policy options and recommendations on AI at Union level, notably the Commission’s white paper of 19 February 2020 on AI; calls for these recommendations to be revised in order to make culture an AI policy priority at Union level; calls on the Commission and the Member States to address the potential impact of the development, deployment and use of AI technologies on the CCSI and to make the most of the Next Generation EU recovery plan to digitise these sectors to respond to new forms of consumption in the 21st century;

59.   Points out that AI has now reached the CCSI, as exemplified by the automatic production of texts, videos and pieces of music; emphasises that creative artists and cultural workers must have the digital skills and training required to use AI and other digital technologies; calls on the Commission and the Member States to promote the opportunities offered by the use of AI in the CCSI, by making more funding available from science and research budgets, and to establish digital creativity centres in which creative artists and cultural workers develop AI applications, learn how to use these and other technologies and test them;

60.   Acknowledges that AI technologies have the potential to boost a growing number of jobs in the CCSI facilitated by greater access to these technologies; emphasises, therefore, the importance of boosting digital literacy in the CCSI to make these technologies more inclusive, usable, learnable, and interactive for these sectors;

61.   Emphasises that the interaction between AI and the CCSI is complex and requires an in‑depth assessment; welcomes the Commission’s report of November 2020 entitled ‘Trends and Developments in Artificial Intelligence – Challenges to the IPRS Framework’ and the study on copyright and new technologies: copyright data management and artificial intelligence; underlines the importance of clarifying the conditions of use of copyright‑protected content as data input (images, music, films, databases, etc.) and in the production of cultural and audiovisual outputs, whether created by humans with the assistance of AI or autonomously generated by AI technologies; invites the Commission to study the impact of AI on the European creative industries; reiterates the importance of European data and welcomes the statements made by the Commission in this regard, as well as the placing of artificial intelligence and related technologies high on the agenda;

62.   Stresses the need to set up a coherent vision of AI technologies in the CCSI at Union level; calls on the Member States to strengthen the focus on culture in their AI national strategies to ensure that the CCSI embrace innovation and remain competitive and that cultural diversity is safeguarded and promoted at Union level in the new digital context;

63.   Stresses the importance of creating a Union-wide heterogeneous milieu for AI technologies to encourage cultural diversity and support minorities and linguistic diversity, while also strengthening the CCSI through online platforms, allowing Union citizens to be included and to participate;

64.   Calls on the Commission and the Member States to support a democratic debate on AI technologies and to provide a regular forum for discussion with civil society, researchers, academia and stakeholders to raise awareness on the benefits and the challenges of its use in the CCSI; emphasises, in that connection, the role which art and culture can play in familiarising people with AI and fostering public debate about it, as they can provide vivid, tangible examples of machine learning, for example in the area of music;

65.   Calls on the Commission and the Member States to address the issue of AI-generated content and its challenges to authorship and copyright infringement; asks the Commission, in that regard, to assess the impact of AI and related technologies on the audiovisual sector and the CCSI , with a view to promoting cultural and linguistic diversity, while respecting authors’ and performers’ rights;

66.   Stresses that the European Institute of Innovation and Technology (EIT), in particular its future Knowledge and Innovation Community (KIC) dedicated to cultural and creative industries (CCI), should play a leading role in developing a European strategy on AI in education, culture and the audiovisual sector and can help accelerate and harvest AI applications to these sectors;

67.   Notes that AI has already entered the creative value chain at the level of creation, production, dissemination and consumption and is therefore having an immense impact on the CCSI, including music, the film industry, art and literature, through new tools, software and AI-assisted production for easier production, while providing inspiration and enabling the broader public to create content;

68.   Calls on the Commission to carry out studies and consider policy options to tackle the detrimental impact of AI-based control of online streaming services designed to limit diversity and/or maximise profits by including or prioritising certain content in the consumer offer, as well as how this impacts cultural diversity and creators’ earnings;

69.   Believes that AI is becoming increasingly useful for the CCSI in creation and production activities;

70.   Emphasises the role of an author’s personality for the expression of free and creative choices that constitute the originality of works [12] ; underlines the importance of limitations and exceptions to copyright when using content as data input, notably in education, academia and research, and in the production of cultural and creative output, such as audiovisual output and user-generated content;

71.   Takes the view that consideration should be given to protecting AI-generated technical and artistic creations in order to encourage this form of creativity;

72.   Stresses that in the data economy context, better copyright data management is achievable, for the purpose of better remunerating authors and performers, notably in enabling the swift identification of the authorship and right ownership of content, thus contributing to lowering the number of orphan works; further highlights that AI technological solutions should be used to improve copyright data infrastructure and the interconnection of metadata in works, but also to facilitate the transparency obligation provided in Article 19 of Directive (EU) 2019/790 on copyright and related rights in the Digital Single Market [13] for up‑to‑date, relevant and comprehensive information on the exploitation of authors’ and performers’ works and performances, particularly in the presence of a plurality of rights holders and of complex licensing schemes;

73.   Calls for the intellectual property action plan announced by the Commission to address the question of AI and its impact on the creative sectors, taking account of the need to strike a balance between protecting IPR and encouraging creativity in the areas of education, culture and research; considers that the Union can be a leader in the creation of AI technologies if it adopts an operational regulatory framework and implements proactive public policies, particularly as regards training programmes and financial support for research; asks the Commission to assess the impact of IPR on the research and development of AI and related technologies, as well as on the CCSI, including the audiovisual sector, with particular regard to authorship, fair remuneration of authors and related questions;

74.   Calls on the Commission to consider the legal aspects of the output produced using AI technologies, as well as cultural content generated with the use of AI and related technologies; considers it important to support the production of cultural content; reiterates, however, the importance of safeguarding the Union’s unique IPR framework and that any changes should be made with the necessary due care, in order not to disrupt the delicate balance; calls on the Commission to produce an in-depth assessment with regard to the possible legal personality of AI-produced content, as well as the application of IPR to AI-generated content and to content created with the use of AI tools;

75.   Calls on the Commission, in addition, to consider developing, in very close cooperation with Member States and the relevant stakeholders, verification mechanisms or systems for publishers, authors and creators in order to assist them in verifying what content they may use and to more easily determine what is protected under IPR legislation;

76.   Calls on the Commission to lay down rules designed to guarantee effective data interoperability in order to make content purchased on a platform accessible via any digital tool irrespective of brand;

Audiovisual sector

77.   Notes that AI is often used to enable automated decision-making algorithms to disseminate and order the cultural and creative content displayed to users; stresses that these algorithms are a ‘black box’ for users; stresses that the algorithms used by media service providers, video sharing platforms (VSPs) and music streaming services should be designed in such a way that they do not privilege specific works by limiting their ‘personalised’ suggestions to the most popular works, for targeted advertising, commercial purposes or to maximise profit; calls for recommendation algorithms and personalised marketing to be explainable and transparent where possible, in order to give consumers an accurate and comprehensive insight into these processes and content and to ensure that personalised services are not discriminatory and in line with the recently adopted Platform to Business Regulation [14] and New Deal for Consumers Omnibus Directive [15] ; calls on the Commission to address the ways in which content moderation algorithms are optimised to engage users, and to propose recommendations to increase user control over the content they see, by guaranteeing and properly implementing the right of users to opt out of recommended and personalised services; underlines, moreover, that consumers must be informed when they are interacting with an automated decision process and that their choices and performance must not be limited; stresses that the use of AI mechanisms for the commercial surveillance of consumers must be countered, even if it concerns ‘free services’, by ensuring that it is strictly in line with fundamental rights and the GDPR; stresses that all regulatory changes must take into consideration the impact on vulnerable consumers;

78.   Underlines that what is illegal offline shall be illegal online; notes that AI tools have the potential and are already used to fight illegal content online, but strongly recalls ahead of the forthcoming Digital Services Act that such tools must always respect fundamental rights, especially freedom of expression and information, and should not lead to a general monitoring obligation for the internet, or to the removal of legal material disseminated for education, journalistic, artistic or research purposes; stresses that algorithms should be used only as a flagging mechanism in content moderation, subject to human intervention, as AI is unable to reliably distinguish between legal, illegal and harmful content; notes that terms and conditions should always include community guidelines as well as an appeal procedure;

79.   Recalls, furthermore, that there should be no general monitoring, as stipulated in Article 15 of the e-Commerce Directive [16] , and that specific content monitoring for audiovisual media services should be in accordance with the exceptions laid down in Union legislation; recalls that AI applications must adhere to internal and external safety protocols, which should be technically accurate and robust in nature; considers that this should extend to operation in normal, unknown and unpredictable situations alike;

80.   Stresses, moreover, that the use of AI in algorithm-based content recommendations on media service providers, such as video on demand services and VSPs, may have a serious impact on cultural and linguistic diversity, notably regarding the obligation to ensure the prominence of European works under Article 13 of the Audiovisual Media Services Directive (Directive (EU) 2018/1808 [17] ); notes that the same concerns are equally relevant for the music streaming services and calls for the development of indicators to assess cultural diversity and the promotion of European works on such services;

81.   Calls on the Commission and the Member States to step up their financial support for the development, deployment and use of AI in the area of the automatic subtitling and dubbing of European audiovisual works, in order to foster cultural and language diversity in the Union and enhance the dissemination of and access to European audiovisual content;

82.   Calls on the Commission to establish a clear ethical framework for the use of AI technologies in media in order to prevent all forms of discrimination and ensure access to culturally and linguistically diverse content at Union level, based on accountable, transparent and inclusive algorithms, while respecting individuals’ choices and preferences;

83.   Points out that AI can play a major role in the rapid spread of disinformation; stresses, in that regard, that the framework should address the misuse of AI to disseminate fake news and online misinformation and disinformation, while avoiding censorship; calls on the Commission, therefore, to assess the risks of AI assisting the spread of disinformation in the digital environment as well as solutions on how AI could be used to help counter disinformation;

84.   Calls on the Commission to take regulatory measures to ensure that media service providers have access to the data generated by the provision and dissemination of their content on other providers’ platforms; emphasises that full data transfer from platform operators to media service providers is vital if the latter are to understand their audience better and thus improve the services they offer in keeping with people’s wishes;

85.   Stresses the importance of increasing funding for Digital Europe, Creative Europe and Horizon Europe in order to reinforce support for the European audiovisual sector, namely by collaborative research projects and experimental pilot initiatives on the development, deployment and use of ethical AI technologies;

86.   Calls for close collaboration between Member States in developing training programmes aimed at reskilling or upskilling workers to make them better prepared for the social transition that the use of AI technologies in the audiovisual sector will entail;

87.   Considers that AI has enormous potential to help drive innovation in the news media sector; believes that the widespread integration of AI, such as for content generation and distribution, the monitoring of comments sections, the use of data analytics, and identifying doctored photos and videos, is key for saving on costs in newsrooms in the light of diminishing advertising revenues and for devoting more resources to reporting on the ground and thus increase the quality and variety of content;

Online disinformation: deepfakes

88.   Stresses the importance of ensuring online and offline media pluralism to guarantee the quality, diversity and reliability of the information available;

89.   Recalls that accuracy, independence, fairness, confidentiality, humanity, accountability and transparency, as driving forces behind the principles of freedom of expression and access to information in online and offline media, are decisive in the fight against disinformation and misinformation;

90.   Notes the important role which independent media play in culture and the daily life of citizens; stresses that disinformation represents a fundamental problem, as copyright and IPR generally are being constantly infringed; calls on the Commission, in cooperation with the Member States, to continue its work on raising awareness of this problem, countering the effects of disinformation as well as the source problems; considers it important, furthermore, to develop educational strategies to specifically improve digital literacy in this regard;

91.   Recalls that with new techniques rapidly emerging, detecting false and manipulated content such as deepfakes may become increasingly challenging due to the ability of malicious producers to generate sophisticated algorithms that can be successfully trained to evade detection, thus seriously undermining our basic democratic values; asks the Commission to assess the impact of AI in the creation of deepfakes, to establish appropriate legal frameworks to govern their creation, production or distribution for malicious purposes, and to propose recommendations for, among other initiatives, action against any AI-powered threats to free and fair elections and democracy;

92.   Welcomes recent initiatives and projects to create more efficient deepfake detection tools and transparency requirements; stresses, in this regard, the need to explore and invest in methods for tackling deepfakes as a crucial step in combatting misinformation and harmful content; considers that AI-enabled solutions can be helpful in this regard; asks the Commission, therefore, to impose an obligation for all deepfake material or any other realistically made synthetic videos to state that the material is not original and a strict limitation when used for electoral purposes;

93.   Is concerned that AI is having an ever greater influence on the way information is found and consumed online; points out that so-called filter bubbles and echo chambers are restricting diversity of opinion and undermining open debate in society; urges, therefore, that the way platform operators use algorithms to process information must be transparent and that users must be given greater freedom to decide whether and what information they want to receive;

94.   Points out that AI technologies are already being used in journalism, for example to produce texts or, in the context of investigative research, to analyse large data sets; emphasises that in the context of producing information of significance to society as a whole, it is important that automated journalism should draw on correct and comprehensive data, in order to prevent the dissemination of fake news; emphasises that the basic principles of quality journalism, such as editorial supervision, must also apply to journalistic content produced using AI technologies; calls for AI-generated texts to be clearly identified as such, in order to safeguard trust in journalism;

95.   Highlights the potential of AI to facilitate and encourage multilingualism by developing language-related technologies and enabling online European content to be discovered;

96.   Instructs its President to forward this resolution to the Council and the Commission.

“I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted ”

Alan Turing, 1947

The last decade has been transformative for AI, arousing both fear and excitement for humanity. Seen as ‘the new electricity’, AI has advanced to the point that it will have such a systemic impact that it could substantially change all aspects of society for the next century.

Whilst it is easy to understand the potential effects of AI on sectors such telecommunications, transportation, traffic management, health care, evaluating its long-term effects on education, culture and the audiovisual sector is considerably more challenging. Although there is a consensus that AI and automation is likely to create more wealth and to simplify a vast array of processes, the use of AI has also raised serious concerns that it may result in an increase in inequality, discrimination and unemployment.

The potential impact of AI on education, culture and the audiovisual sector is however rarely discussed and is mostly unknown. Yet this question is of utmost importance because AI is already being used to teach curricula, as well as to produce movies, songs, stories and paintings.

The purpose of this report is therefore to understand concretely how AI currently impacts these sectors and how future technological advances in AI will impact them further over the next decade. In particular, the Rapporteur reflects on how AI may transform these sectors and which particular regulatory challenges the Union may have to face in that regard.

(i) AI is reshaping education

AI is transforming learning, teaching, and education radically. The whirlwind speed of technological development accelerates the radical transformation of educational practices, institutions and policies. In this field, AI has many applications, such as customisable approaches to learning, AI-based tutors, textbooks and course material with customised content, smart algorithms to determine best teaching methods, AI game engines, and adaptive user models in personalised learning environments (PLE) which can allow early identification of difficulties, such as dyslexia or risks of early school leaving.

Personalised learning experience is the cornerstone of the use of AI in education. It would allow students to enjoy an educational approach that is fully tailored to their individual abilities, needs and difficulties, whilst enabling teachers to closely monitor students’ progress. However in order to make personalised education a reality, large amounts of personal data need to be collected, used and analysed.

The Rapporteur stresses in that regard that the current lack of access to personal data on students is likely to prevent the successful implementation of AI in education. It is thus essential to ensure the safety and transparency of personal data collection, use, management and dissemination, whilst safeguarding confidentiality and privacy of learners’ personal data. Moreover, addressing the risks of AI potential bias, as well as tackling the issue of data storage should be a priority in any initiatives for the wide deployment of AI in the education system at Union level.

Although there is little chance that teachers will be replaced by machines in the near future, the increasing use of AI in education implies the need to rethink education overall, as well as to reflect on the redefinition of teaching, the role of the teachers, and, as a result, the subsequent retraining required to adapt to a AI-based educational system.

Considering that less than 40% of teachers in the Union have received courses on ICT inclusion in the classroom throughout their Initial Teacher Education (ITE), the Rapporteur would like to stress the crucial importance of training teachers so they acquire digital skills as a prerequisite to becoming familiar with AI. They could then take advantage of AI technologies, but also make them aware of the potential dangers of AI.

This issue can also be seen more widely, with 42% of the Union population still lacking basic digital skills. There are also serious regional discrepancies in access to digital infrastructure and in digital skills attainment across the Union.

Emerging technology trends related to digital transformation, such as AI, have profound implications in terms of the skills required for the evolving digital economy. In particular, the notion of lifelong learning in AI has emerged as one of the key strategies for job security and employment in the digital era.

The Rapporteur suggests that citizens are trained to acquire the necessary digital skills, whilst carefully assessing what AI-related skills are needed today and in the future, and that the necessary measures are taken to address existing and emerging skill gaps.

It also crucial to ensure that the prerequisites for the deployment and the relevant use of AI, in terms of internet access, connectivity, networks and infrastructures, are met.

  (ii) AI can be used to safeguard and promote cultural heritage

In recent years, AI has been of increasing relevance to cultural heritage, notably in response to potential modern threats, such as climate change or conflicts. AI can have various applications in that regard: it can be used to enhance users’ experience by enabling visitors of cultural institutions and museums to create personal narrative trails or to enjoy virtual tour guides. Conversational bots could communicate in an interactive way about cultural heritage on any topics and in any language. They would also make the access to information easier whilst providing a vivid cultural experience to users.

AI could also facilitate the understanding of the history of the Union, such as how the ‘Time Machine Project’ aims to create advanced AI technologies to make sense of vast amounts of information from complex historical data sets stored in archives and museums. This enables the transformation of fragmented data into useable knowledge by mapping Union’s entire social, cultural and geographical evolution. This may facilitate the exploration of the cultural, economic, and historical development of European cities, and improve understanding thereof.

(iii) AI changes the way the cultural and creative industries work, in particular the audiovisual sector

AI use is rapidly expanding in media with many applications:

-   Data-driven marketing and advertising, by training machine learning algorithms to develop promotional movie trailers and design advertisements,

-   Personalisation of users’ experience, by using machine learning to recommend personalised content based on data from user activity and behaviour,

-   Search optimisation, by using AI to improve the speed and efficiency of the media production process and the ability to organise visual assets,

-   Content creation, by generating video clips from automatic video segments ready for broadcast and special effects, such as re-creating a younger version of an actor digitally or creating new content with a deceased actor,

-   Script writing such as simple factual text creation (sports and news reports produced by robots), but also for writing fictional stories, such as the experimental short movie ‘Sunspring’,

-   Viewer interaction on complex story lines, such as the last episode of the British series ‘Black Mirror’, ‘Bandersnatch’,

-   Automated captioning and subtitling, such audio-to-text processes, for viewers with disabilities

-   Automated content moderation on audiovisual content.

Whilst AI offers a wide range of opportunities in producing high quality cultural and creative content, the centralised distribution and access to such content raises a number of ethical and legal issues, notably on data protection, freedom of expression and cultural diversity.

Cultural and creative works, notably audiovisual works, are mainly distributed through

large centralised platforms, which conditions media consumption to the proprietary algorithms developed by these platforms.

The Rapporteur points out that algorithm-based personalised recommendations are potentially detrimental to cultural and linguistic diversity, preventing under-represented cultural and creative content from appearing in suggestions provided by these systems. On the largest platforms, the criteria used to select or recommend a work are neither transparent nor auditable, and are likely to be decided on the basis of economic factors that solely benefit these platforms.

The question of cultural and linguistic diversity in recommendation systems is therefore crucial and must be addressed. The Rapporteur stresses the need to set up a clear legal framework for transparent, accountable and inclusive algorithms, in order to safeguard and promote cultural and linguistic diversity .

Regulatory challenges triggered by AI applications within the audiovisual sector are also linked to existing legal acts, such as the AVMSD. Thus a more in-depth assessment might be needed as to the urgency and/or political momentum for future adaptions of these files to AI.

Whilst AI can help empower many creators, making CCS more prosperous and driving cultural diversity, the large majority of artists and entrepreneurs may not still be familiar with AI tools.

There is a lack of technical knowledge among creators precluding them from experimenting with machine learning and reaping the benefits they can bring. Therefore it is essential to assess which skills would be needed in the near future, whilst at the same time improving training systems, including upskilling and reskilling, guaranteeing lifelong learning throughout the whole working life and beyond.

In that context, the Rapporteur suggests setting up an AI observatory with an objective of harmonising and facilitating evidence-based scrutiny of new developments in AI in order to tackle the question of auditability and accountability of AI applications in CCS .

(v) Countering fake news

AI-technologies are increasingly used to disseminate fake news, notably through the use of ‘deepfakes’.

Deepfakes are synthetic images or videos generated by AI using ‘deep learning machines’ and generative adversial networks (GAN). Humans cannot distinguish deepfakes from authentic content. Deepfakes can be used for all kinds of trickery, most commonly used for ‘face swaps’, from harmless satires and film tweaks to malicious hoaxes, targeted harassment, deepfake porn or financial fraud, The danger of deepfakes is to make people believe something is real when it is not, and thus may be used as a particularly powerful and potent weapon for online disinformation, spreading virally on platforms and social media, where they can influence public opinion, voting processes and election results.

Whilst AI is frequently singled out for its role in spreading fake news, it could also play a significant role in countering and combating fake news and disinformation, as evidenced by projects such as “Fake News Challenge”. AI systems can reverse-engineer AI generated fake news, and help spot manipulated content. However algorithms generating deepfakes are getting more and more sophisticated, and detecting them as a result is getting increasingly difficult.

The Rapporteur therefore stresses the need to tackle the misuse of AI in disseminating fake news and online misinformation, notably by exploring ways to efficiently detect deepfakes.

OPINION OF THE COMMITTEE ON CIVIL LIBERTIES, JUSTICE AND HOME AFFAIRS  ( 16.7.2020 )

for the Committee on Culture and Education

Rapporteur for opinion (*): Ondřej Kovařík

(*) Associated committee – Rule 57 of the Rules of Procedure

SUGGESTIONS

The Committee on Civil Liberties, Justice and Home Affairs calls on the Committee on Culture and Education, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

1.   Underlines that the use of AI in the education, culture and audiovisual sectors must fully respect fundamental rights, freedoms and values, including privacy, the protection of personal data, non-discrimination and freedom of expression and information, as enshrined in the EU Treaties and the Charter of Fundamental Rights of the European Union; welcomes the Commission’s White Paper on Artificial Intelligence in this regard, and invites the Commission to include the educational sector, limited to areas posing significant risks, in the future regulatory framework for high-risk AI applications;

2.   Recalls that AI may give rise to biases and thus to various forms of discrimination based on sex, race, colour, ethnic or social origin, genetic features, language, religion or belief, political or any other opinion, membership of a national minority, property, birth, disability, age or sexual orientation; in this regard, recalls that everyone’s rights must be ensured and that AI initiatives must not be discriminatory in any form;

3.   Emphasises that such bias and discrimination can arise from already biased sets of data, reflecting existing discrimination in society; stresses that AI must avoid bias leading to prohibited discrimination and must not reproduce discrimination processes; underlines the need to take these risks into account when designing AI technologies, as well as, the importance of working with AI technology providers to address persistent loopholes facilitating discrimination, and recommends that AI designing and developing teams should reflect the diversity of society;

4.   Notes that the use of AI in education brings a wide range of possibilities and opportunities, for instance to facilitate access to information, to improve research methods or to understand how pupils learn and to offer them customisation, while at the same time posing risks regarding equal access to education and learning equalities at an increasingly younger age and for vulnerable and historically disadvantaged groups; calls for a sufficient data-sharing infrastructure between AI applications and public research entities; points out that equity and inclusion are core values that should be duly taken into account when designing policies for AI in education; calls for the non-discriminatory use of AI in the education sector; recalls the risks and discrimination that may arise from recently developed AI tools used for purposes of school admissions, and calls for them to be rectified as soon as possible; underlines the need for a proper assessment of the AI tools used in the education sector to identify their impact on the rights of the child;

5.   Acknowledges that using digital and AI technologies can help develop increasingly effective educational tools and lead to a more inclusive society, by countering traditional forms of discrimination including lack of access to services, by bringing education to disadvantaged communities, persons with disabilities in line with the EU Accessibility Act, and other categories of European citizens lacking proper access to education, and by providing access to adequate learning opportunities;

6.   Underlines that the benefits of AI should be shared with all parts of society, leaving no one behind; stresses the need to fully take into consideration the specific needs of the most vulnerable groups, such as children, persons with disabilities, elderly people and other groups at risk of exclusion; expresses its concerns about limited accessibility of the internet in some regions across the EU, and calls on the Commission and the Member States to deploy sustained efforts to ameliorate telecommunications infrastructures;

7.   Recognises the possibilities of AI in the culture sector in terms of developing music, art and other cultural expressions; emphasises that freedom of expression is an important freedom and value and that a pluriform cultural landscape is of great value to society; calls on the Commission to keep these values in mind when drafting its proposals on AI;

8.   Welcomes the Commission’s plan to update the Digital Education Action Plan so as to make it more ambitious and integrated with a view to making educational systems fit for the digital age, notably through making better use of data and AI-based technologies; calls on all stakeholders, both public and private, to closely cooperate in implementing these educational reforms;

9.   Stresses the need to ensure more general public awareness of AI at all levels, as a key element to enable the public to make informed decisions and help strengthen the resilience of our societies; underlines that this must also include public awareness of the risks in terms of privacy and biases related to AI; invites the Commission and the Member States to include the above in educational programmes and programmes which support the arts;

10.   Underlines the urgent need to educate the public at every level in the use of AI and to equip all European citizens, including vulnerable groups, with basic digital skills enabling equal social and economic opportunities, as well as the need to have high-quality ICT programmes in education systems, at all levels; calls for the digital gender gap not to be underestimated and for measures to be taken to remedy it; welcomes the upcoming update of the Skills Agenda aimed at allowing everyone to benefit from EU digital transformation; emphasises the importance of training teachers and educators in the use of AI, especially those responsible for underage students; notes that significant skills shortages still exist in the digital and technology sectors; underlines the importance of diversifying this sector and to encourage students, in particular women and girls, to enrol in Science, Technology, Engineering and Mathematics (STEM) courses, in particular in robotics and AI-related subjects, in addition to those related to their career aspirations; calls for more financial and scientific resources to motivate skilled people to stay in the EU and to attract those with skills from abroad; furthermore, notes that there are a considerable number of start-ups working with AI and developing AI technologies; stresses that small and medium-sized enterprises (SMEs) will require additional support and AI-related training to comply with digital and AI-related regulation;

11.   Recalls that data protection and privacy can be particularly affected by AI; underlines the principles established in Regulation (EU) 2016/679 of the European Parliament and of the Council (the General Data Protection Regulation (GDPR)) [18] as binding principles for AI deployment; recalls that all AI applications need to fully respect Union data protection law, namely the GDPR and Directive (EC) 2002/58 of the European Parliament and of the Council (the ePrivacy Directive) [19] (currently under revision);

12.   Recalls that children constitute a vulnerable public who deserve particular attention and protection; recalls that automated decisions about natural persons based on profiling, where they have legal or similar effects, must be strictly limited and always require the right to human intervention and to explicability under the GDPR; underlines that this should be strictly adhered to, especially in the education system, where decisions about future chances and opportunities are taken; observes that a few private companies are dominating the educational technology (edtech) sector in some Member States, and believes this should be scrutinised through EU competition rules; strongly recalls that data of minors is strictly protected by the GDPR, and that children’s data can only be processed if completely anonymised or where consent has been given or authorised by the holder of parental responsibility over the child; therefore calls for stronger protection and safeguards in the education sector where children’s data are concerned; calls for clear information to be provided to children and their parents, including via awareness and information campaigns, about the possible use and processing of children’s data;

13.   Underlines the specific risks existing in the use of AI automated recognition applications, which are currently developing rapidly; recalls that children are a particularly sensitive public; recommends that the Commission and the Member States ban automated biometric identification, such as facial recognition for educational and cultural purposes, on educational and cultural premises, unless its use is allowed in law;

14.   Calls on the Commission and the Member States to implement an obligation of transparency and explainability of AI-based automated individual decisions taken within the framework of prerogatives of public power, and to implement penalties to enforce such obligations; calls for the implementation of systems which use human verification and intervention by default, and for due process, including the right of appeal, and access to remedies; recalls that automated decisions about natural persons based on profiling, where they have legal or similar effects, must be strictly limited and always require the right to human intervention and to explainability under the GDPR;

15.   Calls for independent audits to be conducted regularly to examine whether AI applications being used and the related checks and balances are in accordance with specified criteria, and for those audits to be supervised by independent and sufficient overseeing authorities; calls for specific stress tests to assist and enforce compliance;

16.   Points out that AI can play a major role in the rapid spread of disinformation; therefore calls on the Commission to assess the risks of AI assisting the spread of disinformation in the digital environment, and to propose recommendations, among others, for action against any AI-powered threats to free and fair elections and democracy; observes that deep fakes can also be used to manipulate elections, to disseminate disinformation and for other undesirable actions; notes furthermore that the immersive experiences facilitated by AI can be exploited by malicious actors; asks the Commission to propose recommendations, including possible restrictions in this regard, in order to adequately safeguard against the use of these technologies for illegal purposes; also calls for an assessment of how AI could be used to help counter disinformation; calls on the Commission to ensure that any future regulatory framework does not lead to censorship of legal individual content uploaded by users; recalls that critical thinking and the ability to interact with skill and confidence in the online environment are needed more than ever;

17.   Notes that AI is often used to enable automated decision-making algorithms to disseminate and order the content displayed to users; stresses that these algorithms are a ‘black box’ for users; calls on the Commission to address the ways in which content moderation algorithms are optimised towards engagement of their users; also calls on the Commission to propose recommendations to increase user control over the content they see, and to ask AI applications and internet platforms to give users the possibility to choose to have content displayed in a neutral order, in order to give them more control on the way content is ranked to them, including options for ranking outside their ordinary content consumption habits and for opting out completely from any content curation;

18.   Notes the potential negative impact of personalised advertising, in particular micro-targeted and behavioural advertising, and of assessment of individuals, especially minors, without their consent, by interfering in the private life of individuals, asking questions as to the collection and use of the data used to personalise advertising, and offering products or services or setting prices; calls, therefore, on the Commission to introduce strict limitations on targeted advertising based on the collection of personal data, starting by introducing a prohibition on cross-platform behavioural advertising, while not hurting SMEs; recalls that currently the ePrivacy Directive only allows targeted advertising subject to opt-in consent, otherwise making it illegal; calls on the Commission to prohibit the use of discriminatory practices for the provision of services or products;

19.   Underlines that what is illegal offline shall be illegal online; notes that AI tools have the potential and are already used to fight illegal content online, but strongly recalls ahead of the Digital Services Act expected for the end of this year that such tools must always respect fundamental rights, especially freedom of expression and information, and should not lead to a general monitoring obligation for the internet, or to the removal of legal material disseminated for education, journalistic, artistic or research purposes; stresses that algorithms should be used only as a flagging mechanism in content moderation, subject to human intervention, as AI is unable to reliably distinguish between legal, illegal and harmful content; notes that terms and conditions should always include community guidelines as well as an appeal procedure;

20.   Notes the benefits and risks of AI in terms of cybersecurity and its potential in combating cybercrime, and emphasises the need for any AI solutions to be resilient to cyberattacks while respecting EU fundamental rights, especially the protection of personal data and privacy; stresses the importance of monitoring the safe use of AI and the need for close collaboration between the public and private sectors to counter user vulnerabilities and the dangers arising in this connection; calls on the Commission to evaluate the need for better prevention in terms of cybersecurity and mitigation measures thereof;

21.   Stresses that next-generation digital infrastructure and internet coverage are of strategic significance for providing AI-powered education to European citizens; in light of the COVID-19 crisis, calls on the Commission to elaborate a strategy for a European 5G that ensures Europe’s strategic resilience and is not dependent on technology from states who do not share our values;

22.   Calls on the Commission and the Member States to support the use of AI in the area of digitalised cultural heritage.

INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

Key to symbols:

+   :   in favour

-   :   against

0   :   abstention

OPINION OF THE COMMITTEE ON THE INTERNAL MARKET AND CONSUMER PROTECTION  ( 6.7.2020 )

Rapporteur for opinion: Kim Van Sparrentak

The Committee on the Internal Market and Consumer Protection calls on the Committee on Culture and Education, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

A.   whereas artificial intelligence (AI) has the potential to offer solutions for day-to-day challenges of the education sector such as the personalisation of learning, monitoring learning difficulties, automation of subject-specific content/knowledge, providing better professional training, and supporting the transition to a digital society;

B.   whereas AI could have practical applications in terms of reducing the administrative work of educators and educational institutions, freeing up time for their core teaching and learning activities;

C.   whereas the application of AI in education raises concerns around the ethical use of data, learners’ rights, data access and protection of personal data, and therefore entails risks to fundamental rights such as the creation of stereotyped models of learners’ profiles and behaviour that could lead to discrimination or risks of doing harm by the scaling-up of bad pedagogical practices;

D.   whereas AI applications are omnipresent in the audiovisual sector, in particular on audiovisual content platforms;

1.   Notes that the Commission has proposed to support public procurement in intelligent digital services, in order to encourage public authorities to rapidly deploy products and services that rely on AI in areas of public interest and the public sector; highlights the importance of public investment in these services and the complementary added value provided by public-private partnerships in order to secure this objective and deploy the full potential of AI in the education, culture and audiovisual sectors; emphasises that in the education sector, the development and deployment of AI should involve all those participating in the educational process and wider society and take into account their needs and the expected benefits, especially for the most vulnerable and disadvantaged, in order to ensure that AI is used purposefully and ethically and delivers real improvements for those concerned; considers that products and services developed with public funding should be published under open-source licences with full respect for the applicable legislation, including Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market; stresses the importance of this deployment for reskilling and upskilling the European labour market, and particularly in the culture and audiovisual sectors, which will be severely impacted by the COVID-19 crisis;

2.   Recognises that children are an especially vulnerable group in terms of influencing their behaviour; stresses that while AI can be a tool that can benefit their education, it is necessary to take into account the technological, regulatory and social aspects of the introduction of AI in education, with adequate safeguards and a human-centric approach that ensures that human beings are, ultimately, always able to control and correct the system’s decisions; points to the need for a review and updating of the relevant sectoral rules; underlines in this regard that the legal framework governing AI in the education sector should, in particular, provide for legally binding measures and standards to prevent practices that would undermine fundamental rights and freedoms, and ensure the development of trustworthy, ethical and technically robust AI applications, including integrated digital tools, services and products such as robotics and machine learning;

3.   Notes the potential of AI-based products in education, especially in making high-quality education available to all pupils in the EU; stresses the need for governments and educational institutions to rethink and rework educational programmes with a stronger emphasis on STEAM subjects, in order to prepare learners and consumers for the increasing presence of AI and to facilitate the acquisition of cognitive skills; underlines the need to improve the digital skills of those participating in the educational process and wider society, while having regard to the objectives of ‘A Europe fit for the digital age’;

4.   Underlines that algorithmic systems can be an enabler for reducing the digital divide in an accelerated way, but unequal deployment risks creating new divides or accelerating the deepening of the existing ones; expresses its concern that knowledge and infrastructure are not developed in a consistent way across the EU, which limits the accessibility of products and services that rely on AI, in particular in sparsely populated and socio-economically vulnerable areas; calls on the Commission to ensure cohesion in the sharing of the benefits of AI and related technologies;

5.   Calls on the Commission to consider education as a sector where significant risks can be expected to occur from certain uses of AI applications, which may potentially undermine fundamental rights and result in high costs in both human and social terms, and to take this consideration into account when assessing what types or uses of AI applications would be covered by a regulatory framework for high-risk AI applications, given the importance of ensuring that education continues to contribute to the public good and given the high sensitivity of data on pupils, students and other learners; calls on the Commission to include certain AI applications in the education sector, such as those that are subject to certification schemes or include sensitive personal data, in the regulatory framework for high-risk AI applications; underlines that data sets used to train AI and the outputs should be reviewed in order to avoid all forms of stereotypes, discrimination and biases, and where appropriate, make use of AI to identify and correct human biases where they might exist; points out, accordingly, that appropriate conformity assessments are needed in order to verify and ensure that all the provisions concerning high-risk applications are complied with, including testing, inspection and certification requirements; stresses the importance of securing the integrity and the quality of the data;

6.   Welcomes the efforts of the Commission to include digital skills as part of the qualification requirements for certain professions harmonised at EU level under the Professional Qualifications Directive; highlights the need to ensure mutual recognition of professional qualifications in AI skills across the EU, as several Member States are upgrading their educational offer with AI-related skills and putting in place specific curricula for AI developers; stresses the need for these to be in line with the assessment list of the Ethical Guidelines for Trustworthy AI, and welcomes the Commission’s proposal to transform this list into an indicative curriculum for AI developers; underlines the importance of training highly skilled professionals in this area, including ethical aspects in their curricula, and supporting underrepresented groups in the field, as well as creating incentives for those professionals to seek work within the EU;

7.   Takes note that schools and other public education providers are increasingly using educational technology services, including AI applications; expresses its concern that these technologies are currently provided by just a few technology companies; stresses that this may lead to unequal access to data and limit competition by market dominance and restricting consumer choice; encourages public authorities to take an innovative approach towards public procurement, so as to broaden the range of offers that are made to public education providers across Europe; stresses in this regard the importance of supporting the uptake of AI by SMEs in the education, culture and audiovisual sector through the appropriate incentives that create a level playing field; calls, in this context, for investment in European IT companies in order to develop the necessary technologies within the EU; considers that technologies used by public education providers or purchased with public money should be based on open-source technology where possible, while having full respect for the applicable legislation, including Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market;

8.   Calls for the data used by AI applications in the education sector to be accessible, interoperable and of high quality, and to be shared with the relevant public authorities in a standardised way and with respect for copyright and trade secrets legislation, so that the data can be used, in accordance with the European data protection and privacy rules and ethical, democratic and transparency standards, in the development of curricula and pedagogical practices (in particular when these services are purchased with public money or offered to public education providers for free, considering that education is a common good); calls on the Commission to ensure fair access to data for all companies, and in particular SMEs and cultural and creative companies, which play an essential role in sustaining social cohesion and cultural diversity in Europe, as well as democratic values;

9.   Stresses the importance of developing guidelines for the public procurement of such services and applications for the public sector, including for education providers, in order to ensure the relevant educational objectives, consumer choice, a level and fair playing field for AI solution providers and respect for fundamental rights; stresses the need for public buyers to take into account specific criteria linked to the relevant educational objectives, such as non-discrimination, fundamental rights, diversity, the highest standards of privacy and data protection, accessibility for learners with special needs, environmental sustainability and, specifically when purchasing services for public education providers, the involvement of all those participating in the educational process; stresses the need to strengthen the market by providing SMEs with the opportunity to participate in the procurement of AI applications in order to ensure the involvement of technology companies of all sizes in the sector and thus guarantee resilience and competition;

10.   Underlines the unreliability of the current automated means of removing illegal content from online platforms on which audiovisual content is shared, which may lead to inadvertent removal of legitimate content; notes that neither the E-Commerce Directive nor the revised Audiovisual Media Services Directive on video sharing platforms imposes a general monitoring obligation; recalls, to that end, that there should be no general monitoring, as stipulated in Article 15 of the E-Commerce Directive, and that specific content monitoring for audiovisual services should be in accordance with the exceptions laid down in the European legislation; recalls the key requirements for AI applications, such as accountability, including review structures within business processes, and reporting of negative impacts; emphasises that transparency should also include traceability and explainability of the relevant systems; recalls that AI applications must adhere to internal and external safety protocols, which should be technically accurate and robust in nature; considers that this should extend to operation in normal, unknown and unpredictable situations alike;

11.   Calls for recommendation algorithms and personalised marketing on audiovisual platforms, including video streaming platforms, news platforms and platforms disseminating cultural and creative content, to be explainable, to the extent technically possible, in order to give consumers an accurate and comprehensive insight into these processes and content and ensure that personalised services are not discriminatory and are in line with the recently adopted Platform to Business Regulation and New Deal for Consumers Omnibus Directive; stresses the need to guarantee and properly implement the right of users to opt out from recommended and personalised services; points out in this regard that a description should be provided to users that allows for a general and adequate understanding of the functions concerned, notably on the data used, the purpose of the algorithm, and personalisation and its outcomes, following the principles of explainability and fairness; calls for the development of mechanisms providing monitoring of the consumer’s rights of informed consent and freedom of choice when submitting data;

12.   Notes that the deployment of AI in customs screening procedures may support efforts to prevent the illicit trafficking of cultural heritage, in particular to supplement systems which allow customs authorities to target their efforts and resources on those items presenting the highest risk;

13.   Underlines that consumers must be informed when they are interacting with an automated decision process and that their choices and performance must not be limited; stresses that the use of AI mechanisms for commercial surveillance of consumers must be countered, even if it concerns ‘free services’, by ensuring that it is strictly in line with fundamental rights and the GDPR; stresses that all regulatory changes must take in consideration the impact on vulnerable consumers;

14.   Points out that the deployment, development and implementation of AI must make it easier for consumers and learners with some form of disability to use tools to access audiovisual content;

15.   Underlines the need for upskilling of the future workforce; recognises the benefits of forecasting which jobs will be disrupted by digital technology such as automation, digitalisation and AI;

16.   Points out that the AI systems that are developed, implemented and used in the European Union, in any of the three sectors referred to in this report, must reflect the EU’s cultural diversity and multilingualism.

OPINION OF THE COMMITTEE ON LEGAL AFFAIRS  ( 22.9.2020 )

Rapporteur for opinion: Angel Dzhambazki

The Committee on Legal Affairs calls on the Committee on Culture and Education, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

1.   Underlines the strategic importance of using artificial intelligence (AI) and related technologies, and stresses that the European approach in this regard must be human-centred so that AI genuinely becomes an instrument in the service of people and the common good, contributing to the general interest of citizens, including in the audiovisual, cultural and educational sectors; points out that AI can support content creation in those sectors; stresses that AI can support content creation in the education, culture and audiovisual sectors, alongside information and educational platforms, including listings of different kinds of cultural objects and a multitude of data sources; notes the risks of infringement of intellectual property rights (IPRs) when blending AI and different technologies with a multiplicity of sources (documents, photos, films) to improve the way those data are displayed, researched and visualised; calls for the use of AI to ensure a high level of IPR protection within the current legislative framework, for example by alerting individuals and businesses if they are in danger of inadvertently infringing the rules or assisting IPR rightholders if the rules are actually infringed; emphasises, therefore, the importance of having an appropriate European legal framework for the protection of IPRs in connection with the use of AI;

2.   Highlights that the consistent integration of AI in the education sector has the potential to meet some of the biggest challenges of education, to come up with innovative teaching and learning practices, and finally, to accelerate progress towards achieving the Sustainable Development Goals in order to meet the targets of the 2030 Agenda for Education;

3.   Reiterates the importance of access to culture for every citizen throughout the Union; highlights in this context the importance of the exchange of best practices among Member States, educational facilities and cultural institutions and similar stakeholders; further considers it of vital importance that the resources available at both EU and national level are used to the maximum of their potential in order to further improve access to culture; stresses that there are a multitude of options to access culture and that all varieties should be explored in order to determine the most appropriate option; highlights the importance of consistency with the Marrakech Treaty;

4.   Calls on the Commission to realise the full potential of artificial intelligence (AI) for the purposes of improving communication with citizens, through cultural and audiovisual online platforms, for example by keeping citizens informed of what is happening at decision-making level, narrowing the gap between the EU and the grassroots, and promoting social cohesion between EU citizens;

5.   Highlights that education, culture and the audiovisual sector are sensitive areas for the use of AI and related technologies since they have the potential to impact our societies and the fundamental rights they uphold; contends, therefore, that legally binding ethical principles should be observed in their deployment, development and use;

6.   Notes how artificial intelligence and related technologies may be used in developing or applying new methods of education in areas including language learning, academia generally, specialised learning, etc; highlights the importance not only of using such technologies for educational purposes but also of digital literacy and public awareness of the former; stresses the importance of providing educators, trainers and others with the right tools and know-how with regard to AI and related technologies in terms of what they are, how they are used and how to use them as tools properly and according to the law, so as to avoid IPR infringements; highlights in particular the importance of digital literacy for staff working in education, as well as of improving digital training for the elderly, considering that newer generations already have a basic notion of these technologies, having grown up with them;

7.   Emphasises that European artificial intelligence should safeguard and promote core values of our Union such as democracy, independent and free media and information sources, quality education, environmental sustainability, gender balance and cultural and linguistic diversity;

8.   Calls on the Commission, the Member States and the business community to actively and fully exploit the potential of AI in providing the facts and combating fake news, disinformation, xenophobia and racism on cultural and audiovisual online platforms, while at the same time avoiding censorship;

9.   Notes that the independence of the creative process raises issues related to ownership of IPRs; considers, in this connection, that it would not be appropriate to seek to impart legal personality to AI technologies;

10.   Notes that AI can play an important role in promoting and protecting our European and national cultural diversity, especially when used by audiovisual online platforms in promoting content to customers;

11.   Notes that AI could benefit the research sector, for example through the role that predictive analytics can play in fine-tuning data analysis, for example on the acquisition and movement of cultural objects; stresses that the EU must step up investment and foster partnerships between industry and academia in order to enhance research excellence at European level;

12.   Notes the important role which independent media play in culture and the daily life of citizens; stresses that fake media represent a fundamental problem, as copyright and IPRs generally are being constantly infringed; calls on the Commission, in cooperation with the Member States, to continue its work on raising awareness of this problem, countering the effects of fake media as well as the source problems; considers it of importance, furthermore, to develop educational strategies to improve digital literacy specifically in this regard;

13.   Notes that AI-based software, such as image recognition software, could vastly enhance the ability of educational facilities and teachers to provide and develop modern, innovative and high-quality schooling methods, improve the digital literacy and e-skills of the entire population, and enable education to be more accessible; considers that such schooling methods should nevertheless be assessed as to their reliability and accuracy and should ensure fairness in education, non-discrimination, and the safety of children and minors both within educational facilities and when connected remotely within an educational context; highlights the importance of privacy and data protection legislation in order to ensure adequate protection of personal data, in particular children’s data, through transparent and reliable data sources respectful of IPRs; considers it vital that these technologies are only integrated into the existing systems if the protection of fundamental rights and privacy is an absolute given; stresses, however, that recognition software must be used only for educational purposes and not under any circumstances to monitor access to establishments; highlights in this regard the dependence on external data and a few market-dominating software providers; recalls that technologies procured with public money should be developed as open source software to enable the sharing and reuse of resources, making them available throughout the EU and thus increasing benefits and reducing public spending, while ensuring full respect for the applicable legislation, including Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market [20] ;

14.   Notes that if the use of AI is to benefit the education and research sector, the EU must encourage training in the skills of the future, and in particular an ethical and responsible approach to AI technologies; adds, with that aim in view, that this training must not be reserved for pupils focusing on scientific and technical subjects, who are already more familiar with these tools, but must instead target as many people as possible, in particular in the younger generations;

15.   Stresses that the need for investment in research and innovation regarding the use and development of AI and its cultural, educational and audiovisual applications is a key consideration in this respect; calls on the Commission to find additional funding to promote research and innovation regarding AI applications in these sectors;

16.   Expresses serious concern that schools and other education providers are becoming increasingly dependent on educational technology services, including AI applications, provided by companies with a dominant market position, most of which are based outside the EU;

17.   Underlines the need to ensure EU-wide digital and AI literacy, especially through the development of training opportunities for teachers; insists that the use of AI technologies in schools should contribute to narrowing the social and regional digital gap;

18.   Highlights that the COVID-19 pandemic crisis can be considered as a probation period for the development and use of digital and AI-related technologies in the educational and cultural sectors, as exemplified by the many online schooling platforms and online tools for cultural promotion employed across the Member States; calls on the Commission, therefore, to take stock of those examples when considering a common EU approach to the increased use of such technological solutions;

19.   Points out that data protection and privacy can be particularly seriously affected by AI; advocates compliance with the principles laid down in the General Data Protection Regulation (GDPR);

20.   Calls on the Commission to take more effective steps to protect the personal data of pupils and teachers in the education sphere;

21.   Emphasises that the interaction between AI and the creative industries is complex and requires an in‑depth assessment; welcomes the ongoing study ‘Trends and Developments in Artificial Intelligence - Challenges to the IPRS Framework’ and the study on ‘Copyright and new technologies: copyright data management and Artificial Intelligence’; underlines the importance of clarifying the conditions of use of copyright‑protected content as data input (images, music, films, databases, etc) and in the production of cultural and audiovisual outputs, whether created by humans with the assistance of AI or autonomously generated by AI technologies; invites the Commission to study the impact of AI on the European creative industries; reiterates the importance of European data and welcomes the statements made by the Commission in this regard, as well as the placing of artificial intelligence and related technologies high on the agenda;

22.   Emphasises the role of an author’s personality for the expression of free and creative choices that constitute the originality of works [21] ; underlines the importance of limitations and exceptions to copyright when using content as data input, notably in education, academia and research, and in the production of cultural and audiovisual outputs, including user-generated content;

23.   Emphasises that the interaction between AI and the creative industries is complex and requires an in-depth assessment; takes the view that consideration should be given to protecting AI-generated technical and artistic creations, in order to encourage this form of creativity;

24.   Stresses that in the data economy context, better copyright data management is achievable, for the purpose of better remunerating authors and performers, notably in enabling the swift identification of the authorship and right ownership of content, thus contributing to lowering the number of orphan works; further highlights that AI technological solutions should be used to improve copyright data infrastructure and the interconnection of metadata in works, but also to facilitate the transparency obligation provided in Article 19 of the Directive on Copyright and related rights in the Digital Single Market for up‑to‑date, relevant and comprehensive information on the exploitation of authors’ and performers’ works and performances, particularly in the presence of a plurality of rightholders and of complex licensing schemes;

25.   Stresses the need to work on the most efficient way of reducing bias in AI systems, in line with ethical and non-discrimination standards; underlines that data sets used to train AI should be as broad as possible in order to represent society in the best relevant way, that the outputs should be reviewed to avoid all forms of stereotypes, discrimination and biases, and, when appropriate, AI should be made use of to identify and correct human biases where they could exist; calls on the Commission to encourage and facilitate the sharing of de-biasing strategies for data;

26.   Asks the Commission to assess the impact of AI and AI-related technologies on the audiovisual and creative sector, in particular with regard to authorship and related questions;

27.   Calls for the intellectual property action plan announced by the Commission to address the question of AI and its impact on the creative sectors, taking account of the need to strike a balance between protecting IPRs and encouraging creativity in the areas of education, culture and research; considers that the EU can be a leader in the creation of AI technologies if it adopts an operational regulatory framework and implements proactive public policies, particularly as regards training programmes and financial support for research; asks the Commission to assess the impact of IPRs on the research and development of AI and related technologies, as well as on the audiovisual and creative sectors, in particular with regard to authorship, fair remuneration of authors and related questions;

28.   Highlights the future role that the inclusion of AI-based technological tools should have in terms of conservation, disclosure and heritage control, as also in the associated research projects;

29.   Stresses the need to strike a balance between, on the one hand, the development of AI systems and their use in the educational, cultural and audiovisual sectors and, on the other, measures to safeguard competition and market competitiveness for AI companies in these sectors; emphasises in this regard the need to encourage companies to invest in the innovation of AI systems used in these sectors, while at the same time ensuring that those providing such applications do not obtain a market monopoly;

30.   Stresses that in no scenario could the use of AI and related technologies become a reality without human oversight; reiterates the importance of fundamental rights and the overarching supremacy of data and privacy protection legislation, which is imperative when dealing with such technologies;

31.   Asks the Commission to assess the impact of AI and AI-related technologies in creating new audiovisual works such as deep fakes, and to establish appropriate legal consequences to be attached to their creation, production or distribution for malicious purposes;

32.   Notes that automation and the development of AI could pose a threat to employment, and emphasises once again that priority must be given to safeguarding jobs, in particular in the education, culture and creative sectors;

33.   Calls on the Commission to launch an EU-level education plan on digital and AI literacy, in coordination with the Member States, with a particular focus on school students and youth;

34.   Calls on the Commission to consider the legal aspects of the outputs produced using AI technologies, as well as cultural content generated with the use of AI and related technologies; considers it important to support the production of cultural content; reiterates, however, the importance of safeguarding the Union’s unique IPR framework and that any changes should be made with the necessary due care, in order not to disrupt the delicate balance; calls on the Commission to produce an in-depth assessment with regard to the possible legal personality of AI-produced content, as well as the application of IPRs to AI-generated content and to content created with the use of AI tools;

35.   Calls on the Commission to establish requirements for the procurement and deployment of artificial intelligence and related technologies by EU public sector bodies, to ensure compliance with Union law and fundamental rights; highlights the added value of instruments such as public consultations and impact assessments, to be run prior to the procurement or deployment of artificial intelligence systems, as recommended in the report of the Special Rapporteur to the UN General Assembly on AI and its impact on freedom of opinion and expression [22] ;

36.   Calls on the Commission to lay down rules designed to guarantee effective data interoperability, in order to make content purchased on a platform accessible via any digital tool irrespective of brand;

37.   Emphasises that the challenges brought by the use of artificial intelligence and related technologies can only be overcome by establishing data quality obligations and transparency and oversight requirements, in order to enable the public and authorities to assess compliance with Union law and fundamental rights; awaits the Commission’s proposals following its communication on a European strategy for data [23] as regards the sharing and pooling of datasets;

38.   Calls on the Commission, in addition, to consider developing, in very close cooperation with Member States and the relevant stakeholders, verification mechanisms or systems for publishers, authors, creators, etc, in order to assist them in verifying what content they may use and to more easily determine what is protected under IPR legislation.

OPINION OF THE COMMITTEE ON WOMEN'S RIGHTS AND GENDER EQUALITY  ( 14.9.2020 )

Rapporteur for opinion: Maria da Graça Carvalho

The Committee on Women’s Rights and Gender Equality calls on the Committee on Culture and Education, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

A.   whereas gender equality is a core principle of the European Union enshrined in the Treaties, and should be reflected in all EU policies, including in education, culture, and the audiovisual sector, as well as in the development of technologies such as Artificial Intelligence (AI), these being key channels for changing attitudes and challenging stereotypes and gender biases in existing social norms; whereas the development of digitalisation and technologies like AI are fundamentally transforming our reality and their regulation today will highly influence our future societies; whereas there is a need to advocate for a human-centred approach anchored in human rights and ethics for the development and use of AI;

B.   whereas Article 21 of the EU Charter of Fundamental Rights prohibits discrimination on a wide range of grounds and should be a guiding principle; whereas multiple forms of discrimination should not be reproduced in the design, input, development and use of AI systems based on gender-biased algorithms, or in the social contexts in which such algorithms are used;

C.   whereas past experiences, especially in technical fields, have shown us that developments and innovations are often based mainly on male data and that women’s needs are not fully reflected; whereas addressing these biases requires greater vigilance, technical solutions and the development of clear requirements of fairness, accountability and transparency;

D.   whereas incomplete and inaccurate data sets, the lack of gender-disaggregated data and incorrect algorithms can distort the processing of an AI system and jeopardise the achievement of gender equality in society; whereas data on disadvantaged groups and intersectional forms of discrimination tend to be incomplete and even absent;

E.   whereas gender inequalities, stereotypes and discrimination can also be created and replicated through the language and images disseminated by the media and AI-powered applications; whereas education, cultural programmes and audiovisual content have considerable influence in shaping people’s beliefs and values and are a fundamental tool for combatting gender stereotypes, decreasing the digital gender gap, and establishing strong role models; whereas an ethical and regulatory framework must be in place ahead of implementing automatised solutions for these key areas in society;

F.   whereas science and innovation can bring life-changing benefits, especially for those who are furthest behind, such as women and girls living in remote areas; whereas scientific education is important for obtaining skills, decent work, and jobs of the future, as well as for breaking with gender stereotypes that regard these as stereotypically masculine fields; whereas science and scientific thinking are key to democratic culture, which in turn is fundamental for advancing gender equality;

G.   whereas women are significantly under-represented in the AI sector, whether as creators, developers or consumers; whereas the full potential of women’s skills, knowledge and qualifications in the digital and AI fields as well as that of information, communication and technology (ICT), along with their reskilling, can contribute to boosting the European economy; whereas globally only 22 % of AI professionals are female; whereas the lack of women in AI development not only increases the risk of bias, but also deprives the EU of diversity, talent, vision and resources, and is therefore an obstacle to innovation; whereas gender diversity enhances female attitudes in teams and team performance and favours the potential for innovation in both public and private sectors;

H.   whereas in the EU one woman in ten has already suffered some form of cyberviolence since the age of 15 and cyberharassment remains a concern in the development of AI, including in education; whereas cyberviolence is often directed at women in public life, such as activists, women politicians and other public figures; whereas AI and other emerging technologies can play an important role in preventing cyberviolence against women and girls and educating people;

I.   whereas the EU is facing an unparalleled shortage of women in Science, Technology, Engineering and Mathematics (STEM) careers and education, given that women account for 52 % of the European population, yet only for one in three of STEM graduates;

J.   whereas despite the positive trend in the involvement and interest of women in STEM education, the percentages remain insufficient, especially considering the importance of STEM-related careers in an increasingly digitalised world;

1.   Considers that AI has great potential to promote gender equality provided that already existing conscious and unconscious bias are eliminated; stresses the need for further regulatory efforts to ensure that AI respects the principles and values of gender equality and non-discrimination as enshrined in Article 21 of the Charter of Fundamental Rights; stresses, further, the importance of accountability, of a differentiated and transparent risk-based approach, and of continuous monitoring of existing and new algorithms and of their results;

2.   Stresses the need for media organisations to be informed about the main parameters of algorithm-based AI systems that determine ranking and search results on third-party platforms, and for users to be informed about the use of AI in decision-making services and empowered to set their privacy parameters via transparent and understandable measures;

3.   Recalls that algorithms and AI should be ‘ethical by design’, with no built-in bias, in a way that guarantees maximum protection of fundamental rights;

4.   Calls for policies targeted at increasing the participation of women in the fields related to STEM, AI and the research and innovation sector, and for the adoption of a multi-level approach to address the gender gap at all levels of education, with particular emphasis on primary education, as well as employment in the digital sector, highlighting the importance of upskilling and reskilling;

5.   Recognises that gender stereotyping, cultural discouragement and the lack of awareness and promotion of female role models hinders and negatively affects girls’ and women’s opportunities in ICT, STEM and AI and leads to discrimination and fewer opportunities for women in the labour market; stresses the importance of increasing the number of women in these sectors, which will contribute to women’s participation and economic empowerment, as well as to reducing the risks associated with the creation of so-called ‘biased algorithms’;

6.   Encourages the Commission and the Member States to purchase educational, cultural and audiovisual services from providers that apply gender balance in their workplace, promote public procurement policies and guidelines that stimulate companies to hire more women for STEM jobs, and facilitate the distribution of funds to companies in the educational, cultural and audiovisual sectors that take account of gender balance criteria;

7.   Emphasises the cross-sectoral nature of gender-based discrimination rooted in conscious or unconscious gender bias and manifested in the education sector, the portrayal of women in the media and advertising on-screen and off-screen, and the responsibility of both public and private sectors in terms of proactively recruiting, developing and retaining female talent and instilling an inclusive business culture;

8.   Calls on the Commission and the Member States to take into account ethical aspects, including from a gender perspective, when developing AI policy and legislation, and, if necessary, to adapt the current legislation, also including EU programmes and ethical guidelines for the use of AI;

9.   Encourages the Member States to enact a strategy to promote women’s participation in STEM, ICT and AI-related studies and careers in relevant existing national strategies to achieve gender equality, defining a target for the participation of women researchers in STEM and AI projects; urges the Commission to address the gender gap in STEM, ICT and AI-related careers and education, and to set this as a priority of the Digital Skills Package in order to promote the presence of women at all levels of education, as well as in the upskilling and reskilling of the labour force;

10.   Recognises that producers of AI solutions must make a greater effort to test products thoroughly in order to anticipate potential errors impacting vulnerable groups; calls for work to be stepped up on a tool to teach algorithms to recognise disturbing human behaviour, which would identify those elements that most frequently contribute to discriminatory mechanisms in the automated decision-making processes of algorithms;

11.   Underlines the importance of ensuring that the interests of women experiencing multiple forms of discrimination and who belong to marginalised and vulnerable groups are adequately taken into account and represented in any future regulatory framework; notes with concern that marginalised groups risk suffering new technological, economic and social divides with the development of AI;

12.   Calls for specific measures and legislation to combat cyberviolence; stresses that the Commission and the Member States should provide appropriate funding for the development of AI solutions that prevent and fight cyberviolence and online sexual harassment and exploitation directed against women and girls and help educate young people; calls for the development and implementation of effective measures tackling old and new forms of online harassment for victims in the workplace;

13.   Notes that for the purpose of analysing the impacts of algorithmic systems on citizens, access to data should be extended to appropriate parties, notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; points out that users must always be informed when an algorithm has been used to make a decision concerning them, particularly where the decision relates to access to benefits or to a product;

14.   Calls on the Commission and the Member States to devise measures that fully incorporate the gender dimension, such as awareness-raising campaigns, training and curricula, which should provide information to citizens on how algorithms operate and their impact on their daily lives; further calls on them to nurture gender-equal mindsets and working conditions that lead to the development of more inclusive technology products and work environments; urges the Commission and the Member States to ensure the inclusion of digital skills and AI training in school curricula and to make them accessible to all, as a way to close the digital gender divide; 

15.   Stresses the need for training for workers and educators dealing with AI to promote the ability to identify and correct gender-discriminatory practices in the workplace and in education, and for workers developing AI systems and applications to identify and remedy gender-based discrimination in the AI systems and applications they develop; calls for the establishment of clear responsibilities in companies and educational institutions to ensure that there is no gender-based discrimination in the workplace or educational context; highlights that genderless images of AI and robots should be used for educational and cultural purposes, unless gender is a key factor for some reason;

16.   Highlights the importance of the development and deployment of AI applications in the educational, cultural and audiovisual sectors in collecting gender-disaggregated and other equality data, and of applying modern machine learning de-biasing techniques, if needed, to correct gender stereotypes and gender biases which may have negative impacts; 

17.   Urges the Commission and the Member States to collect gender-disaggregated data in order to feed datasets in a way that promotes equality; also calls on them to measure the impact of the public policies put in place to incorporate the gender dimension by analysing the data collected; stresses the importance of using complete, reliable, timely, unbiased, non-discriminatory and gender-sensitive data in the development of AI;

18.   Calls on the Commission to include education in the regulatory framework for high-risk AI applications, given the importance of ensuring that education continues to contribute to the public good, as well as the high sensitivity of data on pupils, students and other learners; emphasises that in the education sector, this deployment should involve educators, learners and the wider society and should take into account the needs of all and the expected benefits in order to ensure that AI is used purposefully and ethically;

19.   Calls on the Commission to encourage the use of EU programmes such as Horizon Europe, Digital Europe and Erasmus+ to promote multidisciplinary research, pilot projects, experiments and the development of tools including training, for the identification of gender biases in AI, as well as awareness-raising campaigns for the general public;

20.   Stresses the need to create diverse teams of developers and engineers to work alongside the main actors in the educational, cultural and audiovisual sectors in order to prevent gender or social bias being inadvertently included in AI algorithms, systems and applications; stresses the need to consider the variety of different theories through which AI has been developed to date and could be further advanced in the future;

21.   Points out that the fact of taking due care to eliminate bias and discrimination against particular groups, including gender stereotypes, should not halt technological progress.

• [1] OJ C 202 I, 16.6.2020, p. 1.
• [2] OJ C 440, 6.12.2018, p. 37.
• [3] OJ C 449, 23.12.2020, p. 37.
• [4] OJ C 433, 23.12.2019, p. 42.
• [5] OJ C 28, 27.1.2020, p. 8.
• [6] OJ C 252, 18.7.2018, p. 239.
• [7] OJ C 307, 30.8.2018, p. 163.
• [8] Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (OJ L 119, 4.5.2016, p. 1).
• [9] Directive 2002/58/EC of the European Parliament and of the Council of 12 July 2002 concerning the processing of personal data and the protection of privacy in the electronic communications sector (Directive on privacy and electronic communications) (OJ L 201, 31.7.2002, p. 37).
• [10] Report of the UN Special Rapporteur on the promotion and protection of the right to freedom of opinion and expression, 29 August 2018.
• [11] Directive 2005/36/EC of the European Parliament and of the Council of 7 September 2005 on the recognition of professional qualifications (OJ L 255, 30.9.2005, p. 22).
• [12] Court of Justice of the European Union, Case C-833/18, SI and Brompton Bicycle Ltd v Chedech / Get2Get.
• [13] Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market and amending Directives 96/9/EC and 2001/29/EC (OJ L 130, 17.5.2019, p. 92).
• [14] Regulation (EU) 2019/1150 of the European Parliament and of the Council of 20 June 2019 on promoting fairness and transparency for business users of online intermediation services (OJ L 186, 11.7.2019, p. 57).
• [15] Directive (EU) 2019/2161 of the European Parliament and of the Council of 27 November 2019 amending Council Directive 93/13/EEC and Directives 98/6/EC, 2005/29/EC and 2011/83/EU of the European Parliament and of the Council as regards the better enforcement and modernisation of Union consumer protection rules (OJ L 328, 18.12.2019, p. 7).
• [16] Directive 2000/31/EC of the European Parliament and of the Council of 8 June 2000 on certain legal aspects of information society services, in particular electronic commerce, in the Internal Market (OJ L 178, 17.7.2000, p. 1).
• [17] Directive (EU) 2018/1808 of the European Parliament and of the Council of 14 November 2018 amending Directive 2010/13/EU on the coordination of certain provisions laid down by law, regulation or administrative action in Member States concerning the provision of audiovisual media services in view of changing market realities (OJ L 303, 28.11.2018, p. 69).
• [18] Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (OJ L 119, 4.5.2016, p. 1).
• [19] Directive 2002/58/EC of the European Parliament and of the Council of 12 July 2002 concerning the processing of personal data and the protection of privacy in the electronic communications sector (Directive on privacy and electronic communications) (OJ L 201, 31.7.2002, p. 37).
• [20] OJ L 130, 17.5.2019, p. 92.
• [21] Court of Justice of the European Union, SI and Brompton Bicycle Ltd v Chedech Get2Get, Case C-833/18.
• [22] Report of the Special Rapporteur on the promotion and protection of the right to freedom of opinion and expression, https://undocs.org/A/73/348.
• [23] COM(2020) 66 final, https://ec.europa.eu/info/sites/info/files/communication-european-strategy-data-19feb2020_en.pdf.

• News & Events

AI in Education: Considering Ethics and Power

Difficult ethical and moral questions will play a central role in whether artificial intelligence will expand opportunities and equity in STEM education or make things worse, Northwestern University learning scientist  Sepehr Vakil said during his closing keynote at a National Science Foundation convening of principal investigators in Arlington, Virgina.

“Researchers and classroom educators must also be involved and engaged in the evolution of AI in learning and schools,” Vakil said. 

Vakil, whose talk during the  2024  EDU Core Research meeting addressed the future of AI in STEM education, also participated in a Q & A with Yolanda Rankin, associate professor of computer science at Emory University. Rankin is considered  one  of the leading researchers working on issues of computing education and equity.

Vakil directs the  Technology, Race, Ethics, and Equity in Education (TREE) Lab at Northwestern’s School of Education and Social Policy. An associate professor, he  also was  co-lead  of  the Spencer Foundation’s research convening on Equity and AI in Education,  serves  as a senior advisor to Expanding Computing Education Pathways in Illinois and  is serving on a  National Academies of Sciences, Engineering, and Medicine’s Committee on  Computing and Data Science in K12  Education.

“AI is unique in the kinds of moral and ethical questions that it raises about complex issues like the environment, warfare and surveillance, to name a few,” he said. “AI ethics in STEM education cannot be viewed as separate from these broader ethical and moral questions. ”

“Our field must engage with the political economy of AI, asking always who is controlling the tech, the data, and to whose benefit?” Vakil added. “How might the National Science Foundation, and those of us who have NSF-funded projects, intervene and shift this ecosystem?”

• Read Vakil's bio.
• Read his full talk on  Substack.

Summary Artificial Intelligence 2024 Legislation

Artificial intelligence—the use of computer systems to perform tasks that normally require human intelligence, such as learning and decision making—has the potential to spur innovation and transform industry and government. As AI advances, more products and services are coming onto the market. For example, companies are developing AI to help consumers run their household appliances and allow the elderly to stay in their homes longer. AI is used in health care, self-driving cars, digital assistants and many other areas of daily life.

Concerns about potential misuse or unintended consequences of AI, however, have prompted efforts to develop standards. The National Institute of Standards and Technology, for example, is holding discussions with the public and private sectors to develop federal standards for the creation of reliable, robust and trustworthy AI systems.

In the 2024 legislative session, at least 40 states, Puerto Rico, the Virgin Islands and Washington, D.C., introduced AI bills, and six states, Puerto Rico and the Virgin Islands adopted resolutions or enacted legislation. Examples of those actions include:

• Indiana created an AI task force.
• South Dakota revised its statutes to clarify that a person is guilty of possessing child pornography if the person knowingly possesses any visual depiction of a minor engaging in a prohibited sexual act, or in a simulation of a prohibited sexual act, or any computer-generated child pornography. A violation of the revised law is a Class 4 felony.
• Tennessee required the governing boards of public institutions of higher education to promulgate rules and required local education boards and public charter schools to adopt policies, regarding the use of AI by students, teachers, faculty and staff for instructional purposes.
• Utah created the Artificial Intelligence Policy Act.
• The Virgin Islands established a real-time, centralized crime data system within the state police department.
• West Virginia created a select committee on artificial intelligence.

This webpage covers key legislation related to AI issues generally. Legislation related solely to specific AI technologies, such as facial recognition, deepfakes or autonomous cars, is being tracked separately.