How to Identify AI Misinformation in Education

To varying degrees, artificial intelligence (AI) has become integrated into almost every industry: health care, information technology, financial services, media, arts and entertainment, manufacturing. However, few sectors have felt the impact of AI more than education, where it is reshaping instruction, assessment and learning, and transforming how students study and teachers teach.

Precedence Research forecasts that the global AI in education market will grow from about $7 billion in 2025 to more than $122 billion by 2034 — a surge of roughly 1,500 percent. Other key figures that underscore the expanding impact of AI in education include the following:

• According to a 2024 survey by the Digital Education Council, 86 percent of college students use AI tools in their studies, with nearly 1 in 4 saying they use it on a daily basis. The most common reason cited for AI use among students was to search for information (69 percent).
• The percentage of K-12 students familiar with AI rose from 37 percent in 2023 to 75 percent in 2024, while among teachers it jumped from 55 to 79 percent, according to a recent report from Impact Research for the Walton Family Foundation. Actual usage of AI tools among students grew by 27 percent, with nearly half of students — as well as teachers — saying they use it weekly.
• A separate survey by the Walton Family Foundation in 2023 showed that a large majority of students (65 percent) and teachers (71 percent) believe that OpenAI’s ChatGPT will become essential to students’ success in college and the workplace.

The introduction of AI technology into education presents enormous challenges, such as the specter of cheating, privacy and equity concerns, and the potential for exposure to AI misinformation. This latter risk involves AI tools providing convincing but false content that threatens students’ ability to distinguish fact from fiction and makes it harder for educators to evaluate their work.

As AI technology continues to rapidly evolve, educators are recognizing that they need to stay ahead of the curve and implement proactive strategies focused on fostering students’ AI literacy and identifying misuse and misinformation.

What Is AI Misinformation?

Misinformation is generally defined as false or misleading information, often caused by errors rather than intentional deception. Disinformation is considered a type of misinformation that is purposefully designed to deceive its audience.

AI misinformation can take any of numerous forms, such as inaccurate outputs — for example, when an AI chatbot fabricates statistics or sources, which are sometimes referred to as AI “hallucinations” — and fake images or audio, known as deepfakes.

While misinformation has been a concern since the invention of the printing press, AI tools have the potential to supercharge the spread of false or misleading content. The number of detected deepfakes grew fourfold between 2023 and 2024, according to a report from Sumsub, accounting for 7 percent of all fraud attempts globally.

And false AI content is getting more sophisticated and harder to identify. One study found that people detected AI-generated misinformation on the Russia-Ukraine war 59 percent of the time, suggesting they were deceived roughly 40 percent of the time, and that they were more likely to believe an AI-generated lie than one written by a human. As AI technology advances exponentially, these fabrications will likely become even more convincing and difficult to detect.

Consequently, AI falsehoods tend to spread more widely and get more engagement on social media, particularly on X, the platform formerly known as Twitter, than other types of misinformation. A recent analysis revealed that AI-crafted X posts containing misinformation received about 8 percent more impressions, 20 percent more reposts and nearly 50 percent more likes than posts containing conventional forms of misinformation.

AI technology has also proven to be susceptible to manipulation and hallucination. In a 2023 NewsGuard report, researchers fed ChatGPT 100 false narratives on various topics, including COVID-19, the Russia-Ukraine war and school shootings. In 80 instances, the AI chatbot amplified those fabrications, producing detailed but fictitious news articles and essays that reinforced the original false claims.

Examples of AI Misinformation

AI-generated misinformation doesn’t always make headlines. However, some cases are so widespread and/or damaging that they capture national and sometimes global attention.

Some notable recent examples of AI misinformation include:

• Election interference: In 2024, a deepfake audio message claiming to be from former President Joe Biden advised New Hampshire voters not to vote in the state’s primary, just days before the election.
• Fake news content: The Chicago Sun-Times and Philadelphia Inquirer published summer reading lists in 2025 that recommended several books that didn’t exist. The newspapers did not create the lists themselves, receiving them from a third-party content provider, which used AI tools to draft the content.
• Chatbot lies: Air Canada’s AI-powered virtual assistant misled a passenger about bereavement fares, advising him he could get a bereavement discount after purchasing his tickets. When he later sought a refund, the airline denied the claim, telling him he wasn’t eligible for a refund after the tickets were purchased. He successfully sued the airline.

How Does AI Misinformation Impact Education?

AI technology can benefit education in several ways: enhancing students’ learning, enriching classroom instruction and automating teachers’ administrative tasks like grading. However, if it’s used irresponsibly or without a proper understanding of how it works, AI technology can present serious problems, including by generating misinformation.

The possible impacts of AI misinformation in education are multifaceted, encompassing concerns regarding students’ learning, the reinforcement of existing biases and even the spread of bullying.

Can Negatively Affect Students’ Learning

Students are increasingly turning to AI to support their studies, with nearly 70 percent saying they primarily use it to search for information, according to the Digital Education Council.

While this can facilitate students’ research efforts, it can also open the door to them being exposed to misinformation, such as misleading accounts of historical events or fake statistics about public health issues, which can distort their understanding of important topics. Over time, this can undermine students’ academic development and erode their trust in credible sources.

Can Invent Fake Sources

AI tools can not only produce false information, they can also fabricate sources — including journal articles and news stories — to buttress their falsehoods, making them all the more convincing to undiscerning users.

A 2022 Scientific American article described a case where a user prompted ChatGPT to report fake studies on various topics. In one instance, the AI chatbot cited a study published in the Journal of the American Medical Association that claimed that the COVID-19 vaccine was only effective in 2 out of 100 people, yet neither the study nor that statistic existed.

More recently, a report on children’s health from the federal government’s Make America Healthy Again Commission was found to have cited a number of fake sources, including journal articles that don’t exist — suggesting that generative AI technology was used to help produce the report.

Can Perpetuate Biases

AI models are trained on existing datasets. If the data used to feed the AI algorithms is tainted by biases, its outputs can reflect and potentially amplify those biases, compromising students’ learning and the fairness of the AI’s assessments.

For example, a study in the journal Patterns found that AI chatbots incorrectly flagged work written by non-native English speakers as AI generated, which could lead to false accusations of cheating. The National Education Association (NEA) also warns that AI may perpetuate racial or gender biases, such as reinforcing stereotypes about “masculine” versus “feminine” jobs, which can skew students’ work and even limit their perceived opportunities.

Can Lead to Bullying

AI misinformation can also contribute to bullying and harassment, specifically through AI technology’s ability to generate deepfakes. Education Week has reported on cases in which male students have used AI tools to create pornographic images of their female classmates, which they then shared on social media. In another instance, an administrator created a fake audio clip of his school’s principal spouting racist remarks, resulting in the administrator’s dismissal and the relocation of the principal to another school.

These types of incidents can cause emotional distress to students and school employees and damage their reputations.

What Can Teachers Do to Combat AI Misinformation?

AI misinformation is a major concern for educators: 83 percent of faculty members are concerned about their students’ ability to critically evaluate AI outputs, according to a 2024 survey by the Digital Education Council.

Fortunately, teachers and other education professionals can adopt several strategies to both empower their students and more easily spot AI falsehoods when they appear.

Promote Responsible Use of AI Technology

Although students are rapidly adopting AI, many say they’re not confident in their ability to use the technology. In the Digital Education Council’s 2024 global AI student survey, 58 percent of students said they lacked sufficient AI knowledge and skills and nearly half felt underprepared for an increasingly AI-driven workforce.

Teachers can help students build their AI literacy through guided practice and critical evaluation exercises, helping them understand that AI chatbots can generate false or misleading information and even invent fake references and statistics. AI literacy education can also involve explaining to students how these systems work, including the ways the data used to train the AI models may perpetuate bias.

Encourage Fact-Checking

As part of their AI literacy curriculum, educators can also help students build their fact-checking skills, particularly when evaluating AI-generated content. Teaching students to step away from an AI chatbot or leave an unfamiliar website to verify any suspicious claims — especially those that seem exaggerated or lack backing from well-established outlets — can help them better recognize and avoid misinformation. Helping students hone their media literacy so they can discern a credible news source from a questionable one can be helpful as well.

Several online tools, such as FactCheck.org and the Google Fact Check Explorer, are easily accessible to many students.

Use AI Detection Tools

Even if they take proactive steps to help their students avoid AI misinformation, educators will inevitably encounter it in their students’ work. A recent survey by the Center for Democracy and Technology found that nearly 70 percent of middle and high school teachers use AI detection tools when assessing their students’ assignments.

Although these tools have varying degrees of accuracy, experts recommend using them as part of a broader AI literacy strategy. These tools can provide teachers with valuable insights that they can use to initiate conversations with their students about responsible AI technology use.

Resources for Identifying and Preventing AI Misinformation

While combating AI misinformation represents a formidable challenge, educators have a growing arsenal of tools at their disposal to help them avoid and detect AI falsehoods. In addition to resources like FactCheck.org and the Google Fact Check Explorer mentioned above, the following can help teachers and administrators combat AI misinformation in education:

• AI Guidance for Schools Toolkit: From TeachAI — a collaboration between nonprofits, government agencies, and experts in tech and education — this site provides guidance for educators in primary and secondary education on how to help students build AI literacy and how to craft thoughtful policies that leverage AI’s strengths while mitigating its risks.
• News Literacy Project: This nonprofit offers a host of tools for educators to use to help their students sharpen their media literacy and fact-checking skills. Some of the resources it provides include Rumor Guard, which helps dispel viral rumors, and the Newsroom to Classroom program, which connects schools with volunteer journalists who help instruct students on topics like misinformation, social media concerns and the First Amendment.
• Common Sense Media: This organization provides several resources to help students in middle and high school develop AI literacy, including articles explaining how AI models are trained and the impacts of AI bias.
• Grammarly AI Detector: The popular writing assistant platform provides an AI detection tool, which shows teachers (and students) what percentage of a piece of text is likely AI generated, helping them identify potential misinformation or improperly sourced content.

Protecting Learning in the Age of AI

AI technology has the power to transform education, but its power needs to be balanced against the risks of misinformation — especially as the technology becomes more advanced. By fostering their students’ AI literacy, strengthening their fact-checking skills and leveraging detection tools, educators can better spot AI misinformation and prepare their students to navigate an increasingly AI-driven world.