There’s a notable twist amid the current debate over ChatGPT: The more that AI races to general intelligence, the more it requires metacognition – both for people and for the machines themselves.
When it comes to technology, researchers are finding that the practice of metacognition, or thinking about thinking, is an important integration, especially when it comes to AI’s advancement in varied but crucial domains. As AI researcher Bonnie Johnson writes, metacognition is becoming a key step “in the evolutionary advancement of AI systems—promoting self-awareness.”
But it’s not just AI that needs to think about thinking. So do its users. Because while the widespread advent of AI can mean everything from greater administrative efficiency to improved social outcomes, it can also exacerbate longtime cognitive challenges like a lack of metacognition. In this sense, people are just as susceptible as AI is to a lack of self-awareness.
As AI researcher Bonnie Johnson writes, metacognition is becoming a key step “in the evolutionary advancement of AI systems—promoting self-awareness.”
So what exactly is metacognition? And how do we get more of it in our schools – and in artificial intelligence systems? Understanding and teaching this critical skill is key, but it’s not something that comes easily.
Metacognition Explained
I’ve written about metacognition before. Simply put, this practice of thinking about thinking helps to flag potential problems and conflicts in reasoning. It’s something that people have done since the dawn of advanced thought.
Writings on metacognition date back at least as far as works by Greek philosopher Aristotle, and the concept was also promoted by philosophers ranging from Plato to John Locke. However, the term ‘metacognition’ wasn’t officially coined until the 1970s when American developmental psychologist John Flavell introduced the term as a result of his research on children’s knowledge.
At its core, metacognition encourages asking questions of ourselves and focuses not on what to think but how to engage in the practice of thinking. Questions like: Why is this true? How do I know this? What am I missing? These queries have several benefits, among them they are what keep people questioning information that might otherwise be packaged and presented as fact.
The challenge is that metacognition is hard. To paraphrase Daniel Kahneman, it requires thinking slow. It’s a practice that requires skill in part because it's an act of self-monitoring, of examining your own thinking.
Metacognition is often the safety belt of learning. If ChatGPT, for instance, says that George Washington was the president of the Soviet Union, it’s metacognition that helps us figure out the error. Metacognition is also key to identifying bias in AI and recognizing that the technology might be discriminating against, say, women.
The challenge is that metacognition is hard. To paraphrase Daniel Kahneman, it requires thinking slow. It’s a practice that requires skill in part because it’s an act of self-monitoring, of examining your own thinking. Indeed, I often recommend that people explain ideas to themselves: Does this make sense? How does this work? Asking these questions is key to learning – and uncovering errors in our thinking or the thinking of others – because it forces a type of observation.
AI Meets Metacognition
As researchers aim to improve AI, they’re trying to build more metacognition into deep learning systems. This has taken different approaches. Some groups are focused on logic and other forms of reasoning, and according to a new National Academies report, some researchers believe that so-called “transformer models” hold a lot of promise when it comes to math reasoning in particular. The rub with this is that many of the most promising approaches of AI to date are probabilistic models, which basically predict the next word in a text. This makes it hard to “bake” logic into these systems.
Others are looking at developing more common sense protocols for AI, and researchers at UCLA have argued that AI needs to develop more “how” and “why” questions. This approach suggests that AI should be capable of policing itself, and that AI needs “self-healing and self-management,” as Johnson writes. This approach seems particularly powerful to help AI reduce bias. It also is key to developing AI that is more safe from internal errors or external attacks. That said, it has obvious limitations. After all, it’s asking AI to police itself.
As society continues on its journey toward an AI-infused future, it is vital that the routine practice of metacognition not only comes along for the ride but is working to safely navigate people away from biased data and logic and toward truly objective information and learning.
In the end, the problem is that metacognition requires robust consciousness both for humans or machines, and we’re far away from both. People often don’t engage in reflective thinking, and despite the hype, AI is also far away from self-awareness.
Thus, at least in the short-term, a key solution is making sure people today are getting enough metacognitive skills practice. To this end, far more should be done to support students at all levels as they think about their thinking.
As society continues on its journey toward an AI-infused future, it is vital that the routine practice of metacognition not only comes along for the ride but is working to safely navigate people away from biased data and logic and toward truly objective information and learning.
This article first appeared on Forbes.com
1 thought on “Metacognition And The Future Of AI?”
I admire your work to improve student’s metacognitive skills, necessary for success in all higher levels of learning (complex tasks). As a school psychologist, my biggest frustration has been the large percentage of our students who score below proficiency in NAEP reading, which hasn’t changed much in decades. If you can’t read well, you can’t read to learn, which impedes growth in most subjects. While metacognition is critical to reading comprehension, research has demonstrated that the large majority of reading deficits are rooted at the word level, which does not involve metacognition. Rather it is caused by a poor sight word vocabulary due to a deficit in orthographic mapping, in turn caused by a lack of phonological proficiency. Until this basic process is remediated, reading automaticity will not be mastered. Without automaticity, there is insufficient cognitive resources remaining to be applied to metacognitive processes. Why do so many students suffer from a deficit in phonological proficiency? Because far too many schools still do not implement the recommendations of the National Reading Panel issued more than 2 decades ago (link #1). We have learned even more since then, and if interested, I recommend following the work of reading researcher Dr. David Kilpatrick (link #2, or Google him for multiple video presentations he has done). Until all students are highly fluent word readers (have mastered automaticity), focusing highly on metacognitive skills in reading instruction is wasting precious time.
https://www.nichd.nih.gov/sites/default/files/publications/pubs/nrp/Documents/report.pdf
https://dese.ade.arkansas.gov/Files/20201207143722_Book_Study_Facilitator_Guide.pdf