Trudi Jacobson

Post One: AI’s Understanding of Metaliteracy

I have been working extensively with ChatGPT on several personal projects, ranging from research into specific antique objects and the historical details they can yield to pastel painting tips. I’ve also used it to plan meals for an individual with multiple health conditions who has strict dietary restrictions. I have compared responses from Claude and ChatGPT, and have had to engage in more research when their answers differed completely. It has been a fascinating process, but I wondered what would happen if I transferred this approach to one of my academic and professional interests. As a next step, I investigated and compared AI responses about metaliteracy’s relevance in this environment. 

This blog post is the first of several in which I will report the findings after asking Claude, ChatGPT, and Gemini several questions about metaliteracy and its role in a burgeoning AI environment. I initially wanted to hear how they would describe metaliteracy. In this first post, I will consider the differences–and similarities–in the responses when I asked the three AI platforms “What do you understand metaliteracy to be?” I put a 300-word limit on their responses.

All three platforms mentioned metaliteracy in connection with information literacy. Claude labeled metaliteracy a framework for information literacy, ChatGPT called it “a contemporary  learning framework that expands traditional information literacy,” and Gemini identified it as “a comprehensive framework that expands the traditional definition of information literacy.”  Gemini was the only one that did not attribute the development of metaliteracy to Tom and me, as co-originators of the concept, while Claude was the only one to provide a date for its development (“around 2011”). Claude linked metaliteracy with the ACRL Framework for Information Literacy for Higher Education. Unfortunately, it did so incorrectly, noting that it was “adopted as the conceptual foundation” for the document, rather than threshold concepts. It is worth noting that threshold concepts are mentioned but not attributed to their critical role in the Framework. As an emerging concept at the time, metaliteracy was incorporated into the ACRL document less overtly.

The metaliterate roles that individuals take on in “networked, collaborative, and participatory environments” (Claude), in a “digital networked world” (ChatGPT), and in “participatory environments” (Gemini) are noted to varying degrees in the responses. ChatGPT highlighted production, sharing, collaborating, and reflecting on information in its initial paragraph of the response. Later, it mentions the shifting roles of “reader, author, editor, curator, commentator, and community member.” Claude and Gemini focused on the role of “active producer.” Developing as a producer of information is vital to metaliteracy, so it is appropriate that it appears in all three responses. But the fact that two of them added the word “active” in connection with the producer role is unusual, especially compared to our simpler use of the noun alone. However, we do stress active learner roles, and it might be that these two platforms combined them, either due to space constraints or to their interpretations of our key concepts.

While Claude and Gemini included less about the roles, they both mentioned the four domains, while ChatGPT did not. What ChatGPT did was include a list of six core dimensions: critical evaluation, creation and participation, metacognition, ethical awareness, collaboration, and adaptability. While all six are important to metaliterate learners, this is a rather arbitrary list and does not reflect our core components with their goals and learning objectives. That said, ChatGPT does highlight metacognition in its response, as the others did, though not always naming it:

ChatGPT: “What distinguishes metaliteracy from older literacy models is its self-reflective and participatory orientation.” (bold typeface ChatGPT’s)

Claude: “That last domain [metacognition] is arguably the most distinctive feature. Metaliteracy treats self-awareness as central….” 

Gemini: “It is essentially about becoming a responsible, self-aware digital citizen.”

None of the three responses fully grasps metaliteracy, which raises the question of whether the word limit played a role. If I had given the platforms 500 words or 1,000, how might the results have differed? As this is not a scholarly study, I won’t follow up on that currently. Another approach I might have taken, based on the advice of a friend who has worked extensively with the engineers and others developing AI, is to request that a platform ask questions until it can provide a response about which it is 95% certain. How, or would, the responses change significantly? And would the responses alter based on a query from another individual, or a slight wording change to the query?

As the responses stand, none fully grasped the concept of metaliteracy, nor included all the key components. Yet if they got the general gist, would that be an adequate starting point for those interested in metaliteracy? And possibly a stopping point, given how in-depth–or not–their interest is? Reading all three responses together gave a fuller picture of metaliteracy, albeit one with a few mistakes, misdirections, or gaps. Yet how many people will check three AI platforms for the same query? If they are working at that level, they will be using scholarly sources. I think that additional studies taking some of these points into consideration would be useful. 

A couple of obvious notes about the research process behind this series. First, I’ve never accomplished my primary research so quickly. From idea to data gathered took about 45 minutes, and that included several breaks. Second, it is not surprising, but it is disappointing that no references were included in these preliminary interactions with all three platforms. While the process was fast, the results were inconsistent, incomplete, and lacked documentation. Of course, I could have expanded the number of words and requested proper citations, but I was curious about what AI could tell me about metaliteracy in these brief and limited interactions. My approach very much reflected how people are engaging with these systems for quick answers, similar to the personal examples I shared at the start of this post. It makes me wonder about how someone new to metaliteracy may understand the concept through AI interactions rather than our published research and projects.

Now that I have answers from these three platforms about what they feel metaliteracy is, the next post in this series will examine the responses to the question, “What role does metaliteracy have in preparing individuals to work with AI?” I didn’t suggest revisions to their understanding before asking this question, as their general grasp sufficed to obtain applicable responses. Tom and I encourage your comments on this post and its contents.