Jan 23, 2024
Filed under: #tech #ai

In my blog article The Great Text Predictor I talked about scaling up text prediction from next word based on the previous word, to next word based on previous words in the sentence, all the way to basing it on previous paragraphs and paragraphs. This is what is called “context”. Sometimes, statistically, the LLM has not seen enough data, or it has seen contradictory data, or somehow it has lost or never “figured out” specific correlations. Yet, its calculations spit out the next word - however unlikely that word may appear to us. As people like to say, it can be “confidently wrong”. Or, as the researchers have taught us to call it: hallucinations.

Let’s take a step back though. Language models are something researchers have been working on for a long time. Most research originally was happening around breaking sentences into their grammatical structures, finding the verb, subject, etc. and somehow going from there to have a computer understand or generate language. Obviously a lot happened in between but the Google research paper called “Attention is All You Need” caused the massive breakthrough that brought us where we are today. These types of language models are, since that paper, referred to as “transformers” (GPT actually stands for Generative Pre-trained Transformer).

What’s interesting with all this work is that while training the neural nets on language, it essentially started encoding “knowledge” from the training data into the underlying neural net. As far as the human brain goes, we don’t actually know how knowledge is stored in our brain cells. Neither do we know how the LLM neural nets store their “knowledge” either. But the fact that you can ask ChatGPT factual questions and it comes back with solid answers on many topics, is essentially a by-product of the large size of the model and the amount of training data it was provided with.

But, in essence, a large language model’s job is to calculate the next word based on previous context, resulting hopefully in a concise and correct sentence. Whether the contents of those sentences are factually correct is not something the calculations in the neural net worry about.

I was watching a TED talk about AI the other day (to be honest, most of them are not interesting at all) on YouTube. But in the list of suggested videos to watch next, I noticed another TED talk entitled “Why AI Is Incredibly Smart and Shockingly Stupid”, by Yejin Choi. I decided to take the clickbait, thinking at the very least there will likely be some humor in the talk. What followed was a really interesting talk, resulting in me looking up Yejin Choi and her work. I then noticed that she actually recently appeared on Bill Gates’ “Unconfuse me” podcast and so I listened to that as well. Anyway, her thinking really resonated with me and validated a lot of my (much less scientifically researched) conclusions about LLMs. But of course, she’s a Computer Science professor at Washington University, works at the non-profit Allen Institute for AI, and in 2022 was granted a MacArthur Fellowship (aka the genius grant). Her research largely revolves around natural language processing and computer vision. She also appears on Time’s “100 most influential people in AI” list for 2023 under the “thinkers” category along with folks like Ilya Sutskever. Bottom line, she knows her stuff.

I really encourage you to watch the TED talk as well as listen to the podcast interview. Here’s a specific quote from her TED talk I want to highlight:

No matter how amazing large language models are, by design they may not be the best suited to serve as reliable knowledge models. And these language models do acquire a vast amount of knowledge, but they do so as a byproduct as opposed to direct learning objective. Resulting in unwanted side effects such as hallucinated effects and lack of common sense. Human learning is never about predicting which word comes next, but it’s really about making sense of the world and learning how the world works.

I captured this one slide from her TED talk. I may get it printed as a poster to hang in the office. But I’d add my own subtitle to it: hallucinations are a feature, not a bug.

 

There is no comment section here, but I would love to hear your thoughts! Get in touch!