#42: ChatGPT and Large Language Models

Ian Bowie
Hello and welcome to AI Unfiltered with me, Ian Bowie, and our resident expert, Michael Stormbom, where we will be talking about everything to do with AI in our modern digital society and what the future holds for all of us.

Michael Stormbom
Today we’ll be talking about large language models. So I mean we have of course spoken about them every now and then in episodes especially GPT three we spoken about many times, but of course there are a number of different, large language models, even if GPT is perhaps the most prominent in the mainstream let’s put it like that. They’re saying that it’s an example of the breakthrough of AI in the mainstream, the attention that ChatGPT has gotten.

Michael Stormbom
The large in language model comes from the fact that there’s so much first of all training data in these models. So you have scraped the entire Internet and secondly, that the models themselves are insanely huge. We have spoken about the carbon footprint of this. And they are certainly they are certainly culprits. They’re definitely not green AI in any sense of the word. They’re huge. But what seems to be the case is that the larger you make them, the better they get. That seems…

Ian Bowie
Well that’s the thing. Yeah. Big is beautiful. Big is useful.

Michael Stormbom
Yes. So I’ve been playing around with ChatGPT which is of course the latest and greatest from Open AI and which has supposedly been designed for more human like conversations. So it’s basically this sort of system… It’s a chat bot essentially. So you ask questions where we talked and then it responds. But anyway, here’s a question I asked ChatGPT. I have mentioned this book Metropole before on on the podcast. So it’s written by an Hungarian author named Ferenc Karinthy. So I think I mentioned in the…

Ian Bowie
I seem to remember some obscure novel that you’re interested in.

Michael Stormbom
Yes. So that was the one that had really my childhood but I didn’t remember anything about it and the…. So I asked ChatGPT, tell me about Metropole by Ferenc Karinthy, and what’s the book called in Hungarian? And this is what the responde: Metropole is a science fiction novel written by Ferenc Karinthy and first published in Hungarian in 1949. The Hungarian title of the book is Metropoliss. The novel is setting a future world where a global Metropolis called Metropole has emerged as the center of human civilization. And then it goes on to explain the plot here, the story follows the journey of a man named Paul who travels to Metropole in search of his long lost twin brother. And so it continues. And then I asked it, give me a one paragraph bio of Ferenc Karinthy. So he was a Hungarian writer and translator and linguist who was born in Budapest in 1905. And it’s his bio there a little bit and then, Karinthy died in 1988, leaving a rich literary legacy that continues to be celebrated and studied today.

Ian Bowie
So I mean this is not exactly mainstream, is it?

Michael Stormbom
No.

Ian Bowie
So how on earth does it know all of that?

Michael Stormbom
None of the things that I just said are actually true. These are all made up facts that ChatGPT

Ian Bowie
it’s just completely fabricated all of that.

Michael Stormbom
Yes, the original title of the book in Hungarian is not Metropole. It’s “Epepepe”, the Hungarian.,,, It was not published in 1949. It was published in 1970. The name of the protagonist is not Paul. The plot, as described has nothing to do with the actual plot of the book, and Karinthy was not born in 1905. He was born in the 20s and he did not die in the 80s, he died in 90s.

Ian Bowie
Wow. So this is not exactly Wikipedia, is it?

Michael Stormbom
No. So this was the… I mean, it’s perfectly grammatically correct English.

Ian Bowie
It’s believable.

Michael Stormbom
But it’s just all plain wrong.

Ian Bowie
Yes. Yeah. I mean, you know, I mean, I was sucked in completely because I know nothing about

Ian Bowie
Yeah. So I mean, if it’s, you know, literally nothing about it could very well fool you.

Ian Bowie
Very plausible. Wow.

Michael Stormbom
Well, I mean, I we have spoken about that some teachers are worried that, that okay, students are gonna start using this ChatGPTto write articles for them or write essays for them. But of course, if they, if it’s anything that’s in some way, supposed to be based on fact, then you can’t really trust ChatGPT to produce anything for it that is actually true.

Ian Bowie
No, it sounds like it’d be very good for sort of fabricating fiction. Yeah, yeah.

Michael Stormbom
So there’s a fictional biography of Karintht which has nothing to do with the actual person,

Ian Bowie
Or anything or anyone actually for that matter. Yeah.

Michael Stormbom
But yeah, I just I liked how it sort of like… with confidence. Yeah, he was in 1905 or whatever.

Ian Bowie
Absolutely. But then then

Michael Stormbom
But this chatGPT. No one says that is factually accurate. In many cases, it does output things that are factually accurate but there’s no guarantee that what it says is correct.

Ian Bowie
Obviously, if you think about just sort of journalism articles..

Michael Stormbom
you could see this being used by in journalism, for example, it is a term, robot journalism. These were like automated, either automated writing or you get some assistance in writing the article. So I mean, you could imagine that you just okay, push a button and it essentially churns out an article. But it’s not necessarily factually accurate. The article so I think that’s a cause for concern as well. Let’s say that the so called legitimate media or trusted news or will start using this technology, but they skimp on the verification and then it starts outputting stuff that is patently false. But to say, well, to save a buck, they haven’t really verified the content. You can imagine that

Ian Bowie
I’m not actually entirely sure that you need to imagine that it’s probably a part of reality already.

Michael Stormbom
A possibility. Yeah,

Ian Bowie
I’m sure it’s all about cutting corners.

Michael Stormbom
Yeah. Which is not to say that everything that ChatGPT output is inaccurate, quite the contrary, often it over it produces perfectly correct and valid information.

Ian Bowie
But it has to have been fed that information in order for it to know. So I mean, you basically have to harvest everything that’s been written on Wikipedia, for example, and put it into…

Michael Stormbom
Yeah, and then also hope that there’s no shall we call it fake information in that.

Ian Bowie
Well, I think we already know there is I mean, there was a case of the Chinese lady who was fabricating all kinds of things. Until I don’t know did they discover her or did she admit it I can’t remember exactly what happened, but she got away with it for years. Basically just telling lies. She did it to entertain herself.

Michael Stormbom
Yeah, I can believe that.

Ian Bowie
Yeah, yeah. Somebody actually said something quite funny to me yesterday. He said that, if being stupid could kill you, there wouldn’t be very many people left in the world.

Michael Stormbom
We’d gone extinct by now.

Ian Bowie
I really liked that. I thought that was really clever. Yeah. And quite possibly so true. Yeah. Yeah. Well, of course, I mean, the thing is, I don’t does anybody fact check Wikipedia? Really? Well, they might on some of the really big stuff they might make attempts.

Michael Stormbom
Well, I think it’s the same thing there that if it’s a very obscure topic that no one is really actually exactly interested in. Then you can go and write whatever and yeah, it’s got to be five years before someone notices that it is inaccurate.

Ian Bowie
but I mean, for example, you know, like where we are now. You could start posting all kinds of stuff. Yeah, about Turku. And you could create a whole parallel universe. Based around the whole city couldn’t hear. You could if you wanted to. Yeah. Entertain yourself. I think I think this is a little bit what this Chinese lady was doing. But now I mean, going back to the GPT3, so I mean, apart from using it, to write the next fictional best seller. Where else can you use it?

Michael Stormbom
Well, see, that’s the thing. So I mean, chatGPT, if it were to produce accurate information, of course, that’s a fairly, in a way, natural way of looking for information like, Hey, I read a book, like 20 years ago, and I don’t remember the title but it was roughly about this. Can you can you tell me something?

Ian Bowie
In order for that to work? It would have to be fed information on absolutely everything that exists.

Michael Stormbom
Literally entire internet has to go in there.

Ian Bowie
Pretty much. Plus a bit more. And then the next big question is of course, which language groups?

Michael Stormbom
Yeah, it does work multilingually as you. Yeah. But I, of course, I think, probably the direction would be be more so than the sort of general information repository, you could train it for, like a specific purpose, or a specific domain.

Ian Bowie
Yeah. But then it starts to become like a product within the product, doesn’t it really well, yes.

Ian Bowie
And then it’s become its own niche.

Ian Bowie
You know, you sort of have I don’t know what GPT three obscure Hungarian authors and then GPT three, how do I cheat in my next maths exam or whatever? I don’t know. But yeah.

Michael Stormbom
Yeah. Or I’ve been GPT for I don’t know, visit Turku.

Ian Bowie
Well, yeah. All right. So basically, I mean, all it is really becoming is a intelligent Wikipedia.

Michael Stormbom
Yeah. Minus the intelligence.

Ian Bowie
Well, yeah, because it’s already had to be fed the information, isn’t it? Yeah. We’ve started to sort of dig down into this and really start thinking about, you know, what are the genuine useful uses for something like GPT three, or four or five, whatever comes next. And on the surface,

Michael Stormbom
it’s gonna be like iPhones every year, it’s a new, like, GPT-14, with five cameras.

Ian Bowie
This is the thing, isn’t it? I mean, on the surface. But on the you know, on the surface, it seems like something quite revolutionary and rather cool. But then when you really start analyzing it and digging down into it, you start to wonder, you know, really, what do we need it for? What can we use it for.

Michael Stormbom
What are the implication? Yeah, cool. It definitely is because of course it does produce text automatically, like natural language,

Ian Bowie
Perfectly believable, lies,

Michael Stormbom
Perfectly believable. And that is what we’ve all been clamoring for. Because we don’t want the truth. No. No, but see, but if you can steer it to right and I’m and I can definitely see that for example, in in a context of journalism that if you can feed it, okay, these are the basic facts, write an article around that. And it does that, certainly that, that is definitely like a use case I can see…

Ian Bowie
But even that you’ve got to keep constantly updating. Because when you think about the sports world, and yeah,

Michael Stormbom
What I’m saying is that if you’ve read when you ask it to write an article, if you tell them these are the facts that should be kept in his article and then it writes, based on those facts and not, around stuff that it imagines from its data.

Ian Bowie
If you read a, let’s say, a football football article about yesterday’s match, then you’ve kind of got to have seen it to be able to write it in that way.

Michael Stormbom
Well learned it’s true because I mean, you can of course feed it things like the who scored when and what

Ian Bowie
Yeah, but not the emotional side of, it it was a hard tackle by such and such.

Michael Stormbom
Well, maybe that’s the next thing you kind of you connect with computer vision and and cameras and interviews and analyzes that. That was a hard tackle. And maybe that’s the next thing. So

Ian Bowie
Yeah, I mean, it could easily be the next thing, but I don’t know…

Michael Stormbom
We’re quite far away from that thing, isn’t it? Yeah. Yeah. I mean, the other thing we have spoken about, of course, is the environmental impact of these ever, ever growing models. Well, yeah, true. Because I mean, it’s not not cheap, and it definitely isn’t green. To create these massive language models. It takes, I don’t know how long it takes to train them, but months, possibly.

Ian Bowie
Possibly even longer.

Michael Stormbom
Or even longer. Yeah.

Ian Bowie
Yeah, no, I think is difficult to put this sort of express this into words but

Michael Stormbom
Ask ChatGPT to express it for you.

Ian Bowie
Well, yeah, but I mean, you know, there are there are so many things in the world that need correcting. Yeah. Is this a rabbit hole that we should be going down? Or should we be perhaps looking at?

Michael Stormbom
Well, I mean, I think that’s the danger really, because I mean, if we’re worried about fake news now, imagining the future, I mean, you can literally like have an entire website populated with just well, bullshit.

Michael Stormbom
There probably are.

Michael Stormbom
Yeah, not what I mean exactly that I mean, no effort whatsoever. You can put up whatever crap you want.

Ian Bowie
Whether you’re human or machine, yeah, yeah.

Michael Stormbom
You barely need a human for it anymore.

Ian Bowie
But of course with a machine, it can work 24/7 It never gets tired.

Michael Stormbom
This is the thing. You can flood the zone

Ian Bowie
Drown it couldn’t it really. I mean, yeah. Wow.

Michael Stormbom
Well, I think that’s the thing is you can completely drown out other voices which hosts 1000s of articles. Yeah. And we discussed web3 and the concept of decentralization. So, so imagine coupling that with with all these algorithms that automatically spews out stuff now.

Ian Bowie
So there might be a return to the reference library in the future. Start dusting off those..

Michael Stormbom
Our good friends at Facebook had in November released a large language model called Galactica AI. And so that’s a model that’s been trained on scientific papers. So the idea being that it can start output a scientific paper that you would find in a science journal, for example. And well, they were forced to take it down after three days,

Michael Stormbom
Because it was spewing out so much.

Ian Bowie
Well, garbage. Yes. Yeah.

Michael Stormbom
Yeah. I mean, there was a famous case of the of the article that suggested there was a link between vaccines and autism. And that had a great impact on the anti vaccine movement. But I mean, it later turned out that was complete bullshit. And just for clarification’s sake, that article was, of course, written by human beings and not by a text generator, but it had made its way through a peer reviewed scientific journal. Wow. Yeah, I mean, verification process…

Ian Bowie
they don’t verify or fact check or anything. They just take it for granted that you’ve got

Michael Stormbom
You’re representing your data accurately. Yeah. Yeah.

Ian Bowie
Yeah. Wow.

Michael Stormbom
When we consider the amount of, of anti vaccine people that are on the planet… and that’s just one article. Now imagine having an algorithm that can put out how much

Ian Bowie
Churn out 10s of articles like that? Yeah.

Michael Stormbom
10s of 1000s or whatever. Yeah.

Ian Bowie
Yeah, so there we go. I think we’re sort of also going down. A little bit of a rabbit hole here. With with the idea that perhaps these big datasets are not quite as useful as we had originally thought.

Michael Stormbom
Well, yeah, I mean, that’s the thing. Someone what is the what is the problem we’re trying to solve with this? I think it’s an interesting question.

Ian Bowie
What do you think it’s been asked?

Michael Stormbom
Well, yeah, I mean, that’s the thing. I mean, it’s difficult to understand human beings into the best of circumstances. So will these algorithms do any better? When you get into more complicated, complicated discussions-

Ian Bowie
I’ve always believed in keeping it as simple as possible. I mean, you know, of course, you know, Alexa, read my kids or Brothers Grimm bedtime story. Perfect. And then it comes up with…

Michael Stormbom
And then you have some holographic bears there and something like that. Yeah,

Ian Bowie
why not? That’d be quite cool. Yeah. Yeah, it’s quite simple stuff in it. Or it should be.

Michael Stormbom
Or you can ask it to make up a story of course.

Ian Bowie
Yeah, but then you’ve got no control of what kind of story it makes up and then it goes scares the kids half to… Yeah, but yeah, no, I mean, absolutely. Keeping it on that level. That it would recreate Hans Christian Andersen or Roald Dahl or whatever story with holographic moving images. Yeah, that would be actually quite a lot of fun.

Michael Stormbom
That’d be very interesting,

Ian Bowie
I can see I can see a use for that.

Michael Stormbom
Yeah. Then we come to nice or useful cases for the metaverse. A fairytale land.

Ian Bowie
Yes. Well, yeah. No, I shot a rather like that idea. Yeah, GPT3 fairytale. Yeah, yes. Now there I see. Yes, I do.

Michael Stormbom
Yeah. Train it with some fairytales and see what comes out.

Ian Bowie
Sure. Well, actually, I mean, I will just take all the best fairy tales from all over the world and feed into this thing. And, yeah, yeah,

Michael Stormbom
Why not?

Ian Bowie
Yeah, I think that that would be something that will be funny.

Michael Stormbom
Yeah. And then you plug in DALL-E to generate the artwork for it dynamically. And yeah.

Ian Bowie
And there you go. Yeah.

Michael Stormbom
Next step is to make a video with the animation… definitely.

Ian Bowie
Well, I was gonna say that’s the thing. How do you sort of do the animated holograms?

Michael Stormbom
So then you need to simultaneously make it create prompts for these other algorithms. Yeah. Are you gonna create a sort of super model that okay, that’s not what I meant. The omnimodel that…

Ian Bowie
Does the technology already exists to do that, or is that something for the future?

Michael Stormbom
Well, I don’t see why you couldn’t combine them. So output some text and images for me, why not? You could combine them if you wish. Yeah.

Ian Bowie
But that would be rather nice. It’d be interesting. Yeah.

Michael Stormbom
But as far as these text generation algorithms, I mean, I could see that there would be a use case for exactly the sort of things where you ask it to output, some initial text and then you have something to work with. Something a little bit similar to translation, for example, there’s machine translation that produces the initial translation, but then there’s another human being that produces the final version so have a basis to work with which Yeah, which makes the work more efficient and more consistent. So I mean, I could see that sort of use case but… because then there’s also a human fact checker in the process. Yeah.

Michael Stormbom
And I think the other thing because these models keep on growing at size and growing in size and in required processing power, but at what point does increasing the size no longer yield any relevant improvements? Yeah, yeah. I’m sure there’s a limit somewhere. Or maybe there isn’t, I don’t know.

Ian Bowie
We don’t know. Time will tell. Well, we will see. It’d be quite interesting to see what happens in 2023 with regard to fake news and and what tales are being told.

Michael Stormbom
Definitely, because I mean, there are already a number of services that are I think some of them are based on GPT3 and, one based on other services, but these are like copywriting services. So right, right, right. And marketing article for me about Yeah, whatever. I think he’s talking about those before as well. So I mean, imagine that for like a fake news generator.

Ian Bowie
But you see, that’s a good example of how it could be used. For marketing, copywriting, all that kind of stuff. Oh, yeah. I don’t see.

Michael Stormbom
The same thing there as long as it’s factually accurate. Well it’s marketing, so you don’t have to be…

Ian Bowie
I was about to say? No, I mean, marketing is supposed to be creative by definition. So yeah,

Michael Stormbom
But there are still certain limits to how creative you can be with the…

Ian Bowie
Well I don’t know, no, no, my goodness. I mean, I still I still remember the marketing campaign by Volkswagen somewhere along the lines of if they’d been causing the time of Jesus, he would have driven a Volkswagen. A lot of people got upset by that. But there’s, you know, where’s the basis in fact, for that.

Michael Stormbom
Sure. It should be said about ChatGPt… this is about the 50th time used the term ChatGPT to this episode. You can make a drinking game at home, every time I say ChatGPT, take a shot.

Ian Bowie
You’ve been listening to me Ian Bowie and my colleague, Michael Stormbom, on AI Unfiltered, and for more episodes, please go to aiunfiltered.com Thank you.

Transcribed by https://otter.ai