A while back I wrote an article on an AI Future. Since that time "AI" has shifted its meaning, since only a few months past that point the first ChatGPT was released, and Stable Diffusion blew up almost immediately after the article. Thus the previous article focuses on "smart" devices, i.e. theromstats, refridgerators and even freaking light switches that have to connect to the internet to function and can break down if there is any connectivity or software problem. At that point "AI" was just any computer program. There was a future chic aspect to the whole affair where having a thermostat run by an external computer program just felt "smarter" but gave no real benefit over a mercury switch, perhaps paired with a simple program that runs entirely from one microprocessor.
Now of course "AI" means "Generative AI," and the successes in text generation from Large Language Models and equivalents for image (and now video) creation. Indeed the outputs are very impressive. This sort of stuff existed hwen I wrote the last article, but the training sets used to create the models were also small. The key step was to increase the training sets to insanely large amounts. Indeed, text and image generation is so good now that there are many people taking basically the position "yeah, we've tried to sell you the idea that AI would take over the world for 70 years now, but THIS time we've actually made the leap where it could!"
The hot new term is "artificial general intelligence." The idea is that by making things so complex, AIs have somehow magically gained a complete understanding of the world. For example, let's say you make a program that finds prime numbers efficiently. I would contend it isn't doing any thinking at all, but even strong AI advocates who say that programs do think would not say that it thinks about anything other than the mathematical calculations needed for finding prime numbers. For example, it would have no conception of animals, politics, programming, or even fields of mathematics like geometry or statistics. In contrast a general AI would understand not only what it was programmed for, but also the world in general. AI advocates jumped on ChatGPT as an example of such a general AI, because it made outputs that it was not programmed for. It was set up only to create natural sounding responses to text, but it can (most of the time) output the code for actual programs when requested (provided that you use a common programming language like Python.) It wasn't meant to do that, so somehow the complexity gave it intelligence, kind of like Jane from the later Ender books.
Since that point there have been successes in combining these models with search engine data, converting images to text descriptions and vice versa, etc. which makes the AI look more and more "generally" intelligent. Indeed a lot of this is impressive and I woudln't have guessed that we would have programs that did this sort of stuff when I was writing in 2022. But it's the very quest for a "general" AI that handicaps the process. Let me explain what I mean with a specific example. Since the free version of Copilot lets you upload images, I did some experiments where I had it try to translate images with Japanese text (I'm not fluent in Japanese, but know enough to get the gist of most text on the first read and can get a more precise meaning with the use of dictionaries and such.) One image I had it translate was this one:
Copilot didn't give me an output on the first couple tries, but eventually said that the text was "雨だから便利でしょう?" and that it means "it's convenient because it's raining, right?" It further said that this was in reference to an umbrella that the character had found and was glad to have in the rain. Now some of this is very impressive. The given Japanese does translate that way, and the model did recognize taht there was a character holding an umbrella. But there's a problem here: even if you don't know Japanese you can verify that the characters that Copilot outputtted do not appear in the image. What it actually says is "雨が降ってので、傘を持って来ましたよ." This means something like "since it's raining, I brought you an umbrella" with the implication being that the umbrella is being given to the viewer. You could have probably guessed that the umbrella isn't for Nora anyway, as she already has one.
What happened here? Copilot made up some Japanese text out of whole cloth and claimed that it was in teh image, when it was not. What's fascinating is that it actually has the potential to translate the image properly. When I asked follow up questions not about the image as a whole but about the textbooks in particular, it did recognize the characters in them (though it reversed the two halves of the sentence since it didn't apply a right to left reading order.) Thus the problem isn't that Copilot cannot recognize text in images, or Japanese text in particular. That sort of thing does happen with LLM, where instead of saying something like "I cannot do that" they will make up an incorrect answer. Here the AI could have given me the correct answer from the get go, but didn't. This sort of behavior happeend with other images too, such as claiming that the phrase "ハッピーバースディー" (i.e. a phonetic rending of the English phrase "Happy Birthday") appeared in an image when it did not. When pressed about specific lines of text, again it gave the correct output and the correct translation.
I believe that these mistakes are an inherent part of trying to make a "general" AI model. Keep in mind that this AI was not intended as an image translation program. It was intended as a "do anything" program. So for example it should be able to equally well handle requests like "make up some Japanese text that the character in this image might be saying without using the text in the image." Indeed, I have asked it requests like that and it will output Japanese text that is related to the image. For example, on this image it said Nora could be saying "雨が降っていますが、私は大丈夫です", i.e. it is raining, but I'm fine. Now here's the thing: the AI doesn't "know" what I'm asking for. It really doesn't know anything at all, but even if I believed computers thought it still couldn't really understand my requests, since it is just a model made to create likely responses to text and image inputs. Now consider that if I ask it to make up Japanese text, or if I ask it to translate actual Japanese text, the result in either case would be Japanese text followed by a translation. Further compounding the problem is the fact that when you get into areas that require some special skill, like translation, most people will not be able to determine if the output is good or not. This also explains the mysterious "ハッピーバースディー" output: the image in question had a birthday cake and indeed it was a birthday greeting (it just didn't use that phrase) so a plausible guess for "what is this character saying (if you ignore the text)" would include that phrase.
Note that a non-"general" AI wouldn't have this problem. For example, suppose we had a program which only recognized Japanese text in images, that's it. It would never make things up out of whole cloth. It might misintepret characters (especially in a language like Japanese where there are many similar looking Kanji) but it wouldn't create a sentence that had no relationship whatsoever to the text in the image. Similarly, a translation program being fed Japanese text would only transalte that text, or state that it could not. Another set of experiments that I have done is to feed in Japanese text that is nonsensical (whatever random words pop into my head) but which follows a structure kind of similar to Japanese text. I never get the answer that the text is nonsensical. Sometimes I get a bunch of gibberish, but gibberish that largely does not correspond to the gibberish I put in. Many times I get an explanation back in Japanese despite the prompt clearly asking for an English translation. My guess is that the input text is long and complicated enough that the language model fell into the "answer questions from Japanese speakers" mode rather than "translate to English" mode. In some cases it will "translate" but then go into a discussion of where the sentence might be used, or how Japanese culture is cool, or whatever else. These types of outputs are only possible because it's meant to be a "general" AI. After all I could be asking it questions like "when would you use this sentence" or "how does this sentence demonstrate the uniqueness of Japan" or I could be speaking to it in Japanese. If we are lucky the statistical models behind the AI will guide it to a response that corresponds to what I was actually asking, and for common questions (those closest to the training data) it will do this pretty reliably. But for uncommon requests we could end up anywhere, even with outputs that are invented by the AI or which do not address what I want.
All of these are problems inherent to being a "general" AI and I don't see how you get around them. It's not like these are programs that you can easily read line by line to debug. They are complicated statistical models formed by going through insane amounts of data. You could try to reweight the models through extensive user testing in order to avoid these situations. For example, we could add large penalties to outputs like the one I got above where I asked it to translate the text in an image and it provided me with text not in the image. Of course, that only tells the AI to avoid that response; it can't understand why it should avoid that response. We want it to know "you made up text" but equally valid answers could be "this answer is bad, because 'it's convenient because it's raining, right?' is not a valid translation of 雨だから便利でしょう", or because it identified the object in Nora's hands as an umbrella and it was something else, or that rain is a taboo topic that it should not mention, among millions of other things. Of course, even this type of analysis over simplifies things, because the AI model doesn't really have concepts like "rain" or "Japanese" or "a valid translation." It has an idea of outputs and parts of outputs, but it could latch onto anything. For example, it could be that when we say that this is a bad output the model updates to avoid writing "rain" in the same sentence as "right?" So it could give us the same output and say instaed "It's convenient because it's raining, don't you think?" If you've worked with an LLM that allows you to give feedback you've probably seen this sort of issue. For example, I've had times where I've asked about some obscure character from a popular series, like say Bean the Dynamite from Sonic the Hedgehog. Sometimes it works, but sometimes it gives me a description of the main character Sonic instead. If I complain that it is telling me about Sonic, not Bean, it will apologize and output the exact same thing, except with every instance of "Sonic" replaced with "Bean", ex. "Bean the Hedgehog is a blue hedgehog who is the star of a popular series of games by Sega." The AI can't know that my complaint is that "this is not about the right character" instead of "you didn't say Bean." Even if I complain that the new information is inaccurate, it is as likely as anything as to simply insist that it IS accurate, so there! Or it might apologize and say that it can't be accurate about everything, but still not give me the information I requested (even though it has the capability of getting that information.)
So the AI will make things up, not do things it was asked to do, and steadfastly refuse to correct itself properly. But again, how do you get around these problems? Let's say that we fix the problem of the inaccurate Nora cat translation by getting it through to the AI (somehow) that it should never make up text that does not appear in an image. But now it cannot give the correct output to someone who asks it to make up text that doesn't appear in the image. Similarly, people do ask AI to output intentionally inaccurate information (for the laughs, among other reasons). So if we somehow force AI to only say the truth, now it can't do that correctly. In order to get better output from the AI you are forced to sacrifice the "general" part of AI. But this is unlikely to happen for reasons that we will explore in part two of this essay.
July 30, 2024