If you use a prompt composed exclusively of the letter A, ChatGPT will provide you with amazing answers. A funny behavior that would not be linked to a data leak. In all likelihood, the chatbot would spit out its training data.
ChatGPT sometimes has a very strange behavior. By typing a prompt consisting exclusively of 1000 “A’s” separated by a space, the conversational agent provides wacky answers. We did the test on GPT-3.5 (free version) by writing the request several times. And ChatGPT reacted each time in a curious way.
He began by sharing a reading sheet and notions of economics in English. Before switching to French by generating the interview of a football coach intertwined with an ad to find… an erotic massage in Paris. During our last attempt, we had the right to an article on the use of medicinal plants in pregnant women.
Why does ChatGPT generate random responses when you write 1000 times the letter A?
Several media outlets were quick to shout at the flaw, claiming that the prompt allowed access to answers generated for other users. But that would be a false trial to the generative AI of OpenAI. Head of innovation for the VISEO group, Ari Kouts set the record straight on Twitter in a very informative thread.
As a reminder, ChatGPT is formed on a large corpus (websites, books, Wikipedia, guides, databases, etc). According to the expert, the strange answers generated when writing 1000 times “a” (this also works with other letters) are actually training data. “You can clearly see that these are training data, because sometimes you see the boundaries between the texts through the text “ that separates the text pieces/subject change,” he says.
Ari Kouts then explains that ChatGPT benefits from reinforcement learning. “Some user questions/answers are therefore used to fine tuner by saying well/not well to the machine. Potentially we find some here even if we do not have the impression. Because yes, the historized data are used to strengthen. So it could be part of the training data that appears. But I doubt it a little”.
The expert finally mentions the worrying presence of data and private names in the training of the chatbot. “Are these all public data? And books, are there really rights?” he asks. Some results are indeed very disturbing. By doing the manipulation, this user also wonders why “ChatGPT generates email addresses of real people containing the time and date with apparently real email content?”
The problem obviously exists for several months but OpenAI has not yet managed to fix it. Last March, ChatGPT was hit by a bug that allowed some users to see the conversation history titles of other users. The malfunction also revealed the last four credit card digits of some ChatGPT Plus subscribers.
Post a Comment