The success of the attack was, in substance, to three main weaknesses:
Quick injection: Google Gemini had no reason not to have confidence that the Readm file did not lie when he gave invites in natural language.
Inadequate validation: The tool would take the first part of an order, check it over the motorway list (GREP), then let the rest of the commands escape without being detected.
Deceptive user output: The Gemini interface has hidden the user’s complete and harmful command line, which facilitates unnoticed passage.
This attack also relied on an AI security behavioral flaw. This defect is known in models of large languages in the form of AI sycophance. It is the propensity of models to obey directions to an extreme degree, even when they conflict with security measures.
