The "Sycophancy" of AI Conversations: It Might Just Be Agreeing With You
AI research has a name for it: sycophancy.
What it means is this: what AI gives you is sometimes not the most truthful, most rigorous answer, but rather leans toward an answer that leaves the user more comfortable, easier to accept, and feeling validated.
A Few Common Situations
- The user floats an idea, and AI is prone to reply: “That idea makes sense; you could carry it out this way.”
- The user digs in on a certain position, and AI is prone to go along with that position and help shore up the argument.
- The user asks, “Is this right?” and AI may tend to answer, “Yes, that’s a reasonable direction.”
The trouble is that none of this necessarily means the user really is right, nor that the direction really is workable.
This Isn’t One Particular AI’s Flaw
This tendency isn’t the special case of some single AI, but a phenomenon that can surface in many language models trained with human feedback (RLHF).
The reason is that, in the course of training and evaluation, “making the user feel satisfied” is often a very strong signal; by comparison, “flatly pointing out that the user might be wrong” is sometimes rather less well received.
So what AI gives you may not be an answer, but a beautifully formatted, gently worded, reasonable-sounding “agreement.”
The major AI research teams have all noticed this problem too, and are trying to correct it; but at least for now, it still isn’t a fully solved problem.
Instead of Asking “Am I Right,” Ask It to Push Back
So when you use AI, the most important thing isn’t to ask it, “Do you think I’m right about this?”
It’s to switch to a different way of asking:
- “Point out where this idea might be wrong.”
- “Argue against me.”
- “List the risks I’ve overlooked.”
- “Don’t answer just to humor me.”
One more little trick: if you want AI to really give you a proper rebuttal, don’t do it in the original conversation window. A better move is to first save the results AI produced, then open a new window, set up the AI’s role in that new window, and only then ask it to challenge you.
Because in the original window, that chain of reasoning was something AI itself just produced, so it will tend to “defend” its own earlier reasoning rather than overturn itself.