What AI Gives You May Not Be an Answer, but a Beautifully Formatted Hallucination

Today I asked Gemini to help me find air-conditioner installers, with the condition that it draw mainly on user experiences from PTT or Dcard (two of Taiwan’s most popular online forums).

At first, it listed some URLs that looked quite thorough, but when I clicked through, they either wouldn’t open or the content didn’t match.

The Moment I Pointed It Out, It “Patched” Things Together Even More Completely

After I pointed out that the URLs were problematic, it put together another “genuine list of PTT netizens’ air-conditioner recommendations,” this one including shop names, PTT IDs, reasons for recommendation, and even tips on which keywords I could use to search.

At first glance, it really did look as if it had done its homework.

But dig a little deeper and the problem only grew: those shops were almost nowhere to be found, and the so-called PTT posts and netizen recommendations had no corresponding sources either.

It’s “Generative AI,” Not “Refusal AI”

This also reminded me: when data has to be searched for, some AI, if it can’t find reliable information, won’t necessarily be honest and tell you “I couldn’t find it”—instead it’ll find a way to “patch together” a piece of content that looks perfectly reasonable.

Because it’s “generative AI,” not “refusal AI.”

In other words, what AI gives you may not be an answer, but a beautifully formatted hallucination.

For Anything Involving the Real World, You Can’t Rely on AI Alone

AI is well suited to helping us map out a direction, generate search keywords, and summarize content that’s already been confirmed; but the moment it involves real shops, prices, and reviews—or matters like law, medicine, and investment—you absolutely cannot go by AI’s answer alone.

AI can save us search time, but it can’t replace verifying the sources. Looking complete doesn’t mean it’s true.