People often mistake AI for humans, but follow-up questions can expose chatbots

In a study with over 72,000 judgments, participants frequently mistook AI-generated captions, word associations and replies for human work. A smaller experiment found direct questioning more effective at exposing a chatbot, though with caveats.

• Participants made more than 72,000 judgments of human and AI responses
• AI captions, word associations and replies were often taken as human
• Direct follow-up questions exposed chatbots more often in a smaller test