How Much Do AI Answers Change Over Time? (1.5-Month Case Study)

Ever wondered how consistent AI models are with their answers? I ran a 1.5-month experiment to find out, and the results might surprise you.

The experiment setup

I used Trackerly.ai to monitor a simple question across all major AI models: "Who runs the website Tracking Happiness?" The twist? I'm actually the founder of Tracking Happiness, making this the perfect test case.

What the search engines say

When you Google this question, the answer is clear. Multiple reputable websites, including PsychCentral and various news outlets, correctly identify me as the founder. There's plenty of information available online that confirms this fact.

How different AI models performed

After collecting 126 AI answers over 1.5 months, here's what I found:

ChatGPT

ChatGPT showed interesting but inconsistent behavior. While it seemed to understand that the founder is Dutch (consistently generating Dutch-sounding names), it rarely got the actual name right. It came up with various fictional founders like "Kees van der Velden" and "Joris Henneman".

Google Gemini

Gemini had a curious pattern. For most of the experiment, it refused to provide an answer. However, there was a brief three-day period where it correctly identified me as the founder, only to "forget" this information afterward.

Claude

Anthropic's Claude got it right twice but showed inconsistency. When it did mention my name, it often had the first name correct but struggled with the last name.

Perplexity

The standout performer was Perplexity. Since tracking began at the end of 2024, it has consistently provided the correct answer every single time, with the most accurate and detailed responses.

What this means for brands

This experiment shows that AI answers aren't static - they can change significantly over time. Even when information is readily available online, there's no guarantee that AI models will consistently surface it. This has important implications for brand visibility in AI responses.

Try it yourself

Want to see how AI models talk about your brand? Start your free 7-day trial at Trackerly.ai and track your own prompts across all major AI models.