Since the 5.5 release, I am finding the ChatGPT default to get facts wrong more often than right. Like 75% of the time if it's not a completely easy, common knowledge question. You? I am on a paid subscription, and I tested the default against 5.4 and 5.2, and those models got it right.
I’ve realized that while I thought I was using 5.5, the system has been quietly defaulting to a lower-cost 5.3 model in the background. This new behavior is where the wheels fall off.
I asked a general question about eM Client and where to find my Send-Later emails. The default answered with three wrong statements in one response. I checked it in Gemini, and Gemini instantly called out the errors, gave the help files, and noted that Chat was confused by assuming all emails were IMAP, when that was clearly not a safe assumption. I deleted the Chat GPT thread, and started over with the same prompt I had given Gemini. The default doubled down on the wrong answer. I checked its sources. None of them supported the wrong answer. I deleted the thread and switched my model to 5.4. That model quickly noted the errors, but missed some of the better thinking of Gemini. Then I deleted the thread and gave the same request for a double check to 5.2. 5.2 was more concise and readable, 100% correct in its statements, but still lacked Gemini's insight. Then I deleted the thread and switched back to the default. The model swore there was nothing wrong and doubled down on the wrong answers.
Separately, I asked the default for the total sales tax rate in Laurinburg, NC over multiple days. It incorrectly answers 7%. 5.4 gives the correct answer, as does 5.2 and Gemini. The correct answer is 6.75%. When the default last answered 7%, I checked its sources. It gave two unofficial .com websites, nothing terribly authoritative. Indeed, one of the websites reported incorrectly that it was 7.0%, but that same website a few lines down reported correctly, 6.75%. The other source cited stated correctly, 6.75%. So among the two sources linked, the question was answered three times: once 7.0% and twice 6.75%. Chat boldly declares 7.0% as the right answer. Even when challenged, the discrepancy between the sources pointed out, told that was not the answer it gave a few months ago, etc., the default digs in, gaslights, and cannot even acknowledge uncertainty, much less the correct answer.
It's been on and on like this for a little over a week. If I am asking for information, I am now going to have to manually change my model every time, to 5.4 or 5.2.