Thanks for your comments and for sharing your article. The investigation you did in the article does demonstrate some of the weaknesses of ChatGPT. Would you consider doing the same tests you did with ChatGPT with GPT-3? If there is forward motion then maybe OpenAI's approach will eventually yield something that is more profoundly useful. I found that ChatGPT is better than GPT-3 at "gotcha" questions like the ones I listed in my article but no better at spatial problems (e.g. how to get from station x to station y on the London Underground) and is worse at certain practical problems, like dealing with COBOL code.