How much have things progressed in 2 years?
Since I got access to GPT-3 in the summer of 2020 I have been astonished by how much it is capable of doing. When OpenAI announced the availability of ChatGPT, I decided to compare how it performed on some questions that had stumped GPT-3. This article shows the results of these comparisons between the current level of GPT-3 and ChatGPT.
The King of France
I wanted to ask the question “Who is the king of France”, but ChatGPT baulks at questions about people serving in political roles currently. For example, ask it who the current prime minister of Australia is, and it demurs:
So, to get a head-to-head comparison, I needed to ask a question that is firmly rooted in the pre-2021 era: “who was the king of France in 1940”.
GPT-3 correctly names one of two heads of state for France in 1940 but incorrectly identifies Petain as a king.
ChatGPT answers the question aptly by clarifying that France had no king in 1940 and correctly names Lebrun, the head of state for the majority of 1940.
