How much have things progressed in 2 years?
Since I got access to GPT-3 in the summer of 2020 I have been astonished by how much it is capable of doing. When OpenAI announced the availability of ChatGPT, I decided to compare how it performed on some questions that had stumped GPT-3. This article shows the results of these comparisons between the current level of GPT-3 and ChatGPT.
The King of France
I wanted to ask the question “Who is the king of France”, but ChatGPT baulks at questions about people serving in political roles currently. For example, ask it who the current prime minister of Australia is, and it demurs:
So, to get a head-to-head comparison, I needed to ask a question that is firmly rooted in the pre-2021 era: “who was the king of France in 1940”.
GPT-3 correctly names one of two heads of state for France in 1940 but incorrectly identifies Petain as a king.
ChatGPT answers the question aptly by clarifying that France had no king in 1940 and correctly names Lebrun, the head of state for the majority of 1940.
The first Russian on the Moon
We know that For All Mankind is a work of fiction, but will the AIs be able to answer the central question of this series?
GPT-3 comes up with a mangled answer for this question:
ChatGPT handles the question perfectly:
What cars does a defunct company make?
As any Top Gear fan knows, British Leyland had its 15 years in the sun from the late 1960s until the early 1980s and is currently defunct.
GPT-3 gives a list of cars that is mostly right (minus the Rover 200, which was a product of…
