Going back underground with OpenAI’s latest model
Back in 2020 I wrote an article about navigating the London Underground using GPT-3. Back then all kinds of people were getting GPT-3 to solve all kinds of problems, but nobody seemed to be testing it on spatial problems. I decided to see whether GPT-3 could accurately generate instructions for trips between stations on the London Underground. As I described in the 2020 article, GPT-3 didn’t do a very good job of navigating the Underground, but I was impressed that it was able to get any of the trips right. When OpenAI made GPT-4 available yesterday I decided to go back Underground to see if LLMs had made any progress on spatial problems in the last two years. In this article I ask GPT-3.5 and GPT-4 how to get between a variety of stations on the Tube and compare the results.
Starting easy — one line trip
For the first trip I picked a simple, one line trip: Earl’s Court to Green Park:
GPT-3.5 faffs about with an extraneous second route that involves a train change, and gets the number of intermediate stations wrong, but other than that is correct. Also, it’s handy to have the line colours…
