9.11 vs 9.9: the model that read a Bible verse
Language models have long claimed that 9.11 is greater than 9.9. In October 2024, Transluce showed on Llama-3.1-8B-Instruct that the question activates neurons linked to Bible verses and dates; by deactivating them, the model answers correctly. It was not a calculation error: it was a reference lookup.
This is a short excerpt.
The full version, with its sources, and the podcast are for subscribers. The account is free.