Executive TL;DR:

• LLMs are structurally incapable of creating new foundational axioms in certain domains.
• Lack of native adjacency resolution makes problems harder and sample inefficient.
• Experts debate the role of sensory experience in creative leaps.

The Buzz Score

The Internet’s Verdict: 70% Hyped, 30% Skeptical

Expert Insights

One expert notes that LLMs can’t jump to new conclusions without sufficient data:

I felt almost certain that the author would have used Judea Pearl’s ladder of causation but they did not. Would have probably been a better argument to make.

Another expert proposes an experiment:

create an LLM from all text up to 1980 or 1990 and see if it can get back to making itself.

Some argue that LLMs can make creative leaps in abstract fields without sensory grounding:

why could a sufficiently advanced LLM not develop an equivalent high-dimensional topology for domains like physics and use it to make creative leaps?

Focus Keyword: LLM Limitations