Executive TL;DR:

• Models are being designed to be more focused and less general-purpose.
• This shift is driven by the need for more accurate and reliable information.
• New approaches, such as pluggable knowledge bases, are being explored.

The Buzz Score

The Internet’s Verdict: 70% Hyped, 30% Skeptical

Expert Insights

Some experts believe that models should be designed to be more focused and less general-purpose. As one expert notes,

Ideally what I’d like to see is pluggable knowledge bases. So if I’m e.g. coding a SwiftUI app for navigation, I’d take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge.

Others are exploring new approaches, such as tool-calling focused models and reasoning over world knowledge.

Combine these two approaches with a reliable search tool/a safe way of accessing the internet for the model, and you’ve got a probably slightly slower model for factual questions, which on the upside however doesn’t hallucinate.

Challenges and Limitations

However, some experts are skeptical about the feasibility of these new approaches. As one expert notes,

Man, both that Blogpost and some comments here read like science-fiction. Specifically, creative writing driven by nerds dreaming about a future, without proper grounding in reality, constraints and all that stuff.

Focus Keyword: Intelligent Models