Leanstral 1.5: Proof Abundance for All
Executive TL;DR:
- Leanstral 1.5 aims to provide proof abundance for all users.
- The system has been tested on various libraries, including varinteger.
- The results show promise, but some users remain skeptical.
The Internet’s Verdict: 70% Hyped, 30% Skeptical
Introduction to Leanstral 1.5
Leanstral 1.5 is a new system that aims to provide proof abundance for all users. The system has been tested on various libraries, including varinteger.
Forum Voices
Some users have expressed skepticism about the system’s capabilities. For example:
One such bug was in the sign function for zigzag decoding of the datrs/varinteger library. On input Std.U64.MAX, the expression (value + 1) overflowed, causing crashes in debug mode and silent corruption in release mode—an edge case that testing and fuzzing would typically miss.
Others have questioned the significance of the results. As one user noted:
Halfway thru the article it shows a comparison with several frontier-ish LLMs. But they’re all from half a year ago. “Our new model is better than all these Chinese models from 3 generations ago” is pretty funny to me.
Technical Details
The varinteger library has been found to have a bug in the sign function for zigzag decoding. This bug causes crashes in debug mode and silent corruption in release mode.
Critical Analysis
Some users have criticized the example used to demonstrate the system’s capabilities. For example:
In what way would this boundary condition case be considered something that “testing […] would typically miss”? It’s certainly something that bad tests would miss or not think about, but I find that (a) careful people and (b) ML coding systems are actually really good at “oh, I should test the extreme values”.
Others have questioned the choice of Lean 4 for formal verification. As one user noted:
Curious that they are pitching Lean 4 for formal verification. I thought that this was more the domain of Isabelle/HOL and TLA+. At least I would have expected a model trained at using all three.
Focus Keyword: Leanstral 1.5
