Stealth Model 'big-pickle' Tops Mini-SWE-Agent Leaderboard
An independent evaluation shows the free stealth model outperforming established high-end models on Scale AI's SWE Atlas benchmark.
A stealth AI model known as 'big-pickle' has achieved a top-tier ranking on the SWE Atlas Codebase QnA benchmark, outscoring several established high-end models. The results indicate a significant leap in codebase reasoning capabilities for a model currently available for free.
In an independent evaluation conducted by Phillip Chaffee, big-pickle recorded a task resolve rate of 50.8%, successfully completing 63 out of 124 tasks. This performance places the model at the top of the Mini-SWE-Agent scaffold class on the official leaderboard, surpassing other entries including GLM 5.2 and GPT-5.6-Sol. The testing environment utilized the mini-swe-agent scaffold, Harbor v0.18.0, and the official Scale AI judge model, claude-opus-4-5-20251101.
Technical Performance
The model demonstrated varying levels of proficiency across different programming languages. Its strongest performance was in TypeScript, where it achieved a 58.1% resolve rate, followed closely by Python at 55.2%. Performance in Go stood at 50.0%, while C proved the most challenging, with a resolve rate of 38.5%.
Benchmark Context
SWE Atlas is a specialized benchmark developed by Scale AI to measure how large language models handle complex software engineering tasks, specifically focusing on codebase question-and-answering. To ensure a fair comparison between first-party and third-party models, the 'Mini-SWE-Agent' is employed. This is a minimal, bash-only scaffold that strips away proprietary optimizations, forcing models to rely on their core reasoning and tool-use capabilities rather than sophisticated external orchestration.
Industry Implications
These results are significant because they suggest that a stealth model—currently offered for free via OpenCode Zen—can rival or exceed the performance of industry leaders like GPT-5.x in complex software engineering environments. When restricted to the same minimal scaffolding, the gap between proprietary giants and emerging stealth models appears to be narrowing, potentially lowering the barrier for high-performance codebase intelligence.
Future Outlook
As big-pickle remains in its stealth period, the identity of its developers and the underlying architecture remain officially unconfirmed. Observers will be watching to see if these results hold across larger, more diverse benchmarks or if the model's performance is specific to the SWE Atlas environment. For now, the model remains accessible to the public through OpenCode Zen.