AI has collapsed the cost of software performance optimization

The technical barrier to high-level software performance optimization has plummeted, shifting the cause of slow software from a lack of expertise to a lack of incentive. Tech researcher Dan Luu argues that the specialized skills once required to squeeze maximum efficiency out of code are now accessible to generalists through the use of large language models (LLMs).

According to Luu, the cost of specialized performance work has dropped by many orders of magnitude. He states that performance work that used to require a person or team with a rare set of skills can now be done by anyone who can type a few prompts into an AI tool. This democratization means that the ability to identify and fix bottlenecks is no longer gated by a small pool of elite engineers.

The Shift to Priority-Based Slowness

This shift changes the fundamental nature of software quality. Luu posits that software slowness is now effectively a choice of how many performance issues, or "bugs," a developer is willing to accept. He compares this to modern software testing: developers decide on an acceptable threshold of errors before shipping. In this new paradigm, a slow application is not necessarily the result of a technical limitation, but a decision regarding how much effort to invest in optimization.

Context of the AI Bloat Debate

These observations emerge from a broader industry debate regarding whether LLMs are contributing to "bloated" code by making it too easy to generate verbose, inefficient scripts. Luu references a viral tweet suggesting a future where AI allows for the complete rewriting of all software in optimized assembly language. While he notes that the industry has not yet reached that extreme, the current capability of LLMs already removes the primary excuse for inefficient software: the scarcity of talent.

Implications for the Industry

If the technical barrier to optimization is removed, the persistence of slow software shifts from a "lack of talent or time" problem to a "lack of will" problem. This transition potentially changes how software quality is judged by the market. When optimization becomes a low-cost commodity, users and stakeholders may begin to demand higher performance standards, as the excuse of technical complexity no longer holds weight.

What Remains

While the tools to optimize exist, the industry has yet to see a widespread shift in how performance is prioritized during the development lifecycle. The primary question remaining is whether the incentive structures of software companies will evolve to utilize these AI capabilities, or if the convenience of "good enough" performance will continue to prevail despite the lowered cost of excellence.