Buying Licenses Isn't Adoption: Why AI Usage Metrics Mislead Executives

Organizations are rushing to procure AI tools to avoid falling behind, but they frequently confuse the act of buying licenses with actual adoption. Harshal Shah, an Engineering Manager at Webflow, argues that while usage can be tracked through dashboards, true adoption requires a fundamental shift in organizational norms and workflows.

Writing for The New Stack, Shah posits that the distinction between usage and adoption is critical for technical leaders. He notes that buying licenses is essentially a budget conversation, whereas adoption is a cultural change. According to Shah, usage metrics—such as seat activations or token spend—can be easily manipulated. He warns that if leadership signals that usage is a performance metric, employees will increase their activity to meet expectations, regardless of whether the tool provides actual value. "If I tell my team that usage is a performance signal, usage goes up inside a week, and I’ve learned nothing except that they can read incentives," Shah writes.

The Cultural Hurdle of Adoption

The gap between usage and adoption exists because the former is a technical hurdle while the latter is a psychological one. True adoption requires an organization to admit that its previous methods of working were inferior and to accept the inherent instability that accompanies the implementation of new workflows. This process is often avoided in favor of the easier path: purchasing seats for Large Language Models (LLMs) and AI assistants to create a veneer of progress.

Shah emphasizes that this transition is rarely seamless. He explains that changing a norm means "saying out loud that the old way was worse, then owning it when the new way breaks something in week three." Without this willingness to endure temporary instability and acknowledge legacy failures, tools remain peripheral rather than integrated into the core value stream of the company.

The Risk of Misleading Metrics

For executives and technical leaders, relying on prompt counts or license activation rates to measure AI success is a dangerous strategy. When these metrics are used as proxies for progress, they create a false sense of security. The organization may appear to be evolving on paper while continuing to operate under legacy paradigms in practice.

This disconnect ultimately limits the return on investment for AI spending. When tools are used superficially to satisfy a mandate rather than adopted to solve a structural problem, the company fails to realize the efficiency gains promised by the technology. The result is a proliferation of tools that are technically available but functionally ignored in the daily habits of the workforce.

What to Watch

As enterprises move past the initial "land grab" phase of AI procurement, the focus must shift from procurement to integration. The primary indicator of success will not be how many licenses are active, but whether the organization has successfully redefined its internal workflows. Leaders should look for evidence of shifted norms and the willingness to iterate through the "week three" failures that Shah describes, rather than relying on inflated usage data.