AI Marketing Emails Match Human Performance, Chicago Booth Study Finds

AI-generated marketing copy is as effective as professional human writing, according to a new study that suggests a radical shift in digital marketing economics. Researchers found that companies can significantly reduce overhead costs by automating email campaigns without sacrificing conversion rates.

In a series of three randomized controlled trials conducted over one year, researchers from the University of Chicago Booth School of Business partnered with online wine retailer Wine Access. The study, involving approximately 27,500 customers, compared the performance of human-written emails, AI-generated versions, and a hybrid approach where AI drafted content for human editors. The results showed that all three styles roughly doubled purchase rates and gross profit per customer when compared to a control group that received no newsletters.

The Economics of Automation

The study, led by Jean-Pierre Dubé and PhD student Ariel Xu, tested various AI implementations, including a custom model trained on five years of the company's top-performing emails and a simple prompt using Anthropic's Claude. The financial implications were stark: full automation via a trained model could reduce writing costs by approximately 90%, while using Claude would cut those costs by 83%.

For Wine Access specifically, the researchers projected that switching to Claude for automation would increase annual profits from $5 million under human writers to $5.5 million. This efficiency gain stems from the fact that human writers typically spent three hours composing each email, whereas AI can produce similar results nearly instantaneously.

Industry Implications

This research provides empirical evidence that large language models (LLMs) can successfully adopt a specific brand voice and match the commercial performance of professional copywriters. The primary value proposition for businesses is no longer just the speed of production, but the radical reduction of operational overhead.

Because the AI-generated content performed on par with professional staff, Dubé and Xu noted that there is scope for companies to downsize writing teams and shift creative content design to AI. This suggests that the traditional role of the marketing copywriter is facing a structural challenge as the cost of high-converting text drops toward zero.

What's Next

While the study confirms that AI can match human effectiveness in a commercial setting, the long-term impact on brand loyalty and creative innovation remains to be seen. The findings indicate that for direct-response marketing—where the goal is a specific action like a purchase—AI is already a viable replacement for human labor. Industry observers will now be watching to see if these results hold across different product categories and more complex customer journeys.