In a recent thread on Hacker News a commenter wrote, “I don't want to read what you didn't write,” expressing frustration that readers increasingly encounter AI‑generated text presented as human work. The thread attracted 180 comments, many echoing the concern that it has become hard to tell whether a piece originated with a person or a language model and that the uncertainty is often used as an excuse for lazy comprehension. This pattern is not unique to AI; it appears whenever a cheap copy can be mistaken for a costly original because verifying provenance is expensive or absent, and both the maker of the copy and the receiver benefit from treating the copy as genuine.
Consider a medieval guild that stamped its members’ work with a quality mark to guarantee that a piece met the guild’s standards. Producing a genuine mark required apprenticeship, material inspection, and a fee that only a qualified craftsman could afford. Counterfeiters, however, could forge the mark at negligible cost. A buyer who could not readily inspect the interior of a metalwork piece relied on the visible stamp as a shortcut to trust. The forger gained by selling inferior goods at the price of authentic work; the buyer gained by avoiding the time‑consuming task of expert inspection. When the cost of verification exceeded the benefit of detecting a forgery, the guild’s mark ceased to reliably signal quality, and the market filled with mislabeled goods.
A similar divergence arose in the nineteenth‑century United States with the explosion of patent medicines. Manufacturers mixed alcohol, vegetable extracts, and sometimes harmful substances into bottles labeled with extravagant curative claims. Producing the genuine remedy—through rigorous chemical analysis and clinical testing—was expensive and slow. Slapping a bold label and a testimonial onto a cheap mixture cost almost nothing. Consumers, desperate for relief and lacking the means to evaluate chemical composition, relied on the label’s promise as a proxy for efficacy. The manufacturer profited from selling a useless or dangerous product at the price of a real cure; the consumer profited from avoiding the effort of seeking a physician or conducting personal trials. When the official verification system—the nascent food‑and‑drug inspection—was weak or absent, the label’s authority eroded, and the market saturated with nostrums that bore little relation to their advertised benefits.
In the twentieth century, the rating of complex financial securities offered another illustration. Agencies such as Moody’s and Standard & Poor’s assigned AAA grades to tranches of mortgage‑backed securities that bundled thousands of home loans. Assigning a trustworthy rating required detailed loan‑by‑loan analysis, stress testing under varied economic scenarios, and transparent disclosure of models and assumptions—activities that demanded substantial expertise and time. The agencies, however, were paid by the very banks that issued the securities, creating a revenue stream that grew with the volume of ratings granted. Producing a cautious, low rating would have reduced that income, while a generous AAA rating could be issued with comparatively little additional work, relying on opaque models and the assumption that housing prices would continue to rise. Investors, faced with voluminous prospectuses and limited capacity to scrutinize each underlying mortgage, used the AAA stamp as a heuristic for safety. The rating agencies benefited from higher fees; investors benefited from a quick decision rule that spared them deep due diligence. When the actual risk of mortgage defaults rose, the coupling between the rating and the underlying security broke down, and the market suffered a cascade of losses that the stamp had failed to predict.
The same logic appears in the natural world. Certain harmless butterflies, such as the viceroy, mimic the wing pattern of the toxic monarch butterfly. Producing the genuine toxin‑based defense requires metabolic pathways that are costly to maintain. Evolving a visual resemblance that predators associate with unpleasant taste is comparatively cheap. Predators that have learned to avoid the monarch’s pattern rely on that cue as a shortcut to avoid poisoning; they do not typically capture and test each butterfly to confirm its toxicity. The viceroy gains by enjoying reduced predation without investing in toxin synthesis; the predator gains by avoiding the handling time and risk of sampling a potentially dangerous prey. When the mimic becomes abundant enough that predators encounter more harmless imitations than toxic models, the warning signal loses its reliability, and predation on the mimic rises—a breakdown of the signaling coupling analogous to the erosion of trust in human institutions.
Across these cases the mechanism is constant: a producer can create a low‑cost imitation of a high‑cost authentic signal; the consumer relies on that signal as a economizing heuristic because direct verification is costly or impossible; the producer gains by passing the imitation off as authentic; the consumer gains by avoiding verification effort; and when the proportion of imitations rises sufficiently, the signal’s predictive power collapses, leaving the consumer to bear the cost of mistaken trust. The signal itself—whether a guild hallmark, a patent‑medicine label, a financial rating, or a wing pattern—does not disappear; rather, its link to the underlying quality is weakened by the incentive to copy and the incentive to accept copies at face value.
The present debate about AI‑generated text follows this pattern. Producing a genuinely human‑authored essay demands time, domain knowledge, and stylistic effort that many contributors are unwilling or unable to expend. Generating plausible text with a large language model requires only a prompt and modest computational power, making the imitation inexpensive. Readers who lack the time or expertise to discern subtle markers of machine origin treat the fluent output as if it were human work, because doing so saves the cognitive labor of close reading or fact‑checking. The model’s provider (or the user who deploys it) benefits from the appearance of authorship without the associated cost, while the reader benefits from a ready‑made answer that can be consumed with minimal effort. As more AI‑generated passages circulate, the heuristic that “fluency equals human authorship” becomes less reliable, and frustration grows—as expressed in the Hacker News comment—when readers discover they have been consuming material they did not intend to read.
The system does not depend on any particular technology, date, or industry; it emerges whenever the cost of authenticating a signal exceeds the cost of fabricating it, and where both sides of the exchange find advantage in treating the imitation as genuine. Historical examples from guilds, patent medicines, financial ratings, and biological mimicry all exhibit the same incentive structure and the same point of failure: the decoupling of a signal from its provenance due to asymmetric effort in production and verification. Recognizing this recurrence helps to see why calls for better detection tools, stricter labeling, or increased verification costs are responses to the same underlying tension, rather than isolated reactions to a novel AI trend. The challenge remains to align the incentives of signal producers and consumers so that the signal once again reflects the underlying quality it purports to represent.