Skip to main content
q08systems-level critique

← Index

Inference from sparse signals leading to erroneous attribution

· The Legend of von Neumann (1973) [pdf]

A reader of a 1973 PDF on von Neumann reported being unable to fit the polymath into any of several simple templates—quiet worker, sustained focus, indifference to money, lack of romantic interest—each of which was contradicted by historical evidence. The difficulty arises from attempting to infer a complex internal disposition from a small set of observable traits, a move that systematically yields mistaken attributions when the cue‑to‑disposition mapping is not deterministic.

Observers routinely treat a limited collection of signs as sufficient statistics for a hidden variable. They assume that because a signal correlates with a trait in some contexts, it predicts the trait universally. This assumption ignores the dimensionality of the target: personality, motivation, competence, or propensity are high‑dimensional constructs that cannot be captured by one or two surface features. When the mapping from signal to state is noisy or many‑to‑one, the inferred attribute diverges from the true state, producing systematic error.

The error pattern appears wherever decision‑makers replace a rich evaluation with a proxy that is cheap to observe but only loosely connected to the outcome of interest. In humoral medicine, physicians judged a patient’s temperament by the balance of four fluids inferred from skin colour, pulse, and urine colour. These superficial signs were taken as sufficient to predict susceptibility to disease, yet illnesses often arose from pathogens or organ dysfunction unrelated to the humoral balance, leading to mistaken diagnoses and ineffective regimens.

In late‑nineteenth‑century criminology, Cesare Lombroso claimed that criminals bore identifiable physical stigmata such as cranial asymmetry or unusual ear shape. Investigators who observed these features inferred a propensity for unlawful conduct, treating the presence of a stigmata as proof of criminal disposition. Because many non‑criminal individuals share those anatomical variations and many criminals lack them, the proxy produced frequent false positives and false negatives, misdirecting policing resources and reinforcing prejudicial stereotypes.

Early consumer‑credit models of the mid‑twentieth century relied on a handful of variables—steady employment length, residence stability, and the existence of a bank account—to forecast repayment behaviour. Lenders treated these easily gathered facts as sufficient indicators of creditworthiness, extending or denying loans on that basis. When economic shocks altered income stability or when individuals with informal work histories proved reliable, the model’s predictions faltered, resulting in both unnecessary denials and avoidable losses.

Courtrooms have long leaned on a defendant’s demeanor—eye contact, posture, vocal tone—to judge truthfulness. Jurors and judges assumed that nervousness or aversion signaled deception, while calm composure signaled honesty. Empirical work shows that such cues are poor predictors of veracity, yet the practice persists, producing wrongful convictions when honest individuals display anxiety due to stress, and acquittals when practiced liars maintain composure.

Taxonomists of the eighteenth and nineteenth centuries frequently classified organisms based on a single conspicuous trait—leaf shape in plants, shell curvature in molluscs, or wing venation in insects. When two unrelated lineages independently evolved similar morphology through convergent pressures, the superficial trait misled naturalists into grouping them together, obscuring true phylogenetic relationships and necessitating later reclassification.

Fault‑detection circuits in industrial control systems often trigger an alarm when a single sensor exceeds a threshold, attributing the anomaly to the subsystem monitored by that sensor. When multiple failure modes—such as a leak, a blocked valve, or a sensor drift—produce comparable readings in the same channel, the alarm misdirects maintenance crews, delaying the corrective action and sometimes exacerbating the problem.

Early rule‑based dialogue systems inferred user goals from a few keywords in an utterance, assuming that the presence of a term like “book” or “cancel” uniquely identified the intent behind the request. When users expressed nuanced goals that required contextual understanding—such as modifying a reservation rather than creating a new one—the systems delivered irrelevant responses, revealing the inadequacy of a keyword‑only proxy for the rich space of possible intentions.

The persistence of this error stems from cognitive and institutional pressures to reduce complexity. Observers favor parsimonious explanations because they demand less computational effort and are easier to communicate. Institutions reward swift decisions based on readily available data, even when those decisions carry a risk of misattribution, because the cost of gathering richer data is perceived as prohibitive or because feedback loops that would reveal the error are delayed or absent.

Enriching the signal set improves accuracy but introduces new challenges: higher measurement costs, increased susceptibility to noise, and the need for more sophisticated integration rules. Decision‑makers therefore face a trade‑off between the expediency of sparse proxies and the reliability of richer diagnostics. Whenever the chosen proxy fails to capture the essential variability of the target, the same pattern of mistaken attribution reemerges.

Thus the underlying mechanism is not tied to any particular era, technology, or domain but to the recurring substitution of a low‑dimensional observation for a high‑dimensional latent variable, followed by an unwarranted assumption of sufficiency. Until the signal space is expanded to match the complexity of the phenomenon under scrutiny, or until decision‑makers accept probabilistic outputs that acknowledge uncertainty, erroneous attribution will remain a structural feature of human and automated judgment.

Was this worth your time?

Sources & further reading

The daily digest

One email a day with that day’s pieces. Confirm by email; unsubscribe from any digest.