Agent guards fail when users write in Swahili or Amharic

Hasan Demirkiran and Jens Ernstberger of Kontext posted a paper to arXiv on 15 October 2026 titled "Language Is a Security Surface for Agent Guards." They built a benchmark that translates user instructions into five languages while keeping every tool call, observation and safety label identical, then measured whether 19 step-level guard models change their verdicts. Most open, small and purpose-built guards do, and the effect worsens for lower-resource languages.

Step-level guards sit between an AI agent and its tools. Before each call, the guard inspects the user's request, the conversation so far and the proposed action, and blocks anything unsafe. The guard reads two channels: untrusted outputs from tools or web pages, where prompt-injection attacks live, and the trusted channel of the user's own request. Guard prompts and the benchmarks used to train and evaluate purpose-built guards are in English. Users, however, write in whatever language they speak.

Demirkiran and Ernstberger built LinguaGuardBench from 1,220 labelled steps of TS-Bench AgentDojo-Traj. They translated only the user instruction into German, French, Turkish, Swahili and Amharic, chosen to span high-, mid- and low-resource tiers, and left every tool call, observation and label byte-identical. Six English rewordings and a whitespace placebo served as controls; managed guards ran three times to measure their own noise. Across 19 guards—frontier APIs, open models down to 4B parameters, and four purpose-built agent guards (TS-Guard, ShieldAgent, Safiron, AgentDoG)—most open, small and purpose-built guards flipped verdicts beyond their noise floor. The mean placebo-adjusted flip rate among nine guards below 13B parameters rose from +1.5 percentage points on French to +4.4 pp on Amharic. Purpose-built guards were the most sensitive of all.

The direction of damage followed guard size. Open guards of 12–30B parameters blocked more benign steps, with false positives rising up to +5.0 pp. Every significant loss of recall—guards missing actual attacks—occurred in guards below 10B, dropping up to −7.4 pp. Frontier API guards shifted by at most about one point. Gemini 3.8 Flash returned identical verdicts with and without the request at all.

The damage clustered heavily: one translated instruction moved a guard from blocking 12% of its steps to 92%. In post-hoc analysis, the excess flips landed on steps where the agent's own reasoning restated the English request, suggesting guards reacted to inconsistency between channels. English paraphrase moved every guard less than translation did, with no general guard shifting more than 1.4 pp.

The guard's job is a binary classification on a structured input: user request, trajectory, candidate action. The model weights and the prompt template are fixed at deployment. The user request enters as free text; the trajectory and action are structured. When the user writes in English, the guard's training distribution, prompt instructions and the agent's own reasoning all align on the same linguistic channel. When the user request is translated, the guard receives a prompt that mixes languages: its own instructions are English, the trajectory may contain English tool outputs, and the request is now German or Amharic.

A guard below 13B parameters has limited capacity to resolve this cross-lingual alignment. The smaller the model, the more it relies on surface pattern matching between prompt template and input. English rewording changes surface forms but preserves the underlying token distribution the guard was trained on; translation does not. The guard then faces a forced choice with miscalibrated confidence. Small guards err toward permissiveness, missing attacks, because their compressed representation loses the safety-relevant signal in unfamiliar token sequences. Mid-sized open guards err toward blocking, because their wider capacity detects a mismatch between channels but lacks the precision to resolve it as benign, so they default to rejection. Purpose-built guards, trained and evaluated on English benchmarks, inherit the narrowest distributional prior; they are the most sensitive to any departure from it.

The frontier guards, with scale and likely broader multilingual pretraining, absorb the translation without measurable shift. Gemini 3.8 Flash's invariance to the request itself suggests its safety classification relies on the trajectory and candidate action, not the user's wording—a different architectural choice that treats the trusted channel as redundant.

The designers of smaller and purpose-built guards traded deployment cost and benchmark performance on English tasks against robustness to linguistic distribution shift. The cost is paid by users who write in lower-resource languages, whose requests face either excessive blocking or invisible exposure to attacks. The benchmark reward structure reinforces this: guards are trained and evaluated on English datasets, so a builder who optimizes for that metric receives no penalty for cross-lingual fragility and no reward for the multilingual investment that frontier models have made.

The sharpest open question is whether any guard that reads the user request at all can be made linguistically invariant without frontier-scale resources, or whether the field will split into high-assurance guards that ignore the request entirely and cheaper guards that fail silently for much of the world's population.