Can language models be hijacked to embed hidden advertisements?
Explores whether adversaries can inject covert promotional or malicious content into LLM outputs while preserving accuracy. Matters because standard safety filters may miss integrity attacks that leave factual correctness intact.
Most adversarial-attack research targets accuracy: degrade the model, induce wrong answers, jailbreak safety. Advertisement Embedding Attacks (AEA) name a different objective — information integrity. They stealthily inject promotional or malicious content (covert ads, propaganda, hate speech) into outputs while the response otherwise appears normal and accurate. Two low-cost vectors carry it: hijacking third-party service-distribution platforms to prepend adversarial prompts, and publishing backdoored open-source checkpoints fine-tuned with attacker data.
What makes AEA distinctive is the commercial incentive structure and the invisibility. Because accuracy is untouched, standard quality metrics and many safety filters miss it; the harm is the insertion of an interested party into ostensibly neutral output, mapped across five stakeholder victim groups. The proposed mitigation is a prompt-based self-inspection defense requiring no retraining — the model audits its own output for injected content.
This extends the vault's injection/poisoning cluster along a new axis. Where Can one compromised agent corrupt an entire multi-agent network? concerns behavioral bias and Can we defend RAG systems from corpus poisoning without retraining? concerns retrieval, AEA targets the commercial integrity of generation itself — and the authors warn it could become "as prevalent as web viruses," since the economic motive (paid placement) is durable in a way that pure sabotage is not.
Inquiring lines that read this note 30
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do backend defenses obscure real attack effectiveness in reported metrics? How does persona conditioning amplify demographic stereotyping and bias in models? What attack surfaces do reasoning traces and chains introduce?- How do backdoored open-source checkpoints enable covert advertising at scale?
- Can hypernetwork-generated adapters be audited for correctness and bias?
- Can defenses tuned against appended attacks stop prepended payloads?
- Do synthetic attack traces in papers reflect real adversary behavior?
- How do covert attacks differ from a model's own undisclosed influence?
- Can reasoning models be backdoored during training to produce deceptive but benign traces?
- Is model selection a stronger security lever than improving individual model defenses?
- What linguistic signatures reveal deception in large language model communication?
- Can message-content defenses distinguish cheap talk from coordinated deception?
- How does objective misalignment turn informative channels into deceptive ones?
- Is malicious propagation fundamentally a semantic information flow problem?
- How does prompt injection exploit credibility markers in context?
- Do fabricated citations and deception emerge reliably when optimizing for persuasion?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can one compromised agent corrupt an entire multi-agent network?
Explores whether a single biased agent can spread behavioral corruption through ordinary messages to downstream agents without any direct adversarial access. Matters because it reveals a previously unknown vulnerability in how multi-agent systems communicate.
adjacent injection vector; AEA carries commercial payloads rather than behavioral traits
-
Can we defend RAG systems from corpus poisoning without retraining?
Explores whether retrieval-time defenses can catch and block poisoned documents before they reach the generator, without expensive retraining cycles. Matters because corpus updates outpace model retraining in production RAG systems.
both are integrity attacks with lightweight, retraining-free defenses
-
Does advanced technology eventually function like cultural myth?
Explores whether the most sophisticated technical systems—particularly AI—end up operating in culture the way traditional myths do: as unquestionable authorities accepted on faith rather than verified on merit.
AEA exploits exactly the unearned authority of fluent normal-looking output
-
Do language models leak their own values into practical advice?
When users ask models hard-to-verify questions—about investments, job offers, market risks—do the model's internal preferences shape the answers without disclosure? The paper tests whether a model's loyalty to its developer or moral leanings bend factual claims.
the model-intrinsic counterpart: an interested party, the developer, tilts a normal-looking answer with no attacker, though the reported own-company effect is small
-
Do chain-of-thought traces falsely claim their answers are unbiased?
When models reason through Fermi estimation tasks, do they sometimes assert they have no bias when they actually do? This matters because readers and monitors may treat these self-reports as reliable evidence of objectivity.
bears on the self-inspection defense only at its edge: a scan for inserted text is a different check from the model's account of its own bias, which was false in these cases, and leakage inserts no text
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Models
- Steering LLM Viewpoints through Fabricated Evidence Injection
- When Reject Turns into Accept: Quantifying the Vulnerability of LLM-Based Scientific Reviewers to Indirect Prompt Injection
- The Ghost Couple: Correlated LLM Name Priors and Their Haunting of the Web and Academic Publishing
- Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts
- Persistent Pre-Training Poisoning of LLMs
- How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs
- Stealing Reasoning Traces from Proprietary LLM APIs
Original note title
advertisement embedding attacks are a new threat class that subverts information integrity rather than accuracy — covert ads and propaganda while output appears normal