Verbalized Bayesian Persuasion

本文是对论文的通俗导读，由 Claude Code 生成，作者审校。

TL;DR

• What: The first framework that extends Bayesian persuasion to real natural-language dialogues, with convergence guarantees.
• How: BP is mapped to a verbalized mediator-augmented extensive-form game where both sender and receiver are LLMs, and equilibrium finding becomes prompt optimization via a prompt-space response oracle (PSRO with OPRO for static games, FunSearch for multistage games).
• Verbalized game theory: BP’s two signature ingredients are re-implemented in language: the commitment assumption is written into the receiver’s prompt as the sender’s known writing-style distribution, and the obedience constraint becomes a sampled penalty estimated by a theory-of-mind LLM.
• Validation: In classic settings, VBP matches the analytically optimal Bayes correlated equilibrium; in free-form and multistage settings, which numeric solvers cannot even represent, it discovers effective persuasion strategies.
• Bonus finding: RLHF alignment reshapes strategic behavior. Aligned LLMs oscillate between honesty and deception before converging to optimal honesty. The oscillation is driven by alignment’s normative pressure, not by strategic cycling; unaligned models don’t oscillate, but don’t find the optimum either.

Intuition

Persuasion is estimated to account for as much as a quarter of economic activity, and economists have had an elegant theory of it since 2011: Bayesian persuasion (BP). A professor writes recommendation letters to influence an HR department; a prosecutor selects evidence to influence a judge. The theory tells the sender exactly how much to reveal, and how much to shade the truth, to get what they want from a rational receiver.

But the theory only speaks in bits. The famous recommendation-letter example compresses a student into “strong or weak” and the letter into “recommend or not.” Everything that makes real persuasion interesting (what to emphasize, what to omit, how to phrase it) is abstracted away before the math even starts.

This work makes the theory speak in full sentences. An LLM professor writes actual letters about actual student profiles; an LLM HR officer reads them and decides. A game solver then optimizes how the letter is written. The trick that makes this tractable: instead of optimizing the LLM’s impossibly high-dimensional parameters, we optimize in prompt space. The sender’s “signaling scheme” becomes the writing-style instructions in its prompt, a compact, discrete strategy space that classic equilibrium machinery can actually search.

The textbook result survives the translation into language: in the courtroom game, the optimal prosecutor lies with probability exactly $3/7$, and an 8B LLM rediscovers this equilibrium through prompt optimization.

Resources

• Verbalized Bayesian Persuasion.
  Wenhao Li, Yue Lin, Xiangfeng Wang, Bo Jin, Hongyuan Zha, Baoxiang Wang.
  International Conference on Machine Learning (ICML) 2026. Poster.

  [Paper]

Related reading on this blog: Information Design in 10 Minutes, Information Design in MARL, Cheap Talk.

What We Did

1. The Problem

In BP, a sender with private information about a state $\omega$ commits to a signaling scheme $\pi: \Omega \to \Delta(\mathcal{S})$; the receiver observes the signal, Bayes-updates, and best-responds. The classic solution maximizes the sender’s utility subject to obedience constraints on the receiver’s utility $u_1$:

\[\sum_{\omega} P(\omega)\, \pi(a \mid \omega) \left[ u_1(a,\omega) - u_1(a',\omega) \right] \ge 0, \quad \forall a, a'.\]

Moving this into natural language breaks everything at once: the signal space becomes infinite-dimensional (all possible letters), the revelation principle no longer applies cleanly, and if the players are LLMs, their strategies live in a non-convex parameter space where a Nash equilibrium may not even exist.

2. The Framework

Verbalized BP: classic BP plus natural-language dialogue is verbalized into a mediator-augmented game with a convergence guarantee, then solved by approximate best responses in a prompt-space response oracle.

• Verbalized mediator-augmented game. BP is recast as a mediator-augmented extensive-form game with the sender as mediator, then verbalized: states are text (a full student profile), signals are text (an actual letter), rewards stay numerical, and both players are instantiated by Llama-3.1-8B.
• Prompt-space response oracle (Prompt-PSRO). Strategy optimization becomes prompt optimization. PSRO maintains a growing pool of prompt strategies, solves the induced meta-game, and queries an LLM-based oracle for approximate best responses. OPRO generates prompt categories and contents for static settings, while FunSearch evolves a prompt-generating function of the interaction history for multistage settings, enabling strategies like “build trust early, exploit it later.”
• Verbalized commitment. The sender’s optimized writing-style prompts and their selection probabilities are written into the receiver’s prompt, so the receiver can genuinely Bayes-update against a known scheme; this is what separates persuasion from cheap talk.
• Verbalized obedience constraint. The constraint above is turned into a penalty term estimated by sampling: a theory-of-mind LLM predicts the receiver’s likely reaction to each signal.
• Information obfuscation. Aligned LLMs resist being strategic. A few rounds of self-reflection, guided by an LLM scoring how much state information each signal hides, warm up the sender before the main loop.

The solver provably returns an $\varepsilon$-approximate Bayes correlated equilibrium in static BP and an $\varepsilon$-approximate Bayes–Nash equilibrium in multistage BP.

3. What Happens

Three classic problems (recommendation letters, courtroom, law enforcement) in three settings of increasing realism:

• S1 (polarized signals): free-form signals are snapped to the nearest extreme so results can be checked against theory. Exploitability drops to about $0.1$ within 10 PSRO iterations, and VBP matches the analytical optimum and a MARL solver on rewards, lie rates, and honesty rates. The courtroom equilibrium is reproduced: lying with probability $3/7$, the prosecutor gets the judge to convict 60% of defendants even though 70% are innocent.
• S2 (free-form language): signals stay real letters, evidence presentations, patrol announcements, a regime no numeric baseline can enter. Performance stays roughly on par with S1.
• S3 (multistage): the same receiver keeps the full interaction history over 5 rounds. The sender’s deception now goes through bargaining-like oscillations: honest at first, then discovering profitable deception, then dialing it back after receiver retaliation, before converging to low lie rates with honesty trending upward.

Notable Findings

• Alignment has a dual effect on strategic play. Aligned models oscillate (honesty rises, falls, rises again) under normative pressure but end at the optimal honesty level; an unaligned LLaMA-3 8B shows no oscillation yet stabilizes away from the optimum.
• Commitment emerges endogenously. Ablating the explicitly verbalized commitment barely changes the converged behavior, even in the static setting. One candidate explanation, echoing the economics literature, is that reputation-like pressure during the repeated learning interactions does the committing.
• The obedience constraint is the load-bearing piece. Removing it significantly harms convergence of the lie probability; removing information obfuscation barely matters, because the sender learns to withhold and obfuscate on its own.
• Multistage persuasion starts to look like bargaining. Once the receiver can ignore signals and retaliate across rounds, commitment stops being unilateral; it must be accepted. This observation is developed into its own framework in our follow-up work, Information Bargaining.