The Problem
Course-of-action (CoA) generation — producing a small set of viable, strategically distinct plans for a commander or an autonomous mission planner to choose from — is a core planning task in military and security operations. Doing it for a drone swarm facing an adaptive adversary raises the difficulty considerably:
- Adaptive adversaries — A fixed, hand-scripted opponent strategy is easy to plan against but unrealistic. A genuinely adaptive adversary changes its behavior in response to the swarm’s tactics, so any CoA generator trained against a static opponent risks producing plans that fail the moment a real adversary deviates from script.
- Communication-limited environments — Swarm coordination algorithms typically assume reliable inter-agent communication. Real deployments — especially maritime and contested electromagnetic environments — routinely degrade or sever those links, and a CoA that only works with full communication is of limited operational value.
- Scaling from tens to hundreds of agents — Strategies that look reasonable in small-scale simulation can break down entirely as swarm size grows, since the space of joint behaviors and possible failure modes expands with the number of agents.
Game-theoretic approaches are a natural fit for the adaptive-adversary piece of this problem: rather than training against one fixed opponent, they optimize policies against an evolving population of opponents, aiming for strategies that stay robust across the range of adversary behaviors an opponent might actually adopt.
Our Approach: UC-PSRO
UC-PSRO builds on Policy-Space Response Oracles (PSRO), a well-established multi-agent reinforcement learning framework that iteratively grows a population of policies and computes best responses against it — converging toward strategies that are robust across the population rather than overfit to any single opponent. UC-PSRO adapts this idea specifically for swarm course-of-action generation under degraded communication.
Rather than treating adversary modeling, strategy selection, and communication robustness as separate concerns bolted together, UC-PSRO integrates three components into a single training loop, each targeting a different failure mode a real swarm deployment can encounter.
Three Framework Components
Mutual Policy Optimization via Self-Play
Swarm and adversary policies are trained against each other through iterative self-play, in the standard PSRO tradition — each side’s policy improves in response to the current best strategies of the other, rather than against a fixed script. This is what lets the resulting course of action generalize to adversaries that adapt in the field, instead of only to the specific behavior seen during training.
Adaptive Policy Steering via Weighted Utility Conditioning
Rather than optimizing a single scalar objective, UC-PSRO conditions policy selection on weighted utility signals, allowing the framework to steer toward courses of action that trade off different mission priorities — this is the "utility-conditioned" piece of the name, and it’s what lets a single trained system produce a family of distinct, strategically meaningful CoAs rather than one-size-fits-all output.
Communication-Dropout Curriculum
Robustness to degraded communication is trained explicitly, via a curriculum that progressively simulates communication failure during training — starting from reliable links and ramping up disruption over time, rather than testing robustness only at evaluation. This component turned out to matter far more than the paper’s authors initially expected.
Results — and a Counterintuitive Finding
The framework was evaluated on a maritime swarm scenario spanning 25 to 200 agents — large enough to expose scaling behavior that smaller test beds tend to miss. The headline result isn't simply "the full framework wins"; it's more interesting than that:
Used on its own, the communication-dropout curriculum was the most reliable single component — success rate counterintuitively improved from 35% to 62% as communication disruption intensified. That’s the opposite of what most swarm coordination work would predict, and it suggests the curriculum doesn’t just make policies tolerate dropped links — it actively pushes them toward strategies that are structurally simpler and less dependent on tight coordination in the first place, which happen to hold up better precisely when communication gets worse.
Combining all three components, however, substantially slowed convergence within a fixed training budget, without a demonstrable gain in exploitability over fixed-opponent baselines. In plain terms: stacking self-play and utility conditioning on top of the comms curriculum made training harder without making the resulting strategies measurably harder for an adversary to exploit. It’s a useful negative result — it tells us the comms-dropout curriculum is carrying most of the practical value here, and that naively combining game-theoretic components doesn’t automatically compound their benefits within a limited compute budget.
Why This Matters
Federal & Maritime Security
Course-of-action generation for swarm operations in contested, communication-degraded maritime environments — directly relevant to federal reconnaissance and force-protection missions.
Communication-Denied Operations
A concrete demonstration that training explicitly for communication dropout, rather than assuming reliable links, is the highest-leverage way to build swarm robustness under realistic field conditions.
Compute-Efficient Training
The finding that combining components can slow convergence without payoff is directly actionable for teams with fixed training budgets — it argues for validating each component’s marginal contribution rather than assuming more machinery is strictly better.
Adaptive Adversary Modeling
Game-theoretic self-play training produces strategies that stay useful against adversaries who adapt in the field, rather than plans that only work against the specific opponent behavior seen in simulation.
What's Next
UC-PSRO's clearest contribution is isolating which piece of a game-theoretic swarm framework actually drives robustness under realistic conditions. That result opens up several natural next steps:
- Investigating why the communication-dropout curriculum alone outperforms the combined framework, to understand whether the interaction cost is fundamental or an artifact of the current training budget
- Testing the curriculum-first finding on non-maritime swarm scenarios to see how broadly it generalizes
- Extending evaluation beyond exploitability to operationally grounded metrics that matter for real course-of-action selection
- Integrating findings with our broader Drone AI Systems research on swarm coordination robust to jamming, spoofing, and adversarial interference
Read the Full Paper
UC-PSRO: Utility-Conditioned Policy-Space Response Oracles with a Communication-Dropout Curriculum for Game-Theoretic Course-of-Action Generation in Adversarial Swarms
Phillip Jiang · arXiv 2608.15372 · 2026