In distributed and federated learning, many workers contribute updates and a server aggregates them. What if some workers are compromised and send poisoned updates? Naïve averaging fails badly — a few bad actors can wreck the model. In Byzantine Resilience With Reputation Scores (2022) we defend by having the system learn whom to trust: workers accrue reputation, and the aggregate discounts the untrustworthy. Robustness becomes something the system infers over time, not a fixed assumption.
Distributed training is also plagued by asynchrony — workers finish at different times, so updates arrive stale. It’s tempting to think this must hurt generalization. In Distributed SGD Generalizes Well Under Asynchrony (2019) we showed the opposite can hold: done right, asynchronous training still generalizes, which is what makes large-scale distributed learning practical.
At the edge you often can’t separate “collect data” from “make decisions” — you must learn while acting, paying for every mistake. That’s the bandit setting, and it shows up everywhere once you look:
That last point matters. At the edge, the environment is rarely stationary, so a good learner has to weigh fresh evidence against stale.
The systems we increasingly rely on — recommendation, payments, wireless, federated models trained on our devices — are distributed, adversarial, and non-stationary by nature. Algorithms that assume clean, centralized, i.i.d. data quietly fail in exactly the places they’re deployed. Designing for untrustworthiness, lag, and drift is not hardening an ideal system; it’s building the real one.
Three forces make edge learning central to the next decade:
The through-line across all of this — and across much of my group’s theory, from iterated random operators to the convergence guarantees underneath these algorithms — is asking not just does it work? but does it still work when the setting is distributed, adversarial, and drifting? Increasingly, that is the only question that matters.
Papers behind this post: Byzantine Resilience With Reputation Scores (2022) · Distributed SGD Generalizes Well Under Asynchrony (2019) · Maximizing success rate of payment routing using non-stationary bandits (2023) · Interference constrained beam alignment … via kernelized bandits (2022) · Weighted Gaussian process bandits for non-stationary environments (2022). See the Publications page.
]]>.bib file,
push, and the Publications page
rebuilds — grouped by year, split into journal and conference work, with links
and copy-paste BibTeX.If you’re a student curious about the group’s work, the Research and Projects pages are the best place to start. More notes to come.
]]>For bimatrix games, tractability tracks a quantity called the rank of the game (roughly, the rank of the sum of the two payoff matrices). Rank-0 (zero-sum) and rank-1 games are solvable with linear programming; from rank-2 upward, the problem is PPAD-hard. So the natural question is: how many games are secretly easy?
Our answer is strategic equivalence. Two games are strategically equivalent if they have the same best-response structure — and therefore the same equilibria — even if their payoff matrices look different. In Rank reduction in bimatrix games (2023) and A fast algorithm to reduce $2\times n$ bimatrix games to rank-1 games (2019), we show that many apparently hard, high-rank games are strategically equivalent to a rank-1 game, and can therefore be solved with a linear program. This effectively enlarges the class of games we know how to solve efficiently.
When exact solutions are out of reach, approximation takes over: Two Algorithms for Computing Exact and Approximate Nash Equilibria in Bimatrix Games (2021) gives practical methods, built on the idea of a best-response bijection.
Equilibria depend not just on payoffs but on who knows what. In Information structures and values in zero-sum stochastic games (2017) we study how the value of a game shifts as you change the players’ information — a bridge between the game-theory and team-theory threads of my work.
If you cannot compute an equilibrium, you cannot use it — to predict behavior, to design a mechanism, or to train an agent. Mapping the boundary between tractable and intractable games, and finding transformations that move a problem to the easy side of that boundary, is what makes game theory operational.
Multi-agent systems are having a moment. RL agents are trained against each other; autonomous systems negotiate for bandwidth, road space, and compute; LLM agents increasingly interact strategically. Under the hood, all of this is equilibrium computation — and the PPAD wall is real. The practical path forward is exactly the one this work explores: identify the structure that makes your game tractable, or transform it until that structure appears. As we build economies of interacting AI agents, knowing which games we can actually solve stops being an academic curiosity and becomes an engineering constraint.
Papers behind this post: Rank reduction in bimatrix games (2023) · A fast algorithm to reduce $2\times n$ bimatrix games to rank-1 games (2019) · Two Algorithms for Computing Exact and Approximate Nash Equilibria in Bimatrix Games (2021) · Information structures and values in zero-sum stochastic games (2017). See them on the Publications page.
]]>Write the update as
\[x_{k+1} = T_k(x_k),\]where each $T_k$ is a random operator drawn from some distribution, acting on a point $x_k$ in a Polish space $\mathcal{X}$. In the ideal, noiseless world we would iterate a single deterministic contraction $T$ with a unique fixed point $x^\star$. The question is whether the random iteration stays close to that ideal as we collect more data.
The usual Banach fixed-point argument wants a metric under which $T$ contracts. But for several algorithms of interest, the natural notion of “getting closer” is measured by a divergence, not a metric — it need not be symmetric and need not satisfy the triangle inequality. To handle this we introduced a Wasserstein divergence between probability measures over $\mathcal{X}$ and gave sufficient conditions under which contraction under this divergence still yields a limit.
With that machinery, you can treat the algorithm’s iterates as a Markov chain and ask about its limiting behavior directly:
Answering these gives consistency results for a whole family of algorithms at once, including settings with continuous state and action spaces where classical guarantees are hard to come by.
If you want the details, the relevant papers are on the Publications page — look for probabilistic contraction analysis of iterated random operators and convergence of recursive stochastic algorithms using Wasserstein divergence.
]]>The core idea: if you’ve already solved a dynamic program, and the new one is a small perturbation of it, your old solution is a great starting point. In An Algorithm to Warm Start Perturbed (WASP) Constrained Dynamic Programs (2022) we make that precise for constrained problems, and in A Computationally Efficient Algorithm for Perturbed Dynamic Programs (A-PDP) (2022) we push the efficiency further. Informally, if the optimal value function is $V^\star$ and the problem shifts by a small $\varepsilon$, you want the cost of finding the new $V^\star_\varepsilon$ to scale with $\varepsilon$ — not with the cost of solving from scratch. That is the difference between an algorithm that runs on-board and one that doesn’t.
Some systems have dynamics on two very different timescales — fast electrical transients riding under slow mechanical motion. Naïvely these singularly perturbed problems are stiff and hard. In Discrete-time finite-horizon optimization of singularly perturbed nonlinear control systems with state-action constraints (2023) we exploit that timescale separation instead of fighting it, turning one intractable problem into two tractable ones.
This is not abstract. The real-time eco-driving controller I’ve written about elsewhere — Real-Time Ecodriving Control … Using Approximate Dynamic Programming (2022) — is exactly this philosophy in a vehicle: the driving problem changes every second as traffic evolves, and warm-starting is what lets the optimizer keep up on embedded hardware.
The bottleneck in deploying optimal control is rarely the math of the optimum — it is the compute budget for re-deriving it as conditions change. Perturbation methods attack that bottleneck directly, and they do it with guarantees on how the solution changes, not just heuristics.
Everything is trending toward continuous, real-time optimization under changing conditions:
The quiet lesson is that in a live system, the valuable skill isn’t solving the problem — it’s cheaply updating the solution you already have.
Papers behind this post: An Algorithm to Warm Start Perturbed (WASP) Constrained Dynamic Programs (2022) · A Computationally Efficient Algorithm for Perturbed Dynamic Programs (A-PDP) (2022) · Discrete-time finite-horizon optimization of singularly perturbed nonlinear control systems (2023) · Real-Time Ecodriving Control … Using Approximate Dynamic Programming (2022). See the Publications page.
]]>Demand in a city is lopsided: some neighborhoods and some hours generate far more trips, and travel times are asymmetric. Naïve dynamic pricing can entrench those asymmetries. In Fair Pricing of Ridehailing Services with Asymmetric Demand and Travel Time (2021) we formalize fairness constraints and derive prices that respect them, showing you don’t have to throw away efficiency to get there. The point is that “fair” stops being a slogan and becomes a constraint you can design around.
Seamless mobility usually requires several operators — a rideshare company, a transit agency, a bike network — to cooperate, and each is out for itself. In Incentive design and profit sharing in multi-modal transportation networks (2022) we design the incentives and profit-sharing rules that make cooperation each operator’s best move, so a rider can go door-to-door across modes that would otherwise never coordinate.
Cars drift to where the last trips ended, not where the next ones will start. In Multi-objective vehicle rebalancing for ridehailing systems using a reinforcement learning approach (2022) we learn where to reposition idle vehicles, balancing competing objectives — rider wait time, driver earnings, efficiency. And in Fleet sizing and charger allocation in electric vehicle sharing systems (2022) we plan the harder, slower decisions: how many electric vehicles, and where to put the chargers.
Mobility is being rewritten by software, and the rules encoded in that software have real distributional consequences — for riders in underserved areas and for drivers whose livelihoods depend on the matching and pricing logic. Making fairness a formal objective, rather than an afterthought, is how you keep those consequences from being accidental.
Three shifts make this work more relevant, not less:
As transportation, energy, and labor markets fuse into one algorithmic system, the question “what is a fair price for a ride?” turns out to be a question about what kind of city we want the algorithms to build.
Papers behind this post: Fair Pricing of Ridehailing Services with Asymmetric Demand and Travel Time (2021) · Incentive design and profit sharing in multi-modal transportation networks (2022) · Multi-objective vehicle rebalancing for ridehailing system using a reinforcement learning approach (2022) · Fleet sizing and charger allocation in electric vehicle sharing systems (2022). See them on the Publications page and the transportation-markets project.
]]>A recurring thread of my work establishes when team-optimal strategies exist. In On the existence of optimal policies for a class of static and sequential dynamic teams (2015) and Existence of Team-Optimal Solutions in Static Teams with Common Information (2020), we develop a topology-of-information approach: treat each agent’s strategy as shaping a probability measure and study continuity in the right topology. That machinery rests on a genuinely foundational result — The topology of information on the space of probability measures over Polish spaces (2014) — and extends to Teams with Countable Observation Spaces (2021). Existence sounds abstract, but without it, “find the optimal decentralized policy” is not even a well-posed request.
When agents know different things, equilibria are hard to compute. The common information approach cuts through this: reformulate the problem around what everyone knows in common, and a coordinator’s problem emerges that is tractable. We used it to characterize Common information based Markov perfect equilibria for linear-Gaussian games (2014) and for finite games (2014), and to design incentives in Dynamic incentive design in multi-stage linear-Gaussian games with asymmetric information (2014).
If agents can talk, which links actually matter? In Sketching for Elimination of Communication Links in LQG Teams (2021) and Communication Link Elimination in Static LQG Teams (2018) we ask when a communication channel can be removed with negligible loss — a direct handle on the trade-off between coordination and bandwidth.
Any system where autonomy is distributed lives here: you cannot centralize every decision, so agents must act locally toward a global goal. The results say when that is possible, how to structure the reasoning (around common information), and which communication is worth its cost.
We are building multi-agent systems at speed — swarms of robots, fleets of autonomous vehicles, and now teams of LLM-based agents collaborating on tasks. Every one of them faces the team-theory questions in disguise:
Decades before “agentic AI” was a phrase, decentralized control was working out its foundations. As we hand more collective decisions to machines that each see only part of the picture, that foundation is exactly what keeps the picture coherent.
Papers behind this post: On the existence of optimal policies for a class of static and sequential dynamic teams (2015) · Existence of Team-Optimal Solutions in Static Teams with Common Information (2020) · Existence of Team-Optimal Strategies in Teams with Countable Observation Spaces (2021) · Common information based Markov perfect equilibria for linear-Gaussian games (2014) · The topology of information on the space of probability measures over Polish spaces (2014) · Sketching for Elimination of Communication Links in LQG Teams (2021). See the Publications page.
]]>When states and actions form a continuum, value iteration becomes an approximation problem. In Fitted Value Iteration in Continuous MDPs With State Dependent Action Sets (2021) we handle a wrinkle most treatments ignore: the feasible actions change with the state — a car near a wall simply cannot turn as hard. Earlier, An empirical relative value learning algorithm for non-parametric MDPs with continuous state space (2019) and An empirical algorithm for relative value iteration for average-cost MDPs (2015) built empirical dynamic programming: replace exact expectations with samples and ask when the sampled algorithm still converges to the truth as data grows.
Increasingly you can’t explore — you have a fixed log of past behavior and must learn a good policy from it. That’s offline RL, and it’s fragile: the policy wants to try actions the data never covered. In Finite sample analysis of a minmax variant of offline reinforcement learning for general MDPs (2022) we gave finite-sample guarantees — not “it worked,” but bounds on how much data buys how much performance.
Two more gaps between theory and deployment:
Risk. Optimizing the average can be reckless. Robustness to Modeling Errors in Risk-Sensitive Markov Decision Problems with Markov Risk Measures (2025) asks for policies that stay good even when your model is a little wrong and you care about the tail, not the mean: roughly,
\[\min_\pi \; \rho\big(\text{cost}\big) \quad\text{subject to model error},\]where $\rho$ is a risk measure rather than an expectation.
These are the exact failure modes that separate a benchmark score from a deployable system. Continuous dynamics, offline data, hard constraints, and tail risk are not edge cases — they are what “the real world” means for a decision-making agent.
The most consequential RL today trains large language models from logged human feedback — an offline RL problem with an enormous action space, wrapped in safety constraints, where average-case tuning can hide rare but serious failures. The questions we’ve studied in control-theoretic clothing — when does learning from fixed data generalize? how do you respect constraints while learning? how do you guard against model error and the tail? — are precisely the questions facing anyone deploying a learning agent that acts in the world. The vocabulary differs; the mathematics is the same.
Papers behind this post: Fitted Value Iteration in Continuous MDPs With State Dependent Action Sets (2021) · Finite sample analysis of minmax variant of offline reinforcement learning for general MDPs (2022) · Learning in Constrained Markov Decision Processes (2022) · Robustness to Modeling Errors in Risk-Sensitive Markov Decision Problems (2025) · An empirical relative value learning algorithm for non-parametric MDPs (2019). See the Publications page.
]]>Our starting point was auction design. In Auctioning electricity under deep renewable integration using a penalty for shortfall (2019) and its predecessor Selling Renewable Generation with a Penalty for Shortfall (2018), a renewable generator sells energy it might produce, backed by a penalty if it falls short. Using stochastic programming and auction theory, we found something clean: the optimal contracted quantity is a function of the inverse CDF of the renewable supply,
\[q^\star = F^{-1}(\cdot),\]and pairing that allocation with a Myerson-style payment rule makes truthful bidding the buyers’ best strategy. In other words, you can design the market so that honesty is optimal even when supply is random.
The theme continues in Equilibria in two-stage electricity markets (2015) and Dynamic Economic Dispatch … under Ramping Constraints and Uncertain Demand (2018), which study how prices and dispatch evolve when both supply and demand are uncertain and generators can’t ramp instantly.
The other half of the answer is that demand no longer has to be passive. Electric-vehicle charging is a huge, flexible load — it mostly cares that the car is charged by morning, not about the exact hour. In Scheduling EV charging having demand with different reliability constraints (2023) and Preemptive scheduling of EV charging for providing demand response services (2023), we schedule that flexibility to soak up renewable energy when it’s abundant, turning millions of cars into a grid-stabilizing resource. The idea goes back to Scheduling, pricing, and efficiency of non-preemptive flexible loads (2015).
Decarbonizing the grid is not only a hardware problem — it’s a market design problem. Panels and turbines are necessary; mechanisms that make variable supply and flexible demand meet efficiently are what make them usable at scale. Good mechanism design here is worth gigawatts.
As renewables become the cheapest generation on Earth, the binding constraint shifts from cost to coordination: matching stochastic supply with shiftable demand, moment to moment, without blackouts or waste. The principles compound into the near future:
Getting the rules right is quietly one of the highest-leverage things we can do for the energy transition.
Papers behind this post: Auctioning electricity under deep renewable integration using a penalty for shortfall (2019) · Selling Renewable Generation with a Penalty for Shortfall (2018) · Equilibria in two-stage electricity markets (2015) · Dynamic Economic Dispatch … under Ramping Constraints and Uncertain Demand (2018) · Scheduling EV charging having demand with different reliability constraints (2023) · Preemptive scheduling of EV charging (2023). See them on the Publications page and the renewable-markets project.
]]>The classic model of allocating limited resources across many contested fronts is the Colonel Blotto game. We used it as a lens on cyberphysical security in A Three-Stage Colonel Blotto Game with Applications to Cyber-Physical Security (2014) and its companions. A striking, counter-intuitive result runs through this work — captured in the title When to provide more information to an adversary (2014): sometimes revealing information is the optimal move, because it shapes the attacker’s incentives in your favor. Later, in Colonel Blotto Game with Coalition Formation for Sharing Resources (2018), we asked when defenders should pool resources at all.
A related thread treats communication and control under attack as a dynamic game. Across Optimal control in the presence of an intelligent jammer (2010), One-stage control over an adversarial channel (2011), A dynamic transmitter-jammer game with asymmetric information (2012), and Jamming in mobile networks (2013), the recurring theme is asymmetric information: the two sides know different things, and the equilibrium hinges on who knows what. That culminated in Dynamic Games With Asymmetric Information and Resource Constrained Players With Applications to Security of Cyberphysical Systems (2017).
The same framing illuminates privacy. In Privacy-aware stochastic control with a “snoopy” adversary (2016), a controller must accomplish its task while an eavesdropper tries to infer private state from observable behavior. The tension is exactly a game: every action leaks a little information, so the optimal policy trades performance against how much it reveals.
Three implications carry into today’s world:
The unifying idea is simple to state and hard to internalize: design against the best response, not the average case. $\max_{\text{defense}} \min_{\text{attack}}$ is a better mental model for security than any fixed threat list.
Papers behind this post: Dynamic Games With Asymmetric Information … Security of Cyberphysical Systems (2017) · A Three-Stage Colonel Blotto Game … Cyber-Physical Security (2014) · … When to provide more information to an adversary (2014) · Colonel Blotto Game with Coalition Formation (2018) · Privacy-aware stochastic control with a “snoopy” adversary (2016) · the jammer-game series (2010–2013). Browse them on the Publications page.
]]>