How computers solved poker

The strongest poker programs were never taught strategy. They play against themselves, and keep score of their regrets.

Take the bluffing game from the Nash topic. Two learners start badly: one bluffs with almost every miss, the other almost never calls.

After each round, each learner asks of every action: how much better would that have done than what I actually did? That gap is its regret.

Next round, each plays every action in proportion to its regret. Calling would have paid, so calling rises. Then bluffing stops paying, and falls.

The strategies keep chasing each other and never settle. But their average does.

After a few thousand rounds the average sits on the equilibrium: a third of bets are bluffs, and calls come half the time. Nobody gave it the answer.

Counterfactual regret minimization runs this at every decision in poker at once. Cepheus, Libratus and Pluribus all computed their strategies with versions of it.

Start the learners anywhere you like. Wherever they begin, the average finds the same point.