Humans

At the height of its popularity, Party Poker accounted for nearly half of all online poker revenue. Due to the large number of active games, high stakes offered, and notoriously soft player base, it was mecca for a growing community of online professionals trying to take advantage of the poker boom. Party Poker did not, however, offer heads-up cash games at this time, and so while one on one poker was occasionally played online or at a snail’s pace in physical casinos, heads-up play remained relatively unexplored until 2006, when an ideologically motivated US senator suddenly affixed the Unlawful Internet Gambling Enforcement Act to a ‘must pass’ security bill. The UIGEA created a dubious legal environment for online poker operators in the US and Party Poker withdrew from the market, losing the majority of its market share overnight. Full Tilt Poker, a competing website which offered heads-up cash games, elected to remain in the US market and absorbed much of Party’s uprooted American player base. For the first time since the onset of the boom, a large number of players were competing in an environment where heads-up play was readily available.

At the point of Full Tilt’s rise there existed extraordinarily few heads-up limit hold’em players who would have been considered proficient or even competent by present day standards. This is not to say that all players were equally bad, of course. Some were worse than others and the most skilled players were still the best in the business. As more and more competition was brought to bear, however, thousands of players began recording millions of hands and the fundamentals of heads-up play became increasingly well understood. Analysts did their part to suggest improvements and evolution did the rest, as those who frequently made large errors tended to lose money and status and those who developed more robust habits tended to gain money and status. Students of the game copied the proven winners, either by seeking them out on internet forums or coaching websites, and within just a couple years the collective store of common knowledge about what made for good, basic play progressed from a tremendous pile of bullshit to something not far off current standards. By the time the second Man vs Machine match took place in 2008 there existed a handful of players whose skill has not been dramatically improved on since. Since then, the number of heads-up limit hold’em players who could be described as ‘very good, if imperfect’ has grown to many hundreds, if not thousands. Beyond that initial foray into the wilderness there have been no startling overturns of major conventions and it seems, now that we have Cepheus to verify, that human scrutiny coupled with evolutionary pressure did a decent job of getting the fundamentals right.

Robots

No human player will ever beat Cepheus at heads-up limit hold’em. Cepheus is not the best heads-up limit hold’em player in the world. To understand why this is not a contradiction, a person needs to first remember that poker is not chess, backgammon, or StarCraft, where a player either wins or loses a match. The objective of poker is to maximize one’s winnings. It is also important to understand what ‘winning’ means in a poker context. Absolutely anyone can potentially win chips from Cepheus or any other poker player. Just sit down, win a hand, and then leave. ‘Winning’, in the sense of playing better than one’s opponent, means choosing plays which, if repeated an infinite number of times, would always come out ahead. Being ‘the winning player’ means winning in a hypothetical long run, even if you end up losing right now. While this may seem a rather academic way to think about winning and losing it is in fact the ONLY way to think about winning in poker, and here’s why:

Suppose you are playing a simple game of poker against a simple AI. The AI is always dealt either a king or a two and you are always dealt a queen. The AI will always raise all-in when it holds a king and will do the same 50% of the time when it is dealt a two. You then get to decide whether you should call or fold. What should you do? Since the AI will have a king 67% of the time that it raises you should fold your queen 100% of the time. You can’t call the times that it holds a two because it’s impossible to distinguish between the time the AI raises with a king and a two. This principle remains unchanged in much of human on human play. The idea that poker players have the ability to ‘see the souls’ of their opponents is a largely fictitious one, perpetrated largely because people don’t know any better and because it makes for good movies and good TV. In reality, professional poker players almost never try and lie to each other, they simply make assumptions about what their opponents are likely to do with different types of hands and play the odds accordingly. Whether a pre-flop raise means that a player is holding a pair of aces or a pair of kings is a sort of quantum event, where the final outcome is impossible to measure until it is at last observed.

The concept underpinning the program that built Cepheus is, in a sense, relatively simple. A ‘dummy’ AI is generated and begins playing poker against a copy of itself, exploring the ways it might play each hand. It then reviews the hands, and every time that it discovers an action it regrets it reduces the frequency with which it takes that action, implicitly increasing the likelihood that it takes some other action. While this description forgoes the details, the end result is that the dummy AI continues to play and evolve until it eventually arrives at a strategy which it no longer wishes to change, a ‘best’ strategy where any change would increase regret. This strategy is Cepheus. Since poker is a symmetrical game, the end strategy which Cepheus plays is an unbeatable one. While chips can, of course, be won from Cepheus in the short term, there is no decision which can be made against it which will be a winner in the long term. If a perfect opponent, either human or computerized, were to play a semi-infinite number of hands against Cepheus the best possible result would be for them to break even. Any imperfect opponent, which unfortunately includes all human players, would make mistakes along the way and lose. That being said, what Cepheus cannot do is maximize its winnings against weak opponents, a skill which humans excel. Cepheus is simply an invincible, immovable bunker, a Maginot Line that actually works.

The Consequences

So what does the availability of Cepheus’ data mean to limit hold’em play, particularly in the online environment where there are no effective checks against referencing Cepheus while play is ongoing? Not a great, deal unfortunately. While Cepheus would have undoubtedly had a detrimental and traumatic effect on a competitive online environment there is effectively no environment left to traumatize. Due to the rake, which is the share of the pot which the house claims as its fee, poker is a negative sum game. As the fundamental of heads up limit hold’em became better understood and the skill gap between competitors narrowed, many players found themselves in a position where they were able to beat their opponents but not both their opponents and the rake. More and more often, competition between players began to result in both players losing and the situation was exasperated by the decline of the online poker industry, which shifted a large portion of competitive play to lower stakes where the rake represents a larger percentage of a player’s potential winnings. Poker players, being rational people, did the only sane thing they could do, which was decline to play anyone who appeared to be of even remotely similar skill. At of the time of writing this article on a Saturday evening there are, on Pokerstars, the current market leader, thirty-five heads-up limit hold’em tables above the one dollar level where players are waiting for an opponent and one table at which two players are actually competing. Cepheus will undoubtedly prove a valuable sparring partner and research tool for casino players and enthusiasts looking to sharpen their skills, but the heyday of heads-up cash play has, unfortunately, already passed.