ON THE BOREL AND VON NEUMANN POKER MODELS Comparison with Real Poker Real Poker: Around 2.6 million possible hands for 5 card stud Hands somewhat independent for Texas Hold ‘em Let’s assume probability of hands comes from a uniform distribution in [0,1] Assume probabilities are independent The Poker Models La Relance Rules: Each player puts in 1 ante before seeing his number Each player then sees his/her number Player 1 chooses to bet B/fold Player 2 chooses to call/fold Whoever has the largest number wins. von Neumann Rules: Player 1 chooses to bet B/check immediately Everything else same as La Relance The Poker Models http://www.cs.virginia.edu/~mky7b/cs6501poker/ rng.html La Relance Who has the edge, P1 or P2? Why? Betting tree: La Relance The optimal strategy and value of the game: Consider the optimal strategy for player 2 first. It’s no reason for player 2 to bluff/slow roll. Assume the optimal strategy for player 2 is: Bet when Y>c Fold when Y<c Nash’s Equilibrium La Relance P2 should choose appropriate c so that P1’s decision does not affect P2: If PI has some hand X<c, the decision he makes should not affect the game’s outcome. Suppose PI bets B P1 wins 1 if P2 has Y<c (since he folds ‘optimally’) P1 loses B+1 if P2 has Y>c (since he calls ‘optimally’) Suppose P1 folds P1 wins -1 − + 1 1 − = −1 Which yields: = /( + 2) La Relance We knew the optimal strategy for P2 is to always bet when Y>c. Assume the optimal strategy for player I is: Bet when X>c (No reason to fold when X>c since P2 always folds when Y<c) Bet with a certain probability p when X<c (Bluff) Now PI should choose p so that P2’s decision is indifferent: ≝ (1 | < ) Using Bayes’ theorem: < 1 = + (1 − ) La Relance Consider P2’s Decision at Y=c: If P2 calls with Y=c, he/she wins pot if X<c and loses if X>c: 2 = + 1 × < 1 − ( > |1 ) If P2 folds, Value for P2 is -1. Solve the equation: + 1 × < 1 − ( > |1 ) = −1 We get: = 1 − = 2/( + 2) La Relance Now we can compute the value of the game as we did in AKQ game: 1 2 =− ( + 2)2 Result shows the game favors P2. La Relance When to bluff if P1 gets a number X<c? P1 bluffs with c2<X<c, (best hand not betting), bets with X>c and folds with X<c2. Why? Intuitively, If P2 is playing with the optimal strategy, how to choose when to bluff is not relevant. This penalizes when P2 is not following the optimal strategy. La Relance What if player / opponent is suboptimal? Assumed Strategy player 1 should always bet if X > m, fold otherwise player 2 should always call if Y > n, fold otherwise, Also call if n > m is known (why?) Assume decisions are not random beyond cards dealt Alternate Derivations Follow La Relance La Relance (Player 2 strategy) La Relance (Player 2 strategy) What can you infer from the properties of this function? What if m ≈ 0? What if m ≈ 1? La Relance (Player 1 response) Player 1 does not have a good response strategy (why?) La Relance (Player 1 Strategy) Let’s assume player 2 doesn’t always bet when n > m This function is always increasing, is zero at n = β / (β + 2) What should player 1 do? La Relance (Player 1 Strategy) If n is large enough, P1 should always bet (why?) If n is small however, bet when m > What if n = β / (β + 2) exactly? Von Neumann Betting tree: Von Neumann Von Neumann Since P1 can check, now he gets positive value out of the game P1 now bluff with the worst hand. Why? On the bluff part, it’s irrelevant to choose which section of (0,a) to use if P2 calls (P2 calls only when Y>c) On the check part, it’s relevant because results are compared right away. Von Neumann Nash’s equilibrium: Three key points: P1’s view: P2 should be indifferent between folding/calling with a hand of Y=c + 1 − ( + 1) 1 − = −1 P2’s view: P1 should be indifferent between checking and betting with X=a − +1 1− = P2’s view: P1 should be indifferent between checking and betting with X=b + +1 − − +1 1− = Von Neumann What if player / opponent is suboptimal? Assumed Strategy Player 1 Bet if X < a or X > b, Check otherwise Player 2 Call if Y > c, fold otherwise If c is known, Player 1 wants to keep a < c and b > c Von Neumann Von Neumann Von Neumann (Player 1 Strategy) Find the maximum of the payoff function a= b= What can we conclude here? Von Neumann (Player 2 Response) Player 2 does not have a good response strategy Von Neumann (Player 2 Strategy) This analysis is very similar to Borel Poker’s player 1 strategy, won’t go in depth here… c= Bellman & Blackwell Bet tree Where 0 ≤ 1 ≤ 2 Borel: B1= B2 Von Neumann: B1= 0 Bellman & Blackwell mL Fold mH Low B b1 High B b3 Low B b2 High B Bellman & Blackwell Or if Where La Relance: Non-identical Distribution Still follows the similar pattern Where F and G are distributions of P1 and P2, c is still the threshold point for P2. π is still the probability that P1 bets when he has X<c. What if ? La Relance: (negative) Dependent hands X and Y conforms to FGM distribution Marginal distributions are still uniform. is correlation factor. < 0 means negative correlation. La Relance: (negative) Dependent hands Player 1 bets when X > l P(Y < c | X = l) = B / (B + 2) Player 2 bets when Y > c (2*B + 2)*P(X > c|Y = c) = (B + 2)*P(X > l|Y = c) Game Value: P(X > Y) – P(Y < X) + B * [ P(c < Y < X) – P(l < X < Y AND Y > c) ] + 2 * [ P(X < Y < c AND X > l) – P(Y < X < l) ] Von Neumann: Non-identical Distribution Also similar to before (just substitute the distribution functions) a | (B + 2) * G(c) = 2 * G(a) + B b | 2 * G(b) = G(c) + 1 c | (B + 2) * F(a) = B * (1 – F(b)) Von Neumann: (negative) Dependent hands Player 2 Optimal Strategy: Player 1 Optimal Strategy: Discussion / Thoughts / Questions Is this a good model for poker?