PokerBot： Create your poker AI bot in Python

转载自：https://www.data-blogger.com/2017/11/01/pokerbot-create-your-poker-ai-bot-in-python/

Kevin Jacobs
发表于

In this tutorial, you will learn step-by-step how to implement a poker bot in Python.

Step 1 – Setup Python and Install Packages

First, we need an engine in which we can simulate our poker bot. Install the following package (PyPokerEngine) using pip:

1
pip install PyPokerEngine

It also has a GUI available which can graphically display a game. If you are interested, you can optionally install the following package (PyPokerGUI):

Both the engine and the GUI have excellent tutorials on their GitHub pages in how to use them. The choice for the engine (and/or the GUI) is arbitrary and can be replaced by any engine (and/or GUI) you like. However, the implementation of the bot in this tutorial depends on this choice, so you need to rewrite some if the code if you plan to change the engine (and/or GUI).

Small note on the GUI: it did not work for my directly using Python 3. This fix explains how to make it work.

 

Step 2 – Implement your Bot!

The first step is to setup the skeleton of the code such that it works. In order to do so, I created three files. One file containing the code for the bot (databloggerbot.py), another file containing the code for a bot which always calls and another file for simulating one game of poker (simulate.py) in which many runs are simulated. The files initially have the following contents:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
from pypokerengine.engine.hand_evaluator import HandEvaluator
from pypokerengine.players import BasePokerPlayer
from pypokerengine.utils.card_utils import _pick_unused_card, _fill_community_card, gen_cards


# Estimate the ratio of winning games given the current state of the game
def estimate_win_rate(nb_simulation, nb_player, hole_card, community_card=None):
 if not community_card: community_card = []

 # Make lists of Card objects out of the list of cards
 community_card = gen_cards(community_card)
 hole_card = gen_cards(hole_card)

 # Estimate the win count by doing a Monte Carlo simulation
 win_count = sum([montecarlo_simulation(nb_player, hole_card, community_card) for _ in range(nb_simulation)])
 return 1.0 * win_count / nb_simulation


def montecarlo_simulation(nb_player, hole_card, community_card):
 # Do a Monte Carlo simulation given the current state of the game by evaluating the hands
 community_card = _fill_community_card(community_card, used_card=hole_card + community_card)
 unused_cards = _pick_unused_card((nb_player - 1) * 2, hole_card + community_card)
 opponents_hole = [unused_cards[2 * i:2 * i + 2] for i in range(nb_player - 1)]
 opponents_score = [HandEvaluator.eval_hand(hole, community_card) for hole in opponents_hole]
 my_score = HandEvaluator.eval_hand(hole_card, community_card)
 return 1 if my_score >= max(opponents_score) else 0


class DataBloggerBot(BasePokerPlayer):
 def __init__(self):
 super().__init__()
 self.wins = 0
 self.losses = 0

 def declare_action(self, valid_actions, hole_card, round_state):
 # Estimate the win rate
 win_rate = estimate_win_rate(100, self.num_players, hole_card, round_state['community_card'])

 # Check whether it is possible to call
 can_call = len([item for item in valid_actions if item['action'] == 'call']) > 0
 if can_call:
 # If so, compute the amount that needs to be called
 call_amount = [item for item in valid_actions if item['action'] == 'call'][0]['amount']
 else:
 call_amount = 0

 amount = None

 # If the win rate is large enough, then raise
 if win_rate > 0.5:
 raise_amount_options = [item for item in valid_actions if item['action'] == 'raise'][0]['amount']
 if win_rate > 0.85:
 # If it is extremely likely to win, then raise as much as possible
 action = 'raise'
 amount = raise_amount_options['max']
 elif win_rate > 0.75:
 # If it is likely to win, then raise by the minimum amount possible
 action = 'raise'
 amount = raise_amount_options['min']
 else:
 # If there is a chance to win, then call
 action = 'call'
 else:
 action = 'call' if can_call and call_amount == 0 else 'fold'

 # Set the amount
 if amount is None:
 items = [item for item in valid_actions if item['action'] == action]
 amount = items[0]['amount']

 return action, amount

 def receive_game_start_message(self, game_info):
 self.num_players = game_info['player_num']

 def receive_round_start_message(self, round_count, hole_card, seats):
 pass

 def receive_street_start_message(self, street, round_state):
 pass

 def receive_game_update_message(self, action, round_state):
 pass

 def receive_round_result_message(self, winners, hand_info, round_state):
 is_winner = self.uuid in [item['uuid'] for item in winners]
 self.wins += int(is_winner)
 self.losses += int(not is_winner)


def setup_ai():
 return DataBloggerBot()

databloggerbot.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
from pypokerengine.players import BasePokerPlayer
import numpy as np
from sklearn.neural_network import MLPRegressor


class CallBot(BasePokerPlayer):
 def declare_action(self, valid_actions, hole_card, round_state):
 actions = [item for item in valid_actions if item['action'] in ['call']]
 return list(np.random.choice(actions).values())

 def receive_game_start_message(self, game_info):
 pass

 def receive_round_start_message(self, round_count, hole_card, seats):
 pass

 def receive_street_start_message(self, street, round_state):
 pass

 def receive_game_update_message(self, action, round_state):
 pass

 def receive_round_result_message(self, winners, hand_info, round_state):
 pass


def setup_ai():
 return CallBot()

callbot.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
from pypokerengine.api.game import start_poker, setup_config

from callbot import CallBot
from databloggerbot import DataBloggerBot
import numpy as np

if __name__ == '__main__':
 blogger_bot = DataBloggerBot()

 # The stack log contains the stacks of the Data Blogger bot after each game (the initial stack is 100)
 stack_log = []
 for round in range(1000):
 p1, p2 = blogger_bot, CallBot()

 config = setup_config(max_round=5, initial_stack=100, small_blind_amount=5)
 config.register_player(name="p1", algorithm=p1)
 config.register_player(name="p2", algorithm=p2)
 game_result = start_poker(config, verbose=0)

 stack_log.append([player['stack'] for player in game_result['players'] if player['uuid'] == blogger_bot.uuid])
 print('Avg. stack:', '%d' % (int(np.mean(stack_log))))

simulate.py

The bot uses Monte Carlo simulations running from a given state. Suppose you start with 2 high cards (two Kings for example), then the chances are high that you will win. The Monte Carlo simulation then simulates a given number of games from that point and evaluates which percentage of games you will win given these cards. If another King shows during the flop, then your chance of winning will increase. The Monte Carlo simulation starting at that point, will yield a higher winning probability since you will win more games on average.

If we run the simulations, you can see that the bot based on Monte Carlo simulations outperforms the always calling bot. If you start with a stack of $100,-, you will on average end with a stack of $120,- (when playing against the always-calling bot).

It is also possible to play against our bot in the GUI. You first need to setup the configuration (as described in the Git repository) and you can then run the following command to start up the GUI:

1
pypokergui serve poker_conf.yaml --port 8000 --speed moderate

Good luck beating the bot!

Conclusion (TL;DR)

In this simple tutorial, we created a bot based on Monte Carlo simulations. In a later blog post, we will implement a more sophisticated Poker bot using different AI methods. If you have any preference for an AI method, please let me know in the comments!