This work is an interactive game application for human versus AI play. It features an easy-to-use GUI and an AI opponent that continuously improves its performance using reinforcement learning.
This work included four major components: a poker game model, an AI bot, a controller, and a user interface.
The game model defines all the objects and interactions used to play the game of poker, including pocket cards, wagers, and betting statistics. The bot uses reinforcement learning to develop a knowledge base of what actions maximize monetary reward. The controller then takes input from either the bot or the human player and uses it to manipulate the game model. Finally, the user interface visually reflects changes in the model while allowing further input.
In this work, my primary role was to develop and program the bot.
To have the program emulate a human player, it required a policy to map the current state of the environment (i.e. the model) to all possible actions and their expected rewards. In PokerBot, we abstracted the environment to a metric of the player’s current hand strength with possible actions being to fold, check, call, or raise.
Using a Monte Carlo method, the bot begins play via random sampling of possible actions. As games unfold, it will be rewarded or penalized for its actions proportionally to the amount of money it has lost or gained by each game’s end.
The average sampled rewards for each state/action pair it encounters gives the bot an approximation for what possible choice would produce the highest monetary benefit at any state of a game.
To encourage semi-exploratory behavior, the bot pseudorandomly chooses from a weighted matrix of actions such that the “best” action has an increased probability of being chosen, but the algorithm may still take risks to find an even more rewarding strategy.
For more information and installation instructions, see our project website.