Alex Elkholy .com

The four factors of TIT FOR TAT’s success are: niceness, forgiveness, provokability, and clarity. When a certain condition is met, TIT FOR TAT will thrive. However, when the circumstances are different, TIT FOR TAT will do very poorly. We will now introduce the condition which allows cooperation to begin, be maintained, and ended.

Our two gas stations from part one, Station A and Station B, are in an iterated Prisoner’s Dilemma game. Station A could lower the price (defecting) and take business from Station B. Station B would then have to lower his prices (mutual defection) in order to maintain the balance. But at this point, both parties are suffering a lower price as punishment for mutual defection. However, eventually these two gas stations will come to realize that it pays more to have mutual cooperation over the long run than it does to have mutual defection. At this point, cooperation can be established. This is because the future of their relationship casts a shadow, and the threat of poor sales is a real risk to both players.

But what if Station A was going out of business, moving, or otherwise would be aware of the ending of the game between them? The shadow of the future would no longer be looming over the present and it would pay for Station A to defect and go for the short term gain over cooperation. When Station B realizes this, it is no longer in his interests to cooperate (which would be less of a payoff than mutual defection) he also has an incentive to defect. This shadow of the future, the Discount Parameter, is what determines if cooperation is able to exist.

The reason for the name “Discount Parameter”, is because of its mathematical representation. For cooperation to exist, the future must have a weight to it. But for many reasons, the future means less than the future. People are impatient and hold the future to less value or the possibility of change (death, moving, bankruptcy, etc.) can all lower the value of the next move. It is possible to say, for whatever reason, a payoff of the next move could be 1/2 of the current move, meaning the future casts one half of the shadow as it would at full value.

This is the key to building cooperation. Within any interaction there has to be the foreshadow of future cooperation. When cooperation breaks down, often times it’s because the future no longer held any value. This concept shows through in the better designed systems. Large businesses will often group their employees together, forcing continued interaction. Politicians are faced with the threat of falling out of voter’s favor and must not defect if they want to be reelected.

If you suspect that the situation you face is a Prisoner’s Dilemma, then check the variables. Real life never has a clear cut values each person receives, payoffs are far more fluid and based on taste. Knowing if you are in a Prisoner’s Dilemma depends then on the ratio of the payoffs. If it is worth more to defect than it does to cooperate, and if mutual defection is worth more than the other defecting (when you cooperate) (T > R and P > S, respectively), then it is reasonable to suspect a Prisoner’s Dilemma. When in the Dilemma, do not assume that TIT FOR TAT will be the one true strategy. TIT FOR TAT is useful when you want to foster cooperation and when the future has enough hold on the present.

But people are more fluid, they’re not robotic like our strategies in Axelrod’s tournament. You may find that based on your goals, there is a way to achieve what you want with a different strategy, based entirely off the strategy of the other player. As the government, you probably do not want cooperation between businesses, which is regulated with antitrust laws (changing the values of the Prisoner’s Dilemma). There may be weaknesses in the strategy of the other player which you can use.

Whatever the situation you face, the most important thing to remember is the game itself. If you are unaware of the game, then you cannot play it effectively as a strategist. Thus, when playing the game of life, keep in mind the circumstances of cooperation and Prisoner’s Dilemma. If you want to read a more in depth introduction to Prisoner’s Dilemma and the topics I wrote about, then I would recommend picking up Robert Axelrod’s book,

In Robert Axelrod’s 1984 book, the Evolution of Cooperation, a tournament was described where Game Theorists could send in their favorite strategy in dealing with an iterated Prisoner’s Dilemma. The unlikely winner was the TIT FOR TAT strategy, which cooperates on the first turn and thereafter imitates the previous move of the other player. TIT FOR TAT will always cooperate until the other player defects, at which point TIT FOR TAT will punish the other player with a defection of its own the next round.

In Axelrod’s tournament, two strategies were pitted against each other for 200 rounds in Prisoner’s Dilemma. Each strategy would have the history of the previous moves available and would try to obtain the highest scores they could. The strategy which ended up being the winner in the tournament was surprisingly the strategy which never tries to exploit the other player unprovoked. While there were plenty of other strategies, many which tried to exploit the other player. Those which based their method of gaining points on the exploitation of other players found themselves doing poorly in the overall tournament. Those strategies which based their scoring more towards cooperation tended to do better. In fact so much that the top eight entries in the tournament were “nice”, and there was a significant gap in the score between “nice” strategies and “mean” strategies.

What does it mean to be a nice strategy? Simply that as long as the other player does not defect, the strategy will never defect itself. A nice strategy will always open the game with a cooperation, and until the other player defects, will maintain that cooperation. The reason the nice strategies did so well is because they were able to get along with each other, a game between two nice strategies would be constant cooperation.

Interestingly enough, a strategy similar to TIT FOR TAT, called JOSS, did significantly worse than TIT FOR TAT. JOSS behaves exactly the same was as TIT FOR TAT, except that 10% of the time it will defect unprovoked, making it different from TIT FOR TAT only in that it was no longer nice. Consider that after cooperating with TIT FOR TAT, JOSS selects a defection at random. During the next turn, TIT FOR TAT punishes the defection with a defection of its own. In response to this, JOSS does the same to TIT FOR TAT the next turn, and thus a never ending cycle is started.

This leads me to the next factor of TIT FOR TAT’s success, forgiveness. TIT FOR TAT will defect the next turn in response to a defection by the other player. After the first cooperation of the other player, TIT FOR TAT will reestablish cooperation. Contrast this with the FRIEDMAN strategy, which would permanently defect after the first defection of the other player. FRIEDMAN, while nice, was the lowest scoring of the top eight nice strategies because of this.

One other critical factor in the success of TIT FOR TAT was how easy to provoke it was. It would never allow the other player to get away with a defection without punishing it. For example, a strategy which always cooperates, no matter what the other player does, did terribly against the mean strategies. A successful strategy must not be exploitable. Once a different strategy is able to determine that a strategy is exploitable, it will do so. TIT FOR TAT’s punishment prevent other strategies from taking advantage of it’s niceness.

Finally, one factor is the clarity in TIT FOR TAT’s actions. It is easy to see the pattern of behavior in this strategy, and once realized easy to find the best method of gaining points: cooperation.

From these factors we can deduce the possibilities behind how cooperation can be established, maintained, and prevented in a game of Prisoner’s Dilemma. Considering the commonality of Prisoner’s Dilemma, any strategist must be aware of the implications of a situation matching the profile (you cannot play the game if you’re not aware of it). The emergence, maintenance, and the breakdown of cooperation, as well as your options as the strategist will be presented in the stunning conclusion. Can’t wait until Friday.

This article is the first part of a three part series on Prisoner’s Dilemma in Game Theory. Game Theory itself is an attempt to capture the behavior behind conflict and cooperation within a mathematical perspective. It doesn’t predict the best strategy, as that is highly dependent, but it does serve the purpose of providing models to work from.

Originally a branch of Economics, Game Theory has developed into a field of its own. The 1944 book Theory of Games and Economic Behavior, written by John Von Neumann and Oskar Morgenstern marks this point. It was also further developed by Evolutionary Biologists, such as John Maynard Smith and thus branched into Biology. Game theory itself tends to have applications within situations involving action based on conflict and cooperation. For example, the arms race between nations can be considered an example of an Iterated Prisoner’s Dilemma. This game, Prisoner’s Dilemma, will be the focus of my three articles.

Prisoner’s Dilemma is a specific problem in Game Theory conceptualized in 1950. The classic definition is as follows:

Two suspects are arrested by the police. The police have insufficient evidence for a conviction, and, having separated both prisoners, visit each of them to offer the same deal. If one testifies (defects from the other) for the prosecution against the other and the other remains silent (cooperates with the other), the betrayer goes free and the silent accomplice receives the full 10-year sentence. If both remain silent, both prisoners are sentenced to only six months in jail for a minor charge. If each betrays the other, each receives a five-year sentence. Each prisoner must choose to betray the other or to remain silent. Each one is assured that the other would not know about the betrayal before the end of the investigation. How should the prisoners act? From Wikipedia

On its surface, it seems of little use to us in daily life. However, if we were to examine the situation more closely, we would find a multitude of problems which fit the model of Prisoner’s Dilemma. The first thing to notice would be a defining factor which sets it apart from other games, such as Chess. In Chess, both players’ interests are completely opposed to each other. But in the Prisoner’s dilemma, the well being of a player is not mutually exclusive from the well being of the other player.

To formalize, this game can be defined as follows:

Prisoner's Dilemma outcomes

Where R stands for Reward for Cooperation, S stands for Sucker’s Payoff, T stands for Temptation to Defect, and P stands for Punishment for Mutual Defection.

In order for this model to hold true as a Prisoner’s Dilemma, T > R and P > S. Or, mutual cooperation (both players choosing to cooperation, RR) is the best outcome for both parties, while defecting is the best outcome for a single party (Temptation to defect, TS). In this example we can assign numerical values (or “points”) by giving each player 3 points if they both choose cooperation. While the player who defects and the player who cooperates will get 5 and 0 points, respectively. Finally, if both players choose defection, both receive 1 point (Punishment, PP).

Consider the problem faced by two competing gas stations. Suppose Station A choose to lower the price of gasoline by 5 cents while Station B did not. This is equivalent to station A choosing defection while Station B choosing cooperation modeled by T,S (Temptation to Defect & Sucker’s payoff) In the above model. Station A would receive 5 points while Station B received 0 points. If Station B were to then also lower the price of gasoline by 5 cents, both parties would stand even, but each would suffer the lower price and so become P,P (Punishment) each receiving 1 point. However, if each were to maintain a middle price without lowering to undercut the other, then each would receive 3 points for mutual cooperation.

The example above relates to a possible real life situation of Prisoner’s Dilemma. But do not get caught up in numerical values, the payoff can be anything from monetary value to doing a favor. The key to this game is the ratio of scoring. If T > R and P > S, then it can qualify as Prisoner’s Dilemma.

Robert Axelrod’s 1984 book, The Evolution of Cooperation described a tournament in which Game Theorists were asked to send in their favorite strategy where each would be pitted against each other in an iterated Prisoner’s Dilemma game (a game of Prisoner’s Dilemma which would repeat over and over again). The tournament was an attempt to measure the relative success of as many strategies as possible using a scoring system.

Such strategies submitted included a RANDOM strategy, in which cooperation or defection was random, the ALL D strategy, in which defection was the only choice, TIT FOR TAT, which cooperated on the first move and followed by doing whatever the other player did the previous move, strategies which would defect every so often for various reasons, strategies which would choose only defection after the first defection by the other player, and a multitude of other strategies which would use complex algorithms and the like.

Amazingly enough, the strategy that did the best overall in the tournament against all the other strategies was the simplest of them all: TIT FOR TAT, the strategy which was never the first to defect.

Later we will explore the reasons behind TIT FOR TAT’s success when against other strategies and what we can learn based on the TIT FOR TAT strategy. Be sure to check back on Wednesday for the exciting PART TWO!