The game

At the start of the game, each player receives a character card that states what role he will play in the game. Generally, the players are divided into two factions: werewolves and citizens. In the latter group some individuals possess special abilities. Examples of such roles are the seer, the witch and the innocent girl. The goal of all citizens is to discover all werewolves. The goal of the werewolves is to eat all citizens.

The game is played in turns that each consist of a night and a day. During the night common citizens have to keep their eyes closed and wait for the night to end. The seer wakes up and can look at the character card of a player of his own choosing. Then the werewolves wake up and together select a victim to eat. During the waking hours of the werewolves the player with the role of the innocent girl can peek to see who the werewolves are. However, the risk is that the werewolves see her looking at them and can then easily just choose to eat her since her peeking ability is a significant threat to the werewolves. Following these events, the witch wakes up. The game master tells her who the victim of the werewolves was. Only once during the game, she can revive a dead player with her elixir of life. Also once during the game she can give a player a deadly potion which will kill the player that receives it.

During the day the game master announces which players are dead and the dead players reveal their character cards. The living then take a vote about who they suspect of being a werewolf. This player is then lynched by the angry civilians and burned. All dead players are immediately removed from the game, they are no longer allowed to give the other players some information...

When either only werewolves or only citizens are left, the game ends and the faction that's still alive wins the game.

Winning conditions

There are different competing factions in the game, each of which has different goals they have to achieve in order to win the game.

• Citizens: The citizens have won when all werewolves have been killed and there is at least one surviving citizen left to celebrate this joyful fact.

• Werewolves: The werewolves have won when all citizens have been killed and there is at least one surviving werewolf left to celebrate this cruel fact.

• Other factions: Depending on what characters are included in the game there could be factions that fall outside the scope of the general citizens versus werewolves fight. For example, the character Cupido selects two players and shoots "love-arrows" at them during the first night of the game. These two players then have an alternative goal in the game: they win if the both of them survive. When the two players are a mixed couple that consists of a werewolf and a citizen, it means that they might have to work against their own faction in order to save the other one. For example, the lovestruck citizen might try to convince the other citizens that his lovestruck werewolf is really not a werewolf so that the other citizens do not try to burn him or her.

Epistemic notions

There are facts about the current state of the game that players know (knowledge) for sure and things they can only suspect (beliefs).

Knowledge

An example of knowledge is that the werewolves know who is werewolf, and can thus derive that everyone outside their group is not a werewolf and must be citizen. Additionally, the fact that werewolves know this is common knowledge since it is a basic rule of the game: both citizens and werewolves know it, and everyone knows that everyone knows it, etc.

Beliefs

However, there are many things that players do not know for sure, such as for citizens who is werewolf and who is not. A more complex example is using voting behaviour for deducing a player's character. If a certain player is voted to be burned and afterwards he is revealed to be a werewolf, then all citizens could conclude that everyone in favor of burning this werewolf must be a citizen. After all, only citizens want to kill all werewolves, so this seems like a reasonable conclusion. It is not certain knowledge however: there could be werewolves that choose to sacrifice one of their own to appear to be citizen so that the citizens no longer suspect them of werewolf and do not want to burn them anymore.

In our implementation, we will abstract from the distinction between knowledge and belief. The reason is that knowledge as well as beliefs will directly influence the actions; belief is there as strong as knowledge.

Higher-order knowledge

Higher-order knowledge plays in Werewolves an important role for each player in terms of what actions they wish to perform. Players typically have little definite knowledge and have to depend on educated guesswork about what player might play what role. In order to find out about this, they have to apply theory of mind by mentally putting themselves into the position of other players to see what other players know and might do. They can use this information to determine their own best game strategy.

Public announcements

The characters of the dead players will be revealed. This can be seen as public announcements. Also the voting behavior of all players in the day-phase is known to everyone, this is interpretable as a public announcement too.

Different strategies

In this section, we will analyse different strategies for citizens, for werewolves, and for the seer and innocent girl. Although in the full Weerwolven van Wakkerdam there are many more roles these, we will stick to these four roles for our implementation since it turns out that this already gives much to talk and reason about. The strategies of the four roles will be based on higher-order knowledge. In the following table, we structure the updates of knowledge and the voting behavior in the day-phase. We assume that werewolves in the night-phase never eat one of their own nature; this is only useful to misguide the witch, who is not considered in our model. Note that behavior in the zeroth order is based only on knowledge about the world, not on knowledge about the knowledge of others. In the n-th order, the behavior is based on the behavior of the opponent in order n-1. Note that the werewolves can consider the seer/innocent girl and the citizens as different opponents, i.e. they have different behaviors.

Order	Citizen	Werewolf	Innocent girl	Seer
0	No knowledge updates.	No knowledge updates.	No knowledge updates.	Knowledge updates from seeing cards.
0	Votes for: W/random.	Votes for: C.	Votes for: W/random.	Votes for: W/random.
1	Update rule: X is a C if voted for W, a W if voted for C.	Update rule: X is probably I/S if voted for W. Else, definitely not I.	No knowledge updates: already knows everything.	Update rule: X is a C if voted for W, a W if voted for C.
1	Votes for: W/random.	Votes for: preferably I/S, else C.	Votes for: W/random.	Votes for: W/random.
2	Update rule: X is a C if voted for W, a W if voted for C.	Update rule: X is probably I/S if voted for W. Else, definitely not I.	No knowledge updates: already knows everything.	Update rule: X is a C if voted for W, a W if voted for C.
2	Votes for: W/random.	Votes for: preferably I, then S, else C; but sometimes W.	Votes for W, sometimes C.	Votes for: W/random.
3	Update rule: X is probably a W if voted for C, probably a C if voted for W.	Update rule: X is probably I/S if voted for W. Else, still possible to be I.	No knowledge updates: already knows everything.	Update rule: X is probably a W if voted for C, probably a C if voted for W.
3	Votes for: W/random.	Votes for: preferably I, then S, else C; but sometimes W.	Votes for W, sometimes C.	Votes for: W/random.
4	Update rule: X is probably a W if voted for C, probably a C if voted for W.	Update rule: X is probably I/S if voted for W. Else, still possible to be I.	No knowledge updates: already knows everything.	Update rule: X is probably a W if voted for C, probably a C if voted for W.
4	Votes for: W/random.	Votes for: preferably I, then S, else C.	Votes for: W/random.	Votes for: W/random.

In general, the principle behind the listed four orders is as follows.

• In the zeroth order, players can only use knowledge they get from their specific role: werewolves know who the citizens are, the innocent girl knows who the werewolves are, and seers can peek at one player's card per turn.

• In the first order, players try to make educated guesses about other player's roles, using the voting behavior.

• In the second order, players try to deceive each other. The werewolves try to prevent citizens from quickly idenfying them by occasionally voting for other werewolves, and the second order innocent girl does the same against werewolves by sometimes voting for citizens.

• Finally, in the fourth order players no longer try to deceive, because third-order players are already countering it so the deceiving behavior becomes counterproductive.

In an additional fifth order, players would no longer need to account for deception since fourth order players do not attempt to deceive anymore. Fifth order players would also not try to mislead themselves, since fourth order players are still trying to counter that deception strategy. In effect, fifth order players act the same as first order players. This means that sixth order players would act similar to second order players, seventh order players like third order players, etc.

In the next subsections, we will explain these behaviors and illustrate them with Kripke models.

Citizens, zeroth-order knowledge

When the citizens use only zeroth-order knowledge, they don't know anything about the knowledge of other players. Therefore, their voting behavior will be completely arbitrary. We show this in a simple situation, where there are only a seer, a werewolf and a citizen remaining. Assuming that the seer has no additional knowledge, the Kripke model is as follows. We use the following convention: the states describe the characters of the players, for example: swc means that the first player is a seer, the second is a werewolf and the third is a citizen. The underlined state is the true state. The arrows apply to the different players as denoted in the legend. We do not make a distinction between knowledge and belief, because of the complexity and the unapplicability in our implementation, but in general the citizens are more believing, and the werewolves are more knowing.

With the completely arbitrary voting behavior, the seer and the citizen are chosing randomly one of the two possible states. This is denoted in the Kripke-model below, where the seer is fortunately right, but the citizen isn't.

Citizens, first/second-order knowledge

When the citizens are using first-order knowlegde, they assume that the werewolves are playing only on the basis of their knowledge about facts, and not on the basis of their knowledge about other people's knowledge. Therefore, their assumption is that werewolves only will vote for a citizen, because their main goal is to extirpate all citizens. This implies that, if there is a werewolf burned, all players who voted for him are considered citizens. We assume in our model that all players who voted for a citizen are werewolves, although also ignorant citizens can do that.

Let us assume that we have the same situation as above, that there was just another citizen killed, and that player 2 and 3 voted for this player. In this situation, the citizen (player 3) thinks that player 2 is a werewolf. (Obviously, he will not condsider himself to be a werewolf, although he voted for person 4.) The seer, however, will consider both player 2 and 3 to be a werewolf. However, this is not possible (there is only one werewolf), so he will judge none of them, and he doesn't change his beliefs: he still beliefs both to be equally like to be a werewolf. The Kripke-model will be the following, where player 3 has the right belief:

Second-order citizens use the same reasoning scheme as first-order citizens, since the behavior of first-order players still leads to the same conclusions regarding their possible roles.

Citizens, third/fourth-order knowledge

In the case the citizens use third/fourth-order knowledge, they assume that the werewolves use second/third-order knowledge, i.e. that werewolves will vote sometimes for themselves. Therefore, they update their knowledge with the following rules: if a dead player turns out to be a werewolf, the players who voted for him are considered to probably be a citizen. In line with this, we assume that if a dead player turns out to be a citizen, the players who voted for him are probably werewolves. Note however that werewolves do not always vote for the other faction; we will work out an example where the citizens assume that werewolves vote for themselves.

We consider a concrete situation. Assume that there are three players (a seer, a werewolf and a citizen, respectively) left, and that a fourth player, a werewolf, is just burned. Assume that player 1 and 2 voted for player 4, player 3 voted for player 1 and player 4 voted for player 2. (Note that player 2 voted for player 4 to establish his own innocence.) Player 3, the citizen, knows that both player 1 and player 2 voted for player 4, and consideres them therefore equally likely to be a werewolf. However, person 4 voted for person 2, and based on the theory that werewolves can vote for other werewolves, player 3 considers player 2 more likely to be a werewolf. Note that the seer as well considers player 2 to be the werewolf (by the same theory and by knowing that he himself isn't a werewolf). The resulting Kripke model is the following:

Werewolves, zeroth-order knowledge

In this case, the werewolves are not thinking about their opponents' knowledge; they only try to destroy all the citizens. So, they never will vote for one of themselves. They do not try to identify seers or innocent girls either.

Werewolves, first-order knowledge

Werewolves know that both citizens and the innocent girl or the seer will, if they use zeroth-order knowledge, vote for them. However, they know that the innocent girl and seer in general will have more information. They therefore will consider it more likely that anyone who is voting for a werewolf is an innocent girl or a seer than that he is a citizen. Because the innocent girl and the seer are a greater threat, they will vote for them, if possible; else, they will vote for another citizen. Concrete, the werewolves consider it not possible that a first-order innocent girl votes for a citizen (we assume that the innocent girl knows all the werewolves in the first night-phase, and will always vote for a werewolf) and consider a player who votes for a werewolf more likely to be a seer or an innocent girl than a common citizen.

In a concrete situation with one seer, one werewolf and one citizen, we assume that player 1 voted for player 2, who is actually the werewolf. Further, we assume that player 2 and 3 voted for another player, who actually was burned and turned out to be a citizen. In that case, player 2 considers it more likely that player 1 is the seer than that player 3 is the seer. The Kripke model becomes the following:

Werewolves, second-order knowledge

In this case, the werewolves think the same as in the first-order case about the characters of players voting for werewolves. However, they also will use the strategy of voting for other werewolves, to mislead their first-order thinking opponents. In this case, we have therefore that werewolves can vote for the innocent girl or the seer (who remain a threat for them) or for other werewolves. In the implementation, we chose to let them vote for another werewolf with chance 0.2, and for the innocent girl with chance 0.8.

Werewolves, third-order knowledge

Werewolves using third-order knowledge sometimes will vote for other werewolves, just as second-order thinking werewolves. However, in contrast to second-order thinking werewolves, the third-order will not have an explicit rule for identifying the innocent girl or the seer. This is because the second-order thinking innocent girl of seer will not always vote for werewolves, and thus sometimes for citizens. But this in general gives no clue, because the amount of citizens is in general larger than the amount of werewolves, so many players will in general vote for citizens when they have no convincing information about who is a werewolf.

Werewolves, fourth-order knowledge

Fourth-order werewolves have the same knowledge-update as third-order, but they will vote always for citizens, because third order citizens are no longer sensitive to deceptive tactics.

Innocent girl and seer

The innocent girl and the seer will have the same knowledge gathering heuristics as the other citizens. However, additionally they get knowledge updates in another way. The innocent girl can peek while the werewolves are eating, and the seer can see a character card of his choosing at each turn. In our implementation, we assume that the innocent girl uses her properties at every turn, so she knows all the werewolves immediately. The first-order thinking werewolves use this as a strong clue, because the zeroth-order thinking innocent girl always will vote for werewolves. The second- and third-order innocent girl will misguide the werewolves by voting not always for them; in our implementation, the chance is 0.2 to vote for another citizen.

General comments

• In this section, we described different strategies of the players, based on the order of knowledge they think their opponents will have. However, it's of course possible that factions consist of players using knowledge of different orders. For example, if you are a third-order thinking citizen, that will be appropriate for a second-order thinking werewolf. However, if there is also a first-order thinking werewolf in the game, you don't have a strategy appropriate for both werewolves. Strategies for individual players in this case can possibly help.

• There is always a random element in the voting behavior of citizens in the first voting round. Therefore, in the higher-order strategies, which are based on people's voting behavior, there will be a bias too. As the game goes on, the random element of the first round will become smaller and smaller and players will get increasing good at correctly guessing other players' roles.

• One of the strategies of the innocent girl can be to behave in the first night as if she is a werewolf. In that case, the werewolves see that there are too much werewolves, they will know that one of them is cheating, but they don't know who it is. This strategie doesn't correspond to one of the orders and it's too complex to implement it, because of the structure of the implementation; therefore, we only analyse it theoretically.

  Assume there are 1 seer, 2 werewolves, 1 innocent girl, and 2 citizens. We analyse the first round, were the girl is playing a risky game, behaving like a werewolf in the night-phase. The two real werewolves note that one of the three is cheating, but they don't know who. We assume that in the day-phase, they will try to unmask the innocent girl. They know that voting for the other werewolf will, if that person is unmasked, result in a belief of their own innocence by the other players, assuming that they use first- or second-order knowledge.The simplified initial situation is as follows:

  

  We suppose that the sixth person, a citizen, is eaten; the second person is burned and person 3 voted for this player. So, person 3 and 4 know of each other what they are (and they know that they know), but person 1 and 5 don't know anything. However, they believe that person 3 isn't a werewolf, because he voted for person 2. In the simplified Kripke model, we can give these relations: