Open Journal of Political Science
2013. Vol.3, No.1, 16-23
Published Online January 2013 in SciRes (http://www.scirp.org/journal/ojps) http://dx.doi.org/10.4236/ojps.2013.31003
16
Coding Issues in Cognitive Mapping of Games
Roel Popping
Department of Sociology, University of Groningen, Groningen, The Netherlands
Received September 17th, 2012; revised October 28th, 2012; accepted November 13th, 2012
In text analysis studies coders have to make qualitative decisions. These decisions are based on interpreta-
tions of the texts under study. In such situations it is very helpful to have coding rules. These do not only
help as an aid to the coder, but are also useful for readers of the research report that will follow. The rules
make visible in considerable extent how the coding task has been performed, they take care of transpar-
ency. This contribution focuses on motions that have been treated in the Dutch House of Representatives.
Motions usually contain information on why they are needed, the proposing member usually also tells
about it. There is a discussion with the secretary, who is supposed to put the motion into effect if it is ac-
cepted. The secretary even has to give an advice. It is assumed that under these discussion(s) a cognitive
map containing some game theoretic representation can be found. Rules are discussed that are used to
code the types of maps that might be found.
Keywords: Coding; Text Analysis; Content Analysis; Motions
Introduction
Many types of text are available that are not exploited yet.
Looking at political debates we have among others party mani-
festos, speeches, debates. Party manifestos allow investigating
parties with respect to their place on a left-right continuum
(Laver & Garry, 2000) or on a scale denoting a relevant char-
acteristic. This in its turn allows discovering how a party
changes position over time. Slapin and Proksch (2008) demon-
strate changes in economic, societal and foreign policy for sev-
eral political parties in Germany. Speeches might be speeches
by party officials, but also speeches by members of the parlia-
ment or members of the government. McLean and Patterson
(2007) analyzed speeches on Iraq by the British Prime minister,
who was at that time also party leader. Debates might be de-
bates at a party meeting, but also a debate in democratic gov-
ernment’s legislature like a parliament between members of
different parties or between such members and the government.
Bara, Weale and Bicquelet (2007), who actually compare the
results found by using two different computer programs, con-
sider the parliamentary debate. They look for dominant themes
in the debates and try to find out how far speakers who take
different positions follow a distinct pattern of discourse. Very
often, certainly in governmental bodies, transcripts of such
speeches or debates are available. Harris (1996) summarizes
studies in which content analysis was used to understand nego-
tiations. She showed that in these studies there has been a lot of
attention for positive and negative behaviors and that also style
and affect received attention. A disadvantage of discussions in
general however is that it is unclear when a party will come
with a reaction on an issue or statement; it is even possible that
there will be no reaction at all.
In debates in parliament sometimes typical formats are used
for discussing issues. In many countries parliamentarians have
the right to ask questions, to propose amendments to laws that
are under construction or to propose motions. A motion is an
official statement in which one or more members of the parlia-
ment (usually as representative(s) of one or more parties) ask
something (usually a change in policy) from the government. In
the Netherlands the motion is pre-structured; it consists of a
consideration which results in a request. In addition, before the
parliament will vote upon the motion, the government is by law
obliged to give an advice.
In order to really exploit the value of motions one can look at
what it is about. Traditional content analysis can be useful here.
Now one gets information on themes that are discussed, more
interesting might be the possibility to get information on how
themes are discussed. The study on how they are discussed
gives insight in negotiation and decision processes.
However, there is more. The motion hides stations of game
theoretic models, which inform about the structure of the nego-
tiations and decisions. Traditional content analysis does not
show this directly, but can help in finding these models. Game
theoretic models can be represented by using cognitive maps,
mathematical models of belief systems.
The purpose of this text is to indicate that such representa-
tions can be found in a structured way by applying text analysis.
Actor’s utilities with respect to two situations have to be coded
as positive or negative. Based on these codings the game can be
determined. The remaining part of the text is on the coding
process. How should one look at a text, what should be taken
into account to come to a judgment. The decisions made by the
coders in the investigation have to be clear and transparent
(Popping & Roberts, 2009). The text does not deal with a con-
crete analysis of representations of models found. This will
come at some other place.
Cognitive Maps of Games
A cognitive map is “designed to capture the structure of the
causal assertions of a person with respect to a particular policy
domain” (Axelrod, 1979, p. 58). The map consists on the one
hand of concepts that are treated as variables. These variables
refer to entities that can take different values (e.g. “education”

R. POPPING
can be high or low; a campaign can be successful or unsuc-
cessful, etc.). On the other hand the map consists of causal be-
liefs, expressed as assertions. These are found in the relations
between the concept variables.
For each concept, it can be determined whether it affects the
utility of an interdependent actor negatively or positively. The
concepts are pictured as nodes in a network and the relation
between the concepts as a line. The actors can also be presented
as nodes, and the lines from concept to actor are used to express
the utility the concepts have for the actor. Depending on the
values the relations take, it is possible to identify which game is
going on. Figure 1 contains the representation of a pure com-
petition game. Here Uh and Ug refer to the actors. Later on in
the text two actors will be used: the member of the House of
Representatives and the member of the government.
The type of game that is at hand depends on the values the
lines take. Types relevant for analyses of the data are the pure
competition, the social dilemma, the coordination, and the con-
sensus game. However, as this text concerns the coding and not
the analysis, neither treatment of the games is given, nor further
background regarding the substantial research questions. Possi-
bilities of the type of research have been discussed by Holsti
(1976). Among others Anthony, Heckathorn and Maser (1994)
performed research in which the course of negotiations is stud-
ied by using these game theoretic representations. The inter-
ested reader is referred to these publications.
Looking at legislative bodies there are several parties taking
a position with respect to a certain concept. In this study mo-
tions as they play a role in the Dutch House of Representatives
are used. The greater part of the motions is directed to the gov-
ernment. Therefore there are two actors: the House and the gov-
ernment. Within the House however, many subgroups might be
distinguished. A member of the House proposes a motion. Usu-
ally the motion pictures a perceived reality that is judged as
being wrong, and next a solution is proposed that is judged as
positive (or at least as a first step in a positive direction). By
law the government represented by the secretary under whose
responsibility the issue falls has to reflect on the motion. This
can be read as judging the perceived reality position. This re-
flection takes place in a debate in the House. The secretary also
has to give a recommendation, has to advice the members of the
House to be against or in favor. Besides, the secretary has to
present some argumentation for the advice. This advice is given
during the debate.
Studying motions is a way to investigate the structure of ne-
gotiation processes. A motion is proposed by a member of a
coalition or an opposition party. Different game theoretic rep-
U
h
A
c
A
r
U
g
−
−
+
+
Figure 1.
Cognitive map of a pure competition game (Ac
= Anarchy Node for the consideration part, Ar =
Anarchy Node for the request part, Uh = Utility
of the member of the House, Ug = Utility of the
government).
resentations are possible given who is proposing and that what
is asked for in combination with the reply by the government.
Motions might also be proposed by several members of the
House. These can even be members of both coalition and oppo-
sition parties. These members will be united in their view on
the issue under discussion. This might among others be related
to the ideology the parties have. Motions might also be used to
investigate other types of research questions.
The task of the investigator is to find documents in which the
motion is presented and (usually other) documents containing
the transcript of the debate about this motion. In the debate the
members have a certain amount of time to speak, during this
time or in the general debate afterwards they propose their mo-
tion. When all speakers have finished the secretary gets the
opportunity to answer. In the answer the government’s position
is presented and at the end also an advice is given.
When the motion is addressed to the government one might
say that the member of the House first is negative (there is
something to complain about) and comes with a solution that is
positive. For the secretary, representing the government, this is
not that simple. A secretary might have opinions different from
the ones of the member of the House, but the secretary might
also be limited in possible positions due to the coalition agree-
ment. What is documented in this agreement might be different
from the position the secretary takes as a private person or as a
member of a specific political party. The secretary has to an-
swer in the role of member of the government.
Motions
In the Netherlands a motion usually contains four elements:
an introduction, a consideration, a request or judgment and a
formal closure. An opinion or an observation is treated as a
special form of a consideration. Several considerations and
requests are allowed in one motion. Below follow two exam-
ples of a motion. These motions will be used as examples in the
text.
The House, having heard the deliberation, considering that
the National Influenza Center collaborating with WHO
(World Health Organization) has to be an independent
agency, requests the government to accommodate the Na-
tional Influenza Center at the National Institute for Public
Health and the Environment (RIVM), and returns to the
order of the day (Motion 22894, nr. 244).
The second motion:
The House, having heard de deliberations, considering
that in the master plan prison system 2009-2014 has also
been recorded that some institutions having limited guard-
ing (ILG) and some having very limited guarding (IVLG)
will be closed, considering that from several (international)
researches it appears that the chances on repetition by
prisoners after dismissal from open and half-open, less
protected institutions are lower than after dismissal from
more closed and more protected institutions, moreover
considering that places in ILG and IVLG are cheaper than
places in closed institutions with a regular guard level,
requests the government to postpone the planned closing
of ILG and IVLG, and to investigate how these valuable
places for the prison system can be remained preserved,
and returns to the order of the day (Motion 24587, nr.

R. POPPING
361).
Motions are directed towards the government or towards the
House itself. In the second case the motion contains a call to
have the House investigate something itself, a word of gratitude
to a committee that did such or a call containing a judgment
about the government or a secretary (normally the government
or the secretary has to go away). This type of motions is not
relevant for the present study. Motions are needed that are ad-
dressed towards the government.
A motion might be framed by the proposer, a specific con-
sideration or the request can be over-emphasized in order to
attach the attention of the media.
Data
The present data set contains all motions that have been pro-
posed between September 2009 and February 2010 or between
September 2010 and February 2011. These time intervals are
the ones in which the House deals especially with the budget
for each department. If the House wants to influence the gov-
ernment’s policy, then it should be done here.
In these intervals 2719 motions have been proposed, 2149 of
these motions have been put to the vote. A stratified random
sample of 729 motions from the ones that have been voted on
are coded with respect to the positions as have been discussed
before. Stratification is based on dividing the motions in 10
equally sized groups based on time of proposal.
Finding the motions and the debates is not that difficult. The
Dutch government has a good website (officielebekendmakin-
gen.nl) where all information can be found. Motions are in se-
parate documents that can be recognized based on the heading.
One has to learn however how to recognize, based on the date
and heading, whether a document (probably) will contain the
transcription of the debate preceding the proposing of the mo-
tion that is looked for and the reaction by the secretary after-
wards. Usually this all is reported in one document. Transcrip-
tions of all debates are presented on the website. Here content
analysis comes in. The problem is how to code the position the
secretary takes first, the advice and the reason of the advice.
For this a number of guidelines can be formulated. With respect
to the position taken first however, one must be careful, often
the decision is based on an interpretation of how things are said.
The guidelines are very important.
Wijnja (2010) investigated qualitative characteristics of the
motions proposed in the Dutch House of Representatives be-
tween September 2009 and February 2010. Out of the 181 mo-
tions (20% of the total) that have been investigated, only two
contained no request part. In 135 motions both the logical and
causal argumentation to come to what was requested seemed
correct on face value (note, motions do not have an “if … then”
structure), in 29 motions one of these two were weak, and in 15
motions both were weak. The fact that 75% of the motions were
judged as having a correct formulation gives a good basis for
assuming that it should be possible to find the suggested cogni-
tive maps in the motions. The formulation of the consideration
part in many cases demands that knowledge regarding the de-
bate in which the motion is presented is a necessity.
Content Analysis
The dominant social scientific methodologies for the quanti-
tative analysis of texts are ones in which statistics are applied to
data consisting of word-counts within thematic categories (Krip-
pendorff, 2004). These text analyses are part of a broader class
of methodologies called “content analysis”—a generic term for
statistical analyses of qualitative data (e.g., words, gestures, art
forms, etc.).
Instead of thematic categories one might also use concepts. A
concept is “a single idea, or ideational kernel, regardless [of
whether] it is represented by a single word or a phrase” (Carley,
1993, p. 81). Our research situation is close to this one. The
filling out of the concepts however is complex. Motions start
from a present manifestation of an issue as is found in reality
according to the proposer; it refers to the issue that needs im-
provement. The request part refers to the same issue but after
improvement again according to the proposer, therefore this is a
future manifestation. To be coded is whether proposer and gov-
ernment are positive or negative about the present and the fu-
ture view.
Roberts (1989) distinguished between four types on inten-
tions a clause can have: a description or a judgment about a
state of affairs or a process. The consideration part in a motion
contains one or more descriptions of states of affairs and proc-
esses necessary to build up an argument, and judgments about
both in order to express an opinion. The request part contains a
judgment concerning the improved issue. In order to get a mo-
tion completely clear it might be necessary to rephrase is, espe-
cially to fill in the implicit statements. See later.
p
roposer's present
Theattitude towards the
government's improved
negative
view ispossitive




 





Motions are a direct source of the information to be analyzed,
they contain the text produced by the sender (the proposer), and
there is no interpretation, transformation or whatever in be-
tween. Motions contain requests. This invites to ally modality
analysis as for example in Roberts et al. (2009). In this type of
analysis one looks at what in our case the government can (pos-
sible), must (inevitable) or must not (impossible) do. Due to the
required composition of motions however it is not possible to
distinguish between possible and inevitable. This would have
been possible when transcripts of the debates are studied.
The investigator also has to decide on whether concrete con-
cepts are to be recorded or not as in Shapiro and Markoff
(1998). On the one hand it is not necessary for finding the cog-
nitive maps. On the other hand each motion is part of a discus-
sion, in the times under investigation this usually was a discus-
sion on a department’s budget. The discussions are already
labeled by the departments responsible for organizing the
House’s meetings.
Lines Representing the Utilities
First one needs to recognize the concept nodes in a motion
and their relation. As soon as these are specified, one can look
at the positions taken by the actors in the motion. One has to
look at the links between the two concept nodes and the two
actor nodes separately. These will be valued and in that way
indicate the positions taken by the member of the House and by
the government with respect to the consideration part concept,
referring to the present situation, and to the request part concept,
referring to the desired situation. Each time context information
18

R. POPPING
might be necessary to decide on the position taken.
Relating the Member of the House to the
Consideration Part Concept Node
The position taken by the party or the Member of the House
of Representatives responsible for the motion is easiest to grasp.
This is so because the core of the necessary information is to be
found in the motion; usually the perceived present situation is
depicted as bad. In the first motion presented before the pro-
posing member states the influenza center has to be an inde-
pendent institution. The way the motion is formulated shows
that the proposer has doubts on whether this will really be the
case in the actual situation. Therefore there is a negative atti-
tude with respect to the present situation.
As far as the second motion is concerned, the member has
stated that the available knowledge regarding people in prison
is to be judged in a positive way and the closing in a negative
one. The closing is an act the government wants to undertake
according to the proposer; therefore it is the concept of interest.
The member is negative about the closing.
Sometimes the actual wording of the motion suggests that the
government does well, but given the request part of the motion
it becomes clear that actually something is to be added, namely:
“… but the government does not tell how to proceed or how she
will proceed.” In the request part of the motion follows a sug-
gestion on how to do it. On the other hand there are also mo-
tions suggesting there is no policy, the government does not do
anything. In her reaction later on the government will explain
whether this view is correct or not (see the section where the
government is related to the consideration part of the motion).
A concrete example: Motion 32123-x, nr. 63 observes that the
budget for a project has been exceeded, therefore the govern-
ment is encouraged to start new negotiations with the producer
in order to cut back the final price. In fact two observations are
made: 1) the budget has exceeded, and 2) the government does
not start new negotiations, as well as a consideration: the gov-
ernment has to start the negotiations. It looked as if the consid-
eration part is neutral, but given the interpretation of the con-
sideration, it is viewed as negative.
Relating the Member of the House of Representatives
to the Request Part Concept Node
In the consideration part of the motion the message generally
is that something is wrong. The request that follows asks to
take care of improvements, totally or only with respect to the
next step to come in the process that is under discussion. This is
considered as positive. The member does not always question
whether that what is asked for is possible by law or is realistic.
Neither does the member always know what is going on at the
side of the government. This is illustrated by an answer given
by a secretary. In motion 22894, nr. 243 the government is
asked to come with an index in which the relations between
pharmaceutical companies and physicians are documented. The
secretary: “I already informed the House that a sunshine act
will take place that goes further than what you ask for. I do not
only want to make visible the contacts, I want that the financial
relations are completely visible, we need complete transpar-
ency.”
The member in the first example motion asks to accommo-
date the center at a certain institution. Implicitly it is stated that
this guarantees independence. This part of the motion can al-
ways be understood from the text in the motion itself. Context
information might be necessary to find an explanation for some
terminology. Also the position taken by the member of the
House with regard to the request part concept in the second
example motion can be read from the motion.
Relating the Government to the Request Part
Concept Node
Finding the government’s position regarding the motion is
difficult as it is not part of the motion itself. The position must
be looked for in the transcription of the debate. The secretary
advices the House about the motion during the debate. This
advice is not always formulated in a distinct way. In the advice
generally the government’s position with respect to the request
part concept is explained. Finding the view on the consideration
part usually is more difficult. Therefore here the request part is
considered first. The advice is not always in the debate, some-
times it is presented in a letter send to the House after the de-
bate. Such a letter is most clear. It really informs whether a mo-
tion is supported or dissuaded.
Looking at the first motion it turns out that the government
has no authority with respect to where to accommodate the
center. Therefore the motion is dissuaded. This indicates a ne-
gative relation. After this advice was given, the proposer has
withdrawn the motion. The secretary is negative with respect to
the second example motion because the proposer did not start
from the correct information. The measure’s goal is not reduc-
tion of capacity, but re-socialization. The motion was rejected
after the voting.
The government can be either positive or negative with re-
spect to a motion. A motion might support the policy making.
According to the secretary motion 32123-xiv, nr. 82, asks for
something that already takes place. But it might also be that the
policy just started or even has to start. The secretary tells that
she will try to use a reverse burden of proof in case of poaching
what was asked for in motion 32500-xiii, nr. 50. In both cases
the government’s view on the request part is positive.
A negative view is found in very different situations. The
judgment is based on very different arguments however. The
secretary does not agree with the accusation in motion 32500-vi,
nr. 48 that policy with regard to vulnerable groups is not bal-
anced but biased and merciless. With respect to motion 21501-
32, nr. 366 about the government’s attitude and effort at the
amendment of the European directives for experiments on ani-
mals the secretary even insists that the claim has no ground and
that the country plays an active role when this kind of questions
are discussed.
Motion 31322, nr. 110 asks for an increase in the tax burden
for employers, which is not in line with the coalition agreement
saying that a reduction is to be realized. The secretary also
might be unwilling to change a practice that works fine. There-
fore she does not support motion 32123-viii, nr. 25 which re-
quests new methods for testing on conflicts of interest at finan-
cial funds, the present system of peer review works well. In
case something new is to be realized one has to make a decision
and cannot always continue with new research. For this reason
the secretary opposes motion 31209, nr. 106, a lot of research
has been done and that resulted in the bill as it is now.
Sometimes a motion is premature. In motion 32123-xiv, nr.
119 the government is asked to keep the promise that what is in

R. POPPING
a certain law has been realized within three years. The advice
was to wait for three years and otherwise not to follow the mo-
tion. More generally, in principle the government agrees with
that what is asked for (is positive with respect to the considera-
tion part), but nevertheless at the moment she is negative as
some lacking knowledge must be filled in. This might mean
that something related to the content of the motion is studied
and one wants to wait for the results or that something (compa-
rable to or part of what is asked for) is tried out, the results
however are not known yet. It is also possible that a motion is
dissuaded, because what was asked for has already been real-
ized, or in reverse is unrealistic. According to the secretary mo-
tion 32500-vi, nr. 60 is unnecessary, as she already promised
what is asked for in the debate. Motion 32474, nr. 16 is not rea-
listic. The secretary is unable to give a guarantee that is asked
for.
Finally motions can have unwanted or unexpected conse-
quences. Motion 32123-xiv, nr. 103 asks for more air cleaners,
but these contribute to the pollution of the environment and the
secretary does not want this.
Coders have to be careful, it occurs that a negative advice is
given, but that the secretary adds that if some parts in the mo-
tion are changed he or she might become positive.
Arguments for not supporting a motion are the most inter-
esting ones. One is not only confronted with different political
positions. An argument often heard is that the motion is not ne-
cessary, because the secretary already promised to solve the
problem. Looking at the relation between government and re-
quest part concept node one runs into a difficult choice now:
the “not necessary” suggests a negative relation (and is coded
as such), but the promise tells that the government is positive.
In summary reasons for suggesting not accepting a motion are:
 the issue is not in the field covered by the secretary;
 the house is not permitted to decide on the issue;
 the government is not permitted to decide on the issue;
 the government cannot force others to do what is asked for;
 the issue can be solved in another way than proposed;
 the information in the motion is not clear or not correct;
 unwanted consequences are possible;
 not enough time available (as there is a final date).
Relating the Government to the Consideration Part
Concept Node
In his or her response the secretary most of all reflects on the
request part of the motion. Sometimes also a position is taken
with respect to the consideration part, but in most cases the
coder has to read this position from the context (how are things
said) or from what the secretary did say earlier in the debate,
even before it was clear that a motion will be proposed. This
implies that coding the government’s view on the consideration
part of the motion requires an interpretation. The coder some-
times only has the feeling the secretary is positive or negative.
The secretary’s view can be based on several aspects: a fact
(state of affairs) as presented might be incorrect or an opinion
might not be agreed upon. The coder has to be extra careful; the
secretary sometimes gives not only the position as taken by the
government, but also a personal opinion. These two might be
different. In connection with motion 32356, nr. 15, which is
about duties of a person who is over 18, the secretary brings up
to date that in her role as a mother the motion has her sympathy.
As a mother she would consider the person still as a child. She
even mentions that she probably would have supported the
motion if she were a member of the House. As a secretary how-
ever, responsible for the country’s policymaking, she has to
take another position. She has to look at the child as being a
grown person. It might also be that the secretary is limited by
law. Motion 32500-viii, nr. 110 asks for a decrease in VAT rate,
but given European rules, the secretary has no possibility to
make changes. The government agreed with these rules, there-
fore the government has to be positive.
The center that is referred to in the example motion has to be
independent. According to the proposer this can only be real-
ized by the government. The secretary starts her answer by
explaining that it has been a decision by the WHO to do it in
the way it is achieved now and that the government cannot
overrule this decision. This is coded as that the government is
positive with regard to the way the present situation is.
With respect to the second example motion on the prison
system the secretary answers that she already indicated in the
debate that she is focusing on re-socialization of prisoners and
that it is not her intention to reduce capacity. She emphasized
the importance of the fact that prisoners are in a detention cen-
ter close to their own place of living. Therefore the government
is positive with regard to the present situation.
Regarding many issues the government will indicate that she
has some concrete action or policy in mind, but that it is neces-
sary to wait for the right moment to show this. This often hap-
pens with regard to positions to be taken in negotiations within
the European government. This is read as that the government
is positive about what is stated in the consideration part. Often
other situations close to this one are found. In motion 27428, nr.
155 the government is invited to use as point of departure in a
proposal about allowing the growth of gene plants the position
that pollution across borders with genetically modified organ-
isms should be prevented. The secretary interprets this as sup-
port for the present policy; the government is positive with
regard to the consideration part.
As regards a number of motions the secretary states some-
thing like “… it is well known we are working on it. We have
had this debate before.” It might mean the debate was earlier in
the same meeting, but it can also be in a meeting before. It is
relevant to have the information on the debate. The “we are
working on it” can mean that the government is already taking
action, which implies she has already demonstrated to be posi-
tive. It might also mean that the government has the intention to
work on it, but also that the government does not want what is
asked for and that in her view a repetition has started. The
member did not get what was wanted in the debate, therefore he
or she tries it once more by means of the motion. In that case
the government is negative with respect to the consideration
part. Another interpretation however is also possible. The pro-
blem, the consideration part, is acknowledged, but the proposed
solution is not supported.
Sometimes the government is negative with respect to the
consideration part because of non-political arguments, what is
discussed is not part of the responsibility of the secretary or is
not part of the debate.
When all positions with regard to the example motions pre-
sented are combined, it becomes clear that the games going on
here are pure competition games. The proposer is negative with
regard to the coordination part and positive with regard to the
request part. For the government the reverse is found.
The first motion has two parts, a consideration and a request
20

R. POPPING
part. Therefore at most only one game is possible. The second
motion has three consideration parts, therefore three games
might be found.The first consideration part of the motion in-
forms that there is a plan to close prisons. The concept cannot
be the cause for the concept asking for delay. So, here is no
game. The second part was treated above. The third part con-
tains the information that open and half-open prisons are chea-
per than closed prisons. Therefore, the open and half-open pri-
sons should not be closed. Here the reasoning is the same as
with respect to the second part, we have a second pure competi-
tion game. It is up to the investigator to decide whether all
separate parts should be coded or not. We finally used one
score for a motion and did not use sub-units. The resulting fre-
quency distribution of the games is in Table 1.
A Note on Interrater Reliability
The coding task performed is complex and has to be carried
out by human coders. For each motion the coders have to make
five decisions:
 four decisions concern the position of House and govern-
ment with respect to the two concepts;
 the fifth decision concerns the argument used by the gov-
ernment that is at the basis of the position taken with re-
spect to the request part concept.
When the first four decisions are known, it is quite easy to
determine which game is represented in the corresponding map.
In order to get an indication of how well the coders have per-
formed their tasks a random sample of all motions that had
been voted about was independently coded by both coders. This
occurred after instruction had taken place.
The coders received instruction on how to perform the task.
In this instruction thoughts as have been discussed above have
been used. After that the coders were asked to code a sample of
30 motions which had been withdrawn by the proposer, a fact
the coders did not know. The coders coded these motions inde-
pendently; the investigator also performed the task. A discus-
sion on the differences in coding followed. Next the task was
performed. The coders each time received a small set with mo-
tions that were a representative sample of the complete set that
has been voted for. Having coded this set, they received the
next one. A motion could only be part of one set. Almost half
of the sets were coded by both coders, the other sets only by
one coder. For the analysis the index pi (Scott 1955; Popping
2010) is used as a measure of intercoder agreement. The meas-
ure uses the marginal distribution over all coders as the basis
for computing deviations from expected agreement; therefore it
does not assume a priori differences between the coders. The
index is appropriate when comparing a large number of assign-
ments among trained coders. The amount of agreement found is
presented in Table 2.
After the try-out investigator and coders spoke together about
these codings and especially about the why of the differences.
From the moment the coders indicated that all was understood
they started performing the real task. The outcomes in the table
show that the amount of agreement is always over .80, which is
usually considered as a very good result. It also indicates that
the coders having talked together must have been very impor-
tant for getting the correct view on the task. The most difficult
part in the coding task was the finding of the position taken by
the government with respect to the consideration part in the
motion.
Table 1.
Frequency distribution of the games.
House Government
Label
consid. req. consid. req.
Freq.
Social dilemma gameneg pos neg neg 121
Pure competition gameneg pos pos neg 374
Coordination game neg pos neg pos 156
Battle of the sexes gameneg pos pos pos 78
Total 729
Table 2.
Pairwise agreement-results for the coders.
π z p N
Try out 30
Gov. consid. part concept .71 14.96 0
Gov. request part concept .85 25.17 0 Coders
Argument .87 46.46 0
Gov. consid. part concept .80 29.39 0
Gov. request part concept .83 41.84 0
Coders +
Investigator
Argument .87 83.63 0
Final 303
Gov. consid. part concept .85 189.33 0
Gov. request part concept .96 272.43 0 Coders
Argument .84 439.71 0
The final score for each position or argument is the score
given by the coder. In those situations where both coders per-
formed the coding and where a different opinion was found, a
final decision was made by the main investigator. The longer
the coders had been performing the coding task the less differ-
ences in assigning were found.
Conclusion
This paper discussed the coding process to be followed by
researchers interested in applying cognitive mapping within
studies in which the investigator looks for the presence of game
theoretic models in motions. It was demonstrated that the struc-
ture of motions is complex and that coders need a clear under-
standing of what is going on in order to be able to come to the
best coding. The high amounts of agreement show that the final
decisions by the coders resemble each other very much. This is
one step in the process of ascertaining that the correct type of
game is found. The next step is relating the games to different
other variables related to the motions.
These game theoretic models are used to investigate for ex-
ample differences in performance of political parties or changes
in approaches to the government over time. It might concern all
kinds of events from the past. Apart of these also more techni-
cal questions can be posed. In case a motion contains several
game models, should these be of the same type? One might
argue that if this is not true, it indicates that several techniques

R. POPPING
22
are followed by the proposer. This indicates a weak motion, it
is not enforcing towards the government. By critically review-
ing discrepancies in applications of our coding scheme, we
hope to have both improved our methodology and conveyed it
more clearly for others’ use.
REFERENCES
Anthony, D. L., Heckathorn, D. D., & Maser, S. M. (1994). Rational
rhetoric in politics—The debate over ratifying the US constitution.
Rationality and Society, 6, 489-518.
doi:10.1177/1043463194006004004
Axelrod, R. (1976). The analysis of cognitive maps. In R. Axelrod (Ed.),
Structure of decision. The cognitive maps of political elites (pp. 55-
73). Princeton, NJ: Princeton University Press.
Bara, J., Weale, A., & Bicquelet, A. (2007). Analysing parliamentary
debate with computer assistance. Swiss Political Science Review, 13,
577-605. doi:10.1002/j.1662-6370.2007.tb00090.x
Carley, K. M. (1993). Coding choices for textual analysis: A compari-
son of content analysis and map analysis. In P. V. Marsden (Ed.),
Sociological methodology 1993 (pp. 75-126). Cambridge, MA: Basil
Blackwell.
Harris, K. L. (1996). Content analysis in negotiation research: A review
and guide. Behavior Research Methods, Instrumentation, and Com-
puters, 28, 458-467. doi:10.3758/BF03200525
Holsti, O. (1976). Foreign policy formation viewed cognitively. In R.
Axelrod (Ed.), Structure of decision. The cognitive maps of political
elites (pp. 18-54). Princeton, NJ: Princeton University Press.
Krippendorff, K. (2004). Content analysis: An introduction to its meth-
odology. Thousand Oaks: Sage.
Laver, M., & Garry, J. (2000). Estimating policy positions from politi-
cal texts. American Journal of Political Sciences, 44, 619-634.
doi:10.2307/2669268
McLean, C., & Patterson, A. (2006). A precautionary approach to for-
eign policy? A preliminary analysis of Tony Blair’s speeches on Iraq.
British Journal of Politics & International Relations, 8, 351-367.
doi:10.1111/j.1467-856X.2006.00242.x
Popping, R. (2010). Ag09. A computer program for interrater agree-
ment for judgments. Social Science Computer Review, 28, 391-396.
doi:10.1177/0894439309351394
Popping, R., & Roberts, C. W. (2009). Coding issues in semantic text
analysis. Field Methods, 21, 244-264.
doi:10.1177/1525822X09333433
Roberts, C. W. (1989). Other than counting words: A linguistic ap-
proach to content analysis. Social Forces, 68, 147-177.
Roberts, C. W., Popping, R., & Pan, Y. (2009). Modalities of democ-
ratic transformation: Forms of public discourse within Hungary’s
largest newspaper, 1990-1997. International Sociology, 24, 498-525.
doi:10.1177/0268580909334500
Scott, W. A. (1955). Reliability of content analysis: The case of nomi-
nal scale coding. Public Opinion Quarterly, 19, 321-325.
doi:10.1086/266577
Shapiro, G., & Markoff, J. (1998). Revolutionary demands: A content
analysis of the Cahiers de Doléances of 1789. Stanford: Stanford
University Press.
Slapin, J. B., & Proksch, S. O. (2008). A scaling model for estimating
time-series party positions from texts. American Journal of Political
Science, 52, 705-722. doi:10.1111/j.1540-5907.2008.00338.x
Wijnja, M. (2010). Beleidsmakers of hypeparlement? Groningen: De-
partment of Sociology.

R. POPPING
Motions and Debates
Motions and debates are in the Acts of the House of Represen-
tatives [Handelingen van de TweedeKamer] or in reports on
committee meetings. They can be found at the places listed
below.
Motion Act
Page
(motion)
Page
(advice) Date
21501-32, nr. 366 36 3433 3435 December 9, 2009
22894, nr. 243 18 1459 1462 October 29, 2009
22894, nr. 244 18 1459 1462 October 29, 2009
24587, nr. 361 5 309 311 September 23, 2009
27428, nr. 155 52 4798 4799 February 9, 2010
31209, nr. 106 36 3540 3541 December 9, 2009
31322, nr. 110 113 56 61 November 22, 2009
32123-viii, nr. 25 33 36 38 November 2, 2009
32123-x, nr. 63 33 3185 3196 December 3, 2009
32123-xiv, nr. 82 33 3136 3151 December 3, 2009
32123-xiv, nr. 103 33 3143 3152 December 3, 2009
32123-xiv, nr. 119 33 3147 3153 December 3, 2009
32356, nr. 15 47 52 55 February 2, 2011
32474, nr. 16 24 69 71 November 18 2010
32500-vi, nr. 48 29 28 43 December 1, 2010
32500-vi, nr. 60 29 37 45 December 1, 2010
32500-viii, nr. 110 132 58 65 December 15, 2010
32500-xiii, nr. 50 67 39 50 November 11, 2010
Note: In bold are not acts but reports on committee meetings.