J. COSTLEY, S. L. HAN
forum, “How should I prepare for the Korean history exam
required for all education students?” In the first case the whole
post was coded as one unit:
“Don’t worry. U can do it!!!!!!!! I just studied that for 5 days.
I passed the exam at once. It is easy enough for me to pass it at
once. I recommend Mr. Choi in EBS. His full name is Choi,
Tae-seong. He is very famous as Korean-history teacher. If you
take his class in cyber, you can feel that Korean history is easy.
Cheer up! Anyway, it’s time for my dinner, Good luck~~~~~~”
(R+ Relevant statement; N+ Novelty, New information or
ideas, O+ Bringing outside information to help with the prob-
lem. C+ Critical assessment/evaluation of own or others con-
tribution).
In this second example of grain size, as opposed to coding as
a whole post, the coding was done at the level of the sentence:
“Don’t worry. U can do it!!!!!!!! I just studied that for 5 days.
I passed the exam at once. It is easy enough for me to pass it at
once (OK+ Evidence of using previous knowledge). I recom-
mend Mr. Choi in EBSR (R+ Relevant statement, NS+ New
solutions to problems). His full name is Choi, Tae-seong. He is
very famous as Korean-history teacher. If you take his class in
cyber, you can feel that Korean history is easy (C+ Critical
assessment/evaluation of own or others contribution). Cheer
up! Anyway, it’s time for my dinner, (R− Irrelevant statement)
Good luck~~~~~~”.
Depending on the grain size analysis we get two different
versions of the same text using the same coding system. In
these particular examples there are three key differences that
should be noted: 1) Coding at the larger grain size is faster but
allows less detail; 2) Coding at the smaller grain size allows
coders and researchers to see which statements lead to which
examples of the particular code. This can complicate inter-rater
reliability in the short term, but in the long term it allows easier
definitions of particular types of states and operationalization of
the coding system; 3) As can be seen with the example of rele-
vance (R+ or R−), the smaller grain size can allow conflicting
coding within the same post. This allows for greater detail and
more depth in the research.
The most important question when deciding grain size are
the research questions. Coarser grain size allows for more data
to be coded and in some cases it can give a better understanding
of the topic being researched. Finer grains can lead to over
complicating or over detailed analyses that do not contribute to
answering the research questions. In this research, ratios are
created to assess how much any particular learner is exhibiting
critical thinking in an online asynchronous environment. There-
fore it was decided that a smaller grain size was more appropri-
ate and a sentence by sentence grain size was chosen. This is
because the sentence level of analysis allows a greater degree of
analysis and a higher conformance in inter-rater reliability.
Once the choice has been made of grain size a choice needs
to be made on the protocols of separation of each unit. In the
case of using an individual post, the segmentation protocol is
simple as each post represents not only a unit of analysis but
also a unit that is easily separated visually. In the case of re-
corded data, segmentation can become trickier. For example, if
the choice has been made to segment data based at the sentence
level, how are interrupts treated? Student A, “I think the more
important thing about study is hard work….” Student B,
“Yeah!” Student A, “….. and having a good relationship with
your teacher and peers.” Cases like this can seem to be self
explanatory, however, when there is a great deal of data and
multiple coders being used, clear protocols need to be devel-
oped for segmenting each unit of analysis. In this study the
non-content feature of punctuation was used. The data collected
in this study was written as opposed to spoken so separating
each individual sentence was reasonably simple and quick.
There were three clear features that were used to divide each
sentence: a full stop, a question mark, or an exclamation mark.
Furthermore, the nature of the forum was informal therefore
there were cases where “proper” punctuation wasn’t used. In
the above examples from the forum the student used “~~~~~~”
to end a sentence. These cases were very clear and there was
100% inter-rater reliability on a sample of 200 sentences over
where the sentence began and ended. One of the ways the cod-
ing scheme in this research differs from Newman, Webb, and
Cochrane’s scheme is in the difference of unit size delineation.
In Newman, Webb and Cochrane’s coding scheme the unit size
is ill-defined and varies between the level of the sentence, idea,
phrase or message.
Creating or Choosing a Coding Scheme
After the unit size and a method of segmentation have been
chosen, coding can begin. This point in the process is the most
variable and complex, because this is the point at which the
researcher’s varied questions, area being studied and theoretical
background come most into play. In the beginning it must be
noted that a coding scheme has several possible purposes. It
may be to summarize or condense the data. In this type of case
the purpose is to allow the researcher to get an overview of the
data and gain understanding through seeing the larger picture.
In the case of reducing the data, the data is put into a form in
which it can be analyzed outside of the data set from which it
comes (Salanda, 2009). According to Lichtman (2006) when
coding in education generally, approximately 80 original codes
are generated. She goes on to say, those 80 codes are coded into
approximately 15 categories. Those 15 categories can be further
narrowed into 7 or 8 concepts. Creswell (2006) constructs a
differing conceptualization of how many codes to use. Accord-
ing to Creswell there should be 5 or 6 basic codes that can then
be expanded into 30 more specific codes. In this research 45
codes were created based on Newman, Webb, Cochrane’s
(1996) content analysis framework.
The coding choice specifically needs to conform to what the
researcher is investigating. An example of this is a coding
scheme used in Pena-Shaff and Nicholas (2004) wherein one of
the primary research questions was how the users of a bulletin
board service utilized the knowledge construction process in
developing their understanding of the content of their course.
Their coding system was founded on knowledge construction
and what elements in that knowledge construction we most
closely relate to reflection. For this reason their coding scheme
was developed around indicators of knowledge construction:
questioning, replying, clarifying, interpreting, disagreeing, as-
serting, building consensus, judging, reflecting, and supporting
(Pena-Shaff & Nicholas, 2004). As can be seen from this exam-
ple (Pena-Shaff & Nicholas, 2004, p. 256):
“Excerpts from the Discussion: Community Networking,
Week 4
Open questions for discussion: Are the assumptions of the
“informed citizen” and “community cohesion” valid? Are they
necessary for building an on-line Community Network? If so,
why? If not, why should a Community Network be built at all?
Open Access 3