Vol.3, No.1, 51-57 (2013) Open Journal of Preventive Medicine
Mixed methods evaluation of a randomized control
pilot trial targeting sugar-sweetened beverage
Jamie Zoellner1*, Emily Cook1, Yvonnes Chen2, Wen You3, Brenda Davy1, Paul Estabrooks1
1Department of Human Nutrition Foods and Exercise, Virginia Tech, Blacksburg, USA;
*Corresponding Author: zoellner@vt.edu
2Department of Communications, Virginia Tech, Blacksburg, USA
3Department of Agriculture and Applied Economics, Virginia Tech, Blacksburg, USA
Received 6 December 2012; revised 8 January 2013; accepted 17 January 2013
This Excessive sugar-sweetened beverage (SSB)
consumption and low health literacy skills have
emerged as two public health concerns in the
United States (US); however, there is limited re-
search on how to effectively address these is-
sues among adults. As guided by health liter-
acy concepts and the Theory of Planned Be-
havior (TPB), this randomized controlled pilot
trial applied the RE-AIM framework and a mixed
methods approach to examine a sugar-sweet-
ened beverage (SSB) intervention (SipSmartER),
as compared to a matched-contact control in-
tervention targeting physical activity (Move-
More). Both 5-week interventions included two
interactive group sessions and three support
telephone calls. Executing a patient-centered
developmental process, the primary aim of this
paper was to evaluate patient feedback on in-
tervention content and structure. The secondary
aim was to understand the potential reach (i.e.,
proportion enrolled, representativeness) and
effectiveness (i.e. health behaviors, theorized
mediating variables, quality of life) of SipS-
martER. Twenty-five participants were random-
ized to SipSmartER (n = 14) or MoveMore (n = 11).
Participants’ intervention feedback was positive,
ranging from 4.2 - 5.0 on a 5-point scale. Qualita-
tive assessments reavealed several opportune-
ties to improve clarity of learning materials, en-
hance instructions and communication, and re-
fine research protocols. Although SSB con-
sumption decreased more among the SipS-
martER participants (256.9 ± 622.6 kcals), there
were no significant group differences when
compared to control participants (199.7 ± 404.6
kcals). Across both groups, there were signifi-
cant improvements for SSB attitudes, SSB be-
havioral intentions, and two media literacy con-
structs. The value of using a patient-centered
approach in the developmental phases of this
intervention was apparent, and pilot findings
suggest decreased SSB may be achieved through
targeted health literacy and TPB strategies. Fu-
ture efforts are needed to examine the potential
public health impact of a large-scale trial to ad-
dress health literacy and reduce SSB.
Keywords: Beverages; Health Literacy; Health
Education; Public Health; Health Behavior; Pilot
High sugar-sweetened beverage (SSB) consumption
and low health literacy skills have emerged as two broad
public health concerns in the United States (US). For
example, SSB consumption has approximately doubled
in the past two decades and contributes about 10% of the
total calories (kcal) in the US diet [1]. While excessive
SSB intake has been associated with numerous adverse
health outcomes [2], there is limited research on how to
effectively improve SSB behaviors among adults. Fur-
thermore, it is estimated that one-third of Americans
have low health literacy skills [3]. Low health literacy
has been associated with poorer health outcomes [4], and
one study found health literacy was a stronger predictor
of SSB consumption relative to educational achievement
or income [5]. However, taken as a whole, intervention
approaches to mitigate the effects of low health literacy
have been mixed [4]. Two plausible explanations include
the deficiency of health behavior theory to guide health
literacy intervention approaches and the lack of pilot
studies to refine intervention messages, strategies to im-
prove health literacy, and recruitment and retention ap-
Copyright © 2013 SciRes. OPEN ACCESS
J. Zoellner et al. / Open Journal of Preventive Medicine 3 (2013) 51-57
proaches for low literate audiences [4,6]. Collectively,
these findings highlight the potential of addressing SSB
intake through intervention approaches guided by health
behavior theory and health literacy, as well as the need
for pilot studies to help advance intervention develop-
ment and implementation.
To date, there is limited research on how to address
SSB behaviors among adults [7-9], and none of which
report an underlying theoretical approach or the potential
influence of health literacy status on behavior change.
Likewise, no study, to date, has reported on engaging
prospective participants to elicit feedback on the devel-
opment of SSB intervention content and structure to en-
sure that it is relevant to the target population [10].
Therefore, an important starting point for assessing the
acceptability and potential effectiveness of an SSB be-
havioral intervention is to gather information directly
from the target population [11]. In addition to the re-
finement of research methods, instrumentation, and hy-
pothesis, taking advantage of opportunities to execute a
patient-centered developmental process can help more
fully understand patients’ receipt and value of the theory-
driven intervention content and communication approaches
The overall goals of this 5-week, 2-arm randomized
controlled trial was to apply a patient-centered develop-
mental process to inform the refinement of intervention
content and communication approaches, as well as pilot
test the effects of an intervention to decrease SSB con-
sumption (SipSmartER) when compared to a matched-
contact control condition targeting increasing physical
activity behaviors (MoveMore). Both treatment condi-
tions were guided by the Theory of Planned Behavior
(TPB) [13] and concepts in health literacy [14], including
media literacy [15]. Further, the structure and evaluation
of the intervention was informed by the RE-AIM
framework to heighten its likelihood for translation into
practice by considering factors related to reach and ef-
fectiveness at the individual level and the potential adop-
tion, implementation, and maintenance at the organiza-
tional level [16]. Hence, the primary aim of this paper is
to evaluate patient feedback on intervention content and
structure. The secondary aim was to understand the po-
tential reach (i.e., proportion enrolled, representativeness)
and effectiveness (i.e. health behaviors, theorized medi-
ating variables, quality of life) of SipSmartER. Although
the small sample of this pilot study limits statistical
power, it was hypothesized that when compared to the
matched-contact control participants, SipSmartER par-
ticipants would trend towards greater decreases in SSB
intake and improvements in mediating TPB-SSB vari-
After approval by Virginia Tech’s Institutional Review
Board, written informed consent was obtained prior to
enrollment in October 2011. Both conditions consisted of
two 90-minute small group sessions and three 5 - 10-
minute telephone calls (Figure 1). Previously executed
focus groups guided content development for key mes-
sages [17], and program components were specifically
designed to address TPB constructs including attitudes,
subjective norms, percieved behavioral control, and be-
havioral intentions for the referent behaviors (i.e. either
SSB or PA). Integration of health literacy concepts in-
cluded minimization of print materials, use of engaging
visual-based activities, use of simplifed print materials
written at <8th grade level, strong integration of media
literacy concepts, and use of intervention staff trained in
clear communication techniques. Throughout the pro-
gram, participants developed and updated personalized
action plans and used diaries to track behaviors.
Participants were recruited via flyers and word of
mouth from one community and one healthcare center in
Roanoke, Virginia. Eligibility criteria included >18 years
of age, English-speaking, without medical conditions
that contraindicate physical activity, and consuming >
200 SSB kcals/day as assessed with the validated 15-
item Beverage Questionnaire (BEVQ-15) [18]. Forty-
two of sixty-three screened individuals were eligible.
Twenty-five completed enrollment and were randomized
to SipSmartER (n = 14) or Move More (n = 11) (Figure
At the end of each group session, participants com-
pleted a self-administered process evaluation regarding
session content and delivery which included seven 5-
point likert scale questions and three open-ended ques-
After the program, participants completed an inter-
viewer-administered qualitative assessment that included
24 semi-structured questions related to group sessions,
personal action plans, diaries, and telephone calls.
Outcome data collection occurred at baseline and upon
completion of the program (week 6), and each took ap-
proximately 45 - 60 minutes. Previously validated in-
struments were utilized, including: 1) 15-item BEVQ-15
[18]; 2) 20-item Theory of Planned Behavior question-
naire for SSB [19]; 3) 9-item media literacy adapted to
reflect SSB [20]; and 4) 2 quality of life questions from
the Centers for Disease Control and Prevention [21]. Ad-
ditional baseline measures included 9 demographic ques-
tions, the 6-item validated Newest Vital Sign to assess
health literacy [22], and height and weight using stan-
dardized protocol. Participants were provided $25 and
$50 gift cards, respectively, for completing baseline and
follow-up assessments.
Qualitative data were coded as specific to group ses-
sions, personal action plans, diaries, telephone calls, or
non-specific, then coded as positive or negative, and
Copyright © 2013 SciRes. OPEN ACCESS
J. Zoellner et al. / Open Journal of Preventive Medicine 3 (2013) 51-57
Copyright © 2013 SciRes.
Figure 1. CONSORT diagram and program objective overview of SipSmartER and MoveMove.
subsequently examined for emerging themes. Quantita-
tive statistical analyses were performed using SPSS sta-
tistical analysis software, version 20. Descriptive statis-
tics and chi-squared tests were used to summarize all
quantitative measures. ANOVA tests were used to ana-
lyze group effects and group by time effects.
J. Zoellner et al. / Open Journal of Preventive Medicine 3 (2013) 51-57
Of the 25 enrolled participants, 19 (76%) were female
and 6 (24%) were male. Participants’ mean age was 42
(SD = 14) years, and were primarily Caucasian (n = 13;
52%) or African American (n = 12; 48%). Nine (36%)
had a high school education or less and 21 (84%) re-
ported <$25,000 annual household income. Health liter-
acy status indicated 6 (24%) participants with a high
likelihood of limited literacy skills, 7 (28%) with a pos-
sibility of limited literacy skills, and 11 (44%) with ade-
quate literacy skills. Eight participants (32%) were over-
weight and 16 (64%) were obese. There were no signifi-
cant differences between groups for any demographic
variables except education level (SipSmartER > Move-
More; F = 5.57; p = 0.03). When compared to US census
data, our sample appeared representative with the excep-
tion that men were underrepresented, while African
Americans and those with lower income or education
levels were overrepresented. The conditions did not dif-
fer on the reach of different intervention components (i.e.,
attendance, F = 0.01; p = 0.94; call completion, F = 0.91;
p = 0.35; Figure 1).
Related to participants’ assessment of content and
structure of the group classes, mean scores were rela-
tively high for both conditions and both classes, ranging
from 4.2 - 5.0 on a Likert scale of 1 (strongly disagree)
to 5 (strongly agree) (Ta b l e 1 ). Lesson components that
were favored among SipSmartER group sessions emerged:
realizing how much sugar is in beverages, recognizing
the health risks associated with drinking too much sugar,
understanding how much sugar they were consuming,
learning about better alternatives, and learning about the
media’s role in influencing SSB companies and how ad-
vertisements leave out important information on health.
Participants concluded that hands on activities (e.g.
learning about serving sizes, counting sugar packets)
were fun and engaging. Overall, participants thought
group sessions were “very beneficial,” “very informa-
tive,” “fun,” “captivating,” and “time well spent.” Sug-
gested improvements included bringing speakers for the
laptops, increasing the session duration, and encouraging
more participant discussion and questions.
Themes that emerged for the personal action plans
were that it encouraged responsibility and accountability,
offered ideas about strategies to overcome barriers,
helped make goals achievable, and helped to visualize
goals. The primary dislike was about the time needed to
complete it. While some participants enjoyed the chal-
lenge of setting and achieving goals, other participants
stated this challenge as a dislike.
The major positive emergent theme related to drink
diaries included the accountability with tracking daily
amounts of SSB. However, most participants disliked the
amount of time to record behaviors and struggled with
remembering to complete the diary. Most participants
expressed ease when asked about figuring out SSB
weekly averages, “All you have to do is add them up and
divide by the days.” However, a few participants ex-
pressed difficulties, “It was hard to look through each
day and each time per day.”
When asked about the telephone calls, SipSmartER
participants concluded that they were “supportive,” “kept
me motivated,” and “made it fun.” Dislikes included the
timing of the calls with one participant stating, “It was
hard to get calls at work or when I was driving.” Most
participants liked reporting their SSB intake over the
phone with one stating, “It was nice to speak with some-
one and set another goal.” When asked about strategies
offered over the phone, one participant stated, “They
were helpful, gave me new ideas, and nothing that I had
thought about before.”
Only one participant stated that the calls were not
helpful, because they did not have any barriers, while
another participant suggested that the phone calls needed
to be less scripted. Quantitatively, there was a significant
time effect on a number of study outcomes (Table 1).
Specifically, across groups, there were significant im-
provements in SSB behaviors, SSB affective and instru-
mental attitudes, SSB behavioral intention, and two me-
dia literacy outcomes (meanings/messages, e.g., SSB com-
panies create messages for certain purposes; representa-
tion/reality, e.g., SSB commercials omit certain health
information). However, SSB reduction differences be-
tween SipSmartER compared to MoveMore participants
were not significant (SipSmartER 256.9 + 622.6 kcals
versus MoveMore 199.7 + 404.6 kcals). There were no
significant differences for quality of life measures, sug-
gesting no unintended or potential negative conse-
This is the first known study to engage participants in
the refinement of an intervention integrating concepts
from health literacy with the TPB to reduce SSB behav-
iors among adults. As identified in the seminal health
literacy review by Berkman and colleagues [4], pilot
tested interventions, which engage the target population,
result in greater effects. Similarly to conclusions by
Berkman and colleagues [4], the observations of, and
information gathered from representatives of the target
population provided a number of key points to consider
for the larger trial, including: 1) refinement of small
group sessions (e.g. earlier integration of action planning,
promote more participant dialogue, change duration to
120 minutes); 2) incorporate explicit teach back methods
in the calls (e.g. assess understanding of SSB types,
servings sizes, calculating averages) to add clarity to the
instructions and learning materials, as well as reduce
Copyright © 2013 SciRes. OPEN ACCESS
J. Zoellner et al. / Open Journal of Preventive Medicine 3 (2013) 51-57
Copyright © 2013 SciRes. OPEN ACCESS
Table 1. Process & outcome results of SipSmartER and MoveMore conditions.
SipSmartER Mean (SD) (n = 11) MoveMore Mean (SD) (n = 9)
Class #1 Class #2 Class #1 Class #2
Process Evaluationa
The session was well
The information was easy to
The activities were fun 4.7
The session was the right
amount of time
I learned things in the
session that I did not know
The presenters seemed to
understand my concerns
The presenters knew what they
were talking about
SipSmartER Mean (SD) (n = 11) Move More Mean (SD) (n = 9) Overall
Pre Post Pre Post
Outcome E valuation
Health behaviors
Sugar-sweetened beverage
F = 3.58
P = 0.08
F = 0.06
P = 0.82
Sugar-sweetened beverage
F = 3.72
P = 0.07
F = 0.04
P = 0.85
Theory of Planned Scales for
Sugar-sweetened Beveragesb
Affective attitudes (3 items) 3.4
F = 9.57
P = 0.01
F = 0.10
P = 0.76
Instrumental attitudes (3 items) 4.6
F = 10.51
P < 0.01
F = 1.95
P = 0.18
Subjective norms (3 items) 5.0
F = 1.40
P = 0.25
F = 0.06
P = 0.80
Perceived behavioral Control
(3 items)
F = 0.51
P = 0.49
F = 0.09
P = 0.77
Behavioral intention total
(4 items)
F = 7.04
P = 0.02
F = 0.10
P = 0.76
Media literacy scales for
sugar-sweetened beveragesc
Authors/audiences (5 items) 3.4
F = 2.25
P = 0.15
F = 0.77
P = 0.39
J. Zoellner et al. / Open Journal of Preventive Medicine 3 (2013) 51-57
Meanings/messages (9 items) 3.5
F = 16.06
P < 0.01
F = 0.05
P = 0.83
Representation/reality (5 items) 3.4
F = 4.31
P = 0.05
F = 0.57
P = 0.46
Quality of life
Rate your general healthd 2.7
F = 0.31
P = 0.59
F = 0.31
P = 0.59
In past 30 day, how many days
did poor physical or mental
health keep you from usual
F = 0.87
P = 0.36
F = 0.12
P = 0.74
aReported on a 5-point Likert Scale: 1 = strongly disagree, 5 = strongly agree. bReported on a 5-point Likert Scale: 1 = worse, 5 = better attitudes, subjective
norms, perceived behavioral control, behavioral intentions. cReported on a 4-point Likert Scale: 1 = definetly no, 4 = definetly yes. dReported on a 5-point
Likert Scale: 1 = excellent, 5 = poor.
recall bias and variability while addressing the sensitivity
of the primary outcome measure; and 3) refinement of
recruitment and enrollment protocols. The value of using
a patient-centered approach in the developmental phases
of this theory-guided SSB behavioral intervention was
In general, the behavior change, while not signifi-
cantly different between groups, trended in the direction
hypothesized (i.e. greater SSB improvements in the
SipSmartER as compared to the MoveMore). Being
made aware of the study purpose through informed con-
sent procedures and the repeated exposure to SSB rec-
ommendations through the assessment process may have
prompted SSB improvements in the control group. This
is consistent with the literature on mere-measurement
effects, which demonstrates short-term (but not long-
term) behavioral responses to sets of questions related to
measurement of behavioral and psychosocial constructs
similar to those proposed in our study [23]. It is hypothe-
sized that an adequately powered trial, of longer duration
and timing between data assessment points, will over-
come this challenge.
The RE-AIM approach for planning the intervention
seemed to be successful in creating a structure that could
consistently reach the study sample and including con-
tent that they enjoyed [16]. This initial feedback from
participants provides promising directions for under-
standing the reach (including representativeness) and
effectiveness of a TPB and health literacy-based SSB
intervention. Future evaluative efforts will include as-
sessing reach, effectiveness, adoption, implementation,
and maintenance to promote comprehensive understand-
ing of internal and external validity factors, as well as
potential public health impact of a large-scale trial to
reduce SSB.
We acknowledge the research support provided by Terri Corsi, Sarah
Wall, Valisa Hedrick, Lauren Noel, Angie Bailey, Ramine Alexander,
and Felicia Reese. We are particularly grateful for contributions from
Eileen Lepro at New Horizons Healthcare and staff at the Presbyterian
Community Center. This research was funded, in part, by National
Institutes of Health/National Cancer Institute 1R01CA154364-01A1
(Zoellner, PI).
[1] Duffey, K.J. and Popkin, B.M. (2007) Shifts in patterns
and consumption of beverages between 1965 and 2002.
Obesity, 15, 2739-2747.
[2] Vartanian, L.R, Schwartz, M.B. and Brownell, K.D.
(2007) Effects of soft drink consumption on nutrition and
health: A systematic review and meta-analysis. American
Journal of Public Health, 97, 667-675.
[3] Nielsen-Bohlman, L., Panzer, A.M. and Kindig, D.A.
(2004) Health Literacy: A Prescription to End Confusion.
National Academies Press,Washington DC.
[4] Berkman, N.S.S., Donahue, K., et al. (2011) Health liter-
acy interventions and outcomes: An update of the literacy
and health outcomes systematic review of the literature.
RTI International-University of North Carolina Evidence-
Based Practice Center, Chapel Hill.
[5] Zoellner, J., You, W., Connell, C., et al. (2011) Health
literacy is associated with Healthy Eating Index scores
and sugar-sweetened beverage intake: Findings from the
rural lower Mississippi Delta. Journal of American Die-
tetic Association , 111, 1012-1020.
[6] Allen, K., Zoellner, J., Motley, M., et al. (2010) Under-
standing the internal and external validity of health liter-
acy interventions: A systematic literature review using the
RE-AIM framework. The Journal of Health Communica-
tion, 16, 55-72. doi:10.1080/10810730.2011.604381
[7] Tate, D.F., Turner-McGrievy, G., Lyons, E., et al. (2012)
Replacing caloric beverages with water or diet beverages
for weight loss in adults: Main results of the Choose
Copyright © 2013 SciRes. OPEN ACCESS
J. Zoellner et al. / Open Journal of Preventive Medicine 3 (2013) 51-57 57
Healthy Options Consciously Everyday (CHOICE) ran-
domized clinical trial. American Journal of Clinical Nu-
trition, 95, 555-563. doi:10.3945/ajcn.111.026278
[8] Stookey, J.D., Constant, F., Popkin, B.M., et al. (2008)
Drinking water is associated with weight loss in over-
weight dieting women independent of diet and activity.
Obesity, 16, 2481-2488. doi:10.1038/oby.2008.409
[9] Chen, L.W., Appel, L.J., Loria, C., et al. (2009) Reduc-
tion in consumption of sugar-sweetened beverages is as-
sociated with weight loss: The PREMIER trial. American
Journal of Clinical Nutrition, 89, 1299-1306.
[10] Wen, K.-Y., Miller, S.M., Stanton, A.L., et al. (2012) The
development and preliminary testing of a multimedia pa-
tient-provider survivorship communication module for
breast cancer survivors. Patient Education and Counsel-
ing, 88, 344-349. doi:10.1016/j.pec.2012.02.003
[11] Helfand, M., Berg. A., Flum, D., et al. (2012) Draft Meth-
odology report: Our questions, our decisions: Standards
for patient-centered outcomes research.
[12] Venetis, M.K., Robinson, J.D. Turkiewicz, K.L., et al. (2009)
An evidence base for patient-centered cancer care: A
meta-analysis of studies of observed communication be-
tween cancer specialists and their patients. Patient Edu-
cation and Counseling, 77, 379-383.
[13] Ajzen, I. (1985) From intentions to actions: A theory of
planned behavior. In: Kuhl, J. and Beckmann, J., Eds.,
Action-Control: From Cognition to Behavior, Springer,
Heidelberg, 11-39.
[14] Zarcadoolas, C., Pleasant, A. and Greer, D. (2006) Ad-
vancing health literacy: A framework for understanding
and action. Jossey-Bass, San Francisco.
[15] Aufderheide, P. (1993) Part II: Conference Proceedings
and Next Steps. Communications and Society Program of
the Aspen Institute, Washington DC.
[16] Glasgow, R.E., Vogt, T.M. and Boles, S.M. (1999) Evalu-
ating the public health impact of health promotion inter-
ventions: The RE-AIM framework. American Journal of
Public Health, 89, 1322-1327.
[17] Zoellner, J., Krzeski, E., Harden, S., et al. (2012) Qualita-
tive application of the theory of planned behavior to un-
derstand beverage consumption behaviors among adults.
Journal of the Academy of Nutrition and Dietetics, 112,
1774-1784. doi:10.1016/j.jand.2012.06.368
[18] Hedrick, V.E., Savla, J., Comber, D.L., et al. (2012) De-
velopment of a brief questionnaire to assess habitual bev-
erage intake (BEVQ-15): Sugar-sweetened beverages and
total beverage energy intake. Journal of the Academy of
Nutrition and Dietetics, 112, 840-849.
[19] Zoellner, J., Estabrooks, P., Davy, B., et al. (2012) Ex-
ploring the theory of planned behavior to explain sugar-
sweetened beverage consumption. Journal of Nutrition
Education and Behavior, 44, 172-177.
[20] Primack, B.A., Gold, M.A., Switzer, G.E., et al. (2006)
Development and validation of a smoking media literacy
scale for adolescents. Archives of Pediatric and Adoles-
cent Medicine, 160, 369-374.
[21] Klesges, R., Eck, L., Mellon, M., et al. (1990) The accu-
racy of self-reports of physical activity. Medical Science
Sports and Exercise, 22, 690-697.
[22] Weiss, B., Mays, M., Martz, W., et al. (2005) Quick as-
sessment of literacy in primary care: The newest vital
sign. Annuals of Family Medicine, 3, 514-522.
[23] Levav, J. and Fitzsimons, G. (2006) When questions
change behavior: The role of ease representation. Phy-
scological Science, 17, 207-213.
Copyright © 2013 SciRes. OPEN ACCESS