2012. Vol.3, No.7, 525-526
Published Online July 2012 in SciRes (
Copyright © 2012 SciRes. 525
Examining the Neurocognitive Validity of Commercially
Available, Smartphone-Based Puzzle Games
Oonagh Thompson, Suzanne Barrett, Christopher Patterson, David Craig
Centre for Public Health, Qu e e n ’s University Belfast, Belfast, U n i t e d K i n g d o m
Received April 6th, 2012; revised May 2nd, 2012; accepted Ju n e 1st, 2012
Cognitive assessment typically involves assessing a person’s cognitive performance in unfamiliar and ar-
guably unnatural clinical surroundings. User-centred approaches to assessment and monitoring, driven by
issues such as enjoyability and familiarity, are largely absent. Everyday technologies, for example,
smartphones represent an opportunity to obtain an objective assessment of a person’s cognitive capabili-
ties in a non-threatening, discreet and familiar way, e.g. by everyday puzzle games undertaken as a leisure
activity at home. We examined the strength of relationships that exist between performance on common
puzzle games and standard measures of neuropsychological performance. Twenty-nine participants, aged
50 - 65 years, completed a comprehensive neuropsychological test battery and played three smart-
phone-based puzzle games in triplicate: a picture puzzle [Matches Plus], a word puzzle [Jumbline] and a
number puzzle [Sudoku]. As anticipated, a priori, significant correlations were observed between scores
on a picture puzzle and visual memory test (r = 0.49; p = 0.007); a word puzzle and estimated verbal IQ (r
= 0.53; p = 0.003) and verbal learning (r = 0.30; p = 0.039) tests; and a number puzzle and reason-
ing/problem solving test (r = 0.42; p = 0.023). Further analyses making allowance for multiple compari-
sons identified a significant unanticipated correlation (r = 0.49; p = 0.007) between number puzzle scores
and a measure of nonverbal working memory. Performance on these smartphone-based games was in-
dicative of relative cognitive ability across several cognitive domains at a fixed time point. Smart-
phone-based, everyday puzzle games may offer a valid, portable measure of assessing and monitoring
cognition in older adults.
Keywords: Cognitive Assessment; Alzheimer’s Disease; Information & Communication Technology
(ICT); Smartphone
Current methods for assessing and monitoring Alzheimer’s
disease (AD) can be l engthy and obtrusive and rely upon testing
cognitive performance at fixed time-points, often in arguably
unnatural clinical settings. The majority of assessment method-
ologies are developed with neurocognitive assessment princi-
ples as the starting point.
Clinical assessment and diagnosis of dementia in the UK is
based upon a clinical interview and assessment of activities of
daily living, often involving the input of a relative or friend; the
administration of standardised assessment scales to assess the
severity of impairment in the individual; and a physical exami-
nation and/or brain scan where considered appropriate by the
physician. The most commonly used assessment scale is the
30-item Mini-Mental State Examination [MMSE] (Folstein et
al., 1975). The MMSE includes key domains of cognitive func-
tioning: orientation, registration, attention, recall, visuospatial
capabilities and language delivered in a structured setting. Pa-
tient monitoring usually takes the form of follow-up clinic ap-
pointments with the physician or nurse, including further clini-
cal interviews and formal cognitive assessment to identify
Interest is growing in new assessment methods that combine
assessment and monitoring and which reduce the need for mul-
tiple, fixed time-point clinic visits while offering a more con-
venient, home-based alternative. Such user-centred approaches
to assessment and monitoring, driven by issues such as enjoy-
ability, familiarity, convenience and availability, are largely
Information and communication technology (ICT), particu-
larly the smartphone, represents an opportunity to obtain an
objective assessment of a person’s cognitive capabilities in a
non-threatening, discreet and familiar way, e.g. by everyday
puzzle games undertaken as a leisure activity. The mobile
phone is an accessible, portable, everyday technology with 77%
uptake among older adults in the UK, and smartphone market
penetration is rising (Ofcom, 2011). If performance on every-
day puzzles maps to scores attained on conventional neuropsy-
chological tests, then smartphone-based puzzles may offer a
valid portable measure of cognition. To investigate this idea
further, we sought to examine the strength of relationships that
exist between performance on three widely available puzzle
games and standard measures of neuropsychological perform-
Twenty-nine older adults (gender [f:m] = 19[66%]:10[34%];
mean age = 56.2; range = 50 - 65 years) were recruited through
local ageing networks to participate in this study. During single
researcher-supervised sessions participants completed a com-
prehensive neurocognitive assessment and played three time-
limited, smartphone-based puzzles in triplicate (tests and games
listed in Table 1). A standard operating procedure was used to
ensure consistency of test adminstration and game instruction. i
Table 1.
Relationship between puzzle game scores and neurocognitive test performance.
Puzzle Games
Word Game Jumbline Number Game Sudoku Picture Game Matches Plus
Neurocogn it ive Doma in : Neurocognitive Te st
r r r
Verbal IQ Estimate: National Adult Reading Test [NART] 0.53**1 0.22 –0.04
Verbal At tention/Work i ng Memory Span: Letter-Number Spa n [LNS] 0.27 0.23 0.29
Verbal Memory-Immediate: HVLT-R™ 0.271 0.27 0.40*
Verbal Memory-Delayed: HVLT-R™ 0.39*1 0.16 0.22
Visuospatial Attention/Working Memory Span: WMS®-III 0.21 0.49** 0.25
Visual Memory-Immediate: BVMT-R™ 0.32 0.27 0.49**1
Visual Memory-Delayed: BVMT-R™ 0.26 0.15 0.341
Attention/Speed of Processing: Trail Ma ki ng Test—Part A –0.35 –0.39 –0.21
Attention/Vigilance: Continuous Perform ance Test—Identical Pairs 0.1 0.24 0.16
Executive Function/Speed of Processing: BACS—Symbol Coding 0.09 0.28 0.42*
Executive Fu nction/Reasoning & Problem Solving: NAB®—Mazes 0.14 0.42*1 0.35
Verbal Flue ncy/Speed of Processing: Controlled Oral Word Association0.26 0.16 0.28
Pearson’s correlations: moderat e effect size = r > 0.3 ; large effect size = r > 0.5; St atistical si gnificance (2-tailed): *p < 0.05; **p < 0.01; 1a priori predictions; BACS: Brief
Assessment of Cognition in Schizophrenia; HVLT-R™: Hopkins Verbal Learning Test™-Revised; WMS®-III: Wechsler Memory Scale®-Third Edition; NAB®: Neuro-
psychological Assessment Battery® mazes; BMVT-R™: Brief Visuospatial Memory Test-Revised.
Neurocognitive tests were selected on the basis of having lim-
ited floor and ceiling effects, high tolerability for participants
and applicability in clinical trials (Nuechterlein et al., 2008).
The three games were selected on the basis of having good face
validity with regards to verbal memory [Jumbline], problem
solving [Sudoku] and visual memory [Matches Plus], and sat-
isfactory user ratings (minimum 3/5 stars) on the online “App
Jumbline is a word puzzle that involves finding as many
words as possible within a set of given letters. Each level of the
game presents a new set of letters and players work against a
timer to try to complete each level in order to progress to the
next. Sudoku is a number-placement puzzle that involves in-
serting numbers into a partially filled 9 × 9 grid. The player’s
objective is to complete the grid so that the digits 1 to 9 are
contained within each row, each column and each 3 × 3 sub-
grid without duplication. Matches Plus is a matching game that
involves matching pairs of identical symbols as quickly as pos-
sible. Players work against a timer to try to match all pairs to
progress to the next level.
Based on game characteristics, we anticipated that the fol-
lowing would show moderate to large correlations: 1) Word
game [Jumbline] scores and tests of verbal memory; 2 Number
game [Sudoku] scores and tests of problem solving and logical
reasoning; 3) Picture game [Matches Plus] scores and tests of
visual memory.
The relationships between puzzle game and neuropsy-
chological test performance are reported in Table 1. In line
with a priori predictions, significant correlations were observed
between: word game scores and performance on the National
Adult Reading Test [NART] measure of reading ability/esti-
mated IQ and the Hopkins Verbal Learning Test™-Revised
[HVLT-R™] measure of verbal memory; number game scores
and the Neuropsychological Assessment Battery® [NAB®] test
of reasoning and problem solving; and picture game scores and
the Brief Visuospatial Memory Test-Revised [BVMT-R™]
measure of visual memory. Post-hoc exploratory analyses at the
1% significance level identified an additional, significant mod-
erate correlation between performance on the number game and
the Wechsler Memory Scale®-Third Edition [WMS®-III] mea-
sure of visuospatial working memory.
To our knowledge, this is the first study to examine the util-
ity of everyday, smartphone-based puzzles for validly assessing
cognitive abilities in this age group. Of note, in this middle-
aged, cognitively intact cohort, we were able to identify moder-
ately sized associations between conventional cognitive tests
and smartphone-based games in keeping with their face validity.
Further work will explore the ability of smartphone-based puz-
zles to detect cognitive impairment, as well as the level of im-
pairment that might limit smartphone use.
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-mental
state. A practical method for grading the cognitive state of patients
for the clinician. Jou rnal of Psychiatric Research, 12, 189-198.
Nuechterlein, K. H., G reen, M. F., Kern, R. S., Baade, L. E., Barch , D.
M., Cohen, J. D., & Marder, S. R. (2008). The MATRICS consensus
cognitive battery, part 1: Test selection, reliability, and validity.
American Journal of Psychiatry, 165, 203-213.
Ofcom. (2011). The Communications Market Report: United Kingdom.
A nation addicted to smartphones. UK.
Copyright © 2012 SciRes.