
==== Front
Int J Sports Phys Ther
Int J Sports Phys Ther
2159
International Journal of Sports Physical Therapy
2159-2896
NASMI Website: International Journal of Sports Physical Therapy

34386278
24924
10.26603/001c.24924
Original Research
Accuracy of the SWAY Mobile Cognitive Assessment Application
https://orcid.org/0000-0002-1491-1660
VanRavenhorst-Bell Heidi A. PhD 1
Muzeau Melissa A. MES 2
Luinstra Lindsay DAT 1
Goering Jared MID 3
Amick Ryan Z. PhD 4
1 Human Performance Studies Wichita State University https://ror.org/00c4e7y75
2 Engineering and Ergonomics of Physical Activity, University Savoie Mont-Blanc; Sporttesting
3 College of Innovation and Design Wichita State University https://ror.org/00c4e7y75
4 Independent Consultant
Corresponding author: Heidi Ann VanRavenhorst-Bell, PhD Assistant Professor, Department of Human Performance Studies Wichita State University 1845 Fairmount St., Campus Box 016, Wichita, Kansas 67260-0016 Phone: 316.978.5150 Fax: 316.978.5451 E-mail: heidi.bell@wichita.edu
1 8 2021
2021
16 4 9911000
24 12 2020
22 2 2021
© The Author(s)
https://creativecommons.org/licenses/by-nc-sa/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike License (4.0) which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and source are credited. If you remix, transform, or build upon this work, you must distribute your contributions under the same license as the original.

Background

Mobile electronic devices have become integral tools in addressing the need for portable assessment of cognitive function following neurocognitive/motor injury. SWAY Medical, Inc., has employed mobile device motion-based technology in the SWAY Cognitive Assessment (SWAY CA) application to assess cognitive function.

Purpose

The purpose of this study was to assess whether the SWAY CA application (reaction time, impulse control and inspective time) was able to reliably operate on different mobile devices and operating systems (iOS, Android). The study further sought to assess the validity of the SWAY CA application against the FDA approved ImPACT QT mobile device application.

Study Design

Original Research, observational study of validity.

Methods

88 healthy, young adults, 18 to 48 years (mean= 22.09 ± sd=4.47 years) completed four, randomized and counter-balanced, reaction time tests (2- SWAY RT, 2- ImPACT QT) using different operating systems (iOS, Android) of 4 randomly assigned mobile devices.

Results

ANOVAs reported the SWAY CA application (reaction time, impulse control, inspection time) operated reliably with iPhone 6S, Samsung Galaxy S9, and iPad Pro 5 mobile devices (p > 0.05), respectively. Google Pixel 3 reliability with SWAY CA application remains undetermined. SWAY CA simple reaction motion measures were in agreement (r = -0.46 to 0.22, p ≤ 0.05) with several ImPACT QT reaction time measures. SWAY CA impulse control and inspection time measures are weakly correlated (r = -0.25 to -0.46, p ≤ 0.05) with five ImPACT QT reaction time measures.

Conclusion

The motion-based SWAY CA mobile device application appears to reliably operate when being administered on different mobile devices and software operating systems. Furthermore, the SWAY CA application appears to be comparable to the ImPACT QT and serve as a valid tool for assessing reaction time measures.

Level of Evidence

Level 2b (observational study of validity).

impact qt
mobile device
reaction time
cognitive assessment
sway app
==== Body
pmcINTRODUCTION

Reaction time is known as an individual’s rate of response (or amount of time lapsed) following the introduction of a known or unknown stimulus.1 It is an important indicator of one’s neurocognitive and functional health,1–3 as well as being a key factor in many daily activities such as participating in sport, driving a car, and even in emergency situations.3 The assessment of reaction time has long been used to evaluate an individual’s cognitive, neurological, and motor (dys)function,4 and more recently has become a respected measure for return-to-play in sport(s) following mild-traumatic brain injury (MTBI).3,5,6 For example, immediately following a sport-related concussion, it is widely accepted that an individual will present with a prolonged reaction time due to an insult on the brain.3,5,7 With time, concussion-induced disruptions in neurocognitive and functional performance are often shown to gradually dampen, and an improvement in reaction time returns.1,8 Furthermore, routine follow-up reaction time assessments are commonly performed and compared to an individual’s pre-concussion (baseline) reaction time measures to determine post-concussion improvements in neurocognitive and functional health prior to an athlete being released for a return-to-play.5,7

The assessment of reaction time has many benefits which include but are not limited to serving as a parallel indicator of one’s central processing speed and cognitive function.5 Traditionally, qualitative evaluations of neurocognitive function and reaction time measures down to the millisecond have involved some form of computerized testing (e.g., software on a desktop computer with a keyboard and mouse).1,5,7,9 Computerized testing is known for its accuracy and reliability5,10; however, are generally administered in a clinical setting and commonly criticized for their lack of portable practical application.5,9 Even laptops, which are viewed as a portable computerized device, require some set-up, along with an appropriate setting to successfully administer a cognitive and reaction time assessment. Such requirements complicate the feasibility of a portable on-field (i.e., athletic venues, athletic training room, military field hospital) assessment application. This is of concern because timely administration of cognitive and reaction time assessments are critical when assessing a potential on-field neuromotor injury.11 A delay in assessment may allow for misdiagnosis, which could result in harm or death of the patient or athlete. Therein, supports the need for a portable practical application to assess reaction time.

Mobile electronic devices such as smartphones and tablets are portable and user-friendly in most any setting (e.g., clinical, medical, and on-field). Most mobile electronic devices are also capable of operating mobile application software as well as administering various health and sport related assessment measures due to an inertial measurement unit (IMU) system built into the mobile device.12–14 IMUs measure specific force, angular velocity and sometimes the orientation of the body or movement of the device, using a combination of magnetometers, gyroscopes and triaxial accelerometers.12,15 In addition, mobile device applications can provide rapid biofeedback (e.g., neurocognitive measures, neuromotor measures, reaction time measures) based on the device IMU measures.12,16 Due to the portability and cognitive assessment application capabilities of a mobile device, use in assessing neurocognitive and neuromotor injuries has become of interest.7,9

One such mobile device application is the Immediate Post-Concussion Assessment and Cognitive Test Quick Test (ImPACT QT). The ImPACT QT is an FDA cleared mobile device application developed to assess neurocognitive function following a suspected concussion.17 Due to its mobility and ease of use, the ImPACT QT is commonly used for sideline assessments in high school and collegiate athletics, as well as during routine clinical assessments.5,7 The five-minute ImPACT QT test includes a series of neurocognitive modules (symbol matching, three letter memory, reverse number counting, attention tracking) administered on a tablet screen. An individual’s rate of response (e.g., neurocognitive and reaction time measures) is recorded by touching the tablet screen following a visual prompt displayed on the display screen. During the assessment, the tablet may be held with both hands or placed on a flat surface while remaining in a standing posture. Following completion of the test, the ImPACT QT application provides three composite scores that may be compared against a subject’s previously established baseline measures.18,19 A decline in the composite scores is often used as an indication of a potential decline in neurocognitive function and consideration for removal of an individual from activity.6,19,20 Wallace and colleagues,19 however, caution of interpretation based on a single low score without cause of concern of a concussion because healthy non-concuss individuals have been shown to randomly present with an unexplained low score.

The ImPACT QT does present with a few limitations. First, the ImPACT QT is only compatible with an iOS (Apple, Cupertino, CA, USA) touch-screen iPad.6,19,20 An iPad, although well-accepted, is not the universal tablet among all end-users, nor clinical and athletic programs. This greatly marginalizes its accessibility and intended purpose of providing critical and often time sensitive sideline assessments. In addition, due to slower software and processor, the iPad is observed to have screen capacitance latency and test results are susceptible to a wider range of variability compared to a traditional desktop or laptop computer assessment. As screen latency can range from 50 to 200 milliseconds, latency induced variability may have an indirect effect on an individual’s true reaction time scores, potentially impacting clinical decisions.1,5,9,21–23 For example, if during an athlete’s baseline assessment screen was between 100 to 200 milliseconds, and was between 50 to 100 milliseconds during an on-field concussion assessment; the end result could be a missed or failed interpretation. The on-field assessment indicated a faster, although inaccurate reaction time measure in comparison to the athlete’s baseline measure. Such inconsistencies due to latent variability of the iPad may place a patient-athlete at risk for another traumatic event that could potentially be more detrimental to neurocognitive function or even fatal.24 In addition, healthcare professionals may experience limitations in making an appropriate diagnosis when attempting to evaluate data comparisons between computer aided testing and reaction time assessments administered on an iPad due to this wide range variability.

Recently, a new method for assessing cognitive function using a mobile device was introduced by SWAY Medical, Inc. The SWAY Cognitive Assessment (SWAY CA) mobile device application registers movement of the mobile device, instead of registering an applied touch screen response.9,25,26 The SWAY CA introduces a series of neurocognitive modules that evaluate an individual’s reaction time, inspection time, impulse control, and working memory-delayed recall. While an individual holds the device with both hands, a module prompts a visual cue on the display screen and evokes the individual to engage in or refrain from an active response (moving the device). An active response is recognized as a minimum motion-based threshold detected as the device is moved in any direction. The cognitive function and reaction time measures (time lapsed from the presence of a stimulus to the initiation of an action) for each of the three SWAY CA modules (simple reaction motion, impulse control, and inspection time)(see Methods section) are reported in milliseconds (ms) and a proprietary SWAY score calculated on a 100-point scale.25,26 The closer an individual’s score is to 100 the better one’s cognitive function and reaction time.25,26 The SWAY CA working memory-delayed recall module, however, is a single proprietary SWAY score based off lapsed time to recall, number of correct recall, and number of sequential squares tracked and recorded correctly.26

To assess movement of the mobile device and interpret one’s rate of reaction time, SWAY CA’s proprietary algorithm uses a triaxial accelerometer motion-based system that is housed within the mobile device. Due to the orthogonal (right angles) placement of the three sensors in reference to each other, detection of device movement and vibration in any direction is registered with increased sensitivity compared to a system with less than three sensors. This increased sensitivity to motion has been shown to minimize mobile device latency down to one to two milliseconds.21,23,26,27 This is a pronounced improvement compared to touch-based reaction time mobile device detection with an average latency of 50 to 200 milliseconds.1,5,9,21–23 An additional advantage to SWAY CA is that it can be used on multiple platforms (smartphones and tablet) and is compatible with iOS (Apple, Cupertino, CA, USA) and Android (Samsung Group, Seoul, South Korea; Google, Mountain View, CA, USA) operating systems.25,26,28

While the prospect of using mobile electronic devices as a clinical evaluation tool has many advantages, developers must ensure that their applications provide consistent results across all devices on which they are intended to operate. This is because, among the most popular smartphone and tablet devices, the number of different hardware and software combinations being used is numerous. Such differences may result in minor compatibility issues that impact processing speed, display screen refresh rate, and input latency. Additionally, different manufacturers may use different solutions for analyzing raw data from integrated sensors.1,12,23 Ultimately, for a mobile application to be versatile and provide clinically relevant and reliable assessments, it is essential to account for these differences across a spectrum of mobile devices and operating systems.

The purpose of this study was to assess whether the SWAY CA application (reaction time, impulse control and inspective time) was able to reliably operate on different mobile devices and operating systems (iOS, Android). The study further sought to assess the validity of the SWAY CA application against the FDA approved ImPACT QT mobile device application.

METHODS

Site Selection

This study was completed in the Human Performance Laboratory (HPLab) at Wichita State University, Wichita, Kansas. This site was selected as the HPLab is experienced in the development and evaluation of mobile device applications.

Participants

A total of 90, college-aged individuals with a mean age of 22.09 ± sd= 4.42 years volunteered to participate in the study. An a priori power analysis was conducted using G*Power 3.1 software (Heinrich-Heine-Universitat Dusseldorf, Dusseldorf, Germany) to identify appropriate sample size. To achieve a power of 0.80 with an α error of probability ≤ 0.05 and a medium effect size, a sample size of 84 participants was required. Volunteers were recruited through direct contact, and technology-based communication, as well as through print materials posted in public areas on the university campus. The Wichita State University Institutional Review Board approved the study, and an informed consent form was obtained from all volunteers prior to completing any questionnaire(s) or participating in data collection.

Inclusion and exclusion criteria. Any pre-existing condition that could interfere with successfully completing the assessment was identified based on the 2020 Physical Activity Questionnaire Plus (PAR-Q+).29 A participant was excluded from the study if they were under the age of 18 year, and were excluded if they reported any of the pre-existing conditions presented as follows; any current medical condition or medical history of a 1) musculoskeletal injury affecting functional movement and balance, 2) neurological dysfunction, 3) uncorrected vision, 4) vestibular disorder or condition, and/or 5) current, un-prescribed or prescribed pharmacological intervention affecting functional movement and balance.

Of the initial 90 volunteers, one participant was excluded for meeting one or more of the exclusion criteria. The remaining 89 participants met the intake questionnaire and were included in the study. One additional participant was removed from the study due to a technology error and inability to download the data output from the mobile device. For the remaining 88 participants, Table 1 provides the demographic information (age, sex), as well as anthropometric measures (height, weight) collected.

64075 Table 1. Subject Demographic Information

	Male (n = 32)	Female (n = 56)	Total (N = 88)	
	Mean ± SD	
Age (years)	22.38 ± 5.68	21.93 ± 3.63	22.09 ± 4.46	
Stature (cm)	178.73 ± 8.27	167.25 ± 7.42	171.43 ± 9.49	
Weight (kg)	83.25 ± 14.9	76.25 ± 20.26	78.79 ± 18.72	
n = Sum of sample, N = Sum of total sample, cm = Centimeters, kg = Kilograms

SWAY Mobile Application

The SWAY System (SWAY Medical Inc., Tulsa, OK, USA) is a mobile device application designed to assess balance (SWAY Balance) and cognitive performance (SWAY CA) through the use of different assessment modules. Both segments of the SWAY System rely primarily on the analysis of movement, as measured through the mobile device’s integrated triaxial accelerometer, to determine performance scores.9,25,27,28,30,31 Evaluation of the balance assessment segment of the SWAY System has previously been reported and received FDA Class II approval.28 The cognitive (reaction time) testing segment has also been evaluated and established clinically reliable and valid measures in comparison to the standard Computerized Test of Information Processing (CTIP) assessment.9 However, SWAY CAs capacity to execute on various mobile devices and operating systems, as well as deliver measures consistent in comparison to the standardized ImPACT QT mobile application remain to be validated. The cognitive performance segment of the SWAY System, SWAY CA, administers three sensory and neuromotor based modules to assess stimulus recognition, cognitive processing speed, neuromotor response, working memory and reaction time.

Module 1 – Simple Reaction Time

Move the device as fast as one can in any direction when the screen of the device turns orange.

Module 2 – Impulse Control

Move the device as quickly as possible when you see a green check mark.

When you see a red X, keep the device still.

Module 3 – Inspection Time

Two T-shaped lines will be shown on the device. Once the two lines are masked (covered), you will be instructed to move the device to the side with the longer line.

Do not move the device if you are unsure which line was longer. An incorrect response will reduce one’s score.

SWAY CA utilizes tri-axial accelerometers built-in to most mobile devices to detect motion21,25,26,28,32 and measure reaction time in reference to a known stimulus.1,2 Overall, SWAY CA is completed in three to five minutes by the participant.

For each SWAY CA module, participates were instructed to follow the SWAY System instructions displayed on the device screen. Research personnel trained on the SWAY System were continually present to offer further clarification to participants if needed on the application or electronic device. For all modules, participants were instructed to hold the device with both hands and maintain a standing position.

Participants were randomly issued one of four mobile devices preloaded with the SWAY CA application. Device selection was based on convenience of accessibility at time of the study. Mobile devices included:

Apple iPhone 6s Plus, Software Version – iOS 12.2 Model: MKTQ2LL/A, Serial: C38QFBM5GRWT (Apple Computer Inc., Cupertino, CA, USA)

Samsung Galaxy S9, Software Version – Android 9 (8.0.0), Serial: R58M217YT7P (Samsung Group, Seoul, South Korea)

Google Pixel 3, Software Version – Android Version 9 (8.0.0), Serial: 89VXOHN87 (Google LLC., Mountain View, CA, USA)

Apple iPad 5 Air, Software Version – 12.1.1 (16C50), Model: MR7F2LL/A, Serial: DMRY26GRJF8J and Serial: DMRY236PJF8J (Apple Computer Inc., Cupertino, CA, USA)

ImPACT Quick Test Mobile Application

The ImPACT QT (ImPACT Applications, Inc., San Diego, CA, USA) is an FDA approved iPad-based neurocognitive test designed for clinical use (e.g., concussion baseline measures, pre- and post-neurocognitive injuries (concussion assessment)).6,17,33 ImPACT QT administers three neurocognitive modules to assess basic output related to neurocognitive functioning, working memory, processing speed, reaction time and symptom recording in a brief five-seven minutes.6,19,33 The three neurocognitive modules are as follows.

Module 1 – Symbol Matching

Trial 1: Match shapes with numbers using the touch screen as quickly as you can.

Trial 2: Remember which shape goes with what number using the touch screen as quickly as you can.

Module 2 – Three Letter Memory and Reverse Number Counting

Trial 1: Count backwards from 25 to 1 using the touch screen as fast as you can.

Trial 2: Remember a set of letters flashed on the touch screen. Then count backwards from 25 to 1 using the touch screen as fast as you can. Immediately following, type the letters you were asked to remember.

Module 3 – Attention Tracking

Visually track a moving object on the touch screen. When you identify the object change from read to green you will click on the circle as fast as you can.

The ImPACT QT test was administered utilizing an Apple iPad Pro 5, [Software Version – 12.1.1 (16C50), Model: MR7F2LL/A, Serial: DMRY26GRJF8J and Serial: DMRY236PJF8J (Apple Computer Inc., Cupertino, CA, USA)]. For each module, instructions were provided on the device screen as well as each participant was provided verbal instruction from an experienced research administrator. Participants were instructed to lay the device flat on the countertop surface and maintain a standing position while performing each module.

Procedure. Each participant completed a total of four cognitive assessments (two SWAY CA and two impact QT). Following a similar protocol described in detail in a previous study for balance by Amick and colleagues,28 each participant completed one familiarization trial and one experimental (baseline) trial for each application (SWAY CA, ImPACT QT). To control for a learning effect and bias, participants were issued one of the four previously described preloaded SWAY CA application mobile devices, and a preloaded ImPACT QT iPad in a randomized order. In addition, the order of the two application cognitive assessments (SWAY CA, ImPACT QT) was counter-balanced (e.g., SWAY – ImPACT QT – SWAY – ImPACT QT, or ImPACT QT – SWAY – ImPACT QT – SWAY). Each participant was provided a two-three-minute seated rest period between test applications. The research administrator used a stopwatch to maintain consistent rest periods.

Data Analysis

Statistical analysis was conducted using the Statistical Packages for the Social Science (SPSS) version 23.0 with a level of significance set at α ≤ 0.05 and a confidence level of 95%. All test variables were evaluated for normality of distribution.

Three separate one-way analyses of variance (ANOVAs) were conducted to determine group mean difference of Mobile Device (iPhone 6s Plus, Google Pixel 3, Samsung S9, iPad Pro 5) on each of the SWAY CA baseline measures (simple reaction, impulse control, inspection time). The critical alpha level for each ANOVA was set at p ≤ 0.05. A post-hoc test was completed at a p ≤ 0.05 if a significant mean difference was reported.

A Pearson’s Product Moment Correlation Coefficient (r) was conducted to determine the degree of correlation in baseline SWAY CA measures (simple reaction motion, impulse control, inspection time) and the ImPACT Quick Test application battery of modules at a p ≤ 0.05. The Coefficient of Determination (r2) was further calculated to determine the amount of shared variance between the SWAY CA and ImPACT QT scores. A Pearson’s Product Moment Correlation Coefficient Interpretation as follows, weak r= 0.00 to 0.30, moderate r= 0.31 to 0.59, and strong r= 0.60 to 1.00.7,17

RESULTS

All SWAY CA and ImPACT QT measures were inspected and found to fall within an acceptable range and demonstrated a normal distribution. Table 2 provides the means and standard deviations of each SWAY CA measure (simple reaction, impulse control and inspection time) by mobile device (iPad, iPhone 6S, Google Pixel 3 and Samsung Galaxy 9S).

64076 Table 2. Means and Standard Deviations of SWAY CA Simple Reaction, Impulse Control and Inspection Time by Mobile Device

	SWAY CA Modules (Mean ± SD)	
Mobile Device	Simple Reaction	Impulse Control	Inspection Time	
iPad	75.38 ± 0.00	59.58 ± 0.00	92.50 ± 0.00	
iPhone 6S	73.91 ± 7.56	61.85 ± 7.15	81.73 ± 18.27	
Google Pixel 3	71.45 ± 8.15	54.34 ± 0.08	97.50 ± 3.54	
Samsung Galaxy S9	74.73 ± 7.61	62.76 ± 6.72	86.09 ± 13.24	

64077 Figure 1. Mean difference in SWAY CA Simple Reaction Motion, Impulse Control, and Inspection Time Measures between Mobile Devices

A one-way analysis of variance (ANOVA), as shown in Figure 1, determined SWAY CA Simple Reaction mean difference did not significantly differ between Mobile Devices, F (3, 84) = 0.182, p = 0.91, ηρ2 = 0.01. One-way ANOVA, as shown in Figure 1, determined SWAY CA Impulse Control mean difference did not significantly differ between mobile devices, F (3, 84) = 1.02, p = 0.39, ηρ2 = 0.04. In addition, the one-way ANOVA, as shown in Figure 1, determined SWAY CA Inspection Time mean difference did not significantly differ between mobile devices, F (3, 84) = 1.08, p = 0.36, ηρ2 = 0.04. Post-hoc tests were not administered based on the lack of significant differences found between mobile devices for each of the SWAY CA measures. As shown in Table 3, it is important to address that Google Pixel 3 (n = 2) and the iPad Pro 5 (n =1) each reported a very small group sample and will be further addressed in the discussion.

64078 Table 3. Total Number of SWAY CA and ImPACT QT Assessments by Mobile Device

	Assessment	
Device	SWAY CA	ImPACT QT	
iPhone	46		
Google Pixel	2		
Samsung	39		
iPad	1	88	
Total	88	88	

As shown in Table 4, a Pearson Product Moment Bivariate Correlation Coefficient (r) determined that SWAY CA simple reaction motion was negatively correlated, weak to moderate, across all seven ImPACT QT reaction time measures (r = -0.08 to -0.46), however three letter time first click was the only measure not found to be significant at p ≤ 0.05. In addition, the three letters counting correct mean score of the ImPACT QT visual motor speed module was found to have a weak positive correlate with the SWAY simple reaction motion (r = 0.22, p < 0.05). SWAY CA impulse control was found to have a significant negative correlation (p ≤ 0.05) of moderate strength with ImPACT QT attention tracker rectangular average time correct (r = -0.46), attention tracker figure eight average time correct (r = -0.36), and attention tracker complex average time correct (r = -0.31), respectively. The remaining ImPACT QT measures, however, were not found to correlate with SWAY CA Impulse Control (p > 0.05), as indicated in Table 4. SWAY CA inspection time was found to have a significant negative correlation of weak strength with ImPACT QT attention tracker figure eight average time correct (r = -0.29), and symbol match incorrect hidden average answer time (r = -0.25). However, SWAY CA inspection time was not found to correlate with the remaining ImPACT QT measures (p > 0.05) (Table 4).

64079 Table 4. Summary of Bivariate Correlations Between SWAY CA and ImPACT QT Measures

	SWAY CA	
ImPACT QT	Simple Reaction Motion	Impulse Control	Inspection Time	
Visual Motor Speed				
Three Letter Count Correct	0.22	0.17	0.06	
Reaction Time				
Three Letter Time First Click	-0.08	-0.01	-0.17	
Rectangular Average Time	-0.44	-0.46	-0.19	
Figure Eight Average Time	-0.46	-0.36	-0.29	
Complex Average Time	-0.32	-0.31	-0.20	
Symbol Match Correct Visible	-0.27	-0.19	0.04	
Symbol Match Correct Hidden	-0.35	-0.06	-0.05	
Symbol Match Incorrect Hidden	-0.32	-0.10	-0.25	
Bolded values represent p = 0.05

DISCUSSION

This study sought to determine the validity of the SWAY CA application, as well as its reliability across various hardware platforms and operating systems. The results indicated that SWAY CA application appears to be reliable in operating cognitive assessment measures (simple reaction motion, impulse control, inspection time) on various mobile devices (i.e., iPhone 6s Plus, Google Pixel 3, Samsung S9, and iPad Pro 5) and operating systems (e.g., iOS, Android). Such findings are important because this introduces the feasibility of assessing neurocognitive function and reaction time measures regardless of the mobile device available. Although the SWAY CA measures across all mobile devices were found to be in agreement; the iPad Pro 5 and Google Pixel 3 each offered a rather small contribution to the overall analysis. The iPad has been shown to be a valid and compatible mobile device for the SWAY application’s balance segment25,28,30,31 and did not present with any compatibility concerns when in use with the SWAY reaction time segment. The small sample size of the iPad Pro 5 (n = 1) was due to its lack of availability, as it was also being used to administer the ImPACT QT during experimental testing sessions. The Google Pixel 3, however, presented with a login issue that resulted in limited SWAY CA assessments (n = 2) and generally inconclusive findings of its compatibility. Overall, the SWAY CA application introduces the convenience of mobility and mobile device versatility, unlike the ImPACT QT application that requires the adoption of a universal mobile device.6,19,20,31 Furthermore, the lack of significant difference in SWAY CA measures between mobile devices minimizes concern of a difference in an individual’s SWAY CA measures (e.g., comparison of baseline data to data recorded immediately following an insult, and each follow-up assessment) being due to the use of different mobile devices.

The findings of this study further indicated that the SWAY CA segment of the SWAY System is a valid tool for assessing reaction time. Based on correlation values established between the measures using the SWAY system and the ImPACT QT, (0.32 to 0.63, p = 0.05)7,9,17; the Simple Reaction Motion of the SWAY CA application introduced reaction time measures (-0.27 to -0.46, p ≤ 0.05) comparable with reaction time measures of the ImPACT QT reaction time measures, except three letter time click first. The lack of agreement of the SWAY CA simple reaction motion measure with the ImPACT QT three letter time click first measure, as shown in Table 4, may be due to the difference in task(s) administered by each application to assess and calculate the measure as previously described in the methods section. Overall, these findings suggest that the SWAY CA is a comparable mobile neurocognitive and reaction time assessment tool to the FDA approved ImPACT QT.

In addition, several SWAY CA simple reaction motion, impulse control, and inspection time measures reported a negative correlation (-0.25 to -0.46; p ≤ 0.05) in relation to the ImPACT QT reaction time measures. Both SWAY CA and ImPACT QT measure rate of response based on lapse in time (milliseconds) from the moment a stimulus is introduced to the moment a response is recorded.21–23 The negative correlational values introduced in this study indicate that, on average, an individual’s rate of response (milliseconds) following a stimulus was significantly faster (smaller value) with the motion-based system used for SWAY CA in comparison to the slower (greater value) recorded when using the touch-based system for the ImPACT QT. Relatedly, these findings align with previous studies that reported motion-based systems (i.e., SWAY) to be extremely sensitive in recognizing movement as well as minimize mobile device latency down to one to two milliseconds,21,23,26,27 compared to a 50 to 200 millisecond delay when using a touch-based system (i.e., ImPACT QT).1,5,9,21–23 Of additional importance, hardware specifications between the devices used to administer the two applications differ (SWAY, ImPACT QT). The processors for each of the devices ran the respective operating systems at between 1.8 and 2.5 gigahertz.21,26,27,32,34 The screen on the iPhone 6S, Samsung 9s, and the Google Pixel 3, however, which were used to administer all but one of the SWAY application assessments, have a refresh rate of 60hz, compared to double the refresh rate of the iPad Pro 5 screen at 120hz used to administer the ImPACT QT application.21,26,27,32,34 Interestingly, although all SWAY assessments, except the one iPad Pro 5 measure, operated off a device with a slower refresh rate, the SWAY application was shown to recognize and capture a reaction time movement or cognitive response at a faster rate compared to the ImPACT QT based on the negative correlational findings. These findings further support the superior sensitivity of the motion-based SWAY application when seeking to record an individual’s reaction time measures and further assess one’s neurocognitive function and health. This is of particular importance for an individual in sport or other clinical setting where cognitive and reaction time measures may have critical and potentially life-threatening implications.3,25

While measures of agreement between the SWAY CA and ImPACT QT applications were established across several measures; further investigation is needed to determine the fair to low correlation amongst many of the SWAY impulse control and inspection time measures with the ImPACT reaction time and visual motor speed measures, as indicated in Table 4. One consideration for this absence of agreement may be due to distinct differences in measurement design for a particular assessment. Although both applications include assessment of reaction time measures; the SWAY application is a cognitive assessment tool that evaluates an individual’s cognitive and neuromotor measures,9,26 while the ImPACT QT application is known as a post-concussion cognitive test recognized as a neurocognitive and reaction time assessment tool.17,33 Therein, the impulse control and inspection time measures of the SWAY may differ beyond comparison with the ImPACT QT more so due to the measurement approach each uses. An additional consideration may be the notable difference in latency and electrical pulse cycle between the application operating systems.21,26,27,32,34 As shown in Table 4, the faster response rate of the motion-based system of SWAY compared to the slower touch-based system of the ImPACT QT may help explain the lack of associated strength amongst some of the measures and absence of agreement for others. Future test-retest reliability is warranted to further validate; however, the current findings support the use of a motion-based approach and the SWAY application to assess cognitive function and reaction time measures on a mobile device.

Limitations and Future Directions

This study is the first effort to establish concurrent validity of the cognitive assessment modules of the SWAY application as well as its capacity to operate across multiple mobile devices. Overall, the SWAY application was found to deliver reliable and valid cognitive and reaction time measures across all mobile devices; however, the iPad was only used to administer one SWAY assessment and the Google Pixel 3 did present with some concerns. The lack of data recorded from the iPad was due to lack of availability of the device because it was also being used to administer the ImPACT QT. The inclusion of the iPad in future studies is necessary to determine its compatibility with SWAY CA, as well as the potential impact of iPad latency of scores as previously discussed. It is unknown, however, whether the sporadic error message displayed during login and download when using the Google Pixel 3 was a compatibility issue or related to some other unknown. This unknown will require future exploration to determine.

Furthermore, the current findings should be generalized across all mobile device systems (hardware, software) with caution due to known capacity differences across systems21,26,27,32,34 as potentially indicated with the Google Pixel 3. In addition, as mobile device systems, including the devices in this study, frequently introduce updates to the hardware and software, further verification of SWAY compatibility is necessary. In addition, while the findings of this study supported the concurrent validity of the SWAY ’s ability to yield consistent cognitive and reaction time measures comparable to those of the FDA approved ImPACT QT; further test-retest reliability to determine within intrasession reliability and between intersession reliability is necessary.

Conclusion

In conclusion, the results of the current study indicate that the SWAY application is a reliable and valid method for measuring cognitive and reaction time measures across a variety of mobile devices. Furthermore, the faster capture rate technology used by the motion-based SWAY application appears to offer a potentially more reliable assessment of cognitive function and reaction time in comparison to the FDA approved touch-based ImPACT QT measures. Additionally, the SWAY application’s versatility in operating across various mobile device systems may further support its favorability of use in both health and sport.

Conflict of Interest

Authors have no reported conflicts of interest.
==== Refs
Factors influencing the latency of simple reaction time Frontiers in Human Neuroscience Woods David L. Wyma John M. Yund E. William Herron Timothy J. Reed Bruce Frontiers Media SA 26 3 2015
9 131 1662-5161 10.3389/fnhum.2015.00131 10.3389/fnhum.2015.00131 25859198
Sway: A novel approach for simple reaction time measurement American Journal of Occupational Therapy Burghart Mark Craig Jordan Radel Jeff Wu Andy Huisinga Jessie 1 8 2016
70 4 Suppl 1 7011500052p1 0272-9490 10.5014/ajot.2016.70s1-po5064 10.5014/ajot.2016.70s1-po5064
Sport-related concussion in children and adolescents Pediatrics Halstead Mark E. Walter Kevin D. Moffatt Kody 12 11 2018
142 6 e20183074 0031-4005 10.1542/peds.2018-3074 10.1542/peds.2018-3074
Visual reaction time and its relationship to neuropsychological test performance Archives of Clinical Neuropsychology Collins L. F. Long C. J. 1 1 1996
11 7 613 623 0887-6177 10.1093/arclin/11.7.613 10.1093/arclin/11.7.613
Pilot evaluation of a novel clinical test of reaction time in national collegiate athletic association division I football players J Athl Train Eckner James T. Kutcher Jeffrey S. Richardson James K. 1 7 2010
45 4 327 332 1062-6050 10.4085/1062-6050-45.4.327 10.4085/1062-6050-45.4.327 20617905
How do ImPACT Quick Test scores compare with ImPACT online scores in non-concussed adolescent athletes? Arch Clin Neuropsychol Elbin R J D’Amico Nathan R McCarthy Matthew Womble Melissa N O’Connor Sydne Schatz Philip 11 2 2020
35 3 326 331 1873-5843 10.1093/arclin/acz072 10.1093/arclin/acz072
Computerized neurocognitive testing in the management of sport-related concussion: an update Neuropsychology Review Resch Jacob E. McCrea Michael A. Cullum C. Munro 12 2013
23 4 335 349 1040-7308 10.1007/s11065-013-9242-5 10.1007/s11065-013-9242-5
Evaluating the recovery curve for clinically assessed reaction time after concussion J Athl Train del Rossi Gianluca 1 8 2017
52 8 766 770 1062-6050 10.4085/1062-6050-52.6.02 10.4085/1062-6050-52.6.02 28722468
Reliability and validity of a motion-based reaction time assessment using a mobile device Applied Neuropsychology: Adult Burghart Mark Craig Jordan Radel Jeff Huisinga Jessie 2019
26 6 558 563 2327-9095 10.1080/23279095.2018.1469491 10.1080/23279095.2018.1469491 30183372
Differences among three measures of reaction time based on hand laterality in individual sports Sports Badau Dana Baydil Bilgehan Badau Adela MDPI AG 19 5 2018
6 2 45 2075-4663 10.3390/sports6020045 10.3390/sports6020045 29910349
Effect of concussion on clinically measured reaction time in 9 NCAA division I collegiate athletes: a preliminary study PM Eckner James T. Kutcher Jeffrey S. Richardson James K. 3 2011
3 3 212 218 1934-1482 10.1016/j.pmrj.2010.12.003 10.1016/j.pmrj.2010.12.003 21402367
State of the art and future developments of measurement applications on smartphones Measurement Daponte P. De Vito L. Picariello F. Riccio M. Elsevier BV 11 2013
46 9 3291 3307 0263-2241 10.1016/j.measurement.2013.05.006 10.1016/j.measurement.2013.05.006
The use of mobile applications to collect data in sport, health and exercise science: a narrative review J Strength Cond Res Peart Daniel J. Balsalobre-Fernández Carlos Shaw Matthew P. 4 2019
33 4 1167 1177 1064-8011 10.1519/jsc.0000000000002344 10.1519/jsc.0000000000002344 29176384
Mobile devices and apps for health care professionals: uses and benefits P T Ventola C.L. 2014
39 5 356 364 24883008
Tracking the evolution of smartphone sensing for monitoring human movement Sensors del Rosario Michael B. Redmond Stephen J. Lovell Nigel H. 31 7 2015
15 8 18901 18933 1424-8220 10.3390/s150818901 10.3390/s150818901 26263998
Consumption analysis of smartphone based fall detection systems with multiple external wireless sensors Sensors González-Cañete Francisco Javier Casilari Eduardo MDPI AG 22 1 2020
20 3 622 1424-8220 10.3390/s20030622 10.3390/s20030622 31979189
ImPACT Quick Test. ImPACT Quick Test – Traditional 510(K), Computerized Cognitive Assessment Aid for Concussion, Section 5 – 510(k) K170551 Department of Health and Human Services – Food and Drug Administration 21 6 2017
2020-12-17 https://www.accessdata.fda.gov/cdrh_docs/pdf17/K170551.pdf
ImPACT Applications: Concussion Management & Training website ImPACT Applications, Inc. 2020
2020-9-20 https://impacttest.com/
Evaluating the prevalence of low factor scores on the ImPACT™ Quick Test in adolescents and adults using multivariate base rates Arch Clin Neuropsychol Wallace J Covassin T Schatz P Iverson G 7 2019
34 5 747 1873-5843 10.1093/arclin/acz026.17 10.1093/arclin/acz026.17
Brief iPad-based assessment of cognitive functioning with ImPACT Quick Test: prevalence of low scores using multivariate base rates Arch Clin Neuropsychol Wallace Jessica Schatz Philip Covassin Tracey Iverson Grant L 1 10 2020
35 8 1276 1282 1873-5843 10.1093/arclin/acaa078 10.1093/arclin/acaa078
iOS device compatibility reference website Apple Inc. 30 10 2017
2020-12-17 https://developer.apple.com/library/archive/documentation/DeviceInformation/Reference/iOSDeviceCompatibility/DeviceCompatibilityMatrix/DeviceCompatibilityMatrix.html
How much faster is fast enough? User perception of latency & latency improvements in direct and indirect touch. Understanding & Extending Touch Interfaces. Conference Proceedings of the 33rd Annual CHI Human Factors in Computing Systems. April 2015. Seoul, South Korea Deber J. Jota R. Forlines C.. 2020-12-8 10.1145/2702123
Could millisecond timing errors in commonly used equipment be a cause of replication failure in some neuroscience studies? Cognitive, Affective, & Behavioral Neuroscience Plant Richard R. Quinlan Philip T. Springer Science and Business Media LLC 3 5 2013
13 3 598 614 1530-7026 10.3758/s13415-013-0166-6 10.3758/s13415-013-0166-6
Immediate post-concussion assessment and cognitive testing (ImPACT) practices of sports medicine professionals J Athl Train Covassin Tracey Elbin Robert J., III Stiller-Ostrowski Jennifer L. Kontos Anthony P. 1 11 2009
44 6 639 644 1062-6050 10.4085/1062-6050-44.6.639 10.4085/1062-6050-44.6.639 19911091
Normative data for the SWAY balance system Clin J Sport Med Brett B.L. Zuckerman S.L. Terry D.P.. 2020
30 5 458 464 10.1097/jsm.0000000000000632 https://doi.org/doi%3A10.1097%2Fjsm.0000000000000632 30113969
SWAY mobile reaction time overview and scoring system of the SWAY reaction time beta website SWAY Medical, LLC 2020
2020-11-23 https://swaymedical.com/resources/videos
Getting raw accelerometer events website Apple Inc. 2020
2020-12-17 https://developer.apple.com/documentation/coremotion/getting_raw_accelerometer_events
Test-retest reliability of the SWAY balance mobile application Journal of Mobile Technology in Medicine Amick Ryan Z. Chaparro Alex Patterson Jeremy A.. 16 7 2015
4 2 40 47 1839-7808 10.7309/jmtm.4.2.6 10.7309/jmtm.4.2.6
2020 PAR-Q+ the physical activity readiness questionnaire for everyone Warburton D.E. Gledhill N. Jamnik V.. 3 2 2019
2020-12-17 http://eparmedx.com/wp-content/uploads/2013/03/January2020PARQPlusFillable.pdf
Comparison of a mobile technology application with the balance error scoring system Int J Athl Ther Train Patterson Jeremy A. Amick Ryan Z. Pandya Priyanka D. Hakansson Nils Jorgensen Michael J. 5 2014
19 3 4 7 2157-7277 10.1123/ijatt.2013-0094 10.1123/ijatt.2013-0094
Validation of measures from the smartphone SWAY balance application:a pilot study Int J Sports Phys Ther Patterson J.A. Amick R.Z. Thummar T.. 2014
9 2 135 139 24790774
Motion sensors website Android Developers 28 10 2020
2020-12-17 https://developer.android.com/guide/topics/sensors/sensors_motion
The “value added” of neurocognitive testing after sports-related concussion The American Journal of Sports Medicine Van Kampen Derk A. Lovell Mark R. Pardini Jamie E. Collins Michael W. Fu Freddie H. SAGE Publications 10 2006
34 10 1630 1635 0363-5465 10.1177/0363546506288677 10.1177/0363546506288677
The next battle for smartphone screen supremacy is about speed, not resolution Popular Science Horaczek S. 15 5 2019
2020-12-17 https://www.popsci.com/oneplus-7-pro-smartphone-screen-refresh-rate
