
==== Front
PLoS One
PLoS One
plos
PLOS ONE
1932-6203
Public Library of Science San Francisco, CA USA

10.1371/journal.pone.0308853
PONE-D-23-37915
Research Article
People and Places
Population Groupings
Age Groups
Children
People and Places
Population Groupings
Families
Children
Medicine and Health Sciences
Neurology
Cerebral Palsy
Biology and Life Sciences
Physiology
Biological Locomotion
Walking
Engineering and Technology
Electronics Engineering
Electronics
Accelerometers
Biology and Life Sciences
Physiology
Biological Locomotion
Jumping
Medicine and Health Sciences
Public and Occupational Health
Physical Activity
Computer and Information Sciences
Artificial Intelligence
Machine Learning
Medicine and Health Sciences
Medical Conditions
Disabilities
Validation of two novel human activity recognition models for typically developing children and children with Cerebral Palsy
Validation of HAR models for children and children with Cerebral Palsy
https://orcid.org/0000-0002-7029-226X
Tørring Marte Fossflaten Data curation Formal analysis Methodology Software Visualization Writing – original draft Writing – review & editing 1 2 *
Logacjov Aleksej Data curation Methodology Software Writing – review & editing 3
Brændvik Siri Merete Conceptualization Funding acquisition Project administration Supervision Writing – review & editing 1 4
https://orcid.org/0000-0001-7516-3259
Ustad Astrid Data curation Methodology Writing – review & editing 1
Roeleveld Karin Conceptualization Data curation Formal analysis Funding acquisition Project administration Supervision Writing – review & editing 1
https://orcid.org/0000-0003-4827-453X
Bardal Ellen Marie Conceptualization Data curation Funding acquisition Methodology Project administration Writing – review & editing 1 4
1 Department of Neuromedicine and Movement Science, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, NTNU, Trondheim, Norway
2 Physiotherapy Unit, Trondheim Municipal, Trondheim, Norway
3 Department of Computer Science, Faculty of Information Technology and Electrical Engineering, Norwegian University of Science and Technology, NTNU, Trondheim, Norway
4 Clinic of Rehabilitation, St Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
Narayan Jyotindra Editor
Imperial College London, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
Competing Interests: The authors have declared that no competing interests exist.

* E-mail: marte.f.torring@ntnu.no
23 9 2024
2024
19 9 e030885329 11 2023
1 8 2024
© 2024 Tørring et al
2024
Tørring et al
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Human Activity Recognition models have potential to contribute to valuable and detailed knowledge of habitual physical activity for typically developing children and children with Cerebral Palsy. The main objective of the present study was to develop and validate two Human Activity Recognition models. One trained on data from typically developing children (n = 63), the second also including data from children with Cerebral Palsy (n = 16), engaging in standardised activities and free play. Our data was collected using accelerometers and ground truth was established with video annotations. Additionally, we aimed to investigate the influence of window settings on model performance. Utilizing the Extreme gradient boost (XGBoost) classifier, twelve sub-models were created, with 1-,3- and 5-seconds windows, with and without overlap. Both Human Activity Recognition models demonstrated excellent predictive capabilities (>92%) for standardised activities for both typically developing and Cerebral Palsy. From all window sizes, the 1-second window performed best for all test groups. Accuracy was slightly lower (>75%) for the Cerebral Palsy test group performing free play activities. The impact of window size and overlap varied depending on activity. In summary both Human Activity Recognition models effectively predict standardised activities, surpassing prior models for typically developing and children with Cerebral Palsy. Notably, the model trained on combined typically developing children and Cerebral Palsy data performed exemplary across all test groups. Researchers should select window settings aligned with their specific research objectives.

http://dx.doi.org/10.13039/100019902 Fakultet for medisin og helsevitenskap, Norges Teknisk-Naturvitenskapelige Universitet https://orcid.org/0000-0002-7029-226X
Tørring Marte Fossflaten http://dx.doi.org/10.13039/100019902 Fakultet for medisin og helsevitenskap, Norges Teknisk-Naturvitenskapelige Universitet https://orcid.org/0000-0003-4827-453X
Bardal Ellen Marie http://dx.doi.org/10.13039/501100021697 Sykehuset i Vestfold We have received financial support in form of a post-doctorate and doctorate position at the Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology (Norwegian name: Fakultetet for medisin og helsevitenskap, Norges tekniske-Naturvitenskapelige Universitet) and from Vestfold Hospital Trust (Norwegian name: Sykehuset i Vestfold). Data AvailabilityAll metadata and raw acceleration files with annotations are available from Dataverse (https://doi.org/10.18710/EPCXCC). To replicate our experiment one would also need our code, which is published on GitHub (https://github.com/ntnu-ai-lab/harth-ml-experiments/tree/main).
Data Availability

All metadata and raw acceleration files with annotations are available from Dataverse (https://doi.org/10.18710/EPCXCC). To replicate our experiment one would also need our code, which is published on GitHub (https://github.com/ntnu-ai-lab/harth-ml-experiments/tree/main).
==== Body
pmc1. Introduction

Accurate measures of physical activity (PA) are crucial for evidence-based evaluation and regulation of PA in children and adolescents, as well as for assessing the effectiveness of PA interventions. Acceleration-based sensors have gained popularity in recent years as a supplement, and a more reliable alternative to self-reported data in both intervention and population studies [1, 2]. Accelerometers provide acceleration in 3D, which can be used to estimate frequency, intensity, mode and duration of activity and sedentary behaviour, offering valuable insights into natural behaviour in children [3–6].

Traditionally, accelerometer data have been analysed using brand specific software with a count-based approach providing information about the intensity of the performed activities [7, 8]. Even though several population- and age-specific cut-off values for the different intensities have been developed, the validity is questionable both in typically developing (TD) children and in children with disabilities [9, 10]. Cut-off values for children with cerebral palsy (CP) has for example shown to misclassify 30–40% of the performed intensities [11]. An alternative approach is to classify unique behavioural patterns by using Human Activity Recognition (HAR) models, and thereby identify and recognise specific activity types, such as sitting, jumping, or standing [7, 12]. HAR models have shown promising results in recognising activities with over 90% accuracy in free living conditions for various population groups [12–16]. In studies involving children the accuracy is inconsistent and varies between 62–95% [8, 15–19]. This inconsistency is probably depending on factors like window size, activity protocol and algorithm used.

Despite the advantages with HAR models, challenges include the need for specific training data, large data sets, definition of optimal accelerometer placements, window size considerations and feature extraction [13, 20–22]. The HAR models also identifies unique movement patterns which may differ between adults and children. The accuracy of existing models trained on healthy adults may therefore be questioned for accurate recognition of PA in children, especially those with disabilities and deviating movement patterns [23, 24]. CP is one of the most frequent disability among children [25] with the global prevalence of approximately 1.6 /1000 live births in high income countries [26], and thereby it is suitable to include this group for our endeavour. Cerebral Palsy (CP) is a collective term for various neurological impairments resulting from cerebral injuries before the age of two [27–29]. The CP condition may include a wide range of physical impairments like reduced walking speed and stride length [30], impaired balance, [31] and secondary musculoskeletal impairments like contractures and skeletal deformities [32], all which may give deviating movement patterns compared to TD children.

The previous studies validating HAR models for TD children are hard to compare, due to use of different classifiers, window sizes, accelerometer placements and activity protocols [16, 17]. Moreover, accuracies often drop significantly when transitioning from laboratory-based to free-living conditions [15, 19]. Only four publications have validated HAR models in children with CP, and they present the same issues regarding the model specifications [24, 33–35]. All these studies achieved good accuracy for sedentary behaviour (over 80%), but variable results for free-living activities, with some as low as 27% accuracy [34]. The protocols used in these studies focus on sedentary behaviour and walking paces and do not adequately represent free-living childlike activities, lacking elements like ballgames, outdoor activities, and free play [24, 33–35]. Therefore, a new HAR model validated on free living data is needed for TD children and one spesificaly for children with CP.

Various human activities unfold over different durations, which the HAR models need to detect. For instance, a single step takes about 500 milliseconds to complete [36]. However, walking as an activity involves multiple steps, and thereby extending the time frame beyond a mere 1-second interval. Consequently, the choice of window size is a contentious issue, impacting accuracy for both cut-off, deep-learning and HAR methods, particulary in children, who change activity frequently [22, 23, 37]. Some studies suggest increased accuracy with larger window sizes (e.g., 15 seconds), while others argue for shorter windows (1–2 seconds) [15, 18]. Addtionally the use of overlapping windows, which provides more data points, is in some cases favoured for better classification [38, 39].

To meet the challenges described with HAR, the present study aimed to develop and validate two machine learning models for recognizing habitual activities in TD children and ambulatory children with CP. Specifically, the two models differentiate with one model trained with only TD data and the second model with both TD and CP data. Additionally addressing the lack of childlike free play activities in existing models and if deviating movement patterns in children is of significance for model performance. Furthermore, this study investigated the impact of window size and overlapping versus non-overlapping windows on prediction accuracy, and consider practical considerations related to time and storage use.

2. Materials and methods

2.1 Participants

Data from 63 TD children and 16 children with CP are included in this validation study (Table 1). The TD children were recruited through a local primary and junior high school, and through colleagues and friends. The children with CP were recruited during a habilitation stay or through outpatient clinic at the local hospital. Only children classified with Gross Motor Function Classification System (GMFCS) I and II were included [40]. All participants were given written age-appropriate information about the study. This was distributed at school and taken home together with written information and consent forms to their parents/guardians. If the parents gave their written consent, verbal information was given to the children before the study started. The data collection started 19.10.2016 and ended 16.08.2019. The study was approved by the Norwegian Centre for Research Data (NSD-nr:50683). In addition, the study was reported to the Regional Ethical Committee for Medical and Health science (REK-nr:2016/707/REK nord) but was not classified under the act on medical and health research. The medical background information that was collected was age, gender, and presence of CP and GMFCS level.

10.1371/journal.pone.0308853.t001 Table 1 Participant characteristics.

Subjects	TD	CP	
N (B/G)	63 (35/28)	16 (8/8)	
Age (years)	10.5 (+/-2.6) [6–15]	11.4 (+/- 2.2) [8–17]	
Height (cm)	149.1 (+/-15.7) [117–170]	146.7 (+/-10.8) [129–173]	
Weight (kg)	42.7 (+/13.0) [21–76]	43.2 (+/-11.0) [25–62]	
GMFCS			
I		10	
II		6	
1 Mean with (+/- SD) and [range]. TD = Typically developing, CP = Cerebral Palsy, N = Number of subjects, B = Boys, G = Girls, cm = Centimetre, kg = Kilogram, GMFCS = Gross motor function classification scale.

2.2 Validation protocols and test groups

The children conducted standardised semi-structured activities including different modes of running, walking, standing, sitting, and lying down, with varying durations. In addition, they performed a free-living protocol including ball games and free play, conducted both indoors and outdoors. For data-synchronisation purposes, the children also performed heel-drops, or the researcher flicked the accelerometer three times. When testing the HAR models, we divided the participants into three groups, based on activity protocol and if they were diagnosed with CP. The TD group included the typically developing children who completed both standardised activities and five minutes of free play. One CP group, here after called CP Stan, are children with CP who completed the same standardised activities as the TD group, except for the five minutes of free play. The second CP group, here after called CP Free, included children with CP who only engaged in group free play activities. For both CP and TD, all activities were performed in a single session, and all TD participants conducted the whole protocol. See Supporting information (S1 Table) for full list of activities and number of children conducting the different activities.

2.3 Instrumentation

2.3.1 Activity monitors

All participants wore two Axivity AX3 accelerometers (Axivity Ltd, Newcastle, UK), one on the thigh placed along the anterior midline, in the middle between anterior superior iliac spine and proximal patella, and one at the approximate placement of L3. For the CP group the thigh accelerometers were placed on the least affected side. Acceleration was sampled at 100 Hz and 200 Hz for TD and 100 Hz for CP (range ± 8g).

2.3.2 Video recordings

Video recording using GoPro Hero 3+ cameras were used to identify the performed activities. The cameras were mounted in corners of the room during inside protocols, play and group activities. For activities with longer duration and/outside activities the GoPro camera was attached with a chest harness, pointing downwards to detect leg movement, or handheld by researcher. The recordings were sampled at 60 frames per second, resolution of 1080x720 pixels.

2.4 Data processing

2.4.1 Video annotation

The video recordings were used as the ground truth for activity types. The activities in the videos were manually labelled (annotated) frame by frame for each participant using Anvil video annotation tool (version 6) [41]. Thirteen activities were labelled using activity definitions used in previous validation studies with the NTNU-HAR models [7, 13, 14]. After clarifying the activity definitions and discussions based on video examples four raters annotated each video independently. Inter-rater reliability of > 0.95 on this methodology has been reported in earlier studies using the same methods, activity definitions and overlapping raters with the present study [7, 14]. Definitions of activities are listed in S2 Table.

2.4.2 Data pre-processing and feature extraction

Before training our machine learning model, we performed three pre-processing steps (Fig 1). Initially we down sampled and synchronized thigh and back accelerometer signals with activity annotations ensuring data alignment at the recommended 50 Hz [12]. Subsequently, we segmented the signals into signal frames of our selected window sizes (1 sec, 3 sec, and 5 sec), with and without 50% overlap. The overlap was included to investigate potential loss of activities at the endpoint of the window. Majority voting was applied to the annotations, such that each 1, 3, or 5 second signal frames corresponded to exactly one activity, based on the most frequently occurring in the set window. Lastly, we computed 161 time- and frequency-domain features for each signal frame, using the movement and gravitational components of all six sensor axes, and each sensor’s vector magnitude, as described in Logacjov et al., [13]. These resulting features and annotations were used to train the machine learning models.

10.1371/journal.pone.0308853.g001 Fig 1 The pre-processing and feature extraction.

2.4.3 Machine learning approach

We used the Extreme gradient boost (XGBoost) classifier [42] as our machine learning method.

The XGBoost is an ensemble learning approach based on the gradient boosting algorithm [43], where multiple weak classifiers (e.g., decision trees) are trained in a sequential manner. Each weak classifier was trained to minimize the errors made by the previous weak classifier. Our final model prediction is the weighted sum of all weak classifiers’ predictions.

2.4.4 Model training and cross validation (evaluation)

The XGBoost methodology resulted in two HAR models, NTNU-HAR-Children (HAR-Children) and NTNU-HAR-ChildrenCP (HAR-ChildrenCP). The HAR-Children were trained with data from the TD group and the HAR-ChildrenCP was trained with all TD training data and additional data from children with CP. The two main models comprise a total of 12 sub-models, 6 for each main model, covering 1-, 3-, and 5-seconds windows with and without overlapping windows, see Supporting information (S1 Fig) for overview of the 12 sub-model creation. For each model we initially performed a leave-six-subjects-out-cross-validation with hyperparameter optimization in the form of a grid search. This allowed us to find optimal hyperparameters for each of the 12 sub-models, leading to a fairer comparison. After finding optimal hyperparameters for each model, we performed 12 leave-one-subject-out cross-validations (LOSOCV), one for each model. In the LOSOCV, the model was trained on all participants, except for one, and this participant became the test-data. The overall performance of the model was estimated by repeating this process for each individual in the dataset and then averaging the performance across each individual [13, 16]. This gave us less subject-dependent estimates and thereby less subject-based bias [13, 16, 44]. Note that the six HAR-Children models were trained without the CP data. Hence, the performed LOSOCVs only provided test results for the TD group in this model. To get the results for the CP groups from the HAR-Children models we trained the model on the whole TD dataset, and then used the CP data as test data. Additionally, before we compared the results of the different window sizes (1 sec, 3 sec, 5 sec), we unfolded the model predictions to the original 50 samples per second, to make all our models comparable. The complete dataset and model file is available at Dataverse (https://doi.org/10.18710/EPCXCC) and GitHub (https://github.com/ntnu-ai-lab/harth-ml-experiments), and is named NTNU-Children.

2.5 Data post processing

Initially we annotated with all activity definitions provided in supporting information (S2 Table), we choose to do this to provide precision and to avoid confusing the model when training it on with similar movements. For the further processing, we collapsed some activity labels to make it applicable for practical use. We have defined shuffling as standing with small foot movements (S2 Table), and for our current focus the differentiation between standing still and standing with some foot movement is of limited significance. Therefore, shuffling was imbedded into standing. Similarly, bending is an activity that typically occur when standing and is therefore collapsed with standing. To avoid confusing our model’s ability to recognize level walking, walking up and down stairs was collapsed with walking. The original two categories of cycling, sit cycling and stand cycling were collapsed into cycling, as our primary interest was in recognize the cyclic leg movements. These collapsed activity classes are the same as used in previous NTNU-HAR models [7, 13, 14]. The overall preliminary results are provided in S3 Table., in their originally annotated form.

2.6 Statistical analysis

We assessed the HAR models by calculating the overall accuracy for each test group and determining precision, sensitivity, specificity, and F1 Score for each activity type. Sensitivity, also called recall, measured our model’s ability to correctly classify activities when they occurred, while specificity evaluated the ability to avoid false recognition when activities were absent. Precision indicated the ratio of correctly classified activities to the sum of correctly and falsely classified activities. The F1 Score, a harmonic mean between precision and sensitivity, provided a class-wise precision and sensitivity measure. Accuracy was calculated as the ratio of correctly recognized activity samples to the total number of activity samples. These metrics range from 0–1, with higher values indicating superior performance. The confusion matrixes include the same collapsed activity classes. If the subject did not conduct the activity, they were taken out of the average calculations. We performed all these calculations for each subject before calculating group mean and confidence intervals. All calculations were executed in MATLAB.

3. Results

Fig 2 shows the distribution of the performed activities in the three test groups. There was more data from TD children (total 2997.9 minutes) than CP, and more with CP standardised (total 647.4 minutes) than CP Free (total 180 minutes). In all three test groups there were more time spent with walking, standing, sitting, and running than the other activity labels (Fig 2).

10.1371/journal.pone.0308853.g002 Fig 2 Bar plot of total amount of activity in the three test groups.

TD = Typically developing children, CP Stan = Children with CP, standardised activities, CP Free = Children with CP, free play.

3.1 Overall performance of the twelve HAR-models

For all models the overall accuracy was high, with the same median value (0.93) (Table 2), and the accuracy slightly favoured (range: 0.12–0.18) the TD and CP Stan groups compared to CP Free. The 1-second model in both HAR-Children and HAR-ChildrenCP performed exemplary for the respective test groups (Table 2). The largest difference in accuracy was between the corresponding HAR-Children and HAR-ChildrenCP models for the test group CP Free (range: 0.2–0.4, see Table 2). As illustrated in Table 2 the difference between overlap and non-overlap is only present in HAR-Children 1- and 3-seconds, and from this point on we present the non-overlapping models. Non-overlapping models are preferred for their efficiency in processing time and storage, making them more practical for later use. Accuracy for the original activity classes is presented in S3 Table. All experiment results can be found at GITHUB (https://github.com/ntnu-ai-lab/harth-ml-experiments).

10.1371/journal.pone.0308853.t002 Table 2 Overall accuracy for each model with and without overlaps.

NTNU-HAR-Children	1 sec	1 sec overlap	3 sec	3 sec overlap	5 sec	5 sec Overlap	
Test group							
TD	0.94	0.95	0.93	0.93	0.92	0.92	
CP Stan	0.94	0.94	0.94	0.94	0.93	0.93	
CP Free	0.78	0.79	0.77	0.78	0.75	0.75	
NTNU-HAR ChildrenCP	1 sec	1 sec overlap	3 sec	3 sec overlap	5 sec	5 sec overlap	
TD	0.94	0.94	0.94	0.94	0.92	0.92	
CP Stan	0.95	0.95	0.94	0.94	0.93	0.93	
CP Free	0.82	0.82	0.81	0.80	0.78	0.77	
Test groups: TD = Typically developing children, CP Stan = Cerebral palsy with standardised activities, CP Free = Cerebral palsy with Free play. The range of accuracy is 0–1, where higher scores are better.

3.2 Specific activity performance of the six selected HAR-models

In both 1-second models (Table 3a and 3b) all activities were predicted with high accuracy for the TD group, with F1 Score over 0.85. In CP Stan only running (range: 0.79–0.80) and cycling (range: 0.75–0.78) had a F1 Score under 0.90. The CP Free group had slightly lower accuracy and a wider range in both 1-second models (range: 0.29–0.94). This group had superior values in favour of the HAR-Children CP model, with an average difference of 0.09 (Table 3b), where walking, running, and jumping had the largest difference (0.09, 0.07, 0.27).

10.1371/journal.pone.0308853.t003 Table 3 Sensitivity, specificity, precision, and F1 Score for each activity.

a)	NTNU HAR-Children- One second	
Test group	Activity type						
TD		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	62	0.94 [0.92, 0.95]	0.98 [0.97, 0.98]	0.95 [0.94, 0.96]	0.94 [0.93, 0.96]	
	Running	62	0.96 [0.94, 0.97]	0.99 [0.99, 0.99]	0.91 [0.87, 0.94]	0.92 [0.90, 0.95]	
	Standing	62	0.93 [0.89, 0.96]	0.97 [0.97, 0.98]	0.93 [0.91, 0.95]	0.92 [0.89, 0.95]	
	Sitting	62	0.98 [0.97, 0.99]	0.99 [0.97, 1.00]	0.97 [0.94, 0.99]	0.97 [0.95, 0.99]	
	Lying	29	0.99 [0.98, 1.00]	1.00 [0.99, 1.00]	0.98 [0.94, 1.01]	0.98 [0.95, 1.00]	
	Cycling	21	0.87 [0.78, 0.97]	1.00 [1.00, 1.00]	0.93 [0.87, 0.98]	0.89 [0.81, 0.97]	
	Jumping	17	0.87 [0.77, 0.96]	1,00 [1.00, 1.00]	0.92 [0.86, 0.98]	0.87 [0.79, 0.95]	
CP Stan		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	10	0.89 [0.86, 0.93]	0.97 [0.96, 0.98]	0.92 [0.90, 0.94]	0.90 [0.88, 0.93]	
	Running	10	0.77 [0.60, 0.94]	1.00 [0.99, 1.00]	0.88 [0.83, 0.93]	0.80 [0.66, 0.94]	
	Standing	10	0.95 [0.93, 0.97]	0.95 [0.93, 0.96]	0.92 [0.89, 0.94]	0.93 [0.91, 0.95]	
	Sitting	10	0.99 [0.99, 1.00]	1.00 [0.99, 1.00]	0.99 [0.98, 1.00]	0.99 [0.99, 0.99]	
	Lying	10	0.99 [0.98, 1.00]	1.00 [1.00, 1.00]	0.99 [0.98, 1.00]	0.99 [0.98, 1.00]	
	Cycling	2	0.62 [-1.37, 2.61]	1.00 [1.00, 1.00]	0.99 [0.92, 1.07]	0.75 [-0.80, 2.30]	
	Jumping	7	0.91 [0.85, 0.98]	1.00 [1.00, 1.00]	0.95 [0.91, 0.99]	0.93 [0.88, 0.98]	
CP Free		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	6	0.49 [0.34, 0.64]	0.92 [0.90, 0.95]	0.65 [0.59, 0.70]	0.55 [0.45, 0.65]	
	Running	5	0.60 [0.46, 0.73]	0.99 [0.98, 1.00]	0.52 [0.20, 0.84]	0.53 [0.29, 0.77]	
	Standing	6	0.88 [0.84, 0.92]	0.70 [0.59, 0.82]	0.80 [0.75, 0.84]	0.84 [0.81, 0.86]	
	Sitting	6	0.87 [0.75, 0.99]	0.99 [0.98, 1.00]	0.93 [0.86, 1.00]	0.90 [0.82, 0.97]	
	Lying	6	0.89 [0.77, 1.01]	0.99 [0.95, 1.02]	0.71 [0.29, 1.14]	0.74 [0.38, 1.11]	
	Jumping	2	0.19 [-0.03, 0.40]	1.00 [1.00, 1.00]	0.72 [-2.91, 4.34]	0.29 [-0.31, 0.90]	
b)	NTNU HAR-ChildrenCP- One second						
Test group	Activity type						
TD		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	62	0.94 [0.92, 0.96]	0.97 [0.97, 0.98]	0.95 [0.94, 0.96]	0.94 [0.93, 0.96]	
	Running	62	0.96 [0.95, 0.97]	0.99 [0.99, 0.99]	0.90 [0.87, 0.94]
0.94 [0.92, 0.95]	0.92 [0.90, 0.95]	
	Standing	61	0.94 [0.93, 0.95]	0.98 [0.97, 0.98]	0.94 [0.92, 0.95]
0.97 [0.94, 0.99]	0.94 [0.93, 0.95]	
	Sitting	62	0.98 [0.97, 0.99]
0.99 [0.99, 1.00]	0.99 [0.96, 1.01]	0.97 [0.94, 0.99]	0.97 [0.95, 0.99]	
	Lying	29	0.99 [0.99, 1.00]	1.00 [1.00, 1.00]	0.98 [0.95, 1.01]	0.98 [0.97, 1.00]	
	Cycling	21	0.90 [0.82, 0.97]	1.00 [1.00, 1.00]	0.95 [0.92, 0.98]	0.92 [0.86, 0.98]	
	Jumping	17	0.83 [0.72, 0.93]	1.00 [1.00, 1.00]	0.91 [0.84, 0.98]	0.86 [0.77, 0.94]	
CP Stan		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	10	0.91 [0.88, 0.94]	0.97 [0.96, 0.98]	0.91 [0.89, 0.94]
0.87 [0.80, 0.94]	0.91 [0.89, 0.93]	
	Running	10	0.76 [0.57, 0.95]	1.00 [0.99, 1.00]	0.87 [0.80, 0.94]	0.79 [0.63, 0.95]	
	Standing	10	0.95 [0.93, 0.97]	0.96 [0.94, 0.97]
1.00 [1.00, 1.00]	0.93 [0.91, 0.96]	0.94 [0.92, 0.96]	
	Sitting	10	0.99 [0.99, 1.00]	1.00 [1.00, 1.00]	0.99 [0.98, 1.00]	0.99 [0.99, 1.00]	
	Lying	10	0.99 [0.98, 1.00]	1.00 [1.00, 1.00]	0.99 [0.98, 1.00]	0.99 [0.98, 1.00]
0.78 [-0.88, 2.44]	
	Cycling	2	0.66 [-1.59, 2.91]
0.93 [0.86, 1.00]	1.00 [1.00, 1.00]	1.00 [1.00, 1.00]	0.78 [-0.88, 2.44]	
	Jumping	7	0.93 [0.86, 1.00]	1.00 [1.00, 1.00]	0.95 [0.92, 0.98]	0.94 [0.89, 0.99]	
CP Free		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	6	0.61 [0.47, 0.75]	0.92 [0.89, 0.95]
0.99 [0.98, 1.00]	0.69 [0.62, 0.76]	0.64 [0.57, 0.71]	
	Running	6	0.56 [0.44, 0.67]	0.99 [0.98, 1.00]	0.45 [0.14, 0.76]	0.46 [0.22, 0.71]	
	Standing	6	0.89 [0.85, 0.93]	0.79 [0.68, 0.90]	0.85 [0.81, 0.90]	0.87 [0.86, 0.88]	
	Sitting	6	0.92 [0.80, 1.04]	1.00 [0.99, 1.00]	0.96 [0.93, 1.00]	0.94 [0.87, 1.00]	
	Lying	6	0.88 [0.78, 0.98]	0.99 [0.95, 1.02]	0.76 [0.38, 1.14]	0.76 [0.45, 1.07]	
	Jumping	3	0.43 [-0.14, 1.01]	1.00 [1.00, 1.00]	0.84 [0.15, 1.53]	0.56 [-0.04, 1.16]	
c)	NTNU HAR-Children- Three seconds						
Test group	Activity type						
TD		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	62	0.93 [0.92, 0.95]
0.94 [0.93, 0.96]	0.96 [0.96, 0.97]	0.94 [0.92, 0.95]	0.93 [0.92, 0.95]	
	Running	62	0.94 [0.93, 0.96]	0.99 [0.98, 0.99]	0.89 [0.85, 0.92]	0.91 [0.88, 0.93]	
	Standing	62	0.90 [0.87, 0.93]	0.97 [0.97, 0.98]	0.93 [0.92, 0.95]	0.91 [0.87, 0.94]	
	Sitting	62	0.97 [0.96, 0.98]	0.98 [0.97, 1.00]	0.96 [0.94, 0.98]	0.96 [0.94, 0.98]	
	Lying	29	0.98 [0.97, 0.99]	0.99 [0.98, 1.00]	0.96 [0.90, 1.01]
0.93 [0.87, 0.99]	0.96 [0.91, 1.00]	
	Cycling	21	0.88 [0.79, 0.97]	1.00 [1.00, 1.00]	0.93 [0.87, 0.99]	0.90 [0.82, 0.98]	
	Jumping	15	0.92 [0.88, 0.96]	1.00 [1.00, 1.00]	0.91 [0.83, 1.00]	0.90 [0.85, 0.96]	
CP Stan		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	10	0.90 [0.87, 0.93]	0.97 [0.95, 0.98]	0.90 [0.87, 0.93]	0.90 [0.87, 0.93]	
	Running	9	0.78 [0.69, 0.88]	1.00 [0.99, 1.00]	0.90 [0.85, 0.96]	0.83 [0.77, 0.89]	
	Standing	10	0.94 [0.91, 0.97]	0.95 [0.94, 0.96]	0.92 [0.89, 0.95]	0.93 [0.90, 0.96]	
	Sitting	10	0.99 [0.99, 1.00]	0.99 [0.99, 1.00]	0.98 [0.96, 0.99]	0.98 [0.98, 0.99]	
	Lying	10	0.99 [0.98, 1.00]	1.00 [1.00, 1.00]	0.99 [0.98, 1.00]	0.99 [0.98, 1.00]	
	Cycling	2	0.64 [0.11, 1.17]	1.00 [1.00, 1.00]	0.99 [0.92, 1.07]	0.78 [0.41, 1.14]	
	Jumping	7	0.84 [0.71, 0.97]	1.00 [1.00, 1.00]	0.98 [0.95, 1.00]	0.90 [0.81, 0.98]	
CP Free		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	6	0.58 [0.40, 0.75]	0.88 [0.84, 0.92]	0.59 [0.54, 0.63]	0.57 [0.48, 0.66]	
	Running	5	0.49 [0.19, 0.79]	0.98 [0.96, 1.00]	0.43 [0.11, 0.75]	0.41 [0.15, 0.67]	
	Standing	6	0.83 [0.78, 0.89]	0.75 [0.62, 0.88]	0.82 [0.77, 0.87]	0.82 [0.79, 0.86]	
	Sitting	6	0.86 [0.75, 0.97]	0.99 [0.98, 1.00]	0.91 [0.82, 1.01]	0.88 [0.81, 0.95]	
	Lying	6	0.87 [0.75, 1.00]	0.99 [0.95, 1.02]	0.71 [0.31, 1.11]	0.74 [0.39, 1.09]	
	Jumping	0	0.00 [0.00, 0.00]	1.00 [1.00, 1.00]	0.00 [0.00, 0.00]	0.00 [0.00, 0.00]	
d)	NTNU HAR-ChildrenCP- Three seconds						
Test group	Activity type						
TD		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	62	0.94 [0.92, 0.95]
0.95 [0.94, 0.96]	0.97 [0.96, 0.97]	0.94 [0.93, 0.95]	0.94 [0.93, 0.95]	
	Running	62	0.95 [0.94, 0.96]	0.99 [0.98, 0.99]	0.89 [0.86, 0.93]	0.91 [0.89, 0.94]	
	Standing	62	0.90 [0.87, 0.93]	0.97 [0.97, 0.98]	0.93 [0.92, 0.95]	0.91 [0.87, 0.94]	
	Sitting	62	0.97 [0.96, 0.98]	0.98 [0.97, 1.00]	0.96 [0.93, 0.98]	0.96 [0.94, 0.98]	
	Lying	29	0.98 [0.97, 0.99]	1.00 [1.00, 1.00]	0.97 [0.95, 1.00]	0.97 [0.96, 0.99]	
	Cycling	20	0.93 [0.88, 0.97]	1.00 [1.00, 1.00]	0.94 [0.88, 1.00]	0.93 [0.88, 0.98]	
	Jumping	14	0.92 [0.88, 0.95]	1.00 [1.00, 1.00]	0.95 [0.93, 0.97]	0.93 [0.91, 0.95]	
CP Stan		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	10	0.91 [0.89, 0.94]	0.96 [0.95, 0.98]	0.90 [0.86, 0.93]	0.90 [0.88, 0.93]	
	Running	9	0.79 [0.67, 0.90]	1.00 [0.99, 1.00]	0.89 [0.82, 0.95]	0.82 [0.76, 0.89]	
	Standing	10	0.93 [0.90, 0.96]	0.95 [0.94, 0.97]	0.93 [0.90, 0.96]	0.93 [0.90, 0.96]	
	Sitting	10	0.99 [0.99, 0.99]	1.00 [0.99, 1.00]	0.98 [0.97, 0.99]	0.99 [0.98, 0.99]	
	Lying	10	0.99 [0.98, 1.00]	1.00 [1.00, 1.00]	0.99 [0.98, 1.00]
1.00 [1.00, 1.00]	0.99 [0.98, 1.00]	
	Cycling	2	0.66 [0.56, 0.75]	1.00 [1.00, 1.00]	1.00 [1.00, 1.00]	0.79 [0.72, 0.86]	
	Jumping	7	0.86 [0.76, 0.96]	1.00 [1.00, 1.00]	0.96 [0.92, 1.01]	0.91 [0.83, 0.98]	
CP Free		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	6	0.64 [0.49, 0.79]	0.90 [0.87, 0.93]	0.65 [0.58, 0.71]	0.63 [0.57, 0.69]	
	Running	5	0.54 [0.33, 0.74]	0.99 [0.97, 1.00]	0.48 [0.29, 0.68]	0.49 [0.31, 0.68]	
	Standing	6	0.86 [0.80, 0.91]	0.79 [0.68, 0.91]	0.85 [0.80, 0.91]	0.85 [0.84, 0.87]	
	Sitting	6	0.91 [0.79, 1.03]	1.00 [0.99, 1.00]	0.96 [0.93, 1.00]	0.93 [0.86, 1.00]	
	Lying	6	0.84 [0.68, 0.99]	0.99 [0.95, 1.02]	0.72 [0.32, 1.12]	0.72 [0.38, 1.05]	
	Jumping	0	0.00 [0.00, 0.00]	1.00 [1.00, 1.00]	0.00 [0.00, 0.00]	0.00 [0.00, 0.00]	
e)	NTNU HAR-Children- Five seconds						
Test group	Activity type						
TD		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	62	0.93 [0.91, 0.94]	0.96 [0.95, 0.97]	0.92 [0.91, 0.94]	0.92 [0.91, 0.94]	
	Running	62	0.93 [0.91, 0.95]	0.99 [0.98, 0.99]	0.87 [0.83, 0.91]	0.89 [0.86, 0.92]	
	Standing	62	0.87 [0.84, 0.91]	0.97 [0.96, 0.98]	0.92 [0.90, 0.93]	0.89 [0.85, 0.92]	
	Sitting	62	0.96 [0.95, 0.98]	0.98 [0.97, 1.00]	0.95 [0.92, 0.97]	0.95 [0.93, 0.97]	
	Lying	29	0.97 [0.95, 0.98]	0.99 [0.99, 1.00]	0.94 [0.89, 0.99]	0.95 [0.91, 0.99]	
	Cycling	21	0.88 [0.80, 0.96]	1.00 [1.00, 1.00]	0.93 [0.88, 0.97]	0.89 [0.82, 0.96]	
	Jumping	14	0.92 [0.89, 0.96]	1.00 [1.00, 1.00]	0.94 [0.92, 0.97]	0.93 [0.91, 0.95]	
CP Stan		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	10	0.89 [0.86, 0.92]	0.96 [0.95, 0.98]	0.89 [0.85, 0.93]	0.89 [0.85, 0.92]	
	Running	9	0.78 [0.65, 0.92]	0.99 [0.99, 1.00]	0.84 [0.79, 0.90]	0.80 [0.71, 0.88]	
	Standing	10	0.92 [0.88, 0.96]
0.99 [0.98, 0.99]	0.95 [0.93, 0.96]	0.91 [0.87, 0.95]	0.92 [0.88, 0.95]	
	Sitting	10	0.99 [0.98, 0.99]	0.99 [0.99, 1.00]	0.97 [0.96, 0.99]	0.98 [0.97, 0.99]	
	Lying	10	0.99 [0.98, 1.00]	1.00 [1.00, 1.00]	0.98 [0.96, 0.99]	0.98 [0.97, 0.99]	
	Cycling	2	0.59[-0.74, 1.92]	1.00 [1.00, 1.00]	0.99 [0.80, 1.17]	0.73[-0.37, 1.83]
0.87 [0.77, 0.97]	
	Jumping	7	0.83 [0.67, 0.98]	1.00 [1.00, 1.00]	0.95 [0.89, 1.00]	0.87 [0.77, 0.97]	
CP Free		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	6	0.59 [0.42, 0.77]	0.87 [0.82, 0.93]	0.58 [0.52, 0.65]	0.57 [0.48, 0.66]	
	Running	5	0.52 [0.24, 0.80]	0.98 [0.96, 0.99]	0.33 [0.12, 0.55]	0.37 [0.19, 0.54]	
	Standing	6	0.81 [0.73, 0.89]	0.75 [0.59, 0.90]	0.82 [0.76, 0.88]	0.81 [0.76, 0.85]	
	Sitting	6	0.82 [0.69, 0.96]	0.99 [0.98, 1.00]	0.89 [0.75, 1.03]	0.85 [0.74, 0.95]	
	Lying	6	0.91 [0.83, 0.99]	0.98 [0.95, 1.02]	0.65 [0.26, 1.05]	0.70 [0.36, 1.04]	
	Jumping	0	0.00 [0.00, 0.00]	1.00 [1.00, 1.00]	0.00 [0.00, 0.00]	0.00 [0.00, 0.00]	
f)	NTNU HAR-ChildrenCP- Five seconds						
Test group	Activity type						
TD		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	62	0.93 [0.92, 0.95]	0.96 [0.95, 0.97]	0.92 [0.91, 0.94]	0.93 [0.91, 0.94]	
	Running	62	0.94 [0.92, 0.96]	0.99 [0.98, 0.99]	0.87 [0.83, 0.90]	0.90 [0.87, 0.92]	
	Standing	61	0.89 [0.86, 0.91]	0.97 [0.97, 0.98]	0.92 [0.91, 0.94]	0.90 [0.88, 0.92]	
	Sitting	62	0.97 [0.96, 0.98]	0.98 [0.96, 1.00]	0.95 [0.92, 0.97]	0.95 [0.94, 0.97]	
	Lying	29	0.98 [0.97, 0.99]	1.00 [0.99, 1.00]	0.95 [0.92, 0.98]	0.96 [0.94, 0.98]	
	Cycling	21	0.90 [0.84, 0.97]	1.00 [1.00, 1.00]	0.91 [0.86, 0.96]	0.90 [0.84, 0.96]	
	Jumping	14	0.92 [0.88, 0.96]	1.00 [1.00, 1.00]	0.93 [0.90, 0.96]	0.92 [0.90, 0.94]	
CP Stan		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	10	0.90 [0.88, 0.93]	0.96 [0.94, 0.98]
0.99 [0.99, 1.00]	0.89 [0.84, 0.93]	0.90 [0.86, 0.93]	
	Running	9	0.82 [0.70, 0.94]	0.99 [0.99, 1.00]	0.85 [0.80, 0.91]	0.83 [0.75, 0.91]	
	Standing	10	0.92 [0.87, 0.97]	0.95 [0.94, 0.96]	0.92 [0.89, 0.95]	0.92 [0.88, 0.96]	
	Sitting	10	0.99 [0.98, 1.00]	0.99 [0.99, 1.00]	0.97 [0.96, 0.99]	0.98 [0.98, 0.99]	
	Lying	10	0.98 [0.97, 1.00]	1.00 [1.00, 1.00]	0.99 [0.98, 1.00]	0.99 [0.98, 0.99]	
	Cycling	2	0.70 [0.63, 0.77]	1.00 [1.00, 1.00]	1.00 [1.00, 1.00]	0.82 [0.78, 0.87]	
	Jumping	7	0.84 [0.70, 0.98]	1.00 [1.00, 1.00]	0.95 [0.90, 1.00]	0.88 [0.79, 0.97]	
CP Free		Sub	Sensitivity	Specificity	Precision	F1 Score	
	Walking	6	0.59 [0.37, 0.80]
0.58 [0.24, 0.92]	0.89 [0.82, 0.95]	0.61 [0.52, 0.71]	0.57 [0.47, 0.67]	
	Running	5	0.58 [0.24, 0.92]
0.83 [0.74, 0.91]	0.98 [0.95, 1.00]	0.38 [0.24, 0.53]	0.41 [0.28, 0.54]	
	Standing	6	0.83 [0.74, 0.91]	0.78 [0.64, 0.91]	0.84 [0.78, 0.90]	0.83 [0.81, 0.85]	
	Sitting	6	0.89 [0.77, 1.01]	0.99 [0.99, 1.00]	0.90 [0.76, 1.05]	0.89 [0.78, 1.00]	
	Lying	6	0.83 [0.62, 1.05]	0.99 [0.95, 1.02]	0.73 [0.36, 1.10]	0.72 [0.40, 1.05]	
	Jumping	0	0.00 [0.00, 0.00]	1.00 [1.00, 1.00]	0.00 [0.00, 0.00]	0.00 [0.00, 0.00]	
Sensitivity, specificity, precision, and F1 Score calculated as mean across all participants in the test group and [95% confidence interval]. Sub = Subjects, number of participants detected with the activity. Each sub-table represent one of the HAR-models for the three predefined test groups. TD = Typically developing, CP stan = Cerebral Palsy standardised activities, CP Free = Cerebral Palsy, Free play.

In the two 3-seconds models (Table 3c and 3d) all activities had still high accuracy for TD and CP Stan (range: 0.78–0.99), both the difference from the corresponding 1-second models (average: 0.01–0.02) and between the 3-seconds models (Table 3c and 3d) were small (F1 Score <0.03). For the CP Free group, the HAR-ChildrenCP 3-seconds (Table 3d) had better F1 Scores (diff: 0.02–0.06) than HAR-Children 3-seconds (Table 3c). However, for all activities the HAR-ChildrenCP 1-second model (Table 3b) were slightly better (range: 0.01–0.04) than 3-seconds (Table 3d).

For the TD group the HAR-Children 5-seconds model (Table 3e) had lower F1 Scores in all activities (range of decrease in F1: 0.01–0.09) compared to 1- and 3-seconds models (Table 3a–3d), except for cycling, where it was higher (< 0.06 increase in F1) with the 5-seconds models. In the two CP groups the HAR-ChildrenCP 5-seconds (Table 3f) had higher F1 Scores (range increase in F1: 0.01–0.04) than HAR-Children 5-seconds (Table 3e), except for the activity lying. When comparing the HAR-ChildrenCP 1-second (Table 3b) and the 5-seconds model (Table 3f) for the CP Stan group, the average difference in F1 Score was 0.03. Also, here the cycling score was higher with the 5-seconds model. In CP Free, the average difference in F1 Score was 0.05 in favour of the 1-second model. Mark that this was without jumping, while jumping shows the largest difference (Table 3e and 3f).

3.3. Confusion matrixes

In all the confusion matrixes in Figs 3–5 the misclassification was higher in the test group CP Free for all HAR-models, where walking was misclassified as standing on average 37% and standing as walking in 10.8% of the instances. Running was more often misclassified as walking in the two CP groups, with average 34.2% in the CP Free group and 17% in CP Stan. Jumping was often misclassified in CP Free, with 100% misclassification in both 3- and 5-seconds models, however for CP Stan jumping was well predicted for all models and higher than the TD group. The difference between HAR-Children and HAR-ChildrenCP decreased with window size (Figs 3–5). Furthermore, the performance for the HAR-ChildrenCP models decreased with increasing windows while the HAR-Children increase for the CP Free test group, however not for the other test groups (Figs 3–5).

10.1371/journal.pone.0308853.g003 Fig 3 Confusion matrix for the 1 second models with non-overlapping windows.

One matrix for each of the three test-groups. Number of subjects are the same as presented in Table 3. The rows represent the video annotated or labelled activity types. The columns represent the predicted activity types. All numbers are percentages.

10.1371/journal.pone.0308853.g004 Fig 4 Confusion matrix for the 3 second models with non-overlapping windows.

One matrix for each of the three test-groups. Number of subjects are the same as presented in Table 3. The rows represent the video annotated or labelled activity types. The columns represent the predicted activity types. All numbers are percentages.

10.1371/journal.pone.0308853.g005 Fig 5 Confusion matrix for the 5 second models with non-overlapping windows.

One matrix for each of the three test-groups. Number of subjects are the same as presented in Table 3. The rows represent the video annotated or labelled activity types. The columns represent the predicted activity types. All numbers are percentages.

4. Discussion

This study validates two HAR models’ ability to predict habitual physical activities in TD children and children with CP (GMFCS I & II). Both models accurately predict standardised activities, with the best overall accuracy for the CP Stan group. The two 1-second models outperform the other models for all test groups. The HAR-ChildrenCP 1-second had preferable performance for all test groups in all activities except for cycling, where 3- or 5-seconds perform better. In the CP Free group there was more variability between models and wider confidence intervals within activities. There were also larger differences in F1 Scores between HAR-Children and HAR-ChildrenCP models when tested on the CP Free group. The results of the present study show that window size is of importance, and ideal size depends on the target activity. The most challenging activities for the models to correctly predict are running, walking, and jumping for the CP groups.

4.1 The difference in performance between HAR-Children and HAR-ChildrenCP

The goal of activity recognition is to correctly predict daily life activities, particularly in free-living situations. Our overall accuracy for TD children ranges from 92–95%, representing an improvement over prior models designed for the same age group and simulated free living, which report accuracy between 62–86% [17, 18]. Notably, studies with higher accuracy exists, however tested on a treadmill [16], limiting their applicability to real life situations. The results of the present study align favourably with previous studies including standardised protocols and children with CP, that have achieved accuracy exceeding 90% [24, 33, 34]. Moreover, our study’s achievement of accuracies >75% in simulated free-living conditions for children with CP is particularly notable, given the limited existing research with suboptimal accuracy [34]. Our model trained with free play activities also performs best for the CP Free group, highlighting the importance of specific training data [23, 24].

The two models consistently demonstrate superior performance when applied to the TD and CP Stan groups, compared to the CP Free group. Intriguingly, in some models, the CP Stan group achieves higher overall and activity-specific performance than the TD group. This implies that the HAR models may not be primarily challenged by the deviating movement patterns in CP. Instead, the presence of unstandardised activities and sporadic transitions within free play emerges as a potential challenge for HAR models. In our data the inferior performance for the two groups that include free play, TD and CP Free, might also be due to the limited training data, with mere five minutes of free play in TD and six participants in CP Free. Nevertheless, free play activities or variability in movement pattens might be beneficial for the model’s ability to predict standardised activities, both for TD and children with CP. Which can be exemplified by the superior performance of the HAR-ChildrenCP models for both CP Stan and all activities, except for jumping, in the TD group. This is interesting because one would expect that the heterogeneity within the CP population hinders the model’s performance. However, in our experiment, this diversity within the CP population could potentially contribute to improved performance for standardised activities, thereby benefiting both TD and CP Stan. Overall, the HAR-ChildrenCP model includes all 79 children and thereby represent a wide variation of movement patterns, and with the contribution of the CP Free group it offers superior performance in most activities for all test groups.

4.2 Accuracy and misclassification of activities

In regards of misclassification of activities, the two groups with standardised activities had a lower misclassification of jumping, compared to free play. We can theorise that structured, sequential jumping, as seen in these protocols, are easier for the models to recognize compared to sporadic jumping or jumping imbedded in other activities in free play. Conversely, even though all jumping from the TD and CP Stan groups were included in the training, it might lack the specificity required for the CP Free group. The misclassification of jumping as standing and walking was more pronounced in the TD group than the CP Stan group, and thereby supports the notion that the primary challenge is free play activities.

There were better predictions of walking for the TD group compared to the children with CP. This can be attributed to differences in walking patterns between the groups [30, 31]. The asymmetric movement during walking in CP creates a distinct difference in accelerometery from TD children [31]. Furthermore, children with CP require longer time during acceleration, and have a steeper increase in acceleration measures related to increase in speed [31], which might affect the recommended choice of window size. In our results the 5-second HAR-Children model perform best for CP Free, indicating that without CP training data, the models with smaller window sizes have difficulties detecting short bouts of walking and running with different accelerometric features.

The misclassification of running as walking in the two CP groups can also be attributed to asymmetry, which is exacerbated with increasing musculoskeletal demands during running [45]. The misclassification may also be influenced by the representation of different walking and running paces in our training data. In real life scenarios, children tend to transition intermittently between walking and running paces. Therefore, it is crucial for the model to recognize the activities regardless of pace. Notably, this limitation in differentiating walking paces is consistent with other publications on children with CP, none of which include running [24, 33, 34]. Additionally, in existing studies regarding TD children the same misclassifications between walking paces are present, however less misclassification between running and walking [16]. The irregularities in gait characteristics among children with CP can also be positive for model performance, where the heterogeneity, might explain the reduced misclassification of walking in both CP groups and TD with the HAR-ChildrenCP model.

4.3. The effect of window size and overlapping windows

Our models confirm the significance of window settings in activity recognition, and its impact depend on the target activity. For instance, momentary activities such as jumping, are in our models best detected with the 1-second window, while cyclic activities with longer cycle lengths such as cycling is better detected with 5-seconds window. Interestingly, when the window size was increased to 5-seconds, the disparity in performance between HAR-Children and HAR-ChildrenCP models diminished in the CP Free group. This suggest that the intermittent changes in the CP Free group are concealed by majority voting in the 5 second window, and conceivably the intermittent activities in the training data play a reduced role for model performance. This data reduction effect of the larger windows have been beneficial in previous research and for specific activities [46], such as walking, cycling, and running, which have cycle rates exceeding one second. Existing literature involving children with CP typically use window sizes ranging from five to fifteen seconds [24, 33–35], reporting higher F1 Scores with larger windows [24]. Ferrari et al., [38] advocates for windows that are long enough to capture a complete cycle of a specific movement yet short enough to distinguish between similar movements.

Within the context of evaluating children’s daily activities, our data suggest that the 1-second models should be used when the objective is to detect momentary activities, such as jumping. However, it is essential to recognise that these momentary actions often transpire within a broader spectrum of gross motor activities, such as running, walking, and standing, which is often of primary interest. In this context larger window sizes offer a more efficient approach to data reduction and improves model performance in typical daily life scenarios. Hence, in the context of everyday life and health outcomes the utilization of the 3-seconds model emerges as potentially preferrable. This choice is substantiated by our results, where the 3-second model predicts momentary activities, but also activities of extended duration, including cycling, with commendable accuracy.

Regarding the choice between overlapping or non-overlapping windows, our study reveals marginal difference in model performance with slightly better accuracy observed with overlapping windows. The optimal window size represent a trade-off between processing speed and prediction accuracy [21]. Consequently, the benefits of overlapping windows must be weighed against the drawbacks, particularly in clinical application. The time-consuming nature of processing overlapping windows is a notable concern. In our model-training the 1-second with overlap required approximately ten times longer processing time than the 5-seconds without overlap. This substantial time difference is consistent with findings of Dehghani et al., [39], where segmentation with overlapping windows took twice as long, and training took four times as long compared to non-overlapping. Furthermore, in their study, the memory requirements for overlapping windows were nearly nine times greater [39]. Given the trade-off in accuracy, questions arise regarding the practical utility of overlapping windows in clinical context, especially when dealing with large data sets.

4.4. Strengths and limitations

Comparing HAR and ML models present considerable challenges due to variations in classifier usage, study populations, activity protocols, accelerometers, accelerometer placements, among other factors. Our study is unique for our utilization of the XGBoost classifier and two accelerometers. The use of two or more accelerometers and our placements have been emphasized as preferred settings [13, 33]. The relatively infrequent use of the XGBoost classifier in HAR research contrasts with the prevalence and recommendations in other fields, and have shown strengths, particularly due to its sequential learning [7, 13]. Moreover, our study incorporates specific configurations and specifications that have demonstrated excellence in other population groups [7, 13, 14]. Additionally, our study is strengthened and novel by the inclusion of playful behaviour and group activities, and thereby simulated habitual activities for children and children with CP. Furthermore, our data set is relatively large in comparison to other validation studies including children.

Some limitations warrant consideration. We have different group sizes and amount of training data from TD children and children with CP. Following the principle that more training data and variation in movement patterns generally benefits model training we opted to include all available data. Thus, we think that the different group sizes between TD and CP will not substantially influence the model performance. However, adding more CP training data would probably improve the model performance further, due to even more variation in the training data’s movement patterns. We have limited free play training data, potentially impacting prediction accuracy for the CP Free group. Additionally, the absence of a TD Free group prevents a direct examination of the contrasting effects of free play versus the presence of CP on model performance.

4.5 Future perspectives and implications

The present article underscores the specificity of the HAR method, emphasizing the importance of the training data as well as technical specifications for future research and clinical work. One noteworthy consideration is the necessity for a specific HAR model for CP, GMFCS I &II as the disparity between HAR-children and HAR-ChildrenCP appear relatively minor for standardised activities. Conversely, based on our results we can suggest the call for the HAR-ChildrenCP model, or children with disabilities model, is not restricted to children with CP, but also to TD children. Such a model, like our HAR-ChildrenCP, may offer enhanced versatility by greater variation within the training data, facilitating the recognition of a broader spectrum of movement patterns in the general child population. An intriguing avenue for future research is to expand the free play data set, which involves training our model with activity data collected in children’s daily environment, such as in their home, kindergarten, school, and leisure activities. This further expansion into real-world scenarios could yield valuable insights and advance the applicability of our model and HAR methods in child populations. Furthermore, our development of a HAR-model, and future enhancements of such a model for children with CP, enables the examination of treatment effects of habilitation interventions that aims to influence the amount, patterns, or distribution of PA. A potential limitation for future clinical use is sensor misplacement, as it is crucial to ensure correct alignment with gravity for accurate data collection. Therefore, it’s important to educate participants on proper sensor placement, especially when positioning sensors outside the clinic.

The accuracy of our sub-models is between 75–95%, which is comparable with previous studies. As described above our model demonstrates greater performance in certain activities and incorporates free play and childlike behavior. However, there remains a debate if our results are good enough, as we recognize some challenges regarding activities during free play. Therefore, in addition to expanding the free play data set, it may be beneficial to explore alternative approaches, such as deep learning experiments. In fact, to optimize activity recognition for future use, it would be intriguing to compare various HAR methods for children, particularly for those with deviating movement patterns.

5.Conclusion

Both HAR models demonstrate precise predictions for standardised activities in both TD children and those with CP, and slightly less precise predictions for free play activities, but still precises and favourable compared to previous models. Among the three groups, the CP Stan has the most accurate predictions, prompting consideration of the influence of impairment versus free play activities. Based on the highest overall accuracy, the NTNU HAR-ChildrenCP model with 1-second window would be recommended for all three test groups. However, the optimal window size and overlap depend on the target activity. For activities such as cycling, 3- or 5-seconds windows perform better. Considering the ability to predict both momentary activities and activities of extended duration, the 3-second window without overlap would be recommended for population measurements.

Supporting information

S1 Table Overview of conducted activities for each test group.

(TIF)

S2 Table Definition of activities.

(TIF)

S3 Table Overall accuracy for each model with and without overlap and original activity classes.

Test groups: TD = typically developing children, CP Stan = Cerebral palsy with standardised activities, CP Free = Cerebral palsy with Free play. The range of accuracy is 0–1, where higher scores are better.

(TIF)

S1 Fig Overview of the 12 sub-models.

TD children = typically developing children, CP = Cerebral Palsy, O = Overlap, Test groups: TD = typically developing children, CP Stan = Cerebral palsy with standardised activities, CP Free = Cerebral palsy with Free play.

(TIF)

We thank the children and families that participated in this study, and other contributors that assisted with the data collection especially Roar M. Fenne. We also thank Kerstin Bach for the utilization of the NTNU-HAR method.

10.1371/journal.pone.0308853.r001
Decision Letter 0
Narayan Jyotindra Academic Editor
© 2024 Jyotindra Narayan
2024
Jyotindra Narayan
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submission Version0
8 Jan 2024

PONE-D-23-37915Validation of two novel human activity recognition models for typically developing children and children with Cerebral Palsy.PLOS ONE

Dear Dr. Tørring,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Feb 22 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Jyotindra Narayan

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

3. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

4. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. 

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

5. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

6. We notice that your supplementary figures are uploaded with the file type 'Figure'. Please amend the file type to 'Supporting Information'. Please ensure that each Supporting Information file has a legend listed in the manuscript after the references list.

Additional Editor Comments:

==============================

The reviewers have praised the work for significant contributions on human activity recognition in typically developing children and those with Cerebral Palsy but suggests improvement in addressing methodological concerns. Specific suggestions include clarifying data acquisition methods, questioning biases in training models (by both reviewer 1 and 3), and addressing discrepancies in group sizes. The reviewers 2 and 3 have common concerns about tangential literature and references used. The reviewer 1 have marked suggestions over the manuscript's pdf (see attachment), helping authors to address minor technical and langauge concerns. Finally, per the reviewer 3 suggestions, the authors are urged to discuss the implications of group sizes and consider ways to enlarge the free play dataset.

==============================

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: typically developed children. Overall, on the technical side, the presented manuscript is technically sound, and all the result data support the authors' claims in the result and conclusion sections. The authors' choice of F-1, understanding of F-1, and usage are respectable. Under the microscope, however, there are minor editorial/technical errors—for example, the authors' inconsistent citation style or difficulty in explaining their 12 models concept. Furthermore, the authors haven't made any data available for public consumption. Please refer to the reviewer attachment for a complete list of questions/comments in the form of notes embedded in the manuscript PDF. Overall, the reviewer concluded that this manuscript is legible and technically sound, and all the analyses have been done appropriately.

Reviewer #2: Authors have presented an ML-based Human Activity Recognition (HAR) model for typically developing (TD) children and children with Cerebral Palsy (CP).

The study discusses the advantages of HAR models for the same and challenges in the setup and hyperparameters such as training data, optimal accelerometer placements,

and window size considerations. It presents the usage of the XGBoost classifier (12 sub-models) with 1, 3, and 5 sec time windows. The ML-based approach [1, 2, 3] is a very common HAR.

Following are my comments to further improve this article:

1. The applicability of the study may be broadened by including the usage of HAR in different health and rehabilitation fields such as exercise/YOGA pose classification [4], rehabilitation of individuals after sports injury, and improving the posture for the different exercises and training.

2. The study can also include a) a comparison of deep learning-based models [3, 4, 6] along with ML-based models for the HAR model for TD and CP children and b) a study of multimodal features [6] in HAR. Authors may refer to the studies [1, 5, 6] for choosing appropriate ML and DL methods for the task.

3. The author should consider the experimentation of multiple related datasets such as WISDM Activity Prediction, UCI HAR, DSADS, etc. Please refer to Table 1 from [6] for a list of publicly available datasets of HAR.

4. The possible direction to measure the automatic improvement in CP children after the rehab/therapy would further boost the applicability of research in a real-world application.

5. In line 73 of the Introduction section, TD is mistyped as TP.

6. In line 243 of section 2.6, it is written as F-measure is the harmonic mean between precision and sensitivity, where as it is the harmonic mean between precision and recall [7].

References:

1. Gupta, N., Gupta, S.K., Pathak, R.K. et al. Human activity recognition in artificial intelligence framework: a narrative review. Artif Intell Rev 55, 4755–4808 (2022). https://doi.org/10.1007/s10462-021-10116-x

2. Csizmadia, G., Liszkai-Peres, K., Ferdinandy, B. et al. Human activity recognition of children with wearable devices using LightGBM machine learning. Sci Rep 12, 5472 (2022). https://doi.org/10.1038/s41598-022-09521-1

3. D. Sakkos, K. D. Mccay, C. Marcroft, N. D. Embleton, S. Chattopadhyay and E. S. L. Ho, "Identification of Abnormal Movements in Infants: A Deep Neural Network for Body Part-Based Prediction of Cerebral Palsy," in IEEE Access, vol. 9, pp. 94281-94292, 2021, doi: 10.1109/ACCESS.2021.3093469.

4. Vallabhaneni, N., Prabhavathy, P. Segmentation quality assessment network-based object detection and optimized CNN with transfer learning for yoga pose classification for health care. Soft Comput (2023). https://doi.org/10.1007/s00500-023-08863-w

5. Wan, S., Qi, L., Xu, X. et al. Deep Learning Models for Real-time Human Activity Recognition with Smartphones. Mobile Netw Appl 25, 743–755 (2020). https://doi.org/10.1007/s11036-019-01445-x

6. Kaixuan Chen, Dalin Zhang, Lina Yao, Bin Guo, Zhiwen Yu, and Yunhao Liu. 2021. Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges, and Opportunities. ACM Comput. Surv. 54, 4, Article 77 (May 2022), 40 pages. https://doi.org/10.1145/3447744

7. Schütze, H., Manning, C. D., & Raghavan, P. (2008). Introduction to information retrieval (Vol. 39, pp. 234-265). Cambridge: Cambridge University Press.

Reviewer #3: The article “Validation of two novel human activity recognition models for typically developing children and children with Cerebral Palsy” is well written and pleasurable to read. It presents a classifying method of physical activities in CP children and above all the influence of different parameters of this method: learning dataset composed of only TD children or a mix of TD and CP children, the length of the observation window, the choice to use or not an overlap to define this window. Results are clear and complete which helps to understand the effect of each parameter. Collecting and processing the dataset has been a very consequent job that might be very useful for any research team working on physical activity recognition in CP children in real life which is a crucial question. I would like to sincerely thank the authors for that huge and useful work.

Nevertheless, some methodological choices and the ensuing elements of discussion should be address to strengthen the impact of this study.

Specific comments

L44: As soon as the abstract you should indicate how data are acquired (video, IMU, … ?)

L85: It is a bit surprising that in most of previous studies, the authors did not think that training the model with one population (adults) would be a bias when using the model with another one (children). This bias is presented as on major argument for your study. However, I am pretty sure this potential bias is largely exaggerated as in most studies aiming at recognize human activities the training population has the same characteristics as the target one.

L99: And even, the three articles you present come from the same research team in Queensland, Australia. I am very surprised that only two research teams in the world are interested in PA recognition in CP. Are you only focusing on the articles in which the methodology used for this recognition is very close to yours so that comparison is easier? If so that needs to be modified. You should be able to compare your results even when the classifier’s principle is different from yours.

L115: It is not clear what are the ‘two’ models. There is no prior justification for the need of two different models and what will be the advantage of one vs. the other, or in which cases one should be used. Just after, I understand that you mean two models because learning data are not the same, e.g. one with only TD and the other one mixing TD and CP. The question is to know if is the methodology identical in both cases. If it is, I would rather say that there is actually only one model with which you are testing the influence of identifying the model’s parameters from two learning datasets. Indeed, afterwards you are talking about 12 models when combining this learning dataset parameter to two other ones, i.e. length of window (1, 3 or 5 s) and the overlap.

L123: The two groups have very different size (ratio 1:4). It means that in your second dataset, mixing TD and CP, TD activities are actually the large majority. On L201, you don not indicate if the second dataset includes all data from TD children. I thus assume that it is the case. It actually means that your second ‘model’ center of gravity is actually drifted towards TD movements.

L155: What is the sensitivity of your method to accelerometer misplacement? The final goal is to monitor activities in real life situation in which CP children or their parents will place the devices. They are not clinicians trained to the use of such device. One usual error in that case is axes misalignment. It is important to test this robustness to that kind of bias. In your experiment, these biases were absent as all the devices were placed by the team (maybe even the same operator). It implies that the data used in the LOSOCV evaluation is homogeneous regarding these biases. As being outside the learning database, the child left-out represents a subject in real life. But actually, sensors’ placement on a child in real life will be less precise.

L197: Fig. 1 is hardly readable.

Discussion: you should discuss a bit more about the groups’ sizes. I agree with you that your whole dataset with 79 children is quite large compared to other studies. But 80% of them are TD. And moreover, as some activities (e.g. jumping) is more difficult for CP children, when focusing more finely on the dataset activity per activity, this ratio is sometimes even more in favor of TD children.

L462: I agree with you that it is always a bit frustrating not to have more free play datasets. Free play, even in clinical environment, is one important way to be closer to real life. You honestly recognize that your free play dataset should be enlarged but not indicating how this could be done. If you have an idea, could you explain it briefly?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Dr. Durgesh Kumar

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachment Submitted filename: PONE-D-23-37915_with-comments.pdf

10.1371/journal.pone.0308853.r002
Author response to Decision Letter 0
Submission Version1
21 Feb 2024

Response to editor and reviewers.

Our response are provided under the comments.

In the word file (response to editor and reviewers) the response are in blue, page and line numbers refer to the manuscript file with track changes.

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response: Thank you for this, we have checked the guidelines again and made minor changes.

We have changed the file names of our figures after the standard. We have also changed our referral to these in the manuscript according to the guidelines. We have also changed the frontpage according to the guidelines, we have removed the authors contribution, page 2, as it seems to only be required in the submission form.

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

Response: Thank you for pointing this out. We have updated our manuscript and the ethics statement in the submission form. Page 7, line 137-141: “All participants were given written age-appropriate information about the study. This was distributed at school and taken home together with written information and consent forms to their parents/guardians. If the parents gave their written consent, verbal information was given to the children before the study started”.

3. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

Response: Our code is a further development of previous code from our research group at NTNU. This code is already uploaded to GITHUB (https://github.com/ntnu-ai-lab/harth-ml-experiments). Our code includes minor changes to this code to fit the scope of our study. Up on publication our code will be provided as well.

4. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

Response: We have corrected our funding information in the submission form. We do not have a specific grant number for this work. We have also updated our cover letter to include information of financial disclosure.

5. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

Response: We have created a platform using Dataverse (https://dataverse.no/) where all our information regarding the data set will be published. Currently the cite is registered without the raw datafiles. However, all other metadata is provided, and the files will be uploaded on acceptance. Here is the DOI number (https://doi.org/10.18710/EPCXCC) that we refer to in our data availability statement. The DOI will be activated up on acceptance.

6. We notice that your supplementary figures are uploaded with the file type 'Figure'. Please amend the file type to 'Supporting Information'. Please ensure that each Supporting Information file has a legend listed in the manuscript after the references list.

Response: We are sorry for the mislabeling. We have changed the file type and file name according to guidelines.

Additional Editor Comments:

==============================

The reviewers have praised the work for significant contributions on human activity recognition in typically developing children and those with Cerebral Palsy but suggests improvement in addressing methodological concerns. Specific suggestions include clarifying data acquisition methods, questioning biases in training models (by both reviewer 1 and 3), and addressing discrepancies in group sizes. The reviewers 2 and 3 have common concerns about tangential literature and references used. The reviewer 1 have marked suggestions over the manuscript's pdf (see attachment), helping authors to address minor technical and langauge concerns. Finally, per the reviewer 3 suggestions, the authors are urged to discuss the implications of group sizes and consider ways to enlarge the free play dataset.

==============================

Response: We are grateful for the thoughtful comments and suggestions from the reviewers and the editorial office, which allowed us to improve our manuscript significantly. We have addressed the comments about tangential literature by providing additional references in our manuscript. We detail below how we responded to each concern from the reviewers point-by-point.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

________________________________________

5. Review Comments to the Author

Reviewer #1:

Typically developed children. Overall, on the technical side, the presented manuscript is technically sound, and all the result data support the authors' claims in the result and conclusion sections. The authors' choice of F-1, understanding of F-1, and usage are respectable. Under the microscope, however, there are minor editorial/technical errors—for example, the authors' inconsistent citation style or difficulty in explaining their 12 models concept. Furthermore, the authors haven't made any data available for public consumption. Please refer to the reviewer attachment for a complete list of questions/comments in the form of notes embedded in the manuscript PDF. Overall, the reviewer concluded that this manuscript is legible and technically sound, and all the analyses have been done appropriately.

Response:

Thank you for your comments and feedback. We have made changes accordingly.

- Regarding the F1 Score, see point 9 in the specific comment section bellow.

- Our citations and references are corrected according to Vancouver reference style.

- We have made several changes to the document regarding the 12 sub-model explanation, which will make it easier for the reader to follow our argumentation. See new supporting figure 4 (S4 Fig) and revised manuscript on page 10, line 222-224. “The two main models comprise a total of 12 sub-models, 6 for each main model, covering 1-, 3-, and 5-seconds windows with and without overlapping windows, see supporting information (S4 Fig) for overview of the 12 sub-model creation”.

- Regarding the data availability we have created a cite on Dataverse where the data will be published upon exception/publication (https://doi.org/10.18710/EPCXCC). Our code can be found with similar codes from our research team at GITHUB: https://github.com/ntnu-ai-lab/harth-ml-experiments.

Comments from the manuscript file:

1. Line 79. Missing definition.

Response: Thank you for this comment. Definition of TD is typically developing, we have made a typo on page 4 line 76. We have now corrected the typo and the definition is there.

2. Line 80. This notion is in conflict with previous statement.

Response: We agree that the sentences needed adjustments. We have rewritten the paragraph.

Page 4/5, line 81-86.

3. Line 89. Citations?

Response: We have rewritten the sentence and added references for the updated prevalence. Page 5, line 96-98. “CP is one of the most frequent disability among children (25) with the global prevalence of approximately 1.6 /1000 live births in high income countries (26).”

4. Line 99. Wrong citation style.

Response: We have corrected the format.

5. Line 172-174. Was the labelling done with one person or more? If more, an addition of Cohen's kappa agreement coefficient would be beneficial in justifying the dataset’s quality by looking at the reliability between the rater(labeller).

Response: More than one person labelled the data. In Ustad et al., (2023) and Bach et al., (2022) they have reported interrater reliability, with the coefficient of >0.95. Using the same method and activity definitions and overlapping raters as our study. We have added this information in our manuscript on page 9, line 189-191.

Ustad A, Logacjov A, Trollebø SØ, Thingstad P, Vereijken B, Bach K, et al. Validation of an Activity Type Recognition Model Classifying Daily Physical Behavior in Older Adults: The HAR70+ Model. Sensors. 2023;23(5):2368.

Bach K, Kongsvold A, Bårdstu H, Bardal EM, Kjærnli HS, Herland S, et al. A Machine Learning Classifier for Detection of Physical Activity Types and Postures During Free-Living. Journal for the Measurement of Physical Behaviour. 2022;1(aop):1-8.

6. Line 183. Why 50% overlap?

Response: When employing HAR models with fixed windows there is a potential loss of some activities at the endpoint of the window. The choice of a 50% overlap was deliberate to ensure that no single endpoint in the data was overlooked as the new window would be in the middle of the previous window. The 50% overlap is commonly used in HAR, when employing these fixed windows. We considered increasing the overlap, however, as it would substantially increase the processing time it will make the model unsuitable for clinical purposes. Under we have provided two references that also use 50% overlap in their HAR models. We have added a sentence on this in our manuscript on page 9, line 199-200. “The overlap was included to investigate potential loss of activities at the endpoint of the window”.

Lara OD, Pérez AJ, Labrador MA, Posada JD. Centinela: A human activity recognition system based on acceleration and vital sign data. Pervasive and mobile computing. 2012 Oct 1;8(5):717-29.

Ferrari A, Micucci D, Mobilio M, Napoletano P. On the personalization of classification models for human activity recognition. IEEE Access. 2020 Feb 12;8:32066-79.

7. Line 191. Pleas don’t forget to cite the original XGBoost paper: https://arxiv.org/abs/1603.02754

Response: Thank you for providing this reference for us. We have cited at page 10, line 208.

8. Line 201-202. A table or a more descriptive definition, whether in the text or the supplement section, would help greatly digest this sentence.

Response: We agree and have provided a figure in the supplementary information (S4 Fig) that explain how we get to 12 sub models and how these are linked. We have revised our manuscript page 10, line 217-224.

9. Line 204 and 205. Why use k-fold here and LOSOCV in the other section? A K-fold CV might have higher accuracy but is more likely to be artificial due to intra-subject contamination, unlike LOSOCV. Additionally, leave-n-subject-out cross-validations might be more beneficial as they provide a cross-subject generalisability (subject-independent performance) overview while maintaining a manageable number of cross-validation-round.

Response: Regarding the use of K-fold or LOSOCV we have only used the method of LOSOCV. We have used the LOSOCV method as a k-fold on 6 subjects, or as you suggested leave-n-subjects-out-cross-validation. We have revised the manuscript, and we are sorry for the confusion. Modifications on page 10, line 224-225. “For each model we initially performed a leave-six-subjects-out-cross-validation with hyperparameter optimization in the form of a grid search”.

10. Line 239. Are they class weighted or unweighted?

Response: We have calculated macro/ non-weighted F1 Score using the following formula F1 Score= 2x precision x recall / precision + recall. As we only report class-wise F1 Scores we have concluded that there is no need for calculating the weighted F1 Score in our results. Updated manuscript on page 12, line 265.

Reviewer #2:

Authors have presented an ML-based Human Activity Recognition (HAR) model for typically developing (TD) children and children with Cerebral Palsy (CP).

The study discusses the advantages of HAR models for the same and challenges in the setup and hyperparameters such as training data, optimal accelerometer placements,

and window size considerations. It presents the usage of the XGBoost classifier (12 sub-models) with 1, 3, and 5 sec time windows. The ML-based approach [1, 2, 3] is a very common HAR.

Response: Thank you for your comments on our manuscript and suggestions for further work and improving the usability of human activity recognition models. Additionally, we would thank you for

Attachment Submitted filename: Response to editor and reviewers.docx

10.1371/journal.pone.0308853.r003
Decision Letter 1
Narayan Jyotindra Academic Editor
© 2024 Jyotindra Narayan
2024
Jyotindra Narayan
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submission Version1
1 Apr 2024

PONE-D-23-37915R1Validation of two novel human activity recognition models for typically developing children and children with Cerebral Palsy.PLOS ONE

Dear Dr. Tørring,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by May 16 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Jyotindra Narayan

Academic Editor

PLOS ONE

Additional Editor Comments:

==============================

Reviewer #1 recommends accepting the manuscript, acknowledging its contribution to Human Activity Recognition (HAR) in typically developing children and children with cerebral palsy. While noting improvements in presentation and writing, Reviewer #2 raises concerns about the lack of novelty in methodology and research objectives. Suggestions for enhancement include incorporating a deep learning-based model for comparison with XGBoost, utilizing multimodal features combining video image and sensor data, sharing data with the research community, and addressing how the proposed models differ from existing methods. Reviewer #3 appreciates the authors' responses to concerns but suggests including unpublished results regarding sensor misplacement's impact on model performance as a potential limitation.

==============================

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: (No Response)

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: The manuscript solves a real-world problem of Human Activity Recognition (HAR) in normal children and children with celebration palsy. The authors have improved in presentation and writing from the first revised version. However, the manuscript lacks novelty in terms of the proposed methodology and research Objectives. The significant research contributions of the study are as follows:

1. The novel datasets for Human Activity Recognition for Typically Developing Children (TD) and Children with Cerebral Activity.

2. Experiment and Analysis of HAR on the above datasets using XGBoost classifier.

3. Study of the impact of windows size (1 sec, 3 sec, 5 sec) and with overlapping.

In my first review, I have provided possible direction to extend the novelty and technical contribution of the paper by including:

1. Inclusion of a Deep learning-based model (such LSTM) for comparison with XGBoost: The author refused to experiment with the model, citing their assumption that it would not improve their results further without even experimenting with it. LSTM and BiLSTM are known to improve the results in sequential data. Further, the authors claimed there was a lack of space to fit their results, as they had too many large result tables. The results table could appropriately be compressed by representing it with suitable diagrams.

2. Inclusion of multimodal features: I mean to use both the video image and sensor data for the HAR. As the confusion matrix (Figure 5) NNTU HAR ChildrenCP-5 sec shows, some of the Walking and running data is wrongly predicted as Standing. In such a scenario, the image taken from the camera could assist the sensor data in proper activity recognition.

3. The authors have highlighted the lack of publicly available data for HAR in children and children with CP. Unless researchers share their data for academic and research purposes, this problem will remain as it is. Therefore, it is requested that the authors share their data with the research community after the journal accepts it. Please share the data sharing policy and some data samples with the Editorial Team before the publication of the papers.

4. How much do the Model NNTU-HAR-Children and NNTU-HAR-ChildrenCP differ from the NNTU-HAR method proposed by Roar M. Fenne?

Other Minor comments:

1. In section 2.4.2 (Video Annotation), it is not clear how many annotators were there in total, and each video frame annotated by how many annotators, and what was the score inter-annotator agreement?

2. The research has not concluded the optimal time window for the HAR in their conclusion section and results discussion section.

Reviewer #3: The authors provided convincing answers to my concerns and consequently modified the article. There is only one concern for which extra information would be useful. Regarding robustness to sensors' misplacement, the authors explained that "Our research group have tested the consequence of sensor misplacement on model performance in previous studies using

the same methodology (unpublished). As long as the sensors are placed in the correct direction towards gravity minor deviations in placements does not affect the accuracy of the model." Even if unpublished yet, the results of these previous studies could be presented in discussion as a potential limitation already taken into account. In the current version, it is as if the limitation does not exist.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Dr. Durgesh Kumar

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

10.1371/journal.pone.0308853.r004
Author response to Decision Letter 1
Submission Version2
9 May 2024

Additional Editor Comments:

==============================

Reviewer #1 recommends accepting the manuscript, acknowledging its contribution to Human Activity Recognition (HAR) in typically developing children and children with cerebral palsy. While noting improvements in presentation and writing, Reviewer #2 raises concerns about the lack of novelty in methodology and research objectives. Suggestions for enhancement include incorporating a deep learning-based model for comparison with XGBoost, utilizing multimodal features combining video image and sensor data, sharing data with the research community, and addressing how the proposed models differ from existing methods. Reviewer #3 appreciates the authors' responses to concerns but suggests including unpublished results regarding sensor misplacement's impact on model performance as a potential limitation.

==============================

Response to editor:

Dear Jyotindra Narayan

We would like to express our gratitude for the thorough review process and the insightful comments provided by the reviewers and the editorial office on our manuscript. We appreciate the time and effort invested by all involved, which has undoubtedly contributed to the improvement of our work. We have carefully considered each comment and suggestion raised by the reviewers and have made necessary revisions to our manuscript accordingly.

Regarding the suggestions put forth by Reviewer #2, we respectfully acknowledge the reviewer's perspective and the importance of exploring various avenues for advancement in Human Activity Recognition (HAR). However, we must clarify that our primary objective is to develop and validate a HAR model tailored to typically developing children and children with cerebral palsy, with deviating movement pattern, by using existing HAR methodology. Our objective was not to compare two methodologies for activity recognition but validate two models.

Reviewer 2 has raised a valid point regarding the novelty of our study's method and objectives. We acknowledge that our study does not introduce significant novelty in terms of methodology, as there already exist Human Activity Recognition (HAR) models employing various machine learning techniques. However, it is worth noting that few of these models are built on data from children, and only three focus on children with cerebral palsy (CP). From our perspective novelty in our study is evident in several aspects. Firstly, the uniqueness of our data set stands out, as it includes a substantial number of typically developing (TD) children alongside children with CP. This diverse data set provides a rich foundation for our analysis. Secondly, the inclusion of free play activities, including childlike activities and ball games, adds a novel dimension. These activities reflect real-world scenarios and add depth to our understanding of activity recognition in children. Thirdly, our comparison between TD and CP models is unprecedented, particularly testing a model designed for children with deviating movement patterns on typically developing children. Lastly, our exploration of window sizes and overlap also contributes valuable insights to the field, by offering insights into optimal parameter settings for activity recognition systems.

In addition to our points raised in the discussion regarding future perspectives and implications, we acknowledge the need for further improvements to HAR methodology. In response to Reviewer 2's suggestion, we have included a new paragraph under this heading about deep learning as a potential avenue for future optimization of HAR in general. We fully recognize the value of further exploration of different methodologies as deep learning, but we believe they warrant dedicated studies of their own. Therefore, we kindly request your understanding in accepting our manuscript without additional experiments that incorporating multimodal features and deep learning-based models.

Once again, we extend our sincere appreciation to the reviewers and the editorial office for their valuable feedback and consideration of our work. We remain committed to contributing to the advancement of knowledge in the field of Human Activity Recognition and look forward to the opportunity to share our findings with the research community.

Thank you for your attention to this matter.

Yours sincerely,

Marte Fossflaten Tørring

NTNU

Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1:

(No Response)

Reviewer #2:

The manuscript solves a real-world problem of Human Activity Recognition (HAR) in normal children and children with celebration palsy. The authors have improved in presentation and writing from the first revised version. However, the manuscript lacks novelty in terms of the proposed methodology and research Objectives. The significant research contributions of the study are as follows:

1. The novel datasets for Human Activity Recognition for Typically Developing Children (TD) and Children with Cerebral Activity.

2. Experiment and Analysis of HAR on the above datasets using XGBoost classifier.

3. Study of the impact of windows size (1 sec, 3 sec, 5 sec) and with overlapping.

In my first review, I have provided possible direction to extend the novelty and technical contribution of the paper by including:

1. Inclusion of a Deep learning-based model (such LSTM) for comparison with XGBoost: The author refused to experiment with the model, citing their assumption that it would not improve their results further without even experimenting with it. LSTM and BiLSTM are known to improve the results in sequential data.

Response:

Thank you for the time and effort you spent on our manuscript. We appreciate your thoughtful consideration of our work and your suggestions for improvements. We acknowledge that our study does not introduce significant novelty in terms of technical methodology, as there already exist Human Activity Recognition (HAR) models employing various machine learning techniques. However, it is worth noting that few of these models are built on data from children, and even fewer focus on children with cerebral palsy (CP).

From our perspective novelty in our study is evident in several aspects. Firstly, the uniqueness of our data set stands out, as it includes a substantial number of typically developing (TD) children alongside children with CP. This diverse data set provides a rich foundation. Secondly, the inclusion of free play activities, including childlike activities and ball games, adds a novel dimension. These activities reflect real-world scenarios and add depth to our understanding of activity recognition in children. Thirdly, our comparison between a TD and CP model is unprecedented, particularly testing a model designed for children with deviating movement patterns on typically developing children. Lastly, our exploration of window sizes and overlap also contributes to valuable insights to the field, by offering insights into optimal parameter settings for activity recognition systems. Our focus stems from our commitment to addressing clinical needs and facilitating activity identification in children with CP. We have clarified this further in the manuscript.

We apologize for giving the impression that we didn't think the suggested models would have any potential improving activity detection. We acknowledge that including additional experiments, as you suggest, could potentially yield some improved results and novelty in terms of methodology. We have included a new paragraph under the heading future perspectives and implications about deep learning as a potential avenue for future optimization of HAR in general (page 25, line 485-492). We fully recognize the value of further exploration of different methodologies as deep learning, but we believe they warrant dedicated studies of their own. Therefore, in our paper, we adhere to the original scope, objectives, and methods of our study. We have clarified our objective on page 5, line 95-102. We hope this clarification resonates with your understanding of our research goals and constraints.

Further, the authors claimed there was a lack of space to fit their results, as they had too many large result tables. The results table could appropriately be compressed by representing it with suitable diagrams.

Response:

We apologize for the words chosen in our previous answer and we thank you for the suggested solution. However, our decision not to expand the experiments beyond the scope of our current study was based on several factors, including the choice to maintain focus on our research question within the clinical context. Introducing further experiments is not solely restricted to space limitations but also risks complicating the manuscript for readers, particularly those outside the technological community.

Moreover, we would like to refer to our answer to your first remark; that expanding the scope of experiments would necessitate the establishment of new objectives. Given our commitment to ensuring accessibility and understanding, especially among clinicians and other stakeholders in the clinical community, we must prioritize clarity and conciseness in our presentation.

2. Inclusion of multimodal features: I mean to use both the video image and sensor data for the HAR. As the confusion matrix (Figure 5) NNTU HAR ChildrenCP-5 sec shows, some of the Walking and running data is wrongly predicted as Standing. In such a scenario, the image taken from the camera could assist the sensor data in proper activity recognition.

Response:

It seems we may have misunderstood each other, and we apologize for any confusion. In our study, we aim to further develop and validate the use of small wearable sensor technology to monitor activity in everyday life of children with cerebral palsy. In our study, the video recordings are used as a “gold standard” which the model is tested against. If we understand correctly, you're proposing the utilization of video recordings from the children to enhance the model's performance. While integrating multimodal features as you suggest could undoubtedly optimize the activity recognition model, we must consider the practical constraints within our clinical context. Incorporating 24/7 video recordings from children's everyday lives would present several challenges, including privacy concerns, ethical considerations, and increased burden on the patients and is therefore not feasible.

3. The authors have highlighted the lack of publicly available data for HAR in children and children with CP. Unless researchers share their data for academic and research purposes, this problem will remain as it is. Therefore, it is requested that the authors share their data with the research community after the journal accepts it. Please share the data sharing policy and some data samples with the Editorial Team before the publication of the papers.

Response:

We totally agree with the importance of data sharing in advancing research efforts. In response to this concern, we have developed a comprehensive data sharing plan, which we previously shared with the editorial staff and included in our response to reviewer 1 during the previous review round. Here is a recap of our data sharing plan:

Our data will be made publicly available upon acceptance/publication through a dedicated repository on Dataverse, accessible via the following link: https://doi.org/10.18710/EPCXCC. Additionally, we have ensured that our codebase is readily accessible for fellow researchers and interested parties. You can find our code, along with similar codes from our research team, on our GitHub repository: https://github.com/ntnu-ai-lab/harth-ml-experiments.

We are committed to fostering collaboration and contributing to the collective advancement of knowledge in this field. We appreciate your attention to this matter and your dedication to promoting open science principles. We will share data with the editorial office upon request from the editor.

4. How much do the Model NNTU-HAR-Children and NNTU-HAR-ChildrenCP differ from the NNTU-HAR method proposed by Roar M. Fenne?

Response:

Our models, NTNU-HAR-Children and NTNU-HAR-ChildrenCP, are developed using training data from children and adolescents, and are tailored to recognized activity patterns often seen in children. The master’s thesis by Roar M. Fenne tested how well a model trained with data from adults performed in recognizing activities in children. For more information about the adult model used in the study by Fenne, I recommend exploring the recent publication titled "A Machine Learning Classifier for Detection of Physical Activity Types and Postures During Free-Living" by Bach et al. 2022.

Other Minor comments:

1. In section 2.4.2 (Video Annotation), it is not clear how many annotators were there in total, and each video frame annotated by how many annotators, and what was the score inter-annotator agreement?

Response:

Thank you for noting this unclarity in section 2.4.2 regarding the video annotation process. We have considered your suggestion and updated our manuscript accordingly. On page 8, lines 161-165 now provide more clarity on the annotation process:

“After clarifying the activity definitions and discussions based on video examples four raters annotated each video independently. Inter-rater reliability of > 0.95 on this methodology has been reported in earlier studies using the same methods, activity definitions and overlapping raters with the present study (Bach et al., 2022; Ustad et al., 2023).”

Bach, K., Kongsvold, A., Bårdstu, H., Bardal, E. M., Kjærnli, H. S., Herland, S., Logacjov, A., & Mork, P. J. (2022). A Machine Learning Classifier for Detection of Physical Activity Types and Postures During Free-Living. Journal for the Measurement of Physical Behaviour, 1(aop), 1-8.

Ustad, A., Logacjov, A., Trollebø, S. Ø., Thingstad, P., Vereijken, B., Bach, K., & Maroni, N. S. (2023). Validation of an Activity Type Recognition Model Classifying Daily Physical Behavior in Older Adults: The HAR70+ Model. Sensors, 23(5), 2368.

2. The research has not concluded the optimal time window for the HAR in their conclusion section and results discussion section.

Response:

Thank you for pointing this out.

In our results discussion section, we acknowledge the importance of the time window size, and we conclude that it is dependent on the target activity. Specifically, we highlight that the two 1-second models outperform others across test groups and most activities, except for cycling, where 3- or 5-second windows show better performance. Page 18-19, lines 332-335 and 337-340. We would also like to refer to the section where we discuss the window settings and the preference of different time windows on page 21-22, lines 403-440.

We have revised our conclusion to provide a more precise summary of our findings. On page 25, lines 499-507 now state:

"Based on the highest overall accuracy, the NTNU HAR-ChildrenCP model with 1-second window would be recommended for all three test groups. However, the optimal window size and overlap depend on the target activity. For activities such as cycling, 3- or 5-second windows perform better. Considering the ability to predict both momentary activities and activities of extended duration, the 3-second window without overlap would be recommended for population measurements."

Reviewer #3:

The authors provided convincing answers to my concerns and consequently modified the article. There is only one concern for which extra information would be useful. Regarding robustness to sensors' misplacement, the authors explained that "Our research group have tested the consequence of sensor misplacement on model performance in previous studies using

the same methodology (unpublished). As long as the sensors are placed in the correct direction towards gravity minor deviations in placements does not affect the accuracy of the model." Even if unpublished yet, the results

Attachment Submitted filename: Response to Reviewers.docx

10.1371/journal.pone.0308853.r005
Decision Letter 2
Narayan Jyotindra Academic Editor
© 2024 Jyotindra Narayan
2024
Jyotindra Narayan
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submission Version2
24 Jun 2024

PONE-D-23-37915R2Validation of two novel human activity recognition models for typically developing children and children with Cerebral Palsy.PLOS ONE

Dear Dr. Tørring,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Despite the reviewers agreed about the significant revisions made by the authors, they are still not convinced on the novelty and technical contribution of the work. The authors should present the model file and include some results for deep learning-based experiments in the manuscript to improve the technical contributions of the work adequately. 

Please submit your revised manuscript by Aug 08 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Jyotindra Narayan

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Partly

Reviewer #3: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: No

Reviewer #3: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The manuscript lacks significant technical contribution except for noble datasets. However, there had been few publicly available HAR datasets ( but not with children and cerebral palsy). Several works have already been on HAR for different age groups and people with cerebral palsy. However, as per the author's claim, this is the first work on HAR in children with Cerebral Palsy.

The GitHub repository https://github.com/ntnu-ai-lab/harth-ml-experiments, provided by the user, contains well-documented code, except the saved model files. The author should publish the model file in the GitHub repo for the reproducibility of the result. The GitHub repository already contains the code for BILSTM and CNN on the same datasets. Therefore, the authors are requested to include the results of the deep-learning-based experiments (biLSTM, CNN) in the current manuscript to justify the technical contribution and the PLOS ONE journal's reputation. There are already a few existing resources (research papers, HAR datasets, GitHub public repository containing ML, Deep learning and other methods for HAR). There has been no significant improvement in technical contribution from the first submission except for the improvement in writing and clarity of the manuscript.

Some references (published articles, HAR datasets, public GitHub repositories) are provided below:

Existing related papers on the HAR:

1. D. Ravi, C. Wong, B. Lo and G. -Z. Yang, "Deep learning for human activity recognition: A resource efficient implementation on low-power devices," 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN), San Francisco, CA, USA, 2016, pp. 71-76, doi: 10.1109/BSN.2016.7516235. keywords: {Machine learning;Feature extraction;Spectrogram;Convolution;Time-frequency analysis;Data mining;Deep Learning;Low-Power Devices;HAR;ActiveMiles},

2. M. Mostafavizadeh, A. R. Sadri and M. Zekri, "Walking pattern classification in children with cerebral palsy: A wavelet network approach," The 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP 2012), Shiraz, Iran, 2012, pp. 243-249, doi: 10.1109/AISP.2012.6313752. keywords: {Classification algorithms;Acceleration;Force;Legged locomotion;Entropy;Feature extraction;Accelerometers;Cerebral Palsy;kinetic data;Accelerometer;Pattern Classification;Wavelet Network;Shannon entropy},

3. Csizmadia, G., Liszkai-Peres, K., Ferdinandy, B. et al. Human activity recognition of children with wearable devices using LightGBM machine learning. Sci Rep 12, 5472 (2022). https://doi.org/10.1038/s41598-022-09521-1

4. Taborri, J.; Scalona, E.; Palermo, E.; Rossi, S.; Cappa, P. Validation of Inter-Subject Training for Hidden Markov Models Applied to Gait Phase Detection in Children with Cerebral Palsy. Sensors 2015, 15, 24514-24529. https://doi.org/10.3390/s150924514

5. J. Kamruzzaman and R. K. Begg, "Support Vector Machines and Other Pattern Recognition Approaches to the Diagnosis of Cerebral Palsy Gait," in IEEE Transactions on Biomedical Engineering, vol. 53, no. 12, pp. 2479-2490, Dec. 2006, doi: 10.1109/TBME.2006.883697.

6. Pengxi Fu, Jianxin Guo, Hongxiang Luo, LightGBM for Human Activity Recognition Using Wearable Sensors. Automation and Machine Learning (2024) Vol. 5: 113-118. DOI: http://dx.doi.org/10.23977/autml.2024.050114

7. Malekzadeh, M., Clegg, R., Cavallaro, A., & Haddadi, H. (2021). Dana: Dimension-adaptive neural architecture for multivariate sensor data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(3), 1-27.

8. Phyo, C. N., Zin, T. T., & Tin, P. (2019). Deep learning for recognizing human activities using motions of skeletal joints. IEEE Transactions on Consumer Electronics, 65(2), 243-252.

HAR datasets:

1. UCI-HAR : https://archive.ics.uci.edu/dataset/344/heterogeneity+activity+recognition - containing 30 users performing 6 activities. Accelerometer and gyroscope data were collected by a smartphone worn on the waist

2. UTwente: https://www.mdpi.com/1424-8220/16/4/426 includes data of 10 users performing 13 activities using accelerometer, gyroscope, and magnetometer data collected from the device on the (right) wrist.

3. MobiAct: https://www.scitepress.org/papers/2016/57924/57924.pdf

4. MotionSense: https://arxiv.org/abs/1802.07802

Public Github repository on HAR:

1. https://github.com/mmalekzadeh/motion-sense/tree/master

2. https://github.com/mmalekzadeh/dana

3. https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition

4. https://github.com/aqibsaeed/Human-Activity-Recognition-using-CNN

Reviewer #3: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: Yes: Dr. Durgesh Kumar

Reviewer #3: Yes: Armel Crétual

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

10.1371/journal.pone.0308853.r006
Author response to Decision Letter 2
Submission Version3
26 Jul 2024

Response to editor and reviewer

Our response is provided under the comments, page and line number refer to the manuscript file with track changes.

Dear Dr. Tørring,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Despite the reviewers agreed about the significant revisions made by the authors, they are still not convinced on the novelty and technical contribution of the work. The authors should present the model file and include some results for deep learning-based experiments in the manuscript to improve the technical contributions of the work adequately.

Please submit your revised manuscript by Aug 08 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Jyotindra Narayan

Academic Editor

PLOS ONE

Response to editor:

Dear editor, Jyotindra Narayan

Thank you for your understanding and for your considerate suggestion. We appreciate the time and effort you have dedicated to evaluating our manuscript and guidance in the current situation. We would like to refer to our email correspondence for further information about our position.

While we have decided not to engage in an extensive round of revisions that would alter the original scope of our study, we will proceed with making the minimal revisions as recommended.

Specifically, we will:

1. Clarify the novelty of our work, once again emphasizing the clinical relevance of our study focused on children with CP.

2. Provide the model file trained on all subjects to enhance reproducibility on GitHub.

We have also made changes to our manuscript to highlight these points.

Finally, we would once again emphasize that two of the reviewers have already accepted our manuscript. Therefore, we ask you to make an editorial decision based on all three reviewers’ remarks. Thank you once again for your guidance and support. We look forward to your final decision.

Best regards,

Marte Fossflaten Tørring

________________________________________

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The manuscript lacks significant technical contribution except for noble datasets. However, there had been few publicly available HAR datasets ( but not with children and cerebral palsy). Several works have already been on HAR for different age groups and people with cerebral palsy. However, as per the author's claim, this is the first work on HAR in children with Cerebral Palsy.

Response:

We appreciate your recognition of the unique dataset we have presented.

As we mentioned in our previous response, we see our novelty not in technical innovation but in the utilization of a technical methods in real-world scenarios and clinical settings. We would like to clarify a few points regarding the novelty and technical contribution of our work:

1. Not the First HAR Model for Children or Children with CP: While it is true that there have been previous HAR models for children and individuals with Cerebral Palsy, our work differentiates itself by achieving a level of accuracy that makes it practically employable in clinical studies. Our model demonstrates a significant improvement in performance, making it a reliable tool for clinical applications.

2. Addressing Childlike Behavior and Free Play: Our study is the first to specifically address the nuances of childlike behavior and free play in children with Cerebral Palsy. This aspect is critical, as it aligns the HAR model more closely with real-world scenarios and clinical needs. The listed machine learning models and HAR experiments cited by the reviewer do not appear to incorporate this important dimension, which is essential for developing clinically relevant tools.

We believe that these contributions significantly enhance the technical value of our work and its potential impact in the field of activity monitoring for children with Cerebral Palsy.

In our introduction on page 4, lines 79-86, we have presented existing HAR models for children with CP and explained why our HAR model is still needed. This is further emphasized in our aim on page 5, lines 99-101.

Reviewer #2:

The GitHub repository https://github.com/ntnu-ai-lab/harth-ml-experiments, provided by the user, contains well-documented code, except the saved model files. The author should publish the model file in the GitHub repo for the reproducibility of the result. The GitHub repository already contains the code for BILSTM and CNN on the same datasets. Therefore, the authors are requested to include the results of the deep-learning-based experiments (biLSTM, CNN) in the current manuscript to justify the technical contribution and the PLOS ONE journal's reputation. There are already a few existing resources (research papers, HAR datasets, GitHub public repository containing ML, Deep learning and other methods for HAR). There has been no significant improvement in technical contribution from the first submission except for the improvement in writing and clarity of the manuscript.

Response:

We appreciate your suggestion to enhance the reproducibility of our results. To this end, we will publish the model file in our GitHub repository (https://github.com/ntnu-ai-lab/harth-ml-experiments). We have made changes to our manuscript on page 10, line 211.

Regarding the additional deep-learning-based experiments, we would like to refer to our previous response in not altering our scope and not going beyond our aim of a clinically relevant tool. Thereby we would like to clarify a few points:

1. GitHub Repository Content: The existing code in our GitHub repository does not include implementations of biLSTM and CNN models for the specific dataset used in our current study. The code available is for different datasets, and applying these models to our dataset would require extensive additional work, including data preprocessing, hyperparameter tuning, and prolonged training periods.

2. Scope of Our Study: As we have emphasized in previous responses, our study's primary goal is to address a specific clinical need using established methods. Expanding the experiments to include deep learning models like biLSTM and CNN would significantly alter the scope of our research and deviate from our original objectives. Our focus has been on creating a practical, clinically relevant tool, rather than exploring a wide array of technical methodologies.

3. Technical Contribution: We believe that our contributions lie in the successful application of technical methods to real-world clinical settings, achieving a level of accuracy that makes our model practically employable. Additionally, our work addresses childlike behavior and free play in children with Cerebral Palsy, a critical aspect that has not been incorporated in existing models.

We hope this clarifies our position and the rationale behind our decisions not to do additional experiments. We are committed to making our work as reproducible and transparent as possible.

Attachment Submitted filename: Response to reviewers.docx

10.1371/journal.pone.0308853.r007
Decision Letter 3
Narayan Jyotindra Academic Editor
© 2024 Jyotindra Narayan
2024
Jyotindra Narayan
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submission Version3
1 Aug 2024

Validation of two novel human activity recognition models for typically developing children and children with Cerebral Palsy.

PONE-D-23-37915R3

Dear Dr. Tørring,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Jyotindra Narayan

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Following the successive revisions and authors' reponse to the reviewer concerns, the mansucript is now recommened for acceptance and publication. Congratulations to the authors fo the good work.

Reviewers' comments:

10.1371/journal.pone.0308853.r008
Acceptance letter
Narayan Jyotindra Academic Editor
© 2024 Jyotindra Narayan
2024
Jyotindra Narayan
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
11 Sep 2024

PONE-D-23-37915R3

PLOS ONE

Dear Dr. Tørring,

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team.

At this stage, our production department will prepare your paper for publication. This includes ensuring the following:

* All references, tables, and figures are properly cited

* All relevant supporting information is included in the manuscript submission,

* There are no issues that prevent the paper from being properly typeset

If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps.

Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Jyotindra Narayan

Academic Editor

PLOS ONE
==== Refs
References

1 Helmerhorst HHJ , Brage S , Warren J , Besson H , Ekelund U . A systematic review of reliability and objective criterion-related validity of physical activity questionnaires. International Journal of Behavioral Nutrition and Physical Activity. 2012;9 (1 ):1–55. doi: 10.1186/1479-5868-9-103 22233712
2 Lavelle G , Noorkoiv M , Theis N , Korff T , Kilbride C , Baltzopoulos V , et al . Validity of the international physical activity questionnaire short form (IPAQ-SF) as a measure of physical activity (PA) in young people with cerebral palsy: A cross-sectional study. Physiotherapy. 2020;107 :209–15. doi: 10.1016/j.physio.2019.08.013 32026822
3 Trost SG , O’Neil M . Clinical use of objective measures of physical activity. Br J Sports Med. 2014;48 (3 ):178–81. doi: 10.1136/bjsports-2013-093173 24311601
4 Matthews CE , Hagströmer M , Pober DM , Bowles HR . Best practices for using physical activity monitors in population-based research. Med Sci Sports Exerc. 2012;44 (1 Suppl 1 ):S68. doi: 10.1249/MSS.0b013e3182399e5b 22157777
5 O’Neil ME , Fragala-Pinkham M , Lennon N , George A , Forman J , Trost SG . Reliability and validity of objective measures of physical activity in youth with cerebral palsy who are ambulatory. Phys Ther. 2016;96 (1 ):37–45. doi: 10.2522/ptj.20140201 26089043
6 Sousa VE , Lopez KD . Towards usable e-Health. Appl Clin Inform. 2017;8 (02 ):470–90.28487932
7 Bach K , Kongsvold A , Bårdstu H , Bardal EM , Kjærnli HS , Herland S , et al . A Machine Learning Classifier for Detection of Physical Activity Types and Postures During Free-Living. Journal for the Measurement of Physical Behaviour. 2022;1 (aop ):1–8.
8 Ahmadi MN , Trost SG . Device-based measurement of physical activity in pre-schoolers: Comparison of machine learning and cut point methods. PLoS One. 2022;17 (4 ):e0266970. doi: 10.1371/journal.pone.0266970 35417492
9 Trost SG , Loprinzi PD , Moore R , Pfeiffer KA . Comparison of accelerometer cut points for predicting activity intensity in youth. Med Sci Sports Exerc. 2011;43 (7 ):1360–8. doi: 10.1249/MSS.0b013e318206476e 21131873
10 Lettink A , Altenburg TM , Arts J , van Hees VT , Chinapaw MJ . Systematic review of accelerometer-based methods for 24-h physical behavior assessment in young children (0–5 years old). International Journal of Behavioral Nutrition and Physical Activity. 2022;19 (1 ):1–63.34991593
11 Trost SG , Fragala-Pinkham M , Lennon N , O’Neil ME . Decision trees for detection of activity intensity in youth with cerebral palsy. Med Sci Sports Exerc. 2016;48 (5 ):958. doi: 10.1249/MSS.0000000000000842 26673127
12 Farrahi V , Niemelä M , Kangas M , Korpelainen R , Jämsä T . Calibration and validation of accelerometer-based activity monitors: A systematic review of machine-learning approaches. Gait Posture. 2019;68 :285–99. doi: 10.1016/j.gaitpost.2018.12.003 30579037
13 Logacjov A , Bach K , Kongsvold A , Bårdstu HB , Mork PJ . HARTH: A Human Activity Recognition Dataset for Machine Learning. Sensors. 2021;21 (23 ):7853. doi: 10.3390/s21237853 34883863
14 Ustad A , Logacjov A , Trollebø SØ , Thingstad P , Vereijken B , Bach K , et al . Validation of an Activity Type Recognition Model Classifying Daily Physical Behavior in Older Adults: The HAR70+ Model. Sensors. 2023;23 (5 ):2368. doi: 10.3390/s23052368 36904574
15 Narayanan A , Stewart T , Mackay L . A Dual-Accelerometer System for Detecting Human Movement in a Free-living Environment. Med Sci Sports Exerc. 2020;52 (1 ):252–8. doi: 10.1249/MSS.0000000000002107 31361712
16 Stewart T , Narayanan A , Hedayatrad L , Neville J , Mackay L , Duncan S . A Dual-Accelerometer System for Classifying Physical Activity in Children and Adults. Med Sci Sports Exerc. 2018;50 (12 ):2595–602. doi: 10.1249/MSS.0000000000001717 30048411
17 Hagenbuchner M , Cliff DP , Trost SG , Van Tuc N , Peoples GE . Prediction of activity type in preschool children using machine learning techniques. J Sci Med Sport. 2015;18 (4 ):426–31. doi: 10.1016/j.jsams.2014.06.003 25088983
18 Ahmadi MN , Pavey TG , Trost SG . Machine learning models for classifying physical activity in free-living preschool children. Sensors. 2020;20 (16 ):4364. doi: 10.3390/s20164364 32764316
19 Ahmadi MN , Brookes D , Chowdhury A , Pavey T , Trost SG . Free-living evaluation of laboratory-based activity classifiers in preschoolers. Med Sci Sports Exerc. 2020;52 (5 ):1227–34. doi: 10.1249/MSS.0000000000002221 31764460
20 Aadland E , Andersen LB , Anderssen SA , Resaland GK , Kvalheim OM . Accelerometer epoch setting is decisive for associations between physical activity and metabolic health in children. J Sports Sci. 2020;38 (3 ):256–63. doi: 10.1080/02640414.2019.1693320 31735120
21 Banos O , Galvez J-M , Damas M , Pomares H , Rojas I . Window size impact in human activity recognition. Sensors. 2014;14 (4 ):6474–99. doi: 10.3390/s140406474 24721766
22 Chen K , Zhang D , Yao L , Guo B , Yu Z , Liu Y . Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities. ACM Computing Surveys (CSUR). 2021;54 (4 ):1–40.
23 Skovdahl P , Olofsson CK , Arvidsson D . Physical activity in children and adolescents with CHD: review from a measurement methodological perspective. Cardiol Young. 2021;31 (4 ):518–31. doi: 10.1017/S1047951121000627 33722326
24 Ahmadi M , O’Neil M , Fragala-Pinkham M , Lennon N , Trost S . Machine learning algorithms for activity recognition in ambulant children and adolescents with cerebral palsy. J Neuroeng Rehabil. 2018;15 (1 ):1–9.29298708
25 Koman LA , Smith BP , SJ S . Cerebral Palsy Lancet. 2004;363 :1619–31.15145637
26 McIntyre S , Goldsmith S , Webb A , Ehlinger V , Hollung SJ , McConnell K , et al . Global prevalence of cerebral palsy: A systematic analysis. Dev Med Child Neurol. 2022;64 (12 ):1494–506. doi: 10.1111/dmcn.15346 35952356
27 Andersen GL , Irgens LM , Haagaas I , Skranes JS , Meberg AE , Vik T . Cerebral palsy in Norway: prevalence, subtypes and severity. Eur J Paediatr Neurol. 2008;12 (1 ):4–13. doi: 10.1016/j.ejpn.2007.05.001 17574886
28 Smith B Koman L Andrew , Smith Beth Paterson , Shilt Jeffrey S . THE LANCET. 2004;363 :1619–31.
29 Omura J , Fuentes M , Bjornson K . Participation in daily life: influence on quality of life in ambulatory children with cerebral palsy. PM&R. 2018;10 (11 ):1185–91. doi: 10.1016/j.pmrj.2018.05.010 29783065
30 Dallmeijer AJ , Brehm M-A . Physical strain of comfortable walking in children with mild cerebral palsy. Disabil Rehabil. 2011;33 (15–16 ):1351–7. doi: 10.3109/09638288.2010.531374 21073360
31 Saether R , Helbostad JL , Adde L , Brændvik S , Lydersen S , Vik T . Gait characteristics in children and adolescents with cerebral palsy assessed with a trunk-worn accelerometer. Res Dev Disabil. 2014;35 (7 ):1773–81. doi: 10.1016/j.ridd.2014.02.011 24679701
32 Graham HK , Selber P . Musculoskeletal aspects of cerebral palsy. The Journal of bone and joint surgery British volume. 2003;85 (2 ):157–66. doi: 10.1302/0301-620x.85b2.14066 12678344
33 Goodlich BI , Armstrong EL , Horan SA , Baque E , Carty CP , Ahmadi MN , et al . Machine learning to quantify habitual physical activity in children with cerebral palsy. Developmental Medicine &Child Neurology. 2020;62 (9 ):1054–60.32420632
34 Ahmadi MN , O’Neil ME , Baque E , Boyd RN , Trost SG . Machine learning to quantify physical activity in children with cerebral palsy: Comparison of group, group-personalized, and fully-personalized activity classification models. Sensors. 2020;20 (14 ):39–76. doi: 10.3390/s20143976 32708963
35 Rast FM, Jucker F, Labruyere R. Accuracy of sensor-based classification of clinically relevant motor activities in daily life of children with mobility impairments. medRxiv. 2022.
36 Thevenon A , Gabrielli F , Lepvrier J , Faupin A , Allart E , Tiffreau V , et al . Collection of normative data for spatial and temporal gait parameters in a sample of French children aged between 6 and 12. Ann Phys Rehabil Med. 2015;58 (3 ):139–44. doi: 10.1016/j.rehab.2015.04.001 25952820
37 Migueles JH , Cadenas-Sanchez C , Ekelund U , Delisle Nyström C , Mora-Gonzalez J , Löf M , et al . Accelerometer data collection and processing criteria to assess physical activity and other outcomes: a systematic review and practical considerations. Sports Med. 2017;47 :1821–45. doi: 10.1007/s40279-017-0716-0 28303543
38 Ferrari A , Micucci D , Mobilio M , Napoletano P . Trends in human activity recognition using smartphones. Journal of Reliable Intelligent Environments. 2021;7 (3 ):189–213.
39 Dehghani A , Sarbishei O , Glatard T , Shihab E . A quantitative comparison of overlapping and non-overlapping sliding windows for human activity recognition using inertial sensors. Sensors. 2019;19 (22 ):5026. doi: 10.3390/s19225026 31752158
40 Palisano R , Rosenbaum P , Walter S , Russell D , Wood E , Galuppi B . Development and reliability of a system to classify gross motor function in children with cerebral palsy. Dev Med Child Neurol. 1997;39 (4 ):214–23. doi: 10.1111/j.1469-8749.1997.tb07414.x 9183258
41 Kipp M. ANVIL: A Universal Video Research Tool.. In: Durands J , Gut U ., Kristofferson G ., editor. Handbook of Corpus Phonology Oxford, UK: Oxford University Press 2014. p. 420–36.
42 Chen T, Guestrin C, editors. Xgboost: A scalable tree boosting system. Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining; 2016.
43 Friedman JH . Greedy function approximation: a gradient boosting machine. Annals of statistics. 2001:1189–232.
44 Gholamiangonabadi D , Kiselov N , Grolinger K . Deep neural networks for human activity recognition with wearable sensors: Leave-one-subject-out cross-validation for model selection. Ieee Access. 2020;8 :133982–94.
45 Böhm H , Döderlein L . Gait asymmetries in children with cerebral palsy: do they deteriorate with running? Gait Posture. 2012;35 (2 ):322–7. doi: 10.1016/j.gaitpost.2011.10.003 22055251
46 Aadland E , Andersen LB , Anderssen SA , Resaland GK , Kvalheim OM . Associations of volumes and patterns of physical activity with metabolic health in children: A multivariate pattern analysis approach. Prev Med. 2018;115 :12–8. doi: 10.1016/j.ypmed.2018.08.001 30081134
