Commercial smartwatches equipped with wearable sensors are increasingly being used in population studies. However, their utility is often constrained by their limited battery duration, memory capacity, and data quality. This report provides examples of cost-effective solutions to real-life technical challenges encountered during studies involving asthmatic children and elderly cardiac patients.
Wearable sensors, which are often embedded in commercial smartwatches, allow for continuous and non-invasive health measurements and exposure assessments in clinical studies. Nevertheless, the real-life application of these technologies in studies involving a large number of participants for a significant observation period may be hindered by several practical challenges.
In this study, we present a modified protocol from a previous intervention study for the mitigation of health effects from desert dust storms. The study involved two distinct population groups: asthmatic children aged 6-11 years and elderly patients with atrial fibrillation (AF). Both groups were equipped with a smartwatch for the assessment of physical activity (using a heart rate monitor, pedometer, and accelerometer) and location (using GPS signals to locate individuals in indoor “at home” or outdoor microenvironments). The participants were required to wear the smartwatch equipped with a data collection application on a daily basis, and data were transmitted via a wireless network to a centrally administered data collection platform for the near real-time assessment of compliance.
Over a period of 26 months, more than 250 children and 50 patients with AF participated in the aforementioned study. The main technical challenges identified included restricting access to standard smartwatch features, such as gaming, internet browser, camera, and audio recording applications, technical issues, such as loss of GPS signal, especially in indoor environments, and the internal smartwatch settings interfering with the data collection application.
The aim of this protocol is to demonstrate how the use of publicly available application lockers and device automation applications made it possible to address most of these challenges in a simple and cost-effective way. In addition, the inclusion of a Wi-Fi received signal strength indicator significantly improved indoor localization and largely minimized GPS signal misclassification. The implementation of these protocols during the roll-out of this intervention study in the spring of 2020 led to significantly improved results in terms of data completeness and data quality.
Digital health technology applications and wearable sensors enable non-invasive and cost-effective patient monitoring both in healthcare and home settings1. At the same time, the large amount of data collected and the availability of wearable-based analytic platforms enable the development of algorithms for automated health event prediction, prevention, and intervention for a wide range of acute and chronic diseases2. Commercially available wearable sensors, primarily used for fitness tracking, are also increasingly being used by medical professionals in public health research and represent a promising tool for multimodal and continuous data collection under real-life conditions3. More importantly, though, unbiased data collection from wearables sensors allows researchers to overcome the challenges of recall bias that characterize traditional data collection methods such as interviews and diaries4.
However, for the purposes of clinical trials or other population studies, data accuracy, reliability, and integrity are essential. In addition, the credibility of the collected data may also be affected by several other parameters, such as age-group applicability as well as the memory capacity and energy efficiency of the device5. Recent systematic reviews of laboratory and field-based studies with limited numbers of participants have generally confirmed the applicability of commercial smartwatches for activity, heart rate, seizure, and behavior monitoring, although the reviews have also demonstrated poor suitability for elderly users, as well as battery, memory, and data quality limitations6,7. These limitations may be further amplified in larger population studies under real-life conditions where additional parameters such as inconsistent internet connectivity, device comfort, and incorrect smartwatch use come into play8. Specifically, appearance and inconvenience are significant barriers to wearing sensors daily9, while concerns relating to privacy and confidentiality issues may affect recruitment in studies involving wearable sensors10. Concerning the applicability of commercial smartwatches and fitness trackers for measuring physical activity in research studies, a recent study by Henriksen et al. suggested that the selection of an appropriate device for a particular study should not only be based on the available embedded sensors but rather also take into account validation and previous use in research, appearance, battery life, robustness, water resistance, connectivity, and usability11.
For the purposes of this study, we present a protocol to improve on the challenges encountered during the LIFE MEDEA project, an intervention study for the mitigation of the health effects of desert dust storms12. The study involved two distinct population groups: asthmatic children aged 6-11 years and elderly patients with atrial fibrillation (AF). Both groups were equipped with a commercial smartwatch for the assessment of physical activity (using a heart rate monitor, pedometer, and accelerometer) and location (using GPS signals to locate individuals in indoor "at home" or outdoor microenvironments). The participants were required to wear the smartwatch daily, and data were transmitted via a wireless network to a centrally administered data collection platform via the data collection application for the near real-time assessment of compliance. Additional details on the smartwatch and system setup are provided in a previous study13. During the first year of the project implementation, several technical and real-life challenges relating to the device emerged, which affected recruitment, the compliance of participants in wearing the device daily, and the completeness of the collected data. Some challenges were population-specific, such as the requirement of school administrators and many parents that the children wearing the smartwatches should not have access to standard smartwatch features, such as gaming, internet browser, camera, and audio-recording applications. Other challenges were technical in nature, such as loss of GPS signal, especially in indoor environments, and internal smartwatch settings interfering with the data collection application. A detailed overview of the main challenges identified as well as a brief description of their implications and solutions are presented in Table 1.
In this study, we suggest simple, cost-effective, and off-the-shelf solutions to improve user compliance, data quality, and data completeness in population studies employing wearable sensors and provide the relevant protocols. In addition, we demonstrate the data completeness improvements from the implementation of such protocols using representative results from the study13.
Administrative and ethics approvals were obtained from the Cyprus Ministry of Health (YY5.34.01.7.6E) and the Cyprus National Bioethics Committee (ΕΕΒΚ/ΕΠ/2016.01.23). Patients with atrial fibrillation and the guardians of the asthmatic children provided written informed consent prior to participation in the study.
1. Application lockers and device automation applications
NOTE: Freely available application lockers and device automation applications (taskers) can be found for both Android devices and IOS devices. The specific applications used in the present study are listed in the Table of Materials.
2. Development of the automated procedures using the tasker
NOTE: A tasker allows for the step-by-step development of automated processes. These can vary according to the requirements of the project. Previous coding or programming experience is not required. In the following steps, the following terms and definitions are used: trigger (a starting state that, when met, allows the tasker to initiate the process), condition (a condition that, when met, allows the process to continue to the next step), and action (the process outcome). In the provided figures, the parallelogram denotes a trigger, the diamond denotes a condition, and the rectangle denotes an action. Each process may result in more than one action, and these are labeled as actions a, b, c, (…) under each process. A separate process was set up for each individual problem identified during the field implementation of the project. This approach ensured there was no overlap between the conditions set and allowed the smooth operation of the automated process as a whole.
3. Exporting the created processes (steps 2.1-2.6)
4. Transferring and installing the files created to the smartwatch
5. Setting up the smartwatch for field use
The protocol describes simple and cost-effective solutions to real-life challenges affecting recruitment, compliance, and data quality in population studies employing wearable sensors. The steps described here allowed for the successful setup of a consumer wearable device for exposure and health monitoring in a large population study involving children with asthma and adults with atrial fibrillation. Figure 6 provides a graphical overview of the provided protocols and illustrates the key steps undertaken to address the main underlying issues identified.
Here, we present the representative results from a subset of 17 participants (asthmatic children aged 6-11 years old) who engaged in the LIFE MEDEA study in the spring of 202013. The 17 participants were equipped with a smartwatch that provided time-stamped data on physical activity (pedometer, accelerometer, heart rate) and GPS locations before and after the implementation of the protocol. These data were collected via the data collection application and were automatically synchronized with a cloud-based database when the smartwatch was in contact with the Wi-Fi network inside each participant's home, as described previously13. However, through the application of the described protocol, information on Wi-Fi connectivity, Wi-Fi signal strength, battery capacity, and whether the device was charging or not was also made available. The data on these additional variables were not automatically synchronized with the cloud-based database but had to be manually downloaded from each smartwatch via Bluetooth after the end of the study period. By comparing the data collected for a duration of 2 weeks before and 2 weeks after the protocol implementation, we evaluated the impact of these solutions in improving data completeness, defined as the percentage of time with collected data per day. Figure 7A presents the percentage of time with data before and after protocol implementation for each participant separately, while Figure 7B presents the corresponding distributions of the percentage of time with data for the whole group before and after the protocol implementation. Interestingly, the protocol implementation led to a statically significant increase in data completeness, with the percentage of time with data increasing from a median of 36.5% (min: 9.3%, max: 68.1%) to a median of 48.9% (IQR: 18.4%, 77.8%, p = 0.013).
Furthermore, in Figure 8, we present an extreme case of poor GPS data collected during 24 h from a single patient with AF participating in the study. Although the patient was wearing the watch as instructed, the actual raw GPS signal collected was scattered across the 24 h (Figure 8A), and the estimation of the duration of time indoors and the duration of time outdoors was difficult. The implementation of a GPS data filling algorithm (Supplementary Figure 1) allows for replacing missing data with estimated values (Figure 8B). Confirmation that the estimated time indoors and the estimated time outdoors were correct was provided by the logged smartwatch connectivity with the Wi-Fi network signal (Figure 8C). For the same patient, we also showcase another extreme case of a day with poor GPS data collected (Figure 9A). However, in this case, the implementation of the GPS data filling algorithm alone did not accurately estimate all the missing data. Characteristically, the algorithm correctly estimated that the participant was mostly out of their residence between approximately 09:00 to 21:00 that day and that they returned home for a brief period around 18:00, but it failed to capture that the participant also returned home for a period of about 90 min at approximately 13:30 (Figure 9B). Nevertheless, this event was not missed when the data on smartwatch connectivity with the Wi-Fi network signal were also considered (Figure 9C).
Finally, following successful piloting, the protocol was implemented across the full cohort of MEDEA participants during the spring of 2020 in both Cyprus and Greece (n = 108 asthmatic children). However, a few weeks after the smartwatches were distributed to the children and data collection began, the health authorities in Cyprus and Greece enforced a series of public health interventions of increasing intensity to control the COVID-19 pandemic in their respective countries. The public health interventions were initially characterized by social distancing measures and a ban on large public events but quickly escalated to strict national lockdowns during the months of March and April. Considering the unprecedented disturbances in the daily routine and behavior of the population, the decision was taken to continue tracking the location and activity of asthmatic children using the smartwatches during the duration of lockdowns to objectively quantify their compliance with the public health intervention measures and overall changes in physical activity. The data collected were used to calculate individual profiles of daily "fraction time spent at home" and "total steps/day" and were analyzed statistically to assess changes in these parameters over the escalating levels of the COVID-19 lockdown measures. The timeline and description of the escalating levels of the COVID-19 lockdown measures in the two countries are presented in Figure 10 and are described in detail by Kouis et al. in an earlier publication13. In summary, the results indicated a statistically significant mean increase in "fraction time spent at home" in both countries across the increasing intervention levels. The mean increase in "fraction time spent at home" was equal to 41.4% and 14.3% (at level 1), 48.7% and 23.1% (at level 2), and 45.2% and 32.0% (at level 3) for Cyprus and Greece, respectively. Physical activity in Cyprus and Greece demonstrated significant mean decreases of −2,531 and −1,191 steps/day (at level 1), −3,638 and −2,337 steps/day (at level 2), and −3,644 and −1,961 steps/day (at level 3)in Cyprus and Greece, respectively13. The weekly averages of "fraction time spent at home" and "total steps/day"in asthmatic children before COVID-19 and during the three levels of COVID-19 lockdown measures are displayed in Figure 1113.
DATA AVAILABILITY STATEMENT:
The anonymized dataset has been submitted to Figshare online open-access repository (https://doi.org/10.6084/m9.figshare.21601371.v3).
Table 1: Real-life challenges identified in relation to the use of smartwatch devices and the employed solutions. Please click here to download this Table.
Figure 1: Activating the data collection application. Schematic diagram of the process to systematically activate the data collection application. The parallelogram denotes a trigger, the diamond denotes a condition, and the rectangle denotes an action. Please click here to view a larger version of this figure.
Figure 2: Enabling Wi-Fi connectivity. Schematic diagram of the process to systematically enable Wi-Fi connectivity. The parallelogram denotes a trigger, the diamond denotes a condition, and the rectangle denotes an action. Please click here to view a larger version of this figure.
Figure 3: Optimizing battery consumption. Schematic diagram of the processes leading to actions that systematically optimize the battery consumption. The parallelogram denotes a trigger, the diamond denotes a condition, and the rectangle denotes an action. Please click here to view a larger version of this figure.
Figure 4: Logging event information. Schematic diagram of the processes that systematically log event information relevant to the project. The parallelogram denotes a trigger, the diamond denotes a condition, and the rectangle denotes an action. Please click here to view a larger version of this figure.
Figure 5: User notification if the GPS signal is disabled. Schematic diagram of the processes that systematically check the GPS signal status and provide notifications to alert users of problems. The parallelogram denotes a trigger, the diamond denotes a condition, and the rectangle denotes an action. Please click here to view a larger version of this figure.
Figure 6: Schematic overview of the protocols. Schematic overview of the underlying challenges identified and the provided protocols with an illustration of the key steps in the processes. Please click here to view a larger version of this figure.
Figure 7: Data completeness before and after the implementation of the protocol. Data completeness for a representative group of participants (n = 17) for a period of 2 weeks before and after implementation of the protocol. (A) The percentage of time with data before and after the protocol implementation for each participant separately. (B) The corresponding distributions for the whole group. Please click here to view a larger version of this figure.
Figure 8: Implementation of the GPS data filling algorithm (extreme case 1). (A) Example case of a day with poor raw GPS signal data and (B) the implementation of the GPS data filling algorithm to replace the missing data with estimated values. (C) The confirmation of indoor and outdoor classifications based on the Wi-Fi received signal indicator. Please click here to view a larger version of this figure.
Figure 9: Implementation of GPS data filling algorithm (extreme case 2). (A) Example case of a day with poor raw GPS signal data and (B) the implementation of the GPS data filling algorithm to replace the missing data with estimated values. (C) The GPS data filling algorithm led to some indoor and outdoor misclassification, which was resolved using the Wi-Fi received signal indicator. Please click here to view a larger version of this figure.
Figure 10: Timeline of public health interventions in Cyprus and Greece. Timeline of the study recordings in relation to the introduction of public health interventions in (A) Cyprus and (B) Greece during March and April 2020.The image is reproduced under license CC BY 4.0, without any changes, from the original study by Kouis et al., first published in Scientific Reports Journal13. Please click here to view a larger version of this figure.
Figure 11: Changes in mobility in response to public health interventions among asthmatic children. Weekly averages of the fraction time spent at home and steps/day of asthmatic children before and during the three levels of public health interventions in (A) Cyprus and (B) Greece. The image is reproduced under license CC BY 4.0, without any changes, from the original study by Kouis et al., first published in Scientific Reports Journal13. Please click here to view a larger version of this figure.
Supplementary Figure 1: The implementation of a GPS data filling algorithm. Please click here to download this File.
Supplementary File 1: The macros described in this protocol. Please click here to download this File.
Wearable sensors are useful tools that allow the continuous and non-invasive monitoring of health parameters and patient behavior. Commercial smartwatches, which are equipped with a variety of sensors, provide a promising alternative to traditional data collection methods, and their use in clinical and public health research is only expected to rise as a result of increased variety and quality of built-in sensors, stronger academic-industry partnerships, and reductions in retail prices14. In this study, we highlight real-life challenges that may affect recruitment, user compliance, and data quality in population studies and provide examples of simple and cost-effective solutions to overcome them in the field. The implementation of this protocol during the roll-out of the study13 led to significantly improved results in terms of data completeness and data quality. The most critical steps within the protocol are step 2.2 (which ensures the systematic activation of the data collection application at regular time intervals), step 2.5 (which provides a separate log of important events about the smartwatch status), and step 5.2.8 (which allows the uninterrupted operation of the smartwatch background processes).
In the past, several studies have addressed the validity of consumer wearable devices for a variety of health and activity endpoints, and the results were recently synthesized in a large systematic review and meta-analysis15. However, of the total 169 studies identified in the systematic review, only 48 involved populations in a free-living environment, while only 36 studies involved populations with any kind of mobility limitation or a chronic disease. Although the authors concluded that, overall, the commercial devices are accurate for measuring steps and heart rate, especially in laboratory-based settings, they highlighted the risk of overestimation or underestimation in free-living environments, while differences in the usability and validity of measurements between healthy controls and chronic patients were not explored15. Both points are particularly important as one of the main arguments for shifting to digital health is to enable the monitoring of patients with chronic diseases outside of healthcare settings16.
Nevertheless, some prior studies have focused on and quantified problems encountered by participants and researchers during the rollout of clinical studies in free-living environments involving consumer wearable devices17,18,19. In a well conducted feasibility study that involved a small number of participants (n = 26) but observed them for a significant time period (3 months), Beukenhorst et al. reported that, on average, patients wore the watch on 73% of the days and that temporary and permanent non-usage increased over the study weeks17.
In a much larger group, Galarnyk et al. reported that from a total 230 individuals recruited in the study and provided with a smartwatch, only 130 (57%) used it at least once and successfully transmitted some data18. Furthermore, some studies have also highlighted the fact that during the implementation phase, intensive technical support is required18,19. Characteristically, in the Parkinson@Home Study, the authors reported an 88% data completeness rate but also highlighted that almost all participants required at least one support call for device troubleshooting during the 3 month study period19. We reported a similar experience in our study13, although official records of troubleshooting calls and house visits were not kept.
In our study, we also focused on data quality issues related to GPS signals. We had to construct individual participant exposures in outdoor and indoor (at home) microenvironments, a task complicated by the frequent and persistent loss of signal, especially in indoor environments, and for this reason, we developed a data filling algorithm, as suggested in previous studies20,21. Although, the algorithm performed reasonably well, the inclusion of a Wi-Fi received signal strength indicator, as collected by the tasker application, significantly improved the performance of the algorithm and largely minimized misclassification. The utility of this Wi-Fi received signal strength indicator has also been demonstrated in previous studies focusing on indoor localisation22,23, and when coupled with GPS measurements, this indicator can provide a valid measurement of 24 h individual exposure to outdoor and indoor microenvironments.
Finally, the protocol suggested here was implemented and tested in real-life conditions in the spring of 2020 both in children and elderly individuals. Although each solution suggested is simple and does not require prior programming knowledge, all the solutions together addressed all the main issues identified, notably by improving and systematizing the data collection, reducing the battery consumption, blocking unwanted applications and smartwatch settings, and improving the GPS signal. However, the processes, as described in the protocol, were only tested with a smartwatch device using Android version 7.1.1. Although it is possible that the direct replication of these processes will be possible with other Android versions, we cannot exclude the possibility that some adjustments may be required, and as a result, the direct generalizability of the protocol may be limited. In addition, the protocol may have to be modified to reflect variations in the technical specifications of other smartphone devices. For example, the time trigger for data collection may be set according to the battery capacity of the smartwatch device or depending on the time resolution required for the collected variables. Nevertheless, even if the application of this protocol to a different smartphone device or a different Android version may require troubleshooting and the modification of some of the individual steps, overall, similar steps will have to be undertaken (or it will have to be confirmed that certain steps are not required) during the setup of any smartwatch prior to it being given to a study participant. The level of detail provided in the protocol allows for the easy adaptation of these solutions to any smartwatch device. Furthermore, this work did not aim and was not originally designed to assess the reasons that may affect user compliance with wearable devices during the performance of population studies. Future studies using appropriate tools and methodologies are required to further examine this topic. Such studies can provide the additional evidence needed to efficiently improve the existing methods of incorporating wearable devices into research studies, especially under real-life conditions.
Currently, the existing methods are quite limited and primarily include the development of an extended support system (initial training, a user manual, a helpline, and on-site visits)19. In addition, a previous study highlighted that in digitalized clinical trials, a significant dropout rate should be expected and a priori contingency plans, such as access to a wider participant recruitment pool, are required18. The incorporation of the solutions presented in this study can complement and, more critically, reduce the burden on the extended support system while increasing the data completeness and data quality. In addition, based on the observations by Galarnyk et al., making the initiation of device use as easy as possible may further ensure compliance and reduce dropout rates18. Finally, future applications of some of these solutions, especially the use of device automation applications, include the further customization of commercial devices to support mobility in the elderly or in persons with disabilities24,25, support early warning systems26, and ensure Bluetooth and Wi-Fi connectivity in Internet of Bodies (IoB) applications27.
In summary, this work provides a protocol that includes simple and cost-effective solutions to real-life challenges affecting recruitment, compliance, and data quality in population studies employing consumer wearable devices. The protocol relies on freely available software tools and does not require any previous programming knowledge. This approach can be easily replicated or adapted by health researchers working with wearable devices in the fields of clinical research and public health.
The authors have nothing to disclose.
The authors are grateful to all the participants and their families, as well as to the teaching and administrative personnel of the participating primary schools in Cyprus and Greece. The study was financed by the European Union LIFE Project MEDEA (LIFE16 CCA/CY/000041).
APK Extractor | Meher | Version 4.21.08 | Application |
Charger/Adaptor with data cable | Jiangsu Chenyang Electron Co. Ltd | C-P17 | Charger |
Embrace application | EmbraceTech LTD | Version 1.5.4 | Application |
LEMFO LF25 Smartwatch | Shenzhen domino Times Technology Co. Ltd | DM368 Plus | Smartwatch |
Lock App – Smart App Locker | ANUJ TENANI | Version 4.0 | Application |
Macrodroid-Device Automation | ArloSoft | Version 5.5.2 | Application |
Xiaomi Redmi 6A | Xiaomi | M1804C3CG | Smartphone |