A Computerized Functional Skills Assessment and Training Program Targeting Technology Based Everyday Functional Skills

Philip D. Harvey; Lize Tibiriçá; Peter Kallestrup; Sara J. Czaja

doi:10.3791/60330

Medicine

A Computerized Functional Skills Assessment and Training Program Targeting Technology Based Everyday Functional Skills

Published: February 13, 2020 doi: 10.3791/60330

Philip D. Harvey^1,2, Lize Tibiriçá^2,3, Peter Kallestrup², Sara J. Czaja^1,2,4

¹University of Miami Miller School of Medicine, ²i-Function, ³Albizu University, ⁴Weill-Cornell Medical Center

Summary

This training protocol uses computerized training to teach technology-related everyday functional skills. These skills include financial skills, travel and transit, as well as medication management.

Abstract

Today, many functional skills are technology-based, so development of a technology-based training program has broad importance. Here we present a computerized functional skills training program that was paired in half of the participants with a commercially available cognitive training (CCT) program.

Non-impaired older individuals (NC) aged 60+ (n=45) and similarly aged individuals with mild cognitive impairment (MCI; n=50) were randomized to receive 12 weeks of twice-weekly computerized functional skills training (CFST) or 12 weeks of twice-weekly sessions split between CCT and CFST. Skills trained were use of an ATM; internet banking; ticket kiosk; telephone and internet prescription refill; medication management; and internet shopping. As with previous functional capacity assessments, we focus on completion time for each simulation.

51 participants completed the training program, either by mastering all 6 tasks (34) or completing 12 weeks of training. 44 more participants completed 4 or more training sessions so they were also analyzed for improvement up to their last training session. Completion time for all 6 tests significantly improved from the baseline assessment to the final training session in both groups of participants (all p<0.001 with an average improvement in task completion time of 45%). Further, there was no differential improvement in MCI and NC in the 6 tests from baseline to end of training (all t<1.66, all p>0.12). Finally, combined CCT plus CFST did not differ from CSFT alone on any of the percent-change score measures (all t<1.64, all p>0.11).

Both NC and MCI groups evidenced substantial improvements in performance. CCT supplementation led to similar functional gains with half as many training sessions. The NC participants proceeded through the training fairly rapidly even without CCT supplementation; MCI participants required more training but learned equivalently. These findings suggest that even in cases with memory impairments, functional skills can be efficiently learned with training.

Introduction

Many contemporary functional tasks are performed using technology. This includes banking and other financial management tasks, travel and transit tasks, and healthcare management. The challenges of using technology daily are amplified in older people whose lifetime exposure to technology may be more limited. Technology-based tasks can also be cognitively demanding. Older people and people with different challenges, such as severe mental illness¹^,² or cognitive deficits, may not have the financial or cognitive resources to use challenging technologies. These individuals have difficulty negotiating today's digitized world, which is a threat to their independence.

Our previous research has shown that many healthy older people also have problems in performing everyday functional tasks efficiently³. Further, people with mild cognitive impairment have proportionately greater challenges performing these tasks⁴. Our research has shown that component cognitive abilities in healthy aging⁴, severe mental illness⁵ and MCI populations, is consistently correlated with the ability to perform veridical simulations of everyday functional tasks. Thus, cognitive ability is a rate limiter for people's ability to initially perform, and learn, functional tasks using technology. This problem is exacerbated by the fact that many of these tasks, such as using an ATM, are thought of as "walk up tasks" or tasks that are intuitive and do not require training.

Currently, training in the performance of technology-relevant everyday functional is not systematically delivered. New technology, such as iPhones, tablets, and computers, are commonly delivered without any instructions for their use. Instructions for using websites are not commonly available. For example, the New York Metropolitan Transit Authority (MTA), website has no instructions on using their ticket kiosks other than instructions on how to refill a Metrocard.

The incremental deficits in cognitive performance can be partially addressed with computerized cognitive training (CCT) for some populations. Recent data have suggested that both healthy older people⁶ and people with MCI⁷ can make cognitive gains with CCT on the cognitive abilities targeted by the training. Thus, it would be expected that performance of functional tasks would also be facilitated with CCT efforts. However, the widely reported success of CCT in both older adult and MCI populations has not been accompanied by spontaneous improvements in the ability to perform novel functional skills. While previously acquired skills, such as driving⁸, can be facilitated through CCT, there is no evidence across populations that CCT alone is enough to lead to acquisition of novel functional skills.

CCT has also been shown to have protective effects against development of dementia, at least with a limited set of training routines. For instance, the ACTIVE trial showed that computerized speed training was associated with persistent improvements in cognitive performance that could be detected at a 10-year follow-up⁹. A subsequent follow-up study also reported a 30% decrease in dementia associated with all causes at the 10-year period as well¹⁰. As a result, since certain types of CCT appear to result in cognitive benefits among older people, the combination of CCT and computerized functional skills training should result in improvements in functional skills.

Thus, the current program involved the development of a set of ecologically valid functional skills tasks that are typically performed using some sort of technology, either internet, telephone, or in-person on a device such as an ATM. The tasks are presented in Table 1 and were chosen as important to living independently. Within the program, these tasks are performed in fixed difficulty, non-training formats first. Each task has multiple different graduated sub-tasks, which vary in difficulty of functional demands. After completion of the 6 fixed difficulty tasks, all participants are then trained on the Computerized Functional Skills Training (CFST) training simulations. These simulations provide direct performance feedback to the participant. Sample feedback is presented in Table 2. There is no involvement of a human trainer, nor does a human provide feedback. When a participant makes an error on a subtask, graduated feedback is provided across that increases in corrective information. For example, in the ATM task, if an individual makes an initial error inputting their pin number, basic corrective information is provided; if they make the same error a second time more corrective information is provided.

After 4 errors, the task proceeds to the next training step. However, when the participant returns to train later, this step is re-trained until it is passed. Each of the training modules is considered complete and the participant graduates after performing the entire task two times without any errors.

The study includes two research participant groups: (1) cognitively normal (CN) healthy older and (2) medically healthy older people with mild cognitive impairment (MCI). CN was defined as a Montreal Cognitive Assessment (MOCA)¹¹ score of 26 or more and no cognitive complaints. MCI was defined with a systematic assessment that included the MOCA, assessments of subjective complaints, and assessment with a structured neuropsychological assessments. Participants were excluded if their cognitive performance reflected impairment greater than MCI. Training was conducted on Windows computers although the software can be deployed in iOS as well. Training was proctored in a ratio of approximately 6 trainees per proctor.

The goal of the study is to determine 1) if CFST is efficacious in healthy older people, defined by improvements in performance of computer-based functional skills; (2) the relative efficacy of functional skills training for those with MCI as compared to those who are non-impaired; and (3) whether the provision of CCT enhances the CFST and if there is a differential effect for the MCI as compared to the NC.

Protocol

The study is a randomized trial, where half of the research participants (stratified by cognitive status) are randomized 1 to 1 to receive computerized cognitive training (CCT) on the Double-Decision task from the Posit Science Brain HQ training software combined with the CFST and the others are trained on the CFST alone. This research was reviewed and approved by the University of Miami Institutional Review Board and all participants provided signed informed consent.

1. Preparation

Obtain written informed consent.
Screen the participant with the Montreal Cognitive Assessment (MOCA)¹¹.
1. Have the participant connect numbers and letters in sequence.
2. Show the participant 3 pictures of animals and have them name them.
3. Read 3 verbal learning words and have participant recall them.
Perform a baseline cognitive assessment using the tablet version of the Brief Assessment of Cognition (Figure 1)¹². Administer all of the subtests on the tablet app. The subtests are Verbal Learning and Memory, Digit Sequencing, Token Motor Task, Symbol Coding, Verbal Fluency Examinations, and Tower of London.
1. Present the overall instructions for the assessment on the app.
2. Present the instructions for the Symbol Coding Task.
3. Have the participant do practice coding.
4. Watch the participant perform the coding task for 15 seconds.
5. Present the instructions for the Tower of London Task.
6. Have the participant do the practice items.
7. Watch the participant solve the first test item.
Assess the participant with 6 different functional tasks (Figure 2) in a fixed difficulty format.
NOTE: During the fixed difficulty assessment, there is no training provided, and if a participant makes an error, the instructions are simply repeated. When a participant completes a task, they proceed to the next one. When all of the tasks are completed, training begins. The representative task demands of each task are presented below.

2. Task demands for fixed difficulty and training simulations

Launch the Ticket Purchase Task.
1. Select purchase a new ticket.
2. Select Purchase a single ride ticket.
3. Check the balance on the transit card.
4. Add $60.00 to the transit card.
Launch the telephone refill task.
1. Dial the number for the pharmacy.
2. Enter the prescription number.
3. Select the time to pick up the medication.
Launch the ATM banking task.
1. Enter the PIN to start the session.
2. Check the balance in the checking account.
3. Withdraw $180.00 from the checking account.
Launch the medication label comprehension task.
1. Choose the correct time of day to take the medication.
2. Correctly identify how many pills to take each day.
3. Launch the medication organizer subtask.
4. Pack up a day's worth of medications.
Launch the internet banking task.
1. Enter the User ID and password.
2. Check the balance in the checking account.
3. Make a transfer of $15.00 from savings to checking.
Launch the internet prescription refill and online shopping task.
1. Enter the userID and password.
2. Verify your identity by selecting the correct city (Miami), Street (Micapony) and Car color (Blue).
3. Select Prinivil and no generic equivalent.
4. Select Correct date and time for pickup.

3. Training after the fixed difficulty assessment

CFST alone condition
1. Explain the training task to the participant, stating that all 6 tasks will be trained.
2. Launch the ATM banking task.
3. Enter PIN.
4. Select transaction (check balance in checking).
5. Visualize the process of program-delivered feedback for errors.
6. Present 4 errors in sequence to demonstrate the feedback process.
CCT + CFST Condition
NOTE: The CFST training is the same as CFST alone but is for 30 minutes only. Brain HQ DoubleDecision is the computerized cognitive training task in the combined condition. Figure 3 depicts the Double-Decision Task.
1. Explain the process of Training on Computerized cognitive training as well as skills training.
2. Launch Brain HQ double Decision Task.
3. Perform the practice items.
4. Perform the actual test items.
5. Visualize this process for about 30 seconds.

4. Post-training assessment

NOTE: After graduation from all six training tasks or completion of 24 sessions, participants complete a post-test. This next section should be presented as a verbal description of the post training assessments

Have participants perform different versions of all six trained tasks. They will be told that the tasks are the same, but the content demands are different.
Have participants perform the BAC App assessment again.

Representative Results

The patient flow is shown in Table 3. The first subject was screened on July 14, 2018. Screened participants included 78 females and 76 males, of whom 53 (33%) were Latino and 52 (33%) are of African American descent. Mean levels of educational attainment were similar across ethnic groups and averaged 15 years. However, there were cases who were training who had less than 8 years of education. 131 cases completed the eligibility assessment, with 16 cases excluded because of possible dementia and 4 cases excluded because of motor or vision problems. Of those 121 randomized 46% (n=56) were HC and 54% (n=65) were diagnosed at MCI and 50% in each group was randomized to CFST training alone. 51 cases completed training with 34 of them graduating, which translates into perfect performance twice in sequence on 30 individual subtasks. 44 cases are still training and 15 cases are waiting to train. The drop-out rate after any training was 9%. The mean MOCA score was 28.38 (SD=1.70) for the NC group and 22.68 (SD=3.02) for the MCI group.

Figure 4 presents the training results for completers and Figure 5 presents the results for all participants who trained. These data are presented in terms of time to completion, although several other dependent variables are collected. Paired t-tests found that completion time for all 6 tests significantly improved from the baseline assessment to the final training assessment in completers (all t>8.16, all p<.001). Further, none of the simulations improved differentially in the NC and MCI samples, as indexed by percentage of improvement from baseline to end of training, all t<1.66, all p>.12. Finally, combined CCT plus CFSAT did not differ from CSFAT alone on any of the %-change score measures: All t<1.44, all p>.16. CCT supplementation led to similar CFSAT gains with half as many CFSAT training sessions and this improvement was consistent in the MCI and NC groups in both conditions. All t-tests for the six tests, across the MCI vs. NC samples and the skills training only vs. combined assessments are in Table 4. Both groups evidenced substantial improvements in performance on alternative versions of the assessment tasks. Importantly, NC participants who received skills training alone required an average of only 6 trainings per module to perfect their performance. The participants randomized to CCT & CFSAT trained an average number of 11 days on CCT, achieving an average of 50 levels per participant.

Figure 1: Representative stimuli from the BAC App showing the tower of London and Symbol coding subtests. Please click here to view a larger version of this figure.

Figure 2: The 6 CFST training tasks. These tests include Ticket Purchase, ATM and Internet Banking, Phone and Internet Prescription Refill and Shopping, and Medication Management Please click here to view a larger version of this figure.

Figure 3: Brain HQ Double Decision Task Stimulus. Please click here to view a larger version of this figure.

Figure 4: Training gains from baseline in individuals who completed training. This is depicted in terms of time to completion at the first assessment, the final training session, and the alternate form of the simulations, across the 6 training tasks. Please click here to view a larger version of this figure.

Figure 5: Percentage improvement per training session for NC and MCI completers and cases still training. This is depicted in terms of the proportion of the total baseline to last session training gains in time to completion per training session completed. Please click here to view a larger version of this figure.

ATM Banking

Internet Banking

Ticket Purchase with Kiosk

Medication Management and Daily Organizer

Interactive Telephone Voice Menu for Prescription Refill

Internet Prescription Refill and On-Line Shopping

Table 1: Computerized functional skills training (CFST) tasks

Error 1 Repeat Instruction:	Your Pin is 1234. Please enter your PIN
Error 2. Guidance:	Your PIN is 1234. Please use the keypad to enter your PIN
Error 3 Direction:	Your PIN is 1234. Enter 1, followed by 2, followed by 3, followed by 4
Error 4 Demonstration	The four keys are lit up in sequence and the participants is instructed to touch them when they light up.

Table 2: Error feedback.

Participant Flow in Clinical Trial (July 2018 to Present)
Screened and signed consent	154
Ineligible	20
Withdrew before baseline	4
Baseline not completed	9
Completed baseline assessment	121
Completed training	51	42%
Still training	44	36%
Waiting to train	15	12%
Dropped out after training	11	9%

Table 3: The CONSORT Diagram for the clinical trial.

ATM Banking		t	p
	HC VS MCI	0.98	0.33
	Skills only vs. Combined Training	0.86	0.4
Medication Management
	HC VS MCI	0.57	0.57
	Skills only vs Combined Training	0.91	0.37
On-Line Banking
	NC vs. MCI	1.66	0.12
	Skills only vs Combined Training	0.56	0.96
Prescription Refill
	NC vs. MCI	0.21	0.84
	Skills only vs. Combined Training	1.44	0.16
Ticket Purchase Task
	NC vs. MCI	1.25	0.22
	Skills only vs. Combined Training	0.25	0.81
Internet Prescription Refill and Shopping
	NC vs. MCI	1.55	0.19
	Skills Only vs. Combined Training	0.16	0.87

Table 4: Results of t-tests comparing MCI vs. NC training gains and Skills only vs. Combined Treatment.

Discussion

CSFT training led to substantial and rapid treatment gains with as few as 6 training sessions, with results applicable to both NC and MCI participants. Both participant groups evidenced substantial improvements in task performance. CCT supplementation led to similar CFST gains with half as many CFST training sessions. Importantly, NC participants who received skills training alone required an average of only 6 sessions per task (out of a possible 24) to perfect their performance. In summary: 1) both groups of participants demonstrated improvements in performance across all tasks; 2) the HC participants proceeded through the training fairly rapidly even without CCT supplementation; and 3) MCI participants required more training sessions but learned equivalently. These findings replicate our previous findings with older patients with schizophrenia and a separate sample of healthy controls.

Of primary importance is the improvement in training associated with computerized functional skills training in participants with MCI. These cases had substantial impairments in their episodic memory. However, they were still able to make substantial gains, proportionately equivalent to people with NC, across 6 different training simulations. Previous studies have shown dissociation of procedural and verbal memory learning in MCI and amnestic conditions¹³^,¹⁴. Thus, this study shows that functional skills can be learned fairly rapidly and efficiently, with few dropouts.

Supplementation of computerized skills training with CCT increased the efficiency of the skills training considerably, with gains doubled per unit training session compared to skills training alone. Thus, a combined intervention with CCT and CFST in individuals with MCI would likely have multiple benefits. First, dementia prevention may be facilitated by CCT. Skills training may also lead to increased independence or delay progressive functional changes in MCI. As a result, the potential benefits of combined training seem considerable and a major topic for further study with this protocol.

Later studies will focus on real-world functional gains. Demonstration of such real-world gains would solidify the benefits of this training protocol. Implementing the protocol was quite efficient and participants reported high levels of satisfaction with their gains. For example, 98% or more of the participants stated that they "definitely" would be more able to do each of the 6 tasks in the real world.

Disclosures

In the past year Dr. Harvey has received consulting fees or travel reimbursements from Alkermes, Boehringer Ingelheim, Intra-Cellular Therapies, Jazz Pharma, Minerva Pharma, Otsuka America, Roche Pharma, Sanofi Pharma, Sunovion Pharma, Takeda Pharma, and Teva. He receives royalties from the Brief Assessment of Cognition in Schizophrenia. He is Chief Scientific Officer for i-Function. Peter Kallestrup is CEO of i-Function. Lize Tibiriçá is a part time employee of i-Function. Sara Czaja is Chief Scientific officer of i-Function.

Acknowledgments

This research was funded by NIA grant number R43AG057238 to Peter Kallestrup.

Materials

Name	Company	Catalog Number	Comments
Bac App	Verasci, Inc.	N/A	Cognitive testing software
Computerized Functional Skills Assessment and Training Software	i-Function	N/A	Computerized Software

DOWNLOAD MATERIALS LIST

References

Berkowsky, R. W., Sharit, J., Czaja, S. J. Factors Predicting Decisions About Technology Adoption Among Older Adults. Innovations in Aging. 2 (1), (2018).
Harvey, P. D., Keefe, R. S. E. Technology, society, and mental illness: challenges and opportunities for assessment and treatment. Innovations in Clinical Neuroscience. 9, 47-50 (2012).
Taha, J., Czaja, S. J., Sharit, J., Morrow, D. G. Factors affecting usage of a personal health record (PHR) to manage health. Psychology and Aging. 28, 1124-1139 (2013).
Czaja, S. J., Loewenstein, D. A., Sabbag, S. A., Curiel, R. E., Crocco, E., Harvey, P. D. A Novel Method for Direct Assessment of Everyday Competence Among Older Adults. Journal of Alzheimers Disease. 57 (4), 1229-1238 (2017).
Czaja, S. J., Loewenstein, D. A., Lee, C. C., Fu, S. H., Harvey, P. D. Assessing functional performance using computer-based simulations of everyday activities. Schizophrenia Research. 183, 130-136 (2017).
Harvey, P. D., McGurk, S. R., Mahncke, H., Wykes, T. Controversies in computerized cognitive training. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. 3, 907-915 (2018).
Sherman, D. S., Mauser, J., Nuno, M., Sherzai, D. The Efficacy of Cognitive Intervention in Mild Cognitive Impairment (MCI): a Meta-Analysis of Outcomes on Neuropsychological Measures. Neuropsychology Review. 27, 440-484 (2017).
Edwards, J. D., Delahunt, P. B., Mahncke, H. W. Cognitive speed of processing training delays driving cessation. Journal of Gerontology. 64, 1262-1267 (2009).
Rebok, G. W., et al. Ten-year effects of the Advanced Cognitive Training for Independent and Vital Elderly cognitive training trial on cognition and everyday functioning in older adults. Journal of the American Geriatrics Society. 62, 16-24 (2014).
Edwards, J., et al. Speed of processing training results in lower risk of dementia. Alzheimers Dementia. 3, 603-611 (2017).
Nasreddine, Z. S., et al. The Montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society. 53, 695-699 (2005).
Atkins, A. S., et al. Validation of the tablet-administered Brief Assessment of Cognition (BAC App). Schizophrenia Research. 181, 100-106 (2017).
Gobel, E. W., et al. Implicit perceptual-motor skill learning in mild cognitive impairment and Parkinson's disease. Neuropsychology. 27 (3), 314-321 (2013).
Oudman, E., Nijboer, T. C., Postma, A., Wijnia, J. W., Van der Stigchel, S. Procedural Learning and Memory Rehabilitation in Korsakoff's Syndrome - a Review of the Literature. Neuropsychology Review. 25 (2), 134-148 (2015).

Medicine