Frame-by-Frame Video Analysis of Idiosyncratic Reach-to-Grasp Movements in Humans
Summary
This protocol describes how to use frame-by-frame video analysis to quantify idiosyncratic reach-to-grasp movements in humans. A comparative analysis of reaching in sighted versus unsighted healthy adults is used to demonstrate the technique, but the method can also be applied to the study of developmental and clinical populations.
Abstract
Prehension, the act of reaching to grasp an object, is central to the human experience. We use it to feed ourselves, groom ourselves, and manipulate objects and tools in our environment. Such behaviors are impaired by many sensorimotor disorders, yet our current understanding of their neural control is far from complete. Current technologies for investigating human reach-to-grasp movements often utilize motion tracking systems that can be expensive, require the attachment of markers or sensors to the hands, impede natural movement and sensory feedback, and provide kinematic output that can be difficult to interpret. While generally effective for studying the stereotypical reach-to-grasp movements of healthy sighted adults, many of these technologies face additional limitations when attempting to study the unpredictable and idiosyncratic reach-to-grasp movements of young infants, unsighted adults, and patients with neurological disorders. Thus, we present a novel, inexpensive, and highly reliable yet flexible protocol for quantifying the temporal and kinematic structure of idiosyncratic reach-to-grasp movements in humans. High speed video cameras capture multiple views of the reach-to-grasp movement. Frame-by-frame video analysis is then used to document the timing and magnitude of pre-defined behavioral events such as movement start, collection, maximum height, peak aperture, first contact, and final grasp. The temporal structure of the movement is reconstructed by documenting the relative frame number of each event while the kinematic structure of the hand is quantified using the ruler or measure function in photo editing software to calibrate 2 dimensional linear distances between two body parts or between a body part and the target. Frame-by-frame video analysis can provide a quantitative and comprehensive description of idiosyncratic reach-to-grasp movements and will enable researchers to expand their area of investigation to include a greater range of naturalistic prehensile behaviors, guided by a wider variety of sensory modalities, in both healthy and clinical populations.
Introduction
Prehension, the act of reaching to grasp an object, is used for many daily functions including acquiring food items for eating, grooming, manipulating objects, wielding tools, and communicating through gesture and written word. The most prominent theory concerning the neurobehavioral control of prehension, the Dual Visuomotor Channel theory1,2,3,4, proposes that prehension consists of two movements – a reach that transports the hand to the location of the target and a grasp that opens, shapes, and closes the hand to the size and shape of the target. The two movements are mediated by dissociable but interacting neural pathways from visual to motor cortex via the parietal lobe1,2,3,4. Behavioral support for the Dual Visuomotor Channel theory has been ambiguous, largely due to the fact that the reach-to-grasp movement appears as a single seamless act and unfolds with little conscious effort. Nonetheless, prehension is almost always studied in the context of visually-guided prehension in which a healthy participant reaches to grasp a visible target object. Under these circumstances the action does appear as a single movement that unfolds in a predictable and stereotypical fashion. Prior to reach onset the eyes fixate on the target. As the arm extends the digits open, preshape to the size of the object, and subsequently start to close. The eyes disengage from the target just prior to target contact and final grasp of the target follows almost immediately afterwards5. When vision is removed, however, the structure of the movement is fundamentally different. The movement dissociates into its constituent components such that an open-handed reach is first used to locate the target by touching it and then haptic cues associated with target contact guide shaping and closure of the hand to grasp6.
Quantification of the reach-to-grasp movement is most often achieved using a 3 dimensional (3D) motion tracking system. These can include infrared tracking systems, electromagnetic tracking systems, or video based tracking systems. While such systems are effective for acquiring kinematic measures of prehension in healthy adult participants performing stereotypical reach-to-grasp movements towards visible target objects, they do have a number of drawbacks. In addition to being very expensive, these systems require the attachment of sensors or markers onto the arm, hand, and digits of the participant. These are usually attached using medical tape, which can impede tactile feedback from the hand, alter natural motor behavior, and distract participants7. As these systems generally produce numerical output related to different kinematic variables such as acceleration, deceleration, and velocity they are also not ideal for investigating how the hand contacts the target. When using these systems, additional sensors or equipment are required to determine what part of the hand makes contact with the target, where on the target contact occurs, and how the configuration of the hand might change in order to manipulate the target. In addition, infrared tracking systems, which are the most commonly employed, require the use of a specialized camera to track the location of the markers on the hand in 3D space6. This requires a direct line of sight between the camera and the sensors on the hand. As such, any idiosyncrasies in the movement are likely to obscure this line of sight and result in the loss of critical kinematic data. There are, however, a large number of instances in which idiosyncrasies in the reach-to-grasp movement are actually the norm. These include during early development when infants are just learning to reach and grasp for objects; when the target object is not visible and tactile cues must be used to guide the reach and the grasp; when the target object is an odd shape or texture; and when the participant presents with any one of a variety of sensorimotor disorders such as a stroke, Huntington's disease, Parkinson's disease, Cerebral Palsy, etc. In all of these cases, the reach-to-grasp movement is neither predictable nor stereotypical, nor is it necessarily guided by vision. Consequently, the capability of 3D motion tracking systems to reliably quantify the temporal and kinematic structure of these movements can be severely limited due to disruptions in sensory feedback from the hand, changes in natural motor behavior, loss of data, and/or difficulties interpreting the idiosyncratic kinematic output from these devices.
The present paper describes a novel technique for quantifying idiosyncratic reach-to-grasp movements in various human populations that is affordable, does not impede sensory feedback from the hand or natural motor behavior, and is reliable but can be flexibly modified to suit a variety of experimental paradigms. The technique involves using multiple high-speed video cameras to record the reach-to-grasp movement from multiple angles. The video is then analyzed offline by progressing through the video frames one at a time and using visual inspection to document key behavioral events that, together, provide a quantified description of the temporal and kinematic organization of the reach-to-grasp movement. The present paper describes a comparative analysis of visually- versus nonvisually-guided reach-to-grasp movements in healthy human adults6,8,9,10 in order to demonstrate the efficacy of the technique; however, modified versions of the technique have also been used to quantify the reach-to-grasp actions of human infants11 and non-human primates12. The comprehensive results of the frame-by-frame video analysis from these studies are among the first to provide behavioral evidence in support of the Dual Visuomotor Channel theory of prehension.
Protocol
Representative Results
Discussion
The present paper describes how to use frame-by-frame video analysis to quantify the temporal organization, kinematic structure, and a subset of topographical features of human reach-to-grasp movements. The technique can be used to study typical visually-guided reach-to-grasp movements, but also idiosyncratic reach-to-grasp movements. Such movements are difficult to study using traditional 3D motion tracking systems, but are common in developing infants, participants with altered sensory processing, and patients with sen…
Disclosures
The authors have nothing to disclose.
Acknowledgements
The authors would like to thank Alexis M. Wilson and Marisa E. Bertoli for their assistance with filming and preparing the video for this manuscript. This research was supported by the Natural Sciences and Engineering Research Council of Canada (JMK, JRK, IQW), Alberta Innovates-Health Solutions (JMK), and the Canadian Institutes of Health Research (IQW).
Materials
| High Speed Video Cameras | Casio | http://www.casio-intl.com/asia-mea/en/dc/ex_f1/ or http://www.casio-intl.com/asia-mea/en/dc/ex_100/ | Casio EX-F1 High Speed Camera or Casio EX-100 High Speed Camera used to collect high speed video records |
| Adobe Photoshop | Adobe | http://www.adobe.com/ca/products/photoshop.html | Software used to calibrate and measure distances on individual video frames |
| Adobe Premiere Pro | Adobe | http://www.adobe.com/ca/products/premiere.html?sdid=KKQOM&mv=search&s_kwcid=AL!3085!3!193588412847!e!!g!!adobe%20premiere%20pro&ef_id=WDd17AAABAeTD6-D:20170606160204:s | Software used to perform Frame-by-Frame Video Analysis |
| Height-Adjustable Pedestal | Sanus | http://www.sanus.com/en_US/products/speaker-stands/htb3/ | A height adjustable speaker stand with a custom made 9 cm x 9 cm x 9 cm triangular top plate attached to the top with a screw is used as a reaching pedestal |
| 1 cm Calibration Cube | Learning Resources (Walmart) | https://www.walmart.com/ip/Learning-Resources-Centimeter-Cubes-Set-500/24886372 | A 1 cm plastic cube is used to transform distance measures from pixels to centimeters |
| Studio Light | Dot Line | https://www.bhphotovideo.com/c/product/1035910-REG/dot_line_rs_5620_1600w_led_light.html | Strong lamp with cool LED light used to illumate the participant and testing area |
| 3 Dimensional (3D) Sleep Mask | Kfine | https://www.amazon.com/Kfine-Sleeping-Contoured-lightweight-Comfortable/dp/B06W5CDY78?th=1 | Used as a blindfold to occlude vision in the No Vision condition |
| Orange Slices | N/A | N/A | Orange slices served as the large sized reaching targets |
| Donut Balls | Tim Hortons | http://www.timhortons.com/ca/en/menu/timbits.php | Old fashion plain timbits from Tim Hortons served as the medium sized reaching targets |
| Blueberries | N/A | N/A | Blueberries served as the small sized reaching targets |
References
- Karl, J. M., Whishaw, I. Q. Different evolutionary origins for the Reach and the Grasp: an explanation for dual visuomotor channels in primate parietofrontal cortex. Front Neurol. 4 (208), (2013).
- Whishaw, I. Q., Karl, J. M. The contribution of the reach and the grasp to shaping brain and behaviour. Can J Exp Psychol. 68 (4), 223-235 (2014).
- Jeannerod, M., Long, J., Badeley, A. Intersegmental coordination during reaching at natural visual objects. Attention and Performance IX. , 153-169 (1981).
- Arbib, M. A., Brooks, V. B. Perceptual structures and distributed motor control. Handbook of Physiology. 2, 1449-1480 (1981).
- De Bruin, N., Sacrey, L. A., Brown, L. A., Doan, J., Whishaw, I. Q. Visual guidance for hand advance but not hand withdrawal in a reach-to-eat task in adult humans: reaching is a composite movement. J Mot Behav. 40 (4), 337-346 (2008).
- Karl, J. M., Sacrey, L. A., Doan, J. B., Whishaw, I. Q. Hand shaping using hapsis resembles visually guided hand shaping. Exp Brain Res. 219 (1), 59-74 (2012).
- Domellöff, E., Hopkins, B., Francis, B., Rönnqvist, L. Effects of finger markers on the kinematics of reaching movements in young children and adults. J Appl Biomech. 23 (4), 315-321 (2007).
- Karl, J. M., Sacrey, L. A., Doan, J. B., Whishaw, I. Q. Oral hapsis guides accurate hand preshaping for grasping food targets in the mouth. Exp Brain Res. 221 (2), 223-240 (2012).
- Karl, J. M., Schneider, L. R., Whishaw, I. Q. Nonvisual learning of intrinsic object properties in a reaching task dissociates grasp from reach. Exp Brain Res. 225 (4), 465-477 (2013).
- Hall, L. A., Karl, J. M., Thomas, B. L., Whishaw, I. Q. Reach and Grasp reconfigurations reveal that proprioception assists reaching and hapsis assists grasping in peripheral vision. Exp Brain Res. 232 (9), 2807-2819 (2014).
- Karl, J. M., Whishaw, I. Q. Haptic grasping configurations in early infancy reveal different developmental profiles for visual guidance of the Reach versus the Grasp. Exp Brain Res. 232 (9), 3301-3316 (2014).
- Whishaw, I. Q., Karl, J. M., Humphrey, N. K. Dissociation of the Reach and the Grasp in the destriate (V1) monkey Helen: a new anatomy for the dual visuomotor channel theory of reaching. Exp Brain Res. 234 (8), 2351-2362 (2016).
- Timmann, D., Stelmach, G. E., Bloedel, J. R. Grasping component alterations and limb transport. Exp Brain Res. 108 (3), 486-492 (1996).
- Saling, M., Mescheriakov, S., Molokanova, E., Stelmach, G. E., Berger, M. Grip reorganization during wrist transport: the influence of an altered aperture. Exp Brain Res. 108 (3), 493-500 (1996).
- Whishaw, I. Q., Faraji, J., Kuntz, J., Mirza Ahga, B., Patel, M., Metz, G. A. S., et al. Organization of the reach and grasp in head-fixed vs freely-moving mice provides support for multiple motor channel theory of neocortical organization. Exp Brain Res. 235 (6), 1919-1932 (2017).
