paper video summaries
Here you can find video recordings summarizing a few of my main papers.
The Study of Dexterous Hand Manipulation: A Synergy-Based Complexity Index
Abstract: In this work we tackle the question of how to analyze and objectively quantify the complexity of a manipulation task. The study investigates the kinematic behavior of the hand joints in three different manipulation tasks of growing complexity: reaching-to-grasp, tool use and piano playing. The collected data were processed to extract the kinematic synergies of the hand by means of singular value decomposition. A novel, unbiased metric to determine hand manipulation complexity was based on the cumulative variance accounted for. This Variance-Accounted-For Complexity Index (VAF-CI) reliably distinguished between different manipulation tasks. Moreover, an unsupervised learning method (k-means clustering) was able to use the index to accurately identify the 3 distinct manipulation tasks. These results may be leveraged to improve the control of biomimetic dexterous robots during manipulation tasks.
Kinematic hand synergies differ between reach-and-grasp and functional object manipulation
Abstract: Humans possess a unique ability to manipulate tools to help us navigate the world around us. This ability is facilitated by the dexterity of our hands. However, millions lose this capability annually due to conditions like limb amputation or cerebral vascular accident (i.e., stroke). This great loss of human hand function has led to increased study of human hand action. Although previous research focused on coordinated hand motion, known as synergies, during reaching and grasping, manipulation of complex objects remains understudied. Specifically, we aimed to test two hypotheses: 1) the number of synergies underlying manipulation is the same as those underlying reach-and-grasp, and 2) the identity of synergies underlying manipulation is different from those underlying reach-and-grasp. To do so, we measured human hand motion during two experiments: 1) during reach and grasp of a tool or object commonly used in wire harness installation and 2) during manipulation of those objects and tools to install a wire harness on a mock electrical cabinet. Results showed that manipulation generally required more synergies than grasp. Comparison between reach-and-grasp and manipulation revealed a decrease in synergy similarity with synergy-order. Considering that higher-order synergies become significant during manipulation, it is important that we investigate these differences; this study serves as a point of entry to doing so. If we want our prosthetic and rehabilitative devices to restore hand function to those who have lost it, we must study hand function, specifically manipulation, and not just grasping.
New & Noteworthy: This study uncovers new insights into kinematic synergies during functional human hand manipulation of objects and tools, through the study of wire harness installation. It emphasizes the nuanced distinctions between functional hand manipulation and simple grasping, revealing that manipulation tasks require a greater number and distinct subset of hand synergies compared with simple grasp actions. This research marks a significant step toward appreciating the intricacies of hand coordination in complex tasks beyond grasping.
The Study of Complex Manipulation via Kinematic Hand Synergies: The Effects of Data Pre-Processing
Abstract: The study of kinematic hand synergies through matrix decomposition techniques, such as singular value decomposition, supports the theory that humans might control a subspace of predefined motions during manipulation tasks. These subspaces are often referred to as synergies. However, different data pre-processing methods lead to quantitatively different conclusions about these synergies. In this work, we shed light on the role of data pre-processing on the study of hand synergies by analyzing both numerical simulation and real kinematic data from a complex manipulation task, i.e., piano playing. The results obtained suggest that centering the data, by removing the mean, appears to be the most appropriate preprocessing technique for studying kinematic hand synergies.
Dynamic Primitives Limit Human Force Regulation During Motion
Abstract: Humans excel at physical interaction despite long feedback delays and low-bandwidth actuators. Yet little is known about how humans manage physical interaction. A quantitative understanding of how they do is critical for designing machines that can safely and effectively interact with humans, e.g. amputation prostheses, assistive exoskeletons, therapeutic rehabilitation robots, and physical human-robot collaboration. To facilitate applications, this understanding should be in the form of a simple mathematical model that not only describes humans’ capabilities but also their limitations. In robotics, hybrid control allows simultaneous, independent control of both motion and force and it is often assumed that humans can modulate force independent of motion as well. This letter experimentally tested that assumption. Participants were asked to apply a constant 5 N force on a robot manipulandum that moved along an elliptical path. After initial improvement, force errors quickly plateaued, despite practice and visual feedback. Within-trial analyses revealed that force errors varied with position on the ellipse, rejecting the hypothesis that humans have independent control of force and motion. The findings are consistent with a feed-forward motion command composed of two primitive oscillations acting through mechanical impedance to evoke force.
Role of path information in visual perception of joint stiffness
Abstract: Humans have an astonishing ability to extract hidden information from the movement of others. In previous work, subjects observed the motion of a simulated stick-figure, two-link planar arm and estimated its stiffness. Fundamentally, stiffness is the relation between force and displacement. Given that subjects were unable to physically interact with the simulated arm, they were forced to make their estimates solely based on observed kinematic information. Remarkably, subjects were able to correctly correlate their stiffness estimates with changes in the simulated stiffness, despite the lack of force information. We hypothesized that subjects were only able to do this because the controller used to produce the simulated arm’s movement, composed of oscillatory motions driving mechanical impedances, resembled the controller humans use to produce their own movement. However, it is still unknown what motion features subjects used to estimate stiffness. Human motion exhibits systematic velocity-curvature patterns, and it has previously been shown that these patterns play an important role in perceiving and interpreting motion. Thus, we hypothesized that manipulating the velocity profile should affect subjects’ ability to estimate stiffness. To test this, we changed the velocity profile of the simulated two-link planar arm while keeping the simulated joint paths the same. Even with manipulated velocity signals, subjects were still able to estimate changes in simulated joint stiffness. However, when subjects were shown the same simulated path with different velocity profiles, they perceived motions that followed a veridical velocity profile to be less stiff than that of a non-veridical profile. These results suggest that path information (displacement) predominates over temporal information (velocity) when humans use visual observation to estimate stiffness.
Author summary: Stiffness of the arms or legs, the force evoked by displacement, plays an important role in managing physical interaction with objects in the world. Measuring stiffness fundamentally requires physical contact. Nevertheless, previous study showed that humans have a remarkable ability to estimate stiffness solely from visual observation of a computer simulation, with no physical contact. The present study extended that work and found that this ability was robust. In particular, the ability to estimate simulated stiffness was largely unaffected by changing the time course of simulated motion. This was surprising given the extensive prior research reporting that distorting velocity patterns influences motion perception. The results presented in this paper indicate that geometric information (path) predominates over temporal information (velocity) in the perception of stiffness. Given the highly-cited relationship between motor action and perception, it also suggests that the structure of the motor control system we used in the simulations is a reasonable approximation of the neural motor controller. This work provides insight into humans’ representation of motor behavior and how humans interpret and learn from the motor actions of others.