2025
Arditi, Emir; Kunavar, Tjasa; Amirshirzad, Negin; Ugur, Emre; Babič, Jan; Oztop, Erhan
Inferring effort-safety trade off in perturbed squat-to-stand task by reward parameter estimation Journal Article
In: Engineering Applications of Artificial Intelligence, vol. 142, pp. 109778, 2025, ISSN: 09521976.
Abstract | BibTeX | Tags: Human Motor Control, Machine Learning, Neuromusculoskeletal Modelling, Optimal Control, Sensorimotor Learning | Links:
@article{Arditi2025,
title = {Inferring effort-safety trade off in perturbed squat-to-stand task by reward parameter estimation},
author = {Emir Arditi and Tjasa Kunavar and Negin Amirshirzad and Emre Ugur and Jan Babi\v{c} and Erhan Oztop},
doi = {10.1016/j.engappai.2024.109778},
issn = {09521976},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
journal = {Engineering Applications of Artificial Intelligence},
volume = {142},
pages = {109778},
abstract = {In this study, an inverse reinforcement learning (IRL) method is developed to estimate the parameters of a reward function that is assumed to guide the movement of a biological or artificial agent. The workings of the method is shown on the problem of estimating the effort-safety trade-off of humans during perturbed squat-to-stand motions based on their Center of Mass (COM) trajectories. The proposed method involves data generation by reinforcement learning (RL) and a novel data augmentation mechanism followed by neural network training. After the training, the neural network acts as the reward parameter estimator given the Center of Mass (COM) trajectories as input. The performance of the developed method is assessed through systematic simulation experiments, where it is shown that the parameter estimation made by our method is significantly more accurate than the baseline of an optimized template-based IRL approach. In addition, as a proof of concept, a set of human movement data is analyzed with the developed method. The results revealed that most participants acquired a strategy that ensures low effort expenditure with a safety margin, producing COM trajectories slightly away from the effort-optimal.},
keywords = {Human Motor Control, Machine Learning, Neuromusculoskeletal Modelling, Optimal Control, Sensorimotor Learning},
pubstate = {published},
tppubtype = {article}
}
In this study, an inverse reinforcement learning (IRL) method is developed to estimate the parameters of a reward function that is assumed to guide the movement of a biological or artificial agent. The workings of the method is shown on the problem of estimating the effort-safety trade-off of humans during perturbed squat-to-stand motions based on their Center of Mass (COM) trajectories. The proposed method involves data generation by reinforcement learning (RL) and a novel data augmentation mechanism followed by neural network training. After the training, the neural network acts as the reward parameter estimator given the Center of Mass (COM) trajectories as input. The performance of the developed method is assessed through systematic simulation experiments, where it is shown that the parameter estimation made by our method is significantly more accurate than the baseline of an optimized template-based IRL approach. In addition, as a proof of concept, a set of human movement data is analyzed with the developed method. The results revealed that most participants acquired a strategy that ensures low effort expenditure with a safety margin, producing COM trajectories slightly away from the effort-optimal.
2024
White, Olivier; Dehouck, Victor; Boulanger, Nicolas; Dierick, Frédéric; Babič, Jan; Goswami, Nandu; Buisseret, Fabien
Resonance tuning of rhythmic movements is disrupted at short time scales: A centrifuge study Journal Article
In: iScience, vol. 27, iss. 5, pp. 109618, 2024, ISSN: 25890042.
Abstract | BibTeX | Tags: Human Motor Control, Optimal Control, Sensorimotor Learning | Links:
@article{White2024,
title = {Resonance tuning of rhythmic movements is disrupted at short time scales: A centrifuge study},
author = {Olivier White and Victor Dehouck and Nicolas Boulanger and Fr\'{e}d\'{e}ric Dierick and Jan Babi\v{c} and Nandu Goswami and Fabien Buisseret},
doi = {10.1016/j.isci.2024.109618},
issn = {25890042},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {iScience},
volume = {27},
issue = {5},
pages = {109618},
abstract = {The human body exploits its neural mechanisms to optimize actions. Rhythmic movements are optimal when their frequency is close to the natural frequency of the system. In a pendulum, gravity modulates this spontaneous frequency. Participants unconsciously adjust their natural pace when cyclically moving the arm in altered gravity. However, the timescale of this adaptation is unexplored. Participants per- formed cyclic movements before, during, and after fast transitions between hypergravity levels (1g\textendash3g and 3g\textendash1g) induced by a human centrifuge. Movement periods were modulated with the average value of gravity during transitions. However, while participants increased movement pace on a cycle basis when gravity increased (1g\textendash3g), they did not decrease pace when gravity decreased (3g\textendash1g). We highlight asymmetric effects in the spontaneous adjustment of movement dynamics on short timescales, suggest- ing the involvement of cognitive factors, beyond standard dynamical models.},
keywords = {Human Motor Control, Optimal Control, Sensorimotor Learning},
pubstate = {published},
tppubtype = {article}
}
The human body exploits its neural mechanisms to optimize actions. Rhythmic movements are optimal when their frequency is close to the natural frequency of the system. In a pendulum, gravity modulates this spontaneous frequency. Participants unconsciously adjust their natural pace when cyclically moving the arm in altered gravity. However, the timescale of this adaptation is unexplored. Participants per- formed cyclic movements before, during, and after fast transitions between hypergravity levels (1g–3g and 3g–1g) induced by a human centrifuge. Movement periods were modulated with the average value of gravity during transitions. However, while participants increased movement pace on a cycle basis when gravity increased (1g–3g), they did not decrease pace when gravity decreased (3g–1g). We highlight asymmetric effects in the spontaneous adjustment of movement dynamics on short timescales, suggest- ing the involvement of cognitive factors, beyond standard dynamical models.
Díaz, María Alejandra; Bock, Sander De; Beckerle, Philipp; Babič, Jan; Verstraten, Tom; Pauw, Kevin De
Human-in-the-loop optimization of wearable device parameters using an EMG-based objective function Journal Article
In: Wearable Technologies, vol. 5, pp. e15, 2024.
Abstract | BibTeX | Tags: Exoskeleton Design and Control, Human Motor Control, Human Performance Augmentation, Human-in-the-Loop Control, Optimal Control | Links:
@article{nokey,
title = {Human-in-the-loop optimization of wearable device parameters using an EMG-based objective function},
author = {Mar\'{i}a Alejandra D\'{i}az and Sander De Bock and Philipp Beckerle and Jan Babi\v{c} and Tom Verstraten and Kevin De Pauw},
url = {https://doi.org/10.1017/wtc.2024.9},
doi = {10.1017/wtc.2024.9},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Wearable Technologies},
volume = {5},
pages = {e15},
abstract = {Advancements in wearable robots aim to improve user motion, motor control, and overall experience by minimizing energetic cost (EC). However, EC is challenging to measure and it is typically indirectly estimated through respiratory gas analysis. This study introduces a novel EMG-based objective function that captures individuals\' natural energetic expenditure during walking. The objective function combines information from electromyography (EMG) variables such as intensity and muscle synergies. First, we demonstrate the similarity of the proposed objective function, calculated offline, to the EC during walking. Second, we minimize and validate the EMG-based objective function using an online Bayesian optimization algorithm. The walking step frequency is chosen as the parameter to optimize in both offline and online approaches in order to simplify experiments and facilitate comparisons with related research. Compared to existing studies that use EC as the objective function, results demonstrated that the optimization of the presented objective function reduced the number of iterations and, when compared with gradient descent optimization strategies, also reduced convergence time. Moreover, the algorithm effectively converges toward an optimal step frequency near the user\'s preferred frequency, positively influencing EC reduction. The good correlation between the estimated objective function and measured EC highlights its consistency and reliability. Thus, the proposed objective function could potentially optimize lower limb exoskeleton assistance and improve user performance and human-robot interaction without the need for challenging respiratory gas measurements. Impact Statement Wearable devices are important in assisting people, such as patients or older adults, during rehabilitation and everyday activities like walking. Some exoskeletons have been able to reduce the energy cost of walking. However, they require a cumbersome device to quantify it, making it impractical to use in real-life scenarios. Thus, we need to identify a way to assess energetic cost using wearable technologies. To address this, we introduced an EMG-based objective function that captures insights into energetic cost through muscle dynamics and motor coordination. Then, we minimized the proposed objective function online by optimizing walking step frequencies. We found that the EMG-based objective function highly correlates with energetic cost during walking. We also found that our algorithm effectively identifies an optimal step frequency that reduces participants\' energetic cost. These findings will facilitate the customization of the assistance in wearable assistive devices and its application in real situations.},
keywords = {Exoskeleton Design and Control, Human Motor Control, Human Performance Augmentation, Human-in-the-Loop Control, Optimal Control},
pubstate = {published},
tppubtype = {article}
}
Advancements in wearable robots aim to improve user motion, motor control, and overall experience by minimizing energetic cost (EC). However, EC is challenging to measure and it is typically indirectly estimated through respiratory gas analysis. This study introduces a novel EMG-based objective function that captures individuals' natural energetic expenditure during walking. The objective function combines information from electromyography (EMG) variables such as intensity and muscle synergies. First, we demonstrate the similarity of the proposed objective function, calculated offline, to the EC during walking. Second, we minimize and validate the EMG-based objective function using an online Bayesian optimization algorithm. The walking step frequency is chosen as the parameter to optimize in both offline and online approaches in order to simplify experiments and facilitate comparisons with related research. Compared to existing studies that use EC as the objective function, results demonstrated that the optimization of the presented objective function reduced the number of iterations and, when compared with gradient descent optimization strategies, also reduced convergence time. Moreover, the algorithm effectively converges toward an optimal step frequency near the user's preferred frequency, positively influencing EC reduction. The good correlation between the estimated objective function and measured EC highlights its consistency and reliability. Thus, the proposed objective function could potentially optimize lower limb exoskeleton assistance and improve user performance and human-robot interaction without the need for challenging respiratory gas measurements. Impact Statement Wearable devices are important in assisting people, such as patients or older adults, during rehabilitation and everyday activities like walking. Some exoskeletons have been able to reduce the energy cost of walking. However, they require a cumbersome device to quantify it, making it impractical to use in real-life scenarios. Thus, we need to identify a way to assess energetic cost using wearable technologies. To address this, we introduced an EMG-based objective function that captures insights into energetic cost through muscle dynamics and motor coordination. Then, we minimized the proposed objective function online by optimizing walking step frequencies. We found that the EMG-based objective function highly correlates with energetic cost during walking. We also found that our algorithm effectively identifies an optimal step frequency that reduces participants' energetic cost. These findings will facilitate the customization of the assistance in wearable assistive devices and its application in real situations.
2023
Kunavar, Tjasa; Cheng, Xiaoxiao; Franklin, David W.; Burdet, Etienne; Babič, Jan
Explicit learning based on reward prediction error facilitates agile motor adaptations Journal Article
In: PLOS ONE, vol. 18, iss. 12, pp. e0295274, 2023, ISSN: 1932-6203.
Abstract | BibTeX | Tags: Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control | Links:
@article{Kunavar2023,
title = {Explicit learning based on reward prediction error facilitates agile motor adaptations},
author = {Tjasa Kunavar and Xiaoxiao Cheng and David W. Franklin and Etienne Burdet and Jan Babi\v{c}},
doi = {10.1371/journal.pone.0295274},
issn = {1932-6203},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {PLOS ONE},
volume = {18},
issue = {12},
pages = {e0295274},
abstract = {Error based motor learning can be driven by both sensory prediction error and reward prediction error. Learning based on sensory prediction error is termed sensorimotor adaptation, while learning based on reward prediction error is termed reward learning. To investigate the characteristics and differences between sensorimotor adaptation and reward learning, we adapted a visuomotor paradigm where subjects performed arm movements while presented with either the sensory prediction error, signed end-point error, or binary reward. Before each trial, perturbation indicators in the form of visual cues were presented to inform the subjects of the presence and direction of the perturbation. To analyse the interconnection between sensorimotor adaptation and reward learning, we designed a computational model that distinguishes between the two prediction errors. Our results indicate that subjects adapted to novel perturbations irrespective of the type of prediction error they received during learning, and they converged towards the same movement patterns. Sensorimotor adaptations led to a pronounced aftereffect, while adaptation based on reward consequences produced smaller aftereffects suggesting that reward learning does not alter the internal model to the same degree as sensorimotor adaptation. Even though all subjects had learned to counteract two different perturbations separately, only those who relied on explicit learning using reward prediction error could timely adapt to the randomly changing perturbation. The results from the computational model suggest that sensorimotor and reward learning operate through distinct adaptation processes and that only sensorimotor adaptation changes the internal model, whereas reward learning employs explicit strategies that do not result in aftereffects. Additionally, we demonstrate that when humans learn motor tasks, they utilize both learning processes to successfully adapt to the new environments.},
keywords = {Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control},
pubstate = {published},
tppubtype = {article}
}
Error based motor learning can be driven by both sensory prediction error and reward prediction error. Learning based on sensory prediction error is termed sensorimotor adaptation, while learning based on reward prediction error is termed reward learning. To investigate the characteristics and differences between sensorimotor adaptation and reward learning, we adapted a visuomotor paradigm where subjects performed arm movements while presented with either the sensory prediction error, signed end-point error, or binary reward. Before each trial, perturbation indicators in the form of visual cues were presented to inform the subjects of the presence and direction of the perturbation. To analyse the interconnection between sensorimotor adaptation and reward learning, we designed a computational model that distinguishes between the two prediction errors. Our results indicate that subjects adapted to novel perturbations irrespective of the type of prediction error they received during learning, and they converged towards the same movement patterns. Sensorimotor adaptations led to a pronounced aftereffect, while adaptation based on reward consequences produced smaller aftereffects suggesting that reward learning does not alter the internal model to the same degree as sensorimotor adaptation. Even though all subjects had learned to counteract two different perturbations separately, only those who relied on explicit learning using reward prediction error could timely adapt to the randomly changing perturbation. The results from the computational model suggest that sensorimotor and reward learning operate through distinct adaptation processes and that only sensorimotor adaptation changes the internal model, whereas reward learning employs explicit strategies that do not result in aftereffects. Additionally, we demonstrate that when humans learn motor tasks, they utilize both learning processes to successfully adapt to the new environments.
Diaz, Maria Alejandra; Bock, Sander De; Beckerle, Philipp; Babič, Jan; Verstraten, Tom; Pauw, Kevin De
An EMG-Based Objective Function for Human-in-the-Loop Optimization. Proceedings Article
In: 2023 International Conference on Rehabilitation Robotics (ICORR), pp. 1-6, IEEE, 2023, ISSN: 1945-7901.
Abstract | BibTeX | Tags: Human Performance Augmentation, Human-in-the-Loop Control, Optimal Control | Links:
@inproceedings{Diaz2023,
title = {An EMG-Based Objective Function for Human-in-the-Loop Optimization.},
author = {Maria Alejandra Diaz and Sander De Bock and Philipp Beckerle and Jan Babi\v{c} and Tom Verstraten and Kevin De Pauw},
doi = {10.1109/ICORR58425.2023.10304819},
issn = {1945-7901},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 International Conference on Rehabilitation Robotics (ICORR)},
journal = {IEEE ... International Conference on Rehabilitation Robotics : [proceedings]},
volume = {2023},
pages = {1-6},
publisher = {IEEE},
abstract = {Advancements in wearable robots aim to improve the users' motion, performance, and comfort by optimizing, mainly, energetic cost (EC). However, EC is a noisy measurement with a physiological delayed response that requires long evaluation periods and wearing an uncomfortable mask. This study aims to estimate and minimize an EMG-based objective function that describes the natural energetic expenditure of individuals walking. This objective is assessed by combining multiple electromyography (EMG) variables from the EMG intensity and muscle synergies. To evaluate this objective function simply and repeatedly, we prescribed step frequency (SF) via a metronome and optimized this frequency to minimize muscle activity demands. Further, a linear mixed-effects model was fitted for EC, with the EMG variables as fixed-effects and a random intercept that varies by participant. After the model was fitted to the data, a cubic polynomial was used to identify the optimal SF that reduces the overall EMG-based objective function. Our analysis outlines that the proposed objective function is comparable to the EC during walking, the primary objective function used in human-in-the-loop optimization. Thus, this EMG-based objective function could be potentially used to optimize wearable robots and improve human-robot interaction.},
keywords = {Human Performance Augmentation, Human-in-the-Loop Control, Optimal Control},
pubstate = {published},
tppubtype = {inproceedings}
}
Advancements in wearable robots aim to improve the users' motion, performance, and comfort by optimizing, mainly, energetic cost (EC). However, EC is a noisy measurement with a physiological delayed response that requires long evaluation periods and wearing an uncomfortable mask. This study aims to estimate and minimize an EMG-based objective function that describes the natural energetic expenditure of individuals walking. This objective is assessed by combining multiple electromyography (EMG) variables from the EMG intensity and muscle synergies. To evaluate this objective function simply and repeatedly, we prescribed step frequency (SF) via a metronome and optimized this frequency to minimize muscle activity demands. Further, a linear mixed-effects model was fitted for EC, with the EMG variables as fixed-effects and a random intercept that varies by participant. After the model was fitted to the data, a cubic polynomial was used to identify the optimal SF that reduces the overall EMG-based objective function. Our analysis outlines that the proposed objective function is comparable to the EC during walking, the primary objective function used in human-in-the-loop optimization. Thus, this EMG-based objective function could be potentially used to optimize wearable robots and improve human-robot interaction.
2021
Arditi, Emir; Kunavar, Tjaša; Ugur, Emre; Babič, Jan; Oztop, Erhan
Inferring Cost Functions Using Reward Parameter Search and Policy Gradient Reinforcement Learning Proceedings Article
In: IECON 2021 – 47th Annual Conference of the IEEE Industrial Electronics Society, pp. 1–6, IEEE, 2021, ISBN: 978-1-6654-3554-3.
Abstract | BibTeX | Tags: Human Motor Control, Optimal Control, Sensorimotor Learning | Links:
@inproceedings{Arditi2021,
title = {Inferring Cost Functions Using Reward Parameter Search and Policy Gradient Reinforcement Learning},
author = {Emir Arditi and Tja\v{s}a Kunavar and Emre Ugur and Jan Babi\v{c} and Erhan Oztop},
url = {https://ieeexplore.ieee.org/document/9589967/},
doi = {10.1109/IECON48115.2021.9589967},
isbn = {978-1-6654-3554-3},
year = {2021},
date = {2021-10-01},
urldate = {2021-10-01},
booktitle = {IECON 2021 \textendash 47th Annual Conference of the IEEE Industrial Electronics Society},
pages = {1--6},
publisher = {IEEE},
abstract = {This study focuses on inferring cost functions of obtained movement data using reward parameter search and pol-icy gradient based Reinforcement Learning (RL). The behavior data for this task is obtained through a series of squat-to-stand movements of human participants under dynamic perturbations. The key parameter searched in the cost function is the weight of total torque used in performing the squat-to-stand action. An approximate model is used to learn squat-to-stand movements via a policy gradient method, namely Proximal Policy Optimization(PPO). A behavioral similarity metric based on Center of Mass(COM) is used to find the most likely weight parameter. The stochasticity in the training result of PPO is dealt with multiple runs, and as a result, a reasonable and a stable Inverse Reinforcement Learning(IRL) algorithm is obtained in terms of performance. The results indicate that for some participants, the reward function parameters of the experts were inferred successfully.},
keywords = {Human Motor Control, Optimal Control, Sensorimotor Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
This study focuses on inferring cost functions of obtained movement data using reward parameter search and pol-icy gradient based Reinforcement Learning (RL). The behavior data for this task is obtained through a series of squat-to-stand movements of human participants under dynamic perturbations. The key parameter searched in the cost function is the weight of total torque used in performing the squat-to-stand action. An approximate model is used to learn squat-to-stand movements via a policy gradient method, namely Proximal Policy Optimization(PPO). A behavioral similarity metric based on Center of Mass(COM) is used to find the most likely weight parameter. The stochasticity in the training result of PPO is dealt with multiple runs, and as a result, a reasonable and a stable Inverse Reinforcement Learning(IRL) algorithm is obtained in terms of performance. The results indicate that for some participants, the reward function parameters of the experts were inferred successfully.
Jamšek, Marko; Kunavar, Tjaša; Bobek, Urban; Rueckert, Elmar; Babič, Jan
Predictive Exoskeleton Control for Arm-Motion Augmentation Based on Probabilistic Movement Primitives Combined With a Flow Controller Journal Article
In: IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 4417–4424, 2021, ISSN: 2377-3766.
Abstract | BibTeX | Tags: Exoskeleton Design and Control, Force Control, Human Performance Augmentation, Optimal Control, Physical Human Robot Interaction | Links:
@article{Jamsek2021,
title = {Predictive Exoskeleton Control for Arm-Motion Augmentation Based on Probabilistic Movement Primitives Combined With a Flow Controller},
author = {Marko Jam\v{s}ek and Tja\v{s}a Kunavar and Urban Bobek and Elmar Rueckert and Jan Babi\v{c}},
url = {https://ieeexplore.ieee.org/document/9387088/},
doi = {10.1109/LRA.2021.3068892},
issn = {2377-3766},
year = {2021},
date = {2021-07-01},
urldate = {2021-07-01},
journal = {IEEE Robotics and Automation Letters},
volume = {6},
number = {3},
pages = {4417--4424},
abstract = {There are many work-related repetitive tasks where the application of exoskeletons could significantly reduce the physical effort by assisting the user in moving the arms towards the desired location in space. To make such controlmore user acceptable, the controller should be able to predict the motion of the user and act accordingly. This letter presents an exoskeleton control method that utilizes probabilistic movement primitives to generate predictions of user movements in real-time. These predictions are used in a flow controller, which represents a novel velocity-field-based exoskeleton control approach to provide assistance to the user in a predictive way. We evaluated our approach with a haptic robot, where a group of twelve participants had to perform movements towards different target locations in the frontal plane. We tested whether we could generalize the predictions for new and unknown target locations whilst providing assistance to the user without changing their kinematic parameters. The evaluation showed that we could accurately predict user movement intentions while at the same time significantly decrease the overall physical effort exerted by the participants to achieve the task.},
keywords = {Exoskeleton Design and Control, Force Control, Human Performance Augmentation, Optimal Control, Physical Human Robot Interaction},
pubstate = {published},
tppubtype = {article}
}
There are many work-related repetitive tasks where the application of exoskeletons could significantly reduce the physical effort by assisting the user in moving the arms towards the desired location in space. To make such controlmore user acceptable, the controller should be able to predict the motion of the user and act accordingly. This letter presents an exoskeleton control method that utilizes probabilistic movement primitives to generate predictions of user movements in real-time. These predictions are used in a flow controller, which represents a novel velocity-field-based exoskeleton control approach to provide assistance to the user in a predictive way. We evaluated our approach with a haptic robot, where a group of twelve participants had to perform movements towards different target locations in the frontal plane. We tested whether we could generalize the predictions for new and unknown target locations whilst providing assistance to the user without changing their kinematic parameters. The evaluation showed that we could accurately predict user movement intentions while at the same time significantly decrease the overall physical effort exerted by the participants to achieve the task.
2020
Rottmann, Nils; Kunavar, Tjaša; Babič, Jan; Peters, Jan; Rueckert, Elmar
Learning Hierarchical Acquisition Functions for Bayesian Optimization Proceedings Article
In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5490–5496, 2020.
Abstract | BibTeX | Tags: Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control, Postural Balance | Links:
@inproceedings{9341335,
title = {Learning Hierarchical Acquisition Functions for Bayesian Optimization},
author = {Nils Rottmann and Tja\v{s}a Kunavar and Jan Babi\v{c} and Jan Peters and Elmar Rueckert},
doi = {10.1109/IROS45743.2020.9341335},
year = {2020},
date = {2020-01-01},
urldate = {2020-01-01},
booktitle = {2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
pages = {5490--5496},
abstract = {Learning control policies in robotic tasks requires a large number of interactions due to small learning rates, bounds on the updates or unknown constraints. In contrast humans can infer protective and safe solutions after a single failure or unexpected observation. In order to reach similar performance, we developed a hierarchical Bayesian optimization algorithm that replicates the cognitive inference and memorization process for avoiding failures in motor control tasks. A Gaussian Process implements the modeling and the sampling of the acquisition function. This enables rapid learning with large learning rates while a mental replay phase ensures that policy regions that led to failures are inhibited during the sampling process. The features of the hierarchical Bayesian optimization method are evaluated in a simulated and physiological humanoid postural balancing task. The method out- performs standard optimization techniques, such as Bayesian Optimization, in the number of interactions to solve the task, in the computational demands and in the frequency of observed failures. Further, we show that our method performs similar to humans for learning the postural balancing task by comparing our simulation results with real human data.},
keywords = {Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control, Postural Balance},
pubstate = {published},
tppubtype = {inproceedings}
}
Learning control policies in robotic tasks requires a large number of interactions due to small learning rates, bounds on the updates or unknown constraints. In contrast humans can infer protective and safe solutions after a single failure or unexpected observation. In order to reach similar performance, we developed a hierarchical Bayesian optimization algorithm that replicates the cognitive inference and memorization process for avoiding failures in motor control tasks. A Gaussian Process implements the modeling and the sampling of the acquisition function. This enables rapid learning with large learning rates while a mental replay phase ensures that policy regions that led to failures are inhibited during the sampling process. The features of the hierarchical Bayesian optimization method are evaluated in a simulated and physiological humanoid postural balancing task. The method out- performs standard optimization techniques, such as Bayesian Optimization, in the number of interactions to solve the task, in the computational demands and in the frequency of observed failures. Further, we show that our method performs similar to humans for learning the postural balancing task by comparing our simulation results with real human data.
2019
Peternel, Luka; Babič, Jan
Target of initial sub-movement in multi-component arm-reaching strategy Journal Article
In: Scientific Reports, vol. 9, no. 1, pp. 20101, 2019, ISSN: 2045-2322.
Abstract | BibTeX | Tags: Human Motor Control, Optimal Control, Sensorimotor Learning | Links:
@article{Peternel2019,
title = {Target of initial sub-movement in multi-component arm-reaching strategy},
author = {Luka Peternel and Jan Babi\v{c}},
url = {http://www.nature.com/articles/s41598-019-56430-x},
doi = {10.1038/s41598-019-56430-x},
issn = {2045-2322},
year = {2019},
date = {2019-12-01},
urldate = {2019-12-01},
journal = {Scientific Reports},
volume = {9},
number = {1},
pages = {20101},
abstract = {Goal-directed human reaching often involves multi-component strategy with sub-movements. in general, the initial sub-movement is fast and less precise to bring the limb's endpoint in the vicinity of the target as soon as possible. The final sub-movement then corrects the error accumulated during the previous sub-movement in order to reach the target. We investigate properties of a temporary target of the initial sub-movement. We hypothesise that the peak spatial dispersion of movement trajectories in the axis perpendicular to the movement is in front of the final reaching target, and that it indicates the temporary target of the initial sub-movement. the reasoning is that the dispersion accumulates, due to signal-dependent noise during the initial sub-movement, until the final corrective sub-movement is initiated, which then reduces the dispersion to successfully reach the actual target. We also hypothesise that the reaching movement distance and size of the actual target affect the properties of the temporary target of the initial sub-movement. the increased reaching movement distance increases the magnitude of peak dispersion and moves its location away from the actual target. on the other hand, the increased target size increases the magnitude of peak dispersion and moves its location closer to the actual target.},
keywords = {Human Motor Control, Optimal Control, Sensorimotor Learning},
pubstate = {published},
tppubtype = {article}
}
Goal-directed human reaching often involves multi-component strategy with sub-movements. in general, the initial sub-movement is fast and less precise to bring the limb's endpoint in the vicinity of the target as soon as possible. The final sub-movement then corrects the error accumulated during the previous sub-movement in order to reach the target. We investigate properties of a temporary target of the initial sub-movement. We hypothesise that the peak spatial dispersion of movement trajectories in the axis perpendicular to the movement is in front of the final reaching target, and that it indicates the temporary target of the initial sub-movement. the reasoning is that the dispersion accumulates, due to signal-dependent noise during the initial sub-movement, until the final corrective sub-movement is initiated, which then reduces the dispersion to successfully reach the actual target. We also hypothesise that the reaching movement distance and size of the actual target affect the properties of the temporary target of the initial sub-movement. the increased reaching movement distance increases the magnitude of peak dispersion and moves its location away from the actual target. on the other hand, the increased target size increases the magnitude of peak dispersion and moves its location closer to the actual target.
Petrič, Tadej; Peternel, Luka; Morimoto, Jun; Babič, Jan
Assistive Arm-Exoskeleton Control Based on Human Muscular Manipulability Journal Article
In: Frontiers in Neurorobotics, vol. 13, pp. 30, 2019, ISSN: 1662-5218.
Abstract | BibTeX | Tags: Compliance and Impedance Control, Human Performance Augmentation, Optimal Control | Links:
@article{Petric2019,
title = {Assistive Arm-Exoskeleton Control Based on Human Muscular Manipulability},
author = {Tadej Petri\v{c} and Luka Peternel and Jun Morimoto and Jan Babi\v{c}},
url = {https://www.frontiersin.org/article/10.3389/fnbot.2019.00030/full https://www.frontiersin.org/articles/10.3389/fnbot.2019.00030/full?\&utm_source=Email_to_authors_\&utm_medium=Email\&utm_content=T1_11.5e1_author\&utm_campaign=Email_publication\&field=\&journalName=Frontiers_in_Neurorobotics\&id=451266},
doi = {10.3389/fnbot.2019.00030},
issn = {1662-5218},
year = {2019},
date = {2019-05-01},
journal = {Frontiers in Neurorobotics},
volume = {13},
pages = {30},
publisher = {Frontiers},
abstract = {This paper introduces a novel control framework for an arm exoskeleton that takes into account force of the human arm. In contrast to the conventional exoskeleton controllers where the assistance is provided without considering the human arm biomechanical force manipulability properties, we propose a control approach based on the arm muscular manipulability. The proposed control framework essentially reshapes the anisotropic force manipulability into the endpoint force manipulability that is invariant with respect to the direction in the entire workspace of the arm. This allows users of the exoskeleton to perform tasks effectively in the whole range of the workspace, even in areas that are normally unsuitable due to the low force manipulability of the human arm. We evaluated the proposed control framework with real robot experiments where subjects wearing an arm exoskeleton were asked to move a weight between several locations. The results show that the proposed control framework does not affect the normal movement behavior of the users while effectively reduces user effort in the area of low manipulability.Particularly, the proposed approach augments the human arm force manipulability to execute tasks equally well in the entire workspace of the arm.},
keywords = {Compliance and Impedance Control, Human Performance Augmentation, Optimal Control},
pubstate = {published},
tppubtype = {article}
}
This paper introduces a novel control framework for an arm exoskeleton that takes into account force of the human arm. In contrast to the conventional exoskeleton controllers where the assistance is provided without considering the human arm biomechanical force manipulability properties, we propose a control approach based on the arm muscular manipulability. The proposed control framework essentially reshapes the anisotropic force manipulability into the endpoint force manipulability that is invariant with respect to the direction in the entire workspace of the arm. This allows users of the exoskeleton to perform tasks effectively in the whole range of the workspace, even in areas that are normally unsuitable due to the low force manipulability of the human arm. We evaluated the proposed control framework with real robot experiments where subjects wearing an arm exoskeleton were asked to move a weight between several locations. The results show that the proposed control framework does not affect the normal movement behavior of the users while effectively reduces user effort in the area of low manipulability.Particularly, the proposed approach augments the human arm force manipulability to execute tasks equally well in the entire workspace of the arm.
Azad, Morteza; Babič, Jan; Mistry, Michael
Effects of the weighting matrix on dynamic manipulability of robots Journal Article
In: Autonomous Robots, vol. 43, no. 7, pp. 1867–1879, 2019.
Abstract | BibTeX | Tags: Human Motor Control, Optimal Control, Postural Balance | Links:
@article{Azad2019,
title = {Effects of the weighting matrix on dynamic manipulability of robots},
author = {Morteza Azad and Jan Babi\v{c} and Michael Mistry},
url = {http://link.springer.com/10.1007/s10514-018-09819-y},
doi = {10.1007/s10514-018-09819-y},
year = {2019},
date = {2019-02-01},
journal = {Autonomous Robots},
volume = {43},
number = {7},
pages = {1867--1879},
abstract = {Dynamic manipulability of robots is a well-known tool to analyze, measure and predict a robot's performance in executing different tasks. This tool provides a graphical representation and a set of metrics as outcomes of a mapping from joint torques to the acceleration space of any point of interest of a robot such as the end-effector or the center of mass. In this paper, we show that the weighting matrix, which is included in the aforementioned mapping, plays a crucial role in the results of the dynamic manipulability analysis. Therefore, finding proper values for this matrix is the key to achieve reliable results. This paper studies the importance of the weighting matrix for dynamic manipulability of robots, which is overlooked in the literature, and suggests two physically meaningful choices for that matrix. We also explain three different metrics, which can be extracted from the graphical representations (i.e. ellipsoids) of the dynamic manipulability analysis. The application of these metrics in measuring a robot's physical ability to accelerate its end-effector in various desired directions is discussed via two illustrative examples.},
keywords = {Human Motor Control, Optimal Control, Postural Balance},
pubstate = {published},
tppubtype = {article}
}
Dynamic manipulability of robots is a well-known tool to analyze, measure and predict a robot's performance in executing different tasks. This tool provides a graphical representation and a set of metrics as outcomes of a mapping from joint torques to the acceleration space of any point of interest of a robot such as the end-effector or the center of mass. In this paper, we show that the weighting matrix, which is included in the aforementioned mapping, plays a crucial role in the results of the dynamic manipulability analysis. Therefore, finding proper values for this matrix is the key to achieve reliable results. This paper studies the importance of the weighting matrix for dynamic manipulability of robots, which is overlooked in the literature, and suggests two physically meaningful choices for that matrix. We also explain three different metrics, which can be extracted from the graphical representations (i.e. ellipsoids) of the dynamic manipulability analysis. The application of these metrics in measuring a robot's physical ability to accelerate its end-effector in various desired directions is discussed via two illustrative examples.
2018
Čamernik, Jernej; Rueckert, Elmar; Oztop, Erhan; Babič, Jan
Configurable dynamical environment simulation platform for studies of whole- body motor control and learning Proceedings Article
In: Congress programme, 8th World Congress of Biomechanics, 8-12 July 2018, Dublin, Ireland, Dublin, 2018.
Abstract | BibTeX | Tags: Human Motor Control, Optimal Control, Sensorimotor Learning
@inproceedings{\v{C}amernik2018a,
title = {Configurable dynamical environment simulation platform for studies of whole- body motor control and learning},
author = {Jernej \v{C}amernik and Elmar Rueckert and Erhan Oztop and Jan Babi\v{c}},
year = {2018},
date = {2018-01-01},
booktitle = {Congress programme, 8th World Congress of Biomechanics, 8-12 July 2018, Dublin, Ireland},
address = {Dublin},
abstract = {We propose an experimental paradigm with which we aim to replicate force field experiments in reaching studies by using full body motion and applying force controlled perturbation directly to the subject’s centre-of-mass. With our specific experimental setup, we even make a step forward from conventional setups and expose the sensorimotor control mechanisms and adaptations to danger of falling and injury. The two main components of our setup include a custom made force-controlled pulling mechanism and a 6 degrees-of-freedom position controlled Stewart platform. Both devices can be used either individually or simultaneously for inducing full body perturbations and/or reducing the effects of perturbation. To test our setup, we conducted a pilot study where subjects had to perform a series of squat-to-stand movements. By methodologically applying perturbations directly to their centre-of-mass, we created an environment where danger and energy consumption is critical. Preliminary results show that our experimental setup can be used for directly studying full-body motor control and learning.},
keywords = {Human Motor Control, Optimal Control, Sensorimotor Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
We propose an experimental paradigm with which we aim to replicate force field experiments in reaching studies by using full body motion and applying force controlled perturbation directly to the subject’s centre-of-mass. With our specific experimental setup, we even make a step forward from conventional setups and expose the sensorimotor control mechanisms and adaptations to danger of falling and injury. The two main components of our setup include a custom made force-controlled pulling mechanism and a 6 degrees-of-freedom position controlled Stewart platform. Both devices can be used either individually or simultaneously for inducing full body perturbations and/or reducing the effects of perturbation. To test our setup, we conducted a pilot study where subjects had to perform a series of squat-to-stand movements. By methodologically applying perturbations directly to their centre-of-mass, we created an environment where danger and energy consumption is critical. Preliminary results show that our experimental setup can be used for directly studying full-body motor control and learning.
2017
Peternel, Luka; Sigaud, Olivier; Babič, Jan
Unifying Speed-Accuracy Trade-Off and Cost-Benefit Trade-Off in Human Reaching Movements Journal Article
In: Frontiers in Human Neuroscience, vol. 11, pp. 615, 2017, ISSN: 1662-5161.
Abstract | BibTeX | Tags: Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control | Links:
@article{Peternel2017b,
title = {Unifying Speed-Accuracy Trade-Off and Cost-Benefit Trade-Off in Human Reaching Movements},
author = {Luka Peternel and Olivier Sigaud and Jan Babi\v{c}},
url = {http://journal.frontiersin.org/article/10.3389/fnhum.2017.00615/full},
doi = {10.3389/fnhum.2017.00615},
issn = {1662-5161},
year = {2017},
date = {2017-01-01},
journal = {Frontiers in Human Neuroscience},
volume = {11},
pages = {615},
abstract = {Two basic trade-offs interact while our brain decides how to move our body. First, with the cost-benefit trade-off, the brain trades between the importance of moving faster toward a target that is more rewarding and the increased muscular cost resulting from a faster movement. Second, with the speed-accuracy trade-off, the brain trades between how accurate the movement needs to be and the time it takes to achieve such accuracy. So far, these two trade-offs have been well studied in isolation, despite their obvious interdependence. To overcome this limitation, we propose a new model that is able to simultaneously account for both trade-offs. The model assumes that the central nervous system maximizes the expected utility resulting from the potential reward and the cost over the repetition of many movements, taking into account the probability to miss the target. The resulting model is able to account for both the speed-accuracy and the cost-benefit trade-offs. To validate the proposed hypothesis, we confront the properties of the computational model to data from an experimental study where subjects have to reach for targets by performing arm movements in a horizontal plane. The results qualitatively show that the proposed model successfully accounts for both cost-benefit and speed-accuracy trade-offs.},
keywords = {Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control},
pubstate = {published},
tppubtype = {article}
}
Two basic trade-offs interact while our brain decides how to move our body. First, with the cost-benefit trade-off, the brain trades between the importance of moving faster toward a target that is more rewarding and the increased muscular cost resulting from a faster movement. Second, with the speed-accuracy trade-off, the brain trades between how accurate the movement needs to be and the time it takes to achieve such accuracy. So far, these two trade-offs have been well studied in isolation, despite their obvious interdependence. To overcome this limitation, we propose a new model that is able to simultaneously account for both trade-offs. The model assumes that the central nervous system maximizes the expected utility resulting from the potential reward and the cost over the repetition of many movements, taking into account the probability to miss the target. The resulting model is able to account for both the speed-accuracy and the cost-benefit trade-offs. To validate the proposed hypothesis, we confront the properties of the computational model to data from an experimental study where subjects have to reach for targets by performing arm movements in a horizontal plane. The results qualitatively show that the proposed model successfully accounts for both cost-benefit and speed-accuracy trade-offs.
Petrič, Tadej; Simpson, Cole S; Ude, Aleš; Ijspeert, Auke J
Hammering Does Not Fit Fitts' Law Journal Article
In: Frontiers in Computational Neuroscience, vol. 11, no. May, pp. 1–12, 2017, ISSN: 1662-5188.
Abstract | BibTeX | Tags: Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control | Links:
@article{Petric2017b,
title = {Hammering Does Not Fit Fitts' Law},
author = {Tadej Petri\v{c} and Cole S Simpson and Ale\v{s} Ude and Auke J Ijspeert},
url = {http://journal.frontiersin.org/article/10.3389/fncom.2017.00045/full},
doi = {10.3389/fncom.2017.00045},
issn = {1662-5188},
year = {2017},
date = {2017-01-01},
journal = {Frontiers in Computational Neuroscience},
volume = {11},
number = {May},
pages = {1--12},
abstract = {While movement is essential to human wellbeing, we are still unable to reproduce the deftness and robustness of human movement in automatons or completely restore function to individuals with many types of motor impairment. To better understand how the human nervous system plans and controls movements, neuromechanists employ simple tasks such as upper extremity reaches and isometric force tasks. However, these simple tasks rarely consider impacts and may not capture aspects of motor control that arise from real-world complexity. Here we compared existing models of motor control with the results of a periodic targeted impact task extended from Bernstein's seminal work: hammering a nail into wood. We recorded impact forces and kinematics from 10 subjects hammering at different frequencies and with hammers with different physical properties (mass and face area). We found few statistical differences in most measures between different types of hammer, demonstrating human robustness to minor changes in dynamics. Because human motor control is thought to obey optimality principles, we also developed a feedforward optimal simulation with a neuromechanically inspired cost function that reproduces the experimental data. However, Fitts' Law, which relates movement time to distance traveled and target size, did not match our experimental data. We therefore propose a new model in which the distance moved is a logarithmic function of the time to move that yields better results (R2 ≥ 0.99 compared to R2 ≥ 0.88). These results support the argument that humans control movement in an optimal way, but suggest that Fitts' Law may not generalize to periodic impact tasks.},
keywords = {Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control},
pubstate = {published},
tppubtype = {article}
}
While movement is essential to human wellbeing, we are still unable to reproduce the deftness and robustness of human movement in automatons or completely restore function to individuals with many types of motor impairment. To better understand how the human nervous system plans and controls movements, neuromechanists employ simple tasks such as upper extremity reaches and isometric force tasks. However, these simple tasks rarely consider impacts and may not capture aspects of motor control that arise from real-world complexity. Here we compared existing models of motor control with the results of a periodic targeted impact task extended from Bernstein's seminal work: hammering a nail into wood. We recorded impact forces and kinematics from 10 subjects hammering at different frequencies and with hammers with different physical properties (mass and face area). We found few statistical differences in most measures between different types of hammer, demonstrating human robustness to minor changes in dynamics. Because human motor control is thought to obey optimality principles, we also developed a feedforward optimal simulation with a neuromechanically inspired cost function that reproduces the experimental data. However, Fitts' Law, which relates movement time to distance traveled and target size, did not match our experimental data. We therefore propose a new model in which the distance moved is a logarithmic function of the time to move that yields better results (R2 ≥ 0.99 compared to R2 ≥ 0.88). These results support the argument that humans control movement in an optimal way, but suggest that Fitts' Law may not generalize to periodic impact tasks.
2016
Rueckert, Elmar; Čamernik, Jernej; Peters, Jan; Babič, Jan
Probabilistic Movement Models Show that Postural Control Precedes and Predicts Volitional Motor Control Journal Article
In: Scientific Reports, vol. 6, no. 1, pp. 28455, 2016, ISSN: 2045-2322.
Abstract | BibTeX | Tags: Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control, Sensorimotor Learning | Links:
@article{Rueckert2016,
title = {Probabilistic Movement Models Show that Postural Control Precedes and Predicts Volitional Motor Control},
author = {Elmar Rueckert and Jernej \v{C}amernik and Jan Peters and Jan Babi\v{c}},
url = {http://www.nature.com/articles/srep28455},
doi = {10.1038/srep28455},
issn = {2045-2322},
year = {2016},
date = {2016-09-01},
journal = {Scientific Reports},
volume = {6},
number = {1},
pages = {28455},
publisher = {Nature Publishing Group},
abstract = {Human motor skill learning is driven by the necessity to adapt to new situations. While supportive contacts are essential for many tasks, little is known about their impact on motor learning. To study the effect of contacts an innovative full-body experimental paradigm was established. The task of the subjects was to reach for a distant target while postural stability could only be maintained by establishing an additional supportive hand contact. To examine adaptation, non-trivial postural perturbations of the subjects' support base were systematically introduced. A novel probabilistic trajectory model approach was employed to analyze the correlation between the motions of both arms and the trunk. We found that subjects adapted to the perturbations by establishing target dependent hand contacts. Moreover, we found that the trunk motion adapted significantly faster than the motion of the arms. However, the most striking finding was that observations of the initial phase of the left arm or trunk motion (100-400 ms) were sufficient to faithfully predict the complete movement of the right arm. Overall, our results suggest that the goal-directed arm movements determine the supportive arm motions and that the motion of heavy body parts adapts faster than the light arms.},
keywords = {Human Motor Control, Neuromusculoskeletal Modelling, Optimal Control, Sensorimotor Learning},
pubstate = {published},
tppubtype = {article}
}
Human motor skill learning is driven by the necessity to adapt to new situations. While supportive contacts are essential for many tasks, little is known about their impact on motor learning. To study the effect of contacts an innovative full-body experimental paradigm was established. The task of the subjects was to reach for a distant target while postural stability could only be maintained by establishing an additional supportive hand contact. To examine adaptation, non-trivial postural perturbations of the subjects' support base were systematically introduced. A novel probabilistic trajectory model approach was employed to analyze the correlation between the motions of both arms and the trunk. We found that subjects adapted to the perturbations by establishing target dependent hand contacts. Moreover, we found that the trunk motion adapted significantly faster than the motion of the arms. However, the most striking finding was that observations of the initial phase of the left arm or trunk motion (100-400 ms) were sufficient to faithfully predict the complete movement of the right arm. Overall, our results suggest that the goal-directed arm movements determine the supportive arm motions and that the motion of heavy body parts adapts faster than the light arms.
2015
Deniša, Miha; Gams, Andrej; Ude, Aleš; Petrič, Tadej
Generalization of discrete Compliant Movement Primitives Proceedings Article
In: 2015 International Conference on Advanced Robotics (ICAR), pp. 565–572, IEEE, Istanbul, 2015, ISBN: 978-1-4673-7509-2.
Abstract | BibTeX | Tags: Compliance and Impedance Control, Optimal Control | Links:
@inproceedings{Denisa2015,
title = {Generalization of discrete Compliant Movement Primitives},
author = {Miha Deni\v{s}a and Andrej Gams and Ale\v{s} Ude and Tadej Petri\v{c}},
url = {http://ieeexplore.ieee.org/document/7251512/},
doi = {10.1109/ICAR.2015.7251512},
isbn = {978-1-4673-7509-2},
year = {2015},
date = {2015-01-01},
booktitle = {2015 International Conference on Advanced Robotics (ICAR)},
number = {1},
pages = {565--572},
publisher = {IEEE},
address = {Istanbul},
abstract = {This paper addresses the problem of achieving high robot compliance while maintaining low tracking error without the use of dynamical models. The proposed approach uses programing by demonstration to learn new task related compliant movement. The presented Compliant Movement Primitives are a combination of 1) position trajectories, gained through human demonstration and encoded as Dynamical Movement Primitives and 2) corresponding torque trajectories encoded as a linear combination of radial basis functions. A set of example Compliant Movement Primitives is used with statistical generalization in order to execute previously unexplored tasks inside the training space. The proposed control approach and generalization was evaluated with a discrete pick-and-place task on a Kuka LWR robot. The evaluation showed a major decrease in tracking error compared to a classic feedback approach and no significant rise in tracking error while using generalized Compliant Movement Primitives.},
keywords = {Compliance and Impedance Control, Optimal Control},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper addresses the problem of achieving high robot compliance while maintaining low tracking error without the use of dynamical models. The proposed approach uses programing by demonstration to learn new task related compliant movement. The presented Compliant Movement Primitives are a combination of 1) position trajectories, gained through human demonstration and encoded as Dynamical Movement Primitives and 2) corresponding torque trajectories encoded as a linear combination of radial basis functions. A set of example Compliant Movement Primitives is used with statistical generalization in order to execute previously unexplored tasks inside the training space. The proposed control approach and generalization was evaluated with a discrete pick-and-place task on a Kuka LWR robot. The evaluation showed a major decrease in tracking error compared to a classic feedback approach and no significant rise in tracking error while using generalized Compliant Movement Primitives.
2014
Petrič, Tadej; Gams, Andrej; Žlajpah, Leon; Ude, Aleš; Morimoto, Jun
Online approach for altering robot behaviors based on human in the loop coaching gestures Proceedings Article
In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 4770–4776, IEEE, Hong Kong, 2014, ISSN: 10504729.
Abstract | BibTeX | Tags: Human-in-the-Loop Control, Optimal Control | Links:
@inproceedings{Petric2014a,
title = {Online approach for altering robot behaviors based on human in the loop coaching gestures},
author = {Tadej Petri\v{c} and Andrej Gams and Leon \v{Z}lajpah and Ale\v{s} Ude and Jun Morimoto},
url = {http://ieeexplore.ieee.org/document/6907557/},
doi = {10.1109/ICRA.2014.6907557},
issn = {10504729},
year = {2014},
date = {2014-01-01},
booktitle = {2014 IEEE International Conference on Robotics and Automation (ICRA)},
pages = {4770--4776},
publisher = {IEEE},
address = {Hong Kong},
abstract = {The creation and adaptation of motor behaviors is an important capability for autonomous robots. In this paper we propose an approach for altering existing robot behaviors online, where a human coach interactively changes the robot motion to achieve the desired outcome. Using hand gestures, the human coach can specify the desired modifications to the previously acquired behavior. To preserve a natural posture while performing the task, the movement is encoded in the robot's joint space using periodic dynamic movement primitives. The coaching gestures are mapped to the robot joint space via robot Jacobian and used to create a virtual force field affecting the movement. A recursive least squares technique is used to modify the existing movement with respect to the virtual force field. The proposed approach was evaluated on a simulated three degrees of freedom planar robot and on a real humanoid robot, where human coaching gestures were captured by an RGB-D sensor. Although our focus was on rhythmic movements, the developed approach is also applicable to discrete (point-to-point) movements.},
keywords = {Human-in-the-Loop Control, Optimal Control},
pubstate = {published},
tppubtype = {inproceedings}
}
The creation and adaptation of motor behaviors is an important capability for autonomous robots. In this paper we propose an approach for altering existing robot behaviors online, where a human coach interactively changes the robot motion to achieve the desired outcome. Using hand gestures, the human coach can specify the desired modifications to the previously acquired behavior. To preserve a natural posture while performing the task, the movement is encoded in the robot's joint space using periodic dynamic movement primitives. The coaching gestures are mapped to the robot joint space via robot Jacobian and used to create a virtual force field affecting the movement. A recursive least squares technique is used to modify the existing movement with respect to the virtual force field. The proposed approach was evaluated on a simulated three degrees of freedom planar robot and on a real humanoid robot, where human coaching gestures were captured by an RGB-D sensor. Although our focus was on rhythmic movements, the developed approach is also applicable to discrete (point-to-point) movements.
2013
Petrič, Tadej; Žlajpah, Leon
Smooth continuous transition between tasks on a kinematic control level: Obstacle avoidance as a control problem Journal Article
In: Robotics and Autonomous Systems, vol. 61, no. 9, pp. 948–959, 2013, ISSN: 09218890.
Abstract | BibTeX | Tags: Human-in-the-Loop Control, Optimal Control | Links:
@article{Petric2013c,
title = {Smooth continuous transition between tasks on a kinematic control level: Obstacle avoidance as a control problem},
author = {Tadej Petri\v{c} and Leon \v{Z}lajpah},
url = {http://linkinghub.elsevier.com/retrieve/pii/S0921889013000833},
doi = {10.1016/j.robot.2013.04.019},
issn = {09218890},
year = {2013},
date = {2013-01-01},
journal = {Robotics and Autonomous Systems},
volume = {61},
number = {9},
pages = {948--959},
publisher = {Elsevier B.V.},
abstract = {Kinematically redundant robots allow simultaneous execution of several tasks with different priorities. Beside the main task, obstacle avoidance is one commonly used subtask. The ability to avoid obstacles is especially important when the robot is working in a human environment. In this paper, we propose a novel control method for kinematically redundant robots, where we focus on a smooth, continuous transition between different tasks. The method is based on a new and very simple null-space formulation. Sufficient conditions for the tasks design are given using the Lyapunov-based stability discussion. The effectiveness of the proposed control method is demonstrated by simulation and on a real robot. Pros and cons of the proposed method and the comparison with other control methods are also discussed.},
keywords = {Human-in-the-Loop Control, Optimal Control},
pubstate = {published},
tppubtype = {article}
}
Kinematically redundant robots allow simultaneous execution of several tasks with different priorities. Beside the main task, obstacle avoidance is one commonly used subtask. The ability to avoid obstacles is especially important when the robot is working in a human environment. In this paper, we propose a novel control method for kinematically redundant robots, where we focus on a smooth, continuous transition between different tasks. The method is based on a new and very simple null-space formulation. Sufficient conditions for the tasks design are given using the Lyapunov-based stability discussion. The effectiveness of the proposed control method is demonstrated by simulation and on a real robot. Pros and cons of the proposed method and the comparison with other control methods are also discussed.
2010
Gams, Andrej; Petrič, Tadej; Žlajpah, Leon; Ude, Aleš
Optimizing parameters of trajectory representation for movement generalization: robotic throwing Proceedings Article
In: 19th International Workshop on Robotics in Alpe-Adria-Danube Region (RAAD 2010), pp. 161–166, IEEE, Budapest, 2010, ISBN: 978-1-4244-6885-0.
Abstract | BibTeX | Tags: Dynamic Motion, Optimal Control | Links:
@inproceedings{Gams2010,
title = {Optimizing parameters of trajectory representation for movement generalization: robotic throwing},
author = {Andrej Gams and Tadej Petri\v{c} and Leon \v{Z}lajpah and Ale\v{s} Ude},
url = {http://ieeexplore.ieee.org/document/5524592/},
doi = {10.1109/RAAD.2010.5524592},
isbn = {978-1-4244-6885-0},
year = {2010},
date = {2010-01-01},
booktitle = {19th International Workshop on Robotics in Alpe-Adria-Danube Region (RAAD 2010)},
pages = {161--166},
publisher = {IEEE},
address = {Budapest},
abstract = {For effective use of learning by imitation with a robot, it is necessary that the robot can adapt to the current state of the external world. This paper describes an optimization approach that enables the generation of a new motion trajectory, which accomplishes the task in a given situation, based on a library of example movements. New movements are generated by applying statistical methods, where the current state of the world is utilized as query into the library. Dynamic movement primitives are employed as the underlying motor representation. The main contribution of this paper is the optimization of dynamic movement primitives with respect to the kernel function positions and over the entire set of demonstrated movements. We applied the algorithm to a robotic throwing task, where the location of the target is determined by a stereo vision system, which can detect infrared markers. The vision system uses two Nintendo WIIMOTEs for cameras.},
keywords = {Dynamic Motion, Optimal Control},
pubstate = {published},
tppubtype = {inproceedings}
}
For effective use of learning by imitation with a robot, it is necessary that the robot can adapt to the current state of the external world. This paper describes an optimization approach that enables the generation of a new motion trajectory, which accomplishes the task in a given situation, based on a library of example movements. New movements are generated by applying statistical methods, where the current state of the world is utilized as query into the library. Dynamic movement primitives are employed as the underlying motor representation. The main contribution of this paper is the optimization of dynamic movement primitives with respect to the kernel function positions and over the entire set of demonstrated movements. We applied the algorithm to a robotic throwing task, where the location of the target is determined by a stereo vision system, which can detect infrared markers. The vision system uses two Nintendo WIIMOTEs for cameras.
2009
Lenart, Lado; Babič, Jan; Kušar, Janez
Direct use of functional gradients and linear programming in optimal control Proceedings Article
In: Štirn, Lidija Zadnik; Žerovnik, Janez; Drobne, Samo; Lisec, Anka (Ed.): Proceedings of the 10th International Symposium on Operational Research in Slovenia, SOR 2009, pp. 289–298, Nova Gorica, 2009, ISBN: 9789616165303.
Abstract | BibTeX | Tags: Optimal Control
@inproceedings{Lenart2009,
title = {Direct use of functional gradients and linear programming in optimal control},
author = {Lado Lenart and Jan Babi\v{c} and Janez Ku\v{s}ar},
editor = {Lidija {Zadnik \v{S}tirn} and Janez \v{Z}erovnik and Samo Drobne and Anka Lisec},
isbn = {9789616165303},
year = {2009},
date = {2009-01-01},
booktitle = {Proceedings of the 10th International Symposium on Operational Research in Slovenia, SOR 2009},
pages = {289--298},
address = {Nova Gorica},
abstract = {In the paper techniques for writing simple, transparent and effective programs for solving some problems in optimal control are presented. The calculation of functional gradients is the background for the fixed-time problems. The terminal function can be constant or else it has to be minimized. The direct algorithm is based on Euler discretization of a dynamic system and object functional. The discretized system is linearized. Then the best solution is searched using linear programming iteratively. The advantage of this algorithm is that the constraints in the control and state variables reduce the computational complexity of a non constraint problem. Typical examples illustrate the use of the described formalism. All solutions are validated with other methods.},
keywords = {Optimal Control},
pubstate = {published},
tppubtype = {inproceedings}
}
In the paper techniques for writing simple, transparent and effective programs for solving some problems in optimal control are presented. The calculation of functional gradients is the background for the fixed-time problems. The terminal function can be constant or else it has to be minimized. The direct algorithm is based on Euler discretization of a dynamic system and object functional. The discretized system is linearized. Then the best solution is searched using linear programming iteratively. The advantage of this algorithm is that the constraints in the control and state variables reduce the computational complexity of a non constraint problem. Typical examples illustrate the use of the described formalism. All solutions are validated with other methods.
2007
Lenart, Lado; Babič, Jan; Kušar, Janez
Some mixed algorithms in optimal control Proceedings Article
In: Štirn, Lidija Zadnik; Drobne, Samo (Ed.): Proceedings of the 8th International Symposium on Operational Research in Slovenia, SOR 2007, pp. 185–190, Nova Gorica, 2007.
Abstract | BibTeX | Tags: Optimal Control
@inproceedings{Lenart2007,
title = {Some mixed algorithms in optimal control},
author = {Lado Lenart and Jan Babi\v{c} and Janez Ku\v{s}ar},
editor = {Lidija {Zadnik \v{S}tirn} and Samo Drobne},
year = {2007},
date = {2007-01-01},
booktitle = {Proceedings of the 8th International Symposium on Operational Research in Slovenia, SOR 2007},
volume = {1},
pages = {185--190},
address = {Nova Gorica},
abstract = {The algorithms for optimal control system are generally split into two solution classes. The first class uses methods to directly cope with the Hamilton-Jacoby-Bellman equation (HJBE). The second class solves HJBE after transforming it into canonical system of ordinary differential equations (ODE) with split boundary values problem (BVP), also called dual point problem. Among a great number of direct methods the principles of collocation and Galerkins error estimation principle are highly interesting. The canonical equations seem to be practically the more standard way of solution, even if one has to solve two-point problem.},
keywords = {Optimal Control},
pubstate = {published},
tppubtype = {inproceedings}
}
The algorithms for optimal control system are generally split into two solution classes. The first class uses methods to directly cope with the Hamilton-Jacoby-Bellman equation (HJBE). The second class solves HJBE after transforming it into canonical system of ordinary differential equations (ODE) with split boundary values problem (BVP), also called dual point problem. Among a great number of direct methods the principles of collocation and Galerkins error estimation principle are highly interesting. The canonical equations seem to be practically the more standard way of solution, even if one has to solve two-point problem.
2005
Lenart, Lado; Babič, Jan
Dynamic programming in control Proceedings Article
In: Štirn, Lidija Zadnik; Drobne, Samo (Ed.): Proceedings of the 8th International Symposium on Operational Research in Slovenia, SOR 2005, pp. 63–68, Nova Gorica, 2005, ISBN: 9616165208 | 9789616165204.
Abstract | BibTeX | Tags: Optimal Control
@inproceedings{Lenart2005,
title = {Dynamic programming in control},
author = {Lado Lenart and Jan Babi\v{c}},
editor = {Lidija {Zadnik \v{S}tirn} and Samo Drobne},
isbn = {9616165208 | 9789616165204},
year = {2005},
date = {2005-01-01},
booktitle = {Proceedings of the 8th International Symposium on Operational Research in Slovenia, SOR 2005},
pages = {63--68},
address = {Nova Gorica},
abstract = {The use of dynamic programming (DP) in control systems is based on a relation between the Hamilton-Jacoby-Bellman equation (HJBE) and the DP. Without calculating costate variables in canonical HJBE system equations, the iteration algorithm is shown to get solution in minimizing a series of control variables.},
keywords = {Optimal Control},
pubstate = {published},
tppubtype = {inproceedings}
}
The use of dynamic programming (DP) in control systems is based on a relation between the Hamilton-Jacoby-Bellman equation (HJBE) and the DP. Without calculating costate variables in canonical HJBE system equations, the iteration algorithm is shown to get solution in minimizing a series of control variables.
Pages
Network and support
Contact
Laboratory for Neuromechanics and Biorobotics
Jožef Stefan Institute
Jamova cesta 39, SI-1000 Ljubljana, Slovenia
+386 477 3638 | jan.babic@ijs.si | https://nbr.ijs.si