2025
Arditi, Emir; Kunavar, Tjasa; Amirshirzad, Negin; Ugur, Emre; Babič, Jan; Oztop, Erhan
Inferring effort-safety trade off in perturbed squat-to-stand task by reward parameter estimation Journal Article
In: Engineering Applications of Artificial Intelligence, vol. 142, pp. 109778, 2025, ISSN: 09521976.
Abstract | BibTeX | Tags: Human Motor Control, Machine Learning, Neuromusculoskeletal Modelling, Optimal Control, Sensorimotor Learning | Links:
@article{Arditi2025,
title = {Inferring effort-safety trade off in perturbed squat-to-stand task by reward parameter estimation},
author = {Emir Arditi and Tjasa Kunavar and Negin Amirshirzad and Emre Ugur and Jan Babi\v{c} and Erhan Oztop},
doi = {10.1016/j.engappai.2024.109778},
issn = {09521976},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
journal = {Engineering Applications of Artificial Intelligence},
volume = {142},
pages = {109778},
abstract = {In this study, an inverse reinforcement learning (IRL) method is developed to estimate the parameters of a reward function that is assumed to guide the movement of a biological or artificial agent. The workings of the method is shown on the problem of estimating the effort-safety trade-off of humans during perturbed squat-to-stand motions based on their Center of Mass (COM) trajectories. The proposed method involves data generation by reinforcement learning (RL) and a novel data augmentation mechanism followed by neural network training. After the training, the neural network acts as the reward parameter estimator given the Center of Mass (COM) trajectories as input. The performance of the developed method is assessed through systematic simulation experiments, where it is shown that the parameter estimation made by our method is significantly more accurate than the baseline of an optimized template-based IRL approach. In addition, as a proof of concept, a set of human movement data is analyzed with the developed method. The results revealed that most participants acquired a strategy that ensures low effort expenditure with a safety margin, producing COM trajectories slightly away from the effort-optimal.},
keywords = {Human Motor Control, Machine Learning, Neuromusculoskeletal Modelling, Optimal Control, Sensorimotor Learning},
pubstate = {published},
tppubtype = {article}
}
In this study, an inverse reinforcement learning (IRL) method is developed to estimate the parameters of a reward function that is assumed to guide the movement of a biological or artificial agent. The workings of the method is shown on the problem of estimating the effort-safety trade-off of humans during perturbed squat-to-stand motions based on their Center of Mass (COM) trajectories. The proposed method involves data generation by reinforcement learning (RL) and a novel data augmentation mechanism followed by neural network training. After the training, the neural network acts as the reward parameter estimator given the Center of Mass (COM) trajectories as input. The performance of the developed method is assessed through systematic simulation experiments, where it is shown that the parameter estimation made by our method is significantly more accurate than the baseline of an optimized template-based IRL approach. In addition, as a proof of concept, a set of human movement data is analyzed with the developed method. The results revealed that most participants acquired a strategy that ensures low effort expenditure with a safety margin, producing COM trajectories slightly away from the effort-optimal.
2022
Fele, Benjamin; Babič, Jan; Pollak, Senja; Žnidaršič, Martin
Evaluation of curriculum learning algorithms using computational creativity inspired metrics Proceedings Article
In: Hedblom, Maria M. (Ed.): Proceedings of the Thirteenth International Conference on Computational Creativity, ICCC'22, pp. 364–373, Bolzano, Italy, 2022.
Abstract | BibTeX | Tags: AI, Machine Learning | Links:
@inproceedings{Fele2022,
title = {Evaluation of curriculum learning algorithms using computational creativity inspired metrics},
author = {Benjamin Fele and Jan Babi\v{c} and Senja Pollak and Martin \v{Z}nidar\v{s}i\v{c}},
editor = {Maria M. Hedblom},
url = {https://computationalcreativity.net/iccc22/wp-content/uploads/2022/06/ICCC-2022_5L_Fele-et-al..pdf},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the Thirteenth International Conference on Computational Creativity, ICCC'22},
pages = {364--373},
address = {Bolzano, Italy},
abstract = {Curriculum learning, especially in robotics, is an active research field aiming to devise algorithms that speed up knowledge acquisition by proposing sequences of tasks an agent should train on. We focus on curriculum generation in reinforcement learning, where various methods are currently compared based on the agent's performance in terms of rewards on a predefined distribution of target tasks. We want to extend this singular characterization of existing algorithms by introducing metrics inspired by notions from the field of computational creativity. Namely, we introduce surprise, novelty, interestingness, and typicality that quantify various aspects of tasks stochastically proposed by the curriculum learning algorithms for the learner to train on. We model proposed tasks with Gaussian mixture models which enable their probabilistic interpretation, and use Hellinger distances between distributions and training rewards in formulation of the proposed metrics. Results are presented for eight curriculum learning algorithms showcasing differences in prioritization of various aspects of task creation and statistically different mean metric values when comparing agent's best and worst training runs. The latter finding is not only useful for analysis of existing algorithms, but potentially also provides guidance for design of future curriculum learning methods.},
keywords = {AI, Machine Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
Curriculum learning, especially in robotics, is an active research field aiming to devise algorithms that speed up knowledge acquisition by proposing sequences of tasks an agent should train on. We focus on curriculum generation in reinforcement learning, where various methods are currently compared based on the agent's performance in terms of rewards on a predefined distribution of target tasks. We want to extend this singular characterization of existing algorithms by introducing metrics inspired by notions from the field of computational creativity. Namely, we introduce surprise, novelty, interestingness, and typicality that quantify various aspects of tasks stochastically proposed by the curriculum learning algorithms for the learner to train on. We model proposed tasks with Gaussian mixture models which enable their probabilistic interpretation, and use Hellinger distances between distributions and training rewards in formulation of the proposed metrics. Results are presented for eight curriculum learning algorithms showcasing differences in prioritization of various aspects of task creation and statistically different mean metric values when comparing agent's best and worst training runs. The latter finding is not only useful for analysis of existing algorithms, but potentially also provides guidance for design of future curriculum learning methods.
2019
Cevzar, Mišel; Petrič, Tadej; Jamšek, Marko; Babič, Jan
Real-Time Control of Quasi-Active Hip Exoskeleton Based on Gaussian Mixture Model Approach Book Section
In: Wearable Robotics: Challenges and Trends, pp. 244–248, Springer, Cham, 2019.
Abstract | BibTeX | Tags: Exoskeleton Design and Control, Machine Learning | Links:
@incollection{Cevzar2019,
title = {Real-Time Control of Quasi-Active Hip Exoskeleton Based on Gaussian Mixture Model Approach},
author = {Mi\v{s}el Cevzar and Tadej Petri\v{c} and Marko Jam\v{s}ek and Jan Babi\v{c}},
url = {http://link.springer.com/10.1007/978-3-030-01887-0_47},
doi = {10.1007/978-3-030-01887-0_47},
year = {2019},
date = {2019-10-01},
booktitle = {Wearable Robotics: Challenges and Trends},
pages = {244--248},
publisher = {Springer, Cham},
abstract = {Lower back pain is a major cause of disability and sick day absences. As lower back pain can result in decreased life quality as well as lower industrial productivity, multiple research groups and companies are looking into possible solutions. One of such solutions could be exoskeletons, that engage and disengage the actuators depending on the movements performed by the user. Otherwise we risk hindering the users movements and increasing his metabolic costs. We implemented an exoskeleton control using finite state machine combined with a Gaussian mixture model movement classifier. By conducting a test battery with a subject wearing the exoskeleton we were able to engage the exoskeleton actuators when appropriate and keep them disengaged to allow a full and unhindered range of motion. The results show our exoskeleton control correctly engages and disengages actuators based on the movements being performed by the user.},
keywords = {Exoskeleton Design and Control, Machine Learning},
pubstate = {published},
tppubtype = {incollection}
}
Lower back pain is a major cause of disability and sick day absences. As lower back pain can result in decreased life quality as well as lower industrial productivity, multiple research groups and companies are looking into possible solutions. One of such solutions could be exoskeletons, that engage and disengage the actuators depending on the movements performed by the user. Otherwise we risk hindering the users movements and increasing his metabolic costs. We implemented an exoskeleton control using finite state machine combined with a Gaussian mixture model movement classifier. By conducting a test battery with a subject wearing the exoskeleton we were able to engage the exoskeleton actuators when appropriate and keep them disengaged to allow a full and unhindered range of motion. The results show our exoskeleton control correctly engages and disengages actuators based on the movements being performed by the user.
Arditi, Emir; Čamernik, Jernej; Babič, Jan; Ugur, Emre; Nagai, Yukie; Oztop, Erhan
Explorations on inverse reinforcement learning for the analysis of sensorimotor data Proceedings Article
In: Winter Workshop on Mechanism of Brain and Mind 2019, Rusutsu, 9.-11.1. 2019, Rusutsu, 2019.
Abstract | BibTeX | Tags: Human Motor Control, Machine Learning, Sensorimotor Learning
@inproceedings{Arditi2019,
title = {Explorations on inverse reinforcement learning for the analysis of sensorimotor data},
author = {Emir Arditi and Jernej \v{C}amernik and Jan Babi\v{c} and Emre Ugur and Yukie Nagai and Erhan Oztop},
year = {2019},
date = {2019-01-01},
booktitle = {Winter Workshop on Mechanism of Brain and Mind 2019, Rusutsu, 9.-11.1. 2019},
address = {Rusutsu},
abstract = {We set out to explore whether recent Reinforcement Learning methods can serve as a computational tool for investigating optimality principles of motor control and cognitive decision making mechanisms of the brain. For this purpose, we have targeted two different tasks. One includes full body motion which involves possibility of injury and fall; whereas the other involves playing a simple computer game requiring prediction and fast decision making. If we can recover optimality principles employed by the brain for various control and decision making tasks, we can analyse experimental data with less bias, and hope to gain more insights than possible with classical methods.},
keywords = {Human Motor Control, Machine Learning, Sensorimotor Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
We set out to explore whether recent Reinforcement Learning methods can serve as a computational tool for investigating optimality principles of motor control and cognitive decision making mechanisms of the brain. For this purpose, we have targeted two different tasks. One includes full body motion which involves possibility of injury and fall; whereas the other involves playing a simple computer game requiring prediction and fast decision making. If we can recover optimality principles employed by the brain for various control and decision making tasks, we can analyse experimental data with less bias, and hope to gain more insights than possible with classical methods.
2018
Cevzar, Mišel; Petrič, Tadej; Babič, Jan
Open Source EMG Device for Controlling a Robotic Hand Book Section
In: Ferraresi, Carlo; Quaglia, Giuseppe (Ed.): Advances in Service and Industrial Robotics: Proceedings of the 26th International Conference on Robotics in Alpe-Adria-Danube Region, RAAD 2017, pp. 797–804, Springer International Publishing, Cham, 2018, ISBN: 978-3-319-61276-8.
Abstract | BibTeX | Tags: Machine Learning, Muscle Mechanics | Links:
@incollection{Cevzar2018,
title = {Open Source EMG Device for Controlling a Robotic Hand},
author = {Mi\v{s}el Cevzar and Tadej Petri\v{c} and Jan Babi\v{c}},
editor = {Carlo Ferraresi and Giuseppe Quaglia},
url = {http://link.springer.com/10.1007/978-3-319-61276-8_84},
doi = {10.1007/978-3-319-61276-8_84},
isbn = {978-3-319-61276-8},
year = {2018},
date = {2018-01-01},
booktitle = {Advances in Service and Industrial Robotics: Proceedings of the 26th International Conference on Robotics in Alpe-Adria-Danube Region, RAAD 2017},
pages = {797--804},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Off-the-shelf electronic market is large, diverse and easily accessible by many. Credit card size computers (example: Raspberry Pi) or micro-controller boards (example: Arduino) can be used for learning how to code and how to control embedded systems. Nevertheless, there is a lack of off-the-shelf, open source devices that would enable us to learn about and make use of human signal processing. An example of such a device is an electromyograph (EMG). In this paper we investigated, if an EMG device could fulfill the aforementioned gap. EMG device we used for conducting our experiment was a five channel open source EMG Arduino shield. The performance of the device was evaluated on three healthy male subjects. They were instructed to perform basic finger movements which we classified and executed on the robotic hand. The EMG signal classification was performed using a Support Vector Machine (SVM) algorithm. In our experimental setup the average EMG signal classification accuracy was 78.29%. This we believe demonstrates there are EMG devices on the market today that provide access to cost effective prototyping and learning about EMG signals.},
keywords = {Machine Learning, Muscle Mechanics},
pubstate = {published},
tppubtype = {incollection}
}
Off-the-shelf electronic market is large, diverse and easily accessible by many. Credit card size computers (example: Raspberry Pi) or micro-controller boards (example: Arduino) can be used for learning how to code and how to control embedded systems. Nevertheless, there is a lack of off-the-shelf, open source devices that would enable us to learn about and make use of human signal processing. An example of such a device is an electromyograph (EMG). In this paper we investigated, if an EMG device could fulfill the aforementioned gap. EMG device we used for conducting our experiment was a five channel open source EMG Arduino shield. The performance of the device was evaluated on three healthy male subjects. They were instructed to perform basic finger movements which we classified and executed on the robotic hand. The EMG signal classification was performed using a Support Vector Machine (SVM) algorithm. In our experimental setup the average EMG signal classification accuracy was 78.29%. This we believe demonstrates there are EMG devices on the market today that provide access to cost effective prototyping and learning about EMG signals.
2016
Peternel, Luka; Oztop, Erhan; Babič, Jan
A shared control method for online human-in-the-loop robot learning based on Locally Weighted Regression Proceedings Article
In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3900–3906, IEEE, Daejeon, 2016, ISBN: 978-1-5090-3762-9.
Abstract | BibTeX | Tags: Compliance and Impedance Control, Human-in-the-Loop Control, Machine Learning | Links:
@inproceedings{Peternel2016,
title = {A shared control method for online human-in-the-loop robot learning based on Locally Weighted Regression},
author = {Luka Peternel and Erhan Oztop and Jan Babi\v{c}},
url = {http://ieeexplore.ieee.org/document/7759574/},
doi = {10.1109/IROS.2016.7759574},
isbn = {978-1-5090-3762-9},
year = {2016},
date = {2016-01-01},
booktitle = {2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
pages = {3900--3906},
publisher = {IEEE},
address = {Daejeon},
abstract = {In this paper we propose a novel control approach for robots that are physically coupled with humans, such as exoskeletons and assisting devices. In contrast with the conventional controllers, where assistance is usually provided indifferently of the configuration of the arm and direction of the motion, we propose a control method that compensates the anisotropic property of the manipulability related to the human arm. Consequently, the assistive behaviour of the proposed method allows the user to perform tasks in arm configurations that are otherwise unsuitable due to the lack of manipulability. In effect, the proposed method transforms the elliptic shape of the force manipulability ellipse to a circular shape in the whole space of human arm configurations. The proposed approach was evaluated by the arm-reaching task that involved pushing of a heavy object on a plane. We tested the accuracy and efficiency of the proposed method under several conditions that involved motions with added weight and friction. The results of our study show that the proposed approach significantly improves the human motor control ability and maintains the desired accuracy of the movement.},
keywords = {Compliance and Impedance Control, Human-in-the-Loop Control, Machine Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper we propose a novel control approach for robots that are physically coupled with humans, such as exoskeletons and assisting devices. In contrast with the conventional controllers, where assistance is usually provided indifferently of the configuration of the arm and direction of the motion, we propose a control method that compensates the anisotropic property of the manipulability related to the human arm. Consequently, the assistive behaviour of the proposed method allows the user to perform tasks in arm configurations that are otherwise unsuitable due to the lack of manipulability. In effect, the proposed method transforms the elliptic shape of the force manipulability ellipse to a circular shape in the whole space of human arm configurations. The proposed approach was evaluated by the arm-reaching task that involved pushing of a heavy object on a plane. We tested the accuracy and efficiency of the proposed method under several conditions that involved motions with added weight and friction. The results of our study show that the proposed approach significantly improves the human motor control ability and maintains the desired accuracy of the movement.
Petrič, Tadej; Ude, Aleš; Ijspeert, Auke J
Autonomous Learning of Internal Dynamic Models for Reaching Tasks Book Section
In: Borangiu, Theodor (Ed.): Advances in Intelligent Systems and Computing, vol. 371, pp. 439–447, Springer, 2016, ISSN: 21945357.
Abstract | BibTeX | Tags: Compliance and Impedance Control, Human Motor Control, Machine Learning | Links:
@incollection{Petric2016c,
title = {Autonomous Learning of Internal Dynamic Models for Reaching Tasks},
author = {Tadej Petri\v{c} and Ale\v{s} Ude and Auke J Ijspeert},
editor = {Theodor Borangiu},
url = {http://link.springer.com/10.1007/978-3-319-21290-6_44},
doi = {10.1007/978-3-319-21290-6_44},
issn = {21945357},
year = {2016},
date = {2016-01-01},
booktitle = {Advances in Intelligent Systems and Computing},
volume = {371},
pages = {439--447},
publisher = {Springer},
abstract = {The paper addresses the problem of learning internal task-specific dynamic models for a reaching task. Using task-specific dynamic models is crucial for achieving both high tracking accuracy and compliant behaviour, which improves safety concerns while working in unstructured environment or with humans. The proposed approach uses programming by demonstration to learn new task-related movements encoded as Compliant Movement Primitives (CMPs). CMPs are a combination of position trajectories encoded in a form of Dynamic Movement Primitives (DMPs) and corresponding task-specific Torque Primitives (TPs) encoded as a linear combination of kernel functions. Unlike the DMPs, TPs cannot be directly acquired from user demonstrations. Inspired by the human sensorimotor learning ability we propose a novel method which autonomously learns task-specific TPs, based on a given kinematic trajectory in DMPs.},
keywords = {Compliance and Impedance Control, Human Motor Control, Machine Learning},
pubstate = {published},
tppubtype = {incollection}
}
The paper addresses the problem of learning internal task-specific dynamic models for a reaching task. Using task-specific dynamic models is crucial for achieving both high tracking accuracy and compliant behaviour, which improves safety concerns while working in unstructured environment or with humans. The proposed approach uses programming by demonstration to learn new task-related movements encoded as Compliant Movement Primitives (CMPs). CMPs are a combination of position trajectories encoded in a form of Dynamic Movement Primitives (DMPs) and corresponding task-specific Torque Primitives (TPs) encoded as a linear combination of kernel functions. Unlike the DMPs, TPs cannot be directly acquired from user demonstrations. Inspired by the human sensorimotor learning ability we propose a novel method which autonomously learns task-specific TPs, based on a given kinematic trajectory in DMPs.
Deniša, Miha; Gams, Andrej; Ude, Aleš; Petrič, Tadej
Learning Compliant Movement Primitives Through Demonstration and Statistical Generalization Journal Article
In: IEEE/ASME Transactions on Mechatronics, vol. 21, no. 5, pp. 2581–2594, 2016, ISSN: 1083-4435.
Abstract | BibTeX | Tags: Compliance and Impedance Control, Machine Learning | Links:
@article{Denisa2016,
title = {Learning Compliant Movement Primitives Through Demonstration and Statistical Generalization},
author = {Miha Deni\v{s}a and Andrej Gams and Ale\v{s} Ude and Tadej Petri\v{c}},
url = {http://ieeexplore.ieee.org/document/7360201/},
doi = {10.1109/TMECH.2015.2510165},
issn = {1083-4435},
year = {2016},
date = {2016-01-01},
journal = {IEEE/ASME Transactions on Mechatronics},
volume = {21},
number = {5},
pages = {2581--2594},
abstract = {In this paper, we address the problem of simultaneously achieving low trajectory tracking errors and compliant control without using explicit mathematical models of task dynamics. To achieve this goal, we propose a new movement representation called compliant movement primitives (CMPs), which encodes position trajectory and associated torque profiles and can be learned from a single user demonstration. With the proposed control framework, the robot can remain compliant and consequently safe for humans sharing its workspace, even if high trajectory tracking accuracy is required. We developed a statistical learning approach that can use a database of existing CMPs and compute new ones, adapted for novel task variations. The proposed approach was evaluated on a Kuka LWR-4 robot performing 1) a discrete pick-and-place task with objects of varying weight and 2) a periodic handle turning operation. The evaluation of the discrete task showed a 15-fold decrease of the tracking error while exhibiting compliant behavior compared to the standard feedback control approach. It also indicated no significant rise in the tracking error while using generalized primitives computed by the statistical learning method. With respect to unforeseen collisions, the proposed approach resulted in a 75% drop of contact forces compared to standard feedback control. The periodic task demonstrated on-line use of the proposed approach to accomplish a task of handle turning.},
keywords = {Compliance and Impedance Control, Machine Learning},
pubstate = {published},
tppubtype = {article}
}
In this paper, we address the problem of simultaneously achieving low trajectory tracking errors and compliant control without using explicit mathematical models of task dynamics. To achieve this goal, we propose a new movement representation called compliant movement primitives (CMPs), which encodes position trajectory and associated torque profiles and can be learned from a single user demonstration. With the proposed control framework, the robot can remain compliant and consequently safe for humans sharing its workspace, even if high trajectory tracking accuracy is required. We developed a statistical learning approach that can use a database of existing CMPs and compute new ones, adapted for novel task variations. The proposed approach was evaluated on a Kuka LWR-4 robot performing 1) a discrete pick-and-place task with objects of varying weight and 2) a periodic handle turning operation. The evaluation of the discrete task showed a 15-fold decrease of the tracking error while exhibiting compliant behavior compared to the standard feedback control approach. It also indicated no significant rise in the tracking error while using generalized primitives computed by the statistical learning method. With respect to unforeseen collisions, the proposed approach resulted in a 75% drop of contact forces compared to standard feedback control. The periodic task demonstrated on-line use of the proposed approach to accomplish a task of handle turning.
2015
Petrič, Tadej; Colasanto, Luca; Gams, Andrej; Ude, Aleš; Ijspeert, Auke J
Bio-inspired learning and database expansion of Compliant Movement Primitives Proceedings Article
In: 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 346–351, IEEE, Seoul, 2015, ISSN: 21640580.
Abstract | BibTeX | Tags: Compliance and Impedance Control, Kinematics, Machine Learning | Links:
@inproceedings{Petric2015,
title = {Bio-inspired learning and database expansion of Compliant Movement Primitives},
author = {Tadej Petri\v{c} and Luca Colasanto and Andrej Gams and Ale\v{s} Ude and Auke J Ijspeert},
url = {http://ieeexplore.ieee.org/document/7363573/},
doi = {10.1109/HUMANOIDS.2015.7363573},
issn = {21640580},
year = {2015},
date = {2015-11-01},
booktitle = {2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids)},
pages = {346--351},
publisher = {IEEE},
address = {Seoul},
abstract = {The paper addresses the problem of learning torque primitives - the torques associated to a kinematic trajectory, and required in order to accurately track this kinematic trajectory. Learning torque primitives, which can be interpreted as internal dynamic models, is crucial to achieve at the same time (1) high tracking accuracy and (2) compliant behaviour. The latter improves the safety concerns of working in unstructured environments or with humans. In the proposed approach, first learning by demonstration is used to obtain the kinematic trajectories, which are encoded in the form of Dynamic Movement Primitives (DMPs). These are combined with the corresponding task-specific Torque Primitives (TPs), and together they form new task-related compliant movements, denoted as Compliant Movement Primitives (CMPs). Unlike the DMPs, the TPs cannot be directly acquired from user demonstrations. Inspired by the human sensorimotor learning ability, we propose a novel method which can autonomously learn task-specific Torque Primitives (TPs) associated to given kinematic trajectories in the form of DMPs. The proposed algorithm is completely autonomous, and can be used to rapidly generate and expand the database of CMPs motions. Since the CMPs are parameterized, statistical generalisation can be used to obtain an initial TP estimate of a new CMP motion. Thereby, the learning rate of new CMPs can be significantly improved. The evaluation of the proposed approach on a humanoid robot CoMan performing reaching task shows fast TP acquisition and accurate generalization estimates in real-world scenarios.},
keywords = {Compliance and Impedance Control, Kinematics, Machine Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
The paper addresses the problem of learning torque primitives - the torques associated to a kinematic trajectory, and required in order to accurately track this kinematic trajectory. Learning torque primitives, which can be interpreted as internal dynamic models, is crucial to achieve at the same time (1) high tracking accuracy and (2) compliant behaviour. The latter improves the safety concerns of working in unstructured environments or with humans. In the proposed approach, first learning by demonstration is used to obtain the kinematic trajectories, which are encoded in the form of Dynamic Movement Primitives (DMPs). These are combined with the corresponding task-specific Torque Primitives (TPs), and together they form new task-related compliant movements, denoted as Compliant Movement Primitives (CMPs). Unlike the DMPs, the TPs cannot be directly acquired from user demonstrations. Inspired by the human sensorimotor learning ability, we propose a novel method which can autonomously learn task-specific Torque Primitives (TPs) associated to given kinematic trajectories in the form of DMPs. The proposed algorithm is completely autonomous, and can be used to rapidly generate and expand the database of CMPs motions. Since the CMPs are parameterized, statistical generalisation can be used to obtain an initial TP estimate of a new CMP motion. Thereby, the learning rate of new CMPs can be significantly improved. The evaluation of the proposed approach on a humanoid robot CoMan performing reaching task shows fast TP acquisition and accurate generalization estimates in real-world scenarios.
Peternel, Luka; Petrič, Tadej; Babič, Jan
Human-in-the-loop approach for teaching robot assembly tasks using impedance control interface Proceedings Article
In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 1497–1502, IEEE, Seattle, 2015, ISSN: 10504729.
Abstract | BibTeX | Tags: Compliance and Impedance Control, Human-in-the-Loop Control, Machine Learning | Links:
@inproceedings{Peternel2015a,
title = {Human-in-the-loop approach for teaching robot assembly tasks using impedance control interface},
author = {Luka Peternel and Tadej Petri\v{c} and Jan Babi\v{c}},
url = {http://ieeexplore.ieee.org/document/7139387/},
doi = {10.1109/ICRA.2015.7139387},
issn = {10504729},
year = {2015},
date = {2015-01-01},
booktitle = {2015 IEEE International Conference on Robotics and Automation (ICRA)},
pages = {1497--1502},
publisher = {IEEE},
address = {Seattle},
abstract = {In this paper we propose a human-in-the-loop approach for teaching robots how to solve part assembly tasks. In the proposed setup the human tutor controls the robot through a haptic interface and a hand-held impedance control interface. The impedance control interface is based on a linear spring-return potentiometer that maps the button position to the robot arm stiffness. This setup allows the tutor to modulate the robot compliance based on the given task requirements. The demonstrated motion and stiffness trajectories are encoded using Dynamical Movement Primitives and learnt using Locally Weight Regression. To validate the proposed approach we performed experiments using Kuka Light Weight Robot and HapticMaster robot. The task of the experiment was to teach the robot how to perform an assembly task involving sliding a bolt fitting inside a groove in order to mount two parts together. Different stiffness was required in different stages of the task execution to accommodate the interaction of the robot with the environment and possible human-robot cooperation.},
keywords = {Compliance and Impedance Control, Human-in-the-Loop Control, Machine Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper we propose a human-in-the-loop approach for teaching robots how to solve part assembly tasks. In the proposed setup the human tutor controls the robot through a haptic interface and a hand-held impedance control interface. The impedance control interface is based on a linear spring-return potentiometer that maps the button position to the robot arm stiffness. This setup allows the tutor to modulate the robot compliance based on the given task requirements. The demonstrated motion and stiffness trajectories are encoded using Dynamical Movement Primitives and learnt using Locally Weight Regression. To validate the proposed approach we performed experiments using Kuka Light Weight Robot and HapticMaster robot. The task of the experiment was to teach the robot how to perform an assembly task involving sliding a bolt fitting inside a groove in order to mount two parts together. Different stiffness was required in different stages of the task execution to accommodate the interaction of the robot with the environment and possible human-robot cooperation.
Nemec, Bojan; Petrič, Tadej; Ude, Aleš
Force adaptation with recursive regression Iterative Learning Controller Proceedings Article
In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2835–2841, IEEE, Hamburg, 2015, ISBN: 978-1-4799-9994-1.
Abstract | BibTeX | Tags: Force Control, Machine Learning | Links:
@inproceedings{Nemec2015,
title = {Force adaptation with recursive regression Iterative Learning Controller},
author = {Bojan Nemec and Tadej Petri\v{c} and Ale\v{s} Ude},
url = {http://ieeexplore.ieee.org/document/7353767/},
doi = {10.1109/IROS.2015.7353767},
isbn = {978-1-4799-9994-1},
year = {2015},
date = {2015-01-01},
booktitle = {2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
pages = {2835--2841},
publisher = {IEEE},
address = {Hamburg},
abstract = {In this paper we exploit Iterative Learning Controllers (ILC) schemes in force adaptation tasks. We propose to encode the control signal with Radial Basis Functions (RBF), which enhances the robustness of the ILC scheme and allows to vary the execution speed of the learned motion. For that a novel control scheme is proposed, which updates the feedforward compensation signals based on current iteration cycle signals in contrast to the standard ILC, which uses signals from the previous iteration cycle. This reduces the computational burden and enhances the adaptation speed. Stability of the proposed control law is analysed and discussed. The proposed approach is evaluated in simulation and on a Kuka Light Weight Robot Arm where the task was to perform force-based surface following with both discrete and periodic movements.},
keywords = {Force Control, Machine Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper we exploit Iterative Learning Controllers (ILC) schemes in force adaptation tasks. We propose to encode the control signal with Radial Basis Functions (RBF), which enhances the robustness of the ILC scheme and allows to vary the execution speed of the learned motion. For that a novel control scheme is proposed, which updates the feedforward compensation signals based on current iteration cycle signals in contrast to the standard ILC, which uses signals from the previous iteration cycle. This reduces the computational burden and enhances the adaptation speed. Stability of the proposed control law is analysed and discussed. The proposed approach is evaluated in simulation and on a Kuka Light Weight Robot Arm where the task was to perform force-based surface following with both discrete and periodic movements.
2014
Nemec, Bojan; Petrič, Tadej; Babič, Jan; Supej, Matej
Estimation of Alpine Skier Posture Using Machine Learning Techniques Journal Article
In: Sensors, vol. 14, no. 10, pp. 18898–18914, 2014, ISSN: 1424-8220.
Abstract | BibTeX | Tags: Machine Learning, Postural Balance, Sport | Links:
@article{Nemec2014,
title = {Estimation of Alpine Skier Posture Using Machine Learning Techniques},
author = {Bojan Nemec and Tadej Petri\v{c} and Jan Babi\v{c} and Matej Supej},
url = {https://www.mdpi.com/1424-8220/14/10/18898},
doi = {10.3390/s141018898},
issn = {1424-8220},
year = {2014},
date = {2014-10-01},
journal = {Sensors},
volume = {14},
number = {10},
pages = {18898--18914},
abstract = {High precision Global Navigation Satellite System (GNSS) measurements are becoming more and more popular in alpine skiing due to the relatively undemanding setup and excellent performance. However, GNSS provides only single-point measurements that are defined with the antenna placed typically behind the skier's neck. A key issue is how to estimate other more relevant parameters of the skier's body, like the center of mass (COM) and ski trajectories. Previously, these parameters were estimated by modeling the skier's body with an inverted-pendulum model that oversimplified the skier's body. In this study, we propose two machine learning methods that overcome this shortcoming and estimate COM and skis trajectories based on a more faithful approximation of the skier's body with nine degrees-of-freedom. The first method utilizes a well-established approach of artificial neural networks, while the second method is based on a state-of-the-art statistical generalization method. Both methods were evaluated using the reference measurements obtained on a typical giant slalom course and compared with the inverted-pendulum method. Our results outperform the results of commonly used inverted-pendulum methods and demonstrate the applicability of machine learning techniques in biomechanical measurements of alpine skiing.},
keywords = {Machine Learning, Postural Balance, Sport},
pubstate = {published},
tppubtype = {article}
}
High precision Global Navigation Satellite System (GNSS) measurements are becoming more and more popular in alpine skiing due to the relatively undemanding setup and excellent performance. However, GNSS provides only single-point measurements that are defined with the antenna placed typically behind the skier's neck. A key issue is how to estimate other more relevant parameters of the skier's body, like the center of mass (COM) and ski trajectories. Previously, these parameters were estimated by modeling the skier's body with an inverted-pendulum model that oversimplified the skier's body. In this study, we propose two machine learning methods that overcome this shortcoming and estimate COM and skis trajectories based on a more faithful approximation of the skier's body with nine degrees-of-freedom. The first method utilizes a well-established approach of artificial neural networks, while the second method is based on a state-of-the-art statistical generalization method. Both methods were evaluated using the reference measurements obtained on a typical giant slalom course and compared with the inverted-pendulum method. Our results outperform the results of commonly used inverted-pendulum methods and demonstrate the applicability of machine learning techniques in biomechanical measurements of alpine skiing.
Peternel, Luka; Petrič, Tadej; Oztop, Erhan; Babič, Jan
Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach Journal Article
In: Autonomous Robots, vol. 36, no. 1-2, pp. 123–136, 2014, ISSN: 0929-5593.
Abstract | BibTeX | Tags: Compliance and Impedance Control, Human-in-the-Loop Control, Machine Learning, Sensorimotor Learning | Links:
@article{Peternel2014b,
title = {Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach},
author = {Luka Peternel and Tadej Petri\v{c} and Erhan Oztop and Jan Babi\v{c}},
url = {http://link.springer.com/10.1007/s10514-013-9361-0},
doi = {10.1007/s10514-013-9361-0},
issn = {0929-5593},
year = {2014},
date = {2014-01-01},
journal = {Autonomous Robots},
volume = {36},
number = {1-2},
pages = {123--136},
abstract = {We propose an approach to efficiently teach robots how to perform dynamic manipulation tasks in cooperation with a human partner. The approach utilises human sensorimotor learning ability where the human tutor controls the robot through a multi-modal interface to make it perform the desired task. During the tutoring, the robot simultaneously learns the action policy of the tutor and through time gains full autonomy. We demonstrate our approach by an experiment where we taught a robot how to perform a wood sawing task with a human partner using a two-person cross-cut saw. The challenge of this experiment is that it requires precise coordination of the robot's motion and compliance according to the partner's actions. To transfer the sawing skill from the tutor to the robot we used Locally Weighted Regression for trajectory generalisation, and adaptive oscillators for adaptation of the robot to the partner's motion.},
keywords = {Compliance and Impedance Control, Human-in-the-Loop Control, Machine Learning, Sensorimotor Learning},
pubstate = {published},
tppubtype = {article}
}
We propose an approach to efficiently teach robots how to perform dynamic manipulation tasks in cooperation with a human partner. The approach utilises human sensorimotor learning ability where the human tutor controls the robot through a multi-modal interface to make it perform the desired task. During the tutoring, the robot simultaneously learns the action policy of the tutor and through time gains full autonomy. We demonstrate our approach by an experiment where we taught a robot how to perform a wood sawing task with a human partner using a two-person cross-cut saw. The challenge of this experiment is that it requires precise coordination of the robot's motion and compliance according to the partner's actions. To transfer the sawing skill from the tutor to the robot we used Locally Weighted Regression for trajectory generalisation, and adaptive oscillators for adaptation of the robot to the partner's motion.
Gams, Andrej; Petrič, Tadej; Nemec, Bojan; Ude, Aleš
Learning and adaptation of periodic motion primitives based on force feedback and human coaching interaction Proceedings Article
In: 2014 IEEE-RAS International Conference on Humanoid Robots, pp. 166–171, IEEE, Madrid, 2014, ISBN: 978-1-4799-7174-9.
Abstract | BibTeX | Tags: Force Control, Human-in-the-Loop Control, Machine Learning | Links:
@inproceedings{Gams2014,
title = {Learning and adaptation of periodic motion primitives based on force feedback and human coaching interaction},
author = {Andrej Gams and Tadej Petri\v{c} and Bojan Nemec and Ale\v{s} Ude},
url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7041354},
doi = {10.1109/HUMANOIDS.2014.7041354},
isbn = {978-1-4799-7174-9},
year = {2014},
date = {2014-01-01},
booktitle = {2014 IEEE-RAS International Conference on Humanoid Robots},
pages = {166--171},
publisher = {IEEE},
address = {Madrid},
abstract = {Dynamic movement primitives (DMP) allow efficient learning and control of complex robot behaviors for both periodic and discrete point-to-point movements either in joint or Cartesian space. They also allow efficient modulation by changing of parameters. In this paper we introduce and evaluate the means of adapting periodic DMP trajectories with respect to force feedback. We simultaneously consider two aspects: 1) adaptation of whole trajectories to comply with the constraints set by the environment; and 2) partially modifying the trajectories during the execution based on human intervention to improve the task performance. The latter can either be force-based, i. e. through physical contact, or through predefined gestures. By intervening when necessary the human acts as a tutor, instructing the robot how to modify the trajectory and bypassing the need to learn new trajectories by autonomous exploration. We introduce the approach in the context of wiping a surface, where the robot first has to acquire and maintain contact, and where later the human tutor modifies the originally learned trajectory in order to achieve the desired robot behavior. We present simulation and real world results of wiping a surface with a Kuka 7 degree-of-freedom LWR robot.},
keywords = {Force Control, Human-in-the-Loop Control, Machine Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
Dynamic movement primitives (DMP) allow efficient learning and control of complex robot behaviors for both periodic and discrete point-to-point movements either in joint or Cartesian space. They also allow efficient modulation by changing of parameters. In this paper we introduce and evaluate the means of adapting periodic DMP trajectories with respect to force feedback. We simultaneously consider two aspects: 1) adaptation of whole trajectories to comply with the constraints set by the environment; and 2) partially modifying the trajectories during the execution based on human intervention to improve the task performance. The latter can either be force-based, i. e. through physical contact, or through predefined gestures. By intervening when necessary the human acts as a tutor, instructing the robot how to modify the trajectory and bypassing the need to learn new trajectories by autonomous exploration. We introduce the approach in the context of wiping a surface, where the robot first has to acquire and maintain contact, and where later the human tutor modifies the originally learned trajectory in order to achieve the desired robot behavior. We present simulation and real world results of wiping a surface with a Kuka 7 degree-of-freedom LWR robot.
2011
Babič, Jan
Utilization of human sensorimotor learning capacity for obtaining novel robot behaviours: 2-DOF ball balancing on a parallel platform Proceedings Article
In: 2011 IEEE International Conference on Robotics and Biomimetics, pp. 1778–1782, IEEE, Karon Beach, 2011, ISBN: 978-1-4577-2138-0.
Abstract | BibTeX | Tags: Ergonomy, Machine Learning, Sensorimotor Learning | Links:
@inproceedings{Babic2011a,
title = {Utilization of human sensorimotor learning capacity for obtaining novel robot behaviours: 2-DOF ball balancing on a parallel platform},
author = {Jan Babi\v{c}},
url = {http://ieeexplore.ieee.org/document/6181547/},
doi = {10.1109/ROBIO.2011.6181547},
isbn = {978-1-4577-2138-0},
year = {2011},
date = {2011-12-01},
booktitle = {2011 IEEE International Conference on Robotics and Biomimetics},
pages = {1778--1782},
publisher = {IEEE},
address = {Karon Beach},
abstract = {We propose a novel method for acquiring robot skill synthesis that exploits human capability to learn and to control complex tools and devices. The basic idea is to consider the target robotic system as a complex tool akin a computer mouse or a car that can be intuitively controlled by a human. Once the human learns how to control the robotic system effortlessly, the target behaviour can be obtained by the human on the robot through practice. The successful execution of the desired task by the human via the robot implies that the required control commands have been discovered by the human, and can subsequently be used for designing controller that operate autonomously. In this article, we demonstrate this idea by presenting a 2-DOF ball balancing task on a parallel platform obtained with the proposed method.},
keywords = {Ergonomy, Machine Learning, Sensorimotor Learning},
pubstate = {published},
tppubtype = {inproceedings}
}
We propose a novel method for acquiring robot skill synthesis that exploits human capability to learn and to control complex tools and devices. The basic idea is to consider the target robotic system as a complex tool akin a computer mouse or a car that can be intuitively controlled by a human. Once the human learns how to control the robotic system effortlessly, the target behaviour can be obtained by the human on the robot through practice. The successful execution of the desired task by the human via the robot implies that the required control commands have been discovered by the human, and can subsequently be used for designing controller that operate autonomously. In this article, we demonstrate this idea by presenting a 2-DOF ball balancing task on a parallel platform obtained with the proposed method.
2008
Oztop, Erhan; Babič, Jan; Hale, Joshua; Cheng, Gordon; Kawato, Mitsuo
From Biologically Realistic Imitation to Robot Teaching Via Human Motor Learning Book Section
In: Ishikawa, Masumi; Doya, Kenji; Miyamoto, Hiroyuki; Yamakawa, Takeshi (Ed.): Neural Information Processing, vol. 4985, pp. 214–221, Springer, Berlin, 2008, ISSN: 0302-9743.
Abstract | BibTeX | Tags: Human-in-the-Loop Control, Machine Learning, Sensorimotor Learning | Links:
@incollection{Oztop2008,
title = {From Biologically Realistic Imitation to Robot Teaching Via Human Motor Learning},
author = {Erhan Oztop and Jan Babi\v{c} and Joshua Hale and Gordon Cheng and Mitsuo Kawato},
editor = {Masumi Ishikawa and Kenji Doya and Hiroyuki Miyamoto and Takeshi Yamakawa},
url = {http://link.springer.com/10.1007/978-3-540-69162-4_23},
doi = {10.1007/978-3-540-69162-4_23},
issn = {0302-9743},
year = {2008},
date = {2008-01-01},
booktitle = {Neural Information Processing},
volume = {4985},
pages = {214--221},
publisher = {Springer},
address = {Berlin},
series = {Lecture Notes in Computer Science},
abstract = {Understanding mechanisms of imitation is a complex task in both human sciences and robotics. On the one hand, one can build systems that analyze observed motion, map it to their own body, and produce the motor commands to needed to achieve the inferred motion using engineering techniques. On the other hand, one can model the neural circuits involved in action observation and production in minute detail and hope that imitation will be an emergent property of the system. However if the goal is to build robots capable of skillful actions, midway solutions appear to be more appropriate. In this direction, we first introduce a conceptually biologically realistic neural network that can learn to imitate hand postures, either with the help of a teacher or by self-observation. Then we move to a paradigm we have recently proposed, where robot skill synthesis is achieved by exploiting the human capacity to learn novel control tasks.},
keywords = {Human-in-the-Loop Control, Machine Learning, Sensorimotor Learning},
pubstate = {published},
tppubtype = {incollection}
}
Understanding mechanisms of imitation is a complex task in both human sciences and robotics. On the one hand, one can build systems that analyze observed motion, map it to their own body, and produce the motor commands to needed to achieve the inferred motion using engineering techniques. On the other hand, one can model the neural circuits involved in action observation and production in minute detail and hope that imitation will be an emergent property of the system. However if the goal is to build robots capable of skillful actions, midway solutions appear to be more appropriate. In this direction, we first introduce a conceptually biologically realistic neural network that can learn to imitate hand postures, either with the help of a teacher or by self-observation. Then we move to a paradigm we have recently proposed, where robot skill synthesis is achieved by exploiting the human capacity to learn novel control tasks.
Pages
Network and support
Contact
Laboratory for Neuromechanics and Biorobotics
Jožef Stefan Institute
Jamova cesta 39, SI-1000 Ljubljana, Slovenia
+386 477 3638 | jan.babic@ijs.si | https://nbr.ijs.si