Benjamin Fele
PhD Student
Email: benjamin.fele@ijs.si
Education
MSc in Cognitive Science, University of Ljubljana (2020)
BSc in Computer and Information Science, University of Ljubljana (2017)
Research interests
- reinforcement learning
- meta-learning
- cognitive modeling
- developmental robotics
- generative modeling
Selected Publications
Fele, Benjamin; Babič, Jan
Curriculum Learning Algorithms for Reward Weighting in Sparse Reward Robotic Manipulation Tasks Journal Article
In: IEEE Access, vol. 13, pp. 45544-45558, 2025, ISSN: 2169-3536.
@article{Fele2025,
title = {Curriculum Learning Algorithms for Reward Weighting in Sparse Reward Robotic Manipulation Tasks},
author = {Benjamin Fele and Jan Babi\v{c}},
doi = {10.1109/ACCESS.2025.3549639},
issn = {2169-3536},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
journal = {IEEE Access},
volume = {13},
pages = {45544-45558},
abstract = {Robotic learning from sparse rewards can be a considerable challenge due to large amounts of data required for mastering a task. We explore the application of curriculum learning (CL) algorithms for automatic reward weighting to tackle learning from sparse rewards in robotic pick-and-place and stacking tasks. We take several state-of-the-art CL algorithms that were originally designed to generate curriculum by manipulating the environment and appropriate them to weigh multiple sparse reward functions instead. The reward functions are chosen in a way that facilitates staged learning of the task, and the two robotic tasks are designed so that the agent learns to generalize to any initial and goal object position in the scene. The results of our three implemented CL algorithms show large improvement over the naive and state-of-the-art baselines in terms of speed of convergence to a successful policy in experiments with multiple task variations. Various generalization tests showcase some strengths and weaknesses of our approach. Inspection of changes in reward weight values during training further reveals varying curricula generated by the employed approaches, and showcases shifting emphasis from auxiliary to the main reward as the training progresses.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Robotic learning from sparse rewards can be a considerable challenge due to large amounts of data required for mastering a task. We explore the application of curriculum learning (CL) algorithms for automatic reward weighting to tackle learning from sparse rewards in robotic pick-and-place and stacking tasks. We take several state-of-the-art CL algorithms that were originally designed to generate curriculum by manipulating the environment and appropriate them to weigh multiple sparse reward functions instead. The reward functions are chosen in a way that facilitates staged learning of the task, and the two robotic tasks are designed so that the agent learns to generalize to any initial and goal object position in the scene. The results of our three implemented CL algorithms show large improvement over the naive and state-of-the-art baselines in terms of speed of convergence to a successful policy in experiments with multiple task variations. Various generalization tests showcase some strengths and weaknesses of our approach. Inspection of changes in reward weight values during training further reveals varying curricula generated by the employed approaches, and showcases shifting emphasis from auxiliary to the main reward as the training progresses.
Fele, Benjamin; Babič, Jan
Door(s): Junction State Estimation for Efficient Exploration in Reinforcement Learning Proceedings Article
In: 9th Annual Conference on Robot Learning, 2025.
@inproceedings{Fele2025b,
title = {Door(s): Junction State Estimation for Efficient Exploration in Reinforcement Learning},
author = {Benjamin Fele and Jan Babi\v{c}},
url = {https://www.github.com/benquick123/doors.},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
booktitle = {9th Annual Conference on Robot Learning},
abstract = {Exploration is one of the important bottlenecks for efficient learning in reinforcement learning, especially in the presence of sparse rewards. One way to traverse the environment faster is by passing through junctions, or metaphorical doors, in the state space. We propose a novel heuristic, Door(s), focused on such narrow passages that serve as pathways to a large number of other states. Our approach works by estimating the state occupancy distribution and allows computation of its entropy, which forms the basis for our measure. Its computation is more sample-efficient compared to other similar methods and robustly works over longer horizons. Our results highlight the detection of dead-end states, show increased exploration efficiency, and demonstrate that Door(s) encodes specific behaviors useful for downstream learning of various robotic manipulation tasks. The code is available at https://www.github.com/benquick123/doors.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Exploration is one of the important bottlenecks for efficient learning in reinforcement learning, especially in the presence of sparse rewards. One way to traverse the environment faster is by passing through junctions, or metaphorical doors, in the state space. We propose a novel heuristic, Door(s), focused on such narrow passages that serve as pathways to a large number of other states. Our approach works by estimating the state occupancy distribution and allows computation of its entropy, which forms the basis for our measure. Its computation is more sample-efficient compared to other similar methods and robustly works over longer horizons. Our results highlight the detection of dead-end states, show increased exploration efficiency, and demonstrate that Door(s) encodes specific behaviors useful for downstream learning of various robotic manipulation tasks. The code is available at https://www.github.com/benquick123/doors.
Fele, Benjamin; Lampe, Ajda; Peer, Peter; Štruc, Vitomir
C-VTON: Context-Driven Image-Based Virtual Try-On Network Proceedings Article
In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022.
@inproceedings{fele2022cvton,
title = {C-VTON: Context-Driven Image-Based Virtual Try-On Network},
author = {Benjamin Fele and Ajda Lampe and Peter Peer and Vitomir \v{S}truc},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
abstract = {Image-based virtual try-on techniques have shown great promise for enhancing the user-experience and improving customer satisfaction on fashion-oriented e-commerce platforms. However, existing techniques are currently still limited in the quality of the try-on results they are able to produce from input images of diverse characteristics. In this work, we propose a Context-Driven Virtual Try-On Network (C-VTON) that addresses these limitations and convincingly transfers selected clothing items to the target subjects even under challenging pose configurations and in the presence of self-occlusions. At the core of the C-VTON pipeline are: (i) a geometric matching procedure that efficiently aligns the target clothing with the pose of the person in the input images, and (ii) a powerful image generator that utilizes various types of contextual information when synthesizing the final try-on result. C-VTON is evaluated in rigorous experiments on the VITON and MPV datasets and in comparison to state-of-the-art techniques from the literature. Experimental results show that the proposed approach is able to produce photo-realistic and visually convincing results and significantly improves on the existing state-of-the-art.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Image-based virtual try-on techniques have shown great promise for enhancing the user-experience and improving customer satisfaction on fashion-oriented e-commerce platforms. However, existing techniques are currently still limited in the quality of the try-on results they are able to produce from input images of diverse characteristics. In this work, we propose a Context-Driven Virtual Try-On Network (C-VTON) that addresses these limitations and convincingly transfers selected clothing items to the target subjects even under challenging pose configurations and in the presence of self-occlusions. At the core of the C-VTON pipeline are: (i) a geometric matching procedure that efficiently aligns the target clothing with the pose of the person in the input images, and (ii) a powerful image generator that utilizes various types of contextual information when synthesizing the final try-on result. C-VTON is evaluated in rigorous experiments on the VITON and MPV datasets and in comparison to state-of-the-art techniques from the literature. Experimental results show that the proposed approach is able to produce photo-realistic and visually convincing results and significantly improves on the existing state-of-the-art.
Fele, Benjamin; Babič, Jan; Pollak, Senja; Žnidaršič, Martin
Evaluation of curriculum learning algorithms using computational creativity inspired metrics Proceedings Article
In: Hedblom, Maria M. (Ed.): Proceedings of the Thirteenth International Conference on Computational Creativity, ICCC'22, pp. 364–373, Bolzano, Italy, 2022.
@inproceedings{Fele2022,
title = {Evaluation of curriculum learning algorithms using computational creativity inspired metrics},
author = {Benjamin Fele and Jan Babi\v{c} and Senja Pollak and Martin \v{Z}nidar\v{s}i\v{c}},
editor = {Maria M. Hedblom},
url = {https://computationalcreativity.net/iccc22/wp-content/uploads/2022/06/ICCC-2022_5L_Fele-et-al..pdf},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the Thirteenth International Conference on Computational Creativity, ICCC'22},
pages = {364--373},
address = {Bolzano, Italy},
abstract = {Curriculum learning, especially in robotics, is an active research field aiming to devise algorithms that speed up knowledge acquisition by proposing sequences of tasks an agent should train on. We focus on curriculum generation in reinforcement learning, where various methods are currently compared based on the agent's performance in terms of rewards on a predefined distribution of target tasks. We want to extend this singular characterization of existing algorithms by introducing metrics inspired by notions from the field of computational creativity. Namely, we introduce surprise, novelty, interestingness, and typicality that quantify various aspects of tasks stochastically proposed by the curriculum learning algorithms for the learner to train on. We model proposed tasks with Gaussian mixture models which enable their probabilistic interpretation, and use Hellinger distances between distributions and training rewards in formulation of the proposed metrics. Results are presented for eight curriculum learning algorithms showcasing differences in prioritization of various aspects of task creation and statistically different mean metric values when comparing agent's best and worst training runs. The latter finding is not only useful for analysis of existing algorithms, but potentially also provides guidance for design of future curriculum learning methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Curriculum learning, especially in robotics, is an active research field aiming to devise algorithms that speed up knowledge acquisition by proposing sequences of tasks an agent should train on. We focus on curriculum generation in reinforcement learning, where various methods are currently compared based on the agent's performance in terms of rewards on a predefined distribution of target tasks. We want to extend this singular characterization of existing algorithms by introducing metrics inspired by notions from the field of computational creativity. Namely, we introduce surprise, novelty, interestingness, and typicality that quantify various aspects of tasks stochastically proposed by the curriculum learning algorithms for the learner to train on. We model proposed tasks with Gaussian mixture models which enable their probabilistic interpretation, and use Hellinger distances between distributions and training rewards in formulation of the proposed metrics. Results are presented for eight curriculum learning algorithms showcasing differences in prioritization of various aspects of task creation and statistically different mean metric values when comparing agent's best and worst training runs. The latter finding is not only useful for analysis of existing algorithms, but potentially also provides guidance for design of future curriculum learning methods.
Fele, Benjamin; Takác, Martin
A connectionist model of acquisition of noun phrases with syntactic bootstrapping Proceedings Article
In: Kognícia a umelý život 2019, pp. 31-32, 2019, ISBN: 978-80-223-4720-4.
@inproceedings{fele2019connectionist,
title = {A connectionist model of acquisition of noun phrases with syntactic bootstrapping},
author = {Benjamin Fele and Martin Tak\'{a}c},
url = {http://cogsci.fmph.uniba.sk/kuz2019/files/zbornik/fele-takac.pdf},
isbn = {978-80-223-4720-4},
year = {2019},
date = {2019-01-01},
urldate = {2019-01-01},
booktitle = {Kogn\'{i}cia a umel\'{y} \v{z}ivot 2019},
pages = {31-32},
abstract = {We present a model of acquisition of noun phrases as descriptions of objects in a grounded setting. The model uses cross-situational learning to acquire mappings between words and sensory properties represented in self-organising maps. We also explore syntactic bootstrapping\textemdashusing the syntactic knowledge to help meaning acquisition and vice versa. The model can reliably reconstruct inputs and is applicable in real-world scenarios.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present a model of acquisition of noun phrases as descriptions of objects in a grounded setting. The model uses cross-situational learning to acquire mappings between words and sensory properties represented in self-organising maps. We also explore syntactic bootstrapping—using the syntactic knowledge to help meaning acquisition and vice versa. The model can reliably reconstruct inputs and is applicable in real-world scenarios.
Fele, Benjamin
Dynamic Determination of Appropriate Training Instances Using Reinforcement Learning Proceedings Article
In: MEi: CogSci Conference 2019, pp. 23, 2019, ISBN: 978-80-223-4740-2.
@inproceedings{fele2019dynamic,
title = {Dynamic Determination of Appropriate Training Instances Using Reinforcement Learning},
author = {Benjamin Fele},
url = {https://www.meicogsci.eu/downloads/MEiCogSci_Conference_Proceedings_2019.pdf},
isbn = {978-80-223-4740-2},
year = {2019},
date = {2019-01-01},
urldate = {2019-01-01},
booktitle = {MEi: CogSci Conference 2019},
pages = {23},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Luštrek, Mitja; Fele, Benjamin; Reščič, Nina; Janko, Vito
Counting bites with a smart watch Proceedings Article
In: Slovenian Conference on Artificial Intelligence: proceedings of the 22nd International Multiconference Information Society-IS, pp. 49–52, 2019.
@inproceedings{luvstrek2019counting,
title = {Counting bites with a smart watch},
author = {Mitja Lu\v{s}trek and Benjamin Fele and Nina Re\v{s}\v{c}i\v{c} and Vito Janko},
url = {http://ai.ijs.si/MitjaL/documents/Counting_bites_with_a_smart_watch-IS-19.pdf},
year = {2019},
date = {2019-01-01},
urldate = {2019-01-01},
booktitle = {Slovenian Conference on Artificial Intelligence: proceedings of the 22nd International Multiconference Information Society-IS},
pages = {49--52},
abstract = {The work described in this paper is a part of the WellCo project, which is developing a virtual coach for healthy lifestyle. An important aspect of a healthy lifestyle is nutrition, and knowing as much as possible about the users’ current nutrition can contribute to better coaching. We therefore set out to count the number of times the users take food to their mouths (bites) using smartwatch sensors. This enables identifying the meals as well as estimating the caloric intake and the speed of eating. We compare three approaches: two that rely on classical machine learning and hidden Markov models, and one that uses deep learning. The F-scores of the approaches range from 0.63 to 0.91, and the percentages of miscounted bites from 6.9 % to 10.7 %, with a different approach scoring best on each metric.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The work described in this paper is a part of the WellCo project, which is developing a virtual coach for healthy lifestyle. An important aspect of a healthy lifestyle is nutrition, and knowing as much as possible about the users’ current nutrition can contribute to better coaching. We therefore set out to count the number of times the users take food to their mouths (bites) using smartwatch sensors. This enables identifying the meals as well as estimating the caloric intake and the speed of eating. We compare three approaches: two that rely on classical machine learning and hidden Markov models, and one that uses deep learning. The F-scores of the approaches range from 0.63 to 0.91, and the percentages of miscounted bites from 6.9 % to 10.7 %, with a different approach scoring best on each metric.