This article focuses on learning manipulation skills from episodic reinforcement learning (RL) in
unknown environments using industrial robot platforms. These platforms usually do not provide
the required compliant control modalities to cope with unknown environments, e.g., force-sensitive
contact-tooling. his requires to design a suitable controller, while also providing the ability os
adapting the the controller parameters from collected evidence online.
Thus, this work extends existing work on meta learning for graphical skill-formalisms. First, we
outline how a hybrid force-velocity-controller can be applied to an industrial robot in order to
design a graphical skill-formalism. This skill-formalism incorporates available task knowledge and
thus allows for online episodic RL.
In contrast to existing work, we further propose to extend this skill-formalism by estimating the
success-probability of the task to be learned by means of factor graphs. This method allows to
assign samples to the individual factors, i.e., Gaussian processes (GPs) more efficiently and thus
allows to improve the learning performance especially at early stages, where successful samples
are usually only drawn in a sparse manner. Finally, we propose suitable constraint GP-models
and acquisition functions to obtain new samples in order to optimize the information gain, while
also accounting for the success-probability of the task.
We outline a specific application example on the task of inserting the tip of a screwdriver into
a screwhead with an industrial robot, and evaluate our proposed extension against the state-
of-the-art. The collected data outlines that our method allows artificial agents to obtain feasible
samples faster than existing approaches while achieving a smaller regret value. This highlights
the potential of our proposed work for future robotic applications
@article{Gabler2022:Frontiers,author={Gabler, Volker and Wollherr, D.},journal={Frontiers Robotics and {AI}},title={{Bayesian Optimization with Unknown Constraints in Graphical Skill-Models for Compliant Manipulation Tasks Using an Industrial Robot}},volume={9},year={2022},doi={10.3389/frobt.2022.993359},issn={2296-9144},url={https://www.frontiersin.org/articles/10.3389/frobt.2022.993359},}
A Force-Sensitive Grasping Controller Using Tactile Gripper Fingers and an Industrial Position-Controlled Robot
Grasping fragile objects in the presence of uncertainty is a crucial task for robots,
that becomes inherently challenging if the manipulator in use is an
industrial robot platform that does not provide compliant control inputs.
This requires not only to estimate the alignment error during object
contact but also to alter the robot configuration to decrease this
error while taking interaction constraints into account.
Thus, this work proposes a novel grasping controller tailored to
industrial robots by exploiting tactile sensor feedback on the
robot gripper fingers in order to estimate and compensate
for the alignment error when touching the object.
Specifically, we propose two grasping strategies,
that allow to either directly compensate for interaction wrenches
or to solve a model predictive control-problem to minimize the
estimated alignment error. Eventually, we outline how these
modalities can be realized as a hybrid Cartesian force-velocity-controller
on an industrial manipulator.
We evaluate the proposed grasping strategies on a WSG 50 parallel two-finger gripper,
that is equipped with a digital sensor array (DSA) per finger,
for which we also provide an extended ROS-driver that allows to obtain DSA-data
at a communication rate above 5 Hz. Given the collected empirical evidence,
the presented grasping controller increases the skill-set of industrial robots
in the presence of uncertainty and thus allows to apply stiff robots to handle
fragile objects autonomously.
@inproceedings{Gabler2022,title={{A Force-Sensitive Grasping Controller Using Tactile Gripper Fingers and an Industrial Position-Controlled Robot}},author={Gabler, Volker and Huber, G. and Wollherr, D.},booktitle={{IEEE} International Conference on Robotics and Automation~(ICRA)},address={Philladelphia},pages={770--776},publisher={{IEEE}},year={2022},url={https://doi.org/10.1109/ICRA46639.2022.9812278},doi={10.1109/ICRA46639.2022.9812278},keywords={Grasping; Force and Tactile Sensing; Industrial Robots},}
2021
MS2MP: A Min-Sum Message Passing Algorithm for Motion Planning
@inproceedings{conf/icra/BariGW21,author={Bari, S. and Gabler, Volker and Wollherr, D.},author+an={2=highlight},title={{MS2MP:} {A} Min-Sum Message Passing Algorithm for Motion Planning},booktitle={{IEEE} International Conference on Robotics and Automation~(ICRA)},address={Xi'an, China},pages={7887--7893},publisher={{IEEE}},year={2021},url={https://doi.org/10.1109/ICRA48506.2021.9561533},doi={10.1109/ICRA48506.2021.9561533},}
2020
Haptic Object Identification for Advanced Manipulation Skills
order to identify the characteristics of unknown objects, humans - in contrast to robotic systems - are experts in exploiting their sensory and motoric abilities to refine visual information via haptic perception. While recent research has focused on either estimating the geometry or material properties, this work strives to combine these aspects by outlining a probabilistic framework that efficiently refines initial knowledge from visual sensors by generating a belief state over the object shape while simultaneously learn material parameters. Specifically, we present a grid-based and a shape-based exploration strategy, that both apply the concepts of Bayesian-Filter theory in order to decrease the uncertainty. Furthermore, the presented framework is able to learn about the geometry as well as to distinguish areas of different material types by applying unsupervised machine learning methods. The experimental results from a virtual exploration task highlight the potential of the presented methods towards enabling robots to autonomously explore unknown objects, yielding information about shape and structure of the underlying object and thus, opening doors to robotic applications where environmental knowledge is limited.
@inproceedings{Gabler2020,author={Gabler, Volker and Maier, K. and Endo, S. and Wollherr, D.},title={Haptic Object Identification for Advanced Manipulation Skills},booktitle={International Conference on Biomimetic and Biohybrid Systems~({Living Machines)}},year={2020},editor={Vouloutsi, Vasiliki and Mura, Anna and Esser, Falk J. and Speck, Thomas and Prescott, Tony J. and Verschure, Paul F. M. J.},volume={12413},series={Lecture Notes in Computer Science},pages={128--140},publisher={Springer},doi={10.1007/978-3-030-64313-3\_14},url={https://doi.org/10.1007/978-3-030-64313-3\_14},}
2019
Reducing Overestimation Bias in Multi-Agent Domains Using Double Centralized Critics
Many real world tasks require multiple agents to work together. Multi-agent reinforcement learning (RL) methods have been proposed in recent years to solve these tasks, but current methods often fail to efficiently learn policies. We thus investigate the presence of a common weakness in single-agent RL, namely value function overestimation bias, in the multi-agent setting. Based on our findings, we propose an approach that reduces this bias by using double centralized critics. We evaluate it on six mixed cooperative-competitive tasks, showing a significant advantage over current methods. Finally, we investigate the application of multi-agent methods to high-dimensional robotic tasks and show that our approach can be used to learn decentralized policies in this domain.
@article{Ackermann2019,author={Ackermann, J. and Gabler, Volker and Takayuki, O. and Masashi, S.},journal={CoRR},title={Reducing Overestimation Bias in Multi-Agent Domains Using Double Centralized Critics},year={2019},volume={abs/1910.01465},bibsource={dblp computer science bibliography, https://dblp.org},biburl={https://dblp.org/rec/journals/corr/abs-1910-01465.bib},eprint={1910.01465},eprinttype={arXiv},timestamp={Fri, 04 Oct 2019 12:28:06 +0200},}
2017
A Game-Theoretic Approach for Adaptive Action Selection in Close Distance Human-Robot-Collaboration
With the integration of Human-Robot Collaboration (HRC) in industrial assembly scenarios, robot systems face numerous challenges. In contrast to classic robot systems which follow a pre-programmed and fixed sequence of actions, an interaction scenario with humans in the loop requires mutual adaptation. In this paper a framework based on game theory is presented that allows robots to choose appropriate actions with respect to the action of human coworkers when collaborating in close proximity. The proposed framework models HRC scenarios as iterative games and selects action-strategies for the Human-Robot Team (HRT) by finding the Nash-Equilibria (NEs) of these games. In contrast to most common approaches, our proposed HRC-game treats the decision-making behavior equally for all agents involved. Therefore, the concept of game theory is applied to evaluate the mutual interference of all actions on the HRT to obtain pareto-optimal NEs, i.e. team-optimal action-allocations. The general framework of the proposed HRC-game is realized on an interactive pick-and-place scenario in close proximity. This exemplary HRC-game is tested in a human subject experiment of a KUKA LWR 4+ robot and a human coworker assembling toy-bricks in close proximity. The experimental measurements and statistically significant improvements in the subjective feedback hold as a proof-of-concept of the proposed HRC-game model.
@inproceedings{Gabler2017,author={Gabler, Volker and Stahl, T. and Huber, G. and Oguz, O. and Wollherr, D.},booktitle={{IEEE} International Conference on Robotics and Automation~(ICRA)},title={{A Game-Theoretic Approach for Adaptive Action Selection in Close Distance Human-Robot-Collaboration}},address={Singapore},isbn={9781509046324},pages={2897--2903},publisher={{IEEE}},year={2017},url={https://doi.org/10.1109/ICRA.2017.7989336},doi={10.1109/ICRA.2017.7989336},keywords={Autonomous Agents, Cognitive Human-Robot Interaction, Planning, Scheduling and Coordination},}
Legible Action Selection in Human-Robot Collaboration
In 26th IEEE International Symposium on Robot and Human Interactive
Communication, RO-MAN 2017, Lisbon, Portugal, August 28 - Sept.
1, 2017, pp. 354–359, 2017
Humans are error-prone in the presence of multiple similar tasks. While Human-Robot Collaboration (HRC) brings the advantage of combining the superiority of both humans and robots in their respective talents, it also requires the robot to communicate the task goal clearly to the human collaborator. We formalize such problems in interactive assembly tasks with hidden goal Markov decision processes (HGMDPs) to enable the symbiosis of human intention recognition and robot intention expression. In order to avoid the prohibitive computational requirements, we provide a myopic heuristic along with a feature-based state abstraction method for assembly tasks to approximate the solution of the resulting HGMDP. A user study with human subjects in round-based LEGO assembly tasks shows that our algorithm improves HRC and helps the human collaborators when the task goal is unclear to them.
@inproceedings{Zhu2017,author={Zhu, H. and Gabler, Volker and Wollherr, D.},booktitle={26th {IEEE} International Symposium on Robot and Human Interactive
Communication, {RO-MAN} 2017, Lisbon, Portugal, August 28 - Sept.
1, 2017},title={{Legible Action Selection in Human-Robot Collaboration}},address={Lisbon},pages={354--359},publisher={{IEEE}},year={2017},url={https://doi.org/10.1109/ROMAN.2017.8172326},doi={10.1109/ROMAN.2017.8172326},keywords={Assembly, Autonomous Agents, Human-Robot Interaction, Legible Planning},}
An online trajectory generator on SE(3) with magnitude constraints
In 2017 IEEE/RSJ International Conference on Intelligent Robots and
Systems, IROS 2017, Vancouver, BC, Canada, September 24-28, 2017, pp. 6171–6177, 2017
@inproceedings{DBLP:conf/iros/HuberGW17,author={Huber, G. and Gabler, Volker and Wollherr, D.},title={An online trajectory generator on {SE(3)} with magnitude constraints},booktitle={2017 {IEEE/RSJ} International Conference on Intelligent Robots and
Systems, {IROS} 2017, Vancouver, BC, Canada, September 24-28, 2017},pages={6171--6177},publisher={{IEEE}},year={2017},url={https://doi.org/10.1109/IROS.2017.8206518},doi={10.1109/IROS.2017.8206518},}
Motion Recognition by Natural Language Including Success and Failure of Tasks for Co-Working Robot with Human
@inproceedings{Kobayashi2017,title={{Motion Recognition by Natural Language Including Success and Failure of Tasks for Co-Working Robot with Human}},author={Kobayashi, Y. and Matsumoto, T. and Takano, W. and Wollherr, D. and Gabler, Volker},booktitle={{IEEE} International Conference on Advanced Intelligent Mechatronics,
{AIM} 2017, Munich, Germany, July 3-7, 2017},pages={10--15},publisher={{IEEE}},year={2017},url={https://doi.org/10.1109/AIM.2017.8013987},doi={10.1109/AIM.2017.8013987},}
2016
Hybrid Human Motion Prediction for Action Selection Within Human-Robot Collaboration
@inproceedings{Oguz2016,title={Hybrid Human Motion Prediction for Action Selection Within Human-Robot Collaboration},author={Oguz, O. and Gabler, Volker and Huber, G. and Zhehua, Z. and Wollherr, D.},editor={Kulic, Dana and Nakamura, Yoshihiko and Khatib, Oussama and Venture, Gentiane},booktitle={International Symposium on Experimental Robotics, {ISER} 2016, Tokyo,
Japan, October 3-6, 2016},series={Springer Proceedings in Advanced Robotics},volume={1},pages={289--298},publisher={Springer},year={2016},url={https://doi.org/10.1007/978-3-319-50115-4\_26},doi={10.1007/978-3-319-50115-4\_26},isbn={978-3-319-50115-4},}
2015
An approach to integrate human motion prediction into local obstacle avoidance in close human-robot collaboration
@inproceedings{Dinh2015,title={{An approach to integrate human motion prediction into local obstacle avoidance in close human-robot collaboration}},author={Dinh, K.H. and Oguz, O. and Huber, G. and Gabler, Volker and Wollherr, D.},booktitle={2015 {IEEE} International Workshop on Advanced Robotics and its Social
Impacts, {ARSO} 2015, Lyon, France, June 30 - July 2, 2015},pages={1--6},publisher={{IEEE}},year={2015},url={https://doi.org/10.1109/ARSO.2015.7428221},doi={10.1109/ARSO.2015.7428221},address={Lyon, France},isbn={9781467380294},issn={21627576},}