TY  - JOUR
AB  - Increasing product variability and shrinking batch sizes in manufacturing demand robotic systems that can be reconfigured rapidly with minimal engineering effort. While mobile manipulators provide flexibility at the hardware level, task-level specification and coordination remain a major bottleneck in assembly-oriented scenarios. This paper presents a modular multimodal AI agent that acts as a supervisory orchestration layer for mobile robotic tasks in flexible assembly automation. Following a hybrid-intelligence architecture, the agent interprets natural-language or speech instructions, grounds them in contextual and perceptual information, and coordinates navigation, perception, and manipulation via constrained Robot Operating System (ROS) 2 tool interfaces, while execution remains within established autonomy components. The framework is evaluated via quantitative analysis of endto- end request-to-execution behavior across stepwise interaction, one-shot execution, and task reconfiguration scenarios, in simulation and selected physical validation. Results show that orchestration-related processing remains bounded relative to navigation and manipulation runtime, with execution variability dominated by perception and grounding reliability rather than agent reasoning overhead. Robust multimodal grounding substantially reduces long-tailed execution behavior in autonomous task pipelines, providing empirical insight into the runtime characteristics and limitations of agent-based task orchestration for production-relevant mobile manipulation.
AU  - Pinheiro de Souza, Lucas Manassés
AU  - Murugasamy, Vishnu
AU  - Moriz, Alexander
AU  - Göppert, Amon
AU  - Schmitt, Robert
DO  - 10.17619/UNIPB/1-2658
PB  - Universitätsbibliothek
DP  - Universität Paderborn
LA  - eng
PY  - 2026
SP  - 1 Online-Ressource (Seite 89-98) : Illustrationen, Diagramme
T2  - 1st International Symposium: March 24 – 26, 2026, Heinz Nixdorf Institute, Paderborn University
TI  - Multimodal AI agent for orchestration of mobile robotic tasks in flexible assembly automation
UR  - https://nbn-resolving.org/urn:nbn:de:hbz:466:2-58953
Y2  - 2026-09-25T21:01:47
ER  -