Industrial robots are reliable but hard to control remotely. With Augmented Reality, new possibilities arise for applications in sensitive environments such as for the pharmaceutical and semiconductor industries. Helbling has developed a practical, customer-centric solution. It keeps operators safe and ensures that processes remain uncontaminated.
The Challenge: Remote Troubleshooting in Sensitive Environments
For some industries, human intervention poses a safety risk and operators must not be exposed to hazardous conditions. In other industries, such as pharmaceuticals or semiconductor manufacturing, the operators themselves are the risk: even minimal human contact can compromise product integrity or violate the highest cleanliness standards.
All of these developments make concepts where robotic systems replace direct human manipulation entirely, thereby reducing contamination significantly, particularly interesting. In the pharma sector, the gloveless isolator is gaining traction, while in the semiconductor industry automated wafer handling systems have been a reality for decades. The ability to troubleshoot remotely, diagnosing and resolving issues without direct human intervention, is therefore not a luxury but an operational necessity.
Helbling teams were able to pinpoint what an effective solution needs to deliver for these cases. Drawing on expertise in augmented reality (AR) and virtual reality (VR) as well as robotics, a new approach was developed. One such case is described below and demonstrated in a video.
The Approach: Augmented Reality Meets Industrial Robotics
Industrial robots have long proven their reliability in production environments where repetitive tasks need to be performed. However, when an operator needs to intervene manually, for instance during a troubleshooting event, traditional control methods such as teach pendants can feel cumbersome. A more intuitive interaction model is needed.
Inspiration for this comes from emerging robotics paradigms, where intuitive motion-based control using augmented (AR) or virtual reality (VR) has shown that complex robotic systems can be operated naturally and efficiently. Applying this philosophy to industrial robots opens up a compelling new possibility: an operator, stationed safely outside a sensitive environment, controls a robotic arm through AR in real time, seeing what the robot sees, guiding it with precision, and resolving issues without ever entering the sensitive area.
The Technical Solution: Architecture and Toolchain
The solution is built around a modular architecture that connects an AR interface with an industrial robot arm through a communication layer.
At its core, the system consists of three main components. The AR headset – in the example a Meta Quest 3 – tracks the operator's hand-held controllers, translating them into robot control commands in real time. A middleware layer handles communication between the AR interface and the robot controller, ensuring low-latency and reliable data exchange. Finally, the industrial robot arm – in the example a collaborative P-Rob Eco from F&P – executes the commands with the precision that industrial environments demand.
A key strength of the solution is its two complementary operating modes. When no direct line of sight to the robot is available, the operator wears the headset and interacts with a virtual twin of the robot overlaid in their field of view, captured with a camera system. The system is a precise digital replica that mirrors the real robot's state in real time. When direct line of sight is available, the headset can be worn around the neck, acting purely as a tracking device for the hand-held controllers. This allows the robot to be controlled intuitively from any position in the space around it, with natural hand movements directly guiding the arm.
This modular system architecture keeps the solution flexible and hardware-agnostic by design, making it adaptable to different robot brands, AR platforms, and facility layouts. This solution therefore represents a clear commitment to Helbling’s practical and customer-centric engineering approach.
Helbling Showcase
The showcase demonstrates a representative troubleshooting scenario: retrieving a vial that had fallen out of position on a pharmaceutical fill & finish line. This is a typical yet delicate intervention requiring high dexterity in a usually jam-packed environment. In such cases, glove ports can be an additional source of leakage, and hands can disturb the airflow and compromise the sterile environment.
In the test set-up, the operator is stationed outside the isolator and uses the controllers to guide the robot arm to the vial location and retrieve it precisely without ever entering the sensitive space. Where direct line of sight is available, the operator can retrieve the vial in a fluid, position-independent manner. Where visibility is limited, the virtual twin provides a reliable and precise alternative.
The video below captures the solution in action.
Conclusion: Helbling's Engineering Edge unlocks new opportunities by combining high-tech disciplines
This project offers an example of the hallmarks of a Helbling solution: taking a real-world operational challenge and solving it with a pragmatic, customer-centric engineering approach. By combining mature industrial robotics with intuitive AR control, we have delivered a solution that is adaptable to a wide range of sensitive environments and industries.
Helbling experts would be delighted to assist any organization at any time facing similar challenges and to explore how this approach can be tailored to suit specific needs.
Authors: Dr. Franck Robin, Fabian Schneider
Main Image: Helbling
