Dual-Arm Teleoperation for Automating Unstructured Bimanual Tasks in Manufacturing

Professor Ryu’s research group at KAIST has developed skill transfer technology to automate dexterous and contact-rich tasks in manufacturing plants that still rely heavily on skilled human labor. Although many manufacturing processes have already been automated, tasks that involve delicate or damage-prone parts or require the handling of flexible materials continue to depend on human finesse and tactile sensitivity.

In particular, these dexterous and contact-rich tasks demand delicate force sensing and corresponding reactive motion. Traditional automation methods often fall short in this regard, and AI-based deep learning approaches struggle to adapt rapidly to new processes. To overcome these limitations, the research team adopted “imitation learning,” enabling robots to learn quickly by observing human demonstrations. However, limited research has been done on demonstration interfaces for force-based tasks and on techniques to capture and learn from precise force data. As a result, implementing finely tuned force control in real industrial environments remains a significant challenge.

To address this issue, the research team introduced a method using haptic devices for bidirectional teleoperation. Through these devices, the human demonstrator can feel the exact same forces the robot senses in real time (transparent force feedback). This enables the human demonstrator to provide a “pure” set of motion skills to the robot, reflecting detailed force interactions as they naturally occur. Because the forces experienced by both the human demonstrator and the robot are identical, high-quality demonstration data become available even for tasks that demand extremely precise force control.

Furthermore, the research team unveiled a new demonstration approach leveraging virtual environments. Teleoperation systems are often constrained by communication delays and physical latency, restricting real-world interaction. However, by employing a physics-based simulator in a virtual environment, those latency factors can be virtually eliminated. This setup allows the human demonstrator to perform tasks freely—without the kinematic or dynamic constraints of the robot—while maintaining realistic force feedback through a haptic device. This approach yields rich and accurate demonstration data, which can then be applied to real robots, enabling faster and more efficient automation solutions.

Building on these technologies, the team collaborated with Hyundai Motor Company to successfully automate the connector-plugging process in an actual automotive production line. By integrating precise force control with a skill transfer approach, the project shows promise in reducing reliance on human experts and achieving safer, more efficient, fully automated manufacturing processes.

Looking ahead, Professor Ryu’s research group plans to further refine their force-sensing-based skill transfer techniques to offer versatile solutions for various industrial fields and spearhead the future of manufacturing automation.