Concept & Architecture For Pogramming Industrial Robots Using Augmented Reality With Mobile Devices Like Microsoft

Concept and architecture for programming industrial robots using augmented reality with mobile devices like Microsoft.

Institute for Production Systems and Design Technology, Berlin, Germany

INTRODUCTION

Individualization of products is increasing in modern manufacturing and flexible production systems are required in the era of connected industry. Building the robot program in the traditional way results into non-portable programs which are vendor and device specific and even a small change will lead to high cost and more time. The idea of Industry 4.0 suggests an architecture to separate field device from control device and instantiating the control device as a hardware independent cloud-service. A small part of software is installed on each device to connect them to a single server or to the cloud forming a network of connected devices.

Challenges: To be able to interact and program the industrial robots using mobile devices the below challenges must be addressed:

• Monitoring & supervising the data exchanged between devices & robots in such a cloud based infrastructure.
• To visualize the robot programs or any other useful data associated with the robot like workspace of robot, movement restrictions, etc. Goals/Contributions: The goal of this paper is to provide a concept and architecture of human-robot interaction for programming industrial robots in a cyber-physical system using mobile devices.

METHOD/APPROACH/ARCHITECTURE/FRAMEWORK

The authors have proposed a framework to divide the layers according to their functionalities. The first is the interaction layer which must support various devices. Tested devices by the authors includes 9.7 inch Android Tablet, 13.5 inch Windows Tablet, Microsoft HoloLens and Windows 10 Laptop. A program is running on the robot control to interpret the commands sent by the operator from the mobile device into the robot specific languages.

The second is the robot layer with all the robotic systems from various vendors and so its implementation must be vendor specific but all of them should have same interface which is exposed to the broker. Tested robots includes Universal Robots UR5, Comau NJ 130 – 2.6 and KUKA KR 6.

The third layer is the broker which is the central part and is connected to all the devices in interaction layer and robot layer. This part is developed in C++ for better performance and scalability to handle numerous connections. The authors have used the implementation from Anderson [3] for connecting robots to the broker. Each device only needs to handle one connection to the broker using which it sends or receives the information about every other device connected to the network. The Unity 3D game engine is used for visualizing and rendering the model and user interface in augmented reality to for a better human-robot interaction as it is more natural.

Marker tracking is used as a base for enabling the spatial positioning which is achieved by using Vuforia software. TCP/IP is used for communication between the layers and the devices. Protocol buffers are used for serializing and de-serializing of data exchanged between the three layers. The experiment is carried out by establishing a connection to the robot and requesting the list of connected device from interaction and robot layers and receiving the general information like axes angle limits, possible types of interactions, etc. Requesting detailed information with precision about some devices and receiving it with high frequency. On program exit, disconnect from the broker.

RESULTS AND DISCUSSION

The concept and architecture for the interaction and programming of industrial robots is presented using mobile device like Microsoft HoloLens. The broker is able to establish a connection with mobile devices and robots using Protocol buffers on top of TCP/IP connections. The software components of the Unity 3D and Vuforia made it possible to use a variety of mobile devices with different operating systems and hardware especially Microsoft HoloLens enabling a natural interaction with robots. Robots are able to transmit the axes angles from & to the mobile devices. This will help to visualize and evaluate the existing programs on the real robots before execution.

REFERENCES:

DIN91345:2016-04, Referenzarchitekturmodell Industrie 4.0(RAMI4.0).

A. Vick, J. Guhl and J. Kr¨uger, ”Model predictive control as a service - Concept and architecture for use in cloud-based robot control”, 2016 21st International Conference on Methods and Models in Automation and Robotics (MMAR), Miedzyzdroje, 2016, pp. 607-612.

T. T. Andersen, ”Optimizing the Universal Robots ROS driver.”, available at http://orbit.dtu.dk/en/publications/optimizing-the-universal-robotsros-driver(20dde139-7e87-4552-8658-dbf2cdaab24b).html, (18.05.2017), 2015.

Guhl J, Nguyen ST and Kr¨uger J. Concept and architecture for programming industrial robots using augmented reality with mobile devices like microsoft HoloLens. In: 2017 22nd IEEE International Conference on Emerging Technologies and Factory Automation, Limassol, Cyprus, 2017, pp. 1-4. DOI:10.1109/ETFA.2017.8247749.