HapticMaster transmits movements over enormous distance in real time
A surgeon in Utrecht performs a complex operation in Tokyo. This kind of activity requires a fast network, advanced robotics, and extremely high-quality video.
At SURFnet’s Contact Days , we demonstrated how a robot arm can be operated from 1800 kilometres away without any delay. We are explaining the network, the robotics, and the video in a series of blogs. Today’s blog focuses on the robotics.
During the demonstration, visitors in Noordwijkerhout play a game based on the Leaning Tower of Pisa. The actual game is located in Poland, 900 kilometres away, but the light signals need to be transmitted over a distance of 1800 kilometres. This means that the visitors can’t operate the game directly with their hands – there is equipment in between, namely the HapticMaster. This is a robot arm that can perceive and transmit movements very precisely, and provide feedback. Two HapticMasters are used.
Let’s say that I’m a visitor to the Contact Days in Noordwijkerhout and I want to move a figure on the board in Poland. To do so, I operate the arm of my HapticMaster. The HapticMaster in Poland has an identical arm and reproduces my movement without any delay. It also gives me visual feedback via high-quality video. I also get haptic feedback: when I touch a figure, I can feel it with the operating arm of my HapticMaster – there is a slight counterpressure. I can therefore both see and feel what I am doing, and it’s without any delay because of the fast network connection.
The HapticMaster was made available by the University of Twente’s Virtual Reality Lab (VR Lab). This is just one of the devices that the VR Lab has available to make the virtual world tangible. Roy Damgrave, a researcher at the VR Lab, explains: “You can see the VR Lab as a sort of big toolbox containing tools that are highly configurable. That means we can help with product development, for example. Our toolbox enables us to create a virtual world that is just as visible, audible, and tangible as the real world.
“Take a gearbox, for example: the car manufacturer wants to develop one with precisely the right ‘feel’ when you change gear. We can test that feel in our lab before the gearbox is actually developed. That allows us to clarify choices and so arrive at a solution in collaboration with people from various disciplines and with various interests. Moreover, this method can of course save a lot of money.”
In this demonstration in Noordwijkerhout, the enormous distance plays a major role. The VR Lab is also considering this factor. “Videoconferencing allows people to see and hear one another,” says Roy Damgrave, “but could they also shake hands by means of robotics? And of course there’s remote operation: it saves money if a surgeon doesn’t need to fly all over the world to perform operations but can do so remotely, or if an engineer can service a complex machine remotely. This development is now getting going.”
This is where a fast network connection comes in. To operate a device remotely, it is extremely important that there is neither haptic nor visual delay, i.e. no time difference between the action by the operator, the physical action, and the feedback via video. Delay is already annoying when you are making a phone call, but with an open heart operation it can be fatal if the surgeon makes an incision a few millimetres from where it should be because there are a few milliseconds delay between the movement of his hand and the feedback on the video screen.
“It’s the enormous distance that makes this demonstration so interesting,” says Roy Damgrave. “We can see whether any delay is noticed by the user. It’s also interesting for us to see whether using 3D video is an advantage when performing what are after all three-dimensional actions.”
Read all blogs regarding the robot arm demonstration:
- About the demonstration’s design: Complex remote action
- About the network connection: Unique 40 Gbit/s connection across 1800 kilometres
- About video: 3D and 4K video for demonstration using HapticMaster