MIT Created a Robot That Maps 3D Objects by Touch

While AI continues to rapidly advance, we still have a long road ahead when developing the necessary sensors to translate the physical world to data a computer can understand. While sight and sound had a bit of a head start, our other senses have few practical applications in the digital world—but that’s not the case with robots.

MIT recently created a new robot using GelSight sensors that allow it to see the objects it touches and create a 3D map of the texture to better understand it. The video below demonstrates how the GelSight technology, typically used for aerospace applications, can “see” what it touches.

GelSight certainly offers an impressive, detailed means of translating the real world into digital information. But that doesn’t make an intelligent robot—just some very informative fingers that require intelligence to control. MIT saw this potential and created a robot with an AI model that trains itself on the objects it touches using detailed three-dimensional maps generated by its GelSight sensors. While the robot doesn’t actually see what it touches in a traditional, optical sense, it receives so much data through its sensors that it can translate that data into visual information and learn from it just like any ordinary image-oriented convolutional neural network (CNN).

MIT’s robot was trained on 12,000 video recordings of GelSight sensor touch data, broken into still image frames, of 200 household objects. Combined with tactile data, this allows the robot to understand the materials its sensors touch. In a conversation with Engadget, CSAIL Ph.D. student and lead author of this project, explained what their system can now achieve:

“By looking at the scene, our model can imagine the feeling of touching a flat surface or a sharp edge. By blindly touching around, our model can predict the interaction with the environment purely from tactile feelings. Bringing these two senses together could empower the robot and reduce the data we might need for tasks involving manipulating and grasping objects.”

While still in its early stages, MIT’s system works quite well and that’s due to their approach. Many AI researchers and developers tend to create models based on how the human brain works but that often doesn’t make a lot of sense. In some cases, we do want AI that functions as a human because its goal is either to approximate us or help us learn more about ourselves by simulating human processes. In most other cases, however, approaching AI development by imposing a human framework negates the many non-human advantages that computer software and hardware have to offer.

MIT chose to use a sensor far more precise and capable than any human can approximate and leverage the computational power available to AI. By making choices that use the inherent advantages of computers, rather than force a human approach, they’ve created a robot that has the potential to outperform humans in blind-touch identification tasks. In specific cases, it already achieves that.

While it might not seem like the most important problem to solve, touch actually plays a significant role in robotics. Niche applications might be able to take advantage of a robotic ability to feel the difference between cotton and nylon, but the broader applications have much more to offer. To a robot without a sense of touch, all objects feel the same. It might be able to understand some things visually but that’s rarely helpful enough.

Consider how you’d go about your day if everything you touched felt the same—or, more accurately, didn’t feel like anything at all. You wouldn’t know how much force to use when plugging in a cable. You wouldn’t have the ability to understand the practical differences between a printed image of sandpaper and sandpaper itself.

By providing a robot with a sense of touch and the ability to learn from it, that robot can make better judgments about the materials it touches. It can learn faster and more precisely than it can simply from standard visuals. It can then use that information to adjust its actions according to the materials it handles—or, at least, that’s the ideal goal for the future. If robots can understand touch, they pose less of a risk of causing unintentional harm. Right now, if you tasked most intelligent robots with carrying a water balloon they wouldn’t know how to hold it without destroying it. A sense of touch gives robots the ability, through a well trained AI-model, to recognize how to handle various types of objects and act accordingly.

While MIT has only created the beginnings of a component of better, more intelligent robots, it’s still another good step in the right direction. A robot designed to understand and incorporate the data it acquires through touch has far better implications for overall safety than one that does not. It’s developments like this that create safeguards against potential accidents.

Top image credit: Photo by Franck V. on Unsplash

Continue reading

Harvard Astronomer Still Believes Interstellar Object Was Alien Technology

Scientists have classified 'Oumuamua variously as an asteroid or a comet, but Avi Loeb, the chair of Harvard's Department of Astronomy, believes it was really alien — a piece of alien technology we mistook for a naturally occurring space rock.

MIT Robot Sees Hidden Objects with Radio Waves

Creating robots that see the world like humans has been a challenge. While computer vision has come a long way, these systems are still easy to fool. So, why not give robots superhuman perception to compensate?

Mysterious Space Object Could Be Record-Breaking Comet

The outer solar system is littered with big chunks of rock and ice, but rarely do their orbits bring them close enough to Earth for us to get a good look. And then there's 2014 UN271, an approaching object that astronomers believe to be a huge comet on a million-year orbit around the sun.

Most Interstellar Objects Might Evaporate Between Stars

In 2017, astronomers around the world turned their attention to a small point of light zipping through the solar system. The object, known as 'Oumuamua, was the very first interstellar object ever discovered. We have since spotted a second, suggesting that visitors from afar might be common. However, a new analysis claims that almost all the potential visitors drifting between stars evaporate long before they could get here.