MIT Created a Robot That Maps 3D Objects by Touch

While AI continues to rapidly advance, we still have a long road ahead when developing the necessary sensors to translate the physical world to data a computer can understand. While sight and sound had a bit of a head start, our other senses have few practical applications in the digital world—but that’s not the case with robots.

MIT recently created a new robot using GelSight sensors that allow it to see the objects it touches and create a 3D map of the texture to better understand it. The video below demonstrates how the GelSight technology, typically used for aerospace applications, can “see” what it touches.

GelSight certainly offers an impressive, detailed means of translating the real world into digital information. But that doesn’t make an intelligent robot—just some very informative fingers that require intelligence to control. MIT saw this potential and created a robot with an AI model that trains itself on the objects it touches using detailed three-dimensional maps generated by its GelSight sensors. While the robot doesn’t actually see what it touches in a traditional, optical sense, it receives so much data through its sensors that it can translate that data into visual information and learn from it just like any ordinary image-oriented convolutional neural network (CNN).

MIT’s robot was trained on 12,000 video recordings of GelSight sensor touch data, broken into still image frames, of 200 household objects. Combined with tactile data, this allows the robot to understand the materials its sensors touch. In a conversation with Engadget, CSAIL Ph.D. student and lead author of this project, explained what their system can now achieve:

“By looking at the scene, our model can imagine the feeling of touching a flat surface or a sharp edge. By blindly touching around, our model can predict the interaction with the environment purely from tactile feelings. Bringing these two senses together could empower the robot and reduce the data we might need for tasks involving manipulating and grasping objects.”

While still in its early stages, MIT’s system works quite well and that’s due to their approach. Many AI researchers and developers tend to create models based on how the human brain works but that often doesn’t make a lot of sense. In some cases, we do want AI that functions as a human because its goal is either to approximate us or help us learn more about ourselves by simulating human processes. In most other cases, however, approaching AI development by imposing a human framework negates the many non-human advantages that computer software and hardware have to offer.

MIT chose to use a sensor far more precise and capable than any human can approximate and leverage the computational power available to AI. By making choices that use the inherent advantages of computers, rather than force a human approach, they’ve created a robot that has the potential to outperform humans in blind-touch identification tasks. In specific cases, it already achieves that.

While it might not seem like the most important problem to solve, touch actually plays a significant role in robotics. Niche applications might be able to take advantage of a robotic ability to feel the difference between cotton and nylon, but the broader applications have much more to offer. To a robot without a sense of touch, all objects feel the same. It might be able to understand some things visually but that’s rarely helpful enough.

Consider how you’d go about your day if everything you touched felt the same—or, more accurately, didn’t feel like anything at all. You wouldn’t know how much force to use when plugging in a cable. You wouldn’t have the ability to understand the practical differences between a printed image of sandpaper and sandpaper itself.

By providing a robot with a sense of touch and the ability to learn from it, that robot can make better judgments about the materials it touches. It can learn faster and more precisely than it can simply from standard visuals. It can then use that information to adjust its actions according to the materials it handles—or, at least, that’s the ideal goal for the future. If robots can understand touch, they pose less of a risk of causing unintentional harm. Right now, if you tasked most intelligent robots with carrying a water balloon they wouldn’t know how to hold it without destroying it. A sense of touch gives robots the ability, through a well trained AI-model, to recognize how to handle various types of objects and act accordingly.

While MIT has only created the beginnings of a component of better, more intelligent robots, it’s still another good step in the right direction. A robot designed to understand and incorporate the data it acquires through touch has far better implications for overall safety than one that does not. It’s developments like this that create safeguards against potential accidents.

Top image credit: Photo by Franck V. on Unsplash

Continue reading

Chromebooks Gain Market Share as Education Goes Online

Chromebook sales have exploded in the pandemic, with sales up 90 percent and future growth expected. This poses some challenges to companies like Microsoft.

Review: The Oculus Quest 2 Could Be the Tipping Point for VR Mass Adoption

The Oculus Quest 2 is now available, and it's an improvement over the original in every way that matters. And yet, it's $100 less expensive than the last release. Having spent some time with the Quest 2, I believe we might look back on it as the headset that finally made VR accessible to mainstream consumers.

AMD Smashes Revenue Records as Zen 3, Xbox Series X, PS5 Ramp Up

AMD's Q3 2020 results are in, and the results are excellent for the company, in every particular.

MSI’s Nvidia RTX 3070 Gaming X Trio Review: 2080 Ti Performance, Pascal Pricing

Nvidia's new RTX 3070 is a fabulous GPU at a good price, and the MSI RTX 3070 Gaming X Trio shows it off well.