When my Italian grandmother couldn’t find something, she always said a prayer to St. Anthony, the patron saint of lost objects. “Dear St. Anthony, please look around. Something’s lost and can’t be found.” Only she said it in Italian. People still pray to St. Anthony, but now, of course, there’s technology to help us locate something we’ve misplaced.
(Have you ever been in a parking lot or garage and pressed the button on your key fob to see if you could figure out where your car was? Thought so! And if you can’t find your car keys, there are any number of key finders on the market that let you track down your keys using your phone. What if you’ve misplaced your phone? There’s tracking technology for that, too.)
Now, thanks to researchers at MIT, there’s a robot that will not just locate your lost whatever, but will fetch it for you – even when your lost whatever is out of sight: stuck between the couch cushions, in a stack of papers on your desk, in that pile of clothes that have been piling up on the chair in the bedroom.
RFusion, is a robotic arm with a camera and radio frequency (RF) antenna attached to its gripper. It fuses signals from the antenna with visual input from the camera to locate and retrieve an item, even if the item is buried under a pile and completely out of view. The RFusion prototype the researchers developed relies
on RFID tags, which are cheap, battery-less tags that can be stuck to an item and reflect signals sent by an antenna. Because RF signals can travel through most surfaces (like the mound of dirty laundry that may be obscuring the keys), RFusion is able to locate a tagged item within a pile. Using machine learning, the robotic arm automatically zeroes-in on the object’s exact location, moves the items on top of it, grasps the object, and verifies that it picked up the right thing. The camera, antenna, robotic arm, and AI are fully integrated, so RFusion can work in any environment without requiring a special set up. (Source: MIT News)
on RFID tags, which are cheap, battery-less tags that can be stuck to an item and reflect signals sent by an antenna. Because RF signals can travel through most surfaces (like the mound of dirty laundry that may be obscuring the keys), RFusion is able to locate a tagged item within a pile. Using machine learning, the robotic arm automatically zeroes-in on the object’s exact location, moves the items on top of it, grasps the object, and verifies that it picked up the right thing. The camera, antenna, robotic arm, and AI are fully integrated, so RFusion can work in any environment without requiring a special set up. (Source: How does RFusion work?
How does RFusion work?
First off, you have to attach an RFID to the items of interest. The robot then pings the RFID using an antenna. It then uses positioning technology to calculate the distance to the tag. There’s then a 3-step process in which RFusion combines the RF with visual info to pin down the location and retrieve the item.
The first step deploys Dense RF-Visual Geometric Fusion:
Given the round-trip distance to the RFID, the robot maps that distance to a spherical ring centered around the wrist-mounted antenna. Subsequently, it geometrically intersects this spherical ring with the RGB-D data obtained from the wrist-mounted camera, resulting in a list of candidate locations. (Source: MIT Media)
Next up, RF-Visual Reinforcement Learning. The robot then collects additional RF and visual measurements, and with an AI learning network starts homing in on the exact location of the target item. RF-Visual Grasping is the final step.
So far, the retrieval success rate for “fully occluded objects” is very high: 96%.
One of the researchers, MIT’s Fadel Adib says that you can think of it as “a Roomba on steroids.” (I think of it more like a really smart version of those claw machines they have in arcades, supermarkets, and bowling alleys. The ones your kids pester you to use, thinking that they’ll actually be able to claw a toy out of.)
The MIT researchers are looking beyond the “Roomba on steroids” (or claw machine) stage. While finding and fetching lost keys is certainly useful, the researchers envision many applications in manufacturing and warehousing, as well as in providing assistance to the elderly and disabled.
Definitely interesting technology that’s worth keeping an eye on. Meanwhile, prayers to St. Anthony can’t hurt.