Amazon’s robots are getting closer to replacing human hands

In 2019, Amazon founder Jeff Bezos predicted that within a decade, robotic systems will be advanced enough to grasp items with the dexterity of a human hand. Three years later, Amazon looks to be making progress toward that goal.

A recent video published on the company’s science blog features a new “pinch-grasping” robot system that could one day do a lot of the work that humans in Amazon warehouses do today. Or, potentially, help workers do their jobs more easily.

The topic of warehouse automation is more relevant than ever in the retail and e-commerce industries, especially for Amazon, which is the largest online retailer and the second-largest private sector employer in the US. Recode reported in June that research conducted inside Amazon predicted that the company could run out of workers to hire in the US by 2024 if it did not execute a series of sweeping changes, including increasing automation in its warehouses.

At the same time, the company is facing the prospect of US workers starting to unionize after the victory by the Amazon Labor Union in the historic Staten Island vote, and another upcoming union election in October in Upstate New York. Labor activists have long speculated that Amazon might ramp up automation efforts in response to unionization activity.

In a statement provided by an Amazon spokesman, the company’s director of Robotics AI, Siddhartha Srinivasa, said: “[W]e have an incredible opportunity to help advance the science of robotic manipulation in ways that meaningfully benefit our employees and our customers. Our investments in robotics and technology are helping make jobs in our facilities better, easier, and safer, as well as creating new career opportunities for our people.”

The robotic arm in question does not look as futuristic as you might imagine. The proof-of-concept machine uses an off-the-shelf metal pincher rather than some novel grasping device. But it can pick up a new item and deposit it on a metal chute every three seconds. At the rate it’s going in the video, Amazon says the robot could handle more than 1,000 items an hour, meaning it could pick and stow items at rates several times faster than a human worker could. From a box of crayons to a container of what looks like garlic powder to a whisk broom, each item is grasped and moved with no human direction. The robot utilizes multiple cameras to help it “see” the assortment of items in front of it, as well as machine learning to help it decide the best way to pick up a given item, and motion-planning algorithms to help the robot navigate the crowded scene without bumping or damaging any of the goods. Preliminary tests also found that the robot damages certain products at a much lower rate than other manipulation robots Amazon has tested.

The video and the robotic system in it were created late last year in a controlled lab test by Amazon technologists. This robot prototype can only move items weighing less than two pounds. In testing, the robot was asked to handle hundreds of different items in this weight group and successfully grasped and moved around 95 percent of them, according to Amazon spokesman Xavier Van Chau. On a larger scale, the two-pound weight restriction would still allow the robot to grasp a selection of items making up about half of Amazon’s total product assortment. But the company is working on grasping solutions that would be able to handle any and every type of item that could fit inside an Amazon box, perhaps by combining a pincher attachment with a popular suction method, and having the system trained to know which “hand” should be used for which item.

How long it will take for Amazon to create a single robot that can handle the vast majority of products is up for debate, but it’s a question of “when,” not “if.” And when the “when” becomes “now,” we’ll have an answer to one of the great unknowns of this era of automation: Will a new generation of warehouse robots that can grasp goods almost as well as human hands make work better or easier for the people doing these jobs? Or will the technological evolution eliminate the need for these workers and their jobs?

An Amazon spokesperson said the company is betting on the latter, based on the way it has utilized other types of robots in its warehouses up to now. In June, Amazon announced a prototype of a robotic system called Cardinal that lifts and sorts already-packaged orders and, the company claims, “reduces the risk of employee injuries by handling tasks that require lifting and turning of large or heavy packages or complicated packing in a confined space.” The company says it expects to introduce the system into an unspecified number of fulfillment centers in 2023. And last year, the company unveiled another robot arm that it calls Robin, which handles a similar task with lighter packages. Van Chau, the company spokesperson, declined to provide details on the deployment of either the Cardinal or Robin robots.

Amazon’s history in robotics dates back to when it bought a company called Kiva for $775 million. In the decade since, it has rolled out more than 500,000 roaming warehouse robots. During the same period, the company says it has hired more than a million workers and points to this fact to try to dispel the notion that warehouse advancements are leading to worker elimination.

“From the early days of the Kiva acquisition, our vision was never tied to a binary decision of people or technology,” the company said in a recent blog post. “Instead, it was about people and technology working safely and harmoniously together to deliver for our customers. That vision remains today.”

The Kiva robots did make some Amazon warehouse jobs easier. For those workers in picker or stower roles, robots now transport shelves to them at a stationary workstation, where they stand for 10 hours a day with padding beneath their feet. In Amazon’s pre-Kiva days, these workers would walk 10 to 20 miles a day, plucking merchandise from, or adding goods to, aisle after aisle of inventory shelves.

Kiva robots also brought downsides. Before the robots arrived, a picker might have had a goal to handle 100 items an hour; Amazon tripled those expectations when the robots, not the workers, did the traveling. And with the addition of robots, injury rates increased as workers were forced to move faster to keep up with higher quotas.

The tasks being completed by Amazon test robots like the “pinch-grasping” one in the new video potentially have more direct overlap with existing worker tasks. The robot, like an Amazon picker or stower, is retrieving a piece of merchandise from one location and moving it to another, as quickly as possible without damaging it. That said, while the robot prototype is picking items at a rate of more than 1,000 an hour — around triple the typical rate of human pickers in Amazon warehouses — it’s not an apples-to-apples comparison. Amazon pickers in warehouses with robots have to pull each item out of a cluttered shelving unit, and sometimes have to use a step stool to reach merchandise at the top. Similarly, Amazon stowers have to fit each piece of merchandise into an open space on the mobile shelving unit, versus the robot that is simply moving it from one open space to another. Van Chau, the Amazon spokesperson, said the prototype in the video was neither tested nor designed to pick items from shelves as workers do in the company’s current robotic warehouses.

Still, robotics experts are paying attention. Martin Ford, the author of multiple books about robotics including Rule of the Robots, said while it’s unclear how Amazon’s most recent robot prototype would perform in a high-volume warehouse, it still seems to show “remarkable progress.” With advancements like Amazon’s, as well as those of many well-funded startups building robotic systems to try to solve for challenge of grasping with the dexterity of humans, “it’s inevitable that the problem will be solved — perhaps sooner than many of us expect,” Ford told Recode.

“And once that happens,” Ford added, “there’s little doubt that Amazon warehouses, as well as many other environments, are going to become a lot less labor intensive.”

Amazon maintains that robots and people will continue to work together inside its warehouses. But robotics experts say that one day, the company may have a real option to depend on robots to do a lot of the work it currently depends on human employees to do.