.png)
January 21, 2025
Our hands perform thousands of complex tasks every day – can artificial intelligence help robots mimic these extraordinary human limbs?
The human hand is one of the most complex and physiologically intricate parts of the body. It has more than 30 muscles, 27 joints, and a network of ligaments and tendons that give it 27 degrees of freedom. The palm alone contains over 17,000 touch receptors and nerve endings. These features allow our hands to perform an astonishing variety of highly complex tasks through a wide range of different movements.
Sarah de Lagarde can attest to this. In August 2022, she climbed Mount Kilimanjaro. Just a month later, she lost her right arm and part of her right leg in an accident. The prosthetic offered by the UK's National Health Service offered her little freedom of movement. After months of difficulty, she received a bionic arm that uses AI to anticipate her movements by detecting tiny electrical signals from her muscles.
Even the simple act of picking up a pen and rotating it in your fingers to write requires seamless integration of body and brain. Hand tasks that we barely perform consciously require a combination of motor control and sensory feedback – from opening a door to playing the piano.
Given this complexity, it's no wonder that attempts to replicate the versatility and dexterity of human hands have occupied physicians and engineers for centuries. From the simple iron hand of a 16th-century German knight to the first robotic hand with sensory feedback from 1960s Yugoslavia, nothing came close to the natural capabilities of the human hand. Until now.
Advances in AI are ushering in a generation of machines that approach human dexterity. Intelligent prosthetics, like de Lagarde's, can anticipate and refine movements. Robots pick strawberries and carefully place them in punnets without crushing them. Vision-guided robots can even remove nuclear waste from reactors. But can they ever compete with the capabilities of the human hand?
Much like a baby learns to use its hands, dexterous robots with embodied AI follow a similar path. Such robots need to coexist with humans in an environment and learn to perform physical tasks based on experience. They respond to their surroundings and adjust their movements to these interactions. Trial and error play a major role.
“Traditional AI processes information, while embodied AI senses, understands, and responds to the physical world,” says Eric Jing Du, a professor of civil engineering at the University of Florida. “It essentially gives robots the ability to ‘see’ and ‘feel’ their surroundings, allowing them to perform actions in a human-like way.”
However, this technology is still in its infancy. Human sensory systems are so complex and our perceptual abilities so adept that reproducing dexterity at the level of the human hand remains a formidable challenge.
“Human sensory systems can detect minute changes and quickly adapt to shifts in tasks and environments,” says Du. “They integrate multiple sensory inputs like vision, touch, and temperature. Robots currently lack this level of integrated sensory perception.”
But the complexity is rapidly increasing. The DEX-EE robot, developed by the Shadow Robot Company in collaboration with Google DeepMind, is a three-fingered robotic hand that uses tendon-like actuators. With fingertip sensors that provide 3D data, the device can handle delicate objects. DEX-EE is currently only a research tool.
Roboticists have long dreamed of automatons with human-like dexterity. Rustam Stolkin of the University of Birmingham is leading a project to develop AI-powered robots for handling nuclear waste.
A well-known example of a real-world android is Boston Dynamics' humanoid robot Atlas. The latest version combines computer vision with reinforcement learning, allowing the robot to perform complex tasks such as picking up objects.
However, the skills required for many tasks in sectors such as manufacturing, construction, and healthcare pose a particular challenge, says Du. “This is because most hand-guided motor actions in these sectors require not only precise movements but also adaptive responses to unpredictable variables like irregular object shapes, varying textures, and dynamic environmental conditions.”
Tesla, too, has given its humanoid robot Optimus a new hand by the end of 2024. The company released a video of the robot catching a tennis ball in the air. However, according to the engineers, it was remotely controlled and not autonomous.
While some innovators have attempted to replicate human hands and arms in machine form, others have opted for completely different approaches. Robotics company Dogtooth Technologies has developed berry-picking robots that use arms and precision pincers to pick and pack delicate fruits like strawberries and raspberries.
Dogtooth's robots use machine learning models. The robot’s arms each have seven degrees of freedom, like the human arm, allowing these limbs to maneuver to find the optimum angle to reach each berry without damaging others.
Robots that can perform some of the more delicate tasks currently performed by humans could provide a major boost to many industrial sectors, says Pulkit Agrawal of the Massachusetts Institute of Technology.
Over the course of a day, however, human hands perform thousands of different tasks and adapt to various shapes, sizes, and materials. Robotics still has a long way to go to compete with that.
Perhaps the ultimate application for robotic dexterity lies in prosthetics. The myoelectric arm prosthetic that Sarah de Lagarde received hints at future possibilities. A collaboration between several software and hardware companies enabled the development of this intelligent prosthetic.
Bibliography: https://www.bbc.com/future/article/20250121-why-hands-are-one-of-the-biggest-challenges-in-robotics https://darioamodei.com/machines-of-loving-grace https://www.discovermagazine.com/technology/humans-beat-robots-hands-down https://dev.to/evanmarie/the-ai-hand-conundrum-why-generative-models-struggle-with-human-hands-21ib https://medium.com/@bp_64302/the-problems-with-humanoid-robots-9d8684d62008 https://news.ycombinator.com/item?id=24207424 https://www.wired.com/story/darpa-arati-prabhakar-humans-machines/ https://www.linkedin.com/posts/anjanps_great-insights-here-spot-on-and-i-have-observed-activity-7272716476048896001-7R19 https://www.apa.org/monitor/2021/11/cover-artificial-intelligence https://en.wikipedia.org/wiki/Three_Laws_of_Robotics