February 2, 2018
Bionic hand: From a galaxy far, far away to Georgia Tech lab
By Katja Ridderbusch
In Gil Weinberg’s lab, science fiction has moved closer toward reality. The director of Georgia Tech’s Center for Music Technology has developed an ultrasonic sensor that allows amputees to individually move the fingers of their prosthetic hands. It’s a movement that even the most state-of-the-art, commercially available prosthetic devices do not offer.
Weinberg named his device “Skywalker” in honor of the hero of the ”Star Wars” saga who gets a fully functioning bionic hand after losing his own in a duel against villain Darth Vader.
“For sure,” Weinberg says, “Luke Skywalker’s bionic hand is more real today than 40 years ago.”
The first amputee to test Weinberg’s bionic hand was a young musician from McDonough—slim, pale, dressed in black and part of his hair pulled back into a small hipster bun.
“Being able to move all of my fingers again is just life-changing,” says 28-year-old Jason Barnes, as he plays the piano for the first time in six years.
Weinberg and Barnes
“Even though I’m more of a ‘Terminator’ fan than a ‘Star Wars’ type of guy,” Barnes adds with a wink, hinting at another science fiction classic featuring a bionic hand.
Doctors amputated Barnes’ right arm below the elbow after he was electrocuted during a work accident in 2012.
“I never thought I’d be part of developing the future,” he says.
The new sensor is still a prototype and for use only in the lab, says Weinberg. But the technology holds promise. “If you can play music, you can do almost everything,” he adds with a smile.
Weinberg, who is originally from Israel, was well on his way to becoming a concert pianist before he got interested in computer science, engineering and robotics in college.
“I thought, how can we combine these two things by building a computer software that understands music like us humans do,” he says. “Melodies and tensions and releases, all the building blocks of music.”
He finished an interdisciplinary master’s program at Tel Aviv University, then got accepted to the Media Lab at MIT, a program that combines communication, arts and technology. A month after he finished his PhD, he was hired by Georgia Tech to build the music technology program.
“It was a perfect fit,” he says.
The most advanced prostheses on the market today are so-called myoelectric devices, which Jason Barnes wears on a daily basis. The devices are controlled by electromyogram (EMG) sensors — electrodes that are attached to the muscles of the residual limb. The sensors pick up the muscles’ activity. By contracting and relaxing the muscles, amputees can move their prosthetic arms between different configurations.
But EMG technology ‘’isn’t accurate enough to get finger-by-finger movement,” Weinberg says.
Luke Skywalker’s device in ‘Star Wars’ film
His team found that when they looked at the muscle movements using ultrasound—with the same kind of probe that doctors use to monitor babies in a woman’s womb—they can see how the muscles shift to different locations.
“We can then use machine learning to understand the correlation between different muscles and what fingers are being pressed for which movement,” says 50-year-old Weinberg, who is soft-spoken with boyish looks, while displaying an aura of confidence.
He adds that instead of moving the prosthetic arm from one set of pre-programmed turns and grips to another, the amputee can now move fingers slowly and individually and do more sophisticated gestures.
He’s convinced that “this is the next step in powered prosthetics.”
A three-armed cyborg
Gil Weinberg and Jason Barnes first met a few months after Barnes’ accident in 2012. Originally a drummer, Barnes built his own very basic prosthetic device after he lost his arm. One of his teachers at the Atlanta Institute of Music, where Barnes studied at the time, saw a photo of a robotic marimba player that Weinstein designed in the Georgia Tech lab. “And then a friend of a friend introduced me to Gil,” Barnes recalls.
Weinberg says he was “just thinking how awesome it would be to embed the robotic technology into the human body, like, building a wearable robot.” A few days later, he met Barnes. “He walked into my lab at exactly the right time.”
Their first project was developing a robotic drumming arm for Barnes. The high-tech prosthesis that Weinberg’s team designed has two drumsticks. One is powered by EMG muscle sensors; the other moves on its own, using algorithms that enable the stick to “listen” to the music in the room and improvise accordingly.
The prosthesis turned Barnes into a three-armed cyborg, who’s able to play faster than any human drummer possibly could. Weinberg and Barnes became a worldwide sensation. They embarked on an international concert tour that took them, among many other places, to Brisbane, Berlin, Istanbul and Shanghai.
While the robotic drumming arm is only for a highly specialized target group, the ultrasonic sensor may have wider applications and help amputees with everyday activities, like grooming, typing, cooking or feeding. The technology could also boost the capabilities of remote surgical robots, says Weinberg, or help users control their smart homes.
To date, Jason Barnes wears the ultrasound-powered bionic arm only in the lab. “But eventually, I could see myself wearing it in the real world.”
He says the device itself is a little lighter than his everyday prosthetic. But it is not yet portable, Weinberg says, adding that he needs to improve the sensors, “making them smaller and more precise.”
He has patented his invention through Georgia Tech and is speaking with investors in the United States and Israel in order to commercialize the ultrasonic sensor.
Weinberg thinks that if his team keeps improving the sensor, or interface, and another company is working on the actual prosthetic device, “both could integrate with our software and we could have a fully functioning, ultrasonic bionic arm ready for market in two years.”
‘Reality is quite different’
“To see finger articulation happening, joints bending and all those intricate movements coming together is really exciting,” says Melissa Tober, an occupational therapist at Emory University. “And I hope researchers will keep working on this technology.”
But she’s also skeptical, and at times, concerned about too much enthusiasm. “Because in our clinical practice, the reality is quite different,” she says.
Tober, who has been an occupational therapist for 18 years, works with patients of many ages, backgrounds, and injuries. They have lost their limbs in an accident or as a result of an illness like diabetes or, in rare cases, flesh-eating disease, a life-threatening bacterial infection.
Every patient has a different disability, personality and requirement, she says. “Some may not even want to go with a prosthetic device.”
The most common arm and hand prostheses today are the so-called body-powered devices. They work by using the body’s own motion, like shoulders and elbow movements. They come as fake hands or hooks; their range of motion is limited, and they’re mostly worn for aesthetic reasons.
The latest generation of myoelectric devices, like the iLimb or Bebionic arm and hand prostheses, may offer a much wider and more articulate range of motion and a larger selection of pre-programmed grip patterns, “but they are not too common in our clinical practice,” says Tober.
For one, myoelectric devices are expensive. The most state-of-the art, commercially available bionic hands can cost up to $100,000. Unless it’s a worker’s comp case there’s little chance insurance will pick up the tab.
Also, many of those high-tech prosthetic devices are delicate, difficult to use and often, uncomfortable to wear.
Tober points out that training can be tedious. “It requires discipline, dedication, hours and hours of practice and a lot of patience,” she says.
As an occupational therapist, she’s seen a lot of patients give up on dealing with prosthetic devices. “They get aggravated, and nine times out of 10, they start to work just with their residual limb and after a while, forgo the prosthesis altogether,” she says.
Tober says she doesn’t know much about ”Star Wars,” but her husband and daughter have pestered her with questions regarding bionic hands.
“You know, science fiction never really comes up with my patients,” she says, “because I think this is a reality for them, a tool they use to be as functional as they can.” But also a tool that’s a constant reminder of a deeply traumatizing event.
In a perfect world, the future prosthetic arm and hand “would look and feel almost natural,” according to Tober. “It would be flesh-toned and lightweight, and it would be able to make somebody feel something—hot, cold, a simple touch.”
Tober may not be a ”Star Wars” fan, but what she gives is a good description of Luke Skywalker’s bionic hand. The proof is in the movies where, as surgical robots get Luke fitted with the prosthesis, they pinch his fingers, and he winces ever so slightly. In a later episode, he’s injured in his fighter jet and grasps his artificial hand in pain. All of those little details mean he obviously has sensory feeling in the bionic limb.
But except for some rare research into connecting nerves with prostheses through invasive surgery, the option of a prosthetic hand with full sensation won’t be available to patients for a while.
At least, that’s the common assumption. But Gil Weinberg is a bold thinker. So, when is he planning to equip his friend Jason Barnes with a perfect “Luke Skywalker” bionic hand, sensory feeling and full, fast and individual finger movement?
Weinberg doesn’t think that’s too far off. “You can create real innovation, science fiction style, but it takes a collaborative approach,” he says.
If there is a close cooperation between multiple research groups, he’s convinced that “we may soon have bionic hands that don’t just look like Luke Skywalker’s, but also act and feel like it.”
© Georgia Health News / Katja Ridderbusch