DEIXIS 2004 - 2005 THE DOE CSGF ANNUAL

Dances with Computers

By Victor D. Chase

Sommer Gentry was a swing dancer long before she became a fellow in the DOE Computational Science Graduate Fellowship (DOE CSGF) program, so when the demanding world of computers, mathematics and practical engineering entered her life, she did not let it replace her first love.  Rather, she found a way to combine the two seemingly disparate endeavors, while approaching each with equal gusto.

By literally combining business and pleasure, Gentry, 26, was able to bring the precision of mathematical analysis to her more intuitive pastime, while working to make computers more responsive to the subtleties of human touch.  In fact, the vivacious Massachusetts Institute of Technology Ph.D.  candidate says, it was the requirements of the fellowship that provided her with the opportunity to combine her vocation and avocation, and that also led her to change her dissertation topic.

The Los Angeles area native earned a bachelor’s degree in mathematical and computational science and a master’s in operations research from Stanford University in 1998.  Her affiliation with both DOE and swing dancing began right after graduation, when she took a job as a systems engineer at DOE’s Lawrence Livermore National Laboratory near San Francisco, a hotbed of swing dancing, where she caught the bug.

A year later, she decided to brave the cold and moved to Cambridge, Massachusetts, to begin working toward a doctorate at MIT.  At the time, her primary area of research was an esoteric field known as “inverse optimization.” Then one day in 2000, on her way to class, Gentry saw a poster promoting the DOE CSGF, and her life was about to change.

“I thought, ‘computational science, that sounds like me,’ ” said Gentry, so she applied and in 2001 was awarded a fellowship.  One of her first hurdles was to satisfy the requirement that fellows include some real-world engineering classes in their curriculum.  At first she tried to convince the powers that be that her prior work had satisfied that requirement, but “They said, no, no, those are math classes, you need something hands-on,” she recalls.  In response, she signed up for a class entitled “Space Biomedical Engineering,” in which she learned about the mechanics of jointed systems, robots and humans alike.  And that’s where swing dancing comes in.

It’s About Illusions

Swing dancing is a historical dance form, and Gentry and her husband, Dorry Segev — a surgeon and transplant fellow at Johns Hopkins Hospital, whom she met through dancing — specialize in the Lindy Hop, popular during the Big Band Era of the 1930s and ’40s.  In fact, Gentry and her friends attend dance events for which they dress up in clothing from that era and dance to the sounds of Benny Goodman and Glenn Miller.  They also watch videos of the original swing dancers strutting their stuff, which often leads to intense discussions of who is doing what to whom in this largely improvisational dance form.

“A lot of dance is about illusions,” says Gentry.  “It may look like one person is pushing on another when they really aren’t, or it may look like something is really light and effortless but two people are working very hard to hold each other up.” Hence, she adds, “The most difficult thing to do in swing dancing is to learn how something feels.”

So it was to learn from the now-departed swing dance masters of yesteryear that Gentry created her first project for the biomedical engineering class.  To do so, she applied her inverse optimization expertise to calculate the forces between people by observing their moves on the videos.

This involved watching someone move and then calculating how much force was applied to make the move.  “I wanted to take it out of the realm of artistic judgment and say that, physically there’s something going on. There is a person who weighs a certain amount, and is balanced in a certain place, and either he is holding her up or he’s not,” says Gentry.

In doing so, she also brought peace to her swing dance community.  “I just wanted to help my swing dancer friends who were arguing about what a video really showed; was this person pulling on the other person or not? I wanted to be able to give a definitive answer, and that’s really what I did.”

Making a science of art may also have helped Gentry and her husband place high in several international swing dance competitions.  The couple came in fifth in the 2001 and 2003 national Lindy Hop championships and took first place in the 2002 U.K.  Lindy Hop Open.

The judges were not the only ones who were impressed.  Gentry’s advisor, who saw her swing dance analysis as a rich area of investigation for a number of areas of practical applications to human/robotic interactions, suggested that she change her thesis from the theoretical to one that allows her to continue creating engineering models for swing dancing.  Gentry jumped — or perhaps did a flip — at the chance.

A Mouse with Feelings

It was then that she began to dance with a PHANTOM.

In swing dancing, the leader not only guides his partner through touch but also gives signals as to what the next move will be.  As Gentry puts it, “A leader is not just helping his partner by holding her up or fixing her balance, he’s also telling her what move we are doing next.  So there is a communications system laid on top of a physical control system, and they interact without words.” This made the PHANTOM a perfect partner, since it is a haptic force-feedback device that provides virtual tactile sensation.  The device includes a jointed mechanical arm that is used like a mouse but has feelings that can present forces.  If a rubber ball appears on a computer screen, for example, and the user manipulates the PHANTOM to point to it, she feels as though she is pushing against a rubber ball.

Gentry carried her video analysis work several steps further by programming the PHANTOM to act as a swing dance partner through touch, even to anticipating her moves.  In doing so, she helped make human/machine interactions more natural, or less machine and more human.

This work has important potential beyond dancing with computers; robotic surgery provides a case in point.  In this relatively new field, robot “hands” enter the human body to perform surgical tasks under a physician’s guidance.  This makes surgery less invasive, since the robot requires less space in which to operate than do a doctor’s hands.

Gentry envisions a surgical robot that not only follows but also anticipates the moves of the surgeon.  “So if the robot can figure out that you are trying to tie a knot, it might make it easier for you to grab the free end of the suture just by pulling you toward where it knows it is.  Part of that would be figuring out from the surgeon’s hand motion what task he is doing, which is the same thing the follower does in swing dancing.  By a little bit of push-pull, you know he’s doing move A or move B,” says Gentry.

Ship Shape

It was her interest in robotics that led Gentry to investigate DOE’s Sandia National Laboratories, Albuquerque, New Mexico, as a place to do her required practicum.  She was especially attracted to Sandia’s Intelligent Systems and Robotics Center (ISRC), where a robotic ship-welding project named AUTOGEN was in progress.  So last February she packed her bags and headed west.

AUTOGEN is designed to automate completely the robotic welding of ship parts.  Currently, robotic welding is used for relatively simple tasks, since the time required for a human to feed instructions to a robot as to how to navigate complex nooks and crannies is frequently prohibitive.  AUTOGEN would allow ship designs created on a computer to be fed directly into a computerized robot system, which would automatically calculate how best to weld the pieces together and then how best to direct the robot to do so.

For Gentry, the project was made to order, because it involved solving the same sorts of problems she dealt with in her swing dance analyses, namely, where the robot is in space at any given time.

When she arrived on the scene, AUTOGEN was already in existence, but it relied on the positioning of the robot and the piece to be welded in a fixed configuration.  “I asked, ‘Why don’t we ask if there is a better place to put the robot and the piece in relation to each other?’” says Gentry.  In response, she created an optimization routine to move the robot, or the parts, “to allow the robot to reach more of those weird corners,” she explains.

Ordinarily, doing so would require inordinately large calculations.  To get around this problem Gentry made some generalizations.  “I abbreviated the search for ways to weld the piece by estimating how well the robot would be able to reach into hard-to-get-at crevices, instead of doing a full planning procedure.  That resulted in some really neat improvements,” she says.

Her Sandia practicum advisor, Arlo Ames, puts it in bolder terms.  “She produced, more rapidly than I’ve ever seen, an answer that is a very significant piece of the overall product,” he says.  “AUTOGEN is a big, difficult thing.  I suggested to her where to look, and she jumped in.  The next thing I knew, she had code up and functional.  It’s working, it’s very fast, very efficient, and does the right thing.  The code she wrote is essentially going verbatim into AUTOGEN.”

The PHANTOM Returns

Unbeknownst to Ames, a PHANTOM was delivered to a different department at Sandia while Gentry was working on AUTOGEN.  She got wind of the fact and was delighted to have the opportunity to help set it up.  “I jumped on the box as soon as it arrived and started running some more of my swing dance experiments,” she says.  Though Gentry makes it sound easy, setting up the PHANTOM requires know-how that Sandia was considering obtaining through an outside source.  Instead, “just because Sommer was there, had prior experience, and is of a natural inclination to jump in and help out,” the lab saved a considerable amount of money, says Ames.  In fact, he says he was not even aware of Gentry’s extracurricular efforts with the PHANTOM “until it was half done and people were excited,” he says.

The feelings about Gentry’s practicum experience are mutual.  “I spent three months at ISRC.  I had a wonderful time, and it gave me the opportunity to explore robotics more fully,” she comments.

For his part, working with Gentry gave Ames renewed hope.  “In recent years I’ve fallen off a lot on mentoring students because I am tired of the effort.  I’ve gotten rather cynical about things, and the amount of my time required to get somebody up to speed was getting to the point where I was less and less willing to even try.  With Sommer I have renewed enthusiasm that I can get something meaningful done with this kind of temporary workforce.”

“Working with Sommer was a genuine pleasure,” he adds.  “I can’t imagine a better working relationship; she came, she saw, she did.  If I could hire her I would, but she frankly has places she’s going and things she’s doing.”

The Krell Institute
http://www.krellinst.org