February 14, 2002
Thanks to ‘Professor Picasso’ techniques of master artists just a click away
For Aaron Hertzmann, painting like a master is more a matter of algorithms than brush strokes.
The acting assistant professor of computer science in the UW’s Department of Computer Science & Engineering has developed software that can take your favorite snapshot and recast it in the style of a master painter.
Ever wonder how the Cascade view out your back window might look if Van Gogh had taken a shot at capturing it? Or how Georgia O’Keefe might transfer the prized blooms from your garden onto canvas? Hertzmann’s work in a project dubbed “Image Analogies” might give you answers.
“That’s one application of what we’re working on,” Hertzmann said. In collaboration with colleagues at the UW and computer scientists at Microsoft Research, he has developed software that can systematically transform textures and resolution for a wide range of effects.
The idea, according to Hertzmann, is to “teach” a program how to transform a photographic image into one that incorporates trademark patterns by a famous artist or of a certain style. Ideally, that would allow one to throw any picture at the software and come out with a rendition of what that image might have looked like had the artist painted it.
It’s a straightforward concept in theory, he said. But in practice it’s an extraordinarily difficult problem to solve.
The method uses two steps. In the first, the computer studies the pixels in a painter’s work, then looks at the relationship between the painting and an equivalent photo. Once it has the formula down, it applies it to other photos to recast them in the style it has learned.
“The challenge is that there is no unique way to interpret a pair of images computationally,” Hertzmann said. There’s a fine line between making exacting analogies while teaching the program how to recast a photo in a certain style and making those comparisons broad enough to apply effectively to other pictures. “Our algorithm is based on some assumptions about what is important in a painting style, but they are necessarily limited assumptions.”
The software works better with some photos than others — those with consistent light and balance probably work best as a template. And the software alters textures, not geometry. It would probably do a less than satisfactory job transforming a photo of your spouse into a Picasso.
In addition to artistic filters, the software shows promise in converting images to various general mediums. For instance, it can recast a photo as a charcoal sketch or a watercolor. It can also be used for landscape and texture synthesis, creating environments from bare-bones information. A flight simulator aficionado from Finland recently downloaded the software and applied it to maps of his country. The result was a realistic, if not completely accurate, aerial photo of the landscape represented by the map.
At this point, the software is confined to working its magic on still images. But Hertzmann plans to extend the principles to animation. In fact, he said, that was the original idea that started the work.
“I was thinking of ways to make life easier for animators,” he said. The thought was that if animators drew the first few frames, the program could learn how the images relate and do much of the subsequent detail work.
“I figured the computer could take it from there,” he said. “Then I realized that the technology had a more general application.”
Hertzmann has posted the software on the Web, opening the door for other academics and computer wonks to tinker, innovate and improve.
Hertzmann presented the project last August at SIGGRAPH 2001, the annual international showcase event for computer graphics, in collaboration with UW computer science and engineering faculty David Salesin and Brian Curless, and Charles E. Jacobs and Nuria Oliver with Microsoft Research.
For sample pictures and more information, including access to the Image Analogies software, go to http://grail.cs.washington.edu/projects/image-analogies/. A word of warning, however: the program isn’t extremely user-friendly.
“It is research code,” Hertzmann explained. “You kind of have to have an idea of what you’re doing to avoid frustration.”
