Watson College professor named IEEE fellow

Binghamton computer scientist Lijun Yin believes that “the heart of the human problem is the problem of the human heart.” In other words, he said, technology’s growth hinges on the involvement of humans to solve the problems we ourselves have created, whether it’s tinkering with artificial intelligence or detecting deep fakes.

Yin, a SUNY distinguished professor, has dedicated more than 20 years of his career to studying human-computer interactions and computer vision, centering people — their faces, emotions and bodies — in his research.

For his contributions in the areas of automatic facial behavior analysis and recognition, Yin was recently recognized as an Institute of Electrical and Electronics Engineers (IEEE) Fellow. The prestigious distinction is reserved for individuals “with an outstanding record of accomplishments in any of the IEEE fields of interest.”

“That’s an honor, hopefully not only for myself, but also for my group, for my lab, for my students, for my collaborators and also for the University,” said Yin, who joined Binghamton’s faculty in 2001.

Yin, director of both Binghamton’s Research Center for Imaging, Acoustics and Perception Science and the Graphics and Imaging Computing Laboratory, is a trailblazer in the fields of 3D and 4D facial modeling. His research has racked up more than 11,000 citations, according to Google Scholar.

“Over his decades-long career, Professor Yin has been a pioneer of facial analysis, and his extensive database of three-dimensional scans have been used for everything from cybersecurity to animation,” said Atul Kelkar, dean of Thomas J. Watson College of Engineering and Applied Science. “His elevation to IEEE fellow is well deserved and highlights the ongoing importance of his research.”

The first of such databases, released in 2006, was groundbreaking at the time: Nobody had considered working with faces in entirely 3D space before. Each iteration since then has added dynamics, spontaneity and, most recently, corresponding vital signs like heart and breathing rates.

“Professor Yin’s elevation to IEEE fellow is a well-deserved recognition of his important contributions to computer vision, especially to human facial expression modeling and analysis,” said Weiyi Meng, chair of the School of Computing and a SUNY distinguished service professor. “In 2024, Professor Yin was promoted to SUNY distinguished professor in another recognition of his research impact. The School of Computing is very proud to have accomplished faculty members like him.”

Beyond security and entertainment, Yin’s databases present numerous possibilities: Doctors could use modeled expressions as reference points to assess their patients’ pain levels, researchers can detect deepfakes by looking at hidden pulse signatures inside video footage, and 3D avatars could even train children with autism how to recognize certain expressions.

Yin works with psychology experts including Binghamton Professor Peter Gerhardstein, who helped design protocols and recruit students, and he has partnered with Binghamton Artist in Residence Andrew Horowitz to elicit genuine emotional reactions from study subjects. His partnerships also extend outside of Binghamton, with collaborators at the University of Pittsburgh and the Rensselaer Polytechnic Institute aiding with data annotation and generation.

He credits his department, colleagues and students for his successes.

“I cannot achieve this without those people’s help and collaboration. I cannot achieve this by myself,” he said. “The initial idea is me, but [for] the real implementation and execution of those projects, of course, I rely on so many people.”

Yin has earned several patents and other accolades for his work, including election as a senior member of the National Academy of Inventors in 2021. The paper his team published in 2006 detailing the first 3D facial expression database also won the IEEE’s Test of Time Award at the 2024 International Conference on Automatic Face and Gesture Recognition — a distinction given to research with lasting and profound impact.

“This data became the benchmark,” Yin said. Indeed, from Hollywood to scientists to Zoom, his lab’s datasets have been used by more than 1,500 research groups and licensed to about 40 companies around the world. He estimates that in the last two decades, students from various universities have defended around 100 doctoral theses based on that data, too.

With each new generation of data, he also has seen his department transform. Students he worked with have become professors at Binghamton or other institutions, and some have even become collaborators.

“One of the happy things for a professor is not only your achievements by yourself. You also see your students’ achievements. That’s even more rewarding,” said Yin, who keeps his own doctoral students’ theses stacked atop a shelf in his office. “You want to see your next generation be even better than your generation. Then you can feel something has been achieved.”