Have you seen this salamander?

​On a warm, moist night in early spring, yellow-spotted salamanders emerge from the forest floor behind College-in-the-Woods, cross the Connector Road and head for the vernal pools and pond of the Nature Preserve, where they will breed. Because they are nocturnal and live most of their lives in burrows, their annual migration is a rare opportunity to assess their health and population size.

In 2011, Dylan Horvath, MS ’03, steward of natural areas, began using these migrations to collect data and photographs of the elusive salamanders, noting sex ratio, physical length and skin condition. But to better understand migration patterns and breeding practices, he needed a way to identify specific salamanders from one year to the next.

Christian Tejera and Jonathan Milongo Kazadi, computer engineering majors, were visiting the Nature Preserve in 2018 as part of a summer research program. “I met Dylan, and he offhandedly mentioned that he wished there was a program that could identify salamanders,” Tejera says.

The students took the idea to their advisor, Douglas Summerville, MS ’94, PhD ’97, professor and chair of electrical and computer engineering. He provided basic directions, help with algorithms and encouragement.

“It was a challenging project, and they spent the summer trying to see if it was even possible to create a system to recognize these salamanders,” Summerville says.

Turns out it was, using a process often associated with facial recognition. Only for this project, the students would focus on the salamanders’ distinct yellow spots.

Photos Pose a Challenge

What the students created is a system that will basically act as a biometric for the salamanders, to determine if a picture of a salamander has been seen before and identify which one it is, says Scott Craver, associate professor of electrical and computer engineering and undergraduate director.

The system is now an Avangrid Foundation-funded senior capstone project for Tejera. Milongo, a junior, is helping, along with computer science senior Jocelyn Ao. Craver, an expert in image processing, is their advisor. Avangrid provides Binghamton with an annual grant for capstone projects impacting energy or the environment.

This isn’t facial recognition, but the process is the same. Starting with a collection of images — in this case of salamanders — algorithms will identify certain patterns that can be used to distinguish one salamander from another.

But first the students had to solve a big problem: The 100-plus salamander photos provided by Horvath were of uneven image quality.

Distractions — long grass, poor lighting and awkward camera angles — are common in the photos. Then there are the salamanders: “They are sometimes not straight, and they are all over in the picture,” Milongo says. “Their positioning is not easy to work with. If they were all straight, it would be super easy.”

Once the students became familiar with Photoshop, they discovered that “straightening” some of those curved creatures isn’t that difficult. It starts with laying a digital mesh, or grid, on top of an image to isolate key features.

“We can put points inside each little polygon, and those points are connected by a line that will follow the spine of the salamander. When we straighten the line, the entire image will straighten,” Tejera says.

“You see a salamander, and that one spot — you would not find that spot on any other salamander,” he says.

An image library called Open CV has been useful for the image processing, and software called Image Magic is good at processing two images and giving the percentage of how similar they are to each other, Milongo says.

So far, they’ve made three or four matches, identifying the same salamanders in multiple photos.

Upload and Identify

While Tejera and Milongo built a database of identifiable salamanders, Ao worked on the frontend programming, building a web page for future users.

“All they need to do is upload a picture, plot some points to identify the salamander’s spine, and then click a button that will tell the program to start trying to identify the salamander,” she says.

Ao joined the project because she was looking for real-life applications for her programming skills.

“I didn’t start off with a lot of knowledge on the languages and the applications I had to use,” she says. But she invested time in learning more about how to use JavaScript and about hosting a web page.

“Professor Craver laid out the tasks for me and explained conceptually how it all worked, and then I would implement what he said to do,” Ao says.

By the beginning of the spring 2019 semester, the project was ready to move into the next phase: testing to see if the back-end and the front-end work together.

“Making the website was pretty easy, and making the program that can separate the salamanders is pretty easy,” Tejera says. “Incorporating it all together will be pretty difficult.”

Building a Useful Database

Eventually, the salamander photos will be kept on a server so that the database can expand as more photos are uploaded, which will benefit research and maybe answer Horvath’s questions about how often individual salamanders migrate and if they use the same path each time.

“Their habitat is often overlooked and prone to development. Flexibility in their migration patterns may make them just a little less vulnerable,” Horvath says.

Finding matches should become quicker, especially as more years of data accumulate, he says.

The possibility that individual salamanders would generate “years” of data surprised Tejera and Milongo, who initially guessed the amphibians might live three or four years.

Then, Tejera says, “We saw a salamander image from 2006 and it matched one from 2017.”

“I was shocked,” Milongo says. “When I looked it up, their life expectancy is about 15 years.”

Once the website is up and running, there’s just one task left for the students: to go into the Nature Preserve on a warm, moist night in early spring to try and see a live salamander.