Is that couple fighting or flirting? Most humans can tell the difference, but even the most sophisticated digital video cameras still have trouble making sense of what they see, says Graham Taylor.

The University of Guelph engineering professor and expert in machine learning will share $550,000 in federal funding for a three-year international research project intended to help make machine vision technology more discerning.

“Professor Taylor’s research on machine learning and computer technology will generate novel approaches to solve pressing problems in key sectors – food, agriculture, and environment – all areas where University of Guelph researchers also excel,” said Malcolm Campbell, vice-president (research).

Using a new Strategic Project Grant from the Natural Sciences and Engineering Research Council that was announced today, Taylor hopes to help computers “see” better in applications ranging from monitoring hospital wards to improving design of public spaces to deciding “whether that couple is arguing or whether they’re passionately in love.”

Computer vision systems are getting better at figuring out whether someone is running, jumping, waving or eating. They can also track your movements for interactive video games.

But computers still have trouble making sense of complex situations and images, never mind deciphering body language and facial expressions, Taylor says.

With digital cameras recording more footage from building monitoring systems to YouTube videos, we need smarter computers to make sense of massive amounts of image data. “We have to rely on computers to watch videos faster and draw inferences,” said Taylor.

He’s bringing together deep learning and structured models to help machines build accurate representations of what they see and to make sense of images in a manner similar to how humans learn about their surroundings.

Although Taylor is focusing on machine vision, he says these concepts apply to different machine learning problems.

He’s working with French research partners at INSA-Lyon and Pierre and Marie Curie University, and Greg Mori, a researcher at Simon Fraser University.

The project’s industrial partners include a company designing better ways to search through massive amounts of video camera footage. Another company makes interactive kiosk displays that detect users’ gestures rather than input on a keyboard or touch screen.

The project might also feed into Taylor’s work with other U of G researchers, including studies of video systems for detecting crop pests and monitoring farm field conditions.