If you’ve ever picked tomatoes before, you know what to look for: tomatoes that are red and firm. It’s a more challenging exercise for robots working in greenhouses. They need to be programmed to recognize what a ripe tomato looks like, grasp it with the right amount of pressure and package it without damaging it.

Robots are often used in industries where they work in predictable environments, such as car manufacturing plants, where they perform the same tasks every day with accuracy and precision on a well-lit factory floor. Greenhouses pose more challenges for robots.

“Nowadays we would like robots to do tasks that are normally reserved for humans, tasks that are not exactly certain,” says Prof. Medhat Moussa, School of Engineering, who is developing robots for the Guelph Intelligent Greenhouse Automation System at U of G’s Robotics Institute. Unlike factories, greenhouses have “a lot of clutter in the environment. There are lots of changing circumstances and conditions.”

Greenhouse robots face greater uncertainty and unpredictability in their tasks. Tomatoes could be growing in any direction in the greenhouse, and the robot needs to find them, even if they’re hidden behind leaves or other tomatoes. “It’s like a jungle because the intensity of the plants is very high, and it’s hard to see the different fruits,” says Moussa.

Hundreds of different algorithms are involved in programming a robot’s decision-making capabilities. The robot uses a machine-vision algorithm to locate the tomato and analyze its colour to determine what percentage of red indicates ripeness. A tomato might be red on one side and green on the other, for example.

If the robot bumps into the plant while reaching for the tomato, the tomato could move, and the robot would need to recalculate its location. Even with all of these algorithms, the robot still needs to pick tomatoes as quickly and accurately as a human, ideally at the rate of one tomato every five or six seconds.

Using robots designed for automotive assembly, Moussa’s research team adds sensors, cameras and artificial intelligence for greenhouse applications. The robot’s “hand” is especially important, because it needs to grab the tomato without squishing it. The Robotics Institute has a research focus on developing sensitive grippers.

The main benefit of using robots in agriculture is to save on labour costs, says Moussa, who works with farmers in Leamington, Ont., which has one of the largest concentrations of greenhouses in North America. Agricultural labour, which is often supplied by foreign workers earning minimum wage, can amount to 30 per cent of a farmer’s total expenses.

“Agriculture is unique in the sense that it’s hard work and long hours, but the pay is not necessarily competitive with the automotive industry,” he says.

The robots are expected to achieve cost recovery within six years, but research advances and commercialization could make them even more affordable.

Robots won’t entirely replace human agricultural workers, says Moussa. “There are some jobs that are better left for humans that are too complicated and too risky, for example, pruning.” A wrong cut could kill the plant.

Once programmed for greenhouse applications, the robots could be used in car plants, where they could apply their tomato-picking skills to pick up car parts and tools.