Machine learning researchers team up with Chinese botanists on flower-recognition project

Has this ever happened to you? You're out walking with your daughter. She finds a beautiful flower, quizzes you on it, but you're stumped—you have no idea what it is. Instead of having to admit you don't know, what if you could quickly identify the flower or any other plant wherever you happen to be? But how? At least 250,000 species of flowers exist and even experienced botanists have trouble identifying them all. Now there's a way thanks to the rising power and sophistication of image recognition and the ease of taking pictures with your smartphone.

It's called the Smart Flower Recognition System but it might never have happened were it not for a chance encounter last year between Microsoft researchers and botanists at the Institute of Botany, Chinese Academy of Sciences (IBCAS). Yong Rui, assistant managing director of Microsoft Research Asia (MSRA), was explaining image-recognition technology at a seminar—much to the delight of IBCAS botanists whose own arduous efforts to collect data on regional flower distribution were experiencing poor results. The IBCAS botanists soon realized the potential of MSRA's image-recognition technology. At the same time, Yong Rui knew he had found the perfect vehicle to improve image recognition while addressing a reality-based problem that benefits society. It also helped that IBCAS had accumulated a massive public store of 2.6 million images. Since anyone in the world could upload pictures to this flower photo dataset—and no human could possibly supervise the uploads—the MSRA team had to create algorithms to filter out the "bad" pictures. That was the first of many difficult problems facing researcher Jianlong Fu and his team in building a tool capable of discerning tiny anomalies among the many species of flowers.

To do so they trained a deep neural network to recognize images using a set of learnable filters. In a nutshell, it works like this: During the forward pass, each filter is convolved across the width and height of the input volume, computing the dot product between the entries of the filter and the input. This produces a 2-dimensional activation map of that filter. As a result, the network learns filters that activate per specific types of features at a given spatial position in the input.

Inputting millions of pictures into the deep-learning framework, MSRA researchers eventually enabled the engine to accurately identify images more than 90 percent of the time, an astonishing rate just shy of human capabilities.

And the project greatly helped the Chinese botanists in meeting their goals. "The flower-recognition engine enables domain experts to acquire plant distribution in China in an efficient way," said Zheping Xu, assistant director of IBCAS. "Not only that, this engine can help ordinary people who have a strong interest in flowers to gain more knowledge."

In the future, MSRA and IBCAS will continue the collaboration, hoping to create applications based on the flower-recognition engine, so that botanists can conduct their research, parents can appear infallible to their kids, and everyone can appreciate flowers on an even deeper level.