Your #sunset photos might have helped improve Facebook's AI


During the opening F8 2018 keynote, Facebook CEO Mark Zuckerberg showed off the company's latest Instagram updates: Spotify integration, AI-based anti-bullying comment filters, AR camera effects and four-way video chat. Facebook recently designed an A.I. program using Instagram images and hashtags to teach object recognition. Using hundreds of graphics processors, Facebook organized 3.5 billion images across 17,000 accounts and created machine learning models that can beat top-of-the-line industry benchmarks.

While companies and researchers around the world work to build the most advanced and powerful AI systems, Facebook has a special treasure trove that most don't: billions of tagged photos thanks to Instagram.

An image recognition machine needs to first learn what an apple looks like before it can recognize the fruit in other photos.

More news: Hawaii County expands evacuation after eruption

Interestingly, Facebook's AI scientists have found a different use for your hashtags. The increases in image recognition accuracy only were a couple of points in numerous tests, but what's fascinating are the pre-training processes that turned noisy data that was this vast into something effective while being weakly trained. But increased accuracy relies on scaling those models to billions of images, something that's essentially not possible if all the labeling is done manually.

For Facebook training, some models worked on as many as 50 million to billion sets of images, and it is unfeasible when all supervision is supplied by humans.

The effort is vital to Facebook as it aims to use AI for things like automatically identifying objectionable content or generating much more specific audio captions-like "sparrow" instead of "bird", or "Indian cuisine" instead of "food"- for photos for visually impaired users. So the company came up with new approaches to handling that noise, including leaning on the fact that many users add multiple hashtags-and thus more context-to the images they post.