Facebook’s latest AI project: Understanding user videos

As a student in the Master in Business Analytics & Big Data at HST, it’s always interesting to see where the current challenges of big-tech firms lie and how they’re tackling them. While Facebook has already undertaken several projects on computer vision which were trained to analyze user pictures, teaching machines to understand complex relationships like people do is still a tall order. If technology could comprehend user videos, a valuable new mass of data would be created, thus improving recommendation engines even further, to name one possibility. 

Reading user videos: a win-win for UX

Though Facebook is just scratching the surface with this technology, the use of machine learning on videos uploaded to the platform has already improved computer vision and speech-recognition systems. And no more than six months after developing the framework for understanding user videos, Facebook developed and deployed an AI model for its Instagram Reels recommendation system. Preliminary experiments showed that with the technology, there was a 20 percent reduction in speech recognition errors—in other words, the AI quickly got better. 

This technology could lead to improvements across a variety of applications, such as placing automatic subtitles over videos or identifying harmful content like hate speech. In the long term, users could even search their digital memories using specific keywords and parameters such as colors, phrases, or objects that appear in the video. For example, users could search for videos in which they sang a Christmas song for their parents.

By the time AR glasses become the norm, people should be able to retrieve specific moments from their vast collection of digital memories just as easily as they recorded them. Facebook is already working on AR glasses—but whether they’ll be equipped with cameras from the start is still unclear. From a data protection perspective, there are many issues that have remained unresolved since the release of Google Glass.

Future implications of the technology

As of now, Facebook has remained vague about concrete plans on how to use its computer vision machines—and so are their statements about the privacy policy. The corporation told The Verge magazine that such data could be used for creating advanced search capabilities, but did not clarify whether it would be used for targeted ads. And when asked if users would have the option to give their consent to having their videos used to train Facebook’s machines, the organization noted that its policy already states that customers’ uploaded content can be used for “product research and improvement.” Facebook also did not make clear how their researchers would be overseen when accessing user videos.

While this technology is sure to improve UX and make users’ lives easier, it’s yet to be seen whether Facebook will handle users’ data responsibly.

 

Born and raised in western Germany but with Romanian roots, Michelle Michalowski studied Economics with a major in Statistics at the University of Mannheim before coming to Madrid to study the Master in Big Data & Business Analytics. Apart from being interested in tech and data, she is also a fitness junkie and a travel lover. Connect with her on Linkedin.