Facebook has just announced its ambitious new project: AI that can understand user videos the same way humans can.
As a student in the Master in Business Analytics & Big Data at HST, it’s always interesting to see where the current challenges of big-tech firms lie and how they’re tackling them. While Facebook has already undertaken several projects on computer vision which were trained to analyze user pictures, teaching machines to understand complex relationships like people do is still a tall order. If technology could comprehend user videos, a valuable new mass of data would be created, thus improving recommendation engines even further, to name one possibility.
Reading user videos: a win-win for UX
Though Facebook is just scratching the surface with this technology, the use of machine learning on videos uploaded to the platform has already improved computer vision and speech-recognition systems. And no more than six months after developing the framework for understanding user videos, Facebook developed and deployed an AI model for its Instagram Reels recommendation system. Preliminary experiments showed that with the technology, there was a 20 percent reduction in speech recognition errors—in other words, the AI quickly got better.
This technology could lead to improvements across a variety of applications, such as placing automatic subtitles over videos or identifying harmful content like hate speech. In the long term, users could even search their digital memories using specific keywords and parameters such as colors, phrases, or objects that appear in the video. For example, users could search for videos in which they sang a Christmas song for their parents.
By the time AR glasses become the norm, people should be able to retrieve specific moments from their vast collection of digital memories just as easily as they recorded them. Facebook is already working on AR glasses—but whether they’ll be equipped with cameras from the start is still unclear. From a data protection perspective, there are many issues that have remained unresolved since the release of Google Glass.
Future implications of the technology
While this technology is sure to improve UX and make users’ lives easier, it’s yet to be seen whether Facebook will handle users’ data responsibly.
Born and raised in western Germany but with Romanian roots, Michelle Michalowski studied Economics with a major in Statistics at the University of Mannheim before coming to Madrid to study the Master in Big Data & Business Analytics. Apart from being interested in tech and data, she is also a fitness junkie and a travel lover. Connect with her on Linkedin.