While day one of Facebook’s F8 developer conference introduced a range of new, consumer facing functions and features, day two was more focused on the back-end, on the systems and innovations Facebook’s putting in place to power their vision. And while much of it was very developer-centric, there were some interesting notes and features discussed that will have wide-ranging impacts and benefits.
VIRTUAL REALITY AND SOCIAL MEDIA
Facebook Chief Technology Officer Mike Schroepfer was given the opening keynote for day two, and he set the tone of the day by touching on their efforts to connect the world and their advances in the development of new platforms like artificial intelligence and virtual reality.
Of particular interest was their work on social VR, which CEO Mark Zuckerberg noted on day one has the potential to be more social than any other platform. On this, Facebook showcased a video which showed how they’re developing technology to enable users to interact with real-world environments through the use of a VR avatar.
The technology still has a long way to go, but these are the first steps towards creating fully immersive, and interactive VR environments, which will enable people to be virtually present. Such technology could have a transformational impact on eCommerce – imagine being able to re-creates the physical, in-store experience without ever leaving your lounge room?
Facebook’s Applied Machine Learning team discussed the latest developments on artificial intelligence, and how AI powers various aspects of Facebook. Of particular note, artificial intelligence is a key element of Facebook’s News Feed – every day, users are served around 15 to 20 stories of interest to them, chosen out of the thousands of potential updates you could see. In order for Facebook to remain a compelling platform, they need to get this selection right, and AI is the key driver behind this service.
In total, Facebook’s back-end systems are completing around 6 million predictions per second across their AI platform to inform what appears in our News Feeds. That’s a staggering amount, and when you consider that both user numbers and engagement stats on the platform have continued to grow, it’s clear that their AI system is getting it right, at least in the majority of cases, which is a huge endorsement for the power of machine learning.
Facebook’s Applied Machine Learning team are now running 50x more AI experiments per day than they were a year ago, examining the capacity of the technology in a wide range of areas to improve the user experience. For example:
“…the Applied Machine Learning team built an AI-based automatic translation system that helps 800 million people every month see translated posts in their News Feed.”
Facebook’s translation solution is customized due to the way people actually communicate on Facebook, with different, regional language variations and expressions. And emoji.
Facebook’s AI team is also working to develop their automatic image recognition technology, which was released last week as part of Facebook’s new system to provide automated captions for vision impaired users. That technology may one day enable users to search by image content – this example shows how such a system is in progress.
While ‘snow’ is a relatively broad example, the next level of this technology will enable the system to understand images at individual pixel level using ‘image segmentation’. This will enable the system to recognize individual objects within an image, as well as their relation to the wider context of the picture.
As with automated captions, the immediate application of such technology relates to giving those who can’t see a more complete on-platform experience, but the extended applications of such technology are significant. Imagine being able to search Instagram for all images containing a specific product, a specific person. It’s a whole new element of social media data, and a whole new aspect to consider in a wider monitoring context.
Facebook’s also looking to extend their image recognition capabilities to video, including live content. In the video below, Facebook’s used their image recognition AI on video posts to identify related content by keywords, without the assistance of tags or other data points.
This technology could also be incorporated with Facebook’s facial recognition technology to identify and tag people in your videos – which is a little creepy, but may prove beneficial in certain applications.
Facebook’s Connectivity Lab also detailed how they’re developing new technology solutions to “help connect the unconnected and improve the experience of the underserved”. In line with this, Facebook has developed two new terrestrial systems:
“Terragraph, a wireless system for dense urban areas; and ARIES, a proof-of-concept research project to provide wide-coverage connectivity to areas with low population density.”
While these types of systems don’t necessarily have immediate bearing on the everyday users’ experience, the developments in themselves are amazing. When you consider them alongside other connective efforts like Facebook’s Aquila drone planes and investigations into laser technology to beam down internet connections to remote regions, the scope of Facebook’s projects comes into wide view.
Mark Zuckerberg noted in an interview earlier this year that he’s aiming to have 5 billion people using the social network by 2030, up from the 1.59 billion currently. When you look at their plans for connecting more people via these projects, that projection doesn’t seem so crazy after all.
In addition to bringing you a whole lot of insight and announcing new features at F8, Facebook has also worked to ensure everyone is able to experience the event, whether you were physically in attendance or not. As such, Facebook’s included a video page on the F8 website which includes all the F8 sessions – and there’s a heap of content there.
There were a heap of sessions at F8, all of them informative in their own way, yet not all contained big announcements or updates, so won’t get as much coverage. For example, there’s a great presentation from Liz Perle, a community manager from Instagram on the role community plays on the platform and how to build your own.
The full F8 schedule is available online, so you can scan through all the sessions and check out the video of each that takes your interest.