Meta Platform Inc has unveiled its latest solution for processing and hosting next-generational artificial intelligence (AI) — the AI Research SuperCluster (RSC), it announced in a blog post on late Monday.
According to Meta, it plans to build the massive supercluster in mid-2022 and aims to become the world’s fastest supercomputer, namely for natural language processing (NLP), computer vision processing, and deep learning research.
Meta is announcing the AI Research SuperCluster (RSC), our latest AI supercomputer 💻 for AI research. RSC will allow our researchers to do new, groundbreaking experiments in #AI. Learn more about RSC and the important role it will play: https://t.co/l9CcQuFLyM pic.twitter.com/gD8Ve74ZqQ
— Meta AI (@MetaAI) January 24, 2022
Meta states the new system will also facilitate the development of future technologies for the Metaverse, which will rely on the new AI-based computing systems.
The company expects its efforts to develop “more accurate AI models” for current services as well as build “completely new user experiences, especially in the metaverse,” it said, adding future investments and self-supervised learning tools would help facilitate future Metaverse platforms and advance AI.
Future NLP Processing
Kevin Lee, Technical Programme Manager for Meta and Shubho Sengupta, Meta Software Engineer, explained in the joint post the new data centre would perform at “nearly 5 exaflops of mixed precision compute,” adding,
“Through 2022, we’ll work to increase the number of GPUs from 6,080 to 16,000, which will increase AI training performance by more than 2.5x. The InfiniBand fabric will expand to support 16,000 ports in a two-layer topology with no oversubscription. The storage system will have a target delivery bandwidth of 16 TB/s and exabyte-scale capacity to meet increased demand”
The RSC will allow Meta’s AI programmes to learn from trillions of language examples across images, text, and video to “develop new augmented reality tools.”
AI researchers for the Menlo Park-based firm can then train advanced AI programmes to enable real-time voice translations for large groups, including speakers of different languages, for numerous activities such as project collaboration or gaming.
Meta also explained its AI research investments began after founding its Facebook AI Research lab in 2013, leading to advancements in self-supervised learning capabilities.
Meta explained further, stating,
“To fully realize the benefits of self-supervised learning and transformer-based models, various domains, whether vision, speech, language, or for critical use cases like identifying harmful content, will require training increasingly large, complex, and adaptable models. Computer vision, for example, needs to process larger, longer videos with higher data sampling rates. Speech recognition needs to work well even in challenging scenarios with lots of background noise, such as parties or concerts”
Such technologies will also help to facilitate robotics and numerous AI modes to perform tasks in real-time, real-world use cases.
Meta also explained that its first iteration of the facility, in 2017, hosted 22,000 NVIDIA V100 Tensor Core graphical processing units (GPUs) to execute 35,000 training jobs each day.
Several longstanding partners, including Penguin Computing, Pure Storage, and NVIDIA, have provided critical infrastructure to develop the RSC control plane, storage, and cluster components.
The tech giant aimed later in 2020 to create improved infrastructure capable of training models “with more than a trillion parameters on data sets as large as an exabyte.”
The current RSC facility contains 760 NVIDIA DGX A100 systems, totalling 6,080 GPUs to provide “computer vision workflows up to 20 times faster,” the tech giant explained.
Security in COVID, Misinformation
Speaking further, Meta also stated had initially created the RSC amid the ongoing coronavirus pandemic and semiconductor shortages, using remote tools and cloud servers.
Such efforts required Meta to design the RSC “from scratch” with “Meta-specific conventions” to build system components such as its rack layout, cabling, cooling, power supplies, and networking, adding,
“We had to ensure that all the teams, from construction to hardware to software and AI, were working in lockstep and in coordination with our partners”
Meta also adopted its new AI Research Store (AIRStore) for its complex AI modelling techniques to minimise cross-region data flows for cross-region traffic on the server.
New AI models would also learn from encrypted “real-world data” separated from the Internet to protect user privacy while processing at the data centre. The RSC would also perform privacy review processes to safeguard data before encryption to regulate the flow of old and new data at endpoints.
The news follows significant developments on the company’s XLS-R language processing tool designed to effectively self-supervise and operate with more than 10 times more hours of public speech data.
The new NLP system has become one of many new solutions Meta has developed for the Metaverse, the future spatial communication tool expected to succeed the Internet, as well as to protect its users.
Meta revealed its ambitions to build the rising communications platform after Mark Zuckerberg, Meta Chief Executive and Founder, shocked audiences in early August with plans to rebrand as a Metaverse firm, which it later officially announced at the Connect 2021 event in October.
Meta’s latest tool is also a new company strategy aimed at tackling harmful content and online abuse, namely after it faced significant criticisms for allegedly failing to tackle such situations, leading to the death of a young woman on the social media platform, which the company has strongly denied.
Zuckerberg and other Meta executives have vowed to step up efforts to protect future Metaverse users with a series of funding, initiatives, technologies, and collaborative measures to tackle the reported epidemic of hate speech on its family of apps and ethically build the Metaverse.
