Student researchers at Stanford University have created a pair of augmented reality (AR) smart glasses capable of leveraging OpenAI’s ChatGPT-4 chatbot.
The added plugin allows the students to speak directly to ChatGPT via smart glasses, merging generative artificial intelligence (AI) with AR overlays.
say goodbye to awkward dates and job interviews ☹️
we made rizzGPT — real-time Charisma as a Service (CaaS)
it listens to your conversation and tells you exactly what to say next 😱
built using GPT-4, Whisper and the Monocle AR glasses
with @C51Alix @varunshenoy_ pic.twitter.com/HycQGGXT6N
— Bryan Hau-Ping Chiang (@bryanhpchiang) March 26, 2023
Bryan Hau-Ping Chiang, Alix Cui, Varun Shenoy, and Adriano Hernandez, among others, led the experiment by creating RizzGPT. The tool combines AI, AR, and OpenAI’s speech detection tool Whisper to provide responses for conversations. Students generated answers with AR glasses and monocles.
Chiang tweeted
“Say goodbye to awkward dates and job interviews. We made rizzGPT — real-time Charisma as a Service (CaaS). It listens to your conversation and tells you exactly what to say next. Built using GPT-4, Whisiper and the Monocle AR glasses”
Chiang also lauded Brilliant Labs for their AR solution, which could “clip onto any pair of glasses [and] has a camera, microphone, and high-res display.”
He explained how the generative AI programme leveraged multimodal perception capabilities using audio, text, and images to “help AI understand what’s going [on] in your life.”
Chiang added: “This context is key for the AI to provide [hyper-personalized] support.”
forgot to tag @Adriano34554795, the memelord himself
thanks to @brilliantlabsAR for providing the Monocle AR gadget — super easy to clip onto any pair of glasses + has a camera, microphone, and high-res display
this project is perhaps best put by @varunshenoy_
😎 + 🧐+ 🤖 +… pic.twitter.com/97rVVnVC8I— Bryan Hau-Ping Chiang (@bryanhpchiang) March 26, 2023
Stanford’s research team stated their Charisma-as-a-Service (CaaS) had numerous use cases. These could cover diverse applications such as helping people with social anxiety, job interview, public speaking, and others.
The tool uses Whisper’s speech recognition tools and responds to conversations based on context and semantics. Brilliant Labs, creators of AR monocle technologies, have also contributed to the research.
ChatGPT Takes on Extended Reality
The news comes as multiple companies integrate ChatGPT variants across their family of products to streamline user experiences.
ChatGPT uses machine learning (ML) to aggregate data from the internet, generating content such as images, sound, documents, and chat texts.
OpenAI’s solution comes from the Generative Pretrained Transformer (GPT) family of AI apps. Numerous firms such as NVIDIA, Meta, Microsoft, Google, Unity, and many others have incorporated versions of GPT on their own solutions to increase content development dramatically.
Other companies such as Moth and Flame, ARuVR, VirtualSpeech, and others have used Generative AI interfaces to deliver on-the-fly virtual reality (VR) modules.
