XR a ‘Tool for the 21st Century Firefighter’, Execs Say

Three Key Execs in the MR Industry Discussed the Exciting Potential of XR to Save Lives

9
Firefighter
Mixed RealityInsights

Published: August 12, 2021

Demond Cureton

A rising number and frequency of fires in places such as California, Turkey, Greece, Australia, and others, as well as the ongoing coronavirus pandemic, have forced fire departments to seek smart solutions for researching, training, and deploying the firefighters of today and the future.

XR Today spoke with Kevin Sofen, Business Development Manager for WS Darley, Logan Parr, Director for Marketing and PR for Pico Interactive, and Cpt Kirk McKinzie, President of McKinzie Smart Technologies on the growing demand for extended reality (XR) solutions in the firefighting and first responders industries.

The Rise of XR in the Firefighting Industry

Speaking on the initial challenges Darley and others aimed to tackle, Mr Sofen explained preventable deaths among firefighters were “unacceptable” and many trainees had failed to avoid similar accidents.

VR solutions were “practical and useful” tools in training new recruits and Darley began to explore ways to build them to instruct first responders roughly seven to eight years ago.

Sofen and his colleagues had begun working with FLAIM Systems and RiVR at the time, and despite setbacks and an initial reluctance to adopt new technologies, progress had been achieved in the industry.

He explained further:

There’s a phrase that the fire service is ‘100 year of tradition, unimpeded by progress’, but there is some truth to it, considering that with VR training, we’re not replacing live fire training and other hands-on learning, which is still very important, but VR is becoming the continuum of training”

Sofen, who has worked with WS Darley for over nine years, added new solutions developed by the firms allowed new recruits to train “anytime, anywhere” with cost-effective solutions across the United States, North America, and globally.

Speaking further on the matter, Logan Parr told XR Today industry-wide adoption of VR training solutions had initially been “a bit rough” due to reluctance among departments, but the ongoing COVID-19 pandemic had forced firms to seek solutions for remote work, education and training for staff.

Global firms scrambled for new solutions amid COVID-19 to “keep engagement and training flowing,” namely among remote workers.

The Pico exec added post-pandemic demand would trigger an upswing of VR adoption, stating:

“We’re going to see a trend of adoption of these technologies, more and more, as we go into that ‘new normal’ and post-pandemic situation, now that people have been exposed to how easy it is, the benefits they see in virtual reality, and I think it’s a new business model that many people will be investigating”

McKinzie, who has over 30 years as a veteran firefighter with the Cosumnes Fire Department in California, explained how firefighting traditions and initial scepticism towards immersive learning shifted as the industry battled a rise in emergency phone calls and continuous deaths for nearly 30 years.

According to the veteran firefighter, US firefighters faced over 1.25 million fires, 240 million service calls, and roughly 3,000 fire deaths each year, which demanded first responders “innovate, in the moment, on the calls.”

“So, while we appreciate our traditions, we’re also innovative, and I’m grateful for the teams who have produced content that allows for Public Education and training,” he said.

Technologies Used in Immersive Firefighter Training

Speaking further about the technologies employed in immersive training sessions, McKinzie explained modules included 360 videos with embedded temperature content used for public instruction.

Such tools educated learners on the differences between kitchen and wildland fires, as well as important safety protocols, he told XR Today, citing open-source work from the NIST Fire Lab in Maryland.

The lab, run by Dr Matt Hoehler, had published an open-source piece viewable on phones and heads-up displays (HUDs), he explained.

Dr James Mullins, Chief Technology Officer at Deacon University’s FLAIM Systems in Melbourne, Australia, had also showcased a Haptically-Enabled Hot Fire Training System.

While visiting the Game Developers Conference in San Francisco, Mr McKinzie had only imagined it was possible and encouraged his work on VR trainer research.

He also considered ways to incorporate mixed reality (MR) systems with computational fluid dynamic fire models to add realism to simulations, but such solutions were under development at FLAIM.

McKinzie explained the FLAIM system, stating:

“There’s thermal feedback. The nozzle has reaction force and a full system with 40 different scenarios, running from simple to complex, and in the last few days, we’ve talked about ‘digital scents’ to educate the public, especially our young members, about the difference between the smell of a roasted marshmallow and of burning plastics, and how to train our firefighters”

Immersive Trainer Efficacy

According to a growing body of scientific work and peer-reviewed papers, immersive learning platforms showed a high degree of efficacy, McKinzie said, adding new MR labs were being built across the nation.

A colleague working with California’s Cosumnes CSD Fire Department, Justin Quarisa, began comparing traditional training methods with interlinked Pico headsets.

The simulation included a fire built to allow workers at the National Aeronautics and Space Agency (NASA) to train artificial intelligence (AI) programmes on future firefighting techniques.

Quarisa found that, compared to live fire training, lectures, and others, roughly 80 percent of cadets using the 360 video exercises said they had learned more than via traditional means, McKinzie said.

Parr added partners found engagement and learner retention had remained high with VR training programmes with “long-lasting” results. Immersive learning experiences also provided “a more sympathetic approach” compared with reading case scenario texts.

Mr Sofen continued, stating the industry has responded well to immersive content produced by firms such as RiVR and FLAIM, adding:

“I think the contextual nature of creating a hundred different scenes within the confines of a conference room, and going from an electrical power fire to a Hazmat scene, to a residential or apartment fire… I mean, just being able to do all those things allows you this ability, from a trainee or student perspective and within a 30-minute time frame, that you couldn’t do before”

Such work had previously been “nearly impossible” due to costs, environmental and contextual issues, but teams could create 360 scenes with low-cost cameras once for indefinite reuse, Sofen explained.

“You don’t need to have Steven Spielberg or Hollywood-style content to learn from it,” he said.

Sofen also urged fire departments to create 360 video teams to quickly create firefighting scenarios, which he believed the market was revolutionising and adapting across North America.

McKinzie added it was important to minimise cancer exposures and consider environmental, costs, and staff hours needed for traditional training compared to VR, which could be deployed anywhere for public and first responder use.

Citing NIST Fire Lab’s Eye of the Fire project and the New Zealand government’s Fire Escape 360 experience, which encouraged the public to develop fire escape plans, McKinzie explained:

“Kids need to know two ways out. You need to have a Plan A to action, because you don’t have 15, 12, 9, or 7 [minutes]… you might have less than three minutes.” 

Fires could reach 1800 F “at the floor in less than two minutes”, with combustible furniture and materials expediting house fires and quickly reducing escape times, he added.

To combat this, McKinzie urged every agency to develop a 360 kit with live streaming capabilities in headsets for chief officers to review.

According to McKinzie’s vision for future fire departments, firefighters would deploy computer vision AI “run in the cloud” along with edge computing, digital field assistants, and others. 360 cameras would also help teams evaluate fire progress and conduct post-incident analyses.

Pico Interactive Headsets: The Preferred Choice

When asked why companies such as Darley chose Pico Interactive Headsets for their VR immersive training courses, Logan Parr explained his company’s strict purchasing process.

He explained:

“I think the first thing is that you don’t just walk up to a store and buy a Pico headset. We have a global sales team, and our first goal is to understand what the partner’s needs are”

Pico worked with clients to understand specific goals and objectives for their business models based on each customer’s sector to help them choose “which headset would be perfect for them” and why.

The San Francisco-based VR headset firm provided G2 headsets with 4K resolution, which were optimal for producing clear, hi-res video. Pico Interactive’s Kiosk Mode provided a simple interface for content, and the company offered additional SDK and development support for clients.

Pico also worked with technology partners that help clients port and create content, as well as develop apps and facilitate remote device management to upgrade and synchronise devices. Additional feedback from clients allowed Pico to improve research and development (R&D) efforts and build future VR initiatives.

Speaking further, McKinzie recalled a conversation with FLAIM’s Dr Mullins’ experience with Pico headsets, who “sold hundreds” for use with FLAIM extinguisher kits to train civilians and office staff.

Both operators and professionals assisting first responders needed fire extinguisher training, which many places required annually, including for insurance purposes, he explained.

Such training also allowed fire departments to stop using petroleum, and VR trainers had successfully instructed civilians on choosing correct extinguishers for wood, trash can, kitchen, and other fires.

He explained:

“People feel a sense of urgency and get to try the dry chemical or CO2 extinguisher, and if it’s a stovetop fire and they try the water extinguisher, they get the reaction that will help them learn, [which is] that water on a grease fire causes a significant issue”

Kevin Sofen also explained how Darley had “very specific customer needs” due to collaboration with the US Department of Defense (US DoD) and Department for Homeland Security (DHS), among others.

Speaking on security needs, Sofen explained:

“We think, if a headset were to get stolen, how can we ensure that its wiped and they’re password protected? It’s those little features and nuances that have existed, where we would ask Pico and they’d change them all the way, and our customers were happy”

Pico’s open-source platform also facilitated bespoke demands for clients, and Darley’s extinguisher kit simplified end-user processes with over-the-air (OTA) updates via Wi-Fi, replacing USB upgrades.

Rapid Deployment, Rapid Results

According to Cpt McKinzie, California had seen several challenging years of fire outbreaks, with fires growing “in size and complexity,” prompting roughly 7,000 firefighters responding to a recent check.

He explained,

“Three homes that I’ve helped build were lost in the Oakland Hills fire years ago and, worst of all, 25 deaths, but we’ve surpassed that very recently, two years running”

Fire risks were not expected to decrease across California and other parts of the world, he said, citing the US Fire Administration, who found fire-related deaths had increased 24 percent over the last decade.

The ongoing COVID-19 pandemic had worsened training responses due to social distancing and health risks, but VR training allowed learners to train while in quarantine or receiving medicine.

Sofen added first responders would still have to tackle fires “rain or shine, pandemic or not”, despite COVID-19 restrictions, but VR technologies facilitated remote training, stating:

“For example, you could have a trainer in Charlotte, North Carolina, and I could have students in Denver and the Cosumnes Fire Department, all on the same Wi-Fi channel and virtual classroom, with the same instructor teaching the same content. This is now possible today, which gives tools to the 21st century firefighter that we need”

He noted hurricane season also concerned first responders and fire departments faced recruitment shortages, but VR training allowed to streamline recruitment processes and reduce training costs.

Citing work from Dr Skip Rizzo, Research Director at the University of Southern California Institute for Creative Technologies for Medical Virtual Reality, McKinzie discussed how VR could help pre-expose trainees to post-traumatic stress using immersive environments.

Cpt McKinzie also recalled 10,000 calls for service over his career and discussed Cmdr Paul Speight from the UK, who heavily promoted Britain’s “Don’t Text and Drive” campaign using immersive learning.

McKinzie said:

“He did a great immersive learning application […] to see what it’s like to be a patient in the back of a car where the roof is cut off and the patient is put into a helicopter and flown off scene. It’s a very powerful learning message”

According to Logan Parr, initiatives from partners working with Pico showcased “what’s possible in VR” and helped push boundaries for other firms to explore the potential in immersive training.

Commander Speight partnered with colleagues who utilize 3D laser scanning, photogrammetry and 360 video to produce a wide range of Reality in Virtual Reality (RiVR) content for use in Pico HUDs. 

Many companies also saved costs as they were not required to build huge labs for fire scenario training and additionally, could work remotely across the world via their VR headsets.

Logan Parr concluded many clients were at the early stages of exploring VR capabilities, and Pico Interactive aimed to help them “take VR to that next level and help improve their own business model.”

Kevin Sofen added VR was not “just a game,” but gave the industry practical training applications to first responders, adding: “There’s no industry that isn’t immune from the opportunities VR can assist.”

Cpt McKinzie concluded “we live in a 3D world” and immersive learning was “here to stay” with an agnostic, ubiquitous presence across professions, stating: “I’d look for other people that want to come blaze the trail, because I believe it’s here to stay, and I look forward to seeing what comes next.”

For more information, kindly visit Futura, Matchbox, and TechX for further MR experiences and real-world LIVE scenarios.

EducationImmersive CollaborationImmersive LearningVR Headsets
Featured

Share This Post