background
All Cases

Terran CQB Trainer

CEO & Founder Brantley Pace
UI/UX Jennifer McMillen
Project Info

Client

Department of Defense (B2G)

Service

Advanced Nano-Electronic Solutions & Mixed Reality Experiences

Sector

Defense

Compilation

C++, C#, Bluetooth, Infrared, SLAM
Products Vision

Firearms Training Simulator

In the demanding world of military training, the integration of cutting-edge technology has traditionally lagged, often relying on antiquated systems such as the FATS (Firearms Training Simulator), which have become progressively obsolete. The advent of Augmented Reality (AR) technology promised a paradigm shift, yet, until recently, its potential remained underexploited due to developmental limitations.

A forefront software development enterprise was commissioned to pioneer a groundbreaking project named "Terran." The objective was to develop an AR-based simulated weapons and tactics training platform that leveraged the Microsoft HoloLens 1, augmented with infrared tracking capabilities. Terran was conceptualized to operate similarly to the PEQ-16, a laser targeting module capable of being mounted on any Picatinny rail for versatile field or indoor application.

The project faced formidable challenges from its inception. Paramount among these was the necessity for flawless tracking; the simulation needed to achieve unparalleled realism, ensuring that users' interactions with the virtual environment felt utterly authentic. Moreover, the AR interface had to facilitate intuitive lateral communication, mimicking the complexities of real-life military operations.

The solution, Terran, emerged as a state-of-the-art live simulation environment. It encompasses a comprehensive suite of scenarios designed to enhance proficiency in all facets of weapons handling, from basic operation to immediate and remedial action. This case study illustrates not only the technical prowess of the development team but also its contribution to the evolution of military training methodologies, where traditional barriers are transcended through innovative technological solutions.

Known Challenges

At the outset of the Terran project, the team faced the critical challenge of integrating Augmented Reality (AR) into a military training solution that was both universally compatible with any Picatinny rail system and capable of overcoming the inherent limitations of AR technology. The choice of infrared for tracking accuracy brought with it the additional challenge of environmental limitations, notably the interference caused by sunlight in outdoor settings. Moreover, the constraints imposed by AR technology—particularly in terms of users' perception and peripheral vision—demanded a solution that provided a fully immersive, 360-degree training experience.

To address these challenges, the team embarked on a focused development strategy, prioritizing the core components essential for a viable prototype. The selection of Unity 3D as the development platform enabled rapid prototyping, allowing the team to iterate quickly and efficiently. This choice was complemented by the utilization of Visual Studio for Nano Electronics, specifically for Arduino programming, and the integration of thermal cameras to enhance tracking fidelity.

This strategic approach allowed the team to navigate the project's technical complexities effectively. By leveraging these technologies, the team was able to swiftly address and resolve edge cases, thus facilitating the seamless transition from prototype to production within the stringent demands and tight timelines dictated by the project's scope. This case study exemplifies our methodical problem-solving prowess and its ability to deliver innovative solutions under challenging constraints.

  • icon Augmented Reality Trainer.
  • icon Nano Electronics Integration.
  • icon Universal Picatinny Rail Compatibility.
  • icon Infrared Tracking.
  • icon Unity 3D for Rapid Prototyping.
  • icon Thermal Camera Utilization.

The Magic behing the Technology

Incorporating thermal imaging and infrared tracking technologies was critical in mastering Terran's simulation accuracy protocols, particularly under diverse environmental lighting conditions. Achieving seamless integration of these technologies with the ARCore framework allowed for a stable approach. Pro Stack's engineers developed middleware solutions and harnessed existing libraries to facilitate direct communication between the sensory input from thermal and infrared sources and the virtual scenarios attached to the PEQ-16 replacement. This ensured a high degree of accuracy in tracking user movements and interactions within the AR Simulator. Furthermore, ensuring Terran's compatibility with any Picatinny rail system introduced a layer of complexity that demanded a modular and flexible design approach. The use of 3D printing made prototyping quick and painless.

icon

SLAM Technology

icon

Target impact recognition

icon

Visual and Sensation Feedback

icon

Proper Recoil Control

Project
Project
Project
Project

Infrared and Thermal Tracking

Working with infrared (IR) technology for tracking in the Terran product necessitated a sophisticated approach, blending both the physical hardware intricacies with the nuances of software integration. The cornerstone of our solution was the Adafruit MLX90640 IR Thermal Camera, a compact module equipped with a dense array of 768 IR sensors, forming images through individual temperature readings over I2C communication. This sensor offered the detailed thermal imaging capabilities we needed, operating within a temperature range of -40 to 300°C with an accuracy of ±2°C and a refresh rate conducive to real-time interaction​​​​.

The process began with harnessing the Adafruit MLX90640's potential by integrating it into our system through Arduino, a choice influenced by Arduino's versatility and the availability of robust pre-defined libraries for rapid development. The MLX90640's compatibility with Arduino's I2C pins allowed for straightforward connectivity, simplifying the data transmission between the thermal camera and our processing unit. Our engineers focused on optimizing the thermal camera's settings, such as the resolution and refresh rate, to match the Terran system's performance requirements, ensuring that thermal data was captured and processed efficiently without overburdening the system's computational resources​​​​.

The integration also involved developing custom software routines to interpret the thermal images into actionable data within the AR environment, a task that required deep dives into the Adafruit and Arduino libraries. By leveraging these libraries, we could focus on refining the Terran's simulation fidelity rather than the underlying mechanics of thermal imaging. This approach not only accelerated the development process but also ensured that Terran benefited from the latest advancements in IR tracking technology, providing a highly responsive and immersive training experience that could adapt to a wide range of environmental conditions

icon

Advanced IR Integration

icon

Real-Time Interaction

icon

Seamless AR Integration for any Medium

Project

CQB Trainer Summary

The Terran project represents a pioneering step in military training simulations, utilizing advanced augmented reality (AR) technology to provide immersive and realistic training environments. At its core, Terran integrates the Adafruit MLX90640 IR Thermal Camera with Arduino for precise infrared tracking, addressing the challenges of environmental variables and ensuring high-fidelity simulations. This integration, combined with a seamless AR interface and compatibility with any Picatinny rail system, enables real-time interactions and efficient data processing. The system’s modular design facilitates rapid prototyping and adaptation to various military hardware, setting a new standard for realism in tactical training simulations

icon

Leadership

With over 70 years combined experience Pro Stack can lead your engineering teams providing Fractional CTO services that help your team perform at its best.

icon

Development

As a team of seasoned software experts we provide top of the line cutting edge software solutions whether it's building from the ground up or developing your existing products.

icon

Consulting

Have a team or product you'd like to improve? Pro Stack offers consulting services to businesses that need some help from top industry experts.