The National Geospatial-Intelligence Agency is injecting machine learning and computer vision across its operations, from the battlefield to the highest levels of geopolitical analysis. The agency is harnessing rapidly evolving artificial intelligence technologies to help military leaders see what’s happening on the battlefield in more detail and give policymakers a better understanding of global threats and dynamics.
“We’re very excited about the trajectory of AI applications,” NGA’s director, Vice Adm. Frank Whitworth, said in an interview with SpaceNews.
Whitworth said a convergence of rapidly advancing AI capabilities and revised business processes is reshaping how the agency provides intelligence to inform national security decision-making at the highest levels.
The Springfield, Virginia-based agency is building on the work of Project Maven, a Pentagon initiative launched in 2017 to harness AI for analyzing drone footage and satellite imagery.
Growing pains
At its outset, the program wrestled with the limitations of early computer vision technology, which required laboriously annotating drone videos frame-by-frame to teach algorithms how to track vehicles across deserts.
Project Maven was also marred by political and media controversy in 2018 when Google employees raised ethical objections to the company supplying the Pentagon with Google’s open-source machine learning library for computer vision applications, which were being used to improve drone strike targeting.
However, those initial growing pains proved a catalyst, as Maven paved the way for the intelligence community’s embrace of AI over the following years. And NGA is now moving to operationalize machine learning and computer vision across its entire data analysis workflow.
Whitworth explained that with AI, analysts can unearth hidden threats and areas of interest buried within the vast amounts of imagery and data agencies collect daily. He added that a new set of tools is enabling NGA to deliver timelier and more accurate intelligence.
Deluge of data
The government’s embrace of machine learning to extract insights from massive data streams holds significant implications for the commercial remote sensing industry. Companies have poured billions of dollars into next-generation Earth observation satellite constellations producing torrents of imagery and geospatial data.
Whitworth said NGA is eager to take full advantage of the capabilities the private sector is fielding. The challenge is to be able to turn these fire hoses of data into usable intelligence.
“The sheer volume of geospatial data is staggering,” Whitworth said, comparing it to trying to send high-resolution photos from a mobile phone without a signal. “That challenge, multiplied a billionfold, is what we face on a national scale.”
Beyond the tactical applications of AI, he said, the agency wants to leverage the technology more broadly, to monitor global events and provide policymakers with early warnings of potential flashpoints.
A new initiative at NGA is to seamlessly integrate AI into the daily workflow of analysts. These AI-powered systems would, for example, alert analysts to a potential threat emerging halfway across the globe, Whitworth said. “This is a much bigger opportunity that we’re really excited about.”
The race to operationalize AI is seen as paramount in the so-called Great Power Competition between the United States and China. As a DoD intelligence official bluntly stated, “The Chinese see this as a race to be the first to weaponize AI. Winning a war hinges on the quality and accuracy of data.”
Both the Pentagon and China’s People’s Liberation Army view AI and machine learning as crucial for rapid data processing and accelerated decision-making, enabling them to “prosecute large numbers of targets swiftly and at scale,” the DoD official said at a recent briefing at the Pentagon. “This is their vision of the future of warfare.”
AI on the battlefield
AI tools developed under Project Maven are now actively deployed by frontline combat units like the U.S. Army’s 18th Airborne Corps, said Trey Treadwell, NGA’s associate director for capabilities.
Leveraging computer vision, trained on meticulously annotated datasets, analysts can rapidly scan satellite photos and drone video feeds to pinpoint specific targets like vehicles, aircraft and military equipment, giving commanders and intelligence analysts a high-fidelity operational picture, Treadwell said last month at the 39th Space Symposium in Colorado Springs, Colorado.
To be sure, he added, “We are not just going to let the AI take over or the machines take over.” He highlighted how the agency sees AI as a force multiplier to be applied judiciously, such as helping avoid using a multi-million dollar missile to take down a cheap drone.
“With Maven, we’re pushing boundaries and employing technologies on the cutting edge,” he said. “The real driving force is the ingenuity of our field operators,” who constantly challenge analysts to extract new layers of information from the data.
Beyond computer vision
Mark Munsell, NGA’s director of data and digital innovation, said the agency will be “using a lot more AI technology, beyond computer vision.” Computer vision is a subset of AI technology that enables computers to interpret and understand visual information from digital images and videos.
NGA plans to tap the newer foundation models developed by AI leaders OpenAI, Anthropic, Google and Microsoft, and “fine tune and train those models with NGA data and reporting,” Munsell told SpaceNews.
“I think you’ll see a new era of doing analytics with these big computer vision models or these big large language models, and in this new era, there needs to be a lot more human-machine teaming,” he said.
This concept is embodied in ASPEN (Analysis Services Production Environment for the National System for Geospatial Intelligence), a project NGA launched last year to integrate AI tools into analysts’ workflows. The goal is to empower analysts to identify adversaries’ activities that might warrant notification of the White House or the Pentagon.
“AI models are rapidly becoming multimodal,” explained Munsell, referring to systems that can process and understand data from various sources, not just images or videos. This includes text descriptions and audio, allowing for a more comprehensive understanding of a scene.
Munsell, who was NGA’s chief technology officer during Project Maven’s inception, highlighted how the program’s realignment under NGA has “sharpened our focus on combat support activities once more. It has refined our vision of how we should be supporting combatant commands in operations.”
Sean Batir, Project Maven’s chief technology officer, said intelligence analysts are challenged to balance commanders’ thirst for information while streamlining workflows. “One way we address this is by trying to compress the information,” he said.
“I’ve been working on integration of some of the latest open-source large language models, and we’re actually discovering a very positive result,” he said. “I think that the future is multimodal models that can take text and imagery, and we’ve already begun experimentation with that in our workflows.”
Training data
The work done under Project Maven is transitioning to the broader ASPEN initiative, Munsell said. “We have analysts that look at imagery, annotate imagery all day long, and we’re putting that training data creation into the analytic workflow.”
ASPEN program manager Rob Shields explained that while Project Maven is primarily looking at how to support faster and more accurate targeting on the battlefield, ASPEN is about providing insights to “stay ahead of our adversaries.”
“Our goal is really to integrate at scale the analytic tools and processes to include computer vision, big data analytics, geographic information, geospatial intelligence reporting and imagery exploitation services,” he said.
Project Maven for years has relied on contractors to do “image labeling,” or the process of adding descriptive tags or annotations to images to help machines interpret the visual content and understand what’s happening in the image.
One of the objectives of the ASPEN program is for government analysts to do image labeling as part of their normal workflow, said Shields. “And as images are labeled, shared and used across the program, it’s all about having those labels improve the accuracy of our models.”
“We have to increase speed without compromising accuracy,” he said.
Predictive Intelligence
NGA’s vision is to “anticipate adversary action and potential outcomes through pattern of life analysis, patterns of behavior and trends through data analytics, identifying anomalous and escalating indicators,” he said. “ASPEN covers all the workflow from detection through reporting.”
While NGA has been using AI in various capacities for over two decades, Munsell notes that the sheer pace of technological progress in just the past few years is allowing the agency to take these applications to an entirely new level. “Transformers took off in the 2017 and 2018 era when Project Maven started,” he explained. “The availability of high-performance compute, cloud storage — all these innovations converging is what has really unlocked the AI’s potential.”
Transformers are a type of neural network inspired by the structure and function of the human brain, essentially a way for computers to learn and process information the way humans do, albeit in a much simpler way.
The advent of foundation models like Generative Pre-trained Transformers (GPT) represents a generational shift, he added.
These models are associated with popular chatbots like OpenAI’s ChatGPT. “But these things are a lot more powerful than that,” he said. “They are agents of work, and becoming more and more capable of doing human work, tapping into applications and data sources, writing their own code, able to get results that normally a human would have to put together.”
Given the rapid progress of technology and confluence of innovation trends, “I think we’re looking at something very special for the intelligence community and for the Department of Defense over the next several years,” Munsell said. “There’s going to be some big breakthroughs in the way we produce intelligence.”
This article first appeared in the May 2024 issue of SpaceNews Magazine.