Why The Army's New Palantir Contract Won’t Fix Battlefield Intelligence

The Long March
YouTube/New Line Cinema

The U.S. Army recently announced that Palantir won the contract to build the new battlefield intelligence platform.

Palantir has a great reputation for use on the battlefield, especially for counter-IED functions, and has attained an almost legendary status among some analysts and communities in the Army. When compared to the Distributed Common Ground System – Army (DCGS-A), its success is not surprising; most users of DCGS-A would agree that it is problematic.

In particular, Palantir has a much friendlier user interface than DCGS-A, and its Gotham system is excellent at linking reports or other pieces of intelligence together. But Palantir’s Gotham system, the model for a new battlefield intelligence system, is susceptible to quickly becoming the next DCGS-A. Without some important changes, Palantir’s software will not satisfy battlefield intelligence needs and be doomed to repeat the failures of its predecessor.

There are two critical areas that Palantir must address to develop a superior battlefield intelligence platform: inter-operability and customization of analysis by an end-user.

First, Palantir must be interoperable with different analysis suites. Real battlefield intelligence problems require a variety of approaches, both qualitative and quantitative, and will require different tools for appropriate analysis. When performing intelligence analysis—such as an All-Source Analyst will do in a battlefield environment—various tools need to be integrated. Analysts use a plethora of tools such as Analyst Notebook for creating link diagrams, ORA  for network analysis, ArcGIS  for creating maps, Anaconda for data science, and Palantir Gotham for finding linked reports. Many of these tools have specialized algorithms and analysis formats that cannot be replicated easily in one comprehensive format; ORA’s socio-cognitive maps cannot easily be replicated, and few systems can create good link diagram visualizations like Analyst Notebook. But one analysis suite will simply not provide the functionality to perform all of the analyses needed for battlefield intelligence.

Palantir must be able to integrate data with different systems. This means that it must both select and export data into easily ingestible formats like JSON and csv, and load in and parse the native formats from different analysis suites. A closed system that is not easily interoperable with other systems will not provide the functionality needed by battlefield analysts to attack real intelligence problems. For Palantir to be more successful than DCGS-A, it must have a high degree of interoperability with many analysis suites.

Secondly, and perhaps more importantly, if Palantir is to succeed where DCGS-A did not, it must allow for analysts to write their own code with its data. Humanity, and with it, warfare, is becoming increasingly digitized. As a result, information is exploding and the days where one analyst or group of analysts could read through all of the information on a particular area of interest and become an expert in a reasonable amount of time are waning. As intelligence analysis increasingly incorporates machine learning, Big Data, and interactive visualizations, intelligence will demand systems that can incorporate these advances in technology and methodology (see here, here, or here for just a few examples).

Many of the best tools for machine learning and advanced visualization are open source and written in coding languages like Python, R, and Julia. So, an analyst equipped with the right data will need to not only pull in these tools easily to support their analyses but also customize their tools to the particular intelligence data that is relevant to their battlefield environment and do all of this at massive scale and with varied types of information.

Some of the best uses of Big Data and machine learning in industry result from applying numerous methods to data, which can only be done in an open framework where an analyst is not restricted to provided or existing tools and methods. For the Palantir system to be successful for battlefield intelligence, it must, at a minimum, have an application programming interface for analysts to programmatically and easily pull and submit data with the Palantir system.

Ideally, Palantir should feature a full interface that has all of the important programming languages and their associated packages already in it, such that an analyst can build code and query data in the same environment (see Kaggle for an example of this type of environment). Ultimately, as the sheer quantity of information available on any given battlefield continues to increase (and it will do so for the foreseeable future), any successful battlefield intelligence system must be able to fully leverage this information using Big Data and machine learning. To do so, Palantir has to support an analyst coding and programmatically querying data.

The decision to award Palantir the contract for creating the Army’s new battlefield intelligence platform is a step in the right direction. Palantir’s Gotham program is generally better than DCGS-A. However, Palantir is in danger of making the same fundamental mistake that DCGS-A made: trying to create an intelligence analysis system to rule them all. Fundamentally, battlefield intelligence is both a science and an art. So, any successful battlefield system must fundamentally be interoperable and customizable to empower creativity. The change in battlefield intelligence systems brought by Palantir presents the military intelligence community a great opportunity to modernize and ensure dominance in battlefield intelligence both now and in the future. We can’t afford to waste the opportunity by allowing Palantir to make the same mistakes as DCGS-A.

Captain Iain J. Cruickshank is a USMA Graduate, class of 2010, and is currently a PhD Candidate in Societal Computing at Carnegie Mellon University as National Science Foundation Graduate Research Fellow. His previous assignments include Company Commander for D Company, 781st Military Intelligence Battalion (Cyber), and Sub-Element lead for planning and analysis and production on a National Mission Team in the Cyber National Mission Force. The views expressed in this article do not officially represent the views of the U.S. Army, the U.S. military or the United States Government, and are the views of the author only.

The U.S. Embassy in Iraq (Reuters/Lucas Jackson)

BAGHDAD (Reuters) - A rocket was fired in Iraqi capital Baghdad's heavily-fortified Green Zone, which houses government buildings and foreign embassies, but caused no casualties, the Iraqi military said on Sunday.

A blast was heard in central Baghdad on Sunday night, Reuters witnesses said and two Baghdad-based diplomatic sources also said they heard the blast.

Read More Show Less

Officers from the California Highway Patrol arrested a homeless man Thursday morning after he allegedly threw a stolen Caltrans tripod onto Interstate 5 in downtown Sacramento, endangering the occupants of a van as it crashed through its windshield.

The incident happened just after 10:30 a.m., when the Caltrans survey tripod was stolen from the corner of Neasham Circle and Front Street, CHP South Sacramento said in a news release.

Read More Show Less
(U.S. Army/ Sgt. Mike MacLeod)

KABUL (Reuters) - Afghanistan's parliament descended into chaos on Sunday when lawmakers brawled over the appointment of a new speaker, an inauspicious start to the assembly which was sitting for the first time since chaotic elections last year.

Results of last October's parliamentary election were only finalized earlier this month after repeated technical and organizational problems and widespread accusations of fraud.

Read More Show Less
FILE PHOTO: The Carl Vinson VA Medical Center iin Dublin, Georgia

RIVIERA BEACH — When a distraught patient opened fire at the VA Medical Center in February, Albert Gaines' long ago military training kicked into gear.

"When I saw the arm come up, I knew what was next, pow, pow, pow," said Gaines, who was doing his job, cleaning patient rooms, when gunfire erupted. "I hit the deck to minimize the target."

Now, three months after what his bosses at the hospital call "the active shooter incident," the 65-year-old Riviera Beach man still feels like a target is on him.

Read More Show Less

President Donald Trump could issue a pardon on Memorial Day for Navy SEAL Chief Eddie Gallagher, former Special Forces Maj. Matthew Golsteyn, and Marine Scout Snipers accused of urinating on Taliban corpses, the New York Times is reporting.

The White House is working with the Justice Department and military services to get the paperwork necessary for the pardons in order, according to the Times.

Read More Show Less