Why The Army's New Palantir Contract Won’t Fix Battlefield Intelligence

The Long March
A palantir from 'Lord Of The Rings.' GET IT?!
YouTube/New Line Cinema

The U.S. Army recently announced that Palantir won the contract to build the new battlefield intelligence platform.


Palantir has a great reputation for use on the battlefield, especially for counter-IED functions, and has attained an almost legendary status among some analysts and communities in the Army. When compared to the Distributed Common Ground System – Army (DCGS-A), its success is not surprising; most users of DCGS-A would agree that it is problematic.

In particular, Palantir has a much friendlier user interface than DCGS-A, and its Gotham system is excellent at linking reports or other pieces of intelligence together. But Palantir’s Gotham system, the model for a new battlefield intelligence system, is susceptible to quickly becoming the next DCGS-A. Without some important changes, Palantir’s software will not satisfy battlefield intelligence needs and be doomed to repeat the failures of its predecessor.

There are two critical areas that Palantir must address to develop a superior battlefield intelligence platform: inter-operability and customization of analysis by an end-user.

First, Palantir must be interoperable with different analysis suites. Real battlefield intelligence problems require a variety of approaches, both qualitative and quantitative, and will require different tools for appropriate analysis. When performing intelligence analysis—such as an All-Source Analyst will do in a battlefield environment—various tools need to be integrated. Analysts use a plethora of tools such as Analyst Notebook for creating link diagrams, ORA  for network analysis, ArcGIS  for creating maps, Anaconda for data science, and Palantir Gotham for finding linked reports. Many of these tools have specialized algorithms and analysis formats that cannot be replicated easily in one comprehensive format; ORA’s socio-cognitive maps cannot easily be replicated, and few systems can create good link diagram visualizations like Analyst Notebook. But one analysis suite will simply not provide the functionality to perform all of the analyses needed for battlefield intelligence.

Palantir must be able to integrate data with different systems. This means that it must both select and export data into easily ingestible formats like JSON and csv, and load in and parse the native formats from different analysis suites. A closed system that is not easily interoperable with other systems will not provide the functionality needed by battlefield analysts to attack real intelligence problems. For Palantir to be more successful than DCGS-A, it must have a high degree of interoperability with many analysis suites.

Secondly, and perhaps more importantly, if Palantir is to succeed where DCGS-A did not, it must allow for analysts to write their own code with its data. Humanity, and with it, warfare, is becoming increasingly digitized. As a result, information is exploding and the days where one analyst or group of analysts could read through all of the information on a particular area of interest and become an expert in a reasonable amount of time are waning. As intelligence analysis increasingly incorporates machine learning, Big Data, and interactive visualizations, intelligence will demand systems that can incorporate these advances in technology and methodology (see here, here, or here for just a few examples).

Many of the best tools for machine learning and advanced visualization are open source and written in coding languages like Python, R, and Julia. So, an analyst equipped with the right data will need to not only pull in these tools easily to support their analyses but also customize their tools to the particular intelligence data that is relevant to their battlefield environment and do all of this at massive scale and with varied types of information.

Some of the best uses of Big Data and machine learning in industry result from applying numerous methods to data, which can only be done in an open framework where an analyst is not restricted to provided or existing tools and methods. For the Palantir system to be successful for battlefield intelligence, it must, at a minimum, have an application programming interface for analysts to programmatically and easily pull and submit data with the Palantir system.

Ideally, Palantir should feature a full interface that has all of the important programming languages and their associated packages already in it, such that an analyst can build code and query data in the same environment (see Kaggle for an example of this type of environment). Ultimately, as the sheer quantity of information available on any given battlefield continues to increase (and it will do so for the foreseeable future), any successful battlefield intelligence system must be able to fully leverage this information using Big Data and machine learning. To do so, Palantir has to support an analyst coding and programmatically querying data.

The decision to award Palantir the contract for creating the Army’s new battlefield intelligence platform is a step in the right direction. Palantir’s Gotham program is generally better than DCGS-A. However, Palantir is in danger of making the same fundamental mistake that DCGS-A made: trying to create an intelligence analysis system to rule them all. Fundamentally, battlefield intelligence is both a science and an art. So, any successful battlefield system must fundamentally be interoperable and customizable to empower creativity. The change in battlefield intelligence systems brought by Palantir presents the military intelligence community a great opportunity to modernize and ensure dominance in battlefield intelligence both now and in the future. We can’t afford to waste the opportunity by allowing Palantir to make the same mistakes as DCGS-A.

Captain Iain J. Cruickshank is a USMA Graduate, class of 2010, and is currently a PhD Candidate in Societal Computing at Carnegie Mellon University as National Science Foundation Graduate Research Fellow. His previous assignments include Company Commander for D Company, 781st Military Intelligence Battalion (Cyber), and Sub-Element lead for planning and analysis and production on a National Mission Team in the Cyber National Mission Force. The views expressed in this article do not officially represent the views of the U.S. Army, the U.S. military or the United States Government, and are the views of the author only.

Task & Purpose

I don't always drop everything to spend a few hours with a short, squat Marine, but when I do, you can bet it's for Chesty.

Read More Show Less
A U.S. Army Soldier assigned to the 3rd Battalion, 21st Infantry Regiment, Fort Wainwright, Alaska, consoles a fellow Soldier after sleeping on the ground in a designated sleeping area on another cold evening, between training exercises during NTC 17-03, National Training Center, Ft. Irwin, CA., Jan. 15, 2017. (U.S. Army/Spc. Tracy McKithern)

The Defense Visual Information Distribution Service (DVIDS) is the largest official database of U.S. military media available for public consumption. It is also an occasional source of unexpected laughs, like this gem from a live fire exercise that a public affairs officer simply tagged 'Fire mortar boom.' In the world of droll data entry and too many acronyms, sometimes little jokes are their own little form of rebellion, right?

But some DVIDS uploads, however, come with captions and titles that cut right to the core, perfectly capturing the essence of life in the U.S. military in a way that makes you sigh, facepalm, and utter a mournful, 'too real.'

Read More Show Less

The U.S. military does not need Iraqi permission to fly close air support and casualty evacuation missions for U.S. troops in combat, a top spokesman for the U.S.-led coalition fighting ISIS clarified on Tuesday.

Army Col. James Rawlinson clarified that the Iraqis do not need to approve missions in emergency circumstances after Task & Purpose reported on Monday that the U.S. military needed permission to fly CAS missions for troops in a fight.

Read More Show Less
Photo illustration by Paul Szoldra/Task & Purpose

It all began with a medical check.

Carson Thomas, a healthy and fit 20-year-old infantryman who had joined the Army after a brief stint in college, figured he should tell the medics about the pain in his groin he had been feeling. It was Feb. 12, 2012, and the senior medic looked him over and decided to send him to sick call at the base hospital.

It seemed almost routine, something the Army doctors would be able to diagnose and fix so he could get back to being a grunt.

Now looking back on what happened some seven years later, it was anything but routine.

Read More Show Less
The Armed Forces Service Medal has a green, blue and yellow ribbon and a bronze medal featuring a torch like that held by the Statue of Liberty. (U.S. Air Force/Staff Sgt. Alexx Pons)

Editor's Note: This article by Gina Harkins originally appeared onMilitary.com, a leading source of news for the military and veteran community.

Thousands of U.S. service members who've been sent to operate along the Mexico border will receive a military award reserved for troops who "encounter no foreign armed opposition or imminent hostile action."

The Pentagon has authorized troops who have deployed to the border to assist U.S. Customs and Border Protection (CBP) since last April to receive the Armed Forces Service Medal. Details about the decision were included in a Marine Corps administrative message in response to authorization from the Defense Department.

There is no end date for the award since the operation remains ongoing.

Read More Show Less