Why The Army's New Palantir Contract Won’t Fix Battlefield Intelligence

The Long March
YouTube/New Line Cinema

The U.S. Army recently announced that Palantir won the contract to build the new battlefield intelligence platform.


Palantir has a great reputation for use on the battlefield, especially for counter-IED functions, and has attained an almost legendary status among some analysts and communities in the Army. When compared to the Distributed Common Ground System – Army (DCGS-A), its success is not surprising; most users of DCGS-A would agree that it is problematic.

In particular, Palantir has a much friendlier user interface than DCGS-A, and its Gotham system is excellent at linking reports or other pieces of intelligence together. But Palantir’s Gotham system, the model for a new battlefield intelligence system, is susceptible to quickly becoming the next DCGS-A. Without some important changes, Palantir’s software will not satisfy battlefield intelligence needs and be doomed to repeat the failures of its predecessor.

There are two critical areas that Palantir must address to develop a superior battlefield intelligence platform: inter-operability and customization of analysis by an end-user.

First, Palantir must be interoperable with different analysis suites. Real battlefield intelligence problems require a variety of approaches, both qualitative and quantitative, and will require different tools for appropriate analysis. When performing intelligence analysis—such as an All-Source Analyst will do in a battlefield environment—various tools need to be integrated. Analysts use a plethora of tools such as Analyst Notebook for creating link diagrams, ORA  for network analysis, ArcGIS  for creating maps, Anaconda for data science, and Palantir Gotham for finding linked reports. Many of these tools have specialized algorithms and analysis formats that cannot be replicated easily in one comprehensive format; ORA’s socio-cognitive maps cannot easily be replicated, and few systems can create good link diagram visualizations like Analyst Notebook. But one analysis suite will simply not provide the functionality to perform all of the analyses needed for battlefield intelligence.

Palantir must be able to integrate data with different systems. This means that it must both select and export data into easily ingestible formats like JSON and csv, and load in and parse the native formats from different analysis suites. A closed system that is not easily interoperable with other systems will not provide the functionality needed by battlefield analysts to attack real intelligence problems. For Palantir to be more successful than DCGS-A, it must have a high degree of interoperability with many analysis suites.

Secondly, and perhaps more importantly, if Palantir is to succeed where DCGS-A did not, it must allow for analysts to write their own code with its data. Humanity, and with it, warfare, is becoming increasingly digitized. As a result, information is exploding and the days where one analyst or group of analysts could read through all of the information on a particular area of interest and become an expert in a reasonable amount of time are waning. As intelligence analysis increasingly incorporates machine learning, Big Data, and interactive visualizations, intelligence will demand systems that can incorporate these advances in technology and methodology (see here, here, or here for just a few examples).

Many of the best tools for machine learning and advanced visualization are open source and written in coding languages like Python, R, and Julia. So, an analyst equipped with the right data will need to not only pull in these tools easily to support their analyses but also customize their tools to the particular intelligence data that is relevant to their battlefield environment and do all of this at massive scale and with varied types of information.

Some of the best uses of Big Data and machine learning in industry result from applying numerous methods to data, which can only be done in an open framework where an analyst is not restricted to provided or existing tools and methods. For the Palantir system to be successful for battlefield intelligence, it must, at a minimum, have an application programming interface for analysts to programmatically and easily pull and submit data with the Palantir system.

Ideally, Palantir should feature a full interface that has all of the important programming languages and their associated packages already in it, such that an analyst can build code and query data in the same environment (see Kaggle for an example of this type of environment). Ultimately, as the sheer quantity of information available on any given battlefield continues to increase (and it will do so for the foreseeable future), any successful battlefield intelligence system must be able to fully leverage this information using Big Data and machine learning. To do so, Palantir has to support an analyst coding and programmatically querying data.

The decision to award Palantir the contract for creating the Army’s new battlefield intelligence platform is a step in the right direction. Palantir’s Gotham program is generally better than DCGS-A. However, Palantir is in danger of making the same fundamental mistake that DCGS-A made: trying to create an intelligence analysis system to rule them all. Fundamentally, battlefield intelligence is both a science and an art. So, any successful battlefield system must fundamentally be interoperable and customizable to empower creativity. The change in battlefield intelligence systems brought by Palantir presents the military intelligence community a great opportunity to modernize and ensure dominance in battlefield intelligence both now and in the future. We can’t afford to waste the opportunity by allowing Palantir to make the same mistakes as DCGS-A.

Captain Iain J. Cruickshank is a USMA Graduate, class of 2010, and is currently a PhD Candidate in Societal Computing at Carnegie Mellon University as National Science Foundation Graduate Research Fellow. His previous assignments include Company Commander for D Company, 781st Military Intelligence Battalion (Cyber), and Sub-Element lead for planning and analysis and production on a National Mission Team in the Cyber National Mission Force. The views expressed in this article do not officially represent the views of the U.S. Army, the U.S. military or the United States Government, and are the views of the author only.

The U.S. Department of Veterans Affairs paid $13,000 over a three-month period for a senior official's biweekly commute to Washington from his home in California, according to expense reports obtained by ProPublica.

Read More Show Less
Saturday Night Live/screenshot

President Donald Trump said that "retribution" should be "looked into" after this week's opening skit of Saturday Night Live featured Alec Baldwin being mean to him again.

Read More Show Less
Staff Sgt. John Eller conducts pre-flights check on his C-17 Globemaster III Jan. 3 prior to taking off from Hickam Air Force Base, Hawaii for a local area training mission. Sgt. Eller is a loadmaster from the 535th Airlift Squadron. (U.S. Air Force photo/Tech. Sgt. Shane A. Cuomo)

CUCUTA, Colombia — The Trump administration ratcheted up pressure Saturday on beleaguered Venezuelan President Nicolas Maduro, dispatching U.S. military planes filled with humanitarian aid to this city on the Venezuelan border.

Read More Show Less
U.S. Secretary of Defense Patrick Shanahan speaks at the annual Munich Security Conference in Munich, Germany February 15, 2019. REUTERS/Andreas Gebert

ABOARD A U.S. MILITARY AIRCRAFT (Reuters) - Acting U.S. Defense Secretary Patrick Shanahan said on Saturday he had not yet determined whether a border wall with Mexico was a military necessity or how much Pentagon money would be used.

President Donald Trump on Friday declared a national emergency in a bid to fund his promised wall at the U.S.-Mexico border without congressional approval.

Read More Show Less
A pair of U.S. Navy Grumman F-14A Tomcat aircraft from Fighter Squadron VF-211 Fighting Checkmates in flight over Iraq in 2003/Department of Defense

Since the sequel to the 1986 action flick (and wildly successful Navy recruitment tool) Top Gun, was announced, there's been a lot of speculation on what Top Gun: Maverick will be about when it premieres in June 2020. While the plot is still relatively unclear, we know Tom Cruise will reprise his role as Naval aviator Pete "Maverick" Mitchell, and he'll be joined by a recognizable costar: The iconic F-14 Tomcat.

It looks like the old war plane will be coming out of retirement for more than just a cameo. A number of recently surfaced photos show an F-14 Tomcat aboard the USS Theodore Roosevelt, alongside Cruise and members of the film's production crew, the Drive's Tyler Rogoway first reported earlier this week.

Read More Show Less