In 2013, the Department of Defense began an approximately six-year review of 159 mental health programs, many of which were launched after the U.S. invaded Iraq and Afghanistan. According to preliminary DoD records obtained in a Freedom of Information Act lawsuit, the DoD found “a large proportion” of these programs did not track spending and were “unable to document evidence of program outcomes.”

The DoD declined to comment, but the record of one controversial program included in the review, Comprehensive Soldier Fitness, provides a rare glimpse into a wider problem. In a well-funded Post-9/11 panic, mental health programs were rushed out, and their full impact on the lives of troops and their families remains unknown.

On Nov. 25, 2008, at the Pentagon, General George Casey Jr., Army chief of staff, back from commanding a brutal Iraq war, sat in a meeting about manufacturing soldiers' “mental armor,” according to three internal memos. The idea was to provide cognitive training to increase resilience to stress and trauma. The problem was–and still is–no one knew how to do it.

Mental Health photo

Joint Base Balad, Iraq(U.S. Army/Sgt. Pat Caldwell)

“The science wasn't known,” said former Surgeon General Richard Carmona, who also attended the meeting, in a phone interview. “We didn't have time to answer all the questions.” The U.S. military was fighting two wars in Iraq and Afghanistan and suffering a mental health crisis made worse by deficient care back home, all of which made headlines. A sense of urgency to respond outweighed rigorous testing that takes time.

For years, the Army had come up against an indisputable fact: The long psychological damage of war was outpacing the science to treat it.

“We were sending 150,000 troops to combat every year,” said Casey in a phone interview. “We were not going to study it for eight years.”

Retired Army Brigadier General Dr. Rhonda Cornum, CSF's first director, also attended the Pentagon meeting. “The statistics were all going the wrong way,” she said, referring to the high rates of mental health conditions, such as PTSD and depression. “And there was really no evidence that anything was going to change them.”

In the meeting, some experts advocated for psychological literacy training, while others pushed for practical changes, including an open discussion about the psychological consequences of killing. Casey, however, wanted a more “comprehensive whole-life fitness strategy” for an “era of persistent conflict,” according to a memo from his archive at the National Defense University.

Mental Health photo

3rd Cavalry Regiment soldiers, Afghanistan. (U.S. Army/Maj. Vance Trenkel)

The expert who provided the program template was Dr. Martin Seligman, another meeting attendee, who believed that resilience to war and the stress of military life can be learned. He declined to be interviewed.

“Seligman sold his program and he sold it well,” said Dr. Larry Dewey, former chief of psychiatry at Boise VA Medical Center, who was in the meeting. No one else in the room offered up a program besides Seligman.

At the University of Pennsylvania, Seligman's resilience program had shown that trained school teachers can instruct adolescents to be more resilient, reducing depression symptoms. But that program was tested on high-school students, not soldiers at war, one of the many flaws that CSF's critics noted.

Casey adopted Seligman's template anyway. A $31,350,000 no-bid contract was awarded to the Penn Resilience Program that would be implemented Army-wide. In about a year, the Army's “mental armor” was rolled out for roughly 1.1 million soldiers.

After CSF launched in 2009, the program was heavily criticized for unproven claims of reducing suicide and PTSD. “When I first heard about it I was more or less floored,” said Dr. George Bonanno, a clinical psychologist at Columbia University. “I've been studying resilience for 20 years, and I don't know of any empirical data that shows how to build resilience in anybody.”

CSF was one of the few programs to receive critical press and scholarly attention, which partly came from measuring outcomes that were released in public reports.

Eventually, it was renamed Comprehensive Soldier and Family Fitness (CSF2), and in promotional material it seems to be more about helping soldiers with the stress of daily life, such as managing personal relationships, than it is about combat. But behind CSF's long controversy were hundreds of other programs, produced in the same urgent context, that received little outside scrutiny other than the occasional unnoticed report.

After 9/11, military mental health programs proliferated. In 2010, there were 211, broadly defined, according to a report by the RAND Corp. In 2013, the DoD tallied 377 in its records. Its review also ranked at least some of the 159 psychological health programs based on multiple areas, including outcomes, or lack thereof. However, with the exception of CSF, the names of the other programs were kept anonymous, referenced only by a number, making it impossible to know which ones scored highest or lowest, or reported data on outcomes at all.

A second FOIA request produced internal CSF surveys. Though not representative, they showed that while soldiers liked CSF, its effectiveness was questionable. In 2015, soldiers surveyed “with at least two years of service (87 percent)” could not identify scenarios to use at least half of the program's skills. And sometimes, material was delivered incorrectly, likely degrading any potential benefit.

In 2014, at Fort Benning, Georgia, 215 soldiers in basic combat training and 186 in OSUT from and infantry company, received a few hours of CSF training. A survey showed that eighty-four percent of those soldiers said the training was relevant to being a better soldier.

Mental Health photo

Bagram Airfield, Afghanistan. (U.S. Air Force/Tech. Sgt. Tyrona Lawson)

“Out of all the seemingly endless and useless PowerPoints we have thus far received in BCT, this has been the only one that really caught my attention as being significant and relevant to what we will face in the next few months, and in life outside the military,” an anonymous soldier commented in the 2014 survey.

A 2013 survey of 707 soldiers also showed positive feedback. Seventy-two percent said training was “beneficial in my Army life,” and 74% said training was “beneficial in my personal life.”

“In the formations that I've been, it improved our organizational readiness and capabilities,” said Major Joe Nussbaumer, stationed at Fort Benning, in a recent phone interview.

Staff Sergeant Justin Goelz, stationed at Fort Benning, agreed. His son has depression. “And just talking with him, and teaching him those [CSF] skills to deal with whatever challenges he's having,” has helped.

CSF's current leadership declined to be interviewed, but the DoD records ranked the program in the top 25% and categorized it with the term “developing effectiveness.”

The question remains as to the usefulness of the 158 other mental health programs or if the DoD will better track funding and document evidence of outcomes. As of now, the DoD declined to comment.

This article first appeared on The War Horse, an award-winning nonprofit news organization educating the public on military service, war, and its impact.