Education

Pentagon Didn’t Make Sure Evaluations It Paid For Evaluated Anything Useful

Tristyn Bloom Contributor

The Department of Defense, which helps pay for active servicemen and women pursue post-secondary education, wanted to make sure the tuition fees it was paying were going to legitimate institutions, so it did what any reasonable government agency would do — hired a contractor to conduct evaluations. The only problem: It never told the contractor what to evaluate the schools on, and now a newly released government report is saying these taxpayer-funded evaluations are completely useless, and not only that — they were conducted by unqualified observers.

The DOD’s Tuition Assistance Program helps active duty service members pay for undergraduate, graduate, vocational, and other post-secondary courses during their off-hours, spending $540 million on the program in 2013 alone. That year, Congress asked the GAO to look into the evaluations of those programs the Pentagon had paid an outside contractor to conduct, and their findings, released Monday, are disturbing.

The GAO’s report, politely titled “Action Is Needed to Ensure Evaluations of Postsecondary Schools Are Useful,” explains that the evaluations “did not provide the agency the information it needed to assess schools” because “DOD lacked a specific plan to frame the evaluations” — in other words, the Pentagon never told the contractors it hired what to evaluate. Not only that, but “DOD’s contract did not specify all the skills needed by the contractor… to ensure that the contractor provided personnel with the requisite education and experience needed to conduct the evaluations.” (RELATED: Pentagon Still Allowing Contractors To Massively Overcharge For Parts)

While the DOD did tell the contractor its evaluations should cover 15 areas, the GAO found that these areas “were often broad, not clearly defined, and lacked specificity… Based on our review of these 15 areas, it was not always clear what DOD was asking the contractor to evaluate and how the 15 areas would be measured.”

Because the DOD did not initially require evaluation personnel to have appropriate qualifications, “data from student surveys were misinterpreted and erroneously reported.” The Pentagon did eventually modify the contract to require that staff have “a working knowledge of measurement methods and tools; sufficient experience in postsecondary education; and expertise in education theories, principles, and practices”–but not until 2013, and even after the modification the GAO found evidence in 2014 reports of continued incompetence. (RELATED: Three Highest-Paid Pentagon Officials Are All Football Coaches)

“Specifically, the contractor made broad generalizations about student satisfaction with their school based on survey responses from a non-representative sample of students whose responses may not have been the prevailing view of the other students who did not respond,” the GAO explains. This was not an isolated incident — “a recent evaluation report stated that there was a ‘high overall degree of student satisfaction’ with the institution based on responses from a non-representative sample of students.”

The report also found that some of the contractor evaluations made recommendations “only tangentially related to quality.” One contractor recommended “replacing the artwork in the classrooms with pictures of students in the hallways to foster a positive climate,” while another advised changing the names of academic terms to “mid-Fall,” “mid-Winter,” “mid-Spring,” and “mid-Summer,” (presumably as opposed to simply the names of the seasons themselves).

“Although the federal investment in service member education is substantial,” the GAO concludes, “DOD’s current approach has left the department without the information it needs about the quality of schools that served about 280,000 service members in fiscal year 2013.”

With typical bureaucratic understatement, they explain that “by not having a plan in place to guide the evaluation of schools, DOD’s ability to effectively assess the schools has been limited,” and recommend developing “a plan for future school evaluations that includes, among other things, clearly-defined evaluation questions and an assessment of the experience, expertise, and skills needed by the personnel from the entity or entities conducting the school evaluations.”

In a statement responding to, and concurring with, the GAO’s findings, the Pentagon said that “we take our responsibility to act as good stewards of taxpayer dollars and to provide quality opportunities to facilitate educational goal attainment very seriously.”

Follow Tristyn on Twitter.