The Department of Homeland Security is not able to properly assess its biological weapon surveillance technology due to to a lack of testing standards, a government study has revealed.
BioWatch is the only federally managed bio-surveillance system designed to detect airborne-released biological weapons.
During their review of BioWatch’s capabilities, Government Accountability Office investigators found that since Homeland Security lacks technical performance standards, test results have nothing to be compared to, so officials are not able to interpret test results and draw conclusions about the system’s ability to meet its objectives.
“The BioWatch system was deployed quickly in 2003 to address a perceived urgent need; it was deployed without performance requirements and, as the National Academies has reported, without sufficient testing,” the report found. “In keeping with Office of Management and Budget (OMB) guidance on making decisions about federal programs, decisions about upgrades for BioWatch will require comprehensive information about the benefits and costs associated with the current system, including its capability to meet its operational objective.”
“However, DHS officials told us that in the 12 years since BioWatch’s initial deployment, they have not developed technical performance requirements against which to measure the system’s ability to meet its objective.”
Jim H. Crumpacker of the Homeland Security GAO Liaison office disputed the GAO’s findings that Homeland Security lacks technical performance standards. He also “disagrees with GAO’s conclusion that the BioWatch Program does not incorporate empirical data gathered on the currently deployed Gen-2 system to infonn modeling and simulation studies.”
DHS responded that it does use standards known as “Fraction of Population Covered” — abbreviated as “Fp” — which focuses on maximizing the amount of lives protected in a multiple attack scenario.
The GAO report recommended a change in these existing standards due to the fact that it is not clear “how Fp relates to the probability of detecting attacks large enough to cause 10,000 casualties, DHS’s stated objective for the BioWatch program.”
DHS agency officials declined to explain “how specific values of Fp would enable DHS to conclude what types and sizes of attack the system can detect.”
“Furthermore, officials said they have not commissioned or produced an analysis in which the best available test results are used to calculate Fp values and draw conclusions about the system’s ability to detect attacks of defined types and sizes,” the report continued.
The BioWatch program is currently operated locally in over 30 jurisdictions with over 600 Gen-2 aerosol collectors. The current process can take up to 36 hours to analyze for six different biological agents, and an updated autonomous system would speed this up potentially saving thousands of lives.
However, the GAO recommended in its report that the DHS should not pursue these upgrades until it finally establishes technical performance requirements that are “necessary for a biodetection system to meet a clearly defined operational objective for the BioWatch program by detecting attacks of defined types and sizes with specified probabilities.”