Government AI deployments are accelerating across intelligence and tax enforcement: the CIA is building its own ChatGPT-style tool for open-source intelligence analysis, the U.S. military is testing AI to handle classified data and issue field orders, and the IRS is expanding AI-driven audits targeting wealthy taxpayers — raising serious questions about hallucination risk, bias, and the absence of meaningful oversight mechanisms.
Stories Covered
CIA Building Its Own AI Chatbot for Intelligence Analysis
The CIA announced it is developing a large language model-based AI tool for intelligence analysis, described by Randy Nixon, the agency’s Director of Open Source Enterprise, as designed to help officers “find the needle in the needle field.” The system will be trained on publicly available data and is intended to accelerate the analysis of the massive and growing volumes of open-source information the agency collects. Nixon acknowledged that the scale of data collection has grown to the point where it can be “daunting and at times unusable” without better analytical tools. The stated goal — processing large volumes of public data to help analysts verify information faster — is straightforward and has obvious operational logic. The concerning dimension is what “publicly available data” means in practice when deployed by an intelligence agency with broad authorities, and how “quickly verify information” translates into actual use against data involving U.S. persons. Nixon stated the tool would follow U.S. privacy laws but provided limited detail on how the agency would guard against hallucinated outputs being used as the basis for decisions, or how it would prevent sensitive information from being inadvertently incorporated into training data.
The hallucination problem is not abstract in this context. When a general-purpose AI assistant confidently cites a non-existent paper, the consequence is embarrassment. When an intelligence agency’s AI analytical tool confidently associates a person with an event or connection that it fabricated from statistical inference rather than actual data, the consequence can be an investigation predicated on invented evidence. There is no mechanism in current large language models to reliably distinguish between facts they were trained on and facts they generated through pattern completion. A tool that “helps analysts quickly verify information” is only useful if the outputs can actually be trusted — and the hallucination characteristics of current LLMs make that trust difficult to establish without systematic human review of every output, which defeats the speed advantage the tool is supposed to provide. Separately, the CIA’s announcement landed in the same news cycle as reports of the U.S. military deploying thousands of autonomous weapons, building AI to handle state secrets, and deploying AI airspace monitoring over Washington D.C. The pattern across these announcements is consistent: AI as an analytical and decision-support tool being integrated into high-stakes government operations faster than the verification frameworks for its outputs have been established.
U.S. Military Testing AI to Handle Classified Data and Issue Field Orders
The U.S. Air Force disclosed that it is testing AI tools capable of processing secret-level and classified data. Colonel Matthew Strohmeyer told Bloomberg that in testing, the AI was able to complete tasks that would take human analysts hours or days in approximately 10 minutes. The near-term use case is decision support — an AI advisor that commanders can query for situational analysis — rather than fully autonomous command. Strohmeyer was clear that the military is not handing command authority to an AI system at this stage. The concern he flagged explicitly was hallucination: the military is working with developers to evaluate the trustworthiness of AI outputs specifically because of the known tendency of large language models to generate confident but false information. The Air Force also announced an AI-enabled airspace monitoring system being deployed around Washington D.C., using computer vision from a non-traditional Defense Department vendor to enhance threat detection. The framing on that specific application is more defensible — a monitoring system that flags something for human analyst review is using AI as a filter rather than a decision-maker, which limits the damage hallucination or error can cause.
The distinction matters. AI in a “surface for human review” role is meaningfully different from AI in a “reach a conclusion and act” role. The airspace monitoring system sits closer to the former; the classified data analysis tool that produces results in 10 minutes sits closer to the latter if those results are used without adequate verification. The legal and policy infrastructure for governing AI use in military operations has not kept pace with the technical capability being deployed. That lag is the structural risk, independent of whether the specific systems perform well in controlled testing.
IRS Expanding AI Audits: Concerns About Bias and Hallucination in Tax Enforcement
The IRS announced plans to expand use of AI for tax compliance enforcement, with an initial focus on higher earners, large partnerships, hedge funds, real estate investors, and law firms. The agency simultaneously re-stated its commitment to not increase audits on Americans earning under $400,000 and to protect lower-income filers who claim the earned income tax credit. The EITC provision is notable because it acknowledges that lower-income filers have already seen elevated audit rates relative to their income level — the IRS existing systems were already producing outcomes inconsistent with stated enforcement priorities before any AI involvement. Layering AI onto an audit selection system that has demonstrated existing bias concerns raises a legitimate question: if the training data reflects historical audit patterns, and those patterns are already biased toward certain demographics or income levels, the AI will learn and potentially amplify those patterns rather than correct them.
The broader concern is that AI audit selection is a black box to the people it affects. A taxpayer who receives an audit notice has no way to know whether the selection was made based on legitimate signal in their return or based on a pattern the AI inferred from training data that has nothing to do with tax compliance. The IRS’s own experts acknowledged they cannot afford a high-profile mistake in this rollout, particularly given the political pressure around demonstrating results ahead of the 2024 election cycle. Pressure to ship plus new technology with known reliability problems is the classic combination that produces quality failures at scale. The episode raises a systemic point that applies across all three government AI deployments discussed: these systems are being deployed because they offer genuine capability advantages, but the verification frameworks, audit mechanisms, and accountability structures for their outputs are materially less developed than the systems themselves. The question is not whether AI belongs in intelligence analysis, military decision support, or tax enforcement — the question is what oversight architecture needs to exist before the speed advantage these tools provide translates into consequential errors at scale.
Leave a Reply