How do we stop an AI assistant from making things up about our contracts?

The failure has a government contracting record already. In May 2025 the Government Accountability Office dismissed a bid protest, Raven Investigations & Security Consulting, B-423447, as academic after the agency terminated the order, and added a warning: the protester had cited decisions "that did not appear to state the principles for which the protester cited them," others GAO was "unable to locate," and quotations GAO could not locate in any of its decisions. The protester blamed AI-assisted tools. GAO answered that drafting with AI "without engaging in any review of the material for accuracy" is a practice that "wastes the time of all parties and GAO," and that "citation irregularities may result in the imposition of sanctions." Swap the citations for your own contract numbers and the risk has the same shape.

Making things up has a name, and it is not rare

OpenAI's September 5, 2025 explanation, Why language models hallucinate, defines hallucinations as "plausible but false statements generated by language models" and argues that "standard training and evaluation procedures reward guessing over acknowledging uncertainty," the way a multiple-choice test rewards a guess over a blank. Its conclusion matters for a contracts office: hallucinations are not inevitable, "because language models can abstain when uncertain." The OWASP Top 10 for LLM Applications 2025 lists the risk as LLM09, Misinformation: "false or misleading information that appears credible," made costly by overreliance, "failing to verify its accuracy."

The rate is measurable. A Stanford and Yale team (Magesh, Surani, Dahl, Suzgun, Manning and Ho) ran 202 preregistered legal questions through three commercial research tools built on retrieval and found that they "each hallucinate between 17% and 33% of the time", lower than a general chatbot but not close to zero. Their definition is the useful part: "either incorrect information or a false assertion that a source supports a proposition." A real document cited for something it does not say counts, and the authors call those errors "potentially more dangerous than fabricating a case outright, because they are subtler and more difficult to spot." A public database kept by the legal researcher Damien Charlotin, listing decisions where a court or tribunal found, or implied, that a party relied on hallucinated material (plus some where AI use was alleged but not confirmed), held 2,289 cases as of its November 10, 2026 update. Contractors report the worry themselves: in Deltek's 17th annual Clarity study (May 2026, 917 government contractors), 29 percent named inaccurate forecasts and 26 percent difficulty validating AI-provided insights among their top AI concerns.

Contract facts live in records, so the assistant reads records

The facts a contracts manager gets asked about have a home. Deltek's Costpoint 8.2 help describes the Manage Contracts screen ("Use this screen to set up and manage information about your contracts," for firms licensed for Capture & Contracts) with tabs including General, Dates, Modifications and Performance Assessment. Contract Type and Contract Status come from controlled lists. The Dates tab holds the contract start and end dates. The Modifications tab, "to enter and track contract modifications," lists each modification's effective date, its period of performance start and end, a Contract Value Modifications Amount and a Funding Modifications Amount, under a Contract Totals row; the signed and funded values themselves are entered by modification on Manage Modifications, starting with Modification 0000. CPARS assessments sit on the Performance Assessment tab, as set out in Can AI draft government proposals from past performance without compliance risk?

That gives the rule. "What is the funded value on this task order" has one right answer: the cumulative funded total across the modifications, as of a run time, with the latest modification named. "Has the option year been exercised" has one too: the modification that exercised it, or the statement that no modification on record does. Neither comes from what a model remembers about contracts in general. Each comes from a read performed for that question, and the answer carries the read.

What is the funded value on task order 2241.003, and when does the current period of performance end?

$1,842,500 funded across four modifications. The current period of performance ends in 31 weeks.

Source

Costpoint, Manage Modifications, project 2241.003, Modifications 0000 to 0003 (cumulative funded total); Manage Contracts, Modifications tab, period of performance end on Modification 0003

Run

Today, 7:10 am

Filter

All modifications on record for 2241.003; funded value, not signed value

Checked

The cumulative total matches the Contract Totals row on the Modifications tab; the latest modification was entered 6 weeks ago

Has the customer exercised the option year on 2241.003?

I can't find a record that says so. No modification after 0003 exists in Costpoint, and the contract file holds no document dated in the last 6 weeks. Ask the contracts manager whether an option modification is expected.

Looked in

Costpoint, Manage Modifications and the Modifications tab for 2241.003; the contract file folder for 2241.003

Found

Four modifications, the latest 6 weeks ago; no document in the file newer than that

Why no answer

Nothing on record states that an option was exercised, and this assistant does not infer contract events from silence

Sample project, figures and dates, to show the shape of an answer that cannot be invented. The first answer can be checked in one click against the modification it names. The second refuses to guess and says what it checked, which is itself useful: it tells the asker which document is missing.

Four behaviors do the work

First, read before answering. Every figure comes from a record read for that question, never from the model's general knowledge. The Stanford and Yale team's warning about misgrounded citations says why the read has to be specific: checking one "requires users to click through to cited references, read and understand the relevant sources, assess their authority, and compare them to the propositions". A source line that says "Costpoint" cannot be checked. One that names the screen, the project and the modification can.

Second, show the trace under every answer: the record, the run time and the filter. "Every answer shows where it came from. No black boxes." Third, decline. When no record answers, the assistant says so and lists where it looked, and the firm counts that as a success, because it is the behavior OpenAI's paper says the usual scoreboards punish. Fourth, keep the arithmetic and the permissions out of the model. Totals come from the records, not from the model adding numbers in prose, and who may see which contract is decided by Costpoint's own security settings, checked before any record is read, as described in How do you keep an AI assistant from showing employees Costpoint data they can't see? OWASP's LLM07:2025 guidance is blunt on this point: controls such as "authorization bounds checks" must not be delegated to the model and "need to occur in a deterministic, auditable manner."

Deltek tells its own users to double-check

None of this is a criticism of the model inside Costpoint. Deltek's 8.2 help page on using Ask Dela lists, among its guidelines, "Verify Responses: Double-check responses, especially for crucial decisions or data, as occasional inaccuracies may occur," and "Report Issues: Report persistent inaccuracies to Deltek Support Services for improvements." The Ask Dela page says responses "include direct links to specific applications, sub-tasks, and records" and that users can give feedback "by liking or disliking the responses." Deltek's August 2025 overview says "Costpoint uses customer data to generate meaningful prompts before sending them to OpenAI Azure services." Same design: ground the answer in a record, link to it, let a person judge it.

For a question that lives inside one Costpoint record, Dela's linked answer is the place to start, with the caveat that Deltek's own documents disagree on whether Dela is available in GovCon Cloud Moderate, as laid out in Is Deltek's Dela available in GovCon Cloud Moderate, and what are GCCM customers' options for AI? The fabrication risk grows where the question crosses systems: the funded value in Costpoint, the signed modification in the contract file, the contracting officer's email about the option. Stitching those together is where a model has the most room to fill gaps, and where the work of tying every clause of the answer to a document belongs.

A person still signs, and the record stays

A wrong contract fact becomes a compliance problem the moment it leaves the firm. A funded value misstated on an invoice, an option year assumed in a staffing plan, a period of performance misremembered in a proposal: each is a statement the firm made. The False Claims Act's definition of "knowingly" covers "reckless disregard of the truth or falsity of the information," and the Raven decision shows how far blaming the tool gets before a tribunal. So the assistant drafts and a person approves: "AI drafts and flags; people approve and sign." The firm keeps the trail. FAR 52.215-2 defines records to include "other data, regardless of type" and "in the form of computer data," so the question, the records read and the answer stay with the contract file whenever the answer fed a bill, a notice or a proposal. The log also tells you, months later, which answers were wrong and why; What should a government contractor's AI use policy include? covers where that rule lives in writing.

What the first month should look like

Pick the twenty active task orders one contracts manager knows best. Write down thirty questions that manager can already answer from the file: funded value, period of performance, contract type, latest modification, CPARS status, whether an option has been exercised. Have the assistant answer each one as that person, with that person's Costpoint rights. Score every answer one of three ways: right with a trace that opens to the record in one click, declined with a plain list of where it looked, or wrong. Count the declines as wins. One wrong answer under a confident trace is the finding that matters, and it is far cheaper to find in week two than in a bill. Fix the inputs that produced it, run the thirty again, and only then widen the portfolio. If after four weeks the contracts manager stops opening Costpoint to check the answers, extend it. If not, the assistant has shown you what it still cannot read.

Common questions

If we give the AI our own documents, does that stop it from making things up?

It reduces it; it does not end it. A Stanford and Yale study of three legal research tools built on retrieval (Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI) ran 202 preregistered queries and found they hallucinated between 17 and 33 percent of the time, less than a general chatbot but far from zero. Its definition counts a real source cited for something it does not say, which the authors call misgrounded and describe as harder to spot than an invented source. Giving the model your documents is the start; the citation a reader can open, the refusal when nothing on record answers, and the human check are what make the answer safe to act on.

Does Deltek's Dela make things up about contracts?

We found no published error rate for Dela, and we have not measured one. Its Costpoint 8.2 help page on using Ask Dela tells users to verify responses, double-checking them especially for crucial decisions or data because occasional inaccuracies may occur, and to report persistent inaccuracies to Deltek support. The Ask Dela page says responses include direct links to the applications, subtasks and records they draw on, and that users can like or dislike a response. Deltek's August 2025 overview says Costpoint builds the prompt from the customer's data before it goes to the model. So Deltek's own design points the same way: answers tied to records, and a person who checks them.

What should the assistant say when the answer is not in Costpoint?

That it cannot find it, and where it looked. OpenAI's September 2025 explanation of hallucinations argues that models guess because training and evaluation reward a guess over an admission of uncertainty, and that models can abstain when uncertain. An assistant for contract questions should be judged the same way the firm would judge a new contracts analyst: a plain "no record says that, here is what I checked" beats a confident figure with nothing behind it. The absence is itself useful information; it tells the asker which modification or document is missing.

Is an AI answer about a contract a record we have to keep?

Our reading, which is research and not legal advice, is that it can be. FAR 52.215-2 defines records to include books, documents and other data regardless of type and whether in written form or in the form of computer data, and requires them to be kept for three years after final payment on cost-type and similar contracts, or any shorter period FAR subpart 4.7 allows. If an assistant's answer about funded value or period of performance feeds a billing, a notice to the contracting officer or a proposal, keep the question, the records it read and the answer with the file. The log is also how you find out, later, which answers were wrong.