Verifying AI output in digital forensics: what a court needs to see

Every conversation about AI in digital forensics eventually arrives at the same question, usually phrased as a challenge: how do we know it is not making things up.

It is the right question, and the usual answers are bad. Vendors respond with accuracy percentages, which are unfalsifiable in a courtroom because nobody can reproduce the benchmark. Or they respond with reassurance about the quality of the model, which is a claim about a component rather than about the process the evidence went through.

The better answer is uncomfortable and much stronger: assume it will be wrong sometimes, and build so that being wrong is cheap to detect.

What a court is actually asking

A court is not evaluating a model. A court is asking whether a specific factual assertion in a report can be supported.

That question has a shape that predates AI by decades. Where did this come from. Who examined it. Can it be shown to me. Can the defence examine the same material. Was anything changed. The introduction of a language model into the workflow does not change the question, it only changes how easy or hard it is to answer.

So the design requirement is not that the model be right. It is that every statement it produces resolves to a specific record that a human can open, read and, if necessary, put in front of a judge.

If a claim cannot be traced back to a record, it should not be in the report, whether a model produced it or a tired human did at eleven at night.

Citation is a structural property, not a feature

There is a large difference between a system that usually mentions its sources and one that cannot produce output without them.

In the first, citation is a behaviour. It is achieved by asking nicely, it degrades in unusual cases, and it fails exactly where you most need it: the long, complicated answer synthesising several threads, which is the one an examiner is least able to check from memory.

In the second, the retrieval step happens first and the identity of the retrieved records travels with the answer through the whole pipeline. The citation is not something the model wrote. It is a record of which rows were read.

That distinction is what an evaluator should probe. The useful question is not “does it cite sources”. It is “what happens if it cannot find one”, and the only acceptable answer is that no assertion is produced.

Verification has to be cheap or it will not happen

Here is the part that gets underestimated, and it is a human factors problem rather than a technical one.

Any process that requires the examiner to spend an afternoon confirming a finding will, under caseload, produce fewer confirmed findings. Not because examiners are careless, but because time is finite and the confirmation competes with the twelve other seals waiting. A verification step that is expensive is a verification step that gets rationed, and rationed verification is worse than no automation at all, because it carries the appearance of rigour without the substance.

So the measure that matters is the cost of one check. If landing on the source record takes one click and shows the original row in its original context, verification stops being a decision the examiner has to make. It becomes the default because skipping it saves nothing.

This is the argument for building traceability into the record model rather than reconstructing it at display time. A citation that still resolves after the mapping producing it has been revised is a citation an examiner can rely on months later, when the case comes to court and nobody remembers the details.

What to ask a vendor

If you are evaluating any tool with an AI assistant in it, four questions separate the serious ones from the rest.

Can I click any statement and land on the source record? Not a document, not a page, the record. Ask them to do it live, on a claim you choose, in a thread you pick.

What is logged? Every action the assistant took and every result it received should be written somewhere you control and can read. If the log lives with the vendor, that is a different product with a different risk profile.

Where does the data go? If the answer involves a third-party inference provider, that is a decision your service has to make deliberately rather than discover in a security review after deployment.

What does it do when it does not know? Ask a question the data cannot answer and watch. A system that produces a confident, well-written and entirely unsupported paragraph has told you everything you need to know about how it will behave on a real case at two in the morning.

The honest position

Independent forensic experts have been consistent on this: AI output has to be verified against original data before it can be relied on. That is not a limitation to be engineered away, it is the correct relationship between an automated system and an evidentiary process.

The value of putting a model into forensic work is not that it replaces the examiner’s judgement. It is that it makes exercising that judgement possible at a volume where it otherwise quietly stops happening. An examiner who could previously read five percent of a device and now reads the relevant part of all of it, with every conclusion checkable in a click, is doing better forensic work, not more automated forensic work.

That is the only claim worth making, and it is the one I would rather be held to.