Table of contents
ARTIFICIAL INTELLIGENCE & SOCIETY
PART 6
The series so far
Part 1 asked what happens when a system trusts the wrong account. Part 2 asked what happens when it trusts the wrong person. Part 3 asked what happens when the real person becomes the anomaly. Part 4 asked what happens when the public trusts the wrong reality. Part 5 asked what happens when a goal-seeking agent leaves a locked test and steals the answer key.
This part moves the trust failure into the room where a false file can become part of the official record. The courtroom.
The version that counts
I have been through the criminal justice system as a defendant. What I remember most clearly is the difference between having a version of what happened and having a version backed by the machinery of the state. Those are not the same thing.
That does not mean the machinery is automatically corrupt. Somebody has to investigate crime, somebody has to prosecute it, and most of the people doing those jobs are trying to get it right. But the imbalance is real. One side builds the official file and presents it through institutions people are taught to trust. The other side has to challenge it.
For most of history, convincing fabrication came with friction. Faking strong audio or video took equipment, skill, access, time, and risk. It could be done, but it was difficult enough that the difficulty itself acted like a safeguard nobody ever wrote into a rule.
What happens when that friction disappears? That is the question here.
Direct Answer
Federal courts still use a relatively low authentication threshold for evidence, and no AI-specific federal evidence rule is currently in force. The problem is not simply whether a detector can recognize a fake. It is who controls the original file, who gets to test it, what has to be disclosed, and whether the judge deciding the dispute can rule against the government without pressure. [3][4][9] To me, that makes judicial independence more important in an AI world, not less.
Fabricated evidence is not an AI invention
Start with what is already documented. The National Registry of Exonerations found official misconduct in more than 60 percent of the exonerations it studied. Concealed evidence appeared in 44 percent, and fabricated evidence appeared in roughly 10 percent, including planted evidence, false forensic claims, and confessions defendants never gave. [1][2] That number matters because AI does not have to invent a new human failure. It only has to make an old one easier.
I want to be precise about what I am saying. I could not find a documented case of a government using AI-fabricated evidence to convict a political opponent. I am not claiming that has happened. What the exoneration record does show is that people inside justice systems have fabricated evidence before, and that means the possibility cannot be dismissed by saying nobody would ever do it. [1][2]
The starting point is not fear of some futuristic abuse. It is a documented old abuse meeting a new capability. [1][2]
That changes the question. If fabrication becomes cheap, fast, and convincing, what has to become stronger on the other side?
How a recording gets through the door
Most people assume a court verifies a video before a jury sees it. That is not quite what happens. [3]
Under Federal Rule of Evidence 901, the proponent has to produce enough evidence for a reasonable juror to find that an item is what the proponent claims it is. The judge is not certifying that the recording is true. The judge is deciding whether there is enough foundation for the jury to consider it. [3]
A witness may identify the scene. A custodian may explain where a file came from. Metadata may support the chain. Those are sensible ways to authenticate ordinary evidence: they establish origin, custody, and context. [3]
Here is the harder version: what if the room is real, the voice sounds right, the metadata is plausible, and one sentence was manufactured?
That is where an old assumption starts to matter. The ordinary authentication rule was built in a world where most apparently ordinary recordings really were ordinary recordings. When fabrication was expensive, the adversary system could treat a fake as an exception. Deepfakes make the exception cheaper. [3][4]
The federal evidence committee has been wrestling with that exact problem. Its reporter warned that existing authentication rules may be too permissive when deepfakes are easy to create and difficult to detect. The committee considered AI-specific proposals, including a higher burden once a credible fabrication claim is raised, but in May 2026 it concluded that a rule change was not yet warranted and kept the issue under study. [4][9]
I understand the hesitation. A rule written too quickly around fast-moving technology can create a different problem. The hesitation has a consequence too: right now, the old rule is still meeting the new capability.
The problem already runs in both directions
Synthetic evidence in court is no longer just a forecast. In 2025, a California court dismissed a case after finding that parties had submitted AI-fabricated exhibits, including a fake video. The National Center for State Courts noted that the deception was caught partly because metadata exposed an obsolete-device problem. [5][6] That detail stuck with me. The fake was not defeated by some perfect detector. It was defeated because the forger made a mistake.
Government lawyers are not standing outside the problem either. In Kohls v. Ellison, a federal judge excluded an expert declaration filed by Minnesota’s attorney general after GPT-4o generated citations to academic articles that did not exist. The court accepted that the mistake was unintentional. False material still entered a federal court filing in a case about deepfakes. [7][8]
At the same time, there is a danger in overcorrecting. In 2026, New York’s highest court rejected video evidence in a child-abuse case because the authentication foundation was insufficient, with the majority recognizing that deepfakes make circumstantial authentication less reliable. Three dissenting judges warned that baseless deepfake claims could become a ready-made way to attack real evidence. [10][11]
Both sides of that disagreement are pointing at a real risk: synthetic media can undermine false evidence and real evidence at the same time. [10][11]
Raise the bar too far and some genuine evidence gets excluded. Leave it too low and some fabricated evidence gets through. The existence of convincing fakes creates both problems at once.
What should the rule optimize for when there is no setting that only catches lies?
I do not think the answer is a single tougher threshold: I think the answer is better process around the cases where authenticity is genuinely disputed.
No AI-specific federal rule is in force yet
Two proposals have been considered by the federal Advisory Committee on Evidence Rules. Proposed Rule 707 would address machine-generated evidence offered without an expert. A proposed Rule 901(c) would create a burden shift for disputed AI-generated or manipulated evidence after the opponent first makes a sufficient showing that fabrication may have occurred. [4][9]
The committee did not advance either proposal at its May 2026 meeting. It concluded that an amendment was not yet warranted, in part because relatively few federal judges reported encountering deepfake issues, and chose to keep studying the problem. [9]
That is an important qualification. Courts are not drowning in synthetic-evidence disputes, and we should not write policy as though they are. But low case volume does not erase the structural question, because the first serious failure does not become harmless just because it is early. [9]
For now, the tool available is still a judge applying familiar evidence rules to a capability those rules were not written around. [3][9]
The deeper problem is who controls the answer
This is where I think the technology conversation gets too narrow. We talk about better detectors as though the detector settles the dispute. But somebody chooses the detector. Somebody hires the expert. Somebody controls the original file. Somebody decides what the defense gets to inspect.
There is already a useful warning in a 2026 case from India. Police alleged that men created forged forensic reports declaring a politically sensitive video AI-generated, while a forensic examiner alleged pressure and payment for a predetermined conclusion. The political figures involved denied wrongdoing, and the allegations remain contested. [12][13]
I am not using that case as proof of what would happen in the United States. It is not proof of that. What it shows is the mechanism: once expert authentication becomes the answer, control over the expert can become as important as control over the file.
That made me look at the problem differently: the real fight is often over who gets to certify reality.
The question is not only, “Can we detect a fake?” It is also, “Who gets to declare the file real?” Those are not the same question.
Independence is the safeguard
Poland offers a documented example of what happens when judicial discipline becomes politically controlled. The Court of Justice of the European Union held that Poland’s disciplinary structure failed to guarantee judicial independence and created a risk that disciplinary proceedings could be used to control judges. [14]
Notice what makes that dangerous. Nobody has to call a judge and order a particular verdict. If the people above the judge control appointment, discipline, assignment, or removal, the pressure can already be built into the structure.
Now bring that back to a disputed recording. The defense says the file is fabricated. The government says it is real. The government’s lab says it is real. The judge has to decide whether to order independent testing, force disclosure of source files, hold a pretrial authenticity hearing, exclude the item, or sanction the people who offered it.
What happens if that judge knows the wrong ruling can cost the judge the job?
That is why I keep coming back to independence. The hierarchy is pretty simple: authentication rules matter, provenance matters, and experts matter. But every one of those safeguards eventually reaches a human decision-maker who has to be free to say no. If that decision-maker cannot rule freely, the rest can become decoration.
What I would require before the jury sees the file
I think the right response looks like this. What follows is my recommendation, not a description of current law.
I think a credible, particularized fabrication claim should trigger a pretrial authentication hearing before the jury sees the recording. Preserve the original file, not just a compressed export. Disclose provenance information, chain of custody, metadata, and known AI processing. Give the defense access to the same underlying material and enough funding to retain an independent examiner when the defendant cannot afford one.
I think the government’s lab cannot be the only lab, and a black-box authenticity score cannot stand in for an expert who can explain what was tested, what was not, and what would change the conclusion.
The proposed Rule 901(c) framework considered by the federal rules committee used a similar burden-shifting idea. Once an opponent produced enough evidence to support a finding that an item may be AI-generated or manipulated, the burden would move to the proponent to establish authenticity by a higher standard than ordinary Rule 901 requires. The proposal is not law. [4]
To me, that logic makes sense because it does not treat every recording as suspicious. It asks for more only after there is a real reason to doubt the file.
I would add something the rule itself cannot solve: who pays for the defense expert. A hearing is not meaningful if one side arrives with a forensic lab and the other arrives with a public defender and a laptop.
The disclosure duty already exists
There is another safeguard that does not need a new AI rule.
Under Brady v. Maryland, prosecutors have a constitutional duty to disclose evidence favorable to the accused when it is material to guilt or punishment. [15] If the government knows that a file was generated, reconstructed, enhanced, altered, or passed through a tool that materially changed its content, that information can affect authenticity and the defense’s ability to challenge it.
The harder part is enforcement. The government decides in the first instance what it knows and what it discloses. If there is a dispute, a court has to be willing to enforce the duty. [15] Which brings us right back to independence.
Make the penalty match the crime
I also think knowing fabrication for use in court should carry consequences serious enough to match what is being risked. This is again my recommendation, not current law.
Knowingly manufacturing synthetic evidence for use in court, or knowingly offering fabricated material as authentic evidence, should carry serious criminal and professional penalties. The same should apply to people in the confirmed chain who knowingly help move the fake into the record.
I am not talking about an honest mistake, a bad enhancement, or a file somebody misunderstood. Kohls is a useful example of why intent matters: fabricated citations entered a court filing, but the judge accepted that the error was unintentional. [7][8] Mistakes need disclosure, correction, and appropriate sanctions. Deliberate fraud on a court is a different category.
Why draw the line that hard? Because the harm is bigger than one case. A fabricated file can affect detention, employment, reputation, plea decisions, trial strategy, and freedom long before a later review corrects the record. But introducing synthetic evidence into a justice system does something else too: it teaches people that the evidence itself may be manufactured. Once that belief takes hold, every real recording becomes easier to dismiss and every official file becomes a little harder to trust.
That is not just harm to a defendant. It is damage to the system that governs all of us. Courts depend on people believing that evidence can be challenged, tested, and ultimately trusted when it survives that process. If an officer, prosecutor, lawyer, expert, or anyone else knowingly manufactures evidence and tries to pass it through that system as real, I think the law should send an unmistakable message that this is not clever litigation, aggressive prosecution, or a technical violation. It is an attack on the integrity of the justice system itself.
If I were writing the penalty, I would make it a mandatory minimum of five years in prison for knowing fabrication or knowing use of synthetic evidence as authentic evidence in a legal proceeding. I would also prohibit minimum-security placement for that sentence. No prison camp. No version of “Club Fed” for someone who knowingly tried to manufacture the proof that could take another person’s freedom.
That sounds severe because I mean it to be severe. The point is deterrence, but it is also trust. If the public is going to accept the enormous power we give police, prosecutors, courts, and other government actors, there has to be a bright line around manufacturing proof. Cross it knowingly, and the consequence should be serious enough that nobody inside the system can mistake where that line is.
There is another reason I want that line to be unmistakable. The damage does not stay inside the case where the fabrication happened. Once one synthetic exhibit is knowingly planted by someone the public was supposed to trust, every defense lawyer has a stronger reason to challenge the next real file, every juror has a stronger reason to wonder whether the next recording was manufactured, and every legitimate prosecutor has a harder job proving that the evidence in front of the court deserves confidence. One act of fabrication creates doubt that honest cases have to carry afterward. That is why I see this as more than obstruction or misconduct. The perpetrator is spending the credibility of the entire justice system for the sake of winning one case.
The person with the least resources loses first
When people imagine a government manufacturing a file, they picture a political opponent with national reporters outside the courthouse. That is the dramatic version.
The first person I worry about is less visible. It is the defendant with a public defender, no money for a forensic examiner, and a hearing nobody covers.
If a convincing fake appears in that case, who catches it?
The defendant says, “That is not me.” The officer says it is. The lab says the file is clean. The prosecutor says there is no reason to doubt it. The public defender has a full caseload and no independent expert standing by. The judge has a calendar to move.
This is where the old exoneration data matters again. Wrongful convictions do not require every person in the system to be corrupt. They can happen when one bad act survives enough ordinary checkpoints: a hidden report, a false forensic statement, a confession that never happened, or a piece of evidence nobody could challenge in time. [1][2]
AI does not change that structure. It changes the quality and accessibility of the thing that can move through it. That is the part I think matters most. A system does not become safe because most people in it are decent. It becomes safer when one bad act still has to survive independent people with the power, information, and resources to stop it.
The actual safeguard
I think “trust nothing” is the wrong answer to deepfake evidence because courts cannot function that way. I also think “trust the detector” is the wrong answer. The detection race will keep moving, and the best fake next year will be designed against the best detector this year.
What follows from all of this? I think the durable safeguards are procedural and institutional: preserve originals, disclose processing, fund independent examination, raise the burden when there is a credible fabrication claim, punish knowing fabrication hard enough that nobody treats it like a litigation tactic, and keep the judge deciding all of it independent from the people offering the evidence.
I have sat on the side of the courtroom that has to answer the state’s version. I know what it feels like when the file in front of everyone carries more authority than the person saying it is wrong. That experience is part of why I do not want the answer to be “trust the system,” but it is also why I do not think the answer is “trust nothing.”
The better answer is to make the system prove more when there is a credible reason to doubt it, and to make sure somebody independent has the power to require that proof.
If we reach a point where the government can manufacture the file, choose the expert who validates it, withhold the underlying material, and pressure the judge who decides whether it comes in, then the problem is not artificial intelligence anymore.
The problem is that there is nobody left in the room who can say no.
Frequently Asked Questions
Can deepfake evidence be used in court?
A video, audio file, image, or other digital item can be admitted if the party offering it satisfies the applicable authentication rules. In federal court, Rule 901 asks for enough evidence to support a finding that the item is what the proponent claims it is. That is not the same as a judicial finding that the item is genuine. [3]
Is there a federal evidence rule specifically for deepfakes?
No AI-specific evidence rule is currently in force. The federal Advisory Committee on Evidence Rules has considered proposed Rule 707 for machine-generated evidence and proposed Rule 901(c) for disputed AI-generated or manipulated evidence, but as of its May 2026 meeting it had not advanced either proposal. [4][9]
Have deepfakes actually been submitted as evidence?
Yes. In Mendones v. Cushman & Wakefield, a California court imposed terminating sanctions after finding that parties submitted AI-fabricated exhibits, including a deepfake video. The National Center for State Courts has highlighted the case as an early warning for courts. [5][6]
Has the government used AI-fabricated evidence to convict a political opponent?
I could not find a documented case of that. The political-prosecution scenario in this article is a projection built from documented risks, not a claim that such a prosecution has already happened. [1][2][12][13][14]
What should happen when a defendant credibly claims a video is fake?
My recommendation is a pretrial authentication hearing, preservation and disclosure of the original file and metadata, disclosure of known AI processing, access to an independent forensic examiner, and a higher burden on the party offering the file once the opponent makes a credible showing of fabrication. Proposed Rule 901(c) uses a similar burden-shifting concept, but it is not currently law. [4]
Why is judicial independence part of an AI evidence problem?
Because every technical safeguard eventually needs enforcement. A judge may have to order disclosure, authorize independent testing, exclude evidence, or sanction misconduct. If the judge can be punished for ruling against the people in power, the evidence rules become much weaker protections. [14]
What protects defendants today?
Existing authentication rules, cross-examination, discovery, constitutional disclosure duties such as Brady, expert testimony, appellate review, sanctions for misconduct, and judicial independence all matter. Synthetic media increases the importance of those safeguards and exposes where access to them is unequal. [3][15]
References
[1] National Registry of Exonerations. Government Misconduct and Convicting the Innocent: The Role of Prosecutors, Police and Other Law Enforcement (September 1, 2020). https://exonerationregistry.org/sites/exonerationregistry.org/files/documents/Updated%20CP_Government_Misconduct_and_Convicting_the_Innocent.pdf
[2] National Registry of Exonerations. 2024 Annual Report. https://exonerationregistry.org/sites/exonerationregistry.org/files/documents/2024_Annual_Report.pdf
[3] Federal Rule of Evidence 901(a) (requirement of authenticating or identifying an item of evidence). https://www.law.cornell.edu/rules/fre/rule_901
[4] Advisory Committee on Evidence Rules, agenda book for the May 7, 2026 meeting (Reporter’s memorandum on deepfakes and draft Rule 901(c)). https://www.uscourts.gov/sites/default/files/document/2026-05-evidence-rules-agenda-book.pdf
[5] Mendones v. Cushman & Wakefield, Inc., No. 23CV028772 (Cal. Super. Ct., Alameda County, Sept. 9, 2025) (terminating sanctions for deepfake exhibits), as reported in eDiscovery Today. https://ediscoverytoday.com/2025/09/25/deepfake-videos-and-images-lead-to-terminating-sanctions-ediscovery-case-law/
[6] National Center for State Courts. “AI-generated evidence is a threat to public trust in the courts.” https://www.ncsc.org/resources-courts/ai-generated-evidence-threat-public-trust-courts
[7] Kohls v. Ellison, No. 24-cv-3754 (D. Minn. Jan. 10, 2025) (order excluding expert declaration containing AI-generated citations). https://storage.courtlistener.com/recap/gov.uscourts.mnd.220348/gov.uscourts.mnd.220348.46.0.pdf
[8] Reuters. “Judge rebukes Minnesota over AI errors in ‘deepfakes’ lawsuit” (Jan. 13, 2025). https://www.reuters.com/legal/government/judge-rebukes-minnesota-over-ai-errors-deepfakes-lawsuit-2025-01-13/
[9] Advisory Committee on Evidence Rules. Report to the Standing Committee on Rules of Practice and Procedure (May 17, 2026) (Rule 707 and draft Rule 901(c) status; Federal Judicial Center survey). https://www.uscourts.gov/sites/default/files/document/advisory_committee_on_evidence_rules_may_2026.pdf
[10] Matter of M.S. (M.H.), 2026 NY Slip Op 00825 (N.Y. Feb. 17, 2026). https://law.justia.com/cases/new-york/court-of-appeals/2026/7.html
[11] New York State Bar Association. “New York State Law Digest: March 2026” (summarizing Matter of M.S. majority and dissents). https://nysba.org/web-post-new-york-state-law-digest-march-2026/
[12] The Times of India. “AI video controversy: Two arrested in fake forensic report case; Punjab CM Bhagwant Mann linked to clip” (June 6, 2026). https://timesofindia.indiatimes.com/city/chandigarh/ai-video-controversy-two-arrested-in-fake-forensic-report-case-punjab-cm-bhagwant-mann-linked-to-clip/articleshow/121664828.cms
[13] The Tribune. “Gurugram Police arrest two for fabricating forensic reports in Punjab CM video case” (June 6, 2026). https://www.tribuneindia.com/news/punjab/gurugram-police-arrest-two-for-fabricating-forensic-reports-in-punjab-cm-video-case/
[14] Court of Justice of the European Union, Case C-791/19, Commission v. Poland, Judgment of July 15, 2021 (disciplinary regime for judges incompatible with EU law). https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:62019CJ0791
[15] Brady v. Maryland, 373 U.S. 83 (1963). https://supreme.justia.com/cases/federal/us/373/83/