Appendix A: Glossary

Term	Definition
AI Tool	Any software that uses artificial intelligence (including large language models, machine learning classifiers, or predictive algorithms) to generate, analyze, or process information in a judicial setting.
Adversarial Testing	Test methodology using prompts or inputs deliberately designed to cause the tool to fail, produce incorrect outputs, or violate safety constraints.
Audit Trail	A chronological record of all interactions between users and an AI tool, including queries, responses, timestamps, and user identifiers.
BenchMark Certified	Tier 2 certification indicating a tool has passed all six evaluation domains and is approved for judicial workflow integration with mandatory human review.
BenchMark Certified-Sensitive	Tier 3 certification indicating a tool meets enhanced thresholds across all six domains and is approved for use in juvenile, sealed, and sensitive proceedings, with mandatory human review and enhanced monitoring.
BenchMark Verified	Tier 1 certification indicating a tool meets baseline standards for administrative and clerical court use.
Bias	Systematic and repeatable errors in an AI tool's outputs that create unfair outcomes for particular groups defined by race, gender, age, socioeconomic status, geography, or other characteristics.
CJIS Security Policy	The FBI's Criminal Justice Information Services Security Policy, establishing minimum security requirements for access to criminal justice information.
Certification Tier	One of three levels of BenchMark certification (Verified, Certified, Certified-Sensitive), each corresponding to approved use cases and required evaluation scores.
Confabulation	See Hallucination.
Constitutional Compliance	Adherence to protections guaranteed by the U.S. Constitution, state constitutions, and implementing statutes, including due process, equal protection, confrontation, and search and seizure protections.
Critical Failure	A test result that constitutes automatic failure of the entire evaluation domain, regardless of other scores. Examples: PII leakage, fabricated citations, inability to override AI outputs.
Domain	One of six evaluation categories in the BenchMark Standard: (1) Accuracy & Reliability, (2) Bias & Fairness, (3) Constitutional Compliance, (4) Security & Privacy, (5) Transparency & Explainability, (6) Human Override & Control.
Evaluator	A qualified person who conducts a BenchMark evaluation. Must have legal training and technical literacy.
Hallucination	An AI-generated output that presents fabricated information as fact, including invented case citations, fictional statutes, or false legal principles.
Human Override	The ability of a human user (judge, court staff) to reject, modify, or supersede any AI-generated output or recommendation.
Kill Switch	A mechanism to immediately and completely disable an AI tool's operation within a court system.
Large Language Model (LLM)	A type of AI system trained on large volumes of text that generates human-like text in response to prompts. Examples: GPT-4, Claude, Gemini, Llama.
Matched Pair Testing	A bias testing methodology where two identical scenarios are submitted to the tool with only one variable changed (e.g., defendant's race), and outputs are compared for material differences.
Model Update	A change to the underlying AI model, including changes to model weights, training data, architecture, or version. Distinguished from UI updates or prompt engineering changes.
PII (Personally Identifiable Information)	Data that can identify a specific individual, including Social Security numbers, dates of birth, home addresses, phone numbers, and biometric data.
Recertification	Periodic re-evaluation of a certified tool to confirm continued compliance. Frequency depends on certification tier.
Sealed Record	A court record that has been ordered sealed by a judge, restricting public access. Includes juvenile records, expunged criminal records, and certain civil proceedings.
Self-Evaluation	A vendor's own assessment of its tool against the BenchMark Standard, using published methodology and test cases, prior to formal certification submission.
Temperature	A parameter in AI text generation that controls output randomness. Lower temperature produces more consistent, deterministic outputs.
Test Case	A specific scenario, query, or prompt used to evaluate an AI tool against a BenchMark criterion, with a known-correct answer or expected behavior.
Transparency	The ability of an AI tool to explain its reasoning, cite its sources, disclose its limitations, and provide an auditable record of its operations.