Glossary term
Generative AI Risk
Subject: Technology, Cyber and Data, AI Governance and Risk
Also known as: GenAI risk
Generative AI risk is the set of harms from tools that create text, images, code or audio, including false output, data leakage, copyright problems, security abuse and over-reliance by users.
Explanation
Generative models predict plausible content rather than retrieve verified facts, so they can state falsehoods with confidence, a fault known as hallucination. Staff who paste confidential material into public tools may expose it, and some providers may reuse inputs for training. Outputs can copy protected work or reveal data present in the training set, creating intellectual property and privacy exposure. Attackers use the tools to write convincing phishing messages and deepfakes, and can manipulate a deployed model with crafted prompts, known as prompt injection. Bias in training data can surface in the text produced. Controls include approved-tool lists, usage policies, input filters, human review of outputs, contract terms with vendors and logging. Most early incidents come from staff pasting data into public tools, so a map of where the tools are already used is a sensible first control.
Example
A law firm associate uses a public chatbot to draft a court brief, and the tool invents three case citations. A partner catches them before filing. The firm then restricts use to an approved internal tool, bans client names in prompts, and requires a named lawyer to verify every citation and quotation in AI-assisted drafts.
Not to be confused with
- Model Risk
- Model risk covers wrong or misused quantitative models; generative AI adds content-creation harms such as invented facts and leaked prompts.