How to Build a Data Classification Policy for AI Systems | Polygraf AI

How to Build a Data
Classification Policy for
AI Systems

Your classification policy was written for a world where data sits still in a labeled file. AI shreds that assumption: it chunks documents, strips metadata, converts text to vectors, and synthesizes answers from a dozen sources at once. Research shows 92% of short text can be reconstructed from its "anonymous" embedding. Here's how to write a policy that survives the pipeline.

92%
of 32-token text inputs were recovered exactly from their embeddings alone — vectors aren't anonymized
A.5.12
ISO 27001's classification control — the foundation access control, DLP, and crypto all sit on top of
3 × 3
FIPS 199's model: three impact levels across confidentiality, integrity, availability — plus high-water mark
4
ways AI breaks a conventional classification scheme — transformation, aggregation, derivation, drift
Polygraf analysis

Almost every enterprise already has a data classification policy. It's usually four tiers — Public, Internal, Confidential, Restricted — with handling rules for each, an owner assigned per data set, and a labeling scheme. It was written to satisfy ISO 27001's Annex A 5.12, or FIPS 199, or a customer's security questionnaire. It works reasonably well for a world of files, folders, and databases.

Then AI arrives, and the policy quietly stops describing reality. Not because the tiers are wrong — they're fine — but because every assumption underneath them was built for data that stays where you put it. A classification policy assumes a document has a label, that the label travels with it, and that protecting the document protects the information. An AI pipeline breaks all three: it splits the document into chunks that inherit nothing, converts them into vectors that carry no metadata, retrieves fragments from a dozen sources, and synthesizes an answer that has no label at all because it never existed until the moment it was generated. This guide is about writing the policy that survives that — the tiers, the AI-specific handling rules, the inheritance logic almost nobody documents, and where enforcement has to live.

Free Tool · Polygraf AI Risk Calculator

Find out what unclassified data flowing into AI is costing you

Classification exists to size protection to risk — which means you need the risk number first. Polygraf's AI Risk Calculator models your exposure across breach, regulatory, litigation, and reputational risk, and maps which obligations apply to the data your AI systems touch.

  • Quantified exposure across every major risk category
  • A tailored read on which regimes govern your data — GDPR, HIPAA, PCI DSS, GLBA
  • Gaps surfaced across discovery, classification, and enforcement
  • Modeled reduction from adding inline detection at the AI boundary
Run the free AI Risk Assessment →
No signup required · Results in minutes · ai-risk-calculator.polygraf.ai
Sample result
Total Potential Exposure
$49.8M
Data breach
Regulatory
Litigation
Reputational

Start From the Standards — They Still Hold

Before covering what breaks, it's worth being clear about what doesn't. The two foundational standards remain correct; they just don't reach far enough.

ISO/IEC 27001 Annex A 5.12 requires that information be classified according to the organization's information security needs, based on confidentiality, integrity, availability, and relevant interested-party requirements. Annex A 5.13 covers labelling — the visible markers that communicate the classification. The relationship matters: 5.12 is the decision, 5.13 is the marking, and the decision must come first. Classification is the foundation that access control (5.15), data leakage prevention (8.12), cryptography (8.24), and secure disposal (7.14) all sit on top of. Get the scheme wrong and everything downstream inherits the confusion. Notably, ISO doesn't mandate a number of tiers — three or four is the practical norm.

NIST FIPS 199 takes a different cut: rather than sensitivity tiers, it categorizes information by potential impact — Low, Moderate, or High — assessed independently across each leg of the CIA triad. Its most useful export to private-sector policy is the high-water mark principle: a system's overall categorization takes the highest impact level across all objectives and all information types it handles. A system that's Low for confidentiality but Moderate for integrity is Moderate overall.

Why the High-Water Mark Is the Key Idea for AI

Hold onto that principle, because it's the single most transferable concept in this entire article. An AI system doesn't handle one information type — it handles everything it can retrieve. Under a high-water mark reading, a RAG assistant with access to one Restricted document is a Restricted system, no matter how much of its corpus is public. Most organizations classify their AI systems by what they were designed for. FIPS 199 logic says classify by the most sensitive thing they can reach.

The Four Ways AI Breaks Your Policy

Conventional classification rests on assumptions that an AI pipeline systematically violates. Each break is specific and each needs a policy answer.

BREAK 01
Transformation strips the label, not the sensitivity

A labeled document gets chunked and embedded. The label lived in metadata, a header, or a file property — none of which survive into a float array in a vector store. The sensitivity is fully intact. The marking is gone. Your DLP tool, which keys on labels, now sees nothing to protect.

BREAK 02
Aggregation creates sensitivity that didn't exist

AI doesn't read one document at a time. It retrieves fragments from many sources and synthesizes them, surfacing relationships that file-level access control was never designed to prevent. Three Internal facts can compose into a Confidential conclusion that exists in no source document — so nothing was ever labeled to protect it.

BREAK 03
Derived data has no parent to inherit from

A model output is new data. A summary of a Restricted file, an inference about a customer, a vector index — none of these existed when your policy was written, and none inherits a classification automatically. Most policies are silent here, which in practice means derived data defaults to unclassified.

BREAK 04
Labels are static; AI data is in motion

Classification at creation says nothing about the fifty places a chunk has since been copied — vector stores, caches, prompt logs, fine-tuning sets, agent scratchpads. A policy that only labels at creation has no mechanism for reclassification or inheritance as data moves. And AI moves it constantly.

Interactive · Label survival tracker

Follow one document through the pipeline

Watch what happens to a classification label at each stage. Click through the stages.
📄 Q3_Layoff_Plan.docx — contains employee names, severance figures, and unannounced restructuring dates. RESTRICTED
📄
Created & labeled
✂️
Chunked
🔢
Embedded
🗄️
Indexed
🔎
Retrieved
💬
Synthesized
The Myth That Vectors Are Anonymized

The most consequential misconception in AI data governance is that embeddings are safe because they're "just numbers." Peer-reviewed research is unambiguous that they aren't. In Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023), Morris and colleagues framed embedding inversion as controlled generation and showed that an iterative method which repeatedly corrects and re-embeds its guesses recovered 92% of 32-token text inputs exactly from the embedding alone. The same work demonstrated recovery of full names from a dataset of clinical notes.

The policy implication is blunt: a vector store containing embeddings of Restricted documents is a Restricted data store. Not a derived artifact, not a de-identified index — the same classification as the source. If your policy doesn't say that explicitly, your teams will assume otherwise, because "it's just a bunch of floats" is an extremely intuitive and completely wrong mental model.

The aggregation problem — three Internal facts, one Confidential conclusion
Every input is individually approved for this user. The output is not. Facilities: "Building C lease not renewed after Q4" INTERNAL HR: "No new reqs approved for the Ops org" INTERNAL Finance: "Q4 severance accrual increased 340%" INTERNAL RAG synthesis "Any Ops changes?" "Ops is likely being restructured in Q4 with significant layoffs." Exists in no source document. Has no label. RESTRICTED File-level access control cannot prevent this: it never sees a violation. Each retrieval was authorized.

The Tier Table, Rewritten for AI

Keep your four tiers. What changes is that each tier now needs an explicit AI handling rule — because "Confidential" told your staff how to email a file, and told them nothing about whether they can paste it into a chatbot.

TierWhat it coversConventional ruleAI handling rule (the new column)
PUBLIC Marketing material, published docs, public filings Free distribution Any approved AI tool. No restriction. Still logged for inventory.
INTERNAL Policies, org charts, non-sensitive operational data Employees only; basic access control Enterprise AI only — sanctioned tools under a no-training contract. Never consumer AI.
CONFIDENTIAL Customer PII, contracts, financials, source code Role-based access; encryption Governed AI + inline inspection. Must pass detection/redaction at the boundary. Vector stores inherit this tier.
RESTRICTED PHI, cardholder data, MNPI, secrets, crown-jewel IP Need-to-know; MFA; full audit On-prem / air-gapped AI only, or blocked. Never leaves the environment. Every interaction logged immutably.
Don't Add a Fifth Tier for "AI Data"

A tempting mistake: creating a separate classification track for AI. Resist it. Six or seven tiers create decision paralysis, and a parallel scheme guarantees the two drift apart until nobody knows which applies. Sensitivity is a property of the information, not of the system touching it — a Social Security number isn't less sensitive because it's in a prompt instead of a database. Keep one scheme; add an AI handling rule to each tier. One vocabulary, one owner, one source of truth.

The Part Nobody Writes Down: Inheritance Rules

This is where almost every policy goes silent, and it's the most important section you'll write. When data is transformed, what classification does the result carry? Without an explicit rule, the practical default is "none," which is how Restricted content ends up in an unlabeled vector index. Here's a defensible starting matrix.

Classification inheritance — what the output carries
TransformationRuleRationale
Chunking INHERIT Every chunk carries the source document's tier. A fragment of a Restricted file is Restricted — splitting isn't sanitizing.
Embedding INHERIT Vectors are recoverable — up to 92% exact reconstruction for short text. The embedding is the data, in a different coordinate system.
Indexing / vector store HIGH-WATER The store takes the highest tier of anything in it. One Restricted document makes the whole index Restricted unless partitioning is enforced and proven.
Summarization INHERIT A summary of Restricted content is Restricted. Compression isn't declassification — the sensitive facts are usually the ones that survive.
Multi-source synthesis HIGH-WATER
OR ABOVE
Takes the highest tier among inputs — and may exceed all of them. Aggregation can create sensitivity none of the sources had. This is the one that needs human judgment.
Model output / inference INHERIT Derived data inherits from everything in its context window, including the retrieved documents and the prompt — not just the user's question.
Redaction / tokenization DOWNGRADE The only reliable downgrade path — and only if verified. This is why redaction at the boundary matters: it's the one transformation that legitimately lowers a tier.
Aggregate statistics CASE-BY-CASE May downgrade if genuinely non-reidentifiable — but small cohorts re-identify easily. Requires documented analysis, not an assumption.

"Classification isn't a label you attach to a file. It's a claim about information — and information doesn't stop being sensitive because you turned it into a float array, split it into chunks, or asked a model to summarize it. The only transformation that legitimately declassifies data is the one that actually removes the sensitive content."

— Polygraf AI, on classification for AI systems

Building the Policy: Seven Steps

1
Inventory data and the AI systems that reach it
Classic step one, with a new second half. List every data store — structured and unstructured, including code repos, support transcripts, backups — then map which AI systems, copilots, and agents can reach each one. That mapping is what turns a classification scheme into an AI control.
2
Define tiers using terms your staff already use
Four tiers, plain names, no jargon. Anchor each to impact: what happens if this is disclosed, altered, or unavailable? Borrow FIPS 199's discipline of assessing confidentiality, integrity, and availability separately — a config database with no personal data can still be Restricted on availability grounds.
3
Add the AI handling rule to every tier
For each tier, state explicitly which AI systems may process it: consumer tools, sanctioned enterprise AI, governed AI with inspection, or on-prem only. This is the single highest-value paragraph in the document — it's the one your staff will actually need on a Tuesday afternoon with a deadline.
4
Write the inheritance rules down
Use the matrix above as a starting point and adapt it. Specify what chunks, embeddings, indexes, summaries, and outputs inherit — and name redaction as the only routine downgrade path. Unwritten inheritance defaults to unclassified, which is how sensitive data ends up in unlabeled stores.
5
Assign owners from the business, not just IT
The information owner decides the tier; security facilitates and arbitrates. The head of HR knows what employee data is Restricted; the security team doesn't. Policies that hand all custodianship to IT fail because IT lacks the context to classify correctly — and a scheme nobody owns is a scheme nobody maintains.
6
Connect labels to automated enforcement
A classification policy not wired to enforcement is documentation, not security. Each tier's AI handling rule must map to a technical control that can actually stop a violation at the moment it's attempted — not a training slide asking people to remember the rule.
7
Review on a cycle and on change
Review classifications at least annually, and whenever an asset's value or the risk landscape shifts materially. For AI specifically, add triggers: a new data source connected, an agent granted new permissions, or a new tool adopted should all force a re-look.
Five Failure Modes We See Repeatedly
  • Over-reliance on manual labeling. Policies that depend on employees to label every file they create fail quickly at scale — and AI generates artifacts faster than any human labels them.
  • Too many tiers. Six or seven creates decision paralysis; fewer than three is too coarse to be useful. Four covers the range.
  • Classification that stops at creation. A Confidential spreadsheet pasted into an AI tool changes form but not sensitivity. Label-at-creation schemes have no mechanism to track it.
  • No enforcement connection. If tiers don't map to DLP, access control, encryption, and AI inspection, you have a taxonomy, not a control.
  • Silence on derived data. The gap that matters most for AI, and the one almost every policy leaves open.

Why Enforcement Has to Live at the Boundary

There's a structural reason classification for AI can't be solved with labels alone. Labels are metadata attached to containers — files, records, stores. AI operates on content, extracted from those containers and recombined. By the time an employee is pasting a paragraph into a prompt, the paragraph has no container and therefore no label. The only place left to make a classification decision is at the moment the content moves, by inspecting the content itself.

That's the shift: from labeling data at rest to classifying content in motion. The policy still defines the tiers and the rules — that work is essential and it's what makes enforcement legible to an auditor. But the enforcement point has to sit where the content actually crosses the line, because that's the last place the sensitivity is visible and the decision still matters.

Where Polygraf Fits

Polygraf AI's Behavioral Control Plane is the enforcement layer for exactly this problem. It sits inline at the AI boundary and classifies content in motion — detecting PII, PHI, cardholder data, secrets, and source code in the actual text of a prompt, regardless of whether the source file was ever labeled, and regardless of which container it came from. That's what makes it work where label-based DLP doesn't: it reads the content, not the metadata.

From there it enforces the tier rules your policy defines: allow, redact, or block, per data type and per destination. Because redaction is the one transformation that legitimately downgrades a classification, this is also the control that makes your inheritance matrix real rather than aspirational — Confidential content becomes safe to send only because something actually removed the sensitive part and can prove it. Every decision is logged immutably, giving you the evidence that the policy was enforced rather than merely published. And because it runs on-premise with zero data egress at sub-100ms latency, it's the control that makes the Restricted tier's "on-prem only" rule enforceable instead of theoretical.

Educational overview, not legal or compliance advice. This article is prepared by Polygraf AI for general information. The tier definitions and inheritance matrix presented here are an illustrative starting framework, not a standard — classification schemes must be adapted to your own data, jurisdiction, sector, and regulatory obligations. ISO/IEC 27001 and FIPS 199 references are summaries; consult the official texts. The 92% embedding-recovery figure reflects the specific experimental conditions in Morris et al. (EMNLP 2023) — 32-token inputs against particular embedding models — and should not be read as a universal recovery rate.
Polygraf AI

Classify Content in Motion, Not Just Files at Rest

Polygraf AI detects sensitive content inside every AI prompt and output — no label required — then enforces your tier rules inline: allow, redact, or block. The control that makes a classification policy real. On-premise, sub-100ms, zero data egress.

Request a Demo →
ISO 27001 · HIPAA · PCI · SOC 2
Deploys in under an hour

NEWS & More

Insights & Updates from Polygraf.

News

Polygraf AI announced a Memorandum of Understanding with the Innovation and Digital Development Agency (IDDA) of the Republic of Azerbaijan to establish a regional Artificial Intelligence Center of Excellence in

To learn more about Polygraf, please get in touch.

At Polygraf, we envision a future where AI augments human capabilities without compromising safety, privacy, or ethical standards. Trust in our commitment to building this future with you.

Products

thank you

Your download will start now.

Thank you!

Please provide information below and
we will send you a link to download the white paper.