Policy-as-Code or Policy-as-PPT? Getting Real About AI Privacy & Governance

---

Why this matters this week

If you’re piloting or scaling AI in production, you’ve probably heard one or more of these in the last 7 days:

• “Legal says we can’t send PII to that model vendor.”
• “Security wants a DPIA / TIA / something-IA before we go live.”
• “Audit is asking how we know the model isn’t trained on customer data.”
• “Can we prove we deleted prompts after 30 days if a regulator asks?”

Underneath all of that is one core issue: you are running a new kind of data processing stack that doesn’t fit your existing controls cleanly.

Traditional privacy & governance was oriented around:
– Databases that you own
– Services where data flows are stable
– Code paths you can trace

Modern AI systems introduce:
– Black-box inference APIs (and sometimes training APIs)
– Ephemeral but sensitive prompt and context data
– Model behaviors that are hard to predict or explain
– Third-party vendors with unclear data retention / training policies

You’re still on the hook for:
– SOC2, ISO 27001/27701 alignment
– Privacy regs (GDPR, CCPA, LGPD, etc.)
– Internal policies (data minimization, retention, access)
– Regulatory expectations around model risk management and auditability

The risk isn’t just “regulators might be mad someday.” The nearer-term, more likely risks are:
– An incident you can’t easily investigate because logs are garbage
– A deal blocked because you can’t answer a client’s questionnaire
– A moratorium from your own risk committee because governance is hand-wavy

This post is about practical mechanisms for privacy & AI governance that won’t collapse the first time someone asks, “Show me.”

---

What’s actually changed (not the press release)

Three concrete shifts are making your old approaches brittle:

1. Your data perimeters got fuzzy overnight

   Before:

   • Data mostly flowed between services you controlled.
   • Third-party APIs handled narrow, structured payloads.

   Now with AI:

   • Prompts can contain anything (PII, secrets, source code, contractual data).
   • Vector databases and prompt caches hold high-value slices of your crown jewels.
   • “Temporary context” is actually persistent when combined with logs and traces.

2. Vendors’ default behaviors don’t match your policies

   Reality check:

   • Some model APIs log requests for abuse detection or quality.
   • “We don’t train on your data” might apply only to some tiers or SKUs.
   • Retention policies are often vague, conditional, or buried in separate docs.

   For many orgs, the effective data retention policy is whatever the vendor’s default is, plus whatever your engineers accidentally log.

3. Governance is lagging behind the implementation surface area

   Most orgs:

   • Updated a policy doc to add “AI” as a word.
   • Created a review committee or RACI matrix.
   • Maybe wrote a “generative AI acceptable use” PDF.

   Meanwhile, engineers:

   • Call third-party models directly from backend services.
   • Store embeddings without clear retention semantics.
   • Expose model outputs to users without consistent redaction or review.

The net-new thing: governance and privacy controls must now be expressed as code that intercepts and shapes model interactions, not just policy PDFs and checklists.

---

How it works (simple mental model)

Think of AI privacy & governance as three layers of policy-as-code:

1. Data Ingress Policy (what goes into models)
   Goal: Control and minimize what leaves your trust boundary.

   Core questions:

   • What types of data are allowed in prompts/contexts for this model?
   • Under what legal bases / contracts can that data leave?
   • Is the provider allowed to store, log, or train on it?

   Mechanisms:

   • Classification + redaction before prompt construction.
   • Allow/deny lists for data domains (e.g., no payment card data).
   • Service-level configuration choosing which model/vendor based on data sensitivity.

2. Model Execution Policy (how inference is run)
   Goal: Constrain behavior, log meaningfully, and support auditability.

   Core questions:

   • Which model / version is used for which use case?
   • What parameters (temperature, tools, system prompts) are allowed?
   • What is logged, and at what level of detail?

   Mechanisms:

   • A central “model gateway” or orchestration layer that all calls go through.
   • Policy hooks to:
     • Validate inputs and outputs
     • Mask before logging
     • Enforce per-tenant routing or isolation

3. Data Egress & Retention Policy (what stays, for how long, and where)
   Goal: Make sure you can answer “Show me where this data went and when it was deleted.”

   Core questions:

   • How long do we keep prompts, outputs, and embeddings?
   • Who can access which logs and traces?
   • Can we delete on demand for a user or tenant?

   Mechanisms:

   • Tagged storage for prompts, embeddings, and outputs with:
     • Tenant IDs
     • Data sensitivity levels
     • Retention policies
   • Automated deletion / TTL at the storage layer.
   • Indexing that supports “right to be forgotten” operations.

If you don’t centralize at least basic control in each layer, you get:
– Shadow flows (direct calls to vendor APIs).
– Inconsistent redaction.
– Unanswerable audit questions.

---

Where teams get burned (failure modes + anti-patterns)

A few patterns that show up repeatedly in production AI systems:

1. “Governance by Google Doc”

Pattern:
– A doc says “Don’t send PII to external LLMs.”
– No technical controls implement this.
– Teams interpret “PII” differently.

Failure mode:
– One team ships a feature that sends email bodies to a SaaS LLM for summarization.
– Those emails contain names, addresses, and IDs.
– An enterprise customer asks in a security review if any PII leaves the platform.
– The truthful answer is “we’re not sure” or “sometimes.”

Anti-pattern: Policy text with no enforcement point.

Fix: Every policy rule must map to:
– A check in code (pre-prompt classification + blocking/redaction), and
– A log that can be queried to prove it happened.

2. “Free-text to vector store” with no classification

Pattern:
– Product logs or user notes go straight to embedding + vector DB.
– No detection of PII, secrets, or regulated data.
– Vector DB is treated as “just another index.”

Failure mode:
– Security review discovers the vector store has:
– API keys, tokens
– Health info
– Payment conversations
– Access control around the vector DB is weaker than the underlying systems.

Anti-pattern: Assuming embeddings are magically non-sensitive.
Embedding stores are often the highest-value target in your AI stack.

Fix:
– Classify + filter before embedding.
– Tag and partition embeddings by sensitivity and tenant.
– Apply stricter retention and encryption on high-sensitivity embeddings.

3. Model selection ignoring data residency / training risk

Pattern:
– Devs use whichever hosted model gives the best results from a notebook.
– No distinction between:
– “No training on customer data” vs.
– “Data may be used for abuse detection / service improvement.”

Failure mode:
– EU customer demands evidence:
– Data stays in-region.
– Data is not reused outside their contract.
– Your architecture doesn’t separate:
– EU vs. non-EU traffic.
– “Training-allowed” vs. “training-prohibited” models.

Anti-pattern: Treating all LLM endpoints as interchangeable.

Fix:
– Explicitly model:
– Geography
– Training rights
– Retention defaults
– Route based on tenant + sensitivity, not developer convenience.

4. Logging everything “for observability”

Pattern:
– Model gateway logs full prompts and responses to your centralized log platform.
– Masking is an afterthought.

Failure mode:
– SIEM / log platform becomes an unbounded repository of:
– Personal data
– Proprietary documents
– Legal contracts
– Retention for logs (e.g., 1 year) collides with privacy policies (e.g., 30 days).

Anti-pattern: Using observability defaults for AI without scoping.

Fix:
– Capture signals, not full content, unless strictly needed.
– Store content-like data in a scoped, encrypted store with its own retention.
– Redact/partial-log by default; allow full logging only for tightly controlled debug modes.

---

Practical playbook (what to do in the next 7 days)

Assuming you already have some AI in lower environments or early prod, here’s a 7‑day, low-theory, high-leverage plan.

Day 1–2: Inventory and trace your AI data flows

1. Create a quick map of:

   • Which services call:
     • Hosted LLM APIs
     • Embeddings APIs
     • Internal fine-tuned models
   • Where prompts, outputs, and embeddings are stored.

2. For each flow, answer:

   • What types of data are in the inputs? (PII, secrets, contracts, health, etc.)
   • Which vendor / model is used, in which regions?
   • Where is logging happening? (App logs, model gateway logs, vendor logs)

Keep this rough; accuracy beats polish. You want enough to see hot spots.

Day 3: Define a minimal policy-as-code spec

You’re not building a full policy engine this week. Start with 5–10 simple rules that can be expressed in code:

• Inputs:

  • “No secrets (tokens, passwords, keys) in prompts.”
  • “No payment card numbers to external vendors.”
  • “For EU tenants, only route to EU-hosted models.”

• Logs:

  • “Full prompts and outputs are never logged to centralized logs in prod.”
  • “PII in prompts must be masked before logging.”

• Retention:

  • “Prompts with user identifiers: max 30 days.”
  • “Embeddings derived from PII: max 90 days or tied to account lifetime.”

Make them binary and testable.

Day 4–5: Implement a basic model gateway (or harden the one you have)

If you don’t already have a central layer through which all model calls flow, create a thin one:

• A service or library that:

  • Wraps vendor SDKs / HTTP calls.
  • Implements:
    • Input classification and simple redaction.
    • Routing (by tenant, region, sensitivity).
    • Controlled logging (structured, masked).

• At minimum, add:

  • A sensitivity flag on each call (e.g., public, internal, pii-lite, `