Policy-as-Code or Policy-as-PDF? Getting Real About AI Governance Before Audit Season

Bygaz

A dimly lit modern server room with glass walls, layered over with glowing abstract graph lines and policy diagrams, a lone engineer standing in the mid-ground studying a wall of structured data flows, cool blue and white lighting, wide cinematic composition, high contrast, no text

Why this matters this week

If you’re running production AI systems in any regulated-ish environment (B2B SaaS, fintech, health, infra, security), you’re now being asked some version of:

  • “Where does the model send data?”
  • “How long do you keep prompts and outputs?”
  • “Can we audit what the model saw and decided?”
  • “Is this SOC2 / ISO compliant or are you just saying ‘trust us’?”

What’s changed in the last few weeks:

  • Enterprise security reviews for AI are getting more specific:
    • “No training on our data”
    • “Must support regional data residency”
    • “Need prompt + output logging with redaction”
    • “Show me your policy, not your pitch deck”
  • Internal infosec teams are finally catching up and blocking:
    • Ad-hoc LLM usage with unclear retention
    • Shadow use of third‑party AI APIs in production paths
  • Auditors are starting to treat LLM usage like new systems of record, not just “a helper script.”

If you don’t have a credible story around privacy, data retention, model risk, and auditability, the near-term impact is not “existential AGI risk” — it’s:

  • Deals delayed or lost during security review
  • Internal freezes on AI features
  • Surprise remediation work just before SOC2 / ISO audits

This isn’t a governance deck problem. It’s a mechanism problem: you either have enforceable policy-as-code around AI, or you have vibes.

What’s actually changed (not the press release)

Three concrete shifts that matter for anyone operating AI in production:

1. Default vendor behavior is now a risk, not a feature

  • Many AI infrastructure and SaaS vendors:
    • Log prompts and outputs for “quality” by default
    • Retain logs for 30–180+ days
    • May use data for model improvement unless explicitly disabled
  • Six months ago, you could get away with “we’re using a reputable provider.”
    Now, security reviews ask:

    • “Is data used for training? Prove it.”
    • “What’s your retention period? Is it configurable via contract and API?”
    • “Where are logs stored? Which sub-processors see them?”

2. LLM usage is discoverable

Security teams are starting to:
– Scan codebases for:
– openai.ChatCompletion, anthropic_client.messages, “/v1/chat/completions”
– Homegrown “/llm” proxies with no auth/logging
– Compare those to:
– Vendor risk register
– Approved data flows
– Data classification policies

Shadow AI isn’t shadow for long. Once discovered, the default response is “turn it off until governed.”

3. SOC2 / ISO auditors have a mental model now

Early audits treated AI as “just another API.” That’s fading. Auditors now ask:

  • Access control: Who can send which data to LLMs?
  • Retention: How long do you keep prompts, outputs, and derived artifacts?
  • Change management: How do you approve and roll out new models / providers?
  • Monitoring: How do you detect and respond to AI-related incidents or data leakage?

They don’t care which model you use; they care whether your controls for AI match the controls for any other data-processing component.

How it works (simple mental model)

Use this mental model to structure AI governance: Data → Policy → Control Plane → Evidence

1. Data: what’s actually at risk?

Classify data before it hits any model:

  • P0: Regulated or highly sensitive
    • PCI, PHI, credentials, secrets, legal docs under privilege, prod customer data
  • P1: Internal but not regulated
    • Internal roadmaps, runbooks, internal incident data, source code
  • P2: Public or pseudo-public
    • Marketing copy, docs, public status pages

For each AI use case, answer: What’s the highest class of data that might realistically appear in prompts or context?

2. Policy: what’s allowed, where

Translate your existing infosec / privacy rules into AI-specific constraints:

Examples:

  • P0 data:
    • Only allowed with models running in your VPC or a dedicated tenant
    • No vendor-side training
    • Short retention (<= 7–30 days) or “no logging” options where possible
  • P1 data:
    • Allowed with vetted third-party providers in approved regions
    • Must go through a central proxy with redaction and logging
  • P2 data:
    • Fewer restrictions, but still logged and rate-limited to avoid abuse

This is where policy-as-code becomes real: codify this in configuration and enforcement points, not only in PDFs.

3. Control Plane: where enforcement lives

You want a single “AI control plane” in your architecture that does:

  • Model routing: Which provider / model handles which request
  • Policy enforcement:
    • Data redaction / classification
    • Allowed models per data class / team / environment
  • Observability:
    • Log prompts (optionally redacted), outputs, metadata (model, region, latency)
    • Trace back to user / system / use case

This can be:

  • An internal “LLM gateway” service you own
  • A managed policy-focused AI gateway from a vendor
  • A hybrid: your own thin proxy + vendor SDK for controls

4. Evidence: auditability and model risk

Finally, you need evidence that your AI governance is real:

  • Logs: Who sent what (or a redacted view), to which model, when, from which system
  • Policies: Versioned, reviewed, and mapped to controls (config, code, Terraform)
  • Tests: Automated checks that:
    • Disallowed models aren’t used in prod
    • Disallowed data classes can’t be sent without redaction
  • Incidents:
    • Runbooks for “prompt with secrets leaked to third-party LLM”
    • Documented responses and corrective actions

SOC2 / ISO alignment for AI is basically “prove you did this and keep doing it.”

Where teams get burned (failure modes + anti-patterns)

Failure mode 1: “Vendor says they’re compliant, so we’re good”

Reality:

  • Vendor SOC2 doesn’t mean:
    • They won’t use your data for training
    • They meet your data residency needs
    • Their default retention matches your policies
  • You’re responsible for:
    • The configuration you choose (e.g., enable/disable training, regions)
    • Your own logs, prompts, and embeddings stores

Anti-pattern: Treating vendor compliance as a blanket exemption from internal governance.

Failure mode 2: Logs as an afterthought

Common pattern:

  • Team builds an AI feature quickly, ships it
  • Logs:
    • Full prompts + outputs stored in a general logging system (e.g., app logs)
    • No redaction, no retention policy, wide access
  • Security review arrives:
    • Logs now contain P0/P1 data in a system never designed for that class

This quietly becomes your biggest privacy risk, not the LLM itself.

Failure mode 3: Policy-as-PDF

Symptoms:

  • There’s a “gen AI policy” doc approved by legal and security
  • Engineers:
    • Haven’t read it
    • Don’t know how to map it to code
  • No:
    • Linting in CI to prevent disallowed providers
    • Central gateway enforcing data class → model routing
    • Infrastructure-as-code representing model configurations

Result: You pass internal policy reviews on paper, but you fail actual enforcement.

Failure mode 4: Ignoring model risk as a change-management problem

Teams often:

  • Swap models (e.g., “Turbo → Latest”) via config flag
  • Change temperatures or system prompts globally
  • Add new providers during an outage

Without:

  • Approval workflow for “model version” as a change
  • Testing in a staging or shadow environment
  • Rollback mechanisms

This is change management, not just “tuning.”

Practical playbook (what to do in the next 7 days)

You won’t build a full AI governance program in a week, but you can de-risk the big items quickly.

Day 1–2: Inventory and classify

  1. Inventory AI usage

    • Grep / search for LLM-related code:
      • Known SDKs and endpoints
      • Internal “/llm” or “/ai” services
    • Group by:
      • Product feature
      • Internal tool
      • Environment (dev, staging, prod)

  2. Classify each use case

    • For each:
      • What’s the highest data class that can appear?
      • Which provider(s) and models are used?
      • Where are prompts and outputs logged?

Create a simple table; you’ll use it to prioritize.

Day 3–4: Draw a minimal AI control boundary

  1. Stand up (or designate) an AI gateway

    • Short-term: a single service / module through which all prod LLM calls must route
    • Responsibilities for now:
      • Enforce provider + model allowlist
      • Attach standard metadata (user id, feature flag, env)
      • Redirect logs to a specific sink

  2. Lock down direct-to-provider calls

    • CI rule or codeowner review requirement for:
      • Any new dependency on AI SDKs
    • Ideally:
      • Any new LLM call must go through the gateway

Day 5: Configure minimum viable privacy controls

  1. Align vendor configs with your policy

    • For each provider:
      • Turn off “use data for training” where possible
      • Set minimum viable retention (shorten if you can)
      • Restrict to required regions only
    • Document these choices (auditors will ask).

  2. Fix your logging

    • Ensure:
      • No prompts with P0 data end up in generic app logs
    • Implement:
      • Redaction for common patterns (emails, IDs, secrets) in gateway
      • A dedicated “AI interaction log” with:
        • Request ID, user ID, feature, model, region, timestamp
        • Optional partially redacted prompt/output

Day 6: Policy-as-code baseline

  1. Encode basic policy constraints

    • In code or config:
      • Map data classes → allowed model groups
      • E.g., class P0 -> models: [internal_vpc_model], P1 -> [vendor_A_enterprise]
    • Add:
      • A small library/helper that features can call with their declared data class
      • The gateway enforces routing based on this

  2. Add CI checks

    • Static checks for:
      • Disallowed SDKs or endpoints
      • Hard

Similar Posts