Your ML Model Is Not a Function, It’s a System (Start Treating It Like One)

Bygaz

A dimly lit control room filled with server racks and glowing dashboards, data streams visualized as colored lines flowing between machines, a large wall display showing drifting distributions and alert graphs, cinematic side lighting with a cool blue and orange palette, wide-angle composition emphasizing complex interconnected systems

Why this matters this week

Most teams now have at least one machine learning model in production doing something that actually matters:

  • Ranking content or search results
  • Routing tickets or leads
  • Approving/declining low-risk transactions
  • Extracting structure from documents
  • Powering “smart” product features

What’s changed in the last year isn’t that “AI is here” — it’s that:

  1. Inference is cheap enough that you can afford to run more continuous evaluation and monitoring instead of “train once, hope forever.”
  2. Tooling stopped being just notebooks and dashboards and started exposing primitives for monitoring, data drift, and feature pipelines that a real SRE team can hook into.
  3. Leadership expectations shifted: “We launched the model” is no longer success. The question is “What’s the lift? How stable is it? What happens when the data moves?”

Teams that don’t adjust to this new baseline are hitting the same pattern:

  • Month 1–2: Model looks good vs. test set. Everyone’s happy.
  • Month 3–6: Quiet degradation. More tickets. Subtle revenue leakage. False positives/negatives climb.
  • Month 6+: Panic re-train or rollback, trust in “AI” declines, product teams resist future ML bets.

This post is about turning that pattern into an engineering discipline: treat ML as a living system with explicit evaluation, monitoring, drift handling, and cost/perf trade-offs — not a one-off project.

What’s actually changed (not the press release)

A few concrete shifts that matter for people shipping production systems:

  1. Evaluation is moving online, not just offline.
    Instead of only AUC on a held-out set, teams are:

    • Logging per-request model inputs, outputs, and downstream decisions.
    • Measuring business metrics per model version (conversion, latency, cost).
    • Running shadow/ghost deployments for new models to get unbiased, live comparisons.

  2. Feature pipelines are being treated like APIs, not ETL scripts.
    The better setups now:

    • Have versioned feature definitions (code, not queries pasted in docs).
    • Share transformations between training and serving (no re-implementing featurization in prod).
    • Track data contracts between upstream systems and ML (schemas, expectations).

  3. Drift is an SLO issue, not just a data science curiosity.
    Production teams:

    • Monitor input distributions (PSI, KL divergence, population stability index; pick your poison).
    • Monitor output distributions and label delays.
    • Set alerts when drift crosses thresholds that historically preceded performance drops.

  4. Inference cost is a first-class metric.
    With larger models and GPU-heavy inference:

    • Teams are budgeting in $ per 1k predictions, not just infra line items.
    • They’re using tiered models (fast baseline + expensive re-ranker) and caching.
    • Cost is tracked per feature flag / user segment, not just globally.

  5. Tooling is less bespoke.
    You can now:

    • Hook ML monitoring into your existing logging, metrics, and incident pipelines.
    • Use managed feature stores instead of hand-rolled “feature tables + cron jobs”.
    • Instrument evaluation with existing AB testing infrastructure.

Under the marketing veneer, the real change is simple: the cost of doing ML ops properly has dropped enough that it’s now cheaper than repeatedly eating the cost of silent model failure.

How it works (simple mental model)

A practical mental model:

Treat your ML deployment as four interconnected subsystems: Features → Model → Policies → Feedback. You monitor and evolve each explicitly.

  1. Features (data in)

    • What raw signals you use
    • How they’re transformed
    • Where they’re computed (offline vs online)
    • Their freshness and reliability

  2. Model (mapping)

    • The trained artifact (classic ML, deep learning, or LLM)
    • Its hyperparameters and architecture
    • The inference path and latency/cost characteristics

  3. Policies (decisions)

    • How the model’s output is actually used:
      • Thresholds
      • Business rules
      • Safety checks
      • Fallbacks and overrides

  4. Feedback (ground truth & metrics)

    • Labels (fast/slow, direct/indirect)
    • Business KPIs
    • Human corrections
    • Incident reports

You then layer three cross-cutting concerns:

  • Evaluation – How good is this behavior?
    • Offline: test/validation sets, backtests
    • Online: AB tests, pre/post analysis, contextual bandits
  • Monitoring – Is it behaving as expected?
    • Data distributions, latency, error rates, business metrics
    • Per-segment (geo, device, user type, traffic source)
  • Control – What do we do when it’s not?
    • Rollbacks, traffic shifts, canaries
    • Automated retraining or human-in-the-loop review
    • Escalation paths like any other critical service

If you’re missing any one of these, your ML system is fragile.

Example: A B2B SaaS company had a lead scoring model:

  • Strong offline lift in recall/precision.
  • No feature monitoring, no policy layer, no defined feedback loop.
  • Sales started ignoring low-scored leads after a few bad experiences from a new marketing channel.
  • The “model” didn’t fail — the system did. Features drifted, policies were static, feedback was purely social.

Where teams get burned (failure modes + anti-patterns)

Common patterns that keep showing up:

1. “One test set to rule them all”

Anti-pattern:
– Single offline test set, usually from a calm historical period.
– No slicing by segment, time, or traffic source.
– Model approved based on a single metric lift.

How it burns you:
– Model looks great overall but fails for a growing segment (e.g., new country, new device).
– By the time support tickets show the pattern, trust is already damaged.

Mitigation:
– Always evaluate by time window and key segments.
– Treat “last 30–60 days” as separate from older data.
– Make per-segment performance part of your launch gate.

2. Training/serving skew via invisible feature changes

Anti-pattern:
– Feature calculation logic duplicated: one version in notebooks, another in production.
– Upstream systems subtly change definitions (e.g., “active user” changes from 7d to 30d).

How it burns you:
– Offline evaluation no longer matches reality.
– Model drifts for reasons that look like “magic” until you dig into ETL or API changes.

Mitigation:
– Single source of truth for features (versioned code + tests).
– Schema/contract checks with CI that fail deployments on breaking changes.
– Regular train vs serve feature distribution comparisons.

3. No strategy for label delay

Anti-pattern:
– Objective depends on slow feedback (e.g., 30-day churn, 90-day repayment).
– Model updated frequently using only the fastest proxies.
– Offline evaluation lags reality by weeks or months.

How it burns you:
– Overfitting to short-term proxies.
– You don’t notice performance regressions until full-label windows catch up.

Mitigation:
– Separate fast proxies from true label metrics in monitoring.
– Use backtesting over time, not just single snapshots.
– Pin model versions for long enough to get full-label performance before deprecating their data.

4. Treating cost as a footnote

Anti-pattern:
– Move from small models to larger models (or from classic ML to LLMs) without budget gates.
– No per-request cost instrumentation; infra bill just “goes up.”

How it burns you:
– Margins erode.
– You can’t run proper AB tests or shadow models because they’re “too expensive.”

Mitigation:
– Track $ per 1k predictions as a first-class metric.
– Enforce latency and cost budgets per endpoint.
– Use strategies like:
– Cascaded models (cheap filter, expensive re-ranker)
– Caching popular or repeat queries
– Lower-precision inference where acceptable

5. Conflating “more complex” with “better”

Real example pattern:
– Classic gradient boosted trees model yields solid lift and low latency.
– Team rebuilds problem with a big neural net or general-purpose LLM to “future-proof.”
– Gains are marginal; operational complexity and costs multiply.

Anti-pattern:
– Choosing model families/tools for prestige or resume value, not for SLOs.

Mitigation:
– Maintain a simple baseline model in production as a control.
– Only adopt complexity when you can measure:
– X% improvement in key metric
– Within acceptable latency/cost bounds
– Re-evaluate complexity yearly; sometimes the baseline catches up.

Practical playbook (what to do in the next 7 days)

You don’t need a full platform rewrite. You can get meaningful leverage in a week.

Day 1–2: Instrument what you already have

  1. Log the essentials per prediction (if you’re not already):

    • Model version
    • Key input features (or hashed/bucketed versions where sensitive)
    • Prediction output (score + decision)
    • Latency
    • User/request segment tags (geo, device, channel)

  2. Add cost and latency metrics:

    • Per-endpoint P50/P95 latency
    • Estimated cost per request / per 1k requests

Day 3: Define minimal “health” metrics

For each production model, define:

  • 1–2 business metrics it’s supposed to move (e.g., acceptance rate, click-through rate, manual review rate).
  • 1 core quality metric you can compute regularly (even if noisy) — e.g., calibration, error rate on sampled labeled data.
  • Segment dimensions that matter (e.g., new vs. returning users, traffic source, region).

Wire these into your existing dashboards.

Day 4: Basic drift checks

Implement simple drift detection before getting fancy:

  • Track mean, std, and a few quantiles for 5–10 most important features.
  • Track prediction score distribution (min/median/max, histogram buckets).
  • Compare to a moving baseline (e.g., last 7 days vs prior 30 days).
  • Page someone only if drift correlates with drops in core metrics, or clearly surpasses historical ranges.

If you want one number: PSI (population stability index) is crude but widely used; what matters more is trend + correlation with performance, not theoretical purity.

Day 5: Document your policies

For each model:

  • Map where its output

Similar Posts