“We’re already on AWS”
A few months back I was on a call with the head of data at a Pacific Northwest community bank — call it ~$2.8B in assets, four-person analytics team, no dedicated ML platform engineers. They were in the middle of an internal debate: should they build their next credit underwriting model on SageMaker themselves, or buy Consilience?
Her CTO had a strong opinion. “We’re on AWS. We have Clarify, Model Cards, Model Monitor, Autopilot. Why pay a vendor to assemble what we already have?”
He’s right that every primitive Consilience uses is available on SageMaker. The question is what it actually costs to assemble them — and what you give up by stopping at the primitives. That’s the conversation I want to walk through, because I have it almost every week.
One caveat up front. The engineer-week numbers in this post come from our prospect conversations, our own benchmarking, and corroborating industry sources (Cornerstone Advisors’ annual banking technology benchmarks, McKinsey’s published ML build studies, AWS’s own reference architecture timelines). Your numbers will vary depending on team experience, existing infrastructure, and how strict your MRM function is. The directional shape — that stages 4-7 take 2-3x what teams budget for them — is consistent across the dozen-plus build-vs-buy conversations I’ve had this year.
What SageMaker primitives don’t do
Before the cost teardown, the deeper question hidden inside the build-vs-buy debate.
SageMaker has Autopilot. It automates the parts of model training that are easiest to automate — algorithm selection, hyperparameter search, basic feature processing on tabular data. What Autopilot does not do is the part that actually moves AUC: a dual-loop search where you iterate on features against residual errors AND on the prediction target itself, building out dozens of candidate models with different targets, time horizons, and sub-population segmentations, and surfacing the one that fits your portfolio.
That search is the difference between a baseline auto-ML model and a good one. It’s also the work every senior credit ML team would do if they had unlimited engineer-weeks — and that no team has the bandwidth to do by hand on every refresh.
Consilience runs the search automatically, on a measured compute budget, every refresh. Teams building in-house on SageMaker either run a thin version of it manually — multi-week effort each cycle, on the features only, rarely on the target — or skip it entirely. That’s where most of the AUC delta between built-it-yourself and bought-Consilience actually lives, and it’s the part the engineer-week table below doesn’t show.
For reference: Credit Key, a buy-now-pay-later lender, came to us with a fraud problem. The factory built dozens of candidate models, varying the target, the time horizon, and the segmentation, and surfaced an optimal fraud model 9% higher AUC than their incumbent. That kind of lift is what’s structurally available to teams running the search and what’s structurally left on the table for teams that don’t.
The cloud-cost question, addressed up front
Every CTO asks this and most build-vs-buy comparisons dodge it.
Consilience runs on your AWS account and uses your existing compute. There’s no separate vendor environment to pay for. The AWS bill differential between an in-house SageMaker build and Consilience is roughly zero — both consume training and inference compute on the same instances in your account. What changes is the labor budget, not the cloud bill.
The same goes for Bedrock, Amazon Q, and the rest of your generative AI stack. Consilience runs alongside them; we don’t replace your generative AI investment. We focus on credit, fraud, pricing, and loss models specifically — the workloads where the dual-loop search pays off.
The honest build cost, in engineer-weeks
Here’s the teardown I sent the Pacific Northwest CTO after the call. Engineer-weeks because that’s the scarce resource.
| SageMaker (build yourself) | Consilience | |
|---|---|---|
| 1. Data ingestion + nested JSON parsing | 2–4 weeks; ongoing maintenance as bureau schemas drift | Included; nested JSON and variable bureau schemas parsed natively |
| 2. FS feature engineering library + dual-loop search | 4–8 weeks for the first library. The dual-loop search (features against residuals, targets against portfolio fit) requires senior DS time per refresh and is the work most teams quietly skip when timelines slip. | Pre-built FS feature library plus the dual-loop search running automatically per build |
| 3. Model training + tuning | 2–3 weeks per model | Included — iterative optimization runs automatically |
| 4. Adverse-action reason codes (ECOA-compliant) | 1–2 weeks to build reason-code mapping on top of Clarify SHAP | Included as a build output |
| 5. SR 11-7 documentation workflow | 3–6 weeks first model; recurring per refresh. Where most builds stall. | Auto-generated per build |
| 6. Fair-lending / disparate-impact pipeline | 2–4 weeks; bespoke to your fair-lending policy | Included |
| 7. Model registry + versioned feature lineage | 1–3 weeks wiring Model Cards / Registry into your MRM workflow | Included |
| Total — first model end-to-end | 15–30 engineer-weeks before a validated model exists | 3 days to validation-ready |
| Refresh (per cycle, per model) | 4–8 weeks per refresh | <1 week per refresh |
Where SageMaker actually wins
Three places where SageMaker is the right answer and Consilience isn’t.
| Consilience | SageMaker (wins here) | |
|---|---|---|
| Non-credit ML use cases | Out of scope — Consilience is purpose-built for credit, fraud, pricing, and loss models | Marketing mix, ops forecasting, treasury analytics — SageMaker amortizes across business units |
| AWS-native AI surface | We don’t replace your generative AI stack | Native integration with Bedrock, Amazon Q, Comprehend, agentic workflows |
| Training infrastructure at massive scale | Not the strength — we’re packaged for credit/fraud workloads | Hundreds of millions of records with custom architectures, especially with managed spot training |
We don’t compete on any of these and shouldn’t pretend to.
Where in-house builds actually stall
Most CTOs who plan a SageMaker build estimate stages 1-3 well and underestimate stages 4-7 by roughly 3x.
The wall almost everyone hits is stage 5 — SR 11-7 documentation. Clarify gives you SHAP outputs. Model Cards give you a template. Model Monitor gives you drift signals. Stitching them into a package your model risk team will sign off on is the work, and it’s the work that gets cut when the project is two months behind schedule. The model is “done.” The documentation isn’t. Production is blocked.
The other wall is maintenance. Every quarter, bureau schemas change, a transaction feed format changes, an AWS service version changes, an upstream data team deprecates a column. In a managed factory, that’s the vendor’s problem. In an in-house build, it’s a recurring tax on the team that built it — and that team has long since moved to the next priority.
The qualifier that lets you self-select
The teams I’ve seen succeed at an in-house SageMaker build have a dedicated 6+ person ML platform org and a multi-quarter runway. If that’s you, build. The lock-in is low, your team will learn things they couldn’t learn buying, and you’ll end up with infrastructure that compounds across non-credit use cases.
If that’s not you — if you have a 2-4 person analytics team and a credit/fraud model roadmap that needs to ship in months, not quarters — buy. The build-vs-buy math doesn’t favor in-house at smaller team sizes.
That’s the honest split. If you read the teardown above and your reaction was “we don’t have six platform engineers,” you have your answer.
Where Consilience is the better choice
You don’t have a 4-8 person ML platform team and don’t want four months to build one.
The factory replaces the platform engineering work, not the modeling judgment. Your team operates the factory rather than rebuilding it from primitives.
See: Build-vs-buy worksheetSR 11-7, ECOA, and fair-lending docs are an output, not a project.
Auto-generated per build, reproducible from the recorded training script. Stage 5 — where most in-house builds stall — disappears.
See: Sample SR 11-7 docRefresh velocity stays high in year 2 and year 3.
The dual-loop search runs the same way on every refresh. The factory is the same in year 3 as it was on day one — no compounding maintenance debt.
See: Refresh-velocity benchmarkBuilding on SageMaker may still be the right call if ML infrastructure is a strategic in-house capability you want to own end-to-end, you’ve already shipped multiple production ML systems on AWS-native tooling, and your team includes a dedicated ML platform org of 6+ engineers. Most lenders don’t have all three.
FAQ
Does Consilience use SageMaker under the hood?
+
We use AWS services where it makes sense — including parts of the SageMaker stack for training infrastructure — and bypass them where it doesn’t. The dual-loop search, the audit-trail generation, the SR 11-7 documentation workflow, and the fair-lending pipeline are not SageMaker capabilities; they’re Consilience capabilities running on your AWS account. From your AWS billing perspective, the compute load is largely consistent with what you’d see from a manual SageMaker build.
Can we start on SageMaker and migrate to Consilience later, or vice versa?
+
Yes, both directions. SageMaker-to-Consilience migration is common — most of our customers had a partial in-house build before they switched. Consilience-to-SageMaker is also supported because model artifacts, feature engineering code, and training scripts are yours. The lock-in is minimal in either direction; the migration cost is mostly your team’s familiarity with the new workflow.
What’s the realistic engineer-week cost of building an audit-ready credit model in-house?
+
15-30 engineer-weeks for the first model, depending on team experience and prior infrastructure. Stages 1-3 (data ingestion, feature engineering, training) take 8-15 weeks; stages 4-7 (reason codes, SR 11-7 docs, fair-lending pipeline, model registry) take 7-15 more. Refresh cost is 4-8 engineer-weeks per cycle. The interactive worksheet gives an institution-specific estimate.
Where do most in-house SageMaker builds actually stall?
+
Stage 5 — SR 11-7 documentation. Clarify gives you SHAP outputs. Model Cards give you a template. Model Monitor gives you drift signals. Stitching them into a package your model risk team will sign off on is the work that gets cut when the project is two months behind schedule. The model is “done”; the documentation isn’t; production is blocked. We see this pattern across most in-house builds we evaluate.
How does refresh velocity compare in year 2 and year 3?
+
In year 1, build cost dominates. In year 2-3, refresh and maintenance costs dominate. An in-house SageMaker build typically incurs 50-150 engineer-weeks per year on refresh and maintenance across a 4-model portfolio. Consilience runs the refresh loop on the same schedule with operator hours only — the factory is the same in year 3 as it was on day one.
Does Consilience replace our SageMaker investment, or sit on top of it?
+
Sit on top. Your SageMaker infrastructure (Bedrock, training instances, monitoring) keeps running. Consilience uses what it needs and replaces only the credit / fraud / pricing model-building workflow. Your AWS bill stays roughly the same; what changes is the engineering labor budget. We don’t replace your generative AI stack — we focus on the workloads where the dual-loop search pays off.