How every analytics role changed in eighteen months

The headline everybody expected was replacement. What happened instead was a shift in which part of each job is scarce.

In almost every analytics role, the mechanical part — writing the query, scaffolding the model, drafting the documentation — got dramatically cheaper. The part that decides whether the output is any good did not get cheaper at all. It got more valuable, and it moved earlier in the process.

This is what that looks like job by job, based on what we see inside lending and fintech teams rather than what the vendor decks say.

Analytics engineers quietly became the most important people in the building#

Three years ago the analytics engineer was a relatively new title that mostly meant "writes dbt". The work fed dashboards.

Now the same dbt project feeds something else: it is the context an AI system reads. When someone asks an assistant for last month's approval rate, the quality of the answer is decided almost entirely by whether "approval rate" is a defined, tested, described metric or something the model has to infer from a table called fct_apps_v2_final.

The practical changes:

Model generation got cheap; model design did not. An assistant will happily write staging models from a source schema, generate tests from a data profile, and draft column descriptions. That used to be two weeks of grind. The remaining work is deciding the grain, the conformed dimensions and what a metric actually excludes — which is exactly the part nobody can generate for you.

Descriptions became load-bearing. A column comment used to be documentation nobody read. Now it is a prompt. A model reads status and has no idea what 3 means; it reads a description that says which statuses count as active and it stops guessing. We have watched teams get a bigger accuracy improvement from writing descriptions than from changing model.

The semantic layer became an interface, not a convenience. Exposing metrics through the dbt Semantic Layer, and increasingly through an MCP server, means an assistant asks for a metric rather than composing SQL. That single architectural choice removes most of the failure modes people blame on hallucination.

Contracts stopped being optional. If an upstream column silently changes type, a dashboard looks odd and someone raises a ticket. An AI system will confidently answer wrongly and nobody raises anything. Contracts and freshness tests became the thing standing between you and a quiet, expensive mistake.

BI developers moved from building pages to preparing models#

The Power BI job used to be roughly seventy percent building report pages. Copilot changed the ratio.

Copilot is only as good as the semantic model behind it, and preparing that model is now a real, named workstream:

• Synonyms, so the business can ask about "clients" when the table says customer
• Descriptions on tables, columns and measures, written for a machine as well as a person
• Hierarchies and correct date tables, because a model that cannot navigate time cannot answer half the questions people ask
• Verified answers for the questions that matter enough to be wrong about
• Row-level security that carries through into every AI answer, not just into the report

Most Copilot disappointment we are called in to look at is a model preparation problem wearing a Copilot costume. The organisations getting value from it did the unglamorous work first.

The other change: narrative. A dashboard that ships with a generated summary of what moved and why gets read by executives who never opened the previous version. That is a real gain, and it depends on the same thing — a model that knows what its own numbers mean.

Analysts draft with a model and validate by hand#

This is the role where day-to-day work changed most visibly and job security changed least.

The first draft of nearly everything — the SQL, the Python, the chart, the write-up — now comes from a model. What the analyst does is decide what the question actually means, check whether the answer is plausible, and notice the thing nobody asked about.

Two consequences worth naming:

The validation burden went up, not down. A confident wrong answer is more expensive than a blank page, because it gets forwarded. Good analysts have developed a reflex for checking joins, date boundaries and denominators before they look at the result.

Junior work got harder to justify and more important to protect. A lot of what juniors learned on was exactly the mechanical work that is now cheap. Teams that stopped hiring juniors will discover in three years that nobody learned to spot a bad join. The teams handling this well put juniors on validation and definition work early rather than on query-writing.

Data engineers got their time back and spent it on contracts#

Pipeline scaffolding, boilerplate transformations, generated data quality tests and first-pass documentation are all faster. Schema drift detection and failure triage are increasingly assisted.

What that time went into, in the teams doing it properly:

• Data contracts on the sources that matter, because AI systems fail silently on drift
• Lineage that is real rather than documented after the fact
• Freshness guarantees, because an assistant will answer from stale data without flagging it
• Cost control, which became a live concern the first month someone left an agent looping

The job did not shrink. The centre of gravity moved from writing pipelines to guaranteeing properties about them.

Data scientists stopped building things they can now buy#

The most concrete change: a large amount of bespoke natural language work — classification, extraction, summarisation, sentiment — is no longer worth building from scratch. A foundation model with a good prompt beats six weeks of feature engineering on most of those tasks.

What did not change is anything where you have to justify the decision. Credit scoring is still built with methods you can explain, monitor and defend, because "the model said so" has never been an acceptable answer to a regulator and is less acceptable now than it was.

So the role split. Some data scientists moved toward AI engineering. Others moved deeper into decision science, where the constraint is explainability rather than capability. The middle — building a mid-sized bespoke model for a task a foundation model now handles — largely disappeared.

Three roles that did not exist in this shape#

AI engineer. The fastest-growing title in data teams, and it is not a data scientist with a new name. It is systems work: retrieval design, tool and MCP interfaces, agent boundaries, context window budgeting, caching, fallback behaviour, and the eval suite that tells you when a prompt change broke something. Most people doing it came from backend engineering or analytics engineering rather than from ML research.

Evaluation engineering. Sometimes a separate person, usually a hat somebody wears. Owns the golden sets, the LLM-as-judge rubrics, the regression suite and the release gate. On any AI system that matters, this is the difference between shipping and gambling.

AI governance and model risk. This existed in banks and has now spread. What changed is that it became continuous. An annual model validation document does not cover a system whose behaviour changes when someone edits a prompt. And since August 2026, anyone using AI to assess the creditworthiness of natural persons in the EU is carrying an Annex III high-risk system, with the documentation, logging, human oversight and post-market monitoring obligations that come with it.

The thing that did not change#

Somebody still has to decide what a metric means.

Every failure mode above resolves to the same root. The model does not know that trial accounts are excluded from active customers, that the Spanish market books on a different day, or that revenue means gross to finance and net to the board. It cannot know. Nobody wrote it down anywhere it could read.

That was true before any of this. What is different is the cost of leaving it undone. An undefined metric used to produce an argument in a meeting. Now it produces a confident answer, at scale, to people who were not in the meeting.

If you manage one of these teams#

Four things worth doing this quarter, in order:

1. Write down your twenty most-used metric definitions and put them somewhere a machine can read. Not a wiki — the dbt project.
2. Build a golden set of the fifty questions your business actually asks, with correct answers. You cannot improve what you cannot score.
3. Pick one decision and thread it end to end rather than starting a platform programme. A narrow thing in production teaches you more than a wide thing at ninety percent.
4. Work out what you are carrying under the AI Act if you operate in the EU and touch credit, employment or essential services. Better to find out on your own schedule.

None of that requires a model vendor. All of it makes the eventual model work several times more likely to survive.