Latest News & Resources

The Model Nobody Else Understands

Your Head of AI spent 18 months building the company's flagship predictive maintenance model. It runs on three manufacturing plants. It saves $6.2M annually. It is the most-cited AI success story in every board presentation.

In April, she accepts a position at a competitor.

Her replacement starts in July.

By October, the model is behaving differently. Accuracy has dropped. Maintenance teams are getting more false positives. Escalations are increasing.

The new AI lead investigates. There is no model documentation. No feature engineering rationale. No training data lineage. No retraining protocol. No explanation of why specific hyperparameters were chosen. The model is running on a pipeline that was built by someone who is no longer available to explain it.

The company has an $6.2M annual dependency on a model that nobody currently employed understands.

This is the AI succession problem. And it is more common than any organization wants to admit.

Why AI Knowledge Is Uniquely Fragile

Software code can be read and understood by other engineers. Business processes can be documented and transferred. Institutional knowledge, while always vulnerable to turnover, typically transfers through teams and documentation.

AI model knowledge is different. It is embedded in a combination of code, data, decisions, and context that is almost impossible to reconstruct after the fact.

The Tacit Knowledge Problem. Experienced AI practitioners make thousands of small decisions during model development: which features to include, how to handle outliers, why a specific validation approach was chosen, what edge cases required special handling. These decisions are rarely documented. They exist in the creator's memory.

The Data Archaeology Problem. Training data for production models is often transformed through multi-step pipelines. Understanding why a model behaves the way it does requires understanding the training data — which requires understanding the pipeline — which requires understanding choices made months or years earlier.

The Environment Problem. Models depend on specific software versions, infrastructure configurations, and data pipeline states. When these environments drift over time, the model's behavior changes. Understanding what changed and why requires documentation that was never created.

The Context Problem. Models are built to solve specific problems in specific contexts. The business context that justified specific modeling choices — why was F1-score chosen over precision? why was this customer segment excluded from training? — is organizational knowledge that lives outside the model itself.

The Four Succession Failure Modes

The Orphaned Model. The model creator leaves. The model continues running. Nobody retrains it. Nobody monitors it. Nobody knows what it does. It quietly degrades for 18 months until a business outcome failure forces attention.

The Accidental Change. A new engineer inherits the model. Without documentation, they make what seems like a minor change — updating a dependency, modifying a data pipeline step. The change has unintended consequences on model behavior that do not surface for months.

The Knowledge Concentration Problem. One person understands the entire AI stack. That person attends every meeting, reviews every model, makes every critical decision. They are a single point of failure. When they leave, the organization loses years of accumulated context overnight.

The Reconstruction Cost. When an undocumented model fails, the reconstruction cost is 60-80% of the original development cost. You effectively pay to build the model twice — except the second time you are also trying to understand what the first model was doing while simultaneously fixing it.

The AI Succession Framework

Solving the succession problem requires changing development practices, not just documentation practices.

Living Model Documentation. Every production model should have a model card — a structured document covering: problem definition and business context, data sources and preprocessing decisions, feature engineering rationale, training methodology and hyperparameter choices, validation approach and performance benchmarks, known limitations and failure modes, retraining schedule and data requirements, and contact ownership. Model cards are maintained as living documents, updated with every retraining.

Pair Development. No production model should be built by a single engineer. Every model should have a primary developer and a secondary reviewer who understands the model well enough to maintain it independently.

Retraining Runbooks. Every production model should have a documented retraining runbook: what triggers retraining, what data is required, what steps are followed, what validation is required before redeployment, and who approves redeployment.

Quarterly Model Reviews. A quarterly review process that walks through each production model's documentation forces documentation to stay current. Models that cannot be coherently explained in a quarterly review are flagged as succession risks.

Model Knowledge Audits. Semi-annually, ask: if the creator of this model left tomorrow, how long would it take us to explain to a new engineer how the model works and how to maintain it? If the answer is more than two weeks, you have a succession risk.

The ITSoli Knowledge Transfer Standard

ITSoli builds knowledge transfer into every model delivery. When we deploy a model, we deliver it with full documentation, a retraining runbook, a training data inventory, and a validation protocol.

We also conduct knowledge transfer sessions with client team members who will maintain the model. Not a hand-off meeting. A working session where the client team builds the model alongside our engineers.

Our standard: when an ITSoli engagement ends, your team can maintain every model we built without calling us. If they cannot, we have not finished.

AI capability that lives in people rather than systems is not organizational capability. It is individual capability that walks out the door.

Document your models like the engineer who built them is leaving next month. Because eventually, they will.