Arcadion
What an MLOps Pipeline Includes from Training to Production
Close Icon

Stay up to date with the latest news in Managed IT, cybersecurity and Cloud Infrastructure.

What an MLOps Pipeline Includes from Training to Production


Thursday, November 19, 2026
By Simon Kadota
Share

Your model passed every test. So, who is responsible for it on Monday morning?

That question exposes the gap between a successful experiment and a system the business can depend on. Once people rely on predictions, someone needs to manage the data, approve changes, and respond when results stop making sense.

A production MLOps (Machine Learning Operations) pipeline connects validated data, repeatable training, release testing, deployment, and monitoring. Its most important output is an identifiable release that a team can approve, operate, and recover when conditions change.

For an IT leader planning the first production model, the challenge is deciding which controls are essential now and which automation can wait. A small forecasting application does not need the same platform as a business serving hundreds of models. Both need to establish what is running, whether it remains useful, and who can intervene.

This guide follows a conventional predictive model, from data preparation to production. It also explains the extra release dependencies introduced by generative AI. Architecture is a planning example, not a requirement to adopt a particular cloud or toolset.

How the pipeline fits together

An MLOps pipeline is an executable workflow for producing, checking, or releasing machine learning assets. The broader lifecycle includes the people, risk decisions, and retirement activities surrounding those workflows. Keeping that distinction clear prevents teams from trying to automate decisions they have not yet defined.

The core flow below shows the handoffs. Monitoring can trigger an investigation or a new training run; a signal should not automatically authorize a replacement.

StageRecorded outputGate before moving forward
1. Validate and prepare data.Versioned dataset and validation reportInputs satisfy the agreed data contract.
2-Train and compare candidates.Model artifact and experiment recordThe candidate improves a relevant objective.
3-Evaluate the release.Test results and dependency recordQuality, compatibility, and risk criteria pass.
4-Register and approveIdentified model version and approvalThe authorized owner accepts the evidence.
5-Deploy and verifyDeployment record and smoke-test resultsCorrect release serves compatible inputs.
6-Monitor and respondMetrics, alerts, and investigation recordsEvidence determines the next action.

Each output should be addressable later. A report called “latest results” is much less useful than a report linked to a specific dataset, model, and deployment.

The six operational stages

1. Validate data before it reaches training or inference.

Define expected fields, types, units, permitted values, and handling of missing information. These rules form a data contract between the system supplying information and the workflow consuming it. Test the contract during training and at the production boundary.

For example, a maintenance model expecting temperatures in Celsius may accept Fahrenheit values as valid numbers. A schema check alone will not catch that error. Unit declarations, plausible ranges, and changes by source location provide stronger protection.

Preserve a versioned snapshot or reproducible reference for training data, subject to access and retention requirements. Record transformations as code or as versioned configuration. Separate training and evaluation data appropriately so that future information does not leak into historical predictions.

2. Train candidates and record what produced them.

Experiment tracking connects a training run to its code revision, data reference, parameters, environment, and evaluation outputs. Without it, a promising result may be impossible to explain or repeat.

The trained artifact needs a durable identifier. Record the dependencies necessary to load and execute it, including preprocessing. A model that expects a particular feature calculation is not an independent file that can be moved into any application.

Repeatability does not always mean identical weights. Randomness, hardware, and external dependencies can affect training. Document the level of reproducibility that the application actually requires, and retain approved artifacts so that recovery does not depend on retraining.

3. Evaluate the whole release.

Testing should reflect the cost of errors. A document classifier may have acceptable overall accuracy while missing urgent records. Review relevant segments and consequential failure types alongside aggregate metrics.

Include runtime checks: accepted input formats, latency, throughput, memory, access restrictions, and behaviour when dependencies fail. Test the preprocessing and application components that will run with the model, not just a notebook that contains it.

Set acceptance criteria before reviewing a candidate. Otherwise, teams can redefine success around whichever metrics improved. A candidate that fails a critical segment should not pass simply because its average score rose.

4. Please register the model and authorize its release.

A model registry identifies approved and candidate versions and associates them with lineage and evaluation evidence. Registration and approval are different events: storing a model does not make it suitable for production.

Keep a release record connecting the model version to its runtime, configuration, tests, and decision-maker. If the same model is deployed with new preprocessing, that is a new system release even if its weights are unchanged.

Our guide to model versioning and rollback (coming soon) includes a sample release record and explains how to preserve a compatible recovery package.

5. Deploy through a controlled change

A machine learning deployment pipeline installs an approved release in a serving environment or scheduled batch workflow. The deployment record should identify the environment, release time, responsible operator, and exact components installed.

Choose a rollout method that fits the workflow. A real-time service may route a limited share of requests to a candidate. A batch forecast may generate parallel outputs for review before anyone uses them to place orders. Shadow outputs should remain isolated from business actions until approved.

Verify the release after installation. Verify both service health and representative outputs. Keep the previous approved package available, but confirm that its data and application dependencies remain compatible before relying on rollback.

6. Monitor performance and assign the response.

Production monitoring requires three views: service health, input and output behaviour, and measured business or model quality. A healthy endpoint can still produce poor predictions.

Track outcomes when they become available and compare relevant segments over appropriate periods. For earlier signals, monitor data quality and distribution changes. Assign every actionable alert an owner and a defined investigation path.

What CI/CD and continuous training automate?

Continuous integration checks changes to code and components. Continuous delivery prepares validated changes for release; continuous deployment goes further by releasing automatically when defined conditions pass. Continuous training reruns model training when a schedule or event warrants it.

These processes can connect without granting every automated training run permission to replace production. A retraining trigger creates a candidate. Evaluation and approval decide whether that candidate should serve users.

Google Cloud’s MLOps architecture guidance distinguishes automation levels and describes how training pipelines and delivery processes work together. Use those patterns to clarify responsibilities before selecting tooling.

For an initial deployment, automate repetitive validation and packaging first. Keep consequential approvals explicit until the organization has reliable tests and a documented policy for automated promotion.

A worked example for a weekly demand forecast

Consider an illustrative retailer forecasting weekly demand. Its training workflow reads an approved sales snapshot, validates units and stockout indicators, and produces a model plus an evaluation report. The report compares the candidate with the current model and a simple baseline, including performance for products where underforecasting is costly.

The release gate requires acceptable results for those product groups, a successful batch runtime test, and business-owner approval. The deployment package contains the model, feature transformations, and forecasting configuration. Each output file records the release that produced it.

After a data-feed update, forecasts become unusually high at one location. Monitoring identifies the change, and the team pauses automated order recommendations for that location. Investigation finds duplicated transactions. Correcting the feed is the appropriate first repair; retraining on duplicated records would preserve the defect.

The design lesson is that the pipeline must preserve evidence at every handoff. A model score alone does not explain which input changed, which recommendations were affected, or how to resume the workflow safely.

What changes for generative AI?

A generative AI application may depend on a provider model, prompt templates, retrieval settings, document permissions, tools, and an evaluation dataset. Record these as part of the release even when the organization does not train the underlying model.

Evaluation should test the application’s intended task: grounded answers, appropriate refusals, access boundaries, and permitted actions. A conventional accuracy metric is not a substitute for those checks. Keep deployment and recovery controls around the complete application configuration.

Build a minimum viable operating process.

Before buying a platform, ask the team to produce five pieces of evidence:

  1. the current release record
  2. its test results
  3. the deployment history
  4. the monitoring ownership record
  5. and a completed recovery test.

Missing evidence identifies a more useful starting point than a feature comparison between tools.

Include operating costs in that assessment. Training, inference, retained artifacts, monitoring data, evaluation, and engineering support all require resources. Decide how long historical assets must remain available and how retirement will remove endpoints, scheduled jobs, credentials, and downstream dependencies.

NIST’s AI Risk Management Framework Playbook offers governance guidance for ongoing risk management and contingency planning. Translate relevant actions into named owners and observable evidence for the particular system.

Plan the next production decision with Arcadion

Arcadion’s MLOps and AI Lifecycle Management services include pipeline architecture, version control, deployment workflows, monitoring, and retraining orchestration across cloud, hybrid, and on-premises environments.

If your first release is approaching, contact us to discuss pipeline readiness. Describe the model’s purpose, its deployment environment, and the controls you already have. A useful starting agenda is the release handoff, monitoring ownership and recovery path; bring an architecture outline or existing release record if available, without sending sensitive datasets through the enquiry form.