The machine learning lifecycle on SageMaker: from raw data to a monitored endpoint

A model is a small file at the end of a long road. Before it exists someone has to decide what question is worth answering, find the data, clean it, label it, shape it into features, pick an algorithm, train, tune, and prove the result is good enough; after it exists someone has to host it, watch it, and retrain it when the world moves on. Amazon SageMaker is the AWS service built around exactly that road, with one named feature for each step, and the exam tests the road twice: once as the stages of a lifecycle and once as "which SageMaker feature does this". This article walks the road from start to finish, stopping at each feature, at the metrics that decide whether a model is any good, and at the MLOps habits that keep it good.

The map

Read this first when short on time. Every branch is a section below.

The pipeline: ten stages from a business question to a watched model

The machine learning lifecycle is the ordered set of stages a model passes through, from deciding what to predict to noticing that the deployed model has gone stale, and it is a loop rather than a line because the last stage feeds the first.

The exam guide names the stages in one breath: business goal, data collection, exploratory data analysis, data pre-processing, feature engineering, model training, hyperparameter tuning, evaluation, deployment and monitoring. Different books merge or split them (AWS's own well-architected material folds them into "business goal, ML problem framing, data processing, model development, deployment, monitoring"), but the order is always the same and the content of each is stable. The first five are about data and take most of the calendar time; the next three are about the model and take most of the compute; the last two are about production and never end.

Amazon SageMaker (renamed Amazon SageMaker AI at the end of 2024, when "Amazon SageMaker" became the name of a wider data-and-AI studio; the exam and most documentation still say SageMaker) is a managed service with one feature for each stage. That mapping is the spine of this article and, in practice, of the exam: a scenario describes a stage in plain words and the answer is the feature that owns it.

The front door to all of it is Amazon SageMaker Studio, the browser-based development environment: JupyterLab notebooks and a code editor on managed compute, plus the visual views of experiments, pipelines, the model registry, endpoints and monitors, so a whole project is run from one window. Canvas (no code), JumpStart (the model hub) and Data Wrangler are applications inside it. When a question says "an integrated development environment for the whole ML workflow", Studio is the answer.

flowchart TD
  G["Stage 1 · Business goal<br/>what to predict, what success costs"]
  C["Stage 2 · Data collection<br/>S3, Glue, Athena, Ground Truth labels"]
  E["Stage 3 · Exploratory data analysis<br/>Data Wrangler insights report"]
  P["Stage 4 · Pre-processing<br/>clean, impute, encode, split"]
  F["Stage 5 · Feature engineering<br/>Feature Store, Processing jobs"]
  T["Stage 6 · Model training<br/>training job, built-in algorithms"]
  H["Stage 7 · Hyperparameter tuning<br/>Automatic Model Tuning"]
  V["Stage 8 · Evaluation<br/>metrics, Clarify, Experiments"]
  D["Stage 9 · Deployment<br/>endpoint or batch transform"]
  M["Stage 10 · Monitoring<br/>Model Monitor, CloudWatch"]
  G --> C --> E --> P --> F --> T --> H --> V --> D --> M
  V -. "not good enough: more data, other features" .-> F
  M -. "drift: collect fresh data, retrain" .-> C

A useful way to hold the whole thing is as three questions. Is this worth predicting, and do we have the data? (stages 1 to 5). Can we build a model that predicts it well enough? (stages 6 to 8). Can we run it, and keep it good? (stages 9 and 10). Each question has a feature set, a metric set and a failure mode, and each is a section below.

Lifecycle

The ordered stages a machine learning system passes through, from framing the goal to monitoring in production and back.

Amazon SageMaker AI

The 2024 name for the SageMaker machine learning service; "Amazon SageMaker" now also names the unified studio that wraps it with analytics tools. Exam questions use the old, shorter name.

Artefact

Any file a stage produces for the next one: a cleaned dataset, a feature table, a trained model (model.tar.gz in S3), an evaluation report.

Framing the goal and collecting the data: S3, Glue, Athena and Ground Truth

The first two stages decide what to predict and gather the raw material for predicting it, and SageMaker's contribution here is labelling, because storage and querying are the job of the wider AWS data stack.

The business goal

A machine learning project starts as a business question, not a dataset: reduce fraud losses, cut churn, forecast demand within a tolerance, route support tickets faster. Framing turns that into a prediction problem (which technique, per the use-cases article), a success metric that the business will actually feel (money saved, tickets deflected) plus a model metric that stands in for it during development (recall, mean absolute error), and a baseline: the number the current process, or a trivial rule, already achieves. A model that does not beat the baseline is not a result. The framing also decides what an error costs, because that decides which metric matters later: a missed fraud costs the transaction, a false alarm costs a phone call, and the two are not equal.

Data collection

Data lands in Amazon S3, which is the data lake for every SageMaker feature: training jobs read from it, batch transform reads and writes it, Feature Store's offline half lives in it, Model Monitor writes reports to it. Around S3 sit the tools that make raw files usable: AWS Glue crawls the files and catalogues their schema and runs ETL jobs, Amazon Athena queries the files in place with SQL, Amazon Redshift holds the warehouse, Amazon Kinesis Data Streams and Data Firehose bring in events as they happen, and AWS Lake Formation governs who may read which columns. AWS Data Exchange is where third-party datasets are bought. The SageMaker side of collection is Data Wrangler's import step, which connects to S3, Athena, Redshift, Snowflake, Databricks and other sources without code.

Labelling with Ground Truth

Supervised learning needs a label on every training example, and most data arrives without one. Amazon SageMaker Ground Truth is the labelling service: you point it at the data, choose a task template (bounding boxes, image classification, semantic segmentation, text classification, named entities, video frames, 3D point clouds, or a custom template), and choose who labels. Three workforces are on offer: Amazon Mechanical Turk, a public crowd, cheap and fast but not for confidential data; a vendor workforce from AWS Marketplace, specialist companies under contract; and a private workforce, your own employees or contractors, for anything sensitive or requiring domain knowledge. Ground Truth can also run automated data labelling: it trains a model on the human labels as they arrive and lets that model label the items it is confident about, sending only the ambiguous ones to people, a technique called active learning that can cut labelling cost sharply on large datasets. Several people can label the same item and Ground Truth reconciles them by consensus, which is how label quality is controlled.

Ground Truth Plus is the turnkey version: you upload the data and describe what you need, AWS supplies and manages an expert workforce and the labelling workflow, and you review the results. It is the answer when a scenario wants high-quality labels "without building a labelling workflow or managing a workforce". Once the model is live, the same idea of putting humans in the loop for low-confidence predictions is Amazon Augmented AI (A2I), covered in the use-cases article.

Baseline

The score the existing process or a trivial rule achieves. Every model is judged against it, not against zero.

Active learning

A loop where a model labels the examples it is sure of and asks humans only about the ones it is not, so human effort goes where it changes the most.

Consensus

Sending the same item to several labellers and keeping the majority answer, with disagreement as a signal of an ambiguous item or an unclear guide.

Exploring, cleaning and engineering features: Data Wrangler, Processing jobs and Feature Store

Stages three to five turn raw rows into the exact numeric inputs a model will learn from, and they are where most of a project's hours and most of its accuracy come from.

Exploratory data analysis

Exploratory data analysis (EDA) is looking before modelling: the distribution of every column, how many values are missing, which columns correlate with each other and with the target, how imbalanced the classes are, whether there are duplicates or obvious outliers, and whether any column is leakage, a value that would not be known at prediction time but sneaks in from the future (the "refund issued" flag in a fraud dataset). On SageMaker this is done in a Studio notebook with pandas and plots, or in Amazon SageMaker Data Wrangler, whose data quality and insights report produces the statistics, flags target leakage, duplicate rows, class imbalance and anomalous samples, and includes a quick model: a throwaway model trained on the spot to estimate how predictable the target is and which columns carry the signal. Data Wrangler started as a Studio tool and now lives inside SageMaker Canvas, so both names appear in questions.

Pre-processing

Pre-processing makes the data consistent and machine-readable: drop or impute missing values, remove duplicates, fix types, normalise or standardise numeric columns so no feature dominates by its units, encode categorical values as numbers (one-hot encoding turns "red, green, blue" into three 0/1 columns), tokenise text, resize images, and finally split the data into training, validation and test sets, stratified so each holds the same class mix. Data Wrangler offers several hundred of these as point-and-click transforms and exports the recipe as code, a Processing job, a Pipelines step or a Feature Store ingestion. When the data is too large for an interactive tool, a SageMaker Processing job runs the same script on a fleet of instances: it is a managed container run (scikit-learn, Spark or your own image) that reads from S3, does the work, writes to S3 and shuts down, and it is also how evaluation scripts run later in the pipeline.

Feature engineering

Feature engineering creates the inputs the model actually sees, which are rarely the raw columns: the age of an account rather than its creation date, the ratio of this transaction to the customer's average, the day of week, a rolling seven-day count, the embedding of a product description, a bin ("small, medium, large") instead of a raw number, or a dropped feature that only added noise. It is the stage where domain knowledge enters the model, and on tabular problems it moves accuracy more than the choice of algorithm does. Feature selection (keeping the columns that matter) and dimensionality reduction (compressing many columns into a few, PCA being the built-in algorithm for it) both live here.

Feature Store

Amazon SageMaker Feature Store is a purpose-built repository for engineered features, and it exists to solve a specific production problem: the features computed for training must be exactly the features computed at inference, or the model sees a world it was not trained on (training-serving skew). Features are grouped into feature groups, each row identified by a record identifier and an event time. Every group can have two stores. The online store is a low-latency key-value store read at inference time: the endpoint asks for the latest features of customer 4711 and gets them in single-digit milliseconds. The offline store is the full history in S3 (Parquet, catalogued in Glue, queryable with Athena) used to build training sets, including point-in-time queries that reconstruct exactly what was known about each row at the moment its label happened. Write once, and both stores are fed; teams discover and reuse each other's features instead of recomputing them.

flowchart TD
  R["Raw data in S3<br/>events, tables, logs"] --> J["Processing job or Data Wrangler<br/>compute features, one definition"]
  J --> FG["Feature group<br/>record id, event time, features"]
  FG --> ON["Online store<br/>latest value per record, milliseconds"]
  FG --> OFF["Offline store in S3<br/>full history, Athena, point-in-time"]
  OFF --> TR["Training job<br/>builds the training set"]
  ON --> EP["Endpoint<br/>fetches features at inference"]

Leakage

A training feature that carries information about the label which would not exist at prediction time. The model looks brilliant offline and useless live.

Training-serving skew

A difference between how features are computed for training and how they are computed at inference, so the model receives inputs it never learned from.

One-hot encoding

Turning a categorical column with k values into k columns of 0 and 1, one per value.

Point-in-time query

Rebuilding a training row using only feature values that existed at that row's event time, so no future information leaks in.

Where the model comes from: your own training, JumpStart, Hugging Face and open source

A model in SageMaker arrives by one of four routes, and choosing the route is a question of how specific the task is, how much data and expertise the team has, and whether the weights must stay in your account.

Train a custom model when the target is your own (churn, yield, default) and the data is your own. SageMaker gives three ways to do it, in rising order of control: a built-in algorithm where you only supply data and hyperparameters, script mode where you write a training script in PyTorch, TensorFlow, scikit-learn, XGBoost or Hugging Face and SageMaker supplies the framework container, and bring your own container where you package anything that can read from S3 and write a model. The next section covers the training job itself.

Amazon SageMaker JumpStart is the model hub inside Studio: hundreds of pre-trained open-source models (vision, text, tabular, and foundation models such as Llama, Falcon, Mistral, Stable Diffusion and many Hugging Face models) plus solution templates that stand up an entire use case (fraud detection, demand forecasting, document understanding) with the pipelines and endpoints already wired. Any model can be deployed to an endpoint in a few clicks, and most can be fine-tuned on your data with the same interface. JumpStart is the answer when the scenario wants a proven model quickly, wants to fine-tune rather than train from scratch, or wants a foundation model whose weights stay inside the customer's account rather than being called through Amazon Bedrock. Bedrock and its trade-offs are the subject of the Bedrock article.

Hugging Face on SageMaker is a partnership rather than a separate feature: AWS ships Deep Learning Containers with the transformers library installed, the SageMaker Python SDK has a HuggingFace estimator and model class, and any model on the Hugging Face Hub can be pulled by name into a training job or an endpoint. It is how a team takes BERT, a sentence-embedding model or a small LLM and fine-tunes or hosts it without building a container.

Open-source pre-trained models more broadly, from PyTorch Hub, TensorFlow Hub or a research repository, can be uploaded to S3 as a model.tar.gz with an inference script and hosted on any SageMaker endpoint. This is the route when a specific paper's weights are needed or when a model trained elsewhere is being brought onto AWS.

Pre-trained model

A model whose weights were learned on a large general dataset by someone else; you use it as is or fine-tune it on your own data.

Fine-tuning

Continuing training of a pre-trained model on a smaller, specific dataset so it adapts to your task; detailed in the customising article.

Script mode

Supplying only a training script; SageMaker provides the framework container and the machines.

Training and tuning: training jobs, built-in algorithms, Automatic Model Tuning, Autopilot and Canvas

A SageMaker training job is a container run on machines SageMaker provisions, fed from S3 and writing a model back to S3; everything else in this section (algorithms, tuning, AutoML, debugging, experiment tracking) is a layer around that one primitive.

The training job

A training job needs four things: an algorithm image (a container in Amazon ECR, built-in or yours), the S3 location of the training and validation data, an instance type and count, and a set of hyperparameters. SageMaker starts the instances, pulls the image, downloads or streams the data (File mode copies it first, Pipe and FastFile modes stream it so the job starts sooner), runs the container, captures its logs and metrics into Amazon CloudWatch, uploads the result to S3 as model.tar.gz, and terminates the instances, so you pay only for the training seconds. Managed Spot Training runs the job on spare capacity for a large discount (AWS quotes up to ninety percent) at the cost of possible interruption, which is why the script should write checkpoints to S3 so an interrupted job resumes rather than restarts. For models too large for one machine, SageMaker's distributed training libraries split the data (data parallel) or the model (model parallel) across a cluster, and SageMaker HyperPod is the resilient cluster offering for training foundation models over weeks.

flowchart TD
  D["Training data in S3<br/>train and validation splits"] --> J
  I["Algorithm container in ECR<br/>built-in, framework or your own"] --> J
  HP["Hyperparameters<br/>learning rate, depth, epochs"] --> J
  J["Training job<br/>ml instances start, run, stop"]
  J --> CW["CloudWatch<br/>logs and training metrics"]
  J --> A["model.tar.gz in S3<br/>the trained artefact"]
  A --> REG["Model Registry<br/>versioned, approval status"]

Built-in algorithms

SageMaker ships a shelf of built-in algorithms, tuned implementations that need no code, only data in the expected format and hyperparameters. The exam names a few by their signature use.

Hyperparameter tuning with Automatic Model Tuning

Hyperparameters are the knobs the algorithm does not learn: learning rate, tree depth, number of trees, regularisation strength, batch size, number of epochs, number of clusters. Amazon SageMaker Automatic Model Tuning (AMT, also called hyperparameter optimisation or HPO) runs many training jobs with different settings and keeps the best by an objective metric you name (validation AUC, validation RMSE). You give it the ranges to search, the maximum number of jobs and how many to run in parallel, and a strategy.

Two economies matter: early stopping kills a training job whose metric is clearly worse than the best so far, and warm start seeds a new tuning job with the results of a previous one instead of beginning blind. Tuning is expensive by construction (it is training multiplied), so the ranges should be narrow and informed, and the objective metric must be computed on validation data, never on the training set.

Autopilot and Canvas: the automated and the no-code routes

Amazon SageMaker Autopilot is AutoML: give it a tabular dataset and a target column and it does exploration, pre-processing, algorithm selection, training and tuning by itself, producing a leaderboard of candidate models ranked by the objective metric, plus generated notebooks that show exactly what it did, so the result is transparent and editable rather than a black box. It can run in a hyperparameter-tuning mode, an ensembling mode (which combines several models) or choose automatically. Amazon SageMaker Canvas is the no-code front: a visual application for business analysts that connects to data, prepares it (Data Wrangler lives here now), builds models with Autopilot underneath, explains predictions, and also gives point-and-click access to ready-to-use models (Rekognition, Comprehend, Textract behind the scenes) and to foundation models. Canvas produces a model that can be shared into Studio and deployed like any other.

Debugger and Experiments

Two features watch the training itself. Amazon SageMaker Debugger captures tensors and metrics from inside the job while it runs and applies built-in rules (vanishing or exploding gradients, loss not decreasing, overfitting, class imbalance, poor weight initialisation, low GPU utilisation); a triggered rule can stop the job and send a notification through Amazon SNS, so a broken run does not burn a night of GPU time. Its profiler reports CPU, GPU, memory and I/O utilisation so an under-used expensive instance is spotted. Amazon SageMaker Experiments records every run's parameters, inputs, metrics and artefacts and lets them be compared side by side in Studio, which is how "which of the forty runs last week had the best recall, and what were its settings" is answered without a spreadsheet.

Hyperparameter

A setting chosen before training that shapes how learning happens (learning rate, depth, epochs), as opposed to a parameter (a weight) the training learns.

Objective metric

The single validation metric a tuning job maximises or minimises to rank its training jobs.

Checkpoint

A snapshot of the model mid-training saved to S3, so an interrupted spot job resumes from it.

AutoML

Automating algorithm selection, pre-processing and tuning end to end from a dataset and a target column; Autopilot is SageMaker's.

Judging a classifier: the confusion matrix, precision, recall, F1 and AUC

Every classification metric is a ratio of the four counts in the confusion matrix, and the right metric is the one that weights the kind of mistake the business cannot afford.

Evaluation happens on the test set, data the model never saw during training or tuning, so the score predicts how it will do on new data. For a binary classifier each test example falls into one of four cells. A true positive (TP) is a real positive the model called positive: the fraud it caught. A false positive (FP) is a negative the model wrongly called positive: the honest customer whose card was declined, a false alarm, a type I error. A false negative (FN) is a positive the model missed: the fraud that went through, a type II error. A true negative (TN) is a negative correctly left alone. Laid out in a two-by-two grid they are the confusion matrix, and everything else is arithmetic on it.

Accuracy, and why it lies

Accuracy is the share of all predictions that were right: (TP + TN) over everything. It is the metric everyone understands and the one that misleads most often, because of class imbalance. If one transaction in a thousand is fraud, a model that says "honest" to everything is 99.9 percent accurate and catches nothing. Accuracy is fine when the classes are roughly balanced and both kinds of error cost about the same; otherwise reach for the next two.

Precision and recall

Precision is TP over (TP + FP): of everything the model flagged as positive, the fraction that really was. High precision means few false alarms. Recall (also sensitivity, or true positive rate) is TP over (TP + FN): of everything that really was positive, the fraction the model caught. High recall means few misses. The two pull against each other through the decision threshold: a classifier outputs a probability and a threshold turns it into a yes or no; lower the threshold and more things are flagged, so recall rises and precision falls; raise it and the reverse. Which to favour is a business decision made in stage one. When a false negative is costly (a missed cancer, a missed fraud, a missed fault in a jet engine) optimise recall and accept more false alarms. When a false positive is costly (a legitimate email in the spam folder, a good customer's card declined, an innocent person flagged) optimise precision and accept some misses. Specificity, TN over (TN + FP), is recall's mirror for the negative class and shows up in medical contexts.

F1, AUC-ROC and log loss

F1 is the harmonic mean of precision and recall, 2PR over (P + R): one number that is high only when both are high, so it is the summary metric for imbalanced problems where neither error type dominates. The harmonic mean punishes lopsidedness: precision 1.0 with recall 0.1 gives an F1 of about 0.18, not the 0.55 an average would suggest. AUC-ROC (area under the receiver operating characteristic curve) judges the classifier across all thresholds at once: the ROC curve plots true positive rate against false positive rate as the threshold slides from strict to lenient, and the area under it is the probability that the model ranks a random positive above a random negative. AUC of 1.0 is a perfect ranker, 0.5 is a coin toss, below 0.5 means the labels are inverted. It is the metric to compare models before a threshold has been chosen, a common objective metric for Automatic Model Tuning on binary problems, and one of the columns on every Autopilot leaderboard (whose default objective for binary classification is F1). Log loss (cross-entropy) scores the probabilities themselves rather than the decisions: a confident wrong prediction (0.95 for an honest transaction that was fraud) is punished far more than a hesitant one, so it rewards models that are well calibrated. Lower is better, and it is the training loss of most classifiers as well as an evaluation metric.

Confusion matrix

The two-by-two (or k-by-k) table of predicted class against actual class whose cells are TP, FP, FN and TN.

Threshold

The probability above which a classifier says "positive". Moving it trades precision against recall.

ROC curve

True positive rate plotted against false positive rate for every threshold; the area under it is AUC.

Calibration

Whether a predicted probability of 0.8 comes true about eighty percent of the time. Log loss rewards it; accuracy ignores it.

Judging a regressor: MAE, MSE, RMSE, MAPE and R squared

A regression model predicts a number, so its errors are distances rather than categories, and the metrics differ in how hard they punish a big miss and whether they speak in the target's units.

For each test row the residual is predicted minus actual. Mean absolute error (MAE) averages the absolute residuals: on average the forecast is off by 12 units. It is in the target's units, easy to explain, and treats a miss of 100 as exactly ten times as bad as a miss of 10, so it is robust to a few outliers. Mean squared error (MSE) averages the squared residuals, so a miss of 100 counts 10,000 against a miss of 10 counting 100: large errors dominate, which is what you want when one big miss really is worse than many small ones, and it is the loss most regressors minimise during training because it is smooth. Its units are squared, which is awkward, so root mean squared error (RMSE) takes the square root to return to the target's units while keeping the outlier sensitivity. RMSE is always at least as large as MAE, and the gap between them is a quick measure of how many large misses there are.

Mean absolute percentage error (MAPE) averages the absolute residual divided by the actual value, giving a percentage that is comparable across series of different scales (a forecast that is 8 percent off for both a bestseller and a slow item). Its weakness is actual values near zero, where a small miss becomes an enormous percentage, and it is undefined at zero, which is why demand forecasting for intermittent items uses weighted variants. R squared (the coefficient of determination) is the fraction of the variance in the target that the model explains: 1.0 means perfect prediction, 0 means no better than always predicting the mean, and it can go negative for a model worse than that. It is scale-free and good for "how much of what is going on does this model capture", but it says nothing about the size of errors in real units, and adding features never lowers it, which is why adjusted R squared exists to penalise pointless features.

Residual

Predicted value minus actual value for one row; the raw material of every regression metric.

Outlier

A row whose value is far from the rest; squared-error metrics let a few of them dominate the score.

Variance explained

How much of the spread in the target the model's predictions account for; R squared is this as a fraction.

Judging the project: cost per user, return on investment and the other business metrics

Model metrics say whether the model predicts well; business metrics say whether the project was worth doing, and the exam expects both to be named and kept apart.

A model is a means. The end is a business outcome, and the business outcome is measured in the business's own terms. Return on investment (ROI) is the headline: the value the model creates (fraud losses avoided, hours of manual review removed, extra sales from better recommendations) minus what it cost, divided by what it cost. Its inputs are the rest of the list. Development costs cover people, labelling, compute for training and tuning, and the tooling; they are mostly one-off but recur at every retrain. Cost per user (or per prediction, per transaction) is the running cost of inference divided by the volume served, and it is the number that decides between an always-on endpoint and a batch job, or between a large model and a distilled one. Customer feedback is the qualitative signal: ratings, complaints, support tickets, survey scores, thumbs up and down on a chatbot answer; it catches failures no metric was defined for. Time to market is how long from the idea to the first version in front of users, which is the argument for managed services, pre-trained models and AutoML over training from scratch. Revenue lift (or conversion lift, retention lift) is the direct measurement of the outcome, established by an A/B test that gives one group the model and another the old process and compares the business number, not the F1.

The link between the two families is the thing to be able to explain. A recall improvement from 0.80 to 0.90 on a fraud model is a model metric; turned into "an extra twelve percent of fraud losses prevented, worth this much per month, against the extra false alarms costing this much in calls" it is a business metric, and only the second version justifies the project. Conversely a model that improves F1 by two points and changes no business number is a cost. Setting the business metric in stage one, and the model metric as its stand-in, is what stops a team optimising the wrong thing for six months.

A/B test

Splitting users at random between the new model and the old process and comparing an outcome metric, so the difference can be attributed to the model.

Lift

The improvement in an outcome (revenue, conversion, retention) attributable to the model, usually from an A/B test.

Baseline cost

What the outcome costs today, without the model; ROI is measured against it.

Deployment: managed API or self-hosted, and the four ways SageMaker serves a model

Deployment puts the trained artefact where callers can reach it, and the choice runs along two axes: whether you host the model at all, and, if you do, whether callers wait, how big their requests are and how steady their traffic is.

Managed API versus self-hosted

A managed API service (the AI services in the use-cases article, Amazon Bedrock for foundation models) hosts the model for you: no instances, no scaling, no patching, pay per call. A self-hosted model on a SageMaker endpoint gives control over the model, its version, its hardware, its network placement and its data handling, at the price of owning capacity and operations. The exam's keywords for the first are "least operational overhead" and "no ML expertise"; for the second "custom model", "specific instance type", "the model must stay in our VPC" and "high, steady volume where per-call pricing costs more than instances".

The four SageMaker serving options

The fundamentals article introduced batch versus real-time inference as concepts; here are the SageMaker products that implement them. A real-time endpoint keeps the model loaded on instances you choose, answers synchronous HTTPS requests in milliseconds, and supports auto scaling on invocations per instance; payloads are small (about 6 MB) and each call must finish within about a minute. Serverless Inference is a real-time endpoint on capacity SageMaker manages: pick a memory size, get a URL, pay per millisecond of compute and per request, and scale to zero between calls, with a cold start when traffic resumes, a slightly smaller payload limit and the same one-minute cap. Asynchronous Inference queues each request, processes it and writes the result to S3, notifying through Amazon SNS; it takes payloads up to about a gigabyte and processing times up to about an hour, and scales its instances to zero when the queue is empty. Batch Transform is not an endpoint at all: a job that reads a dataset from S3, runs the model on temporary instances, writes the predictions to S3 and stops, the cheapest and simplest way to score everything when nobody is waiting.

flowchart TD
  Q1{"Someone waiting?"}
  Q1 -- "no, score a dataset" --> BT["Batch Transform<br/>S3 in, S3 out, no endpoint"]
  Q1 -- "yes" --> Q2{"Payload large?"}
  Q2 -- "yes, or slow to process" --> AS["Asynchronous Inference<br/>queue, result to S3, SNS"]
  Q2 -- "no" --> Q3{"Traffic steady?"}
  Q3 -- "yes" --> RT["Real-time endpoint<br/>instances always on, auto scaling"]
  Q3 -- "no, spiky or rare" --> SL["Serverless Inference<br/>scales to zero, cold starts"]

Sharing, rolling out and sizing an endpoint

A multi-model endpoint hosts many models behind one endpoint on shared instances, loading each from S3 into memory on first use and evicting cold ones, which is how a company with one small model per customer or per region avoids one endpoint per model. Multi-container endpoints run up to fifteen different containers on one endpoint with direct invocation of each; inference pipelines chain containers in series so a preprocessing step, the model and a post-processing step run as one call. Inference components let several foundation models share one endpoint's accelerators with per-model resource allocation.

Rolling out a new version safely uses production variants: two model versions behind one endpoint with a traffic split (90/10) for an A/B test, or a shadow variant that receives a copy of every request and whose responses are logged but never returned, so a new model is tested on live traffic with zero user impact. Deployment guardrails do blue/green deployments with all-at-once, canary (a small slice first) or linear (step by step) traffic shifting, and roll back automatically when a CloudWatch alarm fires. Before any of that, Amazon SageMaker Inference Recommender load-tests the model across instance types and configurations and recommends the one that meets a latency target at the lowest cost, replacing guesswork about whether the endpoint needs a GPU.

The edge

Some models run on the device rather than in the cloud, for latency, privacy or missing connectivity. Amazon SageMaker Neo compiles a trained model for a specific target (ARM and Intel CPUs, Nvidia GPUs, AWS Inferentia, various embedded chips), producing a smaller, faster binary with no accuracy change; SageMaker Edge Manager was the fleet service that packaged, deployed and monitored those models on devices (AWS has since wound it down, but the name still appears in study material), and AWS IoT Greengrass is the runtime that executes them on the device.

Endpoint

A hosted HTTPS URL with a model behind it. Real-time and serverless endpoints answer in the response; an asynchronous endpoint answers through S3.

Production variant

One model version behind an endpoint, with a share of the traffic; several variants make an A/B test.

Shadow test

Sending a copy of live requests to a new model whose answers are recorded but not returned, so it is judged on real traffic with no user impact.

Canary

Shifting a small slice of traffic to the new version first and watching alarms before moving the rest.

Monitoring: Model Monitor, the four kinds of drift, Clarify, Model Cards and the Model Dashboard

A deployed model decays because the world it was trained on changes, and monitoring is the stage that notices the decay and starts the retraining that fixes it.

Two kinds of change hurt. Data drift (feature drift, covariate shift) is the inputs changing: customers get younger, prices rise, a new product category appears, a sensor is recalibrated, a field starts arriving empty. Concept drift is the relationship between inputs and the right answer changing: the same transaction pattern that was fraud last year is now normal, or fraudsters have moved on to a pattern the model never saw. The first is visible from the inputs alone; the second only when the true labels arrive and the predictions are compared with them.

Amazon SageMaker Model Monitor automates the watching. First data capture is switched on for the endpoint (or the batch transform job), so requests and responses are copied to S3. A baseline job then runs over the training data to produce statistics (distributions, means, ranges) and constraints (the rules the live data should obey: this column is never null, this one is a number between 0 and 1). A monitoring schedule (hourly at the finest) runs a Processing job that compares the captured traffic against the baseline, writes a violations report to S3 and publishes metrics to CloudWatch, where an alarm can notify a team or trigger a retraining pipeline. Four monitor types cover the four things that can drift.

sequenceDiagram
  autonumber
  participant E as Endpoint
  participant M as Model Monitor
  participant C as CloudWatch
  participant P as Retrain pipeline
  E->>E: data capture copies requests to S3
  M->>M: baseline from training data
  M->>E: hourly schedule reads captured data
  M->>M: compare with baseline constraints
  M->>C: metrics and violations report
  C->>C: alarm on drift threshold
  C->>P: EventBridge starts the pipeline
  P->>P: fresh data, train, evaluate, register
  P->>E: approved model deployed, canary first

Clarify, Model Cards, the Model Dashboard

Amazon SageMaker Clarify is the bias and explainability tool that Model Monitor borrows for its last two monitors, and it works at every stage: pre-training bias metrics on the dataset (class imbalance, difference in proportions of labels), post-training bias metrics on predictions (disparate impact, accuracy and recall differences across groups), SHAP explanations of which features drove each prediction and the model overall, and evaluations of foundation models. The metrics and their meaning are in the responsible AI article; here the point is where Clarify sits in the lifecycle: evaluation before deployment, and drift after it. Amazon SageMaker Model Cards are the documentation record for a model: intended use, out-of-scope uses, risk rating, training data and method, evaluation results, observations and approval status, kept with the model so an auditor or a new team member can read it; the responsible AI article covers them alongside AI Service Cards. The Amazon SageMaker Model Dashboard is the single console view of every model in the account: which are deployed, which monitors run on them, which alarms are firing, and their lineage back to training jobs and data, so a fleet of endpoints can be checked from one page.

Data drift

The distribution of the model's inputs changing over time, so live data no longer looks like the training data.

Concept drift

The relationship between inputs and the correct output changing, so the same input now deserves a different answer.

Baseline

In Model Monitor, the statistics and constraints computed from training data that live traffic is compared against.

SHAP

A method that attributes each prediction to the features that drove it, giving per-prediction and overall feature importance.

MLOps: Pipelines, the Model Registry and CI/CD for models

MLOps is applying the discipline of software delivery to machine learning, so that every stage above runs as repeatable, versioned, automated code rather than as a notebook someone ran once, and SageMaker Pipelines with the Model Registry is the AWS implementation.

The principles

The exam names the ideas that MLOps is built from. Experimentation: many runs, tracked, comparable, cheap to repeat (SageMaker Experiments). Repeatable processes: the path from raw data to deployed model is code that produces the same model from the same inputs, not a sequence of manual clicks (Pipelines). Scalable systems: training, processing and serving run on managed infrastructure that grows with the data and the traffic rather than on one person's machine (training jobs, Processing jobs, endpoints with auto scaling). Managing technical debt: ML systems accumulate hidden debt fast (glue code, undeclared consumers of a model's output, feature pipelines nobody owns, data dependencies that silently change), and the remedies are the same as in software (versioning data and models, tests on data and on predictions, documented ownership, retiring what is unused). Achieving production readiness: a model is ready when it has passed evaluation gates, has monitoring, has a rollback path, has documentation, and is approved by someone accountable (Model Registry approval, Model Cards, deployment guardrails). Model monitoring and model re-training: production is a loop, and the loop is automated so that drift triggers retraining without a person noticing first.

SageMaker Pipelines and the Model Registry

Amazon SageMaker Pipelines is the workflow service built for ML: a pipeline is a directed graph of steps (Processing, Training, Tuning, Condition, RegisterModel, Transform, Lambda, Callback and more) defined in Python, parameterised (which dataset, which instance type), versioned, and visualised in Studio with the lineage of every artefact. A typical pipeline processes the data, trains, evaluates in a Processing job, and hits a condition step: if the objective metric beats the threshold (or the current production model), register the model; otherwise stop. Amazon SageMaker Model Registry is where registered models live: a model package group per problem, a numbered version per registered model, metadata (metrics, lineage, Model Card), and an approval status of pending, approved or rejected. A human, or an automated check, flips a version to approved, and that status change is an event.

CI/CD for models

Continuous integration and delivery for ML has two pipelines. The build pipeline runs when code or data changes: a commit to the model repository (GitHub, Bitbucket, or the now-closed-to-new-customers AWS CodeCommit) triggers AWS CodePipeline, which uses AWS CodeBuild to run the SageMaker Pipeline that processes, trains, evaluates and registers. The deploy pipeline runs when a model version is approved: the Model Registry emits an Amazon EventBridge event, CodePipeline picks it up, CodeBuild creates or updates the endpoint (typically through CloudFormation) in a staging account, tests it, waits for a manual approval, and promotes it to production with a canary. Amazon SageMaker Projects ships this whole arrangement as templates (build, train, deploy, with model monitoring included), so a team starts from working CI/CD rather than assembling it. Monitoring closes the loop: a Model Monitor alarm can start the build pipeline again with fresh data, which is model re-training on a trigger rather than on a calendar.

flowchart TD
  SRC["Code or data change<br/>commit, schedule, or drift alarm"] --> CP1["CodePipeline build stage<br/>CodeBuild runs the SageMaker Pipeline"]
  CP1 --> PL["SageMaker Pipeline<br/>process, train, evaluate, condition"]
  PL --> REG["Model Registry<br/>new version, pending approval"]
  REG --> APR{"Approved?"}
  APR -- "no" --> STOP["Rejected<br/>lineage kept, nothing deployed"]
  APR -- "yes, EventBridge event" --> CP2["CodePipeline deploy stage<br/>staging, tests, manual gate, production"]
  CP2 --> EP["Endpoint<br/>canary, then full traffic"]
  EP --> MON["Model Monitor and CloudWatch<br/>drift alarms"]
  MON -. "retrain trigger" .-> SRC

MLOps

The practices and tooling that make training, deploying and maintaining models automated, repeatable and monitored, borrowing from DevOps.

Lineage

The recorded chain from a deployed model back through the training job, hyperparameters, code and data that produced it.

Approval status

The Model Registry flag (pending, approved, rejected) on a model version that gates deployment and fires an event when changed.

Condition step

A Pipelines step that branches on a metric, so only a model that beats the bar is registered.

Recap