Prepare and test your skills

Prepare and test your skills

Data bias in machine learning (ML) happens when training data does not fairly represent the real problem space. A skewed or unrepresentative dataset teaches the model patterns from only part of the world, so predictions can be unreliable or unfair for people and cases that were missing. Diverse data gives the model examples from many conditions, groups, and situations, which helps it generalize to new or less common cases.
Unrepresentative data creates bias when the training set does not match the real conditions where the model will be used. Three data-sourcing decisions shape whether bias enters the model: feature selection, data volume, and variation. A feature is an input attribute used for model training, and features are how the model identifies patterns, so features must be relevant to the problem.
Relevant features carry a true signal about the problem, while irrelevant, noisy, or sensitive features can teach misleading or unfair patterns. Data volume also matters because more training examples generally improve outcomes and required data scales with problem complexity. Binary classification generally needs less data than multi-class classification because predicting one class from two choices is less complicated than predicting one class from many.
Recommended minimums for training data are classification with 50 rows times the number of features, forecasting with 5,000 rows times the number of features plus 10 unique values in the time series identifier column times the number of features, and regression with 200 times the number of features. Missing, incomplete, or inconsistent values also reduce data usefulness, so data quality must be reviewed before training because biased or erroneous data lowers model quality.
Diverse data is training data that captures the variety of the problem space across groups, conditions, and situations. The more diverse examples a model sees during training, the more readily it can generalize to new or less common examples after deployment. This matters for fairness because groups or situations absent from training data are likely to be handled poorly when the model is used.
Fairness review includes asking whether a feature could lead to biased or unfair decision-making for marginalized groups. Teams should understand each feature and its values, avoid random identifier columns that add noise, and remove target columns that are not part of the current inference problem when creating multiple models from one dataset. For example, a retail model trained only on winter purchase data may not predict summer clothing preferences or purchase behaviors.
Data preparation is the work of making training data ready before model training starts. If data is biased or contains missing or erroneous values, model quality suffers, so preparation must correct those problems first. Agent Platform (opens in a new tab) can handle missing values and can impute missing values for forecasting, but optimal results are more likely when all values and rows are available.
Preparation actions focus on completeness, consistency, and correct structure:
Two related data quality problems can also hurt production performance even when test scores look good. The table below compares data leakage and training-serving skew.
Side-by-side comparison of data leakage versus training-serving skew across definition, example, effect, and prevention, both requiring knowledge of feature meaning.
| Data quality problem | What it means | Example | Effect |
|---|---|---|---|
| Data leakage | Training uses input features that leak information about the target that is unavailable when the model is served. | A model predicts subscription sign-up using a future subscription payment as an input feature. | Strong test performance but poor production performance. |
| Training-serving skew | Training-time input features differ from serving-time input features. | A model predicts hourly temperatures but trains only on weekly temperatures. | Poor model quality in production. |
To prevent both problems, teams must know what each feature means, flag high correlations for review, and provide only input features that are available in the same form at serving time.
Exam tip: Biased or unrepresentative training data is a data problem, not just a model problem; if a model behaves unexpectedly, check the data first.
Model evaluation is the review of a trained model's performance on data it must handle. After training, Agent Platform provides a performance summary, and detailed analysis is available through evaluate or see full evaluation. Debugging a model is more about debugging the data than the model itself. If the model behaves unexpectedly before or after deployment, the next step is to return to the data and check where completeness, diversity, or correctness can be improved.
The Garbage In, Garbage Out (GIGO) principle states that a machine learning model learns from the data it is trained on, so bad inputs produce unreliable predictions. A model does not automatically correct inaccurate, incomplete, noisy, biased, or unrepresentative data; it finds patterns in whatever examples it receives, including wrong or misleading ones. Because sourcing and preparation are critical steps for building a model, the available data also shapes which problems can be solved.
Poor data enters a machine learning model through the features used for training. A feature is an input attribute used for model training, and features are how the model identifies patterns to make inferences, so they must be relevant to the problem. For example, a fraud-detection model needs transaction details such as buyer, seller, amount, date and time, and items purchased, plus useful history about the buyer, seller, and how often the item has been involved in fraud.
If features are irrelevant, noisy, or leak information about the target, the model can learn the wrong relationship. A random identifier column with a unique value for each row creates noise, while a future subscription payment used to predict whether a customer will subscribe leaks target information that is unavailable at serving time. The common causes of GIGO and their effects are compared below.
| Data problem | Effect on model | Prevention |
|---|---|---|
| Missing or incomplete values | Less useful training data; Agent Platform (opens in a new tab) can handle missing values, but optimal results are more likely when all values are available. | Correct missing values when possible; leave blank only if the column is nullable; for forecasting, keep consistent intervals between training rows. |
| Noise, errors, or inconsistency | Model learns wrong patterns and produces lower-quality results. | Correct or delete errors and noise; review spelling, abbreviations, and formatting. |
| Data leakage | Strong test performance but poor production performance because a leaked feature is unavailable at serving time. | Know what data means; flag high correlations in the Train tab for review; remove features unavailable at serving time. |
| Training-serving skew | Poor production quality because training inputs differ from serving inputs. | Provide only input features that are available in the exact same form at serving time. |
| Biased or unrepresentative data | Model may fail to generalize to new or less common examples and may make unfair decisions. | Capture diversity of the problem space; review fairness principles and avoid biased features. |
High-quality data preparation prevents GIGO by making training data relevant, sufficient, diverse, clean, and consistent. Before training, teams should select relevant features, include enough examples, capture variation, and clean missing, incomplete, or inconsistent values. In Agent Platform, imported datasets get an overview, and the service automatically detects variable types, but each column's variable type and nullability should still be reviewed.
Recommended minimum example data scales with problem complexity: classification needs 50 rows times the number of features, forecasting needs 5,000 rows times the number of features and 10 unique values in the time series identifier column times the number of features, and regression needs 200 times the number of features. More training examples generally improve outcomes, and a binary classification model needs less data than a multi-class model because predicting one class from two is simpler than predicting among many. A dataset should also capture the diversity of the problem space, because a retail model trained only on winter purchase data may not predict summer clothing preferences well.
When selecting features, teams should avoid noise such as randomly assigned identifier columns, understand each feature and its values, and remove target columns that are not part of the current inference problem. They should also check whether a feature could lead to biased or unfair decisions for marginalized groups.
Data leakage happens when training uses input features that reveal information about the target that is not available when the model is served. It can be detected when a feature is highly correlated with the target column, such as using a future subscription payment to predict whether a customer will subscribe. Leakage can make test performance look strong while production performance fails, because the leaked feature is not available at serving time.
Training-serving skew happens when the input features used during training differ from those provided at serving time, causing poor model quality in production. Examples include training on weekly temperatures to predict hourly temperatures, or always providing student grades during training but not at serving time. To prevent both problems, teams must know what each data field means before using it as a feature, flag high correlations in the Train tab for review, and provide only input features that are available in the exact same form at serving time.
Exam tip: Data leakage can make test results look strong even though the model will fail in production, because the leaked feature is not available at serving time.
A dataset split into a training set, a validation set, and a test set gives each part a different role in the model lifecycle. If splits are not specified, Agent Platform automatically uses 80% of images for training, 10% for validating, and 10% for testing, though the default split depends on the model type and manual splits can be used for more control. The lifecycle order is fixed: first train, then tune, then test on unseen data.
Using the training set to tune structure can make the model overfit and generalize poorly, so a separate validation set helps the model handle new examples. Manual splitting is a good choice when more control is needed or when specific examples must be included in a certain part of the lifecycle. After training, the model's performance is summarized, and if the model acts unexpectedly before or after deployment, debugging is often more about debugging the data than the model itself.
The GIGO principle reduces business value because poor data makes model predictions unreliable, so decisions based on those predictions can be wrong, unfair, or costly. The GIGO principle connects data integrity directly to trust in AI-driven insights: accurate, complete, relevant, and representative training data is what allows a model to generalize and produce useful results. Teams should therefore treat data quality as an ongoing responsibility before training and after deployment, returning to the data whenever model behavior is unexpected.
Data quality characteristics and preparation determine whether a machine learning model can learn true patterns and make reliable predictions. High-quality data is relevant to the question being answered, large enough for the problem type, varied enough to cover real cases, and clean, consistent, and correctly labeled. Preparation removes missing values, errors, and mismatches between training and real-world use before training starts.
Relevant, sufficient, and diverse data gives the model the right signals to identify patterns that generalize to new examples. A feature is an input attribute used for model training, and features must be relevant to the problem because they are how the model finds patterns. For fraud prediction, relevant features include transaction details like buyer, seller, amount, date and time, and items purchased, plus history about the buyer and seller.
For retail email marketing, relevant feature columns might include the list of items purchased with brands, categories, prices, and discounts, the number of items purchased and sum of money spent in the last day, week, month, and year, the total number sold and total in stock each day for each item, whether a promotion is running on a particular day, and the known demographic profile of the shopper. Sufficient data volume scales with problem complexity, and more training examples generally produce better outcomes. Binary classification needs less data than multi-class prediction because choosing between two classes is simpler than choosing among many.
Recommended minimums for example data are 50 rows times the number of features for classification, 5,000 rows times the number of features and 10 unique values in the time series identifier column times the number of features for forecasting, and 200 rows times the number of features for regression. Diverse data captures variation across the problem space so the model can handle new or less common cases. A retail model trained only on winter purchase data will not successfully predict summer clothing preferences, while more diverse training examples help the model generalize beyond the exact examples it was shown.
Clean, consistent, and correctly labeled data ensures the model learns from accurate signals rather than gaps, errors, and formatting differences. Agent Platform (opens in a new tab) can handle some missing values and can impute missing values for forecasting, but results are more optimal when all values and rows are available. Timeliness and consistency matter because forecasting requires a consistent interval between training rows, and inconsistent spelling, abbreviations, and formatting make the same value look like different values.
Proper labeling connects each example to the correct target the model must learn. Data can be imported into Agent Platform from a computer or from Cloud Storage in CSV or JSON Lines format with labels and bounding boxes inline. Unlabeled images can be uploaded and annotated in the Google Cloud console, and multiple annotation sets can be managed for the same images. For example, one annotation set can hold bounding box and label information for object detection while another holds only labels for classification.
Cleanup before training focuses on three checks:
After importing, the dataset overview should be reviewed to confirm each column has the correct variable type and correct nullability. Agent Platform automatically detects variable type from column values, but each column should still be reviewed. Nullability determines whether a column can have missing or NULL values, so it directly affects how gaps are handled during training.
Data leakage and training-serving skew are data mismatches that create strong test performance but poor quality in production. Data leakage happens when training features contain information about the target that will not be available when the model is served. Training-serving skew happens when the features used during training differ in form or frequency from the features provided at serving time.
| Problem | What happens | Example |
|---|---|---|
| Data leakage | A training feature leaks future or unavailable information about the target, often shown by very high correlation with the target column. | Predicting next-month subscription sign-up while including that customer's future subscription payment as an input feature. |
| Training-serving skew | Training inputs differ from serving inputs, so the model faces unfamiliar data after deployment. | Training an hourly temperature model with only weekly temperature data, or training a dropout model with student grades that are not provided at serving time. |
Both problems are prevented by understanding what each feature means before using it as a feature. High correlations should be flagged for review because they can signal leakage, and only features available in the exact same form at serving time should be provided for training. This discipline keeps evaluation honest and protects the reliability of AI-driven insights after deployment.
Reviewed data splits for training, validation, and testing ensure the model learns parameters, tunes structure, and is evaluated without memorizing the training data. The training set holds the vast majority of data and is the data the model sees to learn parameters such as the weights of connections between nodes of a neural network. The validation set, sometimes called the dev set, is also used during training to tune hyperparameters, which are variables that specify the model's structure. The test set is not involved in the training process at all and provides a final check on unseen data.
Using a separate validation set to fine-tune structure helps the model generalize better than tuning on the training set alone. Tuning only on the training set makes the model overly focused on that exact data and less able to handle examples that do not match it exactly. Manual splits can be specified in the CSV or JSON Lines import file when the default 80/10/10 split is not desired.
Exam tip: If splits are not specified manually, Agent Platform uses 80% of images for training, 10% for validating, and 10% for testing.
The Garbage In, Garbage Out (GIGO) principle means a machine learning model learns from the data it is trained on, so inaccurate, incomplete, noisy, biased, or unrepresentative inputs produce unreliable predictions. A model does not automatically correct bad data because it finds patterns in whatever examples it receives, including wrong or misleading ones.
Data leakage means training uses input features that reveal information about the target that is unavailable when the model is served, such as using a future subscription payment to predict subscription sign-up, which causes strong test performance but poor production performance. Training-serving skew means training-time input features differ from serving-time input features, such as training only on weekly temperatures to predict hourly temperatures, which causes poor model quality in production.
The recommended minimums are 50 rows times the number of features for classification, 5,000 rows times the number of features plus 10 unique values in the time series identifier column times the number of features for forecasting, and 200 times the number of features for regression. More training examples generally improve outcomes, and binary classification generally needs less data than multi-class classification because predicting one class from two choices is less complicated than predicting one class from many.
The training set holds the vast majority of data and is what the model sees during training to learn parameters such as the weights of connections between neural network nodes. The validation set, sometimes called the dev set, is used during training to tune hyperparameters which specify the model structure, while the test set is not involved in training at all and provides a final check on unseen data to estimate real-world performance.
A retail organization is training a machine learning model to predict customer purchasing behavior. To accelerate the project timeline, the team imports raw historical transaction records directly into the training pipeline without preparation. The dataset contains the following issues:
When evaluating the trained model, the organization finds that its predictions are erratic and unreliable for guiding marketing decisions.
Which action should the team take to address the root cause of the model's poor performance?
Include the test dataset in the iterative training process so the neural network can adjust its weights to account for the unformatted transaction records
Switch from asynchronous batch inference to synchronous online inference to eliminate processing delays that degrade prediction accuracy
Lower the classification score threshold during model evaluation to compensate for the missing demographic values and increase prediction confidence
Clean the training dataset by resolving missing values, standardizing category text, and removing noisy unique identifier columns before retraining
Last updated