Skip to main content

CPP A/B Experiments: Monitoring Your Experiment

Written by Katia

Your experiment is live. This article explains how to understand alerts, check experiment health, and make sure your results stay reliable while the test is running.

Health Check

In the CPP Experiments dashboard, the Health Check column gives you a quick view of whether your experiment is running normally or if something needs attention.

There are two possible states:

  • Healthy: everything is running as expected, and your experiment is collecting clean, reliable data

  • Alert: something has changed that may affect your experiment setup or results

If you see an alert, you can hover over the icon to open a small popup with more details. This gives you a quick explanation of what changed and why it matters, without needing to dig through the setup manually.

The system continuously monitors your experiment in the background. It tracks changes across campaigns, ad groups, keywords, ads, and traffic distribution. When something could impact your results, it flags it automatically and explains the situation in simple terms so you can act quickly.

What Alert Cards Tell You

Each alert card gives you a simple explanation of what changed and why it matters. In general, it helps you understand:

  • What happened

  • What part of the experiment is affected

  • Whether it needs action or just awareness

Some alerts are more important than others, and the system clearly highlights their level of impact.

Alert Levels

Alerts are grouped into three levels based on their impact:

Need Attention

This is the most important type of alert. It means something is affecting your experiment setup or data collection.

For example, a campaign may be paused, an ad group may not be serving, or traffic may not be properly distributed across variations.

When this happens, it can directly impact the quality of your results. It’s important to review these alerts as soon as possible so your experiment can continue collecting valid data.

Warning

A warning means something has changed that could affect how comparable your variations are, but your experiment is still running.

For example, bids, budgets, or targeting may have been updated during the test.

Warnings do not always require immediate action, but they are worth reviewing to make sure all variations are still set up fairly and consistently.

Information

This type of alert is purely informational. It means a change was recorded, but it does not affect your experiment or its results.

For example, a campaign name or label may have been updated. No action is needed in these cases.

Alert Categories

Each alert also belongs to a category that explains what kind of change happened and why it matters for your experiment:

  • Traffic delivery issue: one of the variations may not be receiving traffic as expected. This can affect whether results are balanced and complete

  • Budget issue: a budget change may have stopped or limited delivery, which can reduce data collection

  • Test conditions changed: settings like bids, keywords, or targeting were modified, which can make variations less comparable

  • Experiment timing changed: schedule updates may shift when traffic is collected, which can impact consistency across the test period

  • Non-impacting recorded: a change was logged, but it does not affect experiment results

These categories help you quickly understand the type of change behind each alert.

How To Keep Your Experiment Healthy

The system automatically monitors your experiment, detects changes, and classifies alerts for you. Still, there are a few things you can do to help ensure your results stay clean, stable, and easy to trust.

What To Avoid During A Test

To keep your experiment consistent across all variations, try to avoid making changes while the test is running:

  • Avoid changing campaign, ad group, keyword, or bid settings, since this can make variations less comparable

  • Avoid updating CPP screenshots or promotional text, since this changes the actual creative being tested

  • Avoid manually pausing or enabling test ad groups, since the system manages experiment delivery automatically

What To Watch Closely

A few simple checks during the experiment can help you stay on top of performance:

  • Keep an eye on the Health Check column, especially in the first few days after launch

  • Act quickly on “Need attention” alerts, since they can affect data collection or block proper measurement

  • Review warnings as signals to double-check consistency, not as urgent failures

  • Expect some traffic fluctuations right after launch, since new test structures are created and Apple Ads needs a short time to stabilize, usually within the first day

By following these guidelines, you can keep your experiment stable, ensure reliable data collection, and run a successful test from start to finish.

Did this answer your question?