Your experiment is currently collecting data. This article will help you understand how to interpret results, identify winning signals, and evaluate full-funnel impact before making a decision.
Results are available at two levels:
1️⃣ CPP Experiments dashboard is a quick overview of all your experiments. Here, you can easily see which experiments have a winner, which are still collecting data, and where meaningful differences are emerging. This view helps you quickly focus on the experiments that matter most.
2️⃣ Experiment details popup is a deeper view of a single experiment. It includes performance trends over time, all key metrics broken down by variation, along with full-funnel validation. This is where you analyze results in detail and make your final decision.
CPP Experiments Dashboard
You can access the CPP Experiments dashboard from the left sidebar by going to Optimization Hub > CPP Experiments.
The dashboard gives you a centralized view of all your experiments, including their current progress, probability signals, health status, key metrics, and settings. This allows you to easily understand experiment performance without opening each experiment individually.
Health Check
The Health Check column is the first place to look at to identify if anything needs your attention. You will see one of two statuses 👇
Health check | What it means |
Healthy | No issues detected. Your experiment is collecting reliable data. |
Alert indicator | At least one issue requires your attention. Hover over the indicator icon to view more details about the detected issues. You can learn more about alerts in 📖 Monitoring Your Experiment. |
Progress Statuses
Each running experiment displays a progress message that updates automatically as more data is collected, keeping you informed about the current experiment status.
Status | Description |
Collecting data
| The experiment has just started. The data volume is still below the minimum required for the system to begin calculating probabilities. |
No leading variation yet
| The system has collected enough data to begin analysis, but no variation currently shows a clear advantage. |
Variation X is leading with Y% probability
| One variation is currently performing better based on your primary metric. The probability indicates how strong the signal is. This is an early indicator of performance, not a final result. |
💡Tip: How you interpret early signals depends on the testing method used in your experiment.
Parallel: Both variations collect data simultaneously under the same market conditions, so early signals reflect a true side-by-side comparison. In some cases, the Apple Ads auction may naturally send more traffic to one variation if it predicts stronger conversion performance, which can itself be a useful signal. Probabilities may still change as more data comes in, but early trends in Parallel experiments are usually directionally meaningful.
Switch / Dayparting: Variations run at different times, so early signals become reliable only after each variation has collected data across comparable periods, ideally including both weekdays and weekends. Before that point, one variation may look stronger simply because it ran during a higher-traffic period, rather than because the CPP itself performs better.
❗Good to know: Conversion rates naturally change from day to day. For example, a page with a true 5% download rate might show 4.3% one day and 5.7% the next. This is expected. The system takes this into account by waiting for enough data over time before confirming a result.
Key Metrics
The dashboard shows several key metrics that help you understand how your variations are performing.
Primary metric: The current value of your main performance metric for each variation.
Estimated uplift vs control: A model-based estimate of how much each variation improves the primary metric compared to the control.
Probability to Win: Most useful in 3–4 variation tests. It shows how likely each variation is to be the best overall performer. This helps you quickly see which variation is currently leading when comparing multiple options.
Probability to Beat Control: The main metric for 2-variation tests. It shows how likely a variation is to outperform the control. A high value (for example, 95%) means the system is very confident that this variation performs better than your current baseline. It answers the question: “Is my test variation better than what I have now?”
You also have access to standard performance metrics, such as Spend, Impressions, TTR, Download Rate (Tap-Through), Impression to Download Rate (Tap-Through), CPT, and Avg CPA (Tap-Through), for deeper analysis.
Deltas Dropdown
In the dashboard, you can turn on the Deltas toggle to compare performance differences across all experiments at once.
There are two comparison types available, each answering a different question:
Change vs control variation: Shows how each variation performs compared to the control within the same experiment. This is your core A/B comparison, helping you determine whether the test variation performs better than the original version.
Change vs ad group historical benchmark: Shows how each variation performs compared to the ad group’s baseline performance before the experiment started. This helps you understand whether the experiment is improving or negatively impacting overall ad group performance.
Using Deltas makes it easy to quickly spot which experiments show meaningful performance differences without opening each one individually.
Final Statuses
When an experiment completes or is stopped, the dashboard shows a final status that helps guide your next steps.
Status | What it means | What to do |
Winner defined | One variation is confirmed as the winner. This is highlighted in green and includes the winning probability. | Review full-funnel metrics and then apply the winner. |
Inconclusive – variation is leading | One variation is ahead, but the system cannot confirm a statistically significant winner. The leading variation and its probability are shown. | Review performance across all metrics. You may choose to act on it with some caution or rerun the experiment with more traffic. |
Inconclusive – no difference | The experiment collected enough data but did not find a meaningful difference between variations. | The variations likely perform similarly. You can keep your current CPP or decide based on other factors such as brand or seasonality. |
Not enough data | The experiment ended before enough data was collected for analysis. | Consider rerunning the experiment with a higher-traffic ad group, fewer variations, or a longer duration. |
❗Good to know: You can stop any running experiment using the Stop experiment button.
When you stop an experiment:
If not enough data has been collected → Not enough data
If enough data has been collected but no winner is confirmed → Inconclusive (may show a leading variation with its probability)
The system evaluates results based on the data available at the time of stopping. If a variation was leading, this will still be reflected in the results, but it won’t be marked as a confirmed winner.
CPP Experiment Details Popup
Click Show Analytics on any experiment in the dashboard to open the experiment details. This is where you can go deeper into performance: daily trends, all metrics broken down by variation, and full-funnel validation with post-install data. Use this view when you’re ready to analyze a specific experiment and make a decision.
Settings
The top section shows your experiment setup, including the number of variations, primary metric, testing method, start and end dates, and estimated duration. This reflects the same configuration you defined when creating the experiment.
Progress & Outcome
Below the Settings section, the Progress & Outcome area shows:
Status text — current experiment status
Progress bar — how far the experiment has progressed
Days counter — number of days passed and remaining duration
Stop experiment button — available for running experiments
For completed experiments with a winner, this section will show a message such as:
“Variation X is the winner, with Y% probability”, highlighted in green.
Probability to Win
Each variation is shown with a probability to win, displayed as a percentage with a visual bar. This is the main signal used to understand how likely each variation is to be the best performer on your primary metric.
What it means:
For example, if Variation B shows 78%, it means that based on the data collected so far, there is a 78% chance that Variation B is the best performer on your primary metric. The remaining 22% reflects the possibility that the difference is due to normal statistical fluctuation.
This probability is calculated using a Bayesian model and is continuously updated as new data is received. Early in the experiment, it may fluctuate more, but it becomes more stable as additional data is collected.
Primary and Key Metrics
Next to the probability section, you can see the current value of your primary metric per variation (for example, CPA: $5.45), along with supporting metrics such as Impressions, TTR, Download Rate (Tap-Through), and others. Each metric includes delta indicators for comparison.
The Uplift vs Control column shows the model-based uplift for each variation compared to the control.
Trend Analysis
A time-series chart shows performance over time for each variation based on the selected metric. You can switch between daily, weekly, and monthly views using the dropdown, and choose different chart types such as line or bar charts.
Use Trend analysis to understand performance patterns:
Is one variation consistently leading, or only improving recently?
Are variations moving in the same direction or diverging?
Are there specific periods where performance changed noticeably?
Metric Deep Dive
The table at the bottom provides a detailed breakdown for each variation, including Product Page, Creative Source, Ad Placement, Spend, Impressions, Taps, Installs, and more.
You can:
Turn on the Deltas toggle to see percentage differences between variations
Use Columns to choose which metrics to display
Click Open in Ads Manager to go directly to the CPP dashboard in Ads Manager
Post-install metrics such as ROAS and conversion rates are also available here. These metrics are not used to determine the winner (which is based only on the primary metric), but they help you understand the overall business impact.
How To Validate Results In The Experiment Details Popup
The primary metric decides the winner, but it doesn’t tell the full story. CPP performance is connected across the funnel. For example, stronger screenshots may increase TTR, but they can also attract lower-quality users, which may lower Download Rate or increase CPA. Before you make a decision:
Check the Metric Deep Dive table and turn on the Deltas toggle
Review all key metrics, not just the primary one, including TTR, Download Rate, CPA, Spend, and post-install results
Watch for trade-offs between metrics. For example, one variation may improve Download Rate, but increase CPA, while another may keep CPA stable with similar TTR
Look at the Trend analysis chart to see whether performance is steady over time or only improves near the end of the experiment.
Results that are consistent throughout the full experiment are generally more reliable than short-lived spikes or late changes in performance.
💡 Good to know: Download Rate is often more stable than TTR in CPP experiments because it reflects a deeper action (install vs. tap). TTR can change more due to auction behavior, while Download Rate better reflects real user interest in the product page.
Best practices
Review full-funnel metrics before acting on a winner — a strong result on the primary metric may come with trade-offs in other areas
Use Deltas on the dashboard to quickly spot meaningful differences across experiments
Check the Health status before making a final decision to make sure the test ran as expected
Use early signals to build context. In Parallel experiments, early signals reflect real-time side-by-side data. In Switch/Dayparting experiments, they become more reliable once each variation has collected comparable time periods. In both cases, early signals help you anticipate results before the experiment finishes
Next steps
You now have your result and have validated the impact. Next, learn how to further optimize using your winning variation.
Next article: Scale Your Winning Variation