Start an experiment after getting an Adyen Uplift recommendation to see the effect of making changes to your integration. A recommendation can include the option to start an experiment when changing one or more settings is likely to lead to an uplift in performance or revenue. You get recommendations and can start experiments that apply to your integration, and that are relevant for the countries or regions you operate in.
Experiments are not simulations. They are performed on real transactions, processed with your live Adyen account.
Requirements
Before you begin, take into account the following requirements and limitations.
| Requirement | Description |
|---|---|
| Integration type | Make sure that you have built an online payments integration that supports Adyen Uplift. For more information, see the detailed requirements. |
| Customer Area roles | All Customer Area users can see the Experiments page in the Revenue & risk section. To start an experiment following an optimization recommendation, you need the following role:
|
| Limitations | You can only start optimization experiments following a recommendation. |
Experiment types
There are different types of experiments you can run to validate the effect of making changes to settings on your account. Based on a recommendation, you can start an optimization experiment to preview the effect of changing one or more settings. Or, you can run a risk experiment to compare the performance of two risk profiles.
Optimization experiments
When a recommendation gives you the option, you can start an experiment on a portion of your transactions.
By analyzing the results during the experiment, or after you stop it, you can validate the effect of any changes. You can then choose to change to the recommended settings, or decide not to act on the recommendation.
Risk experiments
Risk experiments let you A/B test two risk profiles against each other to determine which performs better. For example, you can run a risk experiment to see if changing, adding, or disabling a risk rule has the effect you want on your fraud rate.
See run an A/B test experiments for details on how to start, evaluate and stop risk experiments. Risk experiments require Protect premium.
Start an experiment
To start an experiment, in your Customer Area:
-
Go to Revenue & risk > Uplift overview > Recommendations.
-
Select a recommendation and then select Create experiment.
This button is only available when the recommendation allows you to experiment, and when you meet the requirements to run the experiment. For example, certain recommendations require that you process payments in a specific country or region, or that you have enabled a payment method or feature. The recommendation contains details about what is required before you can create an experiment. -
For each experiment, do the following:
- Enter an experiment Name.
- Select the Merchant account from which you want to include transactions for the experiment. If you are logged in on the company account, you can select multiple merchant accounts.
-
Set a traffic percentage. This is the proportion of transactions that you want to include in your experiment.
Transactions in your experiment will be evenly split between your current (Variant A) and recommended (Variant B) setting. For example, if you set a Traffic percentage of 20%, the current setting is applied to 10% of your transactions, and the recommended setting is applied to 10%.
-
Select Create to start running your experiment.
While the experiment is running, the user who created the experiment can edit the name of the experiment and the traffic percentage.
Your experiment will continue running until you stop the experiment.
View experiment results
You can view data from running and completed experiments. At the start of the experiment, it takes about four hours before you can see the first results. After that, the results of your running experiment are updated regularly.
To view data from an experiment in your Customer Area:
- Go to Revenue & risk > Experiments > Optimization.
- Select Running or Completed and then select an experiment to view the results.
For each experiment, you can view and compare the performance of the different setting variants. The data that you can compare depends on the experiment.
Stop an experiment
You can stop an experiment at any time. We recommend letting an experiment run for at least one month on a considerable percentage of traffic to get more accurate data.
To stop an experiment, in your Customer Area:
- Go to Revenue & risk > Experiments.
- In the Optimizations tab, go to your experiment on the list, and select End.
- Select End to stop the experiment and keep your current setting, or select End & change setting to act on the recommendation and change your settings.