Risk experiments let you A/B test two risk profiles against each other to determine which performs better.
When you run a risk experiment, a proportion of your transactions are randomly assigned to either your current risk profile, or a new profile. You can then make a data-driven decision on which profile produces better authorization, refusal, and fraud rates.
Experiments can be stopped at any time, allowing you to apply your preferred risk profile to all transactions.
Watch our video for a demonstration of how risk experiments work.
You can only run one risk experiment at a time, and only one risk experiment can be active per merchant account.
Risk experiments are not simulations. They are performed on real transactions, processed with your live Adyen account.
Start risk experiment
A risk experiment compares the performance of two risk profiles:
- Base profile. The risk profile you currently apply to transactions.
- Variant profile. This is a risk profile that you want to test. This could be a new risk profile, or a variation of your Base profile. If you do not have a Variant profile, you need to create one before starting the experiment.
To start a risk experiment:
-
Log in to the Customer Area.
-
Go to Revenue & risk > Experiments > Set up experiment.
-
Enter an Experiment name, and select:
- Merchant account: Transactions from this account are included in the risk experiment.
- Variant profile: The risk profile you want to test against your Base profile.
-
Set a Traffic percentage. This is the proportion of transactions that you want to include in your experiment.
Transactions in your experiment will be evenly split between your Base and Variant profiles. For example, if you set a Traffic percentage of 20%, your Base profile is applied to 10% of your transactions, and your Variant profile applied to 10%.
-
Select Create experiment to start running your risk experiment.
Your experiment will continue running until you stop the experiment. You can keep adding shopper attributes to Block/Trust referral lists.
A risk experiment will also stop if you make any risk configuration changes to your Base profile, your Variant profile, or if RevenueProtect premium is disabled.
If a risk profile is explicitly assigned in the /authorise or /payments request, the payment will not be included in the experiment.
View experiment results
The results of your experiment are updated every 15 minutes. You can view data from the experiment in your Customer Area:
- Log in to the Customer Area.
- Go to Revenue & risk > Experiments to view your current and finished risk experiments.
For each experiment, you can view and compare the performance of your Base and Variant profiles:
This includes the following data for each risk profile:
- Transactions: Number of transactions processed.
- Refused by risk: Percentage of transactions with a fraud score exceeding 100.
- Refused by issuer: Percentage of transactions that were refused by the card issuer.
- Authorizations: Percentage of transactions that were successfully authorized.
- Fraud: Percentage of transactions that were reported as fraudulent.
More in-depth data from the experiment is available by clicking See details. These data include chargeback and dispute rates for each profile, and a breakdown of why transactions were refused by issuers.
For more information about running risk experiments, or interpreting the results, contact our risk team.
Stop risk experiment
You can stop an experiment at any time. We recommend letting an experiment run for at least one month, so that you get accurate data for fraud and chargeback rates.
When you stop an experiment, you will be asked to choose which profile you want to apply to future transactions processed with this merchant account.
-
Log in to the Customer Area.
-
Go to Revenue & risk > Experiments.
-
Navigate to your experiment on the list, and select See details.
-
Select Stop experiment.
You are prompted to choose which risk profile you want to apply to future transactions.
-
Select your preferred profile, then select Proceed to use this profile for future transactions.