No momento, esta página não está disponível em português
Risk-team icon

Run A/B test experiments

Compare the performance of two risk profiles.

Risk experiments let you A/B test two risk profiles against each other to determine which performs better.

When you run a risk experiment, a proportion of your live transactions is randomly assigned to either your current risk profile, or a new profile. Risk experiments are not simulations. They are performed on real transactions, processed with your live Adyen account.

Experiments can be stopped at any time. After running an experiment, you can view the results to make a data-driven decision on which profile performs better.

Requirements

Before you begin, take into account the following requirements and limitations.

Requirement Description
Integration type Make sure that you have built an online payments integration, that risk is enabled, and that you have enabled premium features.
Customer Area roles Make sure that you have the following role:
  • Risk admin
Limitations You can only run one risk experiment at a time, and only one risk experiment can be active per merchant account.

During a risk experiment, Dynamic 3D Secure rules in the variant profile that are linked to risk checks will not trigger.
Setup steps Before you begin:
  • Set up risk management.
  • Create at least two risk profiles, one of which must be a Protect risk profile.

Experiments with classic risk profiles

After you transition to Protect, the latest risk engine, your existing RevenueProtect risk profiles will be set to read-only. This means that you can no longer create a new RevenueProtect risk profile or make changes to a RevenueProtect risk profile.

You can still run an experiment comparing a Protect risk profile to an existing RevenueProtect risk profile.

Start risk experiment

A risk experiment compares the performance of two risk profiles:

  • Base profile. The risk profile you currently apply to transactions.
  • Variant profile. This is a risk profile that you want to test. This could be a new risk profile, or a variation of your Base profile. If you do not have a Variant profile, you need to create one before starting the experiment.

To start a risk experiment, in your Customer Area:

  1. Go to Revenue & risk > Experiments > Create new experiment > Risk.

  2. Enter an experiment Name, and select:

    • Merchant account: Transactions from this account are included in the risk experiment.
    • Variant profile: The risk profile you want to test against your Base profile.

  3. Set a traffic percentage. This is the proportion of transactions that you want to include in your experiment.

    Transactions in your experiment will be evenly split between your Base and Variant profiles. For example, if you set a Traffic percentage of 20%, your Base profile is applied to 10% of your transactions, and your Variant profile is applied to 10%.

  4. Select Create experiment to start running your risk experiment.

Your experiment will continue running until you stop the experiment. You can keep adding shopper attributes to risk lists.

A risk experiment will also stop if you make any risk configuration changes to your Base profile, your Variant profile, or if you disable premium features.

Transactions where you explicitly assign a risk profile in the payment request are excluded from risk experiments.

View experiment results

You can view data from running and completed experiments. The results of your running experiment are updated every 15 minutes.

To view data from an experiment in your  Customer Area:

  1. Go to Revenue & risk > Experiments > Risk.
  2. Select Running or Completed and then select an experiment to view the results.

For each experiment, you can view and compare the performance of your Base and Variant profiles.

This includes the following data for each risk profile:

  • Transactions: Number of transactions processed.
  • Refused by issuer: Percentage of transactions that were refused by the card issuer.
  • Authorized: Percentage of transactions that were successfully authorized.
  • Blocked by risk: Percentage of transactions blocked by risk.
  • NoF: Percentage of transactions that were reported as fraudulent by a Notification of Fraud (NoF).

More in-depth data from the experiment is available when you select See details. These data include chargeback and dispute rates for each profile, and a breakdown of why transactions were refused by issuers.

Stop risk experiment

You can stop an experiment at any time. We recommend letting an experiment run for at least one month, so that you get accurate data for fraud and chargeback rates.

To stop an experiment, in your Customer Area:

  1. Go to Revenue & risk > Experiments.
  2. In the Risk tab, go to your experiment on the list, and select See details.
  3. Select Stop experiment.

After you stop an experiment, you can decide which risk profile you want to assign to a merchant account for future transactions.

View experiment transactions

To see a list of transactions that were part of an experiment, in your Customer Area:

  1. Go to Revenue & risk > Experiments > Risk.
  2. Select an experiment from the Running or Completed tab, and then select See details.
  3. Note down or copy the experiment reference.
  4. In the search bar, type riskExperimentReference: and then type or paste your experiment reference.
  5. Select Enter to search.

A list of transactions that were part of the experiment will open in the Payments list.

See also