{"title":"Run A\/B test experiments","category":"default","creationDate":1593079020,"content":"<p>Risk experiments let you A\/B test two risk profiles against each other to determine which performs better.<\/p>\n<p>When you run a risk experiment, a proportion of your live transactions is randomly assigned to either your current risk profile, or a new profile. Risk experiments are <em>not<\/em> simulations. They are performed on real transactions, processed with your live Adyen account.<\/p>\n<p>Experiments can be <a href=\"#stop-risk-experiment\">stopped<\/a> at any time. After running an experiment, you can <a href=\"#view-experiment-results\">view the results<\/a> to make a data-driven decision on which profile performs better.<\/p>\n<h2>Requirements<\/h2>\n<p>Before you begin, take into account the following requirements and limitations.<\/p>\n<table>\n<thead>\n<tr>\n<th style=\"text-align: left;\">Requirement<\/th>\n<th style=\"text-align: left;\">Description<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: left;\"><strong>Integration type<\/strong><\/td>\n<td style=\"text-align: left;\">Make sure that you have built an <a href=\"\/online-payments\/build-your-integration\/\">online payments integration<\/a>, that <a href=\"\/risk-management\/configure-risk-settings\/\">risk is enabled<\/a>, and that you have enabled <a href=\"\/risk-management\/configure-risk-settings\/#risk-general\">premium features<\/a>.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\"><strong><a href=\"\/account\/user-roles\/#risk\">Customer Area roles<\/a><\/strong><\/td>\n<td style=\"text-align: left;\">Make sure that you have the following role: <ul><li markdown=\"1\"><strong>Risk admin<\/strong><\/li><\/ul><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\"><strong>Limitations<\/strong><\/td>\n<td style=\"text-align: left;\">You can only run one risk experiment at a time, and only one risk experiment can be active per merchant account. <br><br>During a risk experiment, <a href=\"\/risk-management\/dynamic-3d-secure\/#configure-a-dynamic-3d-secure-rule\">Dynamic 3D Secure rules<\/a> in the <a href=\"#start-risk-experiment\">variant profile<\/a> that are linked to risk checks will not trigger.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\"><strong>Setup steps<\/strong><\/td>\n<td style=\"text-align: left;\">Before you begin: <ul><li markdown=\"1\">Set up <a href=\"\/risk-management\/\">risk management<\/a>.<\/li><li markdown=\"1\">Create and configure at least two <a href=\"\/risk-management\/configure-your-risk-profile\">risk profiles<\/a>.<\/li><\/ul><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2 id=\"start-risk-experiment\">Start risk experiment<\/h2>\n<p>A risk experiment compares the performance of two risk profiles:<\/p>\n<ul>\n<li><strong>Base profile<\/strong>. The risk profile you currently apply to transactions.<\/li>\n<li><strong>Variant profile<\/strong>. This is a risk profile that you want to test. This could be a new risk profile, or a variation of your Base profile. If you do not have a Variant profile, you need to <a href=\"\/risk-management\/create-a-risk-profile\">create one<\/a> before starting the experiment.<\/li>\n<\/ul>\n<p>To start a risk experiment, in your <a href=\"https:\/\/ca-live.adyen.com\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" class=\"external-link no-image\">Customer Area<\/a>:<\/p>\n<ol>\n<li>\n<p>Go to <strong>Revenue &amp; risk<\/strong> &gt; <strong>Experiments<\/strong> &gt; <strong>Create new experiment<\/strong> &gt; <strong>Risk<\/strong>.<\/p>\n<\/li>\n<li>\n<p>Enter an experiment <strong>Name<\/strong>, and select:<\/p>\n<ul>\n<li><strong>Merchant account<\/strong>: Transactions from this account are included in the risk experiment.<\/li>\n<li><strong>Variant profile<\/strong>: The risk profile you want to test against your Base profile.<\/li>\n<\/ul>\n<\/li>\n<li>\n<p>Set a traffic percentage. This is the proportion of transactions that you want to include in your experiment.<\/p>\n<p>Transactions in your experiment will be evenly split between your Base and Variant profiles. For example, if you set a <strong>Traffic percentage<\/strong> of 20%, your Base profile is applied to 10% of your transactions, and your Variant profile is applied to 10%.<\/p>\n<\/li>\n<li>\n<p>Select <strong>Create experiment<\/strong> to start running your risk experiment.<\/p>\n<\/li>\n<\/ol>\n<p>Your experiment will continue running until <a href=\"#stop-risk-experiment\">you stop the experiment<\/a>. You can keep adding shopper attributes to <a href=\"\/risk-management\/configure-your-risk-profile\/risk-lists\/\">risk lists<\/a>.<\/p>\n<div class=\"notices yellow\">\n<p>A risk experiment will also stop if you make any risk configuration changes to your Base profile, your Variant profile, or if you disable premium features.<\/p>\n<\/div>\n<div class=\"notices blue\">\n<p>Transactions where you <a href=\"\/risk-management\/create-a-risk-profile\/#dynamically-assign-a-risk-profile-to-a-payment\">explicitly assign<\/a> a risk profile in the payment request are excluded from risk experiments.<\/p>\n<\/div>\n<h2 id=\"view-experiment-results\">View experiment results<\/h2>\n<p>You can view data from running and completed experiments. The results of your running experiment are updated every 15 minutes.<\/p>\n<p>To view data from an experiment in your \u00a0<a href=\"https:\/\/ca-live.adyen.com\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" class=\"external-link no-image\">Customer Area<\/a>:<\/p>\n<ol>\n<li>Go to <strong>Revenue &amp; risk<\/strong> &gt; <strong>Experiments<\/strong> &gt; <strong>Risk<\/strong>.<\/li>\n<li>Select <strong>Running<\/strong> or <strong>Completed<\/strong> and then select an experiment to view the results.<\/li>\n<\/ol>\n<p>For each experiment, you can view and compare the performance of your Base and Variant profiles.<\/p>\n<p>This includes the following data for each risk profile:<\/p>\n<ul>\n<li><strong>Transactions<\/strong>: Number of transactions processed.<\/li>\n<li><strong>Refused by issuer<\/strong>: Percentage of transactions that were refused by the card issuer.<\/li>\n<li><strong>Authorized<\/strong>: Percentage of transactions that were successfully authorized.<\/li>\n<li><strong>Blocked by risk<\/strong>: Percentage of transactions blocked by risk.<\/li>\n<li><strong>NoF<\/strong>: Percentage of transactions that were reported as fraudulent by a Notification of Fraud (NoF).<\/li>\n<\/ul>\n<p>More in-depth data from the experiment is available when you select <strong>See details<\/strong>. These data include chargeback and dispute rates for each profile, and a breakdown of why transactions were refused by issuers.<\/p>\n<h2 id=\"stop-risk-experiment\">Stop risk experiment<\/h2>\n<p>You can stop an experiment at any time. We recommend letting an experiment run for at least one month, so that you get accurate data for fraud and chargeback rates.<\/p>\n<p>To stop an experiment, in your <a href=\"https:\/\/ca-live.adyen.com\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" class=\"external-link no-image\">Customer Area<\/a>:<\/p>\n<ol>\n<li>Go to <strong>Revenue &amp; risk<\/strong> &gt; <strong>Experiments<\/strong>.<\/li>\n<li>In the <strong>Risk<\/strong> tab, go to your experiment on the list, and select <strong>See details<\/strong>.<\/li>\n<li>Select <strong>Stop experiment<\/strong>.<\/li>\n<\/ol>\n<p>After you stop an experiment, you can decide which risk profile you want to <a href=\"\/risk-management\/create-a-risk-profile#set-a-risk-profile-as-default-action\">assign to a merchant account<\/a> for future transactions.<\/p>\n<h2 id=\"view-experiment-transactions\">View experiment transactions<\/h2>\n<p>To see a list of transactions that were part of an experiment, in your <a href=\"https:\/\/ca-live.adyen.com\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" class=\"external-link no-image\">Customer Area<\/a>:<\/p>\n<ol>\n<li>Go to <strong>Revenue &amp; risk<\/strong> &gt; <strong>Experiments<\/strong> &gt; <strong>Risk<\/strong>.<\/li>\n<li>Select an experiment from the <strong>Running<\/strong> or <strong>Completed<\/strong> tab, and then select <strong>See details<\/strong>.<\/li>\n<li>Note down or copy the experiment reference.<\/li>\n<li>In the search bar, type <code>riskExperimentReference:<\/code> and then type or paste your experiment reference.<\/li>\n<li>Select <strong>Enter<\/strong> to search.<\/li>\n<\/ol>\n<p>A list of transactions that were part of the experiment will open in the <strong>Payments<\/strong> list.<\/p>\n<h2>See also<\/h2>\n<div class=\"see-also-links output-inline\" id=\"see-also\">\n<ul><li><a href=\"\/risk-management\/create-a-risk-profile\"\n                        target=\"_self\"\n                        >\n                    Create a risk profile\n                <\/a><\/li><li><a href=\"\/risk-management\/configure-your-risk-profile\"\n                        target=\"_self\"\n                        >\n                    Configure your risk profile\n                <\/a><\/li><li><a href=\"\/risk-management\/\"\n                        target=\"_self\"\n                        >\n                    Risk management overview\n                <\/a><\/li><li><a href=\"\/uplift\/uplift-experiments\"\n                        target=\"_self\"\n                        >\n                    Adyen Uplift experiments\n                <\/a><\/li><\/ul><\/div>\n","url":"https:\/\/docs.adyen.com\/risk-management\/run-ab-test-experiments","articleFields":{"description":"Compare the performance of two risk profiles.","last_edit_on":"25-06-2020 18:23"},"algolia":{"url":"https:\/\/docs.adyen.com\/risk-management\/run-ab-test-experiments","title":"Run A\/B test experiments","content":"Risk experiments let you A\/B test two risk profiles against each other to determine which performs better.\nWhen you run a risk experiment, a proportion of your live transactions is randomly assigned to either your current risk profile, or a new profile. Risk experiments are not simulations. They are performed on real transactions, processed with your live Adyen account.\nExperiments can be stopped at any time. After running an experiment, you can view the results to make a data-driven decision on which profile performs better.\nRequirements\nBefore you begin, take into account the following requirements and limitations.\n\n\n\nRequirement\nDescription\n\n\n\n\nIntegration type\nMake sure that you have built an online payments integration, that risk is enabled, and that you have enabled premium features.\n\n\nCustomer Area roles\nMake sure that you have the following role: Risk admin\n\n\nLimitations\nYou can only run one risk experiment at a time, and only one risk experiment can be active per merchant account. During a risk experiment, Dynamic 3D Secure rules in the variant profile that are linked to risk checks will not trigger.\n\n\nSetup steps\nBefore you begin: Set up risk management.Create and configure at least two risk profiles.\n\n\n\nStart risk experiment\nA risk experiment compares the performance of two risk profiles:\n\nBase profile. The risk profile you currently apply to transactions.\nVariant profile. This is a risk profile that you want to test. This could be a new risk profile, or a variation of your Base profile. If you do not have a Variant profile, you need to create one before starting the experiment.\n\nTo start a risk experiment, in your Customer Area:\n\n\nGo to Revenue &amp; risk &gt; Experiments &gt; Create new experiment &gt; Risk.\n\n\nEnter an experiment Name, and select:\n\nMerchant account: Transactions from this account are included in the risk experiment.\nVariant profile: The risk profile you want to test against your Base profile.\n\n\n\nSet a traffic percentage. This is the proportion of transactions that you want to include in your experiment.\nTransactions in your experiment will be evenly split between your Base and Variant profiles. For example, if you set a Traffic percentage of 20%, your Base profile is applied to 10% of your transactions, and your Variant profile is applied to 10%.\n\n\nSelect Create experiment to start running your risk experiment.\n\n\nYour experiment will continue running until you stop the experiment. You can keep adding shopper attributes to risk lists.\n\nA risk experiment will also stop if you make any risk configuration changes to your Base profile, your Variant profile, or if you disable premium features.\n\n\nTransactions where you explicitly assign a risk profile in the payment request are excluded from risk experiments.\n\nView experiment results\nYou can view data from running and completed experiments. The results of your running experiment are updated every 15 minutes.\nTo view data from an experiment in your \u00a0Customer Area:\n\nGo to Revenue &amp; risk &gt; Experiments &gt; Risk.\nSelect Running or Completed and then select an experiment to view the results.\n\nFor each experiment, you can view and compare the performance of your Base and Variant profiles.\nThis includes the following data for each risk profile:\n\nTransactions: Number of transactions processed.\nRefused by issuer: Percentage of transactions that were refused by the card issuer.\nAuthorized: Percentage of transactions that were successfully authorized.\nBlocked by risk: Percentage of transactions blocked by risk.\nNoF: Percentage of transactions that were reported as fraudulent by a Notification of Fraud (NoF).\n\nMore in-depth data from the experiment is available when you select See details. These data include chargeback and dispute rates for each profile, and a breakdown of why transactions were refused by issuers.\nStop risk experiment\nYou can stop an experiment at any time. We recommend letting an experiment run for at least one month, so that you get accurate data for fraud and chargeback rates.\nTo stop an experiment, in your Customer Area:\n\nGo to Revenue &amp; risk &gt; Experiments.\nIn the Risk tab, go to your experiment on the list, and select See details.\nSelect Stop experiment.\n\nAfter you stop an experiment, you can decide which risk profile you want to assign to a merchant account for future transactions.\nView experiment transactions\nTo see a list of transactions that were part of an experiment, in your Customer Area:\n\nGo to Revenue &amp; risk &gt; Experiments &gt; Risk.\nSelect an experiment from the Running or Completed tab, and then select See details.\nNote down or copy the experiment reference.\nIn the search bar, type riskExperimentReference: and then type or paste your experiment reference.\nSelect Enter to search.\n\nA list of transactions that were part of the experiment will open in the Payments list.\nSee also\n\n\n                    Create a risk profile\n                \n                    Configure your risk profile\n                \n                    Risk management overview\n                \n                    Adyen Uplift experiments\n                \n","type":"page","locale":"en","boost":18,"hierarchy":{"lvl0":"Home","lvl1":"Risk management","lvl2":"Run A\/B test experiments"},"hierarchy_url":{"lvl0":"https:\/\/docs.adyen.com\/","lvl1":"https:\/\/docs.adyen.com\/risk-management","lvl2":"\/risk-management\/run-ab-test-experiments"},"levels":3,"category":"Risk Management","category_color":"green","tags":["experiments"]},"articleFiles":{"experiment_results.png":"<img alt=\"\" src=\"https:\/\/docs.adyen.com\/user\/pages\/docs\/10.risk-management\/07.run-ab-test-experiments\/experiment_results.png\" \/>"}}