Bayesian Experiments
Bayesian Testing in Statsig
Experiments are frequentist by default. To switch to Bayesian mode, go to Advanced Settings.
The experiment type cannot be modified once the experiment starts.
Deep dive analysis should also reflect Bayesian statistics
Informed Bayesian
Bayesian experiments allow you to specify a prior belief on the relative average treatment effect. Statsig will combine the prior distribution with the observed data to display the prior-adjusted results. You can enable this by selecting the option to "use informative priors".
Implementation Details
Denote as the prior distribution, where is the average treatment effect and is the standard error. Similarly, as the observed distribution.
The posterior distribution is then calculated as