Rich-text Reply

Hi, how to the calculations of statistical significance level, and how to set the rules?

jrudy 05-13-16

Hi, how to the calculations of statistical significance level, and how to set the rules?

Hi, i want ot know how to the calculations of statistical significance level, and how to set the rules? 

why statistical significance number of variation 1 shows <1, but our report shows 91%? 

Level 1

Lyndsey 05-16-16
 

Re: Hi, how to the calculations of statistical significance level, and how to set the rules?

Hi There!

Optimizely's significance is determined by our Stats Engine, which may be why you are getting different results from any other sample calculator.

Here's more information on the Optimizely Stats Engine and how we determine sample size and significance:
1. https://www.optimizely.com/resources/sample-size-calculator/
2. https://help.optimizely.com/Analyze_Results/Stats_Engine%3A_How_Optimizely_calculates_results_to_ena...

If you would like to adjust your significance setting in the project, you can do that as well. Here is more documentation on how you can adjust that setting, as well as the tradeoffs for doing so: https://help.optimizely.com/Analyze_Results/Stats_Engine%3A_How_Optimizely_calculates_results_to_ena...
Thanks,
Lyndsey
Technical Support Engineer
Optimizely
Optimizely
David_Orr 05-16-16
 

Re: Hi, how to the calculations of statistical significance level, and how to set the rules?

Hi,

Here is a link to a specific section in our "Stats Engine: How Optimizely calculates results to enable business decisions" article for info on how to change the statistical significant setting:

https://help.optimizely.com/Analyze_Results/Stats_Engine%3A_How_Optimizely_calculates_results_to_ena...

The article should answer any questions regarding how Stats Engine determines the Statistical Significance of a variation and how it differs from traditional stats calculators.

David
Senior Technical Support Engineer
Optimizely