I have a question around statistical significance. One test, same number of visitors, 2 different goals. The goal with FEWER total conversions and a SMALLER lift has HIGHER statistical significance. How is that possible?
I have also plugged the goal's numbers into an Excel calculator for statistical significance. Optimizely says the goal has 1% significance while the Excel formula says 100% significance.
Screenshots and further explanation are in this Google Doc.
Optimizely's Stats Engine will "slow down" non-primary metrics to avoid false positives and help you make better business decisions based on actual anomalies and non-random chance.
You can read more about stats engine here: https://www.optimizely.com/statistics/
- Tom Fuertes | CTO @ CROmetrics / LinkedIn
"Most Impactful Use of Personalization" and "Experience of the Year" Optie award winner.