Rich-text Reply
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
Statistical significance decreasing
bruab 07-01-16
Statistical significance decreasing
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Hi there,
This may be a silly question, but based on my reading of the Stats Engine blog post, I expect statistical significance to increase with time. I've got some running experiments where it's going up and down, though, so I must have missed something. The post here:
... says "With Stats Engine, statistical significance increases over time" in no uncertain terms. Can someone help clarify this for me?

CouchPsycho 07-01-16
Re: Statistical significance decreasing
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
To be honest, i did not read the post completely but the sentence itself seems to be irritating. Why?
Because statistical significance depends on user behaviour and comparing two groups ( e.g. original and variation). Since you can not predict behavoiur, mathematics will lead to changing results in both directions.
Maybe the article talks about the difference interval which is explained, when hovering on the graphics.
I did a "few" tests and statistical signifance changed in both directions.
Because statistical significance depends on user behaviour and comparing two groups ( e.g. original and variation). Since you can not predict behavoiur, mathematics will lead to changing results in both directions.
Maybe the article talks about the difference interval which is explained, when hovering on the graphics.
I did a "few" tests and statistical signifance changed in both directions.
Kind regards
Michael
Michael




adamlevinson 07-01-16
Re: Statistical significance decreasing
[ Edited ]- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
so shouldn't the knowledge base article reflect that? It clearly states that stat significance increases with each visitor.
CouchPsycho 07-04-16
Re: Statistical significance decreasing
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Maybe yes. But one of the community managers is in charge now :-)
Kind regards
Michael
Michael



