Newly add hover state results in less engagement
my team and I are having problems interpreting the results of a hover test. We think something went wrong, because the results make no sense. We changed the result list to check if a hover has a positive effect on the click rate, but the results are in general that we have less engagement by -4.5%, a lower click rate on the result item itself -5,3% and less pagination -2,6%.
Has somebody some advice regarding the setup of a hover test? Or can somebody think of an explanation why the hover effect leads to a negative result?
If a sample size isn't large enough, it's common to see differences in goals that may not make sense. These are mostly random fluctuations, and not directly attributable to the change you're testing.