I recently completed an A/A test using a control and an identical experiment. The purpose of which was to test the accuracy of the experiments and gauge whether future tests would be reliable. The test included 22,000 page views, and despite being identical, the experiment generated a conversion rate that was 11.3% lower than the control. Is this a common issue?
What was your baseline conversion rate?
How many visitors to each variation?
Did you check GA segmentation to ensure the traffic quality was split evenly? ( Assuming you run on multiple channels)