Is it ok to pause a test, and start it again two weeks later?
We've recently had to roll back minor changes to our site, and as a result have had to pause live tests.
We're in a position to launch tests again.
I wanted to know whether it's ok to unpause these tests, or whether I should duplicate them and launch them as new tests?
Our users have a research-purchase lifecycle of between 1 to 30 days.
You might copy it but you can also reset the results and start again. This is rather a good idea, if the changes you mentioned have an effect on your users. Your results will be "cleaner" by doing so.
But this is just my opinion :-)
Based on your customer purchase lifecycle, I recommend against un-pausing your experiment after such a lengthy delay.
There may be some wiggle room for experiments where you are using "Visit" conversion metrics instead of "Visitor" conversion metrics. If you are looking at the Add To Cart rate, you could run your reporting over two time-frames and look only at visits during the periods where the experiment is active.
If you are looking at "Visitor Conversion", however, you have a decent number of visitors who were placed into an expderiment and then converted when the experiment was off. Similarly, you have visitors who did their shopping before the experiment began and converted when the experiment was running. Over a long period of time, those differences "was away" into the background. But by pausing it and reactivating it, you are doubling the visitors who fall into this condition and also introducing a new condition of "saw experiment, removed from experiment, now back in the experiment".
Analytics and Testing Guru