A/B Testing, aka Split-testing or multi-variate testing if you wish to sound posh.
The process of randomly assigning website visitors to one of two different groups (group A and B), and presenting a slightly different experience to each group. By recording and analyzing difference in behavior between the two groups you can determine which version of your website is more successful.
A/B Testing requires a fair amount of traffic before you can be confident in the validity of your results: usually it requires thousands of visitors, and certainly more than 100 visitors. "Multi-variate" testing is where you test more than two options at once, including combinations of features, and you are not looking for the best result, but looking to exclude the worst result. Effective multi-variate testing requires much larger traffic volumes. For example if you tested 20 combinations, you can expect it to take 10 times as many visitors as a simple A/B test.
Pssst. Want to be notified when the book is ready? It's written for people just like you.