The Dangers of Genetic Optimization
The Black Box of Genetic Algorithms
SnapAds is a great application of this technology because the guiding metric function is obvious: total ad revenue. Since we've talked about A/B testing, you might wonder why not do this automatically for your website at large and optimize other user behavior?
The answer is what we call "black box testing." You know the results — maybe users are 50% more likely to click a certain link — but you don't understand why.
This is a pitfall of normal A/B and multivariate testing, too. You put up an experiment, measure the outcomes, and pick the one that performs the best according to the metrics that matter. Bingo bango.
And hey, if you automate the optimization step with something like genetic algorithms, you don't even need to do this. The machine makes the decision for you!
The problem with black box testing — when you understand the outcome but not the underlying cause — is that there's no learning. Analysis matters. Customer insight matters.
If you're only doing black box testing you don't really understand your customers. You're just blindly following the dictates of whatever algorithm you've set up.
Your customers might be buying more now, but can you apply that knowledge to your next product?