The Dangers of Genetic Optimization

The guys at Weebly just had a round of press for their latest product, SnapAds, an ad optimization platform that uses genetic algorithms. The technology is very cool, so check it out.

John Resig then posted a link to Genetify, the previous incarnation of this technology, which uses genetic algorithms to optimize your website at large by "evolving" your stylesheets.

The Black Box of Genetic Algorithms

SnapAds is a great application of this technology because the guiding metric function is obvious: total ad revenue. Since we've talked about A/B testing, you might wonder why not do this automatically for your website at large and optimize other user behavior?

The answer is what we call "black box testing." You know the results — maybe users are 50% more likely to click a certain link — but you don't understand why.

This is a pitfall of normal A/B and multivariate testing, too. You put up an experiment, measure the outcomes, and pick the one that performs the best according to the metrics that matter. Bingo bango.

And hey, if you automate the optimization step with something like genetic algorithms, you don't even need to do this. The machine makes the decision for you!

Analysis Matters

The problem with black box testing — when you understand the outcome but not the underlying cause — is that there's no learning. Analysis matters. Customer insight matters.

If you're only doing black box testing you don't really understand your customers. You're just blindly following the dictates of whatever algorithm you've set up.

Your customers might be buying more now, but can you apply that knowledge to your next product?