Comments on: Decisions Without Data http://20bits.com/articles/decisions-without-data/ Driven by Data Wed, 28 Jul 2010 21:50:46 -0700 http://wordpress.org/?v=2.8.4 hourly 1 By: Jesse http://20bits.com/articles/decisions-without-data/comment-page-1/#comment-1043 Jesse Sun, 06 Apr 2008 01:25:37 +0000 http://20bits.com/?p=84#comment-1043 Timothy, I should've added a footnote, but I agree. You can basically make your measurements as complex as you want, but there's some point of diminishing returns. You could, for example, measure the effects of the ads on the growth of your userbase and then make some sort of "net present value" calculation. Really all you need is enough data to make an informed decision — some data is better than none. Timothy,

I should’ve added a footnote, but I agree. You can basically make your measurements as complex as you want, but there’s some point of diminishing returns.

You could, for example, measure the effects of the ads on the growth of your userbase and then make some sort of “net present value” calculation.

Really all you need is enough data to make an informed decision — some data is better than none.

]]> By: Timothy http://20bits.com/articles/decisions-without-data/comment-page-1/#comment-1041 Timothy Sat, 05 Apr 2008 22:10:17 +0000 http://20bits.com/?p=84#comment-1041 I'm definitely the first to suggest A/B testing. It's worthwhile to extol it's numerous uses, but it's no silver bullet. A claim like this: "the best ad network is the one that, for a given level of traffic, offers the highest RPM." is itself a guess. Data can help drive to a decision but it doesn't make the decision. For instance, what about offensive content in the ads? It might drive your RPM up but your total pageviews down. Or it might prevent you from reaching a new audience in three months when you try to enter a more genteel market, something which your initial A/B test won't illuminate. The key is to make data driven decisions, which means balancing analysis, synthesis and data. I’m definitely the first to suggest A/B testing. It’s worthwhile to extol it’s numerous uses, but it’s no silver bullet. A claim like this: “the best ad network is the one that, for a given level of traffic, offers the highest RPM.” is itself a guess. Data can help drive to a decision but it doesn’t make the decision. For instance, what about offensive content in the ads? It might drive your RPM up but your total pageviews down. Or it might prevent you from reaching a new audience in three months when you try to enter a more genteel market, something which your initial A/B test won’t illuminate.

The key is to make data driven decisions, which means balancing analysis, synthesis and data.

]]>