I thought I’d take the time to share some of the numbers.

The Sites

Startup ideas come in bunches, and these social commerce sites are no exception. They all are built around aiding discovery, typically by allowing users on the site to share “finds” from around the web and build a following.

These finds may or may not be products and have a price tag associated with them. They may also be restricted to a specific category (e.g., women’s high fashion) on the site.

Here’s a list of all the sites I know. If you know any other, please, send me an email and I’ll update the list.

• Buyosphere
• Ly.st
• NotCouture
• Pinterest
• Svpply
• The Fancy

Of these sites, only Pinterest, Svpply, and The Fancy sent any traffic to Indie Cases.

The Numbers

The hope for these startups is that referrals from these sites are worth more than the average, non-qualified visitor. Some, like The Fancy, are even experimenting directly with commerce.

Value of a Social Commerce Visitor
Site	Conversion %	$/visitor
Pinterest	0.93%	$0.31
Svpply	3.70%	$1.09
The Fancy	0.48%	$0.13

I left out the traffic number deliberately, but will say this: The Fancy drove approximately 10x the traffic of Svpply, which drove the least amount of traffic of the three.

A Wrinkle

Why did Svpply convert so much better than Pinterest or The Fancy? Well, half the purchases from Svpply were of this iPhone case, which Ben Pieratt, CEO of Svpply, tweeted about directly.

Social Commerce?

Consider the above: a plug from a respected member of a community interested in our products (Svpply) produced roughly the same number of sales as a site driving 10x the traffic (The Fancy).

That, in a nutshell, is the promise of social commerce: the right recommendation at the right time from the right person.

The success of sites like Svpply, Pinterest, and The Fancy will hinge on their ability to consistently produce that.

Additional Info

Just for reference, here are the Compete graphs of the sites that sent traffic to Indie Cases.

Pinterest dominates in raw site traffic, but the question is whether their referrals are coming with intent to purchase. I’m sure they were grilled nonstop about that while they were out fundraising — a traffic graph like that allows for a lot of leeway. ;)

Does this interest you?

At Everlane, we’re not building a social commerce platform, we’re building a full-on store selling our own products. Indie Cases was a small preview of what’s coming.

If you are an engineer or designer interested in defining what online retail should look like in a world where Twitter and Facebook exist, and YouTube creates stars in weeks (not months), shoot me an email at jesse@everlane.com and let’s talk.

What is click hacking?

Aside from being a term I just made up, click hacking is a type of social engineering where the goal is to get someone to click a hyperlink.

The link could be the title of a Reddit post, a button on a landing page form that’s trying to capture your email, or an image in a Facebook ad. Deception can be, but is not necessarily involved.

I’m going to use Facebook, Hacker News, and Reddit as the primary examples throughout this article because I know them best. If you have other examples please leave a comment!

For Good or Evil

All the tactics I’m about to describe can be used for good or evil, and I’ve seen each used for both. Don’t take this article as an endorsement for spamming, scamming, or other internet trickery.

There’s plenty of grey area, too. At its start Reddit faked on-site activity to avoid looking like a ghost town. Was that unethical? A mistake?

None of these tactics are a substitute for generating real value for your customer, though they will help you understand how your customers react to what you present them (and how to incentivize them to react the way you want). The hard (and most important) parts are still up to you.

With that disclaimer aside, let’s dive into specifics.

Understand Your Audience

Above all else, I take the time to understand my audience. Each online community has its own customs, norms, and standards of behavior. You have to understand them before you can blend in or stand out as necessary.

For example, Hacker News values civility¹, straight-dealing, and intellectual honesty, but can be a little short-sighted and dour. Reddit values wit (especially puns and in-jokes/memes) and has a strong sense of community justice. It can also be completely juvenile.

From the click hacker’s perspective, a pun-filled title could do well on Reddit, but would never see the light of day on Hacker News.

Ask for Help or Feedback

One way to get people to pay attention is to ask for help. Hacker News has a “Ask HN” feature which evolved purely from community behavior. There are similar “Tell HN” and “Review my Startup” features.

For example, here is Drew Houston’s original Hacker News post asking for reviews of Dropbox. Every startup I know wants to publish a “showoff” entry on Hacker News and have it get to the front page, and their motives range from honest (they really want feedback) to self-serving (they just want the attention).

Give a Gift

Everyone loves gifts, but when we receive them we also feel pressure to reciprocate. The click hacker can use that pressure to get people to do what they want.

This tactic is the difference between “I baked a cake” and “I baked a cake for you.” A normal person is obliged to accept and even reciprocate.

For example, this Reddit user wrote a CSS hack to change the appearance of the site — a little present for anyone using Reddit. The Reddit community reciprocated by giving him upvotes.

Notice how he says he made it “for Reddit.”

The cake example works, too, though. :)

Bribery

Bribery is the opposite of gift-giving. In this situation the click hacker says, “Do this for me and I’ll give you something.” If that something is really awesome people will do almost anything.

The stereotypical example here are those scammy ads for free iPods and the like. All you have to do is fill out this form and click this link and you’ll be entered to win. Or, hey, instead maybe you can forward this offer to three of your friends, and if one of them wins, you win, too!

This can be done well in certain contexts. For example, Facebook game developers often trade installs by paying players virtual currency, e.g., “Install this game to earn ten farm dollars.” Or a game developer might have a special landing page offering players virtual currency as a way to encourage the player to click the install button.

The player gets what they want (virtual currency) and the game developer acquired a customer for free.

Assuming the terms are clearly outlined there’s nothing wrong with this. Of course, there’s plenty of room for outright dishonesty by promising goods that never arrive. Don’t be that guy.

Scarcity

People want what they can’t have. Or, more accurately, people want what they think they can’t have.

Gilt, for example, relies heavily on this tactic to drive sales. Better click that buy button now, because we’re running out!

In Gilt’s case the scarcity is legitimate, but it can be artificial, too. A game developer might make an in-game object ultra-rare in order to drive specific behavior. Limited beta invites are a tried-and-true method of generating traffic and interest; give out 20 invites to an audience of 20,000 and collect follow-up information for the people who don’t get there in time.

Scarcity also works with time, e.g., “If you install this game within the next thirty seconds, you’ll get a ten credit bonus.” Tick, tick, tick, tock.

Social Proof

Similar to scarcity, people don’t want to feel like they’re missing out. If they see other people doing something, especially people they know or respect, they’ll be more likely to do it.

This is the raison d’être for Facebook’s Facepile widget. Click that Like button. You know you want to. Come on, all your friends are doing it.

Even adding faces of random but friendly-looking people can be effective in getting people to click through. Here’s a screenshot of Match.com’s homepage.

Look at all those happy people. Don’t you want to be happy, too?

Pick a Fight

Rather than trying to blend in with a community, some times it helps to generate controversy. This can either be you vs. the community, or setting two factions within a community against each other.

I’ll share a story myself. Back in 2009 I was working on a polling application for Facebook. People would vote in polls and their vote would appear in their newsfeed.

The most controversial topics were the most successful, so to get things started I created a poll: “Do you support same-sex marriage?”

This was just after the Proposition 8 fight, so I decided to run two sets of ads on Facebook: one targeting 50 miles around San Francisco and another targeting 50 miles around Salt Lake City.

I can’t say it was a win for civil political discourse, but it generated enough controversy to make that poll viral and in turn make the entire application viral.²

Environmental Flaws

Sometimes the environment has flaws which allow the click hacker to do some interesting things.

When Facebook first launched their user-generated polling product, I remember seeing several polls asking questions like, “Which model is hotter?” The possible answers were URLs of images.

Unfortunately for the user, clicking on the text of the answer (the URL in this case) would register a vote and post that vote to your newsfeed. Their friends, who of course also wanted to see pictures of hot models, clicked the URLs, accidentally voted, and spread the poll to their friends. Oops!

Click hackers were using this mis-feature deliberately to spread new polls across Facebook.³

I don’t think many people will disagree when I say that most of the growth on the Facebook Platform from 2007-2009 was built on similar environmental flaws. Here’s a two-year old example from Slide’s Super Pocus:

What else?

What else am I missing? Post examples in the comments.

Have you ever been “tricked” into clicking something? (I know I have.) Have a funny story? Have experience optimizing landing pages, etc. and think about this all the time anyway? Leave a comment and share!

Everlane is Hiring

My startup, Everlane, is hiring. We’re trying to take the best elements of shopping offline — visual merchandising, personality, and curation — and bring them online. Check out our jobs page at http://www.everlane.com/jobs and send an email to jesse@everlane.com if you’re interested in talking.

1. See, e.g., Some Tips to Improve the Civility on Hacker News.
2. The way that application died makes a good story, too.
3. There’s a blog post describing data from this social hack — if anyone has it please share it in the comments!

How worried should we be that the feature we thought improved our product actually does nothing, or worse, hurts our bottom line? How can we ever really know that we’re making the correct decision? And is it better to run tests more quickly or more accurately?

The answers to these questions depend on the cost of a bad decision. If mistakes are cheap then it’s better to make 1,000 decisions and get only 60% of them right than to make 100 decisions and get 100% of them right.

One way to achieve this balance in the context of A/B testing is to tune the confidence level.

Tuning the Confidence Level

Intuitively, the confidence level of an A/B test tells you how certain you can be of the result of the A/B test. For example, a confidence level of 95% means that there’s a 5% chance that a statistically significant result is actually random variation, i.e., there is a 5% chance of a false positive.

Of course, we’re free to choose some other confidence level besides 95%. We could choose 80%, 90%, or 99.999%. A higher confidence level requires more data before reaching statistical significance, but we will be more certain of the result.

If you’re not comfortable with the nuts and bolts of statistical analysis, confidence levels, and A/B testing I recommend reading my article about statistical analysis and A/B testing, which explains exactly how one “chooses” a confidence level.

In short, the confidence level acts as a dial between speed and certainty, and we’re free to choose where to set that dial depending on the priorities of our business or product.

Speed vs. Certainty

So where on the speed-certainty spectrum should you, as a product manager or startup entrepreneur, sit?

Mike Cassidy has a great presentation where he argues that speed is the primary business startegy for startups.

Why is speed great for startups? Because mistakes are cheap and calculated risks are rewarded. Most product decisions can be undone, and important early tests can be redone at a higher confidence level when the product has more traction.

But mistakes aren’t always cheap. Here are some factors that increase the cost of a mistake.

Volume

Volume is leverage. If you have millions of customers, like Google or Amazon, a 1% improvement to the bottom line is a huge win. Conversely, a 1% mistake is a huge hit.

Fortunately this problem helps mitigate itself. Increased volume affords you the luxury of running A/B tests at a higher confidence level in the same amount of time.

Reversibility

Most product decisions in a consumer technology startup can be undone, for a price. For example, it’s easy to undo a bad decision for a web-based product, slightly harder to undo a decision for desktop software, and very difficult (and costly) to undo a decision for a physical product.

The less reversible a decision is the more certain you should be before you make it. In the context of A/B testing a product feature this means a higher confidence level, even if it takes longer to run the test.

Real Money

Imagine you’re an ad network. You’re constantly A/B testing formatting, positioning, offers, etc. to see which performs best. Making a mistake in this regard costs your publishers money.

Like volume, money creates leverage. But it is more complicated than that: publishers don’t just want increased revenues, they want reliable cash flow. That is, when money is involved, not only do you have to perform better but you have to perform more consistently because of phenomena like the peak-end rule.

In this case a “three steps forward one step back” strategy might actually be worse than going step-by-step in the right direction, even if the former averages out to better performance.

Conclusions

Maintaining momentum in a startup isn’t about making only correct decisions — it’s about making enough correct decisions. This presents a continuum from speed to certainty. At one extreme you run the business with a magic eight-ball; at the other you agonize over every detail until you’re 100% certain that you’ve made the correct choice.

This thought process extends naturally to A/B testing where the idea of “certainty” and “cost” can be quantified. To recap:

• A/B testing is a great tactical tool for testing specific hypotheses about your customers.
• However, there is a tradeoff between speed and certainty, controlled by the confidence level of the A/B test.
• The cost of doing A/B tests quickly is that you will make more wrong decisions, but that is ok if mistakes are cheap.
• For example, it’s better to make 1,000 decisions and get only 60% of them right than to make 100 decisions and get 100% of them right, all else being equal.

A Spreadsheet Model

Below is a little spreadsheet model that illustrates all my points above.

The two independent variables are the gain from a good decision and the cost of a bad decision. The spreadsheet assumes a fixed time period, so a higher confidence level means more certainty but fewer tests. The ideal confidence level is highlighted as you change the parameters of the model.

You can download the A/B testing confidence model here.

For the statistically inclined this model assumes that traffic increases linearly over time, that the sample statistic is normally distributed, and that a one-tailed t-test is the appropriate statistical test.

Credits

This article was inspired by a conversation with Sean Ellis and edited by my wonderful girlfriend, who is probably going to yell at me for linking to her Twitter account.

You can decide to measure anything, but what metrics matter and what ones are just for show? Here are some rules I hope will guide you toward creating meaningful metrics that help, rather than hinder, the decision-making process.

Be Actionable

If I had to give a one-sentence answer to the question “What metrics should I implement for my product?” it would be “Whatever metrics are actionable.” This means the line from a question to a metric and the line from a metric to an action should be as short as possible.

Most of the tips below are meant to focus attention on this issue. What can you do to make sure your metrics are actionable?

Be Understandable and Trustworthy

Do you understand what your metric measures? Does everyone in your organization also understand and do they trust it?

Trust is the important part. Everyone has to trust the metric if you’re going to use it to make decisions, otherwise you’ll be getting constant pushback. This will slow the decision-making process and cause a lot of ill-tempered arguments.

Measure Results

Does your metric measure customer behavior or a correlate of customer behavior? In the past approximations and correlations were necessary because measuring behavior directly was hard, but on the web there’s no excuse — you have access to every single thing a person does on your site, down to where their mouse is hovering and for how long.

For example, if you want to know how good Twitter is for your business don’t measure the number of positive tweets about your company. Instead, measure how many customers it drives to your product and how much money those customers put in your hands.

Understand the Downside

What would you do if your metric were 50% off the mark today? Would you be able to articulate why this is a problem for your business? What it costs you? Would you know where to start looking for possible causes?

As an example, I’ve worked with a startup that used “number of MySpace friends” as a go-to metric in every marketing meeting. Is that really material to the business?

What would happen if tomorrow we had half as many MySpace friends? Would we lose $100? $10,000?

This number says nothing about the value of MySpace as a marketing channel, which in the most charitable case is what it is trying to approximate, and the downside is completely ambiguous. Is the number of MySpace friends tied to anything meaningful?

Like the Twitter example above, if I thought MySpace were an important marketing channel for my product I’d be measuring things like the number of qualified leads from MySpace and the value they generate for the business.

Understand the Upside

Conversely, ask yourself, “What value does improving the metric bring to the company?” Some metrics are blindingly obvious in this regard, e.g., top-line revenue numbers and some “efficiency” metrics like effective CPM and revenue per user.

Some metrics are less obvious. What about page views? Can you imagine a scenario where your pageviews double but the net effect was bad for your business?

Metrics like these are dangerous because they lull you into a false sense of security. Everything on your analytics dashboard is going up and to the right, but the fundamentals of the business might still be suffering.

Don’t Be Ambiguous

Does your metric measure one thing and one thing only? Or is it really an aggregation of several variables, each of which can rise or fall independently?

A good example of an ambiguous metric is the notion of a “daily active user,” or the total number of people who interacted with your product today. This number is ambiguous because it is really the sum of two metrics: the number of new users and the number of returning users.

Ambiguous metrics are bad because they obscure the underlying variables that truly reflect customer behavior and delay the decision-making process as you are forced to determine which of these variables is actually affecting the aggregate metric. Furthermore, it’s possible that one variable accounts for all the growth in the metric, e.g., you have millions of new users per day but no returning users.

This latter scenario has been the death of many Facebook apps.

Segment by Purpose

Whenever I’m building a web product I divide usage into key segments. Generally these are acquisition, retention, engagement, and monetization. That is, how do customers find my product? Are those customers coming back? Are they doing the things I want or need them to do? And how much money are those customers making me?

See Dave McClure’s Startup Marketing for Pirates presentation, which focuses on this idea.

Doing this lets you tackle each segment independently and allows for a sharper product focus. Early on you might want to focus on acquisition, or maybe you want to focus on monetization from day one. Eventually you’ll starting caring about longer-term metrics like retention and you won’t be distracted or overwhelmed by unrelated metrics from different segments.

You might select one or two from each category to act as top-line variables in a dashboard that you look at every day, e.g., new users (by channel), returning users (by channel), activity, and revenue.

Appropriate Granularity

Sometimes you need a bird eye’s view and sometimes you need a tunneling electron microscope. Know when you need which.

As a general rule of thumb I focus on the microscopic when I am designing specific optimizations but focus on the macroscopic when I’m determining whether the decisions we’re making are working. Another way is to finish this sentence, “I know my product is healthy because…”

You’d never finish that sentence with “because the click-through-rate on my login page is 20%.” You’d say something like “because 80% of my customers return every week” or “because our revenue is growing by 5% month-over-month.”

Conclusions

These are just tips, not hard-and-fast rules. They are meant to focus the discussion of what metrics matter and why because the choice of metrics has a long-lasting impact if your team is committed to building a data-driven culture.

If you have any tips on deciding what metrics matter and why, leave them in the comments! This list wasn’t meant to be exhaustive and I know (or hope) people will have some strong opinions!

Metcalfe’s Law, which states that the value of a network grows in proportion to the square of the number of users of that network, express this idea formally. The value of a social network rests in its ability to foster communication, in its connections.

If you’re building a social network, whether it’s a destination website or an application that exists on another social network, density must figure into key strategic decisions. Let’s see how.

Islands

One way to think about social networks is as a network of networks. On Facebook, for example, if I graph the connections between all my friends I see distinct groups: my high school friends, my college friends, my current circle of friends, and my professional network.

Each group of people is more or less isolated from each other. Density exists within each of these islands, but not between them. I’m fairly certain this is a topological property of any social network.

Case Study: Facebook

Facebook started at Harvard and was initially college-only. Their growth strategy was explicit from day one: move from school to school as demand warranted. I’ve been told that Sean Parker wouldn’t consider opening up access to a new school until at least several dozen students from that school requested an account.

Each school was an island. Once Facebook saturated a specific set of colleges it moved onto the next round. Eventually there was enough anticipatory buzz that they could launch at large, state schools without risk of fading away.

They still pursue this strategy today. After establishing a critical density among colleges they opened up access to high schools and then to everyone with an email address. From there they started moving country-to-country.

The countries where Facebook is having the most difficulty gaining traction are the ones with already-established social networks, like Germany with StudiVZ. In fact, if you look at this old map of the most popular social network in each country, you get an idea of how isolated this country-by-country growth really is.

Other Networks

Facebook isn’t the only example of a social network who grew this way. hi5 has a similar story, starting with smaller markets overseas and spreading from country-to-country. Or Craigslist, by starting small in San Francisco and eventually becoming a presence in most major US cities.

The MySpace team had a background in direct marketing, which is all about targeting specific offers at the people who are most likely to respond. They started with the club scene in LA and grew from there.

The key to all these strategies was density.

If you’re launching a new social service, even if your end goal is to have everyone and their mother using it, it’s important to understand the impact density has on the growth

Multiple Dimensions of Density

So far the only kind of density we’ve talked about is network density, i.e., multiple people connected through their shared use of a service. You could call this “product density.”

Sometimes product density isn’t enough. Take IM, for example, or any network that requires synchronous communication. Not only do two people have to be using the same product but they have to be using it at the same time. What good is your friend being on IM if you’re never awake at the same time?

Xfire, an IM client for gamers, is an example of a product that innovated in this space by tackling a segment of customers who were already interacting synchronously.

Mobile social networks take this to an even greater extreme. To connect with people on Loopt or Google Latitude not only do we have to be using the same product at the same time, but we have to be in the same place!

This isn’t to say building these networks is impossible. Rather, they come with an extra handicap in the form of reduced density. Overcoming that problem has to be a key part of the product strategy.

Conclusion and Counterexamples?

Most product strategy discussions, in my experience, are focused on acquisition or other topline metrics that go “up and to the right.” Instead, if you’re building social software, I believe density is a necessary condition for long-term success and needs to be a part of the strategy discussion from day one.

First, understand the density requirements for your product. Do customers need to sign up for the same service? Do they need to be using it at the same time? Do they need to be in the same place? Is there anything you can do to lower the density requirement?

Second, build a “depth first” strategy. Are there any naturally dense customer segments that might fit your product? Do you have the ability to target specific demographics or segments for acquisition? Which ones respond positively and is it possible to build density there?

Once you’ve achieved sufficient density on one island hop to the next and repeat.

And if anyone out there can think of any counterexamples — social networks or services that got big “all at once” — leave a comment and let me know! I honestly can’t think of any.

I had two jobs. The first was to help manage the dorm computer labs. If a new virus broke out and computers were banned from the network because they were spamming everybody on campus, I had to go in and clean it up. I still remember when the Blaster worm infected most of the Windows machines on campus.

The second job was to help build a web-based dorm management system called Chopin. This system was at the center of the daily operation of the dorms. Students could use it to submit work requests and report problems with the dorms. Staff could use it to send out mass mailings, sell students printing credits for the dorm printers, and any number of other routine tasks.

Enter Verna

Verna was a front-desk clerk at one of the dorms. She was responsible for helping students if a problem came up and used Chopin to get her job done.

She had also lived in Hyde Park, the neighborhood surrounding the University, for most of her life and just wanted to do her job without having to deal with whiny students or crappy software. Chopin was supposed to help her do that.

One day while fixing the the front-desk computer I asked Verna, “What do you think of Chopin?” She immediately supplied a laundry list of complaints. It didn’t work well, it was confusing, she never knew where to go or what to do, and so forth.

It would have been easy to blame her. Maybe she just needed better training or maybe she wasn’t trying hard enough. But that was all ego: I was just upset because she told me the product I helped build was pretty awful.

Her critique was absolutely fair. I had built solutions I liked for problems I wasn’t even sure existed. In fact, before then, I had never honestly talked to any of the people who actually used Chopin about the product itself. In retrospect that seems completely insane.

And watching her use Chopin I saw so much waste. She would click ten times where two would have sufficed. Some features would never get used, others used for things they weren’t intended for. Once I was there, talking with her and watching how she used Chopin, I saw that the whole process was messed up, from top to bottom. Most of her energy was spent working around problems I had caused!

Go and See

If my job hadn’t required that I work on Chopin and get out of the office I never would have even realized there was a problem.

That experience taught me that whenever I didn’t understand a customer’s frustration or thought that maybe they were feeling this way or that way I should just go ask them before building solutions that might be worse than the problem. When in doubt, go and see for yourself. Actually, scratch that: always go see for yourself.

Too often I’d pass off mere belief as knowledge, or generalize from a specific set of circumstances to a fundamentally different set of circumstances. Maybe this solution worked over there, but why should it work over here? The only people who can validate your product are your customers — everyone else, including yourself, can wait their turn.

Verna taught me that.

This problem might seem artificial. Why not put up an ad on your product, or send a notification to every single person who might be interested? There are several reasons the number of people you can notify might be constrained.

1. There is a technical constraint, e.g., Facebook limits the number of application-to-user notifications at the API level.
2. There is a financial constraint, e.g., you’re sending notifications over SMS and every message costs you money.
3. There is a strategic constraint, e.g., sending notifications too frequently causes fatigue and reduces the effectiveness of future notifications.

So, the situation is not too far fetched. Let’s investigate the issue.

For the rest of the article the “application” is going to be a Facebook application and it can only send 100 application-to-user notifications per week. Which 100 users should we notify?

The Basic Considerations

In the linear cascade model when a user in a social network adopts a new behavior there is a probability that each neighbor in the network will adopt it.

Under this model we probably wouldn’t want to notify two people who are friends, and especially not a cluster of friends or a clique. The new feature wouldn’t spread very far beyond this group.

Likewise, we wouldn’t want to notify people who are very far apart on the social network because a user is more likely to adopt a behavior if more than one of his friends has also adopted it. So there is a balancing act between notifying users who are close together, to achieve density, and notifying users who are far apart, to achieve breadth.

Heuristics and Centrality Measures

The easiest solution is to pick 100 random users to notify, but this is also the most naive since it takes into account neither the structure of the network nor likelihood that a person will influence their neighbors.

A better¹ solution to this problem is to develop a heuristic that ranks every user in the network according to some metric. If we can only send 100 notifications then they are sent to the first 100 people on this ranked list.

The idea here is to use centrality measures to come up with heuristics. In graph theory “centrality” is a measure of how important an individual node is.

The simplest measure is called “degree centrality” and is equal to the number of neighbors of a node. On a social network this is the number of friends of a given user. So, if you wanted to send out 100 notifications using this heuristic we’d send notifications to the 100 users with the most friends. This heuristic involves convincing celebrities to use the new feature.

There are other, more complex heuristics. The Wikipedia article linked above has a list of other centrality measures, and I wrote an article about calculating eigenvalue centrality, which is similar to PageRank. Each of these admits a heuristic which can tell us which users to notify.

Of course, which strategy works best is hard to know beforehand, as it varies with respect to both time and the underlying notification. A/B testing this is difficult because the effects are intentionally dependent. If anyone has a good solution to this that doesn’t involve collecting massive amounts of data about user behavior I’d be interested in hearing it!

It should be noted that each of these heuristics only takes into account the underlying structure of the graph and not the probability of “infection.” By including the latter we can come up with a nearly exact model of the optimal subset of users to notify.

A Global Solution

Brendan Meeder at CMU pointed me to a great paper that discuss this very topic, Maximizing the Spread of Influence through a Social Network by Kempe, et al.

Rather than take a localized view of the problem by ranking each node individually, we create a statistical model of how the new feature propagates through the network.

First, we start with a finite seed set, A. In our case A is a set of 100 users. Say we convert each of these 100 users.

In our model if a user u is converted then for each neighbor v there is some probability

that v will also be converted.

After the process has run its course some set of users has adopted the new feature. Because adoption is probabilistic the size of this final configuration is a random variable. Using the notation from Kempe, et al., for a given seed set A the size of the final set of adopters is a random variable denoted by

Our goal is to pick the set A which maximizes the expected value of this random variable.

Formally, we want to find the subset A such that

is maximized, where

is called the influence function.

The Algorithm and The Results

It turns out that calculating the influence function exactly is NP-hard, but there is a greedy algorithm which approximates the value under certain (unrestrictive) conditions.

If you want more details read the paper linked above or the related Influential Nodes in a Diffusion Model for Social Networks by Kempe, et al.

Using Monte Carlo methods Kempe, et al. simulated the diffusion process using this algorithm versus several of the heuristics I described above. The results are fairly striking: their algorithm performs at least 18% beter than the best-performing heuristic (degree centrality) and 48% better than if the seed set were randomly selected. I’ve embedded a graph of their results below.

The “target set” is the initial seed set of users to notify, and the “active set” is the final set of users who actually adopted the new feature or product. The more users who adopt the feature the better the strategy.

Feasibility

Kempe’s algorithm is more feasible than many of the heuristics discussed above, although the best performing heuristic — degree centrality — is also the easiest to calculate. He also doesn’t include eigenvalue centrality in his analysis, which I’d be interested in comparing.

The biggest downside to his algorithm is that it requires both full knowledge of the underlying graph and an accounting of all the user-to-user transmission probabilities. Modeling these probabilities would require a lot of data about users over an extended period of time.

Whether the additional 18% is worth the extra computation and data collection depends on a lot on specific circumstances, but personally I’m going to try to implement it in my projects and see how the performance compares first-hand.

1. “Better” according to what? As we’ll see, randomly selecting seed users performs worse than all the other heuristics.

If Facebook wants to be the internet’s cable company what are they going to have to do to turn themselves into a $40Bn company?

Are Virtual Goods the Key?

Not all social networks are struggling to find a great business model. Tencent, a Chinese social networking company, pulled in over $1Bn in revenue last year, primarily through its use of virtual goods.³

But Facebook doesn’t need to look overseas to see that virtual goods could work for them. Most of the top Facebook games use virtual currency to make money, powered by leadgen-based ad networks like Offerpal and Gambit. There are reports that some of these apps are pulling in eight figures per year.⁴

And of course there’s Facebook’s own gifting service, which has recently moved to a virtual currency system, pricing gifts in “points” that can be bought with real money.⁵

All of this is to say that it appears that virtual goods are a natural business model for social networks and Facebook has enough data to see that. Why isn’t Facebook pursuing this strategy more aggressively? Why do they seem dead-set on building advertising technologies like Social Ads and Beacon?

The US Advertising Crutch

In the world of advertising not all countries are equal. US traffic is generally valued the highest, followed by other English-speaking countries, the G20 , and finally the rest of the world.

Until recently Facebook was concentrated in the English-speaking world. It’s the second largest social network in the US, after MySpace, and the largest in both Canada and the UK.

Unlike other social networks which don’t have a significant presence in the English-speaking world, Facebook can support itself through advertising. This is a crutch that prevents Facebook making bold decisions with their business model. I believe Facebook sees themselves as the next Google, one piece of technology away from changing the world of advertising.

The Demographic Crunch

Not all social networks have Facebook’s demographics, of course. hi5, the world’s third largest social network after MySpace and Facebook, has an extensive presence throughout Latin America and other countries which advertisers and publishers typically ignore. The same can be said of the advertising market in China, but recall that Tencent pulled in $1Bn last year through virtual goods.

It’s little wonder, then, that hi5 is aggressively pursuing a virtual goods strategy.⁶ Their demographics makes this strategy much more appealing. Facebook has the money and the audience to waste pursuing a pure-advertising strategy for social networks.

What once seemed like a demographic disadvantage might turn out to be a demographic advantage for hi5. Will they beat Facebook to the business model punch?

And a year from now will we be reading articles about Facebook’s virtual goods strategy compares to hi5′s, as opposed to articles about how Facebook’s new homepage compares to Twitter?

You’re crazy. You know that, right?

Obviously hi5 has an uphill battle. Facebook is growing on the order of 500,000 new users per day and shows no signs of slowing. But the same was said of MySpace and Friendster when Facebook launched. I think we still have a few more twists in the story of social networking on the web, and this is just one possible twist among many.

1. Microsoft acquires equity stake in Facebook, expands ad partnership (cnet)
2. Update On Facebook’s Dubai Fundraising Trip (Business Insider)
3. The world’s most lucrative social network? China’s Tencent beats $1 billion revenue mark. (VentureBeat)
4. Developer Analytics: Facebook game Mob Wars making $22,000 a day (VentureBear)
5. Gift Shop Credits Have Arrived (Facebook)
6. Hi5’s virtual entertainment plans could hit a virtual jackpot (VentureBeat)

In this article I’m going to outline two models that describe how new behaviors, ideas, and messages propagate through social networks.

The Threshold Model

The first model is called the Threshold Model.¹ It says that people adopt a new behavior bceause a sufficiently large proportion of their friends have adopted that behavior. Early adopters have a very low threshold, say 5% or 10%, while late adopters would have a much higher threshold. Every person, however, has their own individual threshold.

For example, my girlfriend’s stated reason for signing up for Twitter was that “all my friends were using it.” And during the 2008 US Presidential election, some Obama supporters would adopt Hussein as their middle name.² When I saw that lots of my friends were doing it I was certainly tempted to do the same.

The underlying psychological principle is one of “missing out” or “when in Rome.” The key variable here is the initial distribution of thresholds across a social network, which describes in totality the final extent of the behavior.

It’s worth noting that this model says nothing about how people initially adopt behavior. That is, it says nothing about innovators, only about the spread of innovation through a social network.

The Cascade Model

The second model is called the Cascade or Word-of-Mouth Model³, and is the method of “viral growth” that most social application developers are familiar with. It says that every person has a chance of adopting a new behavior whenever one of their neighbors adopts it.

This model describes phenomena like product recommendations or user-to-user notifications on Facebook. The probability that a person adopts the new behavior is the conversion rate for the notification.⁴

This probability is both a function of the sender and the recipient, so more influential people are more likely to convince you to adopt a behavior (or purchase a product, or install an application).

Practical Implications

Both of these models describe facets of real-world interaction on social networks. My take is that the cascade model is more accurate at the beginning of a social network’s life, where behavior is spreading through sparse areas, connected by influencers. Later on, after a critical density has settled in, people start adopting the behavior because everyone else is adopting it and there’s a social cost to not doing the same.

We see this pattern in services like Facebook and MySpace, both of which got their start by harvesting emails and spreading through word-of-mouth (and spam) across a social network.⁵ Eventually each network reached a point where a sufficient number of people were familiar with the product and new users adopted it not because their friends recommended it (the cascade model), but because there was a social expectation that they do (the threshold model).

Also, with respect to analytics and viral growth, the threshold model is more difficult to track. In the cascade model we record who sent what to whom and which messages they responded to. It’s clear who gets credit for a user’s conversion. In the threshold model you have to track passive exposures, and there’s no clear causal relationship.

If ten of my friends are doing something and I decide to start doing the same thing, who gets credit? Most analytics packages will show this behavior as a direct visit, with no connection to other users’ behavior, even though there is a viral process underlying it.

In short, the threshold model requires a certain level of behavioral density, while the cascade model doesn’t. However, we see both models expressed in how people actually adopt new behaviors in social contexts.

Formalisms

In the threshold model every person u has a threshold

and each of their neighbors v is weighted according to

If

then the person u adopts the behavior.

The set of thresholds, weights, and initial adopters completely determines the extent of the behavior in the social network.

In the cascade model, for every person u and neighbor v there is a random variable

which describes the likelihood of u adopting the behavior if v has adopted it.

Takeaways

I’ll try to boil all this down into a few, practical takeaways.

1. The Threshold and Cascade Models describe two mechanisms of behavior adoption in social networks.
2. The Threshold Model says that people do something if enough of their friends are doing it.
3. The Cascade Model says that people have a chance of doing something if one of their friends is doing it.
4. Both models correspond to different real-life adoption patterns.
5. The typical “viral loop” involves the cascade model, but most successful social networks rely on the mechanics of the threshold model in the long run, i.e., density is important for long-term success.
6. The cascade model is a good tool for analyzing acquisition scenarios, but the threshold model is probably more helpful for understanding retention and engagement — it at least implies that density is a key factor in social network growth, a metric that’s not often discussed publicly.

Agree? Disagree? Leave a comment, send me an email, or follow me on Twitter!

1. See Threshold Models of Collective Behavior (1978) by the famous sociologist Mark Granovetter.
2. See Obama Supporters Adopting Middle Name “Hussein” As Their Own
3. See Talk of the Network: A Complex Systems Look at the Underlying Process of Word-of-Mouth (2001) by Goldenburg, Libari, and Muller.
4. More accurately, we’d model the “probability” as a random variable whose mean was the conversion rate.
5. See Stealing MySpace: The Battle to Control the Most Popular Website in America for details about the MySpace team’s background in direct marketing. The ConnectU vs. Facebook court documents, which you can find via Google, paint a similar story for Facebook’s early years.

“Going viral” has become a sort of holy grail and most people would say they’d rather have a viral application than not, but it has a distinct downside: uncontrolled growth and ever-increasing operational costs. By being almost-but-not-quite viral you can dramatically reduce your cost of acquisition without setting your servers on fire.

The Paid Strategy

The key variables for a paid strategy are cost of acquisition and average revenue per user, or ARPU. If your ARPU is greater than your cost of acquisition then you can buy as many users as your budget allows, reinvesting the new revenue into acquiring new users. Generally this is done by advertising your product through something like AdWords or Facebook’s Social Ads.

The best thing about a paid strategy is its relative simplicity. If your ARPU is $2.00 then you can run as many ads you want so long as the cost of acquisition is less than $2.00 and still be profitable.

Eventually, though, you will run out of ad inventory. There are only so many publishers who are willing to accept $0.10 per click. At this point your only options are to decrease your cost of acquisition¹ or increase your ARPU.

The Viral Strategy

The viral acquisition strategy requires you get current users to invite other users to the application. The two key variables for the viral strategy are the average number of invites each new users sends and the rate at which those invites convert into new users.

The ratio of converting invites to new users is your viral coefficient, k. If this is greater than one you will see self-sustaining, viral growth.² If the coefficient is less than one each user will bring in a fixed number of new users, but the application’s growth is still linear.

People who have decided on a viral acquisition strategy focus on this number obsessively. It’s the first big hurdle to clear and if you haven’t had experience engineering a viral application it can take months to build something viral.

But being viral isn’t an either-or proposition. Increasing your viral coefficient from 0.5 to 0.8 has other advantages, especially if you integrate it with a paid strategy. Let’s see how.

A Hybrid Strategy

Say you’re building a game on Facebook backed by a virtual currency. Users give you money or fill out offers to get coins, so you have a positive ARPU. This means you’re free to pursue a paid acquisition strategy.

On the other hand, you’re on Facebook, which has many viral hooks. Many of the technical hurdles are much smaller there, so it becomes more a question of design and optimization rather than implementation. At the very least you encourage players to invite their friends.

The first version of your application has a viral coefficient of k=0.5, that is, every new user who joins brings in 0.5 new users. Equivalently, for every two users you acquire you get one free.

That’s interesting, especially if you’re also paying for users. If you’re paying $1.50 per user then you paid $3.00 to get two users, but acquired a third user for free! This means that you effectively paid $1.00 per user: $3.00 paid / 3 users.

This process is actually geometric, however. If you purchased 4 users with a viral coefficient of 0.5, you’d first get 2 new users for free. These 2 new users would then bring in 1 additional user, for a total of 3 new users, reducing your cost of acquisition even further. This is an infinite geometric series, which I’ll outline below.

Having a non-zero viral coefficient reduces your effective cost of acquisition.

This means that you get more users for every dollar you spend on ads. Or, if you’ve run out of inventory, this means you can now spend more per user and retain the same net revenue level.

Let’s figure out how to calculate your effective cost per acquisition.

Effective Cost of Acquisition

If you just want the formula, here it is:

Where C is your cost of acquisition, k is your viral coefficient, and C’ is your effective cost of acquisition.

If k=0, i.e., you have no viral acquisition, then C’ = C.

If k = 1 and your application is viral then C’ = 0 and your application grows without spending any additional money. But rather than being an either-or proposition — you’re either viral or you’re not — there’s a sliding scale. The more viral you are the cheaper it is to acquire users.

The Benefits of Being Almost Viral

From the above formula, if you have a viral coefficient of 0.90 then you have reduced your cost of acquisition by 90%. This is a great situation to be in. You might ask, “Why wouldn’t I want go the last 0.10 and make my application viral?”

The one benefit of being viral is huge growth, which looks sexy on a graph and can tip an investor to your side if you’re looking for outside money, but the growth is unpredictable. Not only do you have little control over what demographics come to dominate your application, but sometimes it grows so quickly that you run into operational problems (servers on fire, etc.).

By being almost viral you can grow very cheaply, control your rate of growth and demographics, and get enough traffic to conduct meaningful experiments. Need to grow more slowly? Just decrease your daily ad spend. Need statistically significant results more quickly? Increase your daily ad spend.

Put another way, with a viral coefficient of 0.9 you’ve dealt with your acquisition risk. Rather than going fully viral and dealing with the operational difficulties, it might be worth your time to deal with other market risks: retention, engagement, and monetization.

So, stop sweating about “being viral.” Sometimes it’s better to be almost viral.

Deriving the Formula for Effective Cost of Acquisition

You can skip this and go right to the comments if you’re not interested in the math.

We have an application with a viral coefficient of k = 0.5. Every new user who joins brings in 0.5 new users. Another way of thinking of it is that for every new user who joins there is a 50% probability that he will bring in another user.

But this potential user also has a 50% chance of bringing in a new user, so the expected number of users is now 1 + 0.5 + 0.5*0.5. This continues ad infinitum.

Formally, if we acquire one user and have a viral coefficient of k then the number of users we expect to join is N(k), given by the formula

The initial 1 is our original user, whom we paid for, and each k represents the expected number of users from each step in the viral process.

This is a geometric series, so we know that

Therefore, our effective cost of acquisition is

1. If you increase the conversion rates for your ads then you can pay more per click to get more impressions without hurting your bottom line — you pay the same to acquire a user, but get more users through the door.
2. See my article Three Myths of Viral Growth for more information about viral growth.