Inside the 6 Hypotheses that Doubled Patreon’s Activation Success

Written By

Preface by Brian Balfour:  I rarely accept guest posts on this blog, but this opportunity was too good to pass up. A couple months ago Tal Raviv (Growth PM @ Patreon) sent me a short email - "Brian, the results are in.  We doubled (yeah, doubled) new creators in our onboarding." 

My immediate response, holy f*ing YES!!!! My second response, let's write about this. I've spoken/written about the importance of onboarding a lot before. The number of solid cases/examples out there is few and far between. So Susan Su, Head of Marketing at Reforge, dug in with the Patreon team to produce this amazing piece. It covers:

  • How Patreon decided and defined a leading indicator activation metric.
  • The challenges they faced with that metric.
  • The six hypotheses they tested, where the hypotheses came from, what worked and what didn't.  

l hope you enjoy the post. If you want to learn more about onboarding strategies and frameworks, consider applying for the next Reforge program here.


Patreon is a platform that allows people who make things -- everything from music to blog posts to journalism to comics -- to to run a membership business for their fans.
 
Since it was founded in 2013, the company has reached 1 million monthly active Patrons, paying an average of $12 a month, and 50,000 active creators. Patreon projects that its payout to Creators will be $150 million for 2017.
 
The company is growing, too. Today we'll take a look at how growth PM Tal Raviv and the rest of the team at Patreon identified their single highest impact area for growth -- onboarding for new creators -- and how the experiments they ran doubled the performance of that key metric.

Patreon’s North Star Metric

1 million monthly active Patrons coming to the site to fund projects and Creators is an impressive stat, but Patreon's true north star metric is active, financially successful Creators.
 
While the company won't share the exact number, Raviv says the company sets the bar at an amount that represents life-changing earnings for Creators.

"We have very clear, rigorous internal criteria for what we consider financially successful -- there is a specific threshold that we've found to be "life-changing" for our Creators.
 
It's a number where we believe the platform has meaningfully changed a creator's life and ability to create more, focus more on their craft, and in many cases go full time." 

Active, "financially successful Creators" (FSCs) works as Patreon's north star metric for two important reasons.
 
1. An actionable leading indicator
 
The number of active FSCs functions as a leading indicator for the amount of value being generated on the platform. 
 
Patreon monetizes by taking a 5% cut of transactions -- which the company is points out is 6 to 10 times below average take rates in creative industries -- so it makes sense that the they would want to optimize its growth around Creators whom they count as "financially successful." A Creator earning a very low amount through the platform won't meaningfully contribute to Patreon's monetization model, nor to its viral loop (more on that below). 
 
Further on, a financially successful Creator is not only more likely to have a pre-existing audience that they -- and Patreon -- can monetize, but is also more likely to serve as an aspirational example that attracts other Creators on to the platform.
 
Finally, financially successful Creators are more likely to stay happy, and stay on Patreon. Creators who hit the recurring income threshold have near-perfect retention, explains Raviv. 
 
2. A meaningful life change for a financially successful Creator --> more Creators
 
The second reason why active FSCs works as a north star metric is because the designation indicates that Patreon has effected a meaningful life change for a Creator. 
 
Raviv explains, "We'd rather have our GMV be made up of fewer, but truly life-changed creators rather than a lot of creators making a few dollars." 
 
This is because while active FSCs do bring on significantly more Patrons (the fans who support them), they also bring on more Creators. The bigger their success, the greater the aspirational value it carries.
 
For the company, Patron growth is a linear lever, while Creator growth is an exponential lever. Patron growth is a given as long as there is Creator growth. At Patreon, Creator growth is key not because it triggers more Patrons (it does that as well), but because it seeds more Creator growth. 

Challenges to growing the Financially Successful Creator metric

The Patreon team found themselves facing several challenges to growing the number of FSCs using the platform.
 
1. Lead qualification
 
The first challenge was to determine at the lead acquisition stage which creators had the potential to become "financially successful." 

"The fundamental problem was that our acquisition and activation funnel had a high ratio of people who were still early in the process of building their fan base, meaning they were coming to the site and not succeeding." 

In order to have to potential to succeed, Creators must have an established online following, even if small, and post at a regular frequency online. The specific frequency can vary by Creator, but it's crucial that publishing happens on a regular and recurring basis.

"We were attracting people who saw Creators getting paid and succeeding on Patreon, and they would misunderstand this and attribute a reverse causality. They would think that launching a Patreon page before building a fan base would start sending them money," says Raviv. 

The misconception is a side effect of Patreon's viral acquisition model -- artists and creators not yet on Patreon would get pulled in by publicly successful Creators on the platform thinking that the platform had originated a significant chunk of that success.

"Bottom line is it made our funnels useless. We couldn't really make decisions based on quantitative data because any drop-offs didn't necessarily reflect the qualified people. We're talking 2-3 orders of magnitude more noise than signal," explains Raviv.
 
"Our viral mechanism is creators see other creators launch and promote their pages (a lot like eventbrite) and succeed financially. 
 
As a result, our virality is through visibility more than explicit invitations like what you'd see on Dropbox or on a social network. So when people see Creators succeed on Patreon, they're motivated to give Patreon a shot." 

This posed a conundrum for the growth team. For example, an 80% drop-off at a given onboarding step would typically be alarming to any other growth team working on user activation. 
 
But at Patreon, a big drop-off could actually mean something good is happening -- unqualified leads are getting weeded out.

By the same token, a high-converting step could actually result in turning away the right Creators, because suddenly there would be lots of live Patreon pages with uninspiring revenue results (Creator earnings are publicly visible on each Patreon page).

Learn About Growth, One Email Per Post. 

"Project Mondo" & counterintuitive growth experimentation 

The Patreon team took a hard look at their growth levers, and realized that to meaningfully impact the business, they would have to significantly improve onboarding success. And that didn't just mean juicing the numbers to push more volume through their funnel -- they were looking to increase the conversion of qualified leads only, those Creators with potential for financial success on the platform.
 
Earlier learnings and observations told the team that they had a lot of aspects of the product to change -- or at least to test -- but the team's main constraint was as a low-volume SaaS funnel. 

"We needed each variant to be as much data as possible. We can't run 19-way tests like we do on our Patron-facing e-commerce funnels, for example. If we ran each change as a separate experiment, it would end up taking many, many months to get any results."

The second factor was that the Patreon team strongly believed that the sum total of all the of the changes together would lead to an impact that was greater than its constituent parts. If they ran each change as a different and separate variant, and none resulted in positive gains, it wouldn't disprove anything in their minds, and would instead just be a waste of an experiment.
 
So, it was all or nothing, and that's how the team eventually came to call their experiment bundle Project Mondo.
 
Project Mondo was both expansive in its reach and yet highly limited in its resource consumption. The project consisted of dozens of large and visible changes to the Creator onboarding flow, in support of 6 core hypotheses, and was implemented by a team of 2.1 engineers led by Raviv and inspired by their colleagues in sales, support, and community. 

De-risking an all-or-nothing growth experiment

The downsides of a Project Mondo approach were that if it worked or failed, the team wouldn't be able to parse out specific learnings. All hypotheses would be lumped together.
 
But, Patreon's growth team was less concerned about this risk.

"We laid out all the experiment designs and decided to go big. Because of past successes and failures in creator onboarding, we had a lot of specific learnings built up. That both gave us a lot of confidence in our hypotheses, and lowered our need to get super-specific learnings from this experiment." 

Project Mondo resulted in doubling the rate of onboarding success for the financially successful Creator segment, with compounding impact on Creator virality. 

"Chain reaction" virality 

The highest leverage metric in Patreon’s growth model is new, financially successful Creators -- who trigger further creator virality as well as help to bring on more Patrons to the e-commerce side of the Patreon business.
 
In other words, on the SaaS side, activation in a viral funnel is as valuable as direct shares.

"We have a creator-driven viral SaaS loop. The fact of Patreon Creators publicly making money inspires and triggers other creators to come and check out the platform."

In a viral loop, activation is just as powerful as direct shares because it impacts the viral coefficient and is compounding, setting off a chain reaction of growth.

"It's like making people sneeze more vs. making each recipient slightly more susceptible to sneezing." 

Raviv says the analogy is awkward, but works as a reminder that for Patreon deepening activation of qualified people already in the funnel is more important than broadening the number of people exposed to the funnel.
 
When most people think about accelerating virality in a viral loop, they ask themselves, "How do I get users to share more?" 
 
But Raviv and the Patreon team saw this as a blind spot.
 
Instead, they asked themselves, "What if you kept the same amount of shares but doubled your rate of onboarding?" The result would be the same as if you achieved double the volume of shares. 
 
Improving funnel conversion is a powerful way to accelerate virality for two key reasons:
 
1. Sharing happens "out there."
 
Sharing can be prompted within your product or onboarding flow, but, unlike onboarding funnel conversion, ultimately sharing is an action that takes place outside of the experience that you can control directly.  
 
2. Onboarding success rate is an exponential function.
 
In a SAAS funnel, onboarding success affects all other metrics in the funnel, from retention to referral, and exerts a disproportionate effect on recurring revenue.

Defining "financially successful" creators

In order for the Patreon's onboarding experiment to have a compounding viral effect, the company had to be able to identify and single out the segment of Creators that had the greatest potential to achieve "financial success" on the platform.
 
The criteria for defining a “Financially Successful Creator” are context-driven. Finding Financially Successful Creators hasn't hinged on absolute numbers of followers or an absolute publishing threshold. Instead, the team looks for a qualitative combination of engagement, how consistently the people who are engaging are engaging, and the professionalism of public-facing materials.
 
All of this is why it’s difficult to qualify Creators before they actually launch -- and even more challenging to then applying that definition across the board to all potential Creators in different categories. 
 
Instead, the Growth team decided to stick to the ultimate end quantitative criteria of a certain level of monetary earnings because it’s simple and functional for the growth team to optimize for that.
 
Ultimately, their two most basic criteria evaluate an inbound Creator's breadth of reach and depth of relationship with their audience.
 
The next section will go through each of Patreon's six hypotheses, where they came from, and what the growth team did to test them. 

The 6 Hypotheses that doubled onboarding success

Hypothesis 1: Creators with the potential to earn life changing income are lurking in the funnel somewhere
 
What it meant
 
The growth team's first hypothesis was that qualified Creator leads were "lurking" in their onboarding funnel -- in other words, great leads were hidden but everywhere.
 
What does this hypothesis mean? Let's start with looking at its converse. 

"The opposite would be to say that we're doing a bad job on acquisition and users aren't even clicking through. The opposite would that it's not an onboarding question, it's an acquisition problem."

Instead, the team's first hypothesis asserted that it wasn't an acquisition problem, that users were in fact hitting the site and even making it through part of the funnel, and some were even launching their Patreon page (the completion of onboarding). But, ultimately too many were dropping off in too many places. 

"We believed they were dropping off at every step even though we couldn't necessarily drill down on where because for us, some drop-off is positive as it indicates that unqualified leads are self-selecting out," explains Raviv. 

The team concluded that changes to the product at every single step would have a positive effect.
 
Where it came from
 
Early on, the Patreon growth team performed a small onboarding experiment where they only made changes at the very end of the funnel (the page editor). The improvement was the addition of simple text-based content that educated users on how to make a good Patreon page, and it was the same content that the teams on the front lines -- sales, community and customer support -- found themselves saying over and over again to Creators. 
 
In the experiment, the user had to get through the entire rest of the funnel before they could reach the new and improved help content, which lived alongside the editor and was only presented to the user at the bottom of the funnel. 
 
The results were impressive. According to Raviv, the inclusion of help text had a major significant impact on every level of financial activation: 

"The fact that this bottom-of-funnel change could make such a difference told me that there were probably people lurking all the way to the bottom of the funnel that had potential. 
If they're getting to the bottom of the funnel suddenly converting due to this change instead of dropping off, then they're likely to be hanging out at other parts of the funnel but dropping off before they even see this change." 

What Growth did about it
 
This initial micro experiment told them that if they were really going to move the needle, they would have to address every single step of the funnel, from the first message on the landing page, all the way down to launching the Patreon page. 
 
As a result, the team chose to implement a larger scale overhaul to the entire funnel, and they decided to call it Project Mondo. 
 
Raviv explains:

"'Lurking'" means they're present, but dropping off all along the funnel. I probably won't be able to find exactly where they are with a single pinpoint experiment, so I need to cast the net wider. The results can also be multiplicative that way."  

Learn About Growth, One Email Per Post. 

Hypothesis 2: Creators don’t immediately grasp how to use Patreon, but with a few concrete examples they come alive and totally get it
 
What it meant
 
The second hypothesis of Project Mondo conjectured that Creators didn't need a webinar and a course and a training in order to complete onboarding. They didn't need it all spelled out for them; instead they just needed to see real examples of Patreon pages that people had done.
 
Where it came from
 
The hypothesis came from Creator Relations, Patreon's team that handles sales and customer success, and does outreach to bigger Creators.

"When we talked to Creator Relations, we realized they spent most of their time helping them get over writers' block -- giving them examples and showing them examples of what other people have done in their relevant category. Their best sales was to do this type of coaching." 

The company had also conducted user research for other reasons, and the Growth team noticed a interesting side-note: while it was generally hard for user researchers to get Creators to make and launch their pages, as soon as UX researchers Christine Donaldson (also a musician) and Jamie Shroff showed them a few examples of a sample page, the Creator could run with it all the way to launch.
 
What Growth did about it
 
To test out this hypothesis, Growth implemented category-based examples in key parts of the flow. 

 The examples were displayed in context, so they stepped in at key moments of uncertainty. Additionally, because Creators must self-categorize as one of the earliest steps of onboarding, Growth could display only relevant examples within that same category to make the guidance clear and personalized. 
 
Hypothesis 3: Creators don’t know, and go nuts over, low 5% cut, no contracts, and owning their content
 
What it meant
 
In Silicon Valley, we take for granted that a content creator wouldn't have a contract with a software company, and that they would own their own content. 
 
But most of Patreon's Creators aren't part of Silicon Valley, or even part of tech. Instead, they're working with agents or YouTube, and within challenging ecosystems where 30% to 50% fees are the norm. 

"We forgot that! We didn't even think to put it on the website because it seemed so obvious to us, but we realized from our Sales team that it was something braggable that we needed to put front and center," says Raviv.

Where it came from
 
The hypothesis originated from the Sales and customer-facing teams, who found themselves explaining Patreon's fee system over and over again to Creators who couldn't believe how low it was. 
 
Prior to the change, the 5% fee was explained only in Patreon's help docs, but it turned out to be an important selling point for motivating Creators to complete the onboarding process and launch on the platform. 
 
What Growth did about it
 
The Growth team decided to highlight Patreon's fee structure up front, and showcase the 5% figure as early as possible in the onboarding process. 

"It was buried and you would have to go find it. It just wasn't on the website, so we decided to just say it up front. Growth is often about the addressing the 'obvious' things that no matter how obvious they seem now, flew under the radar for a long time."

Hypothesis 4: Creators are afraid of asking for money, & this can be solved with in-category comparables
 
What it meant
 
This hypothesis suggested that even though Creators have a lot of reasons they come up with why they can't do Patreon -- because it doesn't have x feature or doesn't integrate with y -- those complaints were really just a smoke screen for a more fundamental fear that having a Patreon page was like extending a beggars' bowl.  

"They'd ask for all these features, and then we'd build it. When we'd follow up with them, we would find out that the real objection wasn't the lack of X feature. The real issue was that they didn't want to ask -- or beg -- their fans for money," says Raviv. 

Where it came from
 
The Growth team took the problem to Sales, and asked their colleague Jordan Cope to coach top Creators past their discomfort around earning money from their audiences. 
 
He told them, "It's easy, I just show them someone on Patreon whom they respect, in their category, and then they don't care anymore."
 
What Growth did about it
 
Following Cope's insight, the Growth team implemented category-specific examples and linked to them in multiple places in the onboarding process. They also created an Explore page that's designed for Creators (not Patrons). 
 
The Explore page is a discovery portal for Creators to check out other Creators in a similar or adjacent category, and lists motivating stats like number of Patrons and amount of monthly recurring revenue upfront.

Hypothesis 5: Creators are not 100% secure or confident in the success of their campaigns. They don’t always necessarily themselves as successful. They need to be inspired and supported emotionally.
 
What it meant
 
A big part of the Growth team's job is to make sure qualified leads are coming in through the top of the funnel and succeeding throughout. So, it made sense to focus on the criteria that defined a qualified lead -- how big a given Creator already was, their growth trajectory, and their potential to influence other Creators through inspiration and aspiration.
 
"What we see is Creator success -- meaning, hitting a certain financial threshold. But, they don't see themselves the way we see them," explains Raviv.
 
Where it came from
 
Once again, the Creator Care and Customer Success teams dropped in valuable insights from their conversations on the front lines. These teams worked on Creator retention and on helping people improve their Patreon campaigns over time. Through those interactions, they learned that language is all-powerful, and that Creators, like anyone else taking a leap, are sensitive to the wrong language. 
 
A lot of Creators simply weren't as confident as they seemed to their fans, and their lack of confidence directly impacted the likelihood that they would complete and launch their Patreon campaign.
 
What Growth did about it
 
The Growth team, with learnings from Sales and Support, created a shortlist of Dos and Don'ts. 

"DON'T: say anything that might cause people to qualify themselves out of the funnel. It wasn't worth the risk. 
 
DO: be very proactive in addressing the emotional part of it. Give them a hug within the product."

Specifically, embedded a rustic, self-shot iPhone video of Patreon's co-founder and CEO Jack Conte (who is himself a Creator) cheerleading them at the end of the funnel.

The team thought that the video's positive message and authentic delivery would be an extra boost at crucial moment many of the most qualified Creators still drop off.
 
They also implemented custom confetti that would fire when a Creator finally launched their campaign, to celebrate passage through onboarding's most stressful moment. 

"We have creators visiting our office all the time. They meet our teams and our CEO. They walk in and see our musical instruments and our people, and they feel amazing. Then they get actionable advice, and any misconceptions they had about Patreon melt away. They leave the building with concrete steps and lots of motivation.
 
The product was a far cry from what it’s like to walk into our office. There was no warmth, no reassurance, no education. So, our challenge was, how do we get using the product to be like it is to walk into the building?"

Hypothesis 6: highest-earning creators don’t trust an editor that feels too simple or easy -- they need friction to build trust
 
What it meant & where it came from
 
The last hypothesis runs counterintuitive to most growth "best practices" popular on blogs and listicles, and it came from a very different place.
 
Like Patreon CEO Jack Conte, many Patreon employees are Creators themselves. Andy Coenen is currently an engineer on the Growth team, and was previously one third of the band the M Machine. The band was once signed on the same label as Skrillex (whom they also opened for at sold out shows) and toured the world. The group was successful and had a strong following, but in the music business it's hard to make a living unless you're a super star. Eventually, Andy became an engineer and landed at Patreon.
 
His insight on the platform came straight from his experience as a working artist:

"We thought about Patreon, but any tool we tried we would rock through it with a fake account to vet it: does this feel reliable? Is this going to make us look amateur in front of our fans? Is this something that serious creators would use, not just some guy who took his first guitar lesson?"

This was the riskiest hypothesis of all, says Raviv. "We encountered a lot of opposition in the organization. WYSIWYG is the most common best practice. Make it easy, make it frictionless."
 
Most Growth professionals have heard this before. The default assumption is that "people on the internet are stupid," and to improve conversions or streamline onboarding, we have to dumb down the flows and tasks that we ask our users to complete. 
 
But, the truth behind this "best practice" depends mightily on your specific audience and persona. In Patreon's case, artists -- and especially those who've risen to the top -- are far from stupid. Instead, they're self-aware perfectionists who can be highly suspicious of anything that looks too simplistic to be a match for their work.  
 
What Growth did about it
 
The best Growth strategies stem from deeply understanding the psychology of the core persona you're targeting. 
 
Although Patreon considers itself to be a SAAS product, the team also understands that their tool is fulfilling a use case that's highly personal, and very emotional.

"There's a lot of vulnerability. They might fail in front of their fans, and friends and family. That's a core part of the psychology, and any Growth improvement has to start from there -- the unique psychology of YOUR users. What people call best practices end up being just common practices." 

People on the Internet aren't always looking for low friction scenarios, and high functioning perfectionists aren't always looking for high friction work either. "The same person in another situation could desire low friction. Maybe they're on their commute and just want to play a game, and they want it to be easy," explains Raviv.
 
But the friction that someone can tolerate -- or require -- depends on the user model:

  • What's their psychology?
  • What are their needs and worries?
  • What are their daily processes, and who else in their organization do they have to convince to make a decision happen?

By the time a Creator ends up launching on Patreon, says Raviv, they've thought about it for a very long time. If the process to create and launch a campaign requires no mastery, it can be boring or suspect. 
 
The Growth team launched a new campaign editor that was more like a professional backend tool than a WYSIWYG editor. It was also packed with examples and links to help entries and blog posts. The onboarding flow ended up with higher friction and a higher learning curve, but the right Creators wanted education and were willing to spend extra time to get their campaigns right.

More Essays on Growth, One Email Per Post. 

Conclusion

Project Mondo was just one part of Patreon's growth story, and though it logged very specific wins, it highlights two takeaways that all growth teams can put into action.
 
1. Get insights from the front lines. 
 
Tal and the rest of the growth team at Patreon didn't assume that they'd come up with all the right questions to solve the onboarding riddle. Instead, they looked for insights and clues from the people in the company who were the closest to their core customers -- sales, community, and customer support. 
 
These teams are often traditionally left out of the conversation when it comes to growth and product, but they're also the people most likely to have a direct line to customers' unsolved problems and needs.
 
2. Customized growth problem solving > best practices. 
 
Sometimes you have to reject "best practices" -- like isolating test variables -- in favor of what's more right for your growth needs, as Patreon's growth team did with in one fell swoop with Project Mondo.
 
Finally, here's a list of the solutions that Growth implemented as part of Project Mondo:

  • New Explore Page
  • New Sell Page
  • New Editor
  • In-context examples
  • Included links to help/blog articles
  • Heavyweight backend tool rather than a WYSIWYG editor
  • A new email drip reinforcing everything in the product and all educational knowledge
  • Authentic inspiration and Creator cheering in the form of Jack Conte's iPhone video
  • Custom confetti upon successful launch

Special thanks to Tal Raviv for his generous contributions to this post (read growth essays by Tal here). 
 
Additional recognition and credit belongs to the Patreon team who did the work to make this case study possible:

  • Sean Baeyens, Taryn Arnold, Ellie MacBride, Carly Rose (Marketing, Community, Events) 
  • Christine Donaldson, Jordan Cope, Heather Wilder, Dave Hunt, Jamie Shroff (Sales, Customer success, User research)
  • Angela Raiford, Evan Keith, Elizabeth Rofoli, Jake Alster (Support)
  • Mike Jonas, Andy Coenen, Jason Byttow (Engineering) 

PS -- Patreon is hiring. Learn more about what it’s like to be a part of the Patreon team here.

Build A Growth Machine

-  Unlock growth for your business, product, or yourself
-  Learn how to never run out of new growth ideas
-  Learn how to rapidly test and analyze experiments
-  Learn the 5 key foundational elements for growth

Headline

Comments

My immediate response, holy f*ing YES!!!! My second response, lets write about this.  I've spoken/written about the importance of onboarding a lot before. The number of solid cases/examples out there is few and far between.  So Susan Su, Head of Marketing at Reforge, dug in with the Patreon team to produce this amazing piece.  It covers:

  • How Patreon decided and defined a leading indicator activation metric.
  • The challenges they faced with that metric.
  • The six hypotheses they tested, where the hypotheses came from, what worked and what didn't. 
">