Subscriber Onboarding Study
Overview
Following our 2021 subscriber retention study, the Product Design team had the opportunity to create an on-site onboarding experience focused on personalization. We began with a foundational research study to better understand what benefits messaging new subscribers see organically, what benefits they learn about through onboarding emails, and how their opinions of the site changed after one week as subscribers.
The insights in this study build directly on the first study, where we recommended creating an onboarding flow to help subscribers personalize content and understand their benefits.
Objective
Understand how first-time digital subscribers use and experience their local news site in their first week after subscribing.
My role
I led UX research, including:
Following our 2021 subscriber retention study, the Product Design team had the opportunity to create an on-site onboarding experience focused on personalization. We began with a foundational research study to better understand what benefits messaging new subscribers see organically, what benefits they learn about through onboarding emails, and how their opinions of the site changed after one week as subscribers.
The insights in this study build directly on the first study, where we recommended creating an onboarding flow to help subscribers personalize content and understand their benefits.
Objective
Understand how first-time digital subscribers use and experience their local news site in their first week after subscribing.
My role
I led UX research, including:
- Writing the research plan
- Selecting study participants
- Reviewing videos and survey responses
- Analyzing results through crosstab analysis in dscout
- Running group affinity mapping sessions
- Synthesizing insights
- Co-authoring the presentation with our results and recommendations
Process and methodology
We conducted this study in dscout, a qualitative research platform that enables us to create screeners to recruit users. dscout uses in-the-moment video and screen recordings, along with written questions, where we can observe users' thoughts and behaviors over several unmoderated sessions. Note that in the slides below, we use the branded dscout term "scouts" to discuss study participants.
This study included real local news readers from Gannett’s top 25 markets, half existing readers of a Gannett site and half readers of a competitor site. We gave them a digital subscription to their local Gannett site along with a test email account to match a real subscriber experience as much as possible.
Selecting participants: We selected 12 participants from 1,894 applications from local news readers in our top 25 markets. Participants skewed young (25-44) because we excluded anyone who previously subscribed to their local news. This study was intentionally kept small because it was so in-depth, and 9 people completed the study.
Study structure:
- Part one focused on test setup, where participants signed in to their testing emails and went through the real subscription process on their local news site.
- In part two, existing readers discussed their opinions of their local site and news readers familiarized themselves with their local Gannett site. They captured their impressions in a survey, including Likert scale questions to help quantify their feelings and opinions.
- Part three had participants review onboarding emails, recording their experiences reading three emails on separate days.
- In part four, participants recorded themselves browsing their local news site, discussing their likes and dislikes.
- Part five was completed at the end of the week and asked very similar survey questions to part two to capture how, if at all, their opinions changed.
Analysis: We pulled our insights from close-ended responses and open-ended written and video responses. We exported our notes and user quotes and used affinity mapping to identify themes between participants. See the full affinity map.
What we learned
- Onboarding begins at the end of checkout. Most participants began exploring the site before they opened any onboarding emails, making the subscription confirmation page the first opportunity to showcase subscription value. Users have spent money, given us their information, and want to jump in and explore.
- Benefits information should be actionable, not informational. Subscribers expected to learn about their subscription within the product experience, not by email. They felt the benefits in onboarding emails were not clear and there was little guidance on how to use them. For example, one email discussed sharing a bonus subscription but didn’t include instructions or a link to do so.
- The daily news digest email is our #1 onboarding email. When asked to review emails about their subscription, participants overwhelmingly chose the Daily Briefing over onboarding emails. They did not understand the difference between editorial and marketing emails, nor did they know the Daily Briefing is free for everyone.
Because they received the Daily Briefing after subscribing, participants believed it was a subscription benefit and gave it inherent value. Unfortunately, the lack of fresh and relevant content and the amount of low-quality advertisements were continually cited as a source of frustration.
- Immediate, active personalization is a baseline expectation. Users expected a personalized news experience after subscribing. Participants repeatedly voiced their frustration when presented with content that was of no interest to them in the Daily Briefing newsletter, on the site, and in the e-Edition. They were actively looking for ways to make the products more relevant to them, like filtering content.
- Subscribers want to know we’re paying attention. Generic email messaging reinforces their belief that a subscription experience isn't much different from that of a free user. As subscribers, they give us their money and information and expect a more custom experience in return.
On the first day after subscribing, we send out a survey asking multiple questions about their interests, yet none of this is applied to their experience. If we say “we would love to learn more about you,” it needs to help them as much as it helps us. - Highly engaged readers quickly become candidates for churn. Before subscribing, long-time readers believed they were not accessing the most valuable content on the site. Once they had full access, the content didn't live up to expectations. Participants expected subscriber-only content to be exclusive, higher quality and relevant to their interests. After one week of subscribing, long-time readers dropped their quality perception ratings of their local site from 4/5 to 2.5/5, a 1.5-point decrease.
- We have one week to prove we’re worth it. Participants saw their first week as a test of subscription value. After one week of using their subscription, 50% of existing readers and 40% of new readers said they would cancel based on what they saw in the study. 89% said they would cancel when informed how much a full-price subscription cost.
Users were frustrated by finding repetitive, out-of-date, or irrelevant content. Existing local readers were disappointed by how little they gained after subscribing.
Recommendations
We presented curated recommendations for marketing, editorial, and product outlining how we can move forward. Overall, we validated the need for an on-site onboarding flow focused on personalization, more passive personalization by tracking user preferences, and to redesign the onboarding emails to be more actionable and less redundant.
We presented curated recommendations for marketing, editorial, and product outlining how we can move forward. Overall, we validated the need for an on-site onboarding flow focused on personalization, more passive personalization by tracking user preferences, and to redesign the onboarding emails to be more actionable and less redundant.
Outcomes
This study prompted more interest and quick action than any other I’ve been a part of at Gannett. While our sample size was small, the project success was fueled by connecting our insights to in-market cancellation data, a previous quantitative survey of former subscribers, and four previous qualitative research studies.
The marketing team redesigned their onboarding emails following product recommendations, seeing an almost 100% improvement to email click-through rates. Phil Shroder, our head of digital user engagement, wrote more about improving onboarding emails for INMA.
The Daily Briefing, our daily news digest email, was redesigned along with our other email newsletters. We are working on a subscriber-only version that promotes benefits and exclusive content as a companion to onboarding work.
Following this study, I spent two quarters leading the design and usability testing work for two web onboarding experiences, one for new subscribers and one for registered users (people who create free accounts). While I can’t publicly share the work yet, our former Vice President of Product Design and UX Nicole Dingess talked about our research and product thinking for INMA as well.
This study prompted more interest and quick action than any other I’ve been a part of at Gannett. While our sample size was small, the project success was fueled by connecting our insights to in-market cancellation data, a previous quantitative survey of former subscribers, and four previous qualitative research studies.
The marketing team redesigned their onboarding emails following product recommendations, seeing an almost 100% improvement to email click-through rates. Phil Shroder, our head of digital user engagement, wrote more about improving onboarding emails for INMA.
The Daily Briefing, our daily news digest email, was redesigned along with our other email newsletters. We are working on a subscriber-only version that promotes benefits and exclusive content as a companion to onboarding work.
Following this study, I spent two quarters leading the design and usability testing work for two web onboarding experiences, one for new subscribers and one for registered users (people who create free accounts). While I can’t publicly share the work yet, our former Vice President of Product Design and UX Nicole Dingess talked about our research and product thinking for INMA as well.