Closing the Gap Between SMS Purchase and First Send
Mailchimp was seeing a troubling pattern: users were purchasing SMS credits monthly but letting the majority expire unused. I led a mixed-methods study to understand why adoption was stalling and what it would take to get users from purchase to first send.
Why this research was needed
Mailchimp had launched SMS marketing as a new product line, and early data showed two compounding problems. First, users were purchasing credits monthly but letting a majority expire, even after many months. Second, the time to first action was extremely long.
Unused credits meant users couldn't realize the value of SMS, which directly affected churn and active paid SMS users. The team needed to understand whether this was a comprehension problem, a strategy problem, or something else entirely.
Methodology
Before fielding, I spent time on discovery and stakeholder alignment to make sure I was asking the right questions and had buy-in on the research framing before committing to a fielding timeline.
The survey went to 3,474 users who met the screener criteria: paying for Mailchimp SMS for more than two months without having added contacts or sent messages. We offered compensation and fielded for 2.5 weeks using Qualtrics, with analysis in Excel and R. A survey with 140 responses helped us understand and prioritize challenges at scale.
I conducted 11 structured interviews with users who had been paying for Mailchimp SMS for more than two months and had either added no contacts or fewer than 10. The narrow screener was intentional because I needed people who had tried to get started and stalled somewhere in the process.
I ran a team synthesis session using affinity mapping with two designers and a PM. I wanted stakeholders in the room building the framework with me, and it shaped how the findings were received by the team.
Findings
Three core insights emerged from the combined data, each answering a different facet of why users were paying for credits they never used.
Three core insights
Lack of Marketing Strategy
Users didn't have a marketing strategy before they signed up for SMS. They purchased credits with intent but no plan, which prolonged the time to first send and led to credits expiring unused.
Confusion About Credits
Users didn't understand how credits worked. Key gaps included not realizing credits expired monthly, not knowing that emojis and images affected credit consumption, and not understanding how subscriber count related to usage.
Mental Model for Minimum List Size
Users new to SMS were waiting for their subscriber lists to reach a perceived threshold before sending. They didn't want to "waste" a send on a small audience, creating a self-reinforcing delay loop.
What changed
The research drove three product outcomes. An onboarding checklist paired with highlighted educational content about SMS marketing on the homepage gave users a clear path from purchase to first send, addressing the strategy gap directly. This shipped as a redesigned SMS homepage with step-by-step guidance.
A personalized checklist experiment tested whether tailored onboarding could help users get started more easily and build confidence in their SMS marketing approach. This ran as a controlled experiment to measure activation lift.
A credit expiration banner on the SMS homepage reminded users that their credits were expiring, plus an in-context reminder when drafting an SMS. This closed the comprehension gap around monthly expiration and shipped to all SMS users.
