TurboTax Personalized Onboarding

For this project, I worked on utilizing existing data models to create a more relateable onboarding experience for self-employed customers based on the industry they work in. This is meant to be a proof of concept for personalizing experiences across TurboTax at scale.

Date

September 2019

Skills

Product Design, User Research, Prototyping

Tools

Sketch, Invision

Background

TurboTax is a do-it-yourself tax filing software that breaks down tax language into easy-to-answer questions. As a product, it has various tiers of service dependent on a user’s tax situation. In the tax year of 2019. we wanted to tackle the problem of users dropping off early in the experience as they went through our simple onboarding experience. Some initial findings showed that as one went up in tier (and in turn, complexity), it was more likely for a user to drop off early. This led to testing across SKUs, with more focus on our highest tier product for self-employed customers.

We wanted to challenge the perception that additional screens in onboarding would lead to further drop off, while showing that personalization provided enough value to drive retention instead. This proof of concept around creating personalized content was one of three tests we launched in this space to learn more about why customers were dropping off in the early experience of tax filing.

In terms of current personalization in the product, we have a data model built out to match a customer’s self-reported line of work to an IRS business code, as well as logic for a correlated list of common expenses per industry.

Problem

Through data analysis of the previous tax season, we noticed that people with more complex tax situations, especially those with Self-employment, had a much higher drop off rate early in the experience than those with more “simple” situations. This was easier for us to explore as TurboTax had a complexity-based pricing model. We discovered that users in Deluxe, which covers the most common tax situations, had a 14% drop off… versus the 24% of customers who drop off early in the Self-employed product.

While that could make it easy to jump to conclusions about the problem, I partnered with my Product Management partner to dig a bit deeper. We investigated search verbatims in the first few screens of TurboTax for customers in this tier. We discovered that ~25% of searches were related to self-employment tax questions.

I’m self employed for the first year but didn’t know how to estimate taxes. What do I do?

Aside from this, from our past research, we know that the self-employed customer base is non-homogenous. With so many unique situations across industries, different ways self-employment fitting in one’s overall financial picture and varying personal perceptions of the work they do, we knew we needed to give these customers an experience just as unique as they are. At the time, we had pretty much a one-size-fits-all onboarding solution for all customers (aside from a few details related to the tier they are in), and we knew this wouldn’t cut it.

In order to determine scope for this project, we crafted the following customer problem:

I am new to being self-employed this year. I am trying to file my taxes by giving TurboTax a shot, but I am unsure if it can handle my unique situation and if I can do it all by myself. This is because I have so many questions related to self-employment, which makes me feel unsure, doubtful and hesitant to even give TurboTax my information.

One caveat here is that while we have a relatively narrowed problem, we narrowed in on customers who are new to being self-employed and new to TurboTax from a business perspective for now. We don’t have a lot of info on these customers because of the nature of them dropping off so early in the experience, so we used business goals to narrow for now. This was also based on demographic data as we saw customers who were new to self-employment historically dropped off over 50% more across the full end-to-end experience as well.

Constraints

We were a few short months away from launching the TurboTax product for the year and had to work with a few constraints. We were limited to existing UI, tech and data capabilities, especially given that we were trying to test a few different recipes in the space. This meant that we’d be a bit limited in how we might pay off true personalization in this experience and scalability was not the first priority yet, especially without new machine learning models in our back pockets.

Exploration

While personalization was an overall theme we landed on due to the goal being to create an onboarding experience unique to self-employed customers, we wanted to explore what would be deemed as valuable by customers early in their experience, as well as feasible within our constraints. I led a Brainstorm and some Sketch sessions with various cross functional partners to ideate on potential solutions in the space.

Through this exploration, my PM partner and I started affinitizing the themes we saw across the ideas and landed on two general themes to explore. One track became focused on utilizing tax experts, a new addition to the line up through TurboTax Live, and led to two recipes which aren’t the focus of this case study. This one, which is the third recipe, became focused on exposing personalized content by industry as a means of building confidence.

User Research & Testing

Because the UX Research team at TurboTax had been conducting various research with self-employed customers over the last couple of years, the extent of research here mostly meant for me to revisit past findings. Here’s some that I anchored my iterations on

Self-employed customers find value in “industry specific deductions” that we show them in TurboTax because it makes taxes more relateable to something they know.
Related to that, self-employed people typically feel alone due to the nature of working for one’s self, so they try to find other people in their industry to for advice and mimic behavior.
Customers coming into TurboTax have varying levels of intent. Some people early in the experience have low intent and are likely “shopping around” for tax software options.

I sketched a good variety of ways to personalized and realized there were two pieces to personalizing. First is gathering data to understand who the user is, and the second is the pay off to truly show a personalized experience based on the data we get.

Again, because of our constraints we had to rely on customers to willingly provide this info to us which we recognized as a limitation to improve on in the future. The concepts here mostly reflect the pay offs, but I explored what we’d need to learn about customers in order to deliver personalization. I put these together in a prototype and launched it for user testing.

Initial learnings

Through unmoderated testing, I learned:

Adding questions about self-employment didn’t feel like additional work and many were willing to provide the info.
Participants found the photos related to their industry to be appealing and relateable.
They found value in seeing the value and examples of deductions specific to them they can claim so they can check off what they have.
Social proof was not well received. Many of them felt like it didn’t matter to them how many other people in their industry filed with TurboTax.

Testing a V1

To get in market learnings, wee launched the test to a small demographic for tax filing deadline extenders to help us get directional learnings if this was doing well.

Our hypothesis was, “If we create an onboarding personalized to one’s self-employed industry, then customers will be more confident that TurboTax can handle their unique situation and continue to file with us.”

We focused on the top 80% of industries for creating the personalized content. We provided a generic pay off for those who choose either to not provide us with enough info to personalize, or were part of a cohort of industries we didn’t have support for.

I worked with my content design partner to make the flow feel friendly and segue nicely into self-employment. The original intent behind these onboarding screens, fondly called “Get to Know Me,” was to learn more about customers at a high level across their whole picture versus going deeper on specific aspects, so this was something new for us.

If you know anything about personalization, it’s hard to truly personalize for someone when you don’t have data about them. We had explored a lot on what to do once we knew enough about a user, but not a lot on what to do when we have less info. We opted to show key value props for the product itself to stay true to the goal of addressing anxiety around self-employed taxes.

This semi-interactive prototype shows you at a high level what we asked customers, along with the pay offs we provided them in return. The first shows the personalized one, and the latter shows generic.

Here are the outcomes for the V1:

Those who got a personalized industry match were 11% more likely to continue with TurboTax than those in the baseline recipe.
Users who got the generic pay off were slightly more likely to drop off than baseline.
Extension filers are a different demographic than the overall self-employed cohort filing with the product. Most considered their work to be a business versus a side gig.
There were no negative impacts to adding more interactions, especially when it means asking more specific questions early in the experience

Refining

We iterated a bit further for the next tax season. For next steps, we wanted to dive deeper into seeking meaningful personalization for the people with no industry matches. Also, from a design perspective, I wanted to also simplify the experience so that we’re focused on asking questions that we are paying off versus asking all questions related to self-employment. This means staying focused on asking about the type of work they do.

In terms of iterating on the pay offs, I mostly kept the personalized pay off the same to test it across the general demographic. I looked into what worked well in this variant and ideated to how to add some level of personalization even when a customer is not part of the industries we got to personalize for. I came up with a hybrid solution where we still utilized specific examples that can apply to many industries. I also worked with our PM and development team to add further add benefit for them by adding the description they add as an income source for them to work on later.

Here’s how I updated the screens, with the dynamic content highlighted.

Outcomes

We ran the A/B test through the end of the tax filing season. This test won by 6% during extension season but lost during “regular” tax time by 1%.

Going into the details, starting with those who filed by the traditional tax deadline, we saw an 8% increase in volume of actual filers for customers who saw a personalized pay off. We recognize there’s some selection bias here as these are the customers who gave us information about their industry. A total of 80% of customers who saw the recipe were willing to provide info on their self-employed work early on. The other 20% skipping the experience altogether and we were not able to reduce abandonment from this cohort.

As for extension filers, there was a 6% increase in likelihood to stick with TurboTax upon seeing us ask about their industry early in the experience, even among those who skip it. We attributed it to a halo effect of TurboTax wanting to dive deeper into getting to know them as they start.

We determined that this variance is just proof that personalization, especially with a minimal amount of data, has a fine line between showing you know someone and being just completely unrelateable. This also further proves that self-employed customers are non-homogenous and we need to investigate what works and doesn’t for various cohorts and how to scale that. As for this being a proof of concept, it worked! It was enough of a lift for us to consider further personalization. As of the time of writing, we are investing in going deep on specific industries to provide personalization not just in the start experience but across the end-to-end product, as well as investing machine learning and utilizing various data sources to get more data with reduced selection bias.

Takeaways

I love going over this case study because it was a proof-of-concept for something that we are now doubling down on, and because it was… a bit of a failure. But we did come out with so many learnings that then established further business cases. Here are some of my learnings:

Personalization is a fine art. When something is relateable, customers are more likely to stick. However, without enough info gathered on them, it’s easy to offer an inaccurate personalization which can be a factor for abandonment. Getting rich personalization is heavily reliant on data.
When it comes to doing work or providing information, products need to provide enough of a pay off to validate acquiring the info in the first place. This leads to transparency and trust with data governance.
Scalability is an important factor for any product, but in terms of personalization, that’s difficult to achieve done the way we did. From a business perspective, personalization makes sense but needs to be done in a way that isn’t as manual as we tried for this early proof of concept to make it worth it.
Even when things don’t work out, it’s worth trying because you’ll learn something. It might even determine a new business strategy 🙂