MOBILE GAMIFICATION

How Stuff Works Quiz

HowStuffWorks, a pioneer in online journalism for over 20 years with 30m monthly site visits, hosts the HSW Play platform, offering quiz content on subjects like geography, astrology, and engineering. Despite its popularity, HSW Play faced challenges with user retention metrics such as low questions per session, short session duration, and limited social shares. I spearheaded initiatives to investigate the causes behind these issues and implement strategies to enhance user engagement and experience.

My Role

  • Led UX strategy, research and experience design

  • Connected frequently with product, engineering and data science stakeholders to present my findings.

  • Partnered with UI design on original components

GATHERING DATA

Persona Validation

Up to this point, the product team had depended solely on quantitative data (via Google Analytics) to solve problems. I recognized right away that without understanding user motivations, goals and behaviors, we’d continue to struggle with how to improve retention metrics. I was able to convince stakeholders that design personas were a worthwhile time investment as the deliverable could be shared across marketing, editorial and product teams for future improvements.

In order to build personas, I collected additional data from a number of sources including:

  • In-context Hotjar feedback widget installed on 1% of quiz traffic over 1 week

  • Customer support emails noting delighters and points of friction

  • A 4 question Hotjar user feedback survey about user identities and goals

  • Scroll heat maps

To analyze, I created insight “tags” which I tallied to create a larger picture of our user. Bar graphs and word clouds proved useful in communicating the process. I merged this with our existing analytics to compile 4 core personas.

Product personas

With validated personas we were ready to begin testing on the retention trouble spots. I ran remote moderated tests on our two largest groups: Retired Hobbyist and Busy Know-It-All. I leveraged Usertesting.com’s recruiting panel to find 5 participants from each group (web and mobile) based on the demographic parameters tied to the personas. Additional screener questions about hobbies/interests helped create the most organic quiz experience possible so the participants felt bought-in to the quiz they were tasked with taking.

The richest insights were centered around participant’s motivations for taking quizzes in the first place. I was also able to understand why certain content resonates with these key groups. Some additional notable findings included:

  • Nearly all participants held conservative views about privacy and sharing on social media

  • Challenging an aging mind and keeping in touch with distant friends and relatives were the top reasons for engaging with online quizzes

  • The Busy Know-It-All takes pride in competition with friends and finds the most joy in entertainment-based content

  • Nearly all participants spent leisure time at casinos or played cards socially

USER TESTING

Remote Moderated Interviews

user behavior map image

VISIONING

A Value System

The testing left us with two key learnings I felt could be leveraged to build retention features:

  1. An overwhelming number of participants enjoy card/casino gaming

  2. The quiz was viewed as a good use of time to either socialize or learn valuable information

We decided to experiment with game dynamics strategies to appeal to competitive interests while also highlighting the benefit of acquiring knowledge on a topic of interest. The idea was that the higher a user scored the more likely they were to share, and therefore fulfill socialization goals.

I partnered with our UI Design Lead to work with imagery inspired by casino gaming: points became coins, levels became diamonds. Users could accurately assess how long quizzes would take with the introduction of a red and green progress tracker. A tap on the casino-themed icons would reveal the game rules, a useful tool when traffic isn’t organic.

We saw opportunities to incentivize quiz completion with sharable expert badges. These badges made it easy to challenge a friend while spotlighting the knowledge gained from spending time with us.

Additional Features

 In addition to creating this value system, we saw a retention opportunity in the link between content. Our earlier user tests revealed that with the bulk of our traffic coming from Facebook, quiz-takers struggled to find the exit point to access other quizzes. Once they were able to find a way off the page, it took an average of 15 seconds to select one. To remove this obstacle, I designed a prototype with a countdown component to seamlessly roll the user to a related quiz. In order to ensure related content was the right fit, we shared our goals with the data science team. It was fortunate timing that HowStuffWorks was also in the process of undergoing a taxonomy review which made the likelihood of a user getting the right next quiz more likely than ever.

We also added animations, icons and brighter colors to make quiz results more sharable. Finally, survey results revealed a preference for sharing quiz results via text so we added a copy link button for this purpose.

Results

We released V1 this feature in May of 2019. We first released the progress tracker and new results component with timed, automatic recirculation to next quiz. After implementing the retention features HSW Play saw a 3 question per session increase, a 10 point improvement in Net Promoter Score (NPS) and a neutral Facebook share rate.

We saw less success in the recirculation between first and second quiz. User testing revealed that this interaction felt inorganic and “jumpy”. We’re working on improving this before releasing V2 with updated login expert badges and leaderboards. We’re hopeful with the second release, badges will contribute to an improved share rate.

RETROSPECTIVE

Closing the Loop

We closed out the project with a retrospective across engineering, product and data science. The areas we felt we could grow the most were around meeting structure and pace of decision making. Together we decided for our next feature release we’d commit to reduced meetings with more focused agendas and communicate timeliness more clearly.