Testing with Firebase Remote Config reveals surprising user preference
mikan is a Japanese startup with the mission to make 100 million people bilingual. To reach this goal, they have created a series of over 15 learning apps. When they launched in 2014, their first English vocabulary learning app achieved over 100,000 downloads in just three days. However, sustaining growth and attaining their business goals through monetization has been a challenge for mikan.
After taking a trial-and-error approach to optimizing their monetization, they decided to start A/B testing on Android using Firebase Remote Config. Their goal was to find the price and subscription period that appealed most to their users.
What mikan did
Originally, mikan provided only a monthly subscription. With the goal of getting better annual recurring revenue, mikan tested three subscription plans: monthly, semi-annual, and annual.
Using the features of Google Play Billing and Firebase, mikan created their tests in just 2 weeks. Also, their users did not need to download any new features through Google Play, as Firebase Remote Config enabled mikan to change the app’s UI and UX easily without changing the app code.
mikan also modified the subscription page. Rather than explaining the benefits of the subscription, the new subscription page focused on giving users the option to select one of the subscription plans.
Before launching the test, expectations for the annual plan were low because of its cost when compared to the monthly plan price.
The test saw an increase in conversion rate of over 100% and a positive impact on Lifetime Value (LTV). mikan also found that 75% of users purchased a subscription within 30 days of registering. But, to mikan’s surprise, it was annual subscriptions that contributed most to these improvements.
“We got great results from the tests in just one and a half months, much faster than expected. I could not believe this achievement, the number [of annual subscriptions] was more than we had expected. At first, we thought that this test would give a negative result [for annual subscriptions], but the result was totally the opposite,”
Kazumasa Takaoka, CEO and co-founder of mikan Co., Ltd.
After 40 days of testing and analysis, mikan rolled the tested UI and subscription plans out to all Android users. In the 18 months since completing these tests, more than 70% of users have opted for an annual subscription, generating a stable source of income.
The benefits of testing
Perhaps the most important outcome from this test was that the team at mikan learned that their trial and error approach to monetization was not ideal. Testing gave them the right answer - an answer they were unlikely to have found by trial and error, as it was contrary to their expectations.
While it is possible to learn winning patterns from best practices, only well-designed tests and thorough analysis of the resulting data will reveal the truth about user expectations and behavior. The team at mikan believes that anyone can get great results by running tests using Google Play and Firebase. Learn more about how to add subscription-specific features to your app and use Firebase Remote config to run tests.