Introduction

The purpose of this use case study is to showcase how a mobile subscription flow was optimized through A/B testing, resulting in a significant 15% increase in conversion rate. The focus of the optimization was to address the issue of users entering incorrect PINs during the subscription process, which led to frustration and dropouts.

Problem Identification

Users have to enter their mobile number to receive a 4-digit PIN via SMS and confirm the code on the page before gaining access to the portal.

Upon analyzing the mobile subscription flow, it was identified that a considerable number of users were submitting incorrect PINs. Analytics data showed that these incorrect entries included common patterns such as force-brute attempts using numbers like 0000, 1234, 2222, 7777, and 8888. Additionally, a specific number, 1568, was being frequently entered, which coincided with the system's sender identification number (shortcode) displayed in the SMS message.

The problem was exacerbated by occasional delays in users receiving the SMS containing the PIN, causing them to resort to typing random numbers in the PIN input field. After a few unsuccessful attempts, the system would block the user from trying again for a few minutes, leading to frustration and dropouts.

Hypothesis

If we prevent users from entering numbers there are obviously wrong we'll reduce the number of attempts, saving the user time, frustration and still avoiding overloading the system with invalid transactions.

Solution proposed

To address the identified problem, a solution was proposed that aimed to improve the user experience and reduce frustration during the subscription flow. The solution involved implementing a modified prototype that dynamically responded to incorrect PIN entries, particularly when users entered the blocked numbers or the shortcode.

The proposed prototype introduced the following changes:

a. Error Message Replacement: When a user entered one of the blocked numbers or the shortcode, the subscription flow replaced the "Submit" button with an error message. The error message prompted the user to enter the PIN received in the SMS message.

b. PIN Modification: The modified prototype allowed users to freely delete the incorrect digits they entered and attempt again with the correct PIN received via SMS.

A/B Testing and Results

To assess the effectiveness of the proposed solution, an A/B test was conducted, where the original subscription flow (Control Group A) was compared to the modified prototype (Variant Group B). The A/B test was designed to measure the conversion rate, i.e., the percentage of users who successfully completed the subscription process.

The test was performed on a representative sample of users, with an equal number of participants in both groups. The data collected during the test period was analyzed, and the following results were obtained:

• Control Group A (Original Subscription Flow):

  • Conversion Rate: 0.85%

• Variant Group B (Modified Prototype):

  • Conversion Rate: 0.98%

Conclusion

Through the use of A/B testing and the implementation of a modified prototype addressing the issue of incorrect PIN entries during the mobile subscription flow, a 15% increase in the conversion rate was achieved. By replacing the "Submit" button with an error message and allowing users to freely delete incorrect digits, the modified prototype effectively guided users to enter the correct PIN received via SMS, reducing frustration and dropouts.

The successful optimization of the mobile subscription flow demonstrates the value of user-centric design and iterative testing in enhancing the user experience and achieving business objectives. The insights gained from this use case study can inform future improvements in the subscription flow and guide UX professionals in similar optimization endeavors.