Opinion: Google A/B testing Android features is a necessary evil

At the tail end of last November, Google published a blog post detailing 7 new features to celebrate Messages’ 1 billion RCS users. The features, ranging from shared themes and screen effects to AI-powered reactions, include Photomoji, Voice Moods, Screen Effects, Reaction Effects, Custom Bubbles, Animated Emoji and Profiles.

At the time, Google said interested users can download Messages beta to start enjoying these features, but warned that availability may vary by market and device. This explains why some Google Messages users haven’t received Photomoji and other features on their Android phones while others already have them. In fact, as an Android user, you’ve probably noticed that sometimes your friends have features on their phones that you don’t. Or, maybe you’ve had a feature appear on your phone, only to have it vanish a few days later. This isn’t some kind of weird phone sorcery — it’s called A/B testing.

A/B testing is a way for developers to experiment with different versions of apps and features to see which ones users like best. They’ll show one version of a feature to a small group of users, and another version to a different group. Then, they’ll track how users interact with each version and see which one gets better results.

This is what Google has been doing with the new Messages features released last year. While some people have received and used some of these features, others haven’t. For a few others, some of these features keep appearing and disappearing, which is understandably frustrating to a point that some now want Google to do away with A/B testing and go the Apple way of pushing new features to everyone in the beta program at once.

However, while I understand the frustrations, doing away with A/B testing would be a terrible idea for Google and Android at large for a couple of reasons.

Unlike Apple where beta features become available to everyone in the beta program at once, Google uses A/B testing where features are released to select users and then availability expands gradually. This is helpful for a number of reasons:

We all know how fragmented Android is. The sheer number of devices means Google Messages features have to be optimized to work on each one of them. This means optimizing the features to work on multiple processors, screen sizes and across multiple versions of Android. Unlike Apple that has only 5 or so chipsets to optimize their apps/features for, Google Messages, for instance, must be optimized to work on dozens of Android chipsets on billions of devices. This makes A/B testing necessary for Google.

Ultimately, the goal of A/B testing is to make Android apps and features better for everyone. By seeing how users interact with different versions of these features on multiple devices, Google can learn what works and what doesn’t, and then use that information to improve the final product.

Sometimes, app developers may have an idea for a new feature that they think is going to be great, but then users end up hating it. A/B testing can help Google avoid making those kinds of mistakes by catching problems early on so that Messages users don’t end up with half-baked features.

Given the circumstances above, it’s not possible for Google to push new features to everyone in the beta program. With A/B testing, if you’re in one of the test groups, you get to be one of the first people to try out new features. That can be pretty cool, especially if you’re a techie who loves to tinker with new things.

Of course, there are some drawbacks with A/B testing. For instance, there’s plenty of confusion in the Google Messages subreddit. It’s frustrating to see your friends using features that you don’t have, or to have a feature disappear on you. It may feel like Google is playing favorites, or that they don’t care about your experience. Furthermore, A/B testing can be disruptive to your muscle memory because of features appearing and disappearing. It can make it hard to get used to a new feature if you know it might not be there for long.

Ultimately, A/B testing is a necessary evil that ensures Google and developers in general send out apps and features that are at the best they can be. But, looking at the case of latest Google Messages features, it becomes obvious that devs need to be mindful of the user experience and to be as transparent as possible about what they’re doing.

That said, here are a couple of things I think Google could have done to make the ongoing A/B testing of features less frustrating for Messages users:

• Be more transparent: Let users know which features are being A/B tested and why they might not see them all. This can be included in the release notes of the update that introduces the new features in question.
• Minimize inconsistency: Try to avoid having features appear and disappear. This is by far the most frustrating bit, so it can also be mentioned in the release notes so that users become aware before proceeding with the beta testing process.

By taking these steps, Google can make A/B testing a more positive experience for everyone. And while there’s no doubt A/B testing is a complex issue with both benefits and drawbacks, it’s important to weigh the pros and cons carefully and to make sure that the user experience is always a top priority.

What do you think? Let us know in the comments below.