How I Used A/B Testing Approach for Social Introductions While Abroad

When most people think about A/B testing, they often only think of it as some B2B tool that helps them improve their website or copy. A/B testing isn’t just a tool, it’s an approach to improving upon something that already exists. In a nutshell, A/B testing is simply taking a scientific approach and applying it to a product or process to see if a change affects the desired results.

This approach is applied not only to products, but to people, medicine, and everything around us. Many well-known public figures such as Tim Ferriss or Aziz Ansari use A/B testing to improve their lives or professional skills by using hard data and experimentation to see how their actions affect their results. It’s not just a tool, in an approach to improvement that drives results using hard data.

When I first moved to Japan, I used an A/B testing approach to determine the best way to meet strangers. Having suddenly transposed my life there, I knew that to have an amazing and fun experience, it’d be best to meet and befriend locals who could show me what life was really like in the prefecture.

The A/B Test

My goal was to foster more conversations with strangers in order to make new friends, as well as learn more about the country and its culture. I wanted to see if using different phrases would better serve as segues into conversations, and if using Japanese rather than English would increase the likelihood of continuing a conversation.

I decided to test 3 different phrases in both Japanese and English, introducing myself to strangers.

How are you? (Are you well?)
お元気ですか？

Good morning.
おはようございます

It’s hot today (isn’t it?).
暑いですね？

To track effectiveness, I decided on some simple KPIs: whether or not others would follow up with a question directed at me, and whether the conversation continued onto another topic. Over the next couple months, I conducted this test on about 100 strangers I ran into, and took note of their reactions.

The Results

How are you? / お元気ですか？ (Are you well?)

English:

I found that this phrase made many people flustered, and they typically would respond with a nod or just a nervous smile. Oftentimes, they would give off indications that they wanted the interaction to end as quickly as possible.

Japanese:

Usually, the other person would respond, “I’m good (in Japanese), or nod and acknowledge that they were well, and then the conversation ended. On a rare occasion, they would respond back with the same question and the conversation would end after my response.

Notes:

Since there isn’t a commonly used phrase to ask this that doesn’t prompt a yes/no answer, the results were a little disappointing. I expected people to respond well and ask the same, but oftentimes they did not.

Good Morning / おはようございます。

English:

They would usually smile and say good morning, then ask where I was from, or if I was Japanese (in English). Many seemed excited or curious, and were okay with replying with a simple question back. Oftentimes this would eventually lead to the question “do you speak Japanese?” where we could pick up and continue in Japanese.

Notes:

“Good morning,” and “Where are you from?” are well-practiced phrases in most English curriculums. The individuals I talked to who responded usually knew a logical question to ask in that situation since they had practiced it in school, and either seemed excited at the prospect of using their knowledge, or pre-programmed to respond as they did.

Japanese:

Usually people would give the same greeting back, and maybe 15% of the time would ask me something if they had question that could be related to the situation in mind. (i.e. Where are you headed?) Usually storekeepers or other people paid for customer service would respond back with a segue.

Notes:

People generally reacted as I expected them to, as the same phrase in an English speaking setting achieves similar results.

“It’s hot today isn’t it?” / 暑いですね？

English:

The worst performer by far. Most often they would just smile politely and not say anything, or look incredibly confused.

Notes:

I don’t think most people I asked understood me, since it’s not a familiar question taught in school. Many people seemed bewildered as to what an appropriate response would be, so they just smiled and nodded.

Japanese:

Surprisingly, even though I always received a response of “yes it’s hot,” 80% of the time, the conversation ended there.

Notes:

I expected the conversation to continue about 50% of the time, but more often it just ended there. If they did respond, it would be something related to the situation we were in.

Conclusion

The findings I learned from my experiment helped me improve my experiences while overseas. Since I was there to learn about culture and what life is like halfway around the world, it important to me to build relationships and talk to people. I don’t have exact numbers of how many more interesting conversations I had, but I can say with confidence that using a simple “good morning,” has served as an effective building block during my years in the country towards making many new friends in a foreign country.

A/B testing had a huge impact on my lifestyle while living in Japan. Using the results I gleaned using the method, I was able to optimize openers for conversations that led to ongoing friendships, and to many memories that I’ll never forget. In the same way I applied testing to my life, you can also apply testing to yourself and/or your product. Remember A/B testing is not just a one trick pony for increasing web conversions, it’s a method to improve results of any kind. Once you understand how it works, you’ll know how you can apply it to all sorts of situations, and generate data that will drive important decisions.