All posts tagged testing

Shopping on Facebook: Beta Testing & A/B Testing

There’s no such thing on Facebook as a “staging site” or a “sandbox”. So as you develop your embedded shopping experience you’ve got come up with a way to do it that won’t cause problems for your Facebook Fans or other visitors. This can actually be more difficult than it sounds, since the whole point of social shopping is to enable a shared experience and integrate fully with your Facebook presence, and how can you do that without disrupting users?

And once you’re live, can you manage A/B and multi-variant tests like you would on your own web site, or do you have to go through additional steps? And how do you seamlessly manage code updates so they don’t affect live shoppers?

Facebook Test Users

First, the easy part. If you’re testing involves user interactions, Facebook has a way to associate “test accounts” with applications so you can easily manage manual and automated testing via these accounts. You can read the official documentation here. Test users keep you legal and avoid Facebook’s automated secret police units that weed out fake users. In other words, they remain persistent (with a few caveats) so you can expect problems during testing to be related to your code, not your user.

Creating the Fake Fan Page

I don’t know if this is officially sanctioned or not, and certainly sounds like a Bad Idea, but during most of my testing, I simply create a fake “fan page” for a fake company, so I can link my app in to the menu and do strenuous testing, including browser testing. I use a single fake page for all my testing, and I have read that Facebook finds and removes them somehow, but I’ve never been caught I suppose. Unfortunately over time I seem to be getting real fans, so if I’m not careful I might have to convert the fake company to a real one just to keep them happy. 🙂

With a fake company page you can add the FB App and leave it there for others to test and comment, all the while keeping it hidden from real shoppers who could cause real problems for merchants and themselves. From the fake fan page I can fully flesh out templates and manage iFrame issues like pop-outs and such. Your mileage may vary, and of course it depends on your e-commerce platform, but with the CV3 platform a lot of testing involves making sure the Facebook Theme variable remains set, or gets unset if you pop out of the iFrame.

Moving to Your Real Fan Page

This is the toughest part of all, and I’d love suggestions on how this is best accomplished, but rolling out your shopping app to your Facebook Fans seems to be an all or none proposition. I’ve seen it done in different ways, but the simplest and most obvious is to add the app to the left-side menu with a parenthetical disclaimer and then follow that with in-app messaging and/or removal of checkout buttons.

So, for instance, your users might see in your page menu the link “Shop With Us (beta!)” and when they click on it and your app loads they might see embedded messages explaining what they can and can’t do at the moment. One neat idea is to add a prominent button during testing that simply pops them out if the iFrame and takes them to your real site if they have trouble.

In addition, you can comment out “add to cart” or “continue to checkout” buttons as needed during testing if you are concerned about customers having major issues.

The only saving grace here is that because of your fake fan page testing, by the time you reach this step you are necessarily ready for public testing and should have the entire shopping experience to a point that you shouldn’t have too many issues. One tip that will make this easier: anywhere in your app that calls for the fan page ID should use a variable, so you can swap out the ID in one place as you move back and forth between testing and live.

A/B and Multi-Variant Testing

This is really a topic for a larger conversation, and the resources are pretty limited still, but in general there are two things you need to be wary of when testing the Facebook shopping experience for better conversion. First, the tools you use must function within the iFrame environment. People have had trouble with many of their favorite tools, including Google Website Optimizer. One big issue arises when an embedded iFrame tries to set cookies at a domain other than the parent window, which pretty much defines exactly how Facebook Apps are intended to operate. This predominantly seems to affect Internet Explorer. Don’t ask my why, but setting an additional header seems to have cleared up most of my problems. Here’s the header, in PHP form:

<?php header(‘P3P: CP=HONK’); ?>

The second thing to keep in mind is the mathematics behind your definitions for success. You may choose to simply value the addition of a Facebook fan in the same way you would consider a catalog or e-mail sign-up on your web site, but be careful. We are talking about social shopping after all. Do your fan counts expand at ever increasing rates, faster than e-mail sign-ups? This certainly indicates that the valuation should be different on Facebook.

What about purchases? Are they valued in the traditional way, or do you see an upswing in fans or orders when purchasers automatically post their purchase to their wall?

These are just a few of the things that crop up when considering exactly how to embed your shopping experience inside Facebook. I’m certainly interested in hearing from others, feel free to comment and let me know what is working for you.