Path Based Segmentation as an Input to Test Design

The way analytics is used today in organization is to test new designs. Rarely is analytics used as an input to test design. Path based segmentation can be an effective way to come up with a design that is not only a combination of drawing room designs & user testing but also from best analytics practices.

1. Path Identification
Identify the main purchase paths on your site. Let us take the example of a popular e-commerce site ebay.in as an example. There are several paths to buy:
a. Deals path: Discounted offers on these pages, usually a link from the home page exists, some marcom vehicles land folks here .....
b. Default Path: Traditional path.
E.g: While buying on ebay I would start with the home page, go to a category page,
subcategory page, product details page and then add to cart and buy
c. Some specific navigation mechanisms: e.g Assisted Navigation on the left

d. Menu path: A set of menu options on top that can be used to navigate


Some examples of these paths on the ebay site are in the above buckets.
Haven't provided examples for a. and b. as these are known

2. Data Gathering & Representation
For each path, identify the visits, purchase_visits and revenue for each path.
Calculate RPV (=revenue/visits), conversion (Purchase_visits/visits) & Visit_Mix
(path_visits/visits) for each path

Represent these paths as a bubble chart with X-axis as RPV, Y-axis as visits and size of bubble as conversion.

C. Insights as an input to new test design
The example chart above is a dummy example but leads to key areas for action:
a. Layout Design to change visit mix
If you want to drive more revenues, you could change your layout to drive visits to your best performing path
(In above example, a possible test design could be to increase visits to deals. Reason being deals has the highest RPV - so you get more out of every dollar you spend by driving traffic to deals
and less traffic to menu path)
b. Feature improvement to improve RPV
In the above example, you would try and improve the feature on ebay.in going with the dummy data.
So this is a simple example where path based segmentation analysis can help you come up with new testing ideas. Obviously this has to be customized to your organizational metrics needs
(In above example, a possible test design could be to improve some path features like Menu - reason being low RPV performance but considerable visits)

What to action on in a Page that changed

A page is very important in e-tail. It is not any other webpage. The more the visits to a page, the more important it is in the purchase path - and the higher is the impact that can be delivered.

No wonder merchandising teams are trying out different versions of the page. These are usually based on intuition & experience. It is the job of the analytics teams to work with the merchandising teams to ensure these decisions are based on data.
Unfortunately analytics role today in the world of simple web tools is just about providing visits and leakage information for the page OR running a simple A/B test (resulting in 2.5% upside which when annualized becomes a big number). The role of analytics should be much more than analyzing upside and annualizing the results of the A/B tests. Analytics should be able to get back and say what drove the upside/downside. I am presenting an approach that will help the same.

1. Starting Point: Make a list of levers that the merchandising teams have in changing the page
The levers fall into 2 broad buckets
a. Layout Changes
What changed in the layout? Some sample list could be
i. Rotating Banner instead of a static one OR position of Banner Changed
ii. New Links added, Old links removed
iii. Links rearrranged etc.

b. Feature Changes:
i. Page links to a page with a new feature

Step 2a: Create a segment to isolate the entries to the page
Step 2b: Create separate segments to isolate next clicks to pages from the page X
This is where it is imperative that you have a real web analytics tool at hand. You cannot do this in basic tools like SiteCatalyst or CoreMetrics or Discover. You need Insight/Visual Sciences

Step 3: Repeat Step 2 for the page 'before' and 'after' if you are doing an anachronistic analysis or repeat for the versions of the page being tested

Step 4: Tabulate as follows:


Cursory look reveals that for all landings on the page,
* Overall conversion of page improved
* Next clicks were fairly similar
* Bounces were little reduced
* Newly added Page E doing well
* Page A with changed features doing well

Now this can be translated as per our step 1 framework as follows:

Clearly upside in conversion is coming from the new Page E visits, from changed features on Page A & increased visits to Page A that has higher conversion. So business knows what improved

Advantages of my method.

1. Upside might not be coming from the changes to Page X that is being tested but rather because it leads to page Y with a new feature.
In such cases, my method captures the source of the changes
2. In cases where the page does not work, the merchandising & Design teams know what exactly to go and fix

If you need the raw xls for the analysis or help in how to set these things up in a web analytics tool like Insight, get in touch with me at rkirana@gmail.com