Anu on the WA group has a question - SHe ran a test that had two recipes A and B. She found that B is better with a conversion advantage of 10%.
She wants to know if this means that when she puts it into production if the 10% advantage will remain.
I think there is a need for me to post an article on the basics of beta testing and multi-variate testing - on basic statistics like sample, population etc.
Here is my answer to that
-------------------------------------------------------------------------------------
Well, there cannot be a guarantee - I will tell you WHY.
Say you did a beta/MVT test - It works like this -
Since you had 2 versions - control flow (A) and test flow (B) - what happened is that the tool randomly directed incoming traffic to either A or B. So an incoming visit could randomly be in A or B - however at the aggregate level, the tool tried to keep the visit count in A and B same.
Seeing this behavior continually over a large period of time, it achieved statistical significance that in your case B is better
So if all parameters are the same - namely the site design, the incoming traffic quality and the customer behavior, then the conversion advantage of 10% should remain.
however you will see variation if the site design changes or the incoming traffic mix changed or you got a different set of customers. When you test a month form now, you might even find that recipe A is overperforming.
So continually TEST
Saturday, May 2, 2009
Friday, May 1, 2009
Business metrics for Tracking a non e-commerce site -
I have tracked a non e-commerce business site before but that was tracking a site that had learning content - the objective being to eventually drive the visitor who comes to learn about the site to make a purchase.
In such a non e-commerce case key metrics should be:
1. Repeat Metrics: % of Repeat visits
2. Duration Metrics: Visit Duration/Page Duration
3. Engagement Metrics: Number of clicks
4. Drive to purchase metrics: %age of visitors that end up purchasing on the site
5. Abandonment Metrics: Bounce,Leakage
Some nuggets from experience
* Look for 5-abandonment metrics. It is the most important metric. If the bounce is high for a non e-commerce site, then probably no point in advertising to get traffic in there - IMPROVE THE CONTENT
* Engagement metrics: Number of clicks, visit/page view duration are equally important - Less engagement means that the site is not doing its job
* Also look at the metrics by individual page for the top 10 % of the pages. You might find a few lemons there
* If your site is really good, over a period of time, the natural traffic should increase to it -(word of mouth at work :) )
In such a non e-commerce case key metrics should be:
1. Repeat Metrics: % of Repeat visits
2. Duration Metrics: Visit Duration/Page Duration
3. Engagement Metrics: Number of clicks
4. Drive to purchase metrics: %age of visitors that end up purchasing on the site
5. Abandonment Metrics: Bounce,Leakage
Some nuggets from experience
* Look for 5-abandonment metrics. It is the most important metric. If the bounce is high for a non e-commerce site, then probably no point in advertising to get traffic in there - IMPROVE THE CONTENT
* Engagement metrics: Number of clicks, visit/page view duration are equally important - Less engagement means that the site is not doing its job
* Also look at the metrics by individual page for the top 10 % of the pages. You might find a few lemons there
* If your site is really good, over a period of time, the natural traffic should increase to it -(word of mouth at work :) )
Nikki on the WA group has an interesting question - She wants to know the difference between 'organic clicks' and 'organic referring clicks'
I am first of all against using these kind of metrics at a site level without clearly understanding what they imply.
Since Nikki uses HBX (my favorite WA tool), here is my best approach towards solving her issue:
----------------------------------------------------------------------------------
In a visit, the referrer tag is populated by the WA tools based on the referrer= information supplied by the browser.
E.g:
If you go to www.google.com and search for "buying cheap cameras" and it throws up an
ebay link that you click, the browser will add the tag
referrer = google when you visit ebay
Similarly for other sites that lead you to ebay.
All of these are referral traffic - It is organic if it is natural (not paid for - e.g: Natural search traffic), it is inorganic if it is paid for (e.g: Paid Search traffic)
Now it may so happen that in the same visit, you come to ebay from google, from yahoo, from microsoft search as well as from other sites, who would you give the credit?
The credit should go to the first person that got you to ebay.
This is populated in the referrer dimension in VS/HBX
Use that as a dimension and get the visits/clicks by the referrer. Do not use it on the metrics side - and sum up the values of visits/clicks to get it for the site.
----------------------------------------------------------------------------------
I am first of all against using these kind of metrics at a site level without clearly understanding what they imply.
Since Nikki uses HBX (my favorite WA tool), here is my best approach towards solving her issue:
----------------------------------------------------------------------------------
In a visit, the referrer tag is populated by the WA tools based on the referrer= information supplied by the browser.
E.g:
If you go to www.google.com and search for "buying cheap cameras" and it throws up an
ebay link that you click, the browser will add the tag
referrer = google when you visit ebay
Similarly for other sites that lead you to ebay.
All of these are referral traffic - It is organic if it is natural (not paid for - e.g: Natural search traffic), it is inorganic if it is paid for (e.g: Paid Search traffic)
Now it may so happen that in the same visit, you come to ebay from google, from yahoo, from microsoft search as well as from other sites, who would you give the credit?
The credit should go to the first person that got you to ebay.
This is populated in the referrer dimension in VS/HBX
Use that as a dimension and get the visits/clicks by the referrer. Do not use it on the metrics side - and sum up the values of visits/clicks to get it for the site.
----------------------------------------------------------------------------------
Natural Search vs. Paid Search
Googling has become the synonym for search.
Traditional marketing theory has it that a purchase stems from a need (maslow's hierarchy for the B-schoolers :) ) ----> need for information ---> search for information ----> evaluate alternatives ----> buy decision
No wonder that in the online world, Searching or Googling is the easiest way to get the information - and hence one of the major sources of visits/traffic to a website
Google, Yahoo, Microsoft have used this crucial link in the buying process to monetize/get revenue in the form of advertisement links. Anytime somebody clicks on the sponsored links on the right or wherever they appear on the web page, they make money.
E.g: The diagram is a snapshot of my search on google.com for 'buying books online'. Google has intelligently figured out that I live in India (my airtel account??) and shown me results in the India context.
If I click on the "blue" box, then Google makes no money, while Google makes money if i click on the "red" box
Clicking somewhere in the 'blue' box constitutes 'natural search' while clicking in the 'red' box constitutes paid search
This presents an interesting question:
If the same page, e.g: ebay.in, were to be displayed in both the 'red' box as well as the 'blue' box what would happen.
For all visitors that click on ebay in the red box, ebay would pay money to google, adding to its advertising spend.
For all visitors that click on ebay in the blue box, ebay need not pay any money to Google
So from an online retailer's perspective, increasing the visits from natural search would greatly contribute to optimize the advertising spend.
Traditional marketing theory has it that a purchase stems from a need (maslow's hierarchy for the B-schoolers :) ) ----> need for information ---> search for information ----> evaluate alternatives ----> buy decision
No wonder that in the online world, Searching or Googling is the easiest way to get the information - and hence one of the major sources of visits/traffic to a website
Google, Yahoo, Microsoft have used this crucial link in the buying process to monetize/get revenue in the form of advertisement links. Anytime somebody clicks on the sponsored links on the right or wherever they appear on the web page, they make money.
E.g: The diagram is a snapshot of my search on google.com for 'buying books online'. Google has intelligently figured out that I live in India (my airtel account??) and shown me results in the India context.
If I click on the "blue" box, then Google makes no money, while Google makes money if i click on the "red" box
Clicking somewhere in the 'blue' box constitutes 'natural search' while clicking in the 'red' box constitutes paid search
This presents an interesting question:
If the same page, e.g: ebay.in, were to be displayed in both the 'red' box as well as the 'blue' box what would happen.
For all visitors that click on ebay in the red box, ebay would pay money to google, adding to its advertising spend.
For all visitors that click on ebay in the blue box, ebay need not pay any money to Google
So from an online retailer's perspective, increasing the visits from natural search would greatly contribute to optimize the advertising spend.
Tracking repeat cart visitors
In the WA yahoogroup, Smith had an interesting problem.
His website has setup a mechanism to retain items in cart for a period of 30 days and whenever the person who added these to carts returns to the site, he sees the previously placed item in the cart. Smith wants to know how he can track the effectiveness of this feature.
Here is my solution to his problem:
------------------------------------------------------------------------------------
There are two ways to do this:
WAY 1: WITHOUT SETTING UP ADDITIONAL TRACKING CODE, ONE CAN GET A VERY GOOD APPROXIMATION.
(1) Depending on the WA tool that you are using, you need to have a way to IDENTIFY cart additions
e.g:
Look at the URL when someone adds to cart - On most sites it is like http://......./addtocart.html/aspx.
{Get the page name and look at number of visits to that - This will be the metric cart additions}
(2) create a VISITOR/COOKIE LEVEL SEGMENT for all the folks that added to cart (using the page key above)
Above will help us track all the cookies that have added to cart.
(3) create a derived metric based on purchase_visits.
purchase_visits {Visit_seq_num > 1} - This is possible in Discover OnPremise.
Combine 2 and 3 in a workspace/report - You have your metric.
WAY 2: WITH ADDITIONAL SITE TAGGING
Your website already can identify returning visitors and show them what is in the cart. For these have a tagging mechanism by way of code in the add to cart page that sends specific tagging information to the WA tool.
e.g: this could be through a URL parameter called return_to_cart
Next use your custom report formats in Yahoo Analytics, Google Analytics, Omniture to see the visits and purchase_visits for this parameter return_to_cart
That should give you the answer - There is nothing in the world of tracking WA that is not trackable :)
Thanks
Kiran
rkirana@gmail.com
Web Analytics Consultant
http://webanalyticsnuggets.blogspot.com/
- On Thu, 30/4/09, tlsmith1260 wrote:
From: tlsmith1260
Subject: [webanalytics] Tracking conversion in shopping cart with item retention
To: webanalytics@yahoogroups.com
Date: Thursday, 30 April, 2009, 10:31 PM
------------------------------------------------------------------------------------
So Mr SMith, you are on the right track.
Customization and personalization form the backbone of Web 2.0
You are on the right track!!
His website has setup a mechanism to retain items in cart for a period of 30 days and whenever the person who added these to carts returns to the site, he sees the previously placed item in the cart. Smith wants to know how he can track the effectiveness of this feature.
Here is my solution to his problem:
------------------------------------------------------------------------------------
There are two ways to do this:
WAY 1: WITHOUT SETTING UP ADDITIONAL TRACKING CODE, ONE CAN GET A VERY GOOD APPROXIMATION.
(1) Depending on the WA tool that you are using, you need to have a way to IDENTIFY cart additions
e.g:
Look at the URL when someone adds to cart - On most sites it is like http://......./addtocart.html/aspx.
{Get the page name and look at number of visits to that - This will be the metric cart additions}
(2) create a VISITOR/COOKIE LEVEL SEGMENT for all the folks that added to cart (using the page key above)
Above will help us track all the cookies that have added to cart.
(3) create a derived metric based on purchase_visits.
purchase_visits {Visit_seq_num > 1} - This is possible in Discover OnPremise.
Combine 2 and 3 in a workspace/report - You have your metric.
WAY 2: WITH ADDITIONAL SITE TAGGING
Your website already can identify returning visitors and show them what is in the cart. For these have a tagging mechanism by way of code in the add to cart page that sends specific tagging information to the WA tool.
e.g: this could be through a URL parameter called return_to_cart
Next use your custom report formats in Yahoo Analytics, Google Analytics, Omniture to see the visits and purchase_visits for this parameter return_to_cart
That should give you the answer - There is nothing in the world of tracking WA that is not trackable :)
Thanks
Kiran
rkirana@gmail.com
Web Analytics Consultant
http://webanalyticsnuggets.blogspot.com/
- On Thu, 30/4/09, tlsmith1260
From: tlsmith1260
Subject: [webanalytics] Tracking conversion in shopping cart with item retention
To: webanalytics@yahoogroups.com
Date: Thursday, 30 April, 2009, 10:31 PM
------------------------------------------------------------------------------------
So Mr SMith, you are on the right track.
Customization and personalization form the backbone of Web 2.0
You are on the right track!!
How to find the source of my 404s
'404 - Page Not found'
This is a common error that we have all come across. We see this on our browser screen when the page we are trying to visit does not exist on the server.
for the technically inclined, it is the status code returned by the HTTP server when the page requested does not exist on the server.
On the web analytics group, Sue had a question: She is tracking this website and sees that large number of 404s come from the most visited page on her website.
Here is my proposed solution in Omniture Discover
Follow the below logic. It should be same in any WA tool
Let us call the "most popular page that is redirecting to the 404s" as X
and the 404 page as Y
1. Create a segment for all the folks who visited X and Y
2. For this segment look at the traffic source split
E.g: In Discover
* create a segment by dragging the visit container to RHS
* Click on edit
* add the 2 pages X and Y
* save
Then go to the traffic source report and pull the segment as a column and choose the date range.
You should be good!!
Thanks
Kiran
>
> --- On Wed, 4/29/09, Sue wrote:
>
> From: Sue
> Subject: [webanalytics] probably a very dumb ? but I need help finding source of 404
> To: webanalytics@ yahoogroups. com
> Date: Wednesday, April 29, 2009, 9:00 AM
>
> Community, please help me...
> I use Google Analytics and Webtrends version 8.0. WebTrends looks at my log files.
>
> I have read this article: http://googlewebmas tercentral. blogspot. com/2008/ 10/webmaster- tools-shows- crawl-error. html but it is no help to me. Google analytics shows no errors.
>
> but my page that 404's are redirected to is my number one visited page on my site. Where are these people coming from? Can I get this info from WebTrends?
>
> I have been so frustrated, I have actually clicked every link I can find on my web site to try to find the culprit.
>
> HELP!
> Thanks!
> Sue
>
This is a common error that we have all come across. We see this on our browser screen when the page we are trying to visit does not exist on the server.
for the technically inclined, it is the status code returned by the HTTP server when the page requested does not exist on the server.
On the web analytics group, Sue had a question: She is tracking this website and sees that large number of 404s come from the most visited page on her website.
Here is my proposed solution in Omniture Discover
Follow the below logic. It should be same in any WA tool
Let us call the "most popular page that is redirecting to the 404s" as X
and the 404 page as Y
1. Create a segment for all the folks who visited X and Y
2. For this segment look at the traffic source split
E.g: In Discover
* create a segment by dragging the visit container to RHS
* Click on edit
* add the 2 pages X and Y
* save
Then go to the traffic source report and pull the segment as a column and choose the date range.
You should be good!!
Thanks
Kiran
>
> --- On Wed, 4/29/09, Sue
>
> From: Sue
> Subject: [webanalytics] probably a very dumb ? but I need help finding source of 404
> To: webanalytics@ yahoogroups. com
> Date: Wednesday, April 29, 2009, 9:00 AM
>
> Community, please help me...
> I use Google Analytics and Webtrends version 8.0. WebTrends looks at my log files.
>
> I have read this article: http://googlewebmas tercentral. blogspot. com/2008/ 10/webmaster- tools-shows- crawl-error. html but it is no help to me. Google analytics shows no errors.
>
> but my page that 404's are redirected to is my number one visited page on my site. Where are these people coming from? Can I get this info from WebTrends?
>
> I have been so frustrated, I have actually clicked every link I can find on my web site to try to find the culprit.
>
> HELP!
> Thanks!
> Sue
>
Subscribe to:
Posts (Atom)
