Ran a test - will the results be same in production?

Anu on the WA group has a question - SHe ran a test that had two recipes A and B. She found that B is better with a conversion advantage of 10%.
She wants to know if this means that when she puts it into production if the 10% advantage will remain.

I think there is a need for me to post an article on the basics of beta testing and multi-variate testing - on basic statistics like sample, population etc.

Here is my answer to that
-------------------------------------------------------------------------------------



Well, there cannot be a guarantee - I will tell you WHY.

Say you did a beta/MVT test - It works like this -
Since you had 2 versions - control flow (A) and test flow (B) - what happened is that the tool randomly directed incoming traffic to either A or B. So an incoming visit could randomly be in A or B - however at the aggregate level, the tool tried to keep the visit count in A and B same.
Seeing this behavior continually over a large period of time, it achieved statistical significance that in your case B is better

So if all parameters are the same - namely the site design, the incoming traffic quality and the customer behavior, then the conversion advantage of 10% should remain.

however you will see variation if the site design changes or the incoming traffic mix changed or you got a different set of customers. When you test a month form now, you might even find that recipe A is overperforming.
So continually TEST

Nikki on the WA group has an interesting question - She wants to know the difference between 'organic clicks' and 'organic referring clicks'

I am first of all against using these kind of metrics at a site level without clearly understanding what they imply.

Since Nikki uses HBX (my favorite WA tool), here is my best approach towards solving her issue:
----------------------------------------------------------------------------------
In a visit, the referrer tag is populated by the WA tools based on the referrer= information supplied by the browser.

E.g:
If you go to www.google.com and search for "buying cheap cameras" and it throws up an
ebay link that you click, the browser will add the tag
referrer = google when you visit ebay

Similarly for other sites that lead you to ebay.

All of these are referral traffic - It is organic if it is natural (not paid for - e.g: Natural search traffic), it is inorganic if it is paid for (e.g: Paid Search traffic)

Now it may so happen that in the same visit, you come to ebay from google, from yahoo, from microsoft search as well as from other sites, who would you give the credit?

The credit should go to the first person that got you to ebay.

This is populated in the referrer dimension in VS/HBX
Use that as a dimension and get the visits/clicks by the referrer. Do not use it on the metrics side - and sum up the values of visits/clicks to get it for the site.
----------------------------------------------------------------------------------

Tracking repeat cart visitors

In the WA yahoogroup, Smith had an interesting problem.

His website has setup a mechanism to retain items in cart for a period of 30 days and whenever the person who added these to carts returns to the site, he sees the previously placed item in the cart. Smith wants to know how he can track the effectiveness of this feature.

Here is my solution to his problem:

------------------------------------------------------------------------------------

There are two ways to do this:

WAY 1: WITHOUT SETTING UP ADDITIONAL TRACKING CODE, ONE CAN GET A VERY GOOD APPROXIMATION.

(1) Depending on the WA tool that you are using, you need to have a way to IDENTIFY cart additions
e.g:
Look at the URL when someone adds to cart - On most sites it is like http://......./addtocart.html/aspx.

{Get the page name and look at number of visits to that - This will be the metric cart additions}

(2) create a VISITOR/COOKIE LEVEL SEGMENT for all the folks that added to cart (using the page key above)

Above will help us track all the cookies that have added to cart.

(3) create a derived metric based on purchase_visits.
purchase_visits {Visit_seq_num > 1} - This is possible in Discover OnPremise.

Combine 2 and 3 in a workspace/report - You have your metric.

WAY 2: WITH ADDITIONAL SITE TAGGING
Your website already can identify returning visitors and show them what is in the cart. For these have a tagging mechanism by way of code in the add to cart page that sends specific tagging information to the WA tool.
e.g: this could be through a URL parameter called return_to_cart

Next use your custom report formats in Yahoo Analytics, Google Analytics, Omniture to see the visits and purchase_visits for this parameter return_to_cart

That should give you the answer - There is nothing in the world of tracking WA that is not trackable :)

Thanks
Kiran
rkirana@gmail.com
Web Analytics Consultant
http://webanalyticsnuggets.blogspot.com/

- On Thu, 30/4/09, tlsmith1260 wrote:


From: tlsmith1260
Subject: [webanalytics] Tracking conversion in shopping cart with item retention
To: webanalytics@yahoogroups.com
Date: Thursday, 30 April, 2009, 10:31 PM

------------------------------------------------------------------------------------

So Mr SMith, you are on the right track.

Customization and personalization form the backbone of Web 2.0
You are on the right track!!

How to find the source of my 404s

'404 - Page Not found'

This is a common error that we have all come across. We see this on our browser screen when the page we are trying to visit does not exist on the server.

for the technically inclined, it is the status code returned by the HTTP server when the page requested does not exist on the server.

On the web analytics group, Sue had a question: She is tracking this website and sees that large number of 404s come from the most visited page on her website.

Here is my proposed solution in Omniture Discover
Follow the below logic. It should be same in any WA tool

Let us call the "most popular page that is redirecting to the 404s" as X
and the 404 page as Y

1. Create a segment for all the folks who visited X and Y
2. For this segment look at the traffic source split

E.g: In Discover
* create a segment by dragging the visit container to RHS
* Click on edit
* add the 2 pages X and Y
* save

Then go to the traffic source report and pull the segment as a column and choose the date range.

You should be good!!

Thanks
Kiran



>
> --- On Wed, 4/29/09, Sue wrote:
>
> From: Sue
> Subject: [webanalytics] probably a very dumb ? but I need help finding source of 404
> To: webanalytics@ yahoogroups. com
> Date: Wednesday, April 29, 2009, 9:00 AM
>
> Community, please help me...
> I use Google Analytics and Webtrends version 8.0. WebTrends looks at my log files.
>
> I have read this article: http://googlewebmas tercentral. blogspot. com/2008/ 10/webmaster- tools-shows- crawl-error. html but it is no help to me. Google analytics shows no errors.
>
> but my page that 404's are redirected to is my number one visited page on my site. Where are these people coming from? Can I get this info from WebTrends?
>
> I have been so frustrated, I have actually clicked every link I can find on my web site to try to find the culprit.
>
> HELP!
> Thanks!
> Sue
>