Referring Domains Demystified

A while ago a user of the yahoo webanalytics group asked why their own domain was showing up as the number one referring domain. Interestingly enough another person asked me the same question the same day. I have also seen this question pop up several times in past so I thought why not clarify for everybody.

Note: I know the images on this post are very small. I am trying to figure out how I can make them bigger but had no luck so far. As soon as I figure out hot to fix it, I will update this post with better pictures

This will be a two part series. In this part I will explain how Web Analytics tools report the referring domains and pages. I think this is a very critical aspect before we dig into why your domain shows up the referrer in referring domain report. Part II will explain why your own referrer shows up in the referring domain.

So let' begin with understanding what a log file contains:.

What is contained in a log file:: (Note: Even tag based solutions generate a log file, although they don’t call it a log file. This log file is in the vendor’s proprietary format but for this example you can assume it looks the same as the example below).

Every log file contains a line for each of the server ip, request file (URI stem of the file), with time stamp, a cookie (if one exists), and the referrer (URL of the page that referred user to the current file/page). There are other fields as well but they are not required for this example. The assumption here is that if you are using your server logs and they are in W3C Extended Log File format or you are using the JavaScript solution of your web analytics vendor.

Here is a list of all the fields in W3C Extended Log File Format:




Here is an example of the log file


You can read more about is at: http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/ffdd7079-47be-4277-921f-7a3a6e610dcb.mspx?mfr=true

Below is a an example of the log file from http://www.seattleindian.com

#Fields: date time c-ip cs-method cs-uri-stem sc-status sc-bytes cs-version cs(User-Agent) cs(Referer) cs(Cookie)
2007-02-21 07:07:30 66.171.173.27 GET /Seattle/moviepictures/bollywood.asp 200 33453 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) http://www.seattleindian.com/seattle/ USERID=1234;+RID=3423


In this log file /Seattle/moviepictures/bollywood.asp was the current page that the visitor viewed and it was referred by http://www.seattleindian.com/seattle/. Every log line has the referring page that contains the URL of the page that referred the user to the current page.

Let’s take an example of visits for one visitor. For this example I am only showing 5 fields (s-ip, data, time, URI stem, cs(referrer)

Below is the data for a visitor:




The visit started with a referral from http://www.google.com/?q=seattleindian. The referring domain in your web analytics tool will be Google.com

Let’s assume, this visitor goes on a lunch break leaving the site open in her browser. Come back after an hour (note the time change) and clicks on the home page links, here is how the log file will look like




This constitutes a second visit (assuming a 30 min session time out). The referring page will be http://www.seattleindian.com/seattle/advetise.asp and the referring domain will be SeattleIndian.com for this second visit.

This is one of the reasons why you will see lots of visitors referred by your own site. We will cover other reasons in Part II.

Let’s assume that this user clicks on a link on SeattleIndian.com that takes her to a new domain, SFIndian.com. This domain is identified by a different server ip (9.8.7.6) in the log file. Here is the log file for this other domain




Referring page for this visit will be http://www.seattleindian.com/seattle/advetise.asp the referring domain will be Seattleindian.com.

SeattleIndian.com and SFIndian.com are both owned by the same owner who wants to see the combined traffic. Let’s assume you were to combine both the domains in one profile (or reporting suite) and assume they were using the same third party cookie. This is how the combined log file will look like



When your web analytics tool parses these log lines, it will report 2 visits

One visit started at 10:59:50:00 and ended at 10:59:50:04 and the referring domain was Google.com

The second visit started at 11:59:50:05 and ended at 11:59:50:13 on SFIndian.com, the referring domain was SeattleIndian.com

Now let’s take an example where user goes out to another site and then comes back to the site




In this example: The visit started by referring domain being google.com, the user leaves the site from bollywood.asp page and comes back via yahoo search to aboutus.asp page. In this scenario the visit started via google so yahoo will never show up in the referring domains (that is the case with most of the web analytics tool).

Note: If you run a campaign on Google and Yahoo and tag them so that your web analytics tool can track them, then you can configure the tool to show either the last or the first campaign. However; the out-of-the-box referring domain report will only show the first referrer of the visit.

Hope this clarifies some of the confusion around referring domains and pages.

I would like to thank Brad Gagne for helping me the flow of this article. Brad is also responsible for making me split this article in two parts.

As I mentioned earlier, I will be out of the country for next 10 day so you will see Part II when I come back.