Then I discovered something interesting in Google Analytics.

Google Analytics tracks and stores connection speeds for your visitors as part of the Network Properties category. The data provided includes Connection Speed and this corresponds with Pages per visit, Average time on Site, Percentage of new visits, and the Bounce rate. But how in the world does Google compile this data? It appears that people are guessing that Google uses the visitor’s IP address or hostname to do a lookup in an IP database or a reverse lookup, and then makes a guess based on what ISP that IP address belongs to. But Google also loads a small .gif file with each urchin request. This article explains a little trick for using time stamped image loading and log analysis which seems related.

So if your IP belongs to RoadRunner, then we can assume you are on a cable connection and file you under that category. This can sometimes get somewhat hairy and lead to ambiguity as evidenced by the Unknown category shown in Google Analytics. It is also interesting to note the Unknown category is the single largest category in my case with 36% of the total data collected. What happens if you are on AOL or EarthLink? Will the database contain the accurate information to pinpoint if you are on dial-up or broadband? How often is the database updated with new information? What about edge cases, for example if you work in an office where you have a “broadband” connection but share it with a large group of people?

The good news is that with Analytics the work happens without interfering with the user experience and no additional HTTP connections are needed. The bad news is, it is not done in real time. So if you are looking for a real-time test this will not work for you, unless you build your own seriously fast IP database web service that you can use asynchronously. The other option is to use the binary image payload test, but that also makes life hard for your users.

So, If you are looking for general statistics for a webapp over a date range then Google Analytics might be the way to go. Keep in mind you will end up with a chunk of Unknown data. As for making snap decisions, you can sniff user bandwidth while they surf with payload testing. Here is another sample page at AOL where they put up a simple fork page and determine your bandwidth speed using JavaScript image loading and setting a user cookie. I can think of a dozen reasons why this is a bad idea but I won’t go into that here.

PS. It is worth mentioning that Internet Explorer supports a built in behavior called clientCaps, which provides information about features supported by Microsoft Internet Explorer, as well as a way for installing browser components on demand. The connectionType property of clientCaps returns either lan, modem or offline. Since this only works in IE and doesn’t offer too much granularity, it might be a good test if you are only trying to test IE users specifically.

if (window.console && window.console.firebug) {
  alert("found Firebug");
}else{
  alert("can't find Firebug");
}

Demo: Sniff for FireBug

You may have noticed this message in your Gmail homepage recently:

The reason for that is Firebug by default logs all XMLHttpRequests made for all the websites you visit. The problem is that the data that is logged never gets disposed of or cleared out. Therefore, FireFox having to collect and store that information and the memory for Firefox will eventually get large enough that will cause your computer to start using paged cache. This will then make Gmail seem like you are swimming through molasses. Gmail by default seems to make 22 XMLHttpRequests when I load the main page. It then makes a new XMLHttpRequest every 30 seconds, to check for new emails, so in the span of 1 hour you will have 120 new Ajax calls tacked on. This is a nice feature of Gmail which makes life simple for its users (Notice there is no “Get new mail” button anywhere). But now every new Ajax call is another 1kb of data which adds to the memory usage and tacks on more CPU and RAM consumed by Firebug.

No need to worry though since, another nice feature of Firebug is that you can disable it on a domain name basis.

To turn off Firebug for mail.google.com first open Firebug by hitting the green icon or F12. Click on the bug icon in the top left and choose the second option labeled “Disable Firebug for mail.google.com”. Now Firebug will be turned off and the green icon will be changed to a silver icon for this domain. You should notice a smoother and faster Gmail experience. You can also turn it back on by clicking on the link “Enable Firebug for this website”.

The other and better method of speeding up your Firefox and still keeping Firebug around, is to disable Firebug all together and enable it for specific sites that you are working on. To disable Firebug, again you click on the Bug icon and this time choose the first option “Disable Firebug”.
Now visit any site that you would like to use Firebug on and then “right click” on the silver icon on the bottom and choose “Allowed Sites…” From here you can add the domain name for any sites you want to override. This makes your life easier and makes general surfing faster in FireFox, yet allows for using the powers of the almighty Firebug when needed.

If your application is similar to Gmail where XMLHttpRequests are called in a loop you may want to use the sniff script to warn users to disable Firebug for your domain as well.

Here are top 7 reasons why you really should check out Get Clicky.

1. Spy

Spy is a live view of visitors interacting with your web site. Its like having your own digg like spy page. This gives you the power of knowing in real time when and where your visitors are coming from, what pages they are visiting and where they are going when they leave. You see the user IP address as well as their browser and operating system. You can even trace specific users and see a breakdown of the history of their visits and actions made on the site. For each visitor you also see a nice set of stats such as their browser, operating system, language, resolution and whether JavaScript is enabled. As a side note Clicky works regardless of JavaScript support using image source loading.

2. Tagging and Filtering

You can go a bit further with IP address tracking in Clicky. You can tag certain IP addresses so you can easily identify them. You can also filter out specific IP addresses from the stats. For example If you want to exclude your own hits to your blog from being counted in your stats, enter the IP address(es) you use regularly and have them filtered out.

3. RSS

Stats via RSS is another amazing feature that gives you the power of keeping a tab on your site. Clicky gives you customizable RSS feeds that are updated every 30 minutes. Using these RSS feeds you are able to for example take a quick look at the action on your blog from any RSS reader. Now you can easily monitor the recent incoming links, popular incoming links, popular urls, recent searches, popular searches, recent visitors and dashboard data. A cool trick you can try is to change the duration for a feed. For example to see the hits in the past hour, append hours=1 to the feed url. Or to see popular incoming links in the past week append days=7 to the feed url. Another neat feature of the RSS feed is that it obviously doesn’t require authentication, so you can share the url with others as you wish and or transform the RSS data and publicly display your stats on your blog.

4. Clouds, CSV, XML, JSON, API

Stats data is presented to you in tabular format by default, however you can easily change the view of any table to a cloud view by clicking on a little cloud icon. I find the cloud visual more useful, especially when looking at long tables of data. For example, the Search keywords page in cloud view easily highlights the most popular search terms and search engines that are sending you traffic. For offline viewing you can also download your stats in popular formats such as CSV for Excel, and also XML or JSON which can be useful for feeding them to your own applications. If you are comfortable with programming, you can even use the API provided to access the data from Clicky using your sitekey and site_id.

5. Powerful Dashboard

You can edit your main dashboard page to see the blocks that you are most interested in. By default you get the basics, Links, Searches, and Content. I also like Recent Visitors and Locale. On the Basics widget a summary stats table gives you the birds eye view of what is going on. You see the number of visitors, the number of actions performed, total time spent on your blog, as well as averaged numbers such as Average actions per visit and Average time per visit. If that wasn’t enough you also see a comparison percentage column which shows you how today’s stats compare against the previous day’s data, or the same day last week.

6. Google Maps

For those of you that like to see geographical data, Google Maps is very nicely intergrated into Clicky in a few places. You can see the geolocation of any specific user as well as the locations of all visitors to your site on the map.

7. WordPress Plugin

If you use WordPress, and you should be, this plugin automatically puts your tracking code at the bottom of all pages on your site and filters out all traffic from site admins like yourself. It also does the whole ‘custom data tracking’ thing to automatically name your visitors inside the Clicky interface based on what they entered on your comment form.

There are a ton of other features that I don’t have time to go into here, such as tracking custom data, tracking downloads, FeedBurner integration, HTTPS support and more. You can try out a demo of a sample tracking site to get a feel for the clean and easy to use user interface.

When you first register, you get 21 days of premium service free, so you can try out all the features. You can track up to 3 web sites and 1,000 daily page views per day per site for Free. Beyond that level there are reasonably priced premium services which get you more sites, more features, and higher traffic levels – starting at less than $2/month.

var XMLHttpArray = [
        function() {return new XMLHttpRequest()},
        function() {return new ActiveXObject("Msxml2.XMLHTTP")},
        function() {return new ActiveXObject("Msxml2.XMLHTTP")},
        function() {return new ActiveXObject("Microsoft.XMLHTTP")}
];
function createXMLHTTPObject(){
        var xmlhttp = false;
        for(var i=0; i<XMLHttpArray.length; i++){
                try{
                        xmlhttp = XMLHttpArray[i]();
                }catch(e){
                        continue;
                }
                break;
        }
        return xmlhttp;
}////
function AjaxRequest(url,callback,method){
        var req = createXMLHTTPObject();
        req.onreadystatechange= function(){
                if(req.readyState != 4) return;
                if(req.status != 200) return;
                callback(req);
        }
        req.open(method,url,true);
        req.setRequestHeader('User-Agent', 'My XMLHTTP Agent');
        req.send(null);
}////
function AjaxResponse(req){
        var respXML=req.responseXML;
        if(!respXML) return;
        var respVAL=respXML.getElementsByTagName('family')[0]
.getAttribute('result');
        document.getElementById("status").innerHTML += respVAL;
}////
function ClearResponse(){
        mdiv=document.getElementById("status");
        mdiv.innerHTML = "";
}////
function MakeRequst(){
        AjaxRequest("ajax.xml",AjaxResponse,"get");
}////

And our sample XML file looks like this:

<?xml version="1.0" encoding="utf-8"?>
<family result="successful ajax request &amp; response!">
 <parents>
 <father name="Mike" last="Brady" />
 <mother name="Carol" last="Martin" />
 </parents>
 <children>
  <sons>
   <son>Greg</son>
   <son>Peter</son>
   <son>Bobby</son>
  </sons>
  <daughters>
   <daughter>Marcia</daughter>
   <daughter>Jan</daughter>
   <daughter>Cindy</daughter>
  </daughters>
 </children>
 <maid name="Alice" />
</family>

Here is a working sample of this code.

Ajax Request Reset

Some things to note are:

I’m using a GET here, but yo can easily change that to a POST in the call to AjaxRequest.

The URL passed to AjaxRequest can contain parameters as needed.

You can handle other readyStates(0,1,2,3) and HTTP status codes as you wish using the onreadystatechange handle.

I am sending null since I am not posting anything back to the server, but you can swap that with your own Post data if you wish. To set that up you have to also add this line of code before you call send to set the proper request headers.

req.setRequestHeader('Content-type' , 
'application/x-www-form-urlencoded');

The responseXML property is only available when the request is made to an XML document, so If you are using other document types (HTML,Text,CSV,JSON) you will not have access to this property. If you use anything other than XML data you will have access to the responseText property instead.

Once you have the handle to the responseXML property, you can traverse through the XML DOM like you would with any other DOM object and you can use all the DOM element methods, properties and events that you already know.

For example to get a list of all the sons in the family we can do:

respSONS=respXML.getElementsByTagName("son");
respSonVals = "";
for(x=0; x<respSONS.length; x++){
    respSonVals += " " + respSONS[x].firstChild.nodeValue;
}

Try Get Son Names