Spam making an impact on browser stats?
12.10.05 @ 9:39 pmJust like every other bloke out there that has a blog, I’ve had my problems with comment spam. What you might not know though is that your spam may be giving you a false sense of who is visiting your site.
I regularly check my site logs to see who is coming to my site. I am especially interested in browser statistics - not only for this site, but in general as well. Last month, I noticed that the segment of visitors using Internet Explorer had steadily risen to around 45 to 50 percent. I found this to be a strange occurance since I was usually seeing a vast majority of Firefox users coming to my site with the occasional Safari boom when I’d post to Pixelsurgeon, Newstoday and Artdorks.
It wasn’t just a coincidence that during that same timespan, I was seeing my highest concentration of blog spam to date. I would sometimes get over 100 comment notifications that were spam and that doesn’t count the number that were actually blocked by my spam filters.
After a had my full of nearly being spammed to death, I decided to put in a number of changes that would make it much more difficult for spam- bots to successfully get to my comments page. Literally the day after I made those changes, I noticed some astonishing results. The table below charts browser percentages for the three main browsers (Firefox, Internet Explorer and Safari). The table cells in red are the dates before I made changes to block spam traffic. The light blue cells are the dates where my initial changes were made and the dark blue cells were the dates where additional changes were made.
| Date | Firefox/Mozilla | Internet Explorer | Safari |
|---|---|---|---|
| 9/12/2005 | 33.27% | 45.90% | 1.64% |
| 9/13/2005 | 32.13% | 44.44% | 1.80% |
| 9/14/2005 | 37.50% | 38.46% | 1.28% |
| 9/15/2005 | 35.08% | 45.58% | 1.93% |
| 9/16/2005 | 32.94% | 43.11% | 2.69% |
| 9/17/2005 | 34.07% | 42.49% | 1.10% |
| 9/18/2005 | 33.68% | 48.26% | 1.39% |
| 9/19/2005 | 39.83% | 39.44% | 1.24% |
| 9/20/2005 | 38.61% | 44.88% | 1.65% |
| 9/21/2005 | 36.23% | 40.73% | 2.53% |
| 9/22/2005 | 43.48% | 32.17% | 1.74% |
| 9/23/2005 | 53.77% | 24.06% | >1% |
| 9/24/2005 | 29.31% | 20.69% | 1.15% |
| 9/25/2005 | 33.81% | 20.56% | 2.22% |
| 9/26/2005 | 41.36% | 22.51% | 2.09% |
| 9/27/2005 | 40.44% | 23.50% | 1.09% |
| 9/28/2005 | 48.76% | 30.86% | 1.85% |
| 9/29/2005 | 50.28% | 28.49% | 1.68% |
| 9/30/2005 | 52.21% | 26.99% | 2.65% |
| 10/1/2005 | 52.48% | 23.76% | 3.47% |
| 10/2/2005 | 42.46% | 17.43% | 5.96% |
| 10/3/2005 | 43.02% | 22.79% | 22.47% |
| 10/4/2005 | 43.14% | 22.66% | 19.61% |
| 10/5/2005 | 40.40% | 16.16% | 6.40% |
| 10/6/2005 | 34.57% | 21.32% | 4.26% |
| 10/7/2005 | 49.50% | 18.36% | 4.26% |
| 10/8/2005 | 37.58% | 22.03% | 1.76% |
| 10/9/2005 | 39.33% | 17.63% | 1.69% |
| 10/10/2005 | 33.58% | 11.63% | 3.10% |
| 10/11/2005 | 33.78% | 14.86% | 2.70% |
As you can see, the percentage of visitors using Internet Explorer dropped dramatically from a high of 48.26% to a low of 11.63%. The percentage of Firefox users went from being the minority to being the vast majority (over twice as much as the next browser below it). What I also soon noticed was how much non-browser traffic I get to my site. This became much more apparent after all spam-bot traffic was eliminated.
While this study is obviously far from scientific, for myself, it definitely puts into question all the browser usage statistics we get from various studies. Unless those studies have either taken great care in removing spam-bots or have a way to detect them and remove them from the statistical analysis, those studies may be far from accurate.browser statistics, comment spam, firefox, internet explorer spam
October 12th, 2005 at 11:05 pm
$0.00 in Comment Love for September
This is interesting for me because I work in the corporate advertising world and we always have to build and test for IE because random browser stats show that IE is more widely used. What may I ask did you do to block bots and spam?
October 13th, 2005 at 9:51 am
$0.00 in Comment Love for September
A couple of things. First, I agree with your suspicion. The problem is that many of these numbers are either tabulated by the companies themselves, by non-scientific means, or are otherwise flawed. For example, Yahoo and Google’s estimations of their index size are just fudges. (How can Google’s index suddenly increase by a factor of 4?) Thus, I would not be at all surprised if you were onto something.
However, there are a few weak points in the analysis. First, bots are more likely to visit blogs since they can easily post to them. For example, Google probably does not get many bots doing searches because it does not make much sense to do so. Second, your blog probably draws more Firefox users due to the audience to which it is targetted–eg. techsavvy designers that don’t tend to use crappy IE. This would not influence the effect on stats of weeding out bots but would not make it possible to map your stats regarding browser share to the web as a whole. I think those are the only critisisms that I can see for now.
Regardless, it’s a good observation. The next step would be to learn a little more about how the common browser stats are amassed. If that also points to overestimation of IE stats, it would be worthwhile to investigate further.
October 13th, 2005 at 5:05 pm
$0.00 in Comment Love for September
One good thing about Blogspot (readmade though it is) is that you can filter spam from your blog.
October 15th, 2005 at 9:33 am
$0.00 in Comment Love for September
I am using Firefox and I’m real…great site, you’ve been bookmarked
November 21st, 2005 at 9:01 am
$0.00 in Comment Love for September
I am using Firefox too…
Greetings from Austria / Vienna!
June 18th, 2006 at 8:21 pm
$5.00 in Comment Love for September
[…] Denying Spammer’s Rights? Oh, the Humanity! ad, blog spam, internet advertising, internet rights, spam spammersI do not like spam. I can guess that you are not too keen on spam either. It wastes our time, distorts site statistics and just makes the world a less enjoyable place to live. We know spammers are jerks and spammers know they are jerks as well. This fact makes the pill a little easier to swallow. Imagine for a second that a spammer actually begin to speak out about the atrocities of bloggers being unwilling to allow their “ads” to be on their site. Begin the eye-rolling, this one is a pure nugget of gold. This is by far the best piece of blog spam I have ever received. Bar none. Yes Hello, My name is Paul Germana. And I have a question which I know you cannot answer. But I’m just going to ask it to make you look at yourselves. Why is it not up to the “owner” of the blog whether or not he allows ‘adds’ to be posted on his site? Not allowing people to place adds on blogs in like not allowing a salesman to knock on your neibors door, because you think you own the “block”. Try as you might you will not own the internet. That is to monopolize, which inspite of what we see today is still illegal. Furthermore, it is simple to prevent “spamming”. The owner simply posts a request at his blog: “No soliciting please” . Here is how I know it would work. One, because it is a golden rule as old as time; and two, any good blog community has a “nextblog” button. And that is the one and only community a “spammer” (as you call him) is going to be interested in. If he sees “no soliciting” he knows he has hundreds of thousands of other blogs at the click of the mouse. Instead of wasting his time with someone who has said “no soliciting” he is going to gamble on the person who has’nt said “no soliciting”.(Old As Time) What you are trying to do is own the world wide web, which you know is unethical; but you have plan. Blame it all on the “spammer”, a fictional person which you all have created in order to distract attention from the more sinister act of taking the email address of the person who pays you for adds at his or her own site and in turn gives you emails of the people who are supposed to be going to THEIR site!!! Then you take those emails and put them into a server which you or one of your fellow “business associates” owns, to share your stolen emails. This is why it’s so easy for you to lie when you say, “You are automatically approved for a visa-gold card”. But there is no visa card. You merely buy ads from underhanded webpage builders who used to work for, (or for that matter) are the credit-holders, who just want you to pay them in what else, “email addresses”. It’s the oldest foot in the door trick in the world. You say, “I’m not a salesman, I’m a businessman.” Then you have the gall to talk about manners to the “lower-class” citizens who have more talent in their pinky toes than you have in your whole body. It does’nt take talent to be a cheat. But you know what does take talent? The way that I go anywhere I want on the web and there is nothing you can do about it,Why?..well because there are millions upon millions of spinsters just like you who are confined within their little tiny no effort spaces, begging me to come and join their community. You think we don’t see you. And that is why yet another blogcommunity is up for sale. Anonymous spammer […]