Browser Sniffers

Introduction

"Why can't Opera just work?" is a regular and understandable question on the Opera web forums and newsgroups. Who wouldn't want Opera to "just work"? I certainly would!

Unfortunately, for many reasons (some technical), it's just not possible for the browser alone to be made to "just work". For the web to work requires the cooperation and agreement on the parts of both the web browser and the web site on what's going on. Perhaps I should repeat that: a working web requires effort from both the web site and web browser. Neither one by themselves can do it.

In almost every case where Opera appears to fail at a web site, the problem can be traced back to poor scripting on the web site. In most cases the scripting is trying to determine what browser is being used, and in almost every case where Opera appears to fail, it's the script that has got things wrong, and identified Opera as the wrong browser, correctly identifies Opera and intentionally stops working, or does not even identify Opera as a browser at all!

If you were to imagine Opera's "conversation" with a problem web site, it might sound something like the following:

Opera:	[dials the web site]
Web site:	"Hello, you have reached our web site! To ensure you have the best browsing experience, we need to know a little bit about you before we start."
Opera:	"Ok, I'm..."
Web site:	[interrupts] "Are you Internet Explorer?"
Opera:	"No, but..."
Web site:	"Oh, are you Netscape?"
Opera:	"No, I'm not that either..."
Web site:	*clunk* *beep* *beep* *beep*
Opera:	"Hello?"
Web site:	*beep* *beep* *beep*
Opera:	"Is anybody there?"

By far the most common script offenders are menu scripts. So much so, I've created a page dedicated to describing some of the various menu script issues.

Some people have questioned why web sites should have to update their scripts to support Opera - something along the lines of "Opera is in the minority, it should be the one to conform to the majority". That's true, web sites should spend most of their time and effort supporting the major browser(s) and a smaller amount of time and effort supporting the minor browsers. That's smaller, not none at all. Web site maintainers have no excuses to just sit back and do nothing, especially in cases when updated scripts are readily and freely available, or when the changes required to support Opera involve modifying a single line!

In most cases it's not so much updating scripts "to support Opera", it's more like just "updating scripts". It's absurd to think that a script that was installed three, four, five years ago is working just as well today as it was then. The world is always moving on, the web is changing. Scripts are changing too. You probably service your car once a year, web sites need to "service" their scripts regularly too. Surely an annual review and update isn't too much to ask?

Opera's Efforts

Even the most charitable supporter would agree that Opera browsers before version 7 were, at best, very ordinary in their support for web sites. Version 7 changed all that. It was an incredible leap forward in terms of supporting what web sites were currently doing. Opera's CSS, Javascript, and DHTML were all significantly improved.

Opera had become a "modern browser".

Web Site Browser Support

Unfortunately, the damage had been done. Opera had acquired a reputation as being a "difficult" browser, and certainly not one worth wasting much time supporting.

The time of Opera 3, 4, 5, 6 was also the time of the "browser wars". Netscape 3, 4 and Internet Explorer 3, 4, 5 were fighting it out for web supremacy. Web site developers were fighting to support both, so that no matter which one ended up "winning", they would still have a working web site. There was no time to spare on minority browsers.

Web site developers were forced to "sniff out" which browser their visitors were using and send web pages tailored to whichever browser they detected. Such development work was complex and time consuming. Web developers frequently only worried about the top two - Netscape and Internet Explorer. Every other browser was left to fend for itself.

The Fallout

The fallout from the wars remains today. Some web sites today are still using code that was developed during the browser wars - and hence only caters to Netscape and Internet Explorer.

Now, Netscape isn't much of a force these days. Its torch has been taken up by the Mozilla Foundation and its stable of browsers: Mozilla and Firefox. Both of these browsers stand on the shoulders of Netscape. In its heyday, Netscape was as big a force on the web as Internet Explorer is today. Some would say it was even bigger. In terms of web site compatibility, Mozilla and Firefox reap huge rewards as a consequence of being descended from Netscape.

Opera has no such legacy to benefit from. It has always had to stand by itself and make the best of things.

That means that old code left over from the browser wars, which was designed for Netscape, also largely continues to work with Mozilla and Firefox. Without Netscape's legacy to help it out, very little has changed for Opera, at least with respect to supporting old code.

Where Opera benefits is in its ability to handle standards-based code. The current Mozilla and Firefox browsers are very well regarded as browsers that can handle standards-based code. Recent versions of Netscape are the same, since they are more-or-less a rebadged Mozilla! More and more web sites are turning towards standards based development and away from proprietary techniques. Opera's excellent standards compatibility allows web site developers to use the same code for Mozilla, Firefox and Opera.

That is, they could, except many sites set up their page code two or three years ago, and haven't so much as glanced at it since! Quite often code that Opera could handle is witheld from Opera because Opera is neither Netscape nor Internet Explorer.

Explanation

To explain the sorts of problems Opera faces, it's necessary to get technical. I've even got to show some web code! I'll try to explain things as simply and clearly as possible.

The first exhibit is some browser sniffing code, in its entirety:

var agt=navigator.userAgent.toLowerCase(); var is_major = parseInt(navigator.appVersion); var is_minor = parseFloat(navigator.appVersion); var is_ie = ((agt.indexOf("msie") != -1) && (agt.indexOf("opera") == -1)); var is_nav = ((agt.indexOf('mozilla')!=-1) && (agt.indexOf('spoofer')==-1) && (agt.indexOf('compatible') == -1) && (agt.indexOf('opera')==-1) && (agt.indexOf('webtv')==-1) && (agt.indexOf('hotjava')==-1)); var is_nav4up = (is_nav && (is_major >= 4)); var is_nav6up = (is_nav && (is_major >= 5)); var is_ie4up = (is_ie && (is_major >= 4)); var is_mac = (agt.indexOf("mac")!=-1);

The first line gets the user agent string. For example, the user agent string of your browser is "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)". It also converts all capital letters to littles, to make later tests easier to code.

The next two lines get the major and minor version numbers of your browser. For example, if your browser was version 1.23, is_major would be 1 and is_minor would be 1.23.

The rest of the lines detect which browser you're using, based on tests involving the user agent string and the version numbers. Notice that there is nothing like is_opera. Effectively, the code is completely ignoring Opera.

Now, you may have heard that Opera can "spoof", or fake, its identity. If you're using Opera, press F12 and look at the "Identify as..." options. These settings are intended to fool browser sniffing code similar to the above into thinking Opera is really Mozilla (Netscape) or Internet Explorer. They work by having Opera send out a different user agent string:

Opera User Agent Strings

Identify as...	User Agent String
Opera	Opera/7.52 (Windows NT 5.0; U) [en]
Mozilla 5.0	Mozilla/5.0 (Windows NT 5.0; U) Opera 7.52 [en]
Mozilla 4.78	Mozilla/4.78 (Windows NT 5.0; U) Opera 7.52 [en]
Mozilla 3.0	Mozilla/3.0 (Windows NT 5.0; U) Opera 7.52 [en]
MSIE 6.0	Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) Opera 7.52 [en]

Notice, however, that no matter which one you pick, the word "Opera" is always somewhere in the user agent string. There are many reasons for this, both technical and commercial. More on those reasons later. Let's continue with the next few lines of the sniffing code:

var is_ie = ((agt.indexOf("msie") != -1) && (agt.indexOf("opera") == -1)); var is_nav = ((agt.indexOf('mozilla')!=-1) && (agt.indexOf('spoofer')==-1) && (agt.indexOf('compatible') == -1) && (agt.indexOf('opera')==-1) && (agt.indexOf('webtv')==-1) && (agt.indexOf('hotjava')==-1));

The code works by testing to see if the user agent string (agt) does or does not contain one or more particular strings. agt.indexOf(string) calculates to -1 if the string string IS NOT somewhere in the user agent string, and calculates to some other value if it is.

You can now see that as part of the checks for both is_ie and is_nav that they are checking to ensure that "opera" is not part of the user agent string. Since "opera" will always be in the user agent string, both is_ie and is_nav will be calculated as false.

Of course, that's the correct result! Opera is neither Internet Explorer nor Navigator (Netscape/Mozilla/Firefox). The problem is that while the code deliberately excludes Opera from some tests, it fails to include it in other tests. There is no test for "is_opera"!

Now let's look at where these browser sniffing results are used:

function displayAll(){ if (is_ie){ /* code for Internet Explorer snipped */ } if (is_nav){ /* code for Navigator (Netscape/Mozilla/Firefox) snipped */ } }

You'll note that there is nothing like if (is_opera){. Of course there isn't! The browser sniffing code was written to completely ignore Opera! So, what does happen for Opera? is_ie is (correctly) false, so that section of code is not used. is_nav is (correctly) false too, so that section of code is not used either. There are no more sections of code so for Opera, no code is used at all!

It should be obvious that displayAll can only work correctly if one or the other of is_ie or is_nav is true. It assumes that only two browsers exist in the entire universe! Internet Explorer and Navigator. What a shame about Opera, Safari, Konqueror, etc. The only way that the code could possibly work is if Opera removed the word "opera" from the user agent string.

What's most disappointing, and frustrating, is that Opera can handle either the Internet Explorer code section, or the Navigator code section! Opera developers have done such a good job making Opera compatible with all the various ways of coding for browsers, Opera can do both! It's just prevented from working by the short-sighted browser sniffing code, and the poorly written code that uses the sniffing results.

While removing "opera" from the user agent string would be good for that particular site, it would be extremely bad for Opera on the web as a whole.

User Agent Spoofing

As a commercial organisation, Opera Software would never want their browser to go about the web pretending to be a competitors browser! After all, Opera Software would like their browser to become as popular as possible, and the only way to do that is to ensure that their name gets out there. If Opera were to remove their name from the user agent string, web site developers would have to assume that a browser identifying itself as, say, Mozilla 5.0, really is what it says it is, and not, say, Opera. It would be as if Opera did not exist. That would be commercial suicide.

The technical reasons for keeping "Opera" in the user agent string comes down to compatibility. No matter how good Opera developers are, the Opera browser will always be different to every other browser. It will have its own quirks and supported features. While developing web sites these days is much easier than it used to be, eventually a web site developer is going to need to do something special, for whatever browser, be it Opera, Internet Explorer, or Mozilla/Firefox.

For example, Internet Explorer does not support alpha-transparent PNG images, Mozilla/Firefox do not support the Javascript document.all, and Opera does not support the Javascript XMLHttpRequest. A developer who wants to use any of those features needs to know what browser a visitor is using to ensure things don't stuff up.

If Opera was to become indistinguishable from either Internet Explorer or Mozilla/Firefox, a developer would, by necessity, have to assume Opera was what it was claiming it was (either Internet Explorer or Mozilla/Firefox) and send out code fixing a bug Opera didn't have (the lack of PNG and document.all support respectively). The code fixing the non-existent bug could very easily cause the web page to fail completely for Opera - the exact opposite of what we want to achieve!

It should be obvious that completely hiding Opera's identity would not only be dishonest, it is not an acceptable solution to bad browser sniffing either.

So why does Opera provide the user agent spoofing it does? Well, it allows some sites to work. It's the best solution Opera can offer to allow poorly developed sites to work, without being dishonest about what browser Opera is, and allowing those sites that make the effort to be cross-browser compatible to detect Opera and work appropriately.

Leaving "opera" in the user agent string is a double-edged sword. It allows those sites that want to work with many different browsers to distinguish between those browsers and deliver code tailored to their various strengths and weaknessess. However, it also allows those sites that do the wrong thing the ability to single out and block particular browsers from working on their sites, usually because the web site developer was too lazy to make the effort.

By not providing the ability for Opera to completely disguise itself, Opera allows sites that want to do the right thing, to do the right thing. Sites that persist in doing the wrong thing are a shame, but really, is the problem Opera? Or is it the site?

What To Do

It should be obvious that the displayAll code is doing the wrong thing. It was written for exactly two browsers and ignores everything else. It's not up to browsers to fix wrong code - it's up to web site developers to do that. It's not that hard either. The above example could be fixed by incorporating a simple "else" clause (that's right - only one line of code needs to change for Opera to be able to work):

function displayAll(){ if (is_ie){ /* code for Internet Explorer snipped */ } else{ /* code for every other browser (even give unknown ones a chance!) */ } }

If your favourite web site doesn't work with Opera, it could be something as simple as I've shown here. Often the web site developer has used a third-party script (common for menus). This should make it easy for the web site developer to keep up-to-date. All they need to do is check for new versions of their script and update. The third-party developer does all the hard work.

Keeping third-party scripts up-to-date is really basic web site maintenance that all web site developers should do on a regular basis. Even once a year is better than not at all.

What you should do is ask, either in the Opera web forums or the Opera newsgroups, why it doesn't work. In most cases a knowledgable person will be able to tell you. You're then expected to take it further with the web site (since you raised the issue in the first place). Feel free to point the web site developer in the direction of your forum or newsgroup threads. Note that Opera's newgroups are all archived by Google Groups.

While we're all waiting for dead-head web site developers to get their stuff together, it is possible to use third-party software to totally change Opera's user agent string. We've already established why Opera Software will never build this sort of thing into Opera, that's why it's entirely up to outside people.

I have a page, Faking Browser IDs, that would allow the site I used as an example to work. Please, for all the reasons I've outlined on this page, don't go about masking Opera's ID willy-nilly. Only use it for those sites you really have to use.