Testing on previous versions of browsers?

In recent years, Chrome and Firefox have started having regular release cycles on their browsers, which automatically update every 6 weeks or so.

But the major problem is you can’t assume that everyone has restarted their browser to ensure they’re using the latest version, and you can’t easily download an old version of the browser and not have it automatically update while you’re testing.

How does everybody go about browser testing, keeping older versions in mind?

1 Like

This is how I’ve dealt with the problem in the past, (Note that most of my testing experience is as part of a in-house team, and not as an agency, which may do things differently):

  • I concentrate most of my testing on the current stable version of each browser.
  • I’ve made use of analytics/browser stats, so I can be confident of how many people are using what version of what browser. If nobody is (or very few people are) using a browser below a particular version, then I no longer take this into consideration.
  • I make these analytics stats known across the team: if a lot of people are using a particular version of a browser, but I’m unable to test it, I mark this down as a risk.
  • If I get a bug report which I can’t reproduce on the current version, I will compare version numbers. If the reporter is using an old version of the browser, I will request that they try the latest version.
  • I use regular regression testing to catch anything which a browser update may have broken.
  • If something breaks in a browser update, I will treat it as any other bug: if it is an ongoing project we’ll discuss fixing the bug depending on severity and importance. If it’s a one-off project we may decide not to fix the bug, especially if there’s no scope to do so.
2 Likes

We both assume and expect that non-default browser users are keeping up to date. If someone is ten releases out of date on Firefox and something doesn’t work we advise them to update their browser. Wipe hands on pants.

For IE we’re a little more forgiving, but will still stay within the most recent - 1. This is mostly due to environments where users are forced by IT to only use vetted software and they are locked into IE.

What we’ve seen is that people skip using IE and go straight to the mobile apps if they are hamstrung by IT. IE makes up less than 10% of our user base, all versions considered, and we’re in the construction industry where users are typically under trained and less capable with tech.

3 Likes

I have been in a situation where we had to fix a bug on a previous version of Chrome as it was a public funded, bureaucracy-filled org where they literally told us that it would probably be quicker for us to fix the bug than for them to get their Chrome updated by the IT department (and when your oldest client says that, you fix the bug).

I told them it would be awkward to test it, but Browserstack had older versions so I’d test it on that imperfect system then they could test it on theirs. All worked out in the end.

3 Likes

I was going to say, Browserstack is our main tool for testing older versions. It’s not ideal but it gets the job done most of the way. Our marketing team keep stats pretty regularly updated as to which platforms / versions the majority of our customers are using.

I’m also lucky that at our company, Chrome updates aren’t automatically rolled out, IT have a blocker that notifies them when one is pending and allows them to process it manually (some of the Chrome updates in the past have messed with our internet-based phone lines, not ideal for a company with a massive customer-facing phone department). This gives us ability to test our existing functionality on newer versions before they’re released to the company, and also to roll back to a previous version if something stops working.

1 Like

Ideally, a stable version of a browser is preferred to test the feature, as maximum people will be using the same. Also, bug fixing and application testing on all versions of all browsers is not possible. IE itself range from IE 7 - 11. In this case, testing same thing of 5 different versions of IE will be wastage of time as the maximum number of Audience will be using the latest browser. However sometimes need to arise, when we need to test old versions of browser to provide web application testing services. In that situation, it is impossible to roll back to previous version of browser that will consume a lot of time. We can test previous versions of browser using a Browserstack tool. Following features are provided by this tool which could help solve the problem:

  1. Access to real desktop browsers are provided like IE, FF, Chrome, Safari (All these browsers are archived from old to new versions).
  2. It is practically impossible to maintain an internal lab of virtual machines, devices or emulators having different versions of browsers as they are getting updated to a new version on a very frequent basis. Various changes are introduced with every browser version like plugin and player support. So, Browserstack can keep all browser version handy at one location that can be used for testing specific version of the browser.
  3. With this tool, we can test browsers of mobile and tablet devices (a wide variety of devices are listed in their database).

Other tools are also there, that can provide similar support to test the old version of browsers:

  • Browsershots
  • IE NetRender
  • Browserling
  • IE Tab
  • CrossBrowser Testing
  • Super Preview

Hope this information is helpful for you.

1 Like

“you can’t easily download an old version of the browser and not have it automatically update while you’re testing”

Actually, it’s pretty easy to create containers where you have a specific browser version and other associated software - such as browser drivers:

If you want Firefox for version FIREFOX_VERSION, you can download it at:

https://download-installer.cdn.mozilla.net/pub/firefox/releases/$FIREFOX_VERSION/linux-x86_64/en-US/firefox-$FIREFOX_VERSION.tar.bz2

you can’t assume that everyone has restarted their browser to ensure they’re using the latest version

You can monitor the usage and drop out testing for the versions you don’t want to support and/or that your users no longer use.

If we need to manually test something on an older browser or a browser that’s not installed on our computers we use Browserstack. Not perfect but often does the job.

For our automated tests we use Browserstack as well as Selenoid on our local (virtual) machines. There we can also specify what version of Chrome/Firefox/Opera we want to run the tests on.

https://aerokube.com/selenoid/latest/#_browser_images

1 Like