Testing on previous versions of browsers?

(Daniel) #1

In recent years, Chrome and Firefox have started having regular release cycles on their browsers, which automatically update every 6 weeks or so.

But the major problem is you can’t assume that everyone has restarted their browser to ensure they’re using the latest version, and you can’t easily download an old version of the browser and not have it automatically update while you’re testing.

How does everybody go about browser testing, keeping older versions in mind?

(Daniel) #2

This is how I’ve dealt with the problem in the past, (Note that most of my testing experience is as part of a in-house team, and not as an agency, which may do things differently):

  • I concentrate most of my testing on the current stable version of each browser.
  • I’ve made use of analytics/browser stats, so I can be confident of how many people are using what version of what browser. If nobody is (or very few people are) using a browser below a particular version, then I no longer take this into consideration.
  • I make these analytics stats known across the team: if a lot of people are using a particular version of a browser, but I’m unable to test it, I mark this down as a risk.
  • If I get a bug report which I can’t reproduce on the current version, I will compare version numbers. If the reporter is using an old version of the browser, I will request that they try the latest version.
  • I use regular regression testing to catch anything which a browser update may have broken.
  • If something breaks in a browser update, I will treat it as any other bug: if it is an ongoing project we’ll discuss fixing the bug depending on severity and importance. If it’s a one-off project we may decide not to fix the bug, especially if there’s no scope to do so.

(David Shute) #3

We both assume and expect that non-default browser users are keeping up to date. If someone is ten releases out of date on Firefox and something doesn’t work we advise them to update their browser. Wipe hands on pants.

For IE we’re a little more forgiving, but will still stay within the most recent - 1. This is mostly due to environments where users are forced by IT to only use vetted software and they are locked into IE.

What we’ve seen is that people skip using IE and go straight to the mobile apps if they are hamstrung by IT. IE makes up less than 10% of our user base, all versions considered, and we’re in the construction industry where users are typically under trained and less capable with tech.

(Gem) #4

I have been in a situation where we had to fix a bug on a previous version of Chrome as it was a public funded, bureaucracy-filled org where they literally told us that it would probably be quicker for us to fix the bug than for them to get their Chrome updated by the IT department (and when your oldest client says that, you fix the bug).

I told them it would be awkward to test it, but Browserstack had older versions so I’d test it on that imperfect system then they could test it on theirs. All worked out in the end.

(Patrick) #5

I was going to say, Browserstack is our main tool for testing older versions. It’s not ideal but it gets the job done most of the way. Our marketing team keep stats pretty regularly updated as to which platforms / versions the majority of our customers are using.

I’m also lucky that at our company, Chrome updates aren’t automatically rolled out, IT have a blocker that notifies them when one is pending and allows them to process it manually (some of the Chrome updates in the past have messed with our internet-based phone lines, not ideal for a company with a massive customer-facing phone department). This gives us ability to test our existing functionality on newer versions before they’re released to the company, and also to roll back to a previous version if something stops working.