How to decide what tests should be done across multiple browsers?

Processes
#1

As the topic says, I’m not sure the best way to approach this question. I have a whole bunch of tests to run but I need to decide which ones need to be cross-browser tested. The obvious ones that I think do not need it done are where I manually check that a database record is created (for example when a user is created).

However, I’m not sure about things like js client side validation (like email address format is correct), should that also be tested across all browsers? They all should work the same I think? (except for IE11, everything breaks in IE11). Any feedback is appreciated.

#2

Always a tricky subject. Do you have any real user information you can inform your focus? Stats from out in the wild of use? If not look for another source like Gov.uk user stats although I think they have been not as reliable lately. I tend to swap browsers every couple of days so my natural exploration becomes cross browser. And yes, there’s always more weird issues in IE than any other. More browsers are being built or moving to a Chromium base so that helps. I tend to focus on the top 4/5 so Chrome, IE, Firefox, Safari and Opera. Hope that little bit of info helps but keep asking questions and we will all try to help

1 Like
#3

Thanks Ady, we do have user stats but that doesn’t quite relate to what my question is. My question is more about a test library and how to split browser testing depending on functionality (or even non-functional stuff) rather than percentage of user-browser. Do you have any ideas on how to manage that?

#4

We use a selenium grid for our UI tests which distributes across various browsers and versions. Is that more the thing you meant?

#5

Yes that sounds more like the thing I meant. So you mentioned you automate UI tests across various browsers and versions, that is one category of testing. Do you have a general rule of thumb for things that do (or do not) need to be tested across different browsers?

The example I gave in my initial post is to manually check that a database record is created. For example, if I fill a registration form and then click a button, I should see a db record is created for a new user. I don’t think there’s value to test this one across multiple browsers as they all (in theory) should work, would you agree?

Here are a few more examples and maybe you can get the idea of what I am asking, the question is, do I need to test these across different browsers? or not?

  1. Check that job search sections are in accordion format and that I can expand and collapse them (just check they open and close one at a time)
  2. Detect geolocation - click on an icon and I can see my postcode pops up
  3. Validation - I should not be able to register again if I use an existing email address
  4. Validation - Check that the fields are not empty and should be filled in

I hope that is clearer, I’m basically trying to work out how I can save time and what exactly should be tested across different browsers and what does not need to.

#6

Ah, ok. Think I’m understanding a little better now.

For the db example I wouldn’t class that as anything to do with the browser, that’s more the API sending the collected information so any automation would be focused there.

For identifying cross browser specifics I tend to look in two main areas, functional and visual.

Functional: So can I complete the journey in different browsers? It may render slightly differently but all the functions, error messages, success messages, confirmation pop outs / ups / overlays etc. lead to the same output. You can do it. Any browser where you can’t do it is an issue. (usually IE or a variation on IE)

Visual: Does the page look similar enough not to be a problem for using, branding etc. Is the layout close enough.

These are both to identify if we need specific browser handling to replicate the user experience in different ones.

So for your list I’d say they could all apply depending on your context but like all testing I’d focus on what’s the highest risk to your users. If geolocation is important definitely check. If an accordion renders always open might not be much of an issue.

For Validation I’d say displaying the error message re already registered e.g. ‘Did you mean to sign in?’ would be a better check as it is highly unlikely a browser difference would let the same details register twice.

Without more context its really hard for me to be more specific on tests but I hope that helps in some small way?

#7

Thanks Ady,

I will look at those 2 big areas that you mentioned Functional and Visual. We do already have a prioritised list in terms of risk but not cross browser. So the next step is basically to look at the whole list, assess each test in turn and check “Do I need to do it cross browser?” and label accordingly.

Thanks for taking the time to help me out!

1 Like
#8

You are very welcome Philip :slight_smile:

#9

If there is any JS changes, then we we do a cross browser test.