Best strategy for testing multiple desktop platforms

Our SaaS product has multiple platforms, and I struggle to complete all of the testing in a timely manner. It’s laborious to go through a full regression suite for each platform, manually, especially when there are so many os versions that need at least some level of confirmation.

I test productions on the following platforms (latest os versions):

  • Mac
  • Windows
  • Chrome extension
  • iOS

The first three work off of the same code base, however, each platform has some specific trouble areas e.g. the product may work in MS Word on Windows, but outputs don’t precisely match the Mac version of MS Word; a few unique settings for each application, specific to their platform, etc.

They are all built at the same time, so I end up with three client builds that need testing all at once.

My current strategy is:

  • I fill out a Risk Analysis sheet I created to prioritize potential risks.

  • Run through my confirmation tests natively on the Mac and with a Parallels desktop instance of Windows running. I test it on the Mac, then I test it on Windows.

  • Run through my regression tests natively on the Mac and with a Parallels desktop instance of Windows running. I test it on the Mac, then I test it on Windows.

  • I do a basic smoke test for earlier macOS versions - just to confirm general functionality is working.

I had a couple of releases where everything didn’t get checked due to time, and I missed at least one basic thing and a couple of bugs I didn’t even consider. I’d like to tighten this up.

Nothing is automated on these platforms, currently - I don’t have that skill-set at this time. Ideally, I could find an easy to use automation tool that would work on the Mac to test native clients, as that is where I spend 75% of my time.

So, any recommendations for tightening up my procedure?

Thanks!

1 Like