How granular should I go when testing that settings changes are being registered in a config service?

I am wondering how granular I should go with my automation tests. I am test that when I make changes to an apps setting those changes are registered in the config service by comparing the config file. the settings in question are spread over 10 windows with each window having a minimum of 20 options to change.

so I guess the question is, do I test it window by window, and change all the options in the window in one test (this is what my boss wants me to do) or do i have a test for each option to make sure that each one registers on its own and then do the them all together?

What are peoples thoughts on this? and would I be wasting my time testing each option separately.

4 Likes

Do what your boss wants until it doesnโ€™t work. You could also think about mental models that might focus on the most impactful change. Pareto principle (80/20 rule), Occamโ€™s Razor, and what is the bottleneck?

You should experiment and play around without spending all your time on a default config that most people wonโ€™t change.

2 Likes

Technically there is the risk that having one test test 20 windows or panels, that your test becomes a shotgun and fails to point to a specific feature that might have changed. But when the boss says do a thing, just do it like they say, refactoring code later on is easier than writing 10 separate tests in some ways.

I suspect the real problem comes in when the configuration means that some windows or dialogs are hidden entirely or just not available due to a client setup, and thatโ€™s when I start splitting things out, because itโ€™s going to happen, but until it does, do only the minimum. Personally, @whitwes I would only test the most important config settings window and ignore the others, but thatโ€™s just the lazy tester in me who always has bigger fish to fry. And by the way, welcome to the most helpful software testing community in the entire Milky Way Galaxy. Good question, hope you do stay.

2 Likes

Thank you @alanmbarr @conrad.braam, that is really helpful. I have mainly been The sole tester so I have really never had anyone to ask these sorts of things to.
very glad I found this place.

3 Likes