How do I test email subscriptions?

Sharing Lounge How Would You Test That?
#1

Hi all,

QA newb here. I have a question and hopefully those who are expert testers can advise how I can do my job better. I’m working on a project where a person can sign up to different emails depending on their job criteria. There is a vast amount of different options available for searching a job as you can imagine:

Keyword
Location
Parent Category (like Computer Sciences)
Subcategory (like Software Engineering)
Job Type (Academic, Professional, etc)
Salary
Hours
Contract Type

I need to test that the emails are receiving the right content of jobs depending on the subscription. Obviously if I were to test every permutation, I don’t think I can live long enough to finish my test! Is there a way compress my data set to the most critical? My math isn’t great but maybe I can do 8 x 8 = 64 sets of tests to cover every combination? The issue is, there are options below the above options! (like I mentioned with the Job type etc).

I googled some software testing techs before such as equivalence partitioning and boundary value analysis.I don’t know if they would help here, they seem more to do with a range of values rather than selections. I hope I’m making sense here but if I need to clarify, let me know.

#2

First things first, if you are testing an application created to filter responses sent to someone based on an email subscription they set up, you must understand how the filter works. Once you know how the filter matches the text entered (“Software Tester” vs. “Software” or “Tester”), create 2 or 3 basic tests for each scenario (e.g., one word search, multi-word search, exact job title).

For example, if the user selected a keyword search for a “Software Tester” job, I would expect jobs that contained that exact match to be returned (whether the match is on the job title, job title or job description, or the job title/description contains any of the entered text for “Software Tester” vs. “Software” or “Tester”) depending on the specificity of the filter function. If I searched on “Analyst”, I would expect a range job to be returned that contain the word “Analyst”. If I searched for “Full Stack Developer”, I would expect jobs with that exact title and/or those exact words in the job description.

If you start off with these simple tests for each of the ways you can filter (again, only 2 or 3 basic tests), as you write, you will find your brain will automatically start thinking about the “what ifs” (e.g., what if I misspelled a word, etc.). Write those down to keep track of them then go back and determine the level of importance for each (probability it will happen and potential impact).

This still sounds like a lot of complexity, but if you can start with the basics, the outliers will generally become obvious. If you are not sure how to judge the level of importance of the outliers, sit down with a developer, BA, or other tester so get someone else’s opinion. This will help you think outside the box and gain a different perspective.

#3

Thanks for your help @m.pawloski.

Yes you are right, it’s incredibly complex and it’s a headache. The good news is, I already kind of know how each of the categories work. I can go through and identify the scenarios as you mentioned, but there is only so far I can go. My question was more about finding out whether there are techniques to come up with a minimal data set as project timelines are quite tight.

#4

Hello @pwong!

I understand your testing challenge to be two parts: email content, and the email delivery. The content depends on the criteria you listed (and more), and that content is jobs or job descriptions.

I would focus on the email content and suggest that the email is less important. The task of placing content into an email and sending it is far easier than applying search criteria to a database of job descriptions. I suggest you determine how you might isolate the process of generating email content so that it is easier to evaluate.

There are a few methods for generating the content: direct queries to the database, or using a testing front end that creates and submits queries for you.

Joe

#5

Since you are already into the project, I would try to make a mind map of the different scenarios that you have. Again, bring in a second (or third or fourth) person to do this with you. This gives you a broader perspective and it also gives you help in determining which are will be the more common scenarios, or the scenarios with the most potential risk. It will also help you identify ways you can consolidate and provide more coverage with fewer scenarios.
Those are the two things that you should look at when determining which tests you will run vs. which can wait in case you have time at the end, Commonality and high risk should be your primary focus when time is an issue.

1 Like
#6

Hello @pwong!

Specific techniques might be equivalence partitioning (also known as class equivalence and more broadly as domain equivalence - see this BBST course), and all pairs.

Equivalence partitioning helps discover functional similarities and boundaries. A good conversation with your developer might give you information on this. All pairs is a method of pairing inputs to maximize coverage in a few tests. There are some tools listed on this page that might assist with this technique.

Joe