Failed (?) my test task, how would You solve it?

Hello ppl,
A few days ago I applied for one of QA positions, the HR dept sent me a test task that i had to solve before being invited for the technical interview.

I did this task and got a negative response, could you guys take a look and advice what mistakes have I made?
(Thanks!)

TASK:
Please solve the task below and send requested test cases to us.

Function takes two and only two characters as input. It returns 1 when each entered symbol is either letter or number and 0 in other cases.
Numbers are Hindu-Arabic numerals from 0 to 9. Letters are Latin, Cyrillic or Modern Greek alphabet.

The bug was found before, slogan is: “Cyrillic letters are treated as non-letter symbols”
Actions to reproduce:
Function input: БГ.
Expected output: 1. (Logic: only Cyrillic letters).
Observed output: 0.

The bug was fixed. Existing tests were already ran as a regression test after bugfix, here is the test report:
Input Expected output Status
AA 1 Passed
1A 1 Passed
11 1 Passed
A$ 0 Passed
(1 0 Passed

What tests cases will you do for bug’s verification?
What tests cases will you propose to enforce regression testing of the function?
Negative tests like input less than 2 symbols and input more than 2 symbols are out of scope for both bug verification and additional regression test cases.

MY RESPONSE:
Q1 - What tests cases will you do for bug’s verification?

The test cases that are needed for bug’s verification (Cyrillic letters are not treated as letter symbols) could be easily derived from the acceptance criteria that is provided and expected result in the bug report.

The test case Summary will be like:

“Verify that function treats Cyrillic characters as letter symbols” (other wording might be used or this could be organized as a checklist containing checks for Latin, Cyrillic and Modern Greek alphabets - this depends on a real context).

Test case(s) must contain steps(checks) for following cases:
(probably I would prefer to organize this as a checklist or test cases with multiple steps where every step has its own expected result)

  • 2 Cyrillic characters entered at once (e.g. “БГ”)
  • 1 Cyrillic and 1 Greek Alphabet character (e.g. “λГ")
  • 1 Cyrillic and 1 numeric (e.g. “Б1”)

Another test case is needed in order to verify that combination of Cyrillic, non-letter and non-numeric characters will result as “0”.

  • “Б#”
  • “!Г”

Q2 - What tests cases will you propose to enforce regression testing of the function?

Basing on the list of test that were executed the test coverage is not complete.
Test cases for Greek Alphabet (and variations) must be added in order to cover this functionality (function).

PLEASE NOTE: I assume that Cyrillic input tests were executed (See Q-1).

Now all the possible inputs (alphabets, numbers and their combinations) are covered with the tests.

*Ignored negative test cases in accordance with the task

**Design techniques used: Equivalence classes (exhaustive testing will not be efficient)

1 Like

First of all, I would have stated clearly that the existing tests in the test report had not tested for non-Latin character inputs, only for Latin alphabet, numbers and special characters. Your other tests should then be read in that context.

I would then have challenged the way in which the tests were carried out and in particular how the non-Latin characters were to be input. In many applications, non-Latin characters have to be input as special characters, but the test scenario seems to suggest that there will be non-Latin characters used as normal. Therefore you are looking to discover whether the function erroneously identifies non-Latin characters as special characters.

Alternatively, the application might be expecting non-Latin characters to be entered using ASCII codes, which many users who regularly come across these things as exceptions enter using a keyboard shortcut (alt key + a three-digit code punched into the keypad of a standard keyboard. This can also be used to enter stress marks unique to languages that otherwise use Latin alphabets, such as é, ä and so on). It might have been worth testing for different ways of inputting the same characters.

A lot hinges on what the application is. If it is a website, for instance, then it might be expected to receive inputs from any user in the world, some of whom may not use Latin characters. Never having worked in a country where non-Latin characters are the norm, I do not know how IT kit in such countries generate characters, but I assume that in such countries, the Cyrillic or other character set is the default, and so characters being entered into a field would not be identified as “special characters” as such. Perhaps this is the gist of the bug - in which case, the original fix should not have been accepted as a fix.

If, instead, the application is just something resident on a user’s machine and the user is not a native user of non-Latin alphabets, then we are back to discriminating between different types of special character. I’m not such a code expert as would make me be able to say how that works or how to test for it at a code level.

In short, I think it’s a bit of a trick question. There are a range of issues relating to the scenario that step way outside the simple matter of determining test cases. So either HR are looking for people who think outside the box (so just listing test cases is the wrong answer), or whoever set the question doesn’t really understand testing.

Part of the answer you are looking for is perhaps more about the language used in responding to the question; even if providing a list of the test cases does turn out the be the right answer, how you introduce the answer to show the way you were thinking is also important.

This, of course, assumes that there even was a right answer to the question to begin with, or that HR are telling you the real reason why you weren’t invited for interview. Quite often, tests like this are used to whittle down a wide field of candidates to a realistic number to interview. If fifty people apply for a role but the panel can only realistically interview five, then HR have to reduce the list of candidates by 90%, so their criteria for doing that will be pretty broad-brush and may well focus on things that aren’t related to “the right answer”.

I went through this exercise a couple of years ago, and had a number of equally inexplicable decisions from companies. One quite big software house sent me a link to an online test that was actually riddled with viruses and wouldn’t run on my machine anyway, and even if it had worked the instructions on what the test required were at best unclear. It looked as though this was an old test that HR were administering to anyone without understanding either what they were testing for or why they were even administering the test, let alone being able to maintain the test application itself. Their reaction when I challenged this was to clam up and not even respond to my e-mails!

So by all means, check back with the wisdom of crowds to see if there’s anything obvious that you did wrong, but be prepared to find out that it was just one of those things and perhaps it was not your destiny to work for that company or in that role. If after asking around and thinking about it, you still can’t spot what you did “wrong”, then just chalk it up to experience and move on to the next time. Sometimes there are no right or wrong answers; sometimes it’s something completely out of left field that you would never be able to identify or do anything about in a month of Sundays. If it’s nothing you can fix, don’t beat yourself up about it.

3 Likes

This looks like a simple test design task.
You need to perform verification and regression testing like you do it during your work in the office: use best practices like boundary values testing, equivalence classes, etc.
The cases proposed by you have really few coverage for verification and I can’t see any additional cases for regression testing proposed from your side, though the ones in the task were very poor. Therefore, additional cases are obviously needed.
The task was solved at level of a student who applies for his first QA job and had never read any book or even a wiki article about test design and testing methodologies.
I believe the employer request to solve this easy task not to spend time for interviews of candidates with inappropriate skills and knowledge. There are so many testers and so few good testers: the cheapest way of preliminary segregation is a small practical task before the interview.
Good luck for future applications and read something like “The Art of Software Testing” by Glenford J. Myers, or watch some youtube vids, this is even easier.

1 Like

If I was handed a test task like that (even if it was from HR dept) I’d hand it back, say “Not for me, thanks” and then continue my search elsewhere, thankful for the headsup that it wasn’t the place for me.

2 Likes

That’s why you do not have permanent tester job; right, Pat? =)

Same for me, honestly. This question says to me that the person who set it does not understand the role of context, that the company relies on test case artifacts over real testing and that testing at this company will have all the creativity and fun sucked out of it. I don’t want to work on nothing but checklists like a robot. The most I’d do for them is to hand back my honest answer, stating my assumptions and factors affecting my choices, and if I “failed” then I know we’d not be suitable for each other.

3 Likes

Well, I wouldn’t particularly want one at a place like that anyways :grinning:

Going back to the original question, people have to remember that the recruitment process is a two-way thing - not just do they want to hire you - but do you want to work with them? And looking at the task supplied, alarm bells would be ringing for me - get out of there, before they make you write 100 test cases before you can start testing.

2 Likes