What surprised you?
Just how bad the overall quality of the candidate pool is. Particularly for SDET roles. I would understand if there were many candidates that were either great programmers but with a poor testing mindset, or great testers with poor coding skills, but Iāve interviewed an awful lot of candidates who absolutely stunk at both.
How did you become better at interviewing?
I actually started out as an interviewer with interviewing intern candidates, not necessarily just for testing roles. We had a set script of questions, intended to objectively measure candidates equally, but the more interviews I did the more I realized some candidates did poorly because of our script rather than because of their actual ability. My biggest āaha!ā moment was interviewing a candidate with an impressive resume who did quite poorly on pretty much every question I asked. Since at that point I had gotten pretty skeptical of our script, I flipped the script and told him something like ālook, I see these various accomplishments on your resume and I can tell youāre smart, but your answers to my scripted questions arenāt very good. Is there a particular CS topic you feel like youāre stronger on that we should talk about?ā Sure enough, he suggested an area that we didnāt cover in our script, but I knew enough about to come up with some questions off the cuff, and he did great. Unfortunately, not all stories have a happy endingāI advanced him to the next round but this was in 2020, and all our intern positions were cancelled shortly afterwards due to the pandemic.
But Iāve kept that lesson with meāhaving an initial script can be helpful, and I constantly look for ways to improve it, but Iām not afraid to go off script particularly if Iām trying to find a candidateās strengths.
What were your biggest challenges when you first started interviewing testers?
Asking questions that didnāt look for creativity or critical thinking. Plenty of candidates can give you a stock answer they read in a book (or <cough> the ISTQB syllabus), but disappointingly few can think on their feet and go deeper when you push them on context. Which leads intoā¦
What advice would you give to someone new to interviewing testers?
Focus on creativity and critical thinking. I ask several questions that are intentionally badly worded, because I want to see if the candidate will think deeper and challenge me, or just go along with answering the wrong question. For example, in one question I describe a āsimpleā program and then ask āhow many test cases would you need to test this program?ā
Good candidates will ignore that specific request for a number and just start brainstorming, quickly realizing that the āsimpleā program isnāt so simple. Great candidates will start challenging, asking questions, pointing out flaws in the program design/requirements (and I throw some softballs thereāitās amazing how few candidates point out that using a single text field to accept three distinct input values is a really bad idea).
What are some common mistakes people make when interviewing testers?
Well, if the ātop 100 interview questionā cheat sheets on sites like LinkedIn are any indication, a huge mistake would be focusing too much on arbitrary definitions rather than, as I keep saying, looking for creativity and critical thinking.
Another I would postulate is not digging deeper. I get a lot of candidates that might provide a reasonable initial response, but if I try going deeper or asking the question a different way to see if they think about another aspect, they basically just keep repeating what they said the first time. Even if I specifically ask something like āwhat else would you considerā or āis there any situation where that wouldnāt be a good idea?ā