Well first I should add, if it wasn’t obvious, that my expectations have to match the person I’m interviewing. If they use different words but can make me understand them, I’m okay with that, and if they’re more “senior” then I expect them to be able to explain themselves better. I need to be a great interviewer to get the best out of an interviewee especially if they are nervous. I find that people testing at interview get stuck into a focused state, and reminding them to defocus in the wrong way can cause them to be even more nervous, and I have to take all of that into account.
I also have to be careful about what a practical interview is not. I’m looking to see their internal testing structure, what they know about what they know, and that they know about their skills or what information and tools they’re missing. I like to see their approach. I’m not going to find out answers to specific questions without asking specific questions, so if someone doesn’t mention an answer I need I’ll have to ask that later in the interview.
Testing is huge, we know this. So to evaluate it all in an hour or three is impossible. I don’t think this is exclusive to testing. I’ve seen and been on both sides of many interviews, some of which were virtually useless at evaluating a candidate. So we do the best we can with what we have.
To give you some idea of the depth of complexity I’m talking about here’s Caner’s writing on recruitment which is the closest thing I know to required reading before interviewing a tester: http://www.testingeducation.org/BBST/foundations/Kaner_JobsRev6.pdf