I have interviewed a lot of engineers and been interviewed by a lot of companies, and I think most interviews are trying to learn the wrong things. Or rather, they are trying to learn the right things with instruments that cannot measure them.
I have come to believe a good engineering interview is trying to answer three questions, and only three.
Can they do the work
Not can they do any work, or impressive work, or work under pressure in front of strangers. Can they do the work this role involves, at the level the role needs.
I think the best evidence is a conversation about work they have done, in enough depth that it cannot be faked. What was the problem, what did they try first, what did they change their mind about, what would they do differently. A person who did the work can go as deep as you like. A person who was nearby cannot.
The second best evidence is a small, real problem of the kind the job contains, worked through together rather than performed. I would rather watch someone reason through a problem for twenty minutes than solve a puzzle in five.
How they think when stuck
Everyone gets stuck. The job is mostly getting unstuck. So the interview should contain a moment of being stuck, and what I am watching is what happens next.
Do they say what they are thinking, or go quiet. Do they ask a question, or guess. Do they try something and look at the result, or defend the first idea. None is disqualifying on its own, but the pattern is the clearest preview I know of what the person will be like in a hard week.
This is why I think the surprise question is a poor instrument. It produces being stuck, which is useful, but in a way that mostly measures composure under ambush, which the job rarely requires.
What they are like to work with
Most interviews skip this question or reduce it to whether the interviewer liked the person, which is a different thing.
What I try to get at is how they handle disagreement, how they talk about people they have worked with, and whether they can take a suggestion mid-conversation and use it. The questions they ask about the team tell me a great deal. So does how they describe a past team's failure, and whether anyone else in the story gets credit.
What an interview is not for
I think an interview is not a test of stamina, so I do not run six hours of them. It is not a test of trivia, so I do not ask things that a search would answer.
Where this stops being true
Some roles need a specialized test, and a deep technical role may need an exercise that takes real time. Even then, I think the three questions are the ones being answered. The exercise is the instrument, and it is worth asking of every instrument which of the three it serves.
Photo source: https://photos.robertstowe.com/imperial-sand-dunes
