How I learned to interview engineers

Hiring new engineers is always an exciting part of being on a growing team. But I’ve found that learning how to interview potential new teammates well is a skill that requires training and practice as much as any other part of the job. I joined Metaview without much interviewing experience and have now run over 50 of our frontend and backend delivery interviews. Given all of this, I thought it would be helpful to other early-career engineers, or people that are looking to grow their pool of interviewers, to share how I went about learning to interview engineers.

Virtual shadows

My first step in learning to interview engineers was to watch a few past interviews at Metaview. We dogfood our own product to watch recordings and read transcripts of actual interviews. This is a ‘virtual’ or ‘async’ shadow, and unlike a normal shadow interview, I completed it on my own schedule, playback at 1.5x speed, and skipped to the most useful parts of the interview.

Another benefit of shadowing virtually is that only the best examples of interviews to learn from were chosen for me, whereas every live shadow is a roll of the dice with respect to how useful it is as a training tool. Watching a strong candidate steamroll a task wouldn’t be as good of a learning opportunity for me as seeing examples where common pitfalls happened, or where subtle signal could be picked up.

What did I learn from virtual shadows?

One thing that was especially helpful about my virtual shadows was that they gave me a chance to calibrate my introduction. When I first started interviewing, I was surprised by how important the first few minutes of the interview are and how easy it is to miss aspects of a good intro. For example, from shadowing I learned that it’s important to:

• Set the candidate at ease with a friendly, conversational tone—this helps them perform at their best.
• Outline the structure and purpose of the interview, e.g. setting expectations for how long the task will take and mentioning that they’ll get a chance to ask you questions at the end.
• Sell the company. Every candidate should leave the interview excited about the prospect of joining Metaview, so when introducing myself I’ll talk about what made me join the company and briefly outline something exciting I’ve recently been working on.
• Start the task on time. Letting the introductions go on too long will give the candidate less time than others to complete the task.

There’s a lot to keep in mind here just for the first few minutes of the interview! Shadowing allowed me to calibrate on how long my introduction should be, what sort of language I should use to set the candidate at ease, and what I can say about Metaview that will help the candidate understand what we do and why it is important.

Virtual shadowing also gave me familiarity with the frontend and backend delivery tasks and showed me a few different routes that candidates are likely to take, along with some common pitfalls or sidetracks. This resulted in a few data points on good and bad examples of various aspects of the task, which helped me calibrate on real candidates.

Reverse shadows

After completing my virtual shadow path, it was time to step into the driver’s seat and take the lead in a real interview—with an experienced interviewer shadowing me. Again, using our own product, the experienced interviewer didn’t need to be present in the live interview—they played the interview back on Metaview after the fact and left comments on the interview transcript with feedback and tips for me to improve. Examples of some of the areas of improvement they pointed out to me:

• Forgetting to outline the structure of the interview in the introduction.
• Asking multiple questions at once (stacking questions).
• Missing out on signal by not digging deeper into particular aspects of candidates’ answers/thoughts.

One struggle for me early on was knowing when to give help and advice to the candidate. (My teammate Lyubo has written a great guide that touches on this.) I learned a better sense of when to help vs. when to let the candidate work through an issue as I improved at picking up on signal for the key competencies I’m assessing—the main way I did that was through comparing scorecards.

Comparing scorecards

Detailed scorecards with real examples, interviewer opinions, and explained reasoning are not only a useful resource for others to see how the candidate did in their interviews, but also an invaluable tool for calibration and alignment between interviewers.

Especially when I was first learning to interview, I would compare my scorecard to that of more experienced interviewers to see where we were aligned or misaligned, and see what I missed. Sometimes we would have a quick call after the interview to discuss differences in our scorecards in further detail. I would jot down anything I missed and make sure to look out for in future interviews.

I have also found the scorecard-writing time a useful opportunity to reflect on how I did as an interviewer—it’s easy to self-identify certain points to improve such as whether I finished the interview on time.

Onwards

After going through a thorough training process where I had a chance to learn from more experienced interviewers and get feedback on my own skills, I felt much more confident leading interviews independently and assessing candidates.

Nowadays, I rarely lead an interview where I can’t think of anything I could have done better—I’m still learning! I find points to improve on through scorecard comparison with fellow interviewers or comments on my interview transcript from more experienced interviewers. My main takeaway from this ongoing learning process is that interviewing is an active activity—even if I’m watching someone code, not saying much, I have to be alert and sharp to squeeze as much signal out of the interview as possible.