What I learnt about interviewing while at Uber

“The old adage “People are your most important asset” turns out to be wrong. People are not your most important asset. The right people are.”

Jim Collins, Good to Great.

Just over four years ago, I joined the fastest growing startup ever: Uber. It was truly a privilege to be a tiny part of the ferocious growth and expansion brought about by the sweetest example of product/market fit of recent times. Having joined Uber from a much smaller startup, and before that having worked at highly-successful but mature corporates, it was the scale-up experience I was after. And that is exactly what I got. My time there was littered with learnings about how to scale a business, a product, and even myself. But none feel more important than what I learned about hiring and scaling teams.

I was on the hiring panel for most Product Manager, Designer, Data Scientist, and Product Marketing Manager roles in EMEA. While interviewing was always something I thought important and enjoyed, I only really started to develop a keen interest in the art and science of interviewing once I started doing it so regularly for such varied roles while at Uber. The reason for this: this was the first time it legitimately felt like the most high-leverage thing I could possibly be doing. Uber had (and has) so much potential. So many high-impact projects across so many teams and for so many different customers (riders, drivers, couriers, restaurants, API partners). With so many high-potential initiatives in-flight (including those being owned by my Driver Payments team), it was impossible for Uber to adopt anything other than a high-autonomy environment for their employees if they were to have any chance of chasing these opportunities down. High-autonomy environments demand high-competency people and — perhaps more importantly in this case — magnify the negative impact of people that join the company that aren’t a fit. So, just like Jim Collins says in the quote above, getting the right people felt truly vital.

As I started to take keener notice of what goes in to making good hiring decisions, I noticed some things I wanted to share…

The stuff we should all be doing

Let’s start with the good stuff. These are the behaviours and processes I witnessed which showed me what ‘next-level’ hiring looked like.

It’s (a key) part of the job

Given Uber’s headcount growth — c.7k when i joined in March 2016, 16k+ by the end of 2017 — and the rigorous process through which we put all candidates, interviewing was obviously something that lots of people had to dedicate a lot of time to. What was an eye opener to me was the attitude that many (not all) people brought to the task. This was particularly true within engineering, where taking on the moniker of ‘lead interviewer’ (after shadowing experienced interviewers, then being shadowed) was an achievement often tied to personal objectives.

In summary, time spent interviewing was legitimately seen as a key part of the job, not a distraction away from your day-to-day. And so it should be. It made me realise that it’s very unlikely anything else I am doing today is more important than influencing who the people are that will be responsible for delivering on the vision of the company.

Divide and assess

Though not consistently enforced (more on that later), I found that clearly defining areas of assessment for each interviewer worked really well for a few reasons:

• It forces the hiring manager to be clear in her own mind about what the most important attributes are for the role and ensures they are tested for.
• Making the universe smaller made it easier for interviewers to prepare lines of questioning or scenarios for the interview.
• It focuses the feedback of the interviewer. Interviewers are no longer making a ‘hire/no-hire’ decision on the candidate as a whole, merely on a specific attribute which can then be used as a data-point for the overall decision later in the process.

Crucially, I also believe the explicitness and focus that this approach demands can have a neutralizing effect on any unconscious biases which more often hang out in the ambiguities of a more general interview.

Feedback on the candidate

Before Uber, I had the bad habit of failing to supply adequate feedback on ‘no-hire’ candidates. My mental model at the time — which had me believe that hiring was a distraction from my task at hand — meant that I saw interviewing poor candidates as a ‘waste of time’, and any additional time spent on the task of providing feedback to the rest of the hiring panel was merely wasting more time. As such, I often provided just a sentence or two of feedback.

This attitude left me a little embarrassed a couple of times when contributing to hiring panel discussions on a candidate (another thing I saw as a waste of time for a clear ‘no-hire’). Not having written down detailed feedback, I found myself unable to contribute in a meaningful manner to discussions about the suitability of the candidate. I wasn’t doing my job properly, and was not in a position to support my own opinion on the candidate in the face of rigorous, thoughtful feedback from others on the panel.

Insisting on high levels of quality feedback on a candidate is also a way — in retrospect — to ensure interviews are being conducted thoroughly and thoughtfully. As soon as someone knows they are expected to provide extensive feedback on a candidate, they become incentivised to interview them in a thorough and thoughtful way.

The best interviewers get into the habit of committing 15–20 minutes after the interview to expand on their written notes and provide proper feedback for all candidates. Do this. Everytime.

The stuff we can all still get better at

However, even Uber — a company that has successfully grown quicker than any other — is on a journey to improve their hiring and still has lots to do. These are the areas in the hiring process where I believe all companies still can improve.

Lack of consistency

I came to a realisation that — from a candidate perspective — there is a large degree of arbitrariness about the outcome of a job interview. It is dependent on so many things that are outside of your own control:

• The questions that are asked — candidates may get lucky and shine with some questions, or unlucky and struggle with different ones.
• Unconscious bias — characteristics or traits that have little to do with aptitude may reduce chances.
• The people on the hiring panel — different people will make different decisions.
• Interviewer context — if the interviewer has just come out of a meeting that didn’t go well for them and is on the hiring panel for a role that does not directly benefit them or make their life easier, it will be harder to impress them. Conversely, if the interviewer is currently the one suffering at the hands of ‘lack of resources’, they are more likely to view the candidate positively.

The purpose of hiring is to identify and attract the best people for your organisation. It is incredibly hard to do this reliably with such a hearty dose of arbitrariness thrown in the mix.

One thing that many companies do — including Uber — is encourage interviewers to conduct Structured Interviews. Structured Interviews are interviews that are conducted with a common set of questions, irrespective of who is responsible for asking them. This certainly helps recruitment teams to achieve some sort of consistency in the interview process. However it is at a cost: The ability to pull at emerging threads of relevant questioning is sacrificed, and it becomes nigh-on-impossible to engage in a more conversational form of interviewing which can be alienating for the candidate.

Lack of transparency

Like many high-performing companies, at Uber we used OKRs to set clear objectives and ensure ownership and accountability was transparent. OKRs are also a great way to demonstrate team or company priorities, and aligns people on how their performance will be assessed.

With that in mind, it is noticeable that these OKRs very rarely include hiring or interviewing goals (outside of the recruitment team), despite the fact everyone agrees — all the way up to the CEO — that spending time and energy interviewing is a key task. Being good at identifying and attracting the best candidates during the interview process is a key skill, and therefore one that should be rewarded.

The opaqueness of the interview can lead to bad habits for the interviewer. Most commonly: failing to prepare and therefore not knowing basic background about the candidate, spending too much time talking and not enough time actually assessing the candidate, or assessing in a way that doesn’t help the company make an accurate judgment about the candidates suitability for the role. Lack of transparency also contributes to the lack of consistency mentioned above.

To address these bad habits, companies usually invest their employees time in interviewer training. Yet the impact of it is completely unclear. Does it result in better interviews? Better hires being made? More quickly?

Dealing with uncertainty

There are some occasions where the hire/no-hire decision is an easy one. Investing in interviewer training, creating structured interview processes, and engaging in rigorous feedback can help increase certainty further, but inevitably many candidates will continue to fall somewhere in the middle. This leaves you with three options, all with their downsides:

• Err on the side of caution: call it a ‘no hire’.
• Downside — you miss out on some good people, and increase your time-to-hire.
• Err on the side of risk: ‘hire’ the candidate anyway.
• Downside — you hire some bad people.
• Invest more time: Get the candidate in for an additional interview to get a final decision.
• Downside — You increase time-to-hire, and it’s not a good candidate experience.

These decisions on the margin are massively influential in determining the calibre and culture of your teams. Given this, It feels wrong to me that they are made from positions of such relative ignorance.

What can we do about it?

High performing businesses are increasingly taking a data-led approach to recruitment — which is awesome — but it looks like interviews have been left behind. The reality is that without being able to measure what is going on in interviews, they are very hard to manage at scale. And without engendering a repeatable and consistent interview process, hiring decisions will remain far too arbitrary.

That’s where Interview Analytics comes in.

At Metaview, we believe emergent voice recognition and AI technology can be used to plug this black hole of data and contribute towards better, quicker, more reliable hiring decisions. I’ll be writing more about the field of Interview Analytics in the coming weeks, but for now you can check out our website to discover what is already possible with Metaview.