<-- home

Interviewing as a Software Engineer

Interviewing as a software engineer is effectively broken. As an interviewer, it’s basically impossible to really determine someone’s skill level, and as an interviewee it’s basically impossible to prove to someone why they should hire you.

Let’s look at the structure of interviewing, and then talk about what interviewing would be like in a perfect world.

When you’re an interviewer, there is one question that you need to answer. That question is, “Is this person capable of providing value to this company?”. Businesses need to generate revenue. Ideally they even generate profit. Employees are vehicles to generating revenue, and regardless of how ideal or positive you might be, that’s exactly what you’re there for. As the person making a hiring decision, or just providing feedback to someone who will make a hiring decision, you have to determine whether or not this person will help in that goal. The company will typically be paying some salary (or equivalent) to this person, which means at the very minimum, that person needs to provide enough value to match their salary.

Say someone gets paid $100,000 a year. This potential hire at the absolute bare minimum, must generate $100,000 a year in revenue or value for the company, otherwise that hire is a money pit. It’s worth pointing out, that this breaks for a company like Google. Almost all of their money comes from search ads, but not every single engineer is working on search. When a company prints money like they do, the rules just break in general. Anyways, back on topic.

In what ways can someone provide value? Well, maybe you work on a feature that gets the company more customers. Maybe you wrote a clone of some service that’s paid for, and you reduce costs elsewhere. Maybe you come in and refactor something which helps other people implement a feature that provides value, which means you’ve also, albeit indirectly, but you’ve also provided value. The ways that someone can provide value is endless. This is a problem. This is a real problem because that means that with no limitation of parameters, everyone can decide differently how to determine if someone will provide value. There’s a bias in how everyone is going to think about providing value. This also means that everyone is going to have a different way to evaluate what kind of value you would provide. This is entirely broken.

Interviewing is too emotional. If you interview with someone on a bad day, they’re going to fixate on reasons why you couldn’t possibly provide value. If you interview with someone whom with you have a lot in common, then they’re going to inherently believe that you could provide value, because they think they provide value. Of course they probably provide value, but they’re attributing the wrong characteristics to why you could provide value. It doesn’t matter if you both like the same TV shows, or you guys come from the same small town. Those factors will not help you provide value, but they will make the interviewer like you more, and they’ll make them want to find reasons to hire you.

Interviewing is too blind. Writing software is so much more than just the act of writing code. If you want to write software professionally, you’re talking about planning, debugging, writing, testing, bug reporting, all sorts of things. But most people will only interview on two things: culture fit, and technical ability in its broadest definition. It’s impossible to be able to measure all of that, let alone in a reasonable amount of time to ask of an interviewee.

The first step in the job application pipeline is typically the resume screen. You have to somehow distill your most impressive information into a single page. How can you do that though? Well, the best way is for you to describe situations where you provided quantifiable metrics of value for your employer. As we’ve established though, sometimes figuring out what that exact amount is is impossible. Put it into some scale of numbers though. For example:

  • Increased code coverage of a 200k LOC project from 70% to 75%
  • Reduced a search query from taking 200ms down to 25ms
  • Increased average use time by 3 minutes with feature X

These examples don’t translate to an exact dollar value, but they provide some understanding of improvements that you’ve made that will in turn translate to value.

Ultimately though, resumes aren’t really a great indicator of who might be best for the job, because someone might have truncated information to get it to fit to a single page. Maybe someone is just a bad writer and can’t express their accomplishments well. Sometimes the work you do has to be written as to remove specific but impressive details due to an NDA or some other reason. In that case, what are your options? What can you do that’s better than representing yourself with a resume? You can get referred.

A long career usually means you’ve had many coworkers. Not all of them will stay at the same company for their entire life. If you have friends at a different company, you can ask them to refer you. This works well for college new graduates as well. Make friends with people who are graduating before you, and when it’s your turn to graduate, ask around your alumni friends and see what they’ve all been up to. This is one of the reasons that networking and going to meetups is really useful, because you can use it to help skip the resume screening step which is painfully noisy. Sometimes you’ll be approached by recruiters on websites like LinkedIn, and maybe you can find someone who works at a company you want to be at on the same website. You can politely message them and ask about getting referred. Many companies will have a referral bonus, so some employees will happily refer you just for their chance to get that bonus.

A relatively new opportunity to skip the resume screening process is to be blindly interviewed. The website interviewing.io will let you do anonymous practice interviews, and if you do well in an interview, you might be invited to do a final round interview. Also, I’m not affiliated in any way with interviewing.io. I’ve used their service for practicing for a Google interview, and I enjoyed it so I wanted to give it a shout out.

After your resume screening, you’ll probably be contacted by a recruiter or someone from HR. Their job is to figure out whether or not they think it’s worth spending any more time to see if you could be a potential hire. Sometimes this step doesn’t exist if a company is smaller and doesn’t have someone dedicated to this job. After the recruiter chat, you’ll likely go into technical steps of the interview. This is where things begin to get incredibly arbitrary and this is where no real solution has been discovered.

There’s two common camps for technical portions of the interview process. The first one is common amongst a lot of larger companies like Google, Facebook, Uber, AirBnB, and so on. These are the infamous whiteboard interviews where you get asked algorithmic problems that are supposed to test you for your problem solving skills. You can find examples of these types of interview problems all over the internet.

Perhaps one of the more famous cases of these problems is when Max Howell, the author of the brew package management system on OSX, interviewed at Google. He wasn’t happy with his experience. Whiteboard problems can be telling, but they tend not to be. These problems are supposed to assess your abilities to think critically, and to check your depth of knowledge for Computer Science. One of the problems with that is that we’re not really computer scientists when we’re on the job. In most cases, we’re software engineers that need to deliver some product to our users. This means that we’re querying databases, or we’re provisioning servers on AWS. Knowing how to use a trie to find words on a boggle board isn’t necessarily going to help us. Of course there are exceptions, like if you’re writing a boggle clone, but many times this isn’t the case. Some of these questions are genuinely inciteful, but it’s up to the interviewer to make them so. The interviewer should be trying to map out your knowledge. Algorithm questions have a tendency to check for one specific area of knowledge. At Google they interview while checking your knowledge in five distinct areas, but they always ask you these algorithm questions. There might be better ways to assess those five areas, but they’ll try to gleam your ability to write object oriented code from how you traverse a graph. A lot of these questions also have some data structure or algorithm that if you knew what it was, you could write a solution very quickly.

Some people will tell you that getting the solution doesn’t matter, and that they’re actually testing your abilities to communicate your thoughts. If this were true, why give someone a question that makes them super uncomfortable? Why take away the resources that they get to use on the job like StackOverflow or auto complete? It’s like a hazing ritual that persists because “I went through it so now you should too”. I don’t like this method of interviewing, even if I find it personally fun. It doesn’t really give you a good idea about whether or not someone might be a good employee for the job.

The common alternative to whiteboard problems is to receive some take home coding challenge. Some people really like these because it lets you show off your abilities in an environment that you get to determine. The problem I find with a lot of these is that they’re super vague. The employers will know exactly what they’re looking for, but they won’t give you any actual direction. They want to see if you can “figure it out” or see if you follow “best practice”. They justify it by saying that you need to be a self-starter, but this is garbage. Someone might spend 30 minutes working on your challenge, but some people might spend 20+ hours doing all sorts of testing, CI/CD, and who knows what else. The size of these projects also tends to be way off base with what’s reasonable. Different people work at different paces, but this isn’t really telling for a potential employee either. Someone might submit a working solution for your vague problem and it only takes them an hour or two. However, you’ll quickly find that their solution doesn’t generalize well. Maybe someone takes a while to do the challenge, and you think that means they’re slow, but it turns out they have 100% code coverage and have tests for a large number of edge cases. I think most people would say they want the latter case to happen, but what if this takes them 10 hours because they’re using a stack that they’ve never used? Are you going to ding them for not having experience in your EXACT stack? That’s also unreasonable because of the insane number of combinations of tools someone might have in a given stack. Also, lack of experience in Rails doesn’t mean someone would make a bad Rails developer.

Sometimes people give challenges that they expect to take 40+ hours. That means just for an interview, you’re expected to put in over a week’s worth of work. And you’re not even guaranteed the job. The solution to this that I’ve seen is when a company will ask you to develop such a large feature, but they’ll actually pay you a consulting fee for your time. I really like this because it means that the company is more respectful of your time. There’s still problems here though. You might be interviewing multiple people at once, and maybe one person has external time commitments that means it takes them 3 weeks to do the 40 hours worth of work. There’s another candidate who knocked it out in one week + the weekend and now he’s waiting to hear back. What do you do? Do you tell all candidates that they have to wait for the slowest person? The last person to submit the challenge might be the best programmer but you need to hire someone sooner, so you go with the person who submitted first. You’re giving an advantage to the person who can drop everything and work on your large challenge. Even in the opposite perspective, if you wait for the person who takes longer, that means you’re drawing out the interview process for the person who was more responsive to you.

There are problems with both of these types of interviews, and sometimes you’ll have companies who will do both to extreme ends. I’ve read about interview processes that take months of constant interviewing with different people. As a candidate you might be trying to get a job as soon as possible, and you just don’t have that kind of time to wait. Unfortunately, if there’s a better method of interviewing, I’m not really sure what it is. This mock interview done by Casey Muratori and Shawn McGrath is an interesting approach that I find useful. It’s basically just a really long conversation about your history and experience, where the interviewer is dynamic and flexible about asking questions to help guide the interviewee to talk about things that are useful to know. Other than that, I think the best way to hire is to use your personal network and take strong recommendations very seriously. There are many people that I’ve met, where if they were looking for a job, I would swear on their abilities to my current employer. This still only works with people with longer careers, and its biased to help the people who are social and build strong networks, but I think using a web of trust is the best way to hire. You trust your inner circle enough to hire them, now you need to trust that their circles are good enough to be hired as well.

If you have stories or ideas about ways you’ve found effective to interview people, feel free to email me: [email protected]

Thanks.