Hiring engineers in the age of AI

Technical interviews have traditionally centered on coding: which languages and frameworks a candidate knows, and whether they can write out the solution to an algorithm problem on a whiteboard.

In the age of AI, that is no longer an effective bar. Someone who isn’t familiar with a particular programming language can still deliver a valuable product with AI’s help.

Two abilities deserve a closer look.

The first is an understanding of the whole system: which parts a complete system should have, how they work together, and how to confirm it is running properly once it is live. Without that understanding, there is no way to evaluate the designs AI proposes.

The second is curiosity: actively keeping up with new knowledge, especially in AI. Tools and models change quickly, and today’s best practice may be outdated in six months. Whether someone keeps learning, trying and reflecting on new technology has a direct effect on how they grow as an engineer.

The six abilities that matter most now

1. Expressing intent precisely to AI. AI can only work with the information it is given: the goal, the constraints, what counts as done, what is decided and what isn’t. The more clearly these are stated, the closer the output comes to what was intended. Vague input tends to produce a result that looks reasonable but misses the requirement.

2. Understanding the AI’s intent. When AI hands back a plan or a piece of code, an engineer has to be able to follow its reasoning: how it understood the requirement, what it assumed, and where it is guessing. Recognizing that the AI has misunderstood matters more than being able to write the same code by hand.

3. Forming an initial view of a system from a vague goal. Requirements are usually vague at the start. Before starting work, a good engineer forms an initial view of the system: roughly which parts it will need and where the hard problems may lie. They revise that view as new information arrives. Only with that view can they tell whether the AI’s design is heading in the right direction. (How I do it: How I design systems now.)

4. Designing tests. However fast code is generated, verification can’t be skipped. Which parts must be tested, in what way, and whether the tests actually cover the behavior they are meant to verify: all of this takes an engineer’s judgment.

5. Designing monitoring. Before release, tests check key behavior and reduce the risk of defects; after release, monitoring watches how the system runs and surfaces problems early. Neither guarantees the system is always correct, but both are needed. Which logs and metrics are needed, and who finds out about a failure and how quickly, should be considered at design time, not patched in after an incident.

6. Managing an AI workflow. Whether with one agent or several, how work is divided, how output is reviewed, how context is kept, and how to avoid becoming the bottleneck are engineering skills in their own right. (How I do it: How I run a dozen AI agents at once.)

How to assess them in an interview

Using grill-me to assess judgment and communication

grill-me is an AI skill: given a plan, the AI asks questions in rounds, following how the decisions depend on each other, and attaches a recommended answer to each question until every decision is settled.

In the interview, give the candidate a vague requirement and ask them to talk it through with the AI using grill-me, until it becomes a clear design. Before the conversation, ask the candidate to outline the system’s boundaries, key assumptions and open questions; afterwards, ask which judgments changed and why. This step assesses ability 3: what the candidate’s own judgment was before the AI made any suggestions.

During the conversation, the interviewer watches for two things:

• Independent judgment. The test is not whether the candidate disagrees with the AI, but whether they can explain why they accept or reject each suggestion, and how they would verify it. Someone who relies on the AI entirely will struggle to notice when it is wrong.
• Precise communication. Are the answers clear, with the context and reasons the AI needs? When the AI misunderstands, does the candidate notice and correct it?

Compared with a whiteboard problem, this is much closer to the candidate’s daily work.

System design: do they consider testing and monitoring on their own?

System design questions are still valuable, but what they assess has changed. AI can draw components and data flows too. What deserves attention is whether the candidate brings up, unprompted, how the system will be tested, how it will be monitored once live, and how failures will be detected. Bringing these up is only a lead; follow up with a concrete failure scenario: how a test would expose the problem, which signal would reveal it in production, and who would respond. These answers provide evidence of whole-system understanding, but one scenario is not enough to establish it.

How the candidate manages AI agents day to day

Finally, ask the candidate how they work with AI day to day: how many agents they use at once, how they divide the work, how they review the output, what problems they have run into, which new tools they have tried recently, and why they kept or dropped them.

Concrete experiences and lessons are a basis for further questions, but actual ability can’t be judged by how fluently a story is told. The question also assesses curiosity: what someone has been trying lately says more than the list of skills on their résumé.

Keeping it fair

The hiring team should provide the same AI tool, model and account for everyone, with the needed skills installed in advance and time to get familiar with the setup, and use the same task, time limit and scoring criteria. No points for the number of agents or for experience with paid tools: a candidate who hasn’t had the chance to use these tools can still show, in the live task, how they organize work, verify results and learn new methods.

In closing

What the age of AI needs is not the fastest coder, but engineers who can work with AI to turn a vague goal into a complete system that can be tested and monitored, and who keep learning.