halcight · System design and AI coding

Hiring engineers in the age of AI

Fri, 13 Nov 2026 00:00:00 GMT

Technical interviews have traditionally centered on coding: which languages and frameworks a candidate knows, and whether they can write out the solution to an algorithm problem on a whiteboard.

In the age of AI, that is no longer an effective bar. Someone who isn’t familiar with a particular programming language can still deliver a valuable product with AI’s help.

Two abilities deserve a closer look.

The first is an understanding of the whole system: which parts a complete system should have, how they work together, and how to confirm it is running properly once it is live. Without that understanding, there is no way to evaluate the designs AI proposes.

The second is curiosity: actively keeping up with new knowledge, especially in AI. Tools and models change quickly, and today’s best practice may be outdated in six months. Whether someone keeps learning, trying and reflecting on new technology has a direct effect on how they grow as an engineer.

The six abilities that matter most now

1. Expressing intent precisely to AI. AI can only work with the information it is given: the goal, the constraints, what counts as done, what is decided and what isn’t. The more clearly these are stated, the closer the output comes to what was intended. Vague input tends to produce a result that looks reasonable but misses the requirement.

2. Understanding the AI’s intent. When AI hands back a plan or a piece of code, an engineer has to be able to follow its reasoning: how it understood the requirement, what it assumed, and where it is guessing. Recognizing that the AI has misunderstood matters more than being able to write the same code by hand.

3. Forming an initial view of a system from a vague goal. Requirements are usually vague at the start. Before starting work, a good engineer forms an initial view of the system: roughly which parts it will need and where the hard problems may lie. They revise that view as new information arrives. Only with that view can they tell whether the AI’s design is heading in the right direction. (How I do it: How I design systems now.)

4. Designing tests. However fast code is generated, verification can’t be skipped. Which parts must be tested, in what way, and whether the tests actually cover the behavior they are meant to verify: all of this takes an engineer’s judgment.

5. Designing monitoring. Before release, tests check key behavior and reduce the risk of defects; after release, monitoring watches how the system runs and surfaces problems early. Neither guarantees the system is always correct, but both are needed. Which logs and metrics are needed, and who finds out about a failure and how quickly, should be considered at design time, not patched in after an incident.

6. Managing an AI workflow. Whether with one agent or several, how work is divided, how output is reviewed, how context is kept, and how to avoid becoming the bottleneck are engineering skills in their own right. (How I do it: How I run a dozen AI agents at once.)

How to assess them in an interview

Using grill-me to assess judgment and communication

grill-me is an AI skill: given a plan, the AI asks questions in rounds, following how the decisions depend on each other, and attaches a recommended answer to each question until every decision is settled.

In the interview, give the candidate a vague requirement and ask them to talk it through with the AI using grill-me, until it becomes a clear design. Before the conversation, ask the candidate to outline the system’s boundaries, key assumptions and open questions; afterwards, ask which judgments changed and why. This step assesses ability 3: what the candidate’s own judgment was before the AI made any suggestions.

During the conversation, the interviewer watches for two things:

Independent judgment. The test is not whether the candidate disagrees with the AI, but whether they can explain why they accept or reject each suggestion, and how they would verify it. Someone who relies on the AI entirely will struggle to notice when it is wrong.
Precise communication. Are the answers clear, with the context and reasons the AI needs? When the AI misunderstands, does the candidate notice and correct it?

Compared with a whiteboard problem, this is much closer to the candidate’s daily work.

System design: do they consider testing and monitoring on their own?

System design questions are still valuable, but what they assess has changed. AI can draw components and data flows too. What deserves attention is whether the candidate brings up, unprompted, how the system will be tested, how it will be monitored once live, and how failures will be detected. Bringing these up is only a lead; follow up with a concrete failure scenario: how a test would expose the problem, which signal would reveal it in production, and who would respond. These answers provide evidence of whole-system understanding, but one scenario is not enough to establish it.

How the candidate manages AI agents day to day

Finally, ask the candidate how they work with AI day to day: how many agents they use at once, how they divide the work, how they review the output, what problems they have run into, which new tools they have tried recently, and why they kept or dropped them.

Concrete experiences and lessons are a basis for further questions, but actual ability can’t be judged by how fluently a story is told. The question also assesses curiosity: what someone has been trying lately says more than the list of skills on their résumé.

Keeping it fair

The hiring team should provide the same AI tool, model and account for everyone, with the needed skills installed in advance and time to get familiar with the setup, and use the same task, time limit and scoring criteria. No points for the number of agents or for experience with paid tools: a candidate who hasn’t had the chance to use these tools can still show, in the live task, how they organize work, verify results and learn new methods.

In closing

What the age of AI needs is not the fastest coder, but engineers who can work with AI to turn a vague goal into a complete system that can be tested and monitored, and who keep learning.

How I run a dozen AI agents at once

Sat, 07 Nov 2026 00:00:00 GMT

I work on eight projects in parallel, with as many as 17 agents running at peak.

I manage them in Herdr, a terminal workspace for coding agents. Each agent gets its own pane, and Herdr can tell whether it is working, blocked, done or idle. Agents can also read each other’s output and send each other messages.

But the tool only solves visibility. The real bottleneck is my attention: every switch to another pane means rebuilding that project’s context in my head. The more agents I run, the more I become the slowest part of the system.

My answer is a structure with two levels, plus one agent whose only job is to check on the others for me.

Deep conversations with main agents, shallow reads of everyone

I align in depth with each main agent
the orchestrator reads, notifies, relays
alerts to me
a main agent starts subagents and accepts their work

Two kinds of conversation run at the same time:

Me and the main agents: in depth. Each project has one main agent that holds the project’s full context. I align with it directly on the design, the plan and the open decisions.
The orchestrator and every agent: shallow reads. The orchestrator owns no project. Every 5 minutes it reads the latest output of every agent, and it reports to me only what needs me.

So I see the whole picture without switching between projects. When I need to go deep, I enter one project and talk to one main agent.

Only two levels. A main agent may start subagents: a reviewer, the implementer of one PR, or a few groups working in parallel. A subagent never starts another level. Every agent is therefore at most one step from a main agent that holds the context, and that main agent accepts its work. Any deeper, and responsibility blurs while context leaks away at every level.

Four roles

Me: I set the direction with each main agent, and I handle the decisions that need me, approval dialogs, and the steps only I can take in a web UI. Risky operations, such as writes to cloud resources or infrastructure changes, need my explicit OK.
Main agent: owns one project: its context, its plan, its PRs, and a file that records its current state. It owns every PR it opens until merge, including CI, review comments and conflicts. It may start subagents, and it accepts their work.
Subagent: does one well-bounded task for its main agent, such as implementing a PR, reviewing a design or making one cleanup pass. It reports to its main agent and is closed when it is done.
Orchestrator: owns no project and no PR. Every 5 minutes it checks every agent, reports what needs me, notifies both sides of any overlap, and relays my answers word for word.

Every 5 minutes: overlaps first, then attention

Every run of the orchestrator follows the same order: the overlap check, the attention check, then the report.

These two checks are the core of the whole setup. With many agents, the usual problem is not one agent getting something wrong. It is two agents each doing the right thing and colliding, or an agent blocked and waiting for me while I have no idea.

Overlap check: one direction per area

Main agents cannot see each other’s context.

An overlap is one area that two agents are both changing or deciding on: a module, a document or architecture decision record (ADR), an interface contract, a PR, a deployment. The orchestrator matches by meaning, not only by path: services/foo and docs/spec/foo are the same area.

When it finds an overlap, the orchestrator notifies both sides at once. Before either one changes that area again, both must know the other’s plan and agree.

These are not overlaps:

a main agent and its own subagents, which share one task brief;
an agent that only reads or reviews;
an area where one side’s change is already merged.

A pair that was notified in the last 24 hours, with no new change since, is not notified again.

Example: two frontend PRs touching the same files

Projects B and E are building separate apps in the same frontend repository.

The day before, the orchestrator first told both about the shared parts of the repository (the component library, the mock backend, the interface contracts), and they agreed which shared files each one could change.
10:11: a PR from project E changed the developer guide and the lock file, and it was next in line to merge. Project B’s newly opened skeleton PR changed the same two files. The orchestrator’s read: aligned (separate apps), but whichever merged second would conflict. It notified both and recorded the pair.
10:12: the two agents talked directly and agreed that E merges first and B rebases after. Each wrote this into its state file.
10:16: E’s PR merged and was deployed to the development environment.
10:36: B finished its rebase. The guide kept both changes, the lock file was unchanged, and type checks, lint, tests and the build all passed.

Nobody came to me. My report gained only one line: “Notified … aligned”.

The notice to project B went roughly like this; project E got the mirror image:

Overlap check (from the orchestrator): you and project E are both changing shared files in the frontend repository: the developer guide and the lock file.
- You: your skeleton PR changes both files.
- Them: a PR that also changes both files is about to merge.
- My read: aligned (separate apps), but the second PR to merge will conflict.
Before you change these files again, send them one message with your plan, read their answer, agree who rebases, and write the agreement into your state file.
If you cannot agree, or the choice belongs to a human, stop changing these files and say so in your reply.

The last sentence is the important one. A conflict the two cannot settle, or a decision that should be mine, is never settled privately between agents. That work stops and shows up in my report as a conflict for me.

Attention check: what counts as “needs me”

After the overlap check, the orchestrator decides for each agent whether it is waiting for me. The key is a precise definition of “needs me”. Too broad, and the report fills with noise I soon stop reading. Too narrow, and a truly blocked agent slips through.

Needs me:

It is blocked on an approval or question dialog. The report quotes the dialog.
Its last reply asks me to decide, choose or do something, and nothing has answered it since. A step refused by a permission check counts.
It reports a failure it cannot fix on its own.
The host is in trouble: less than 30 GiB of disk left, or Docker not responding. Every agent’s builds and end-to-end tests depend on both.
A hand-off watcher has not polled for more than 10 minutes.

Doesn’t need me:

Waiting for a teammate’s approval or for CI. That goes under Waiting.
Text in an agent’s input box. That is my own unsent input or a suggestion, not a question.
A question that an answer I relayed has already closed.

Each agent that doesn’t need me appears on exactly one line: Working, Waiting or Idle. Idle agents note their last area of work, so I can decide whether to give them a new task or close them.

The report

The report always has the same shape: overlaps first, then what needs me, then one line each for Working, Waiting and Idle. Here is one from a morning, with the names replaced:

Overlaps: nothing needs you. Notified: developer guide and lock file in the frontend repository (project B ↔ project E, aligned).

Needs you:
1. Project B (new app skeleton): grant the frontend repository read access to an internal npm package.
2. Project A (performance): an infrastructure change that cuts a service to 1 vCPU. Keep it or close it?

Working: project B's implementer (app skeleton); project C's group c (16 end-to-end scenarios)
Waiting: project C's group a (1 of 2 approvals); project D (1 of 2 approvals)
Idle:    chores (Docker and disk cleanup); a test coverage audit
Host:    313 GiB free, Docker OK

It takes a few seconds to read, and then I go back to my own work.

That morning’s three alerts

09:45 Docker had stopped, so project C’s group c could not run its 16 end-to-end scenarios, and the main agent asked in a dialog. I chose “start Docker”; the orchestrator started it and answered the dialog. The scenarios ran, and group c opened a PR.
09:53 Project B needed a step only I could take in the GitHub web UI: granting the frontend repository read access to an internal package. Once I had, the orchestrator relayed “done”, and project B started on its first frontend PR right away.
09:53 Project A had measured the load burst at morning start-up, and asked whether to keep or close an infrastructure change that cuts a service to 1 vCPU. I answered “keep it and start”. A few minutes later it opened a PR whose Terraform plan showed only the expected change.

Without the orchestrator, I would have found out that Docker had stopped only when I happened to look at project C.

Attention goes to decisions

What matters here is not how many agents run, but where attention goes:

Only I can set the direction and make the trade-offs, so I talk in depth with the main agents.
“Who is waiting for me” and “who is colliding with whom” can be defined precisely and checked on a schedule, so they go to the orchestrator.
Two levels mean that every agent’s work is accepted by a main agent that holds the context.

You can run many agents in parallel, but only one person makes the decisions. Keep that person’s attention for decisions, and leave the rest to the process.

Is Japanese IT behind?

Wed, 04 Nov 2026 00:00:00 GMT

People often say Japanese IT is behind. My answer is yes and no.

No shortage of great engineers

Japanese engineers have given the industry a lot. The QR code came out of Denso Wave in 1994, with Masahiro Hara leading its development. Yukihiro Matsumoto released Ruby in 1995. Lean and kanban in software development both trace back to the Toyota Production System. Japan has never lacked great engineers.

Those examples say nothing about the average, of course. I do think the average Japanese software engineer is behind their peers in the US and China, and universities have a lot to do with it: software engineering as a discipline is neither widespread nor taken very seriously here. But I don’t think that is the main point. Plenty of engineers in the US came from other majors, and it hasn’t held them back.

The real problem is how companies are built

The real problem is that the structure of the companies hasn’t kept up.

The numbers show it too. According to IPA, Japan’s Information-technology Promotion Agency, 73.6% of Japan’s IT workers worked at IT companies in 2020, and only about a quarter at companies in other industries. The US is the other way round: only 35.1% were at IT companies in 2021, and most work at the companies that use IT. In other words, many Japanese companies have no engineering team of their own. Their systems go to system integrators, often through layers of subcontractors. The people who decide are far from the people who write the code, and engineers have little say in the product itself.

On November 2, 2026, GMO Internet Group held a press conference about unauthorized access to its Onamae.com Desktop Cloud service. The attackers got in through remote desktop servers used by customers and stole an administrator account, and the group’s own security company had pointed out problems with the system back in 2024. What stayed with me, though, wasn’t the security failure. It was one moment at the conference: as the responsible manager began to explain to the press, the group’s chief interrupted him to scold him for his unbuttoned jacket. “We are apologizing, and you leave your jacket open?”

It’s hard to imagine Satya Nadella or Elon Musk stopping someone over a button, in public or in private. Most people would probably find it absurd, yet the person who did it clearly saw nothing wrong. To me this is exactly why so many Japanese IT companies fall behind: a strict hierarchy built around power doesn’t fit with the open, equal environment that software engineering needs.

By hierarchy I don’t mean reporting lines. Microsoft and Amazon have hierarchies and reporting lines too, but they divide the work; they are not a chain of command. There, many projects start from the bottom up. In Japanese companies, ideas rarely travel up like that. Whether the deeper cause lies in Japanese society and culture is beyond the scope of this post, and I won’t go into it here. What is clear is that complex software engineering struggles to get anywhere under a strong hierarchy of power.

Reasoned debate, and knowledge tested by argument, need an environment of equals. Google’s Project Aristotle studied 180 teams and found that, among those teams, of the five factors behind team effectiveness, the most important was psychological safety: team members trust that no one will embarrass or punish them for admitting a mistake, asking a question or offering a new idea. Diversity matters as well. A Harvard Business Review article that sums up the research finds that diverse teams focus more on facts, re-examine them more often and make fewer factual errors. These are exactly what many Japanese companies lack.

Do the leaders know the technology?

Another difference is whether the people leading teams know the technology. In Japanese IT companies, project managers tend to sit near the top of the hierarchy. American and Chinese IT companies are different. At Microsoft, I saw many PMs (program managers) who wanted to become engineers who write code, and very few engineers who wanted to become PMs; most team managers came up as engineers too.

What responsibility means

Then there is how responsibility is understood. In many Japanese companies, one important question behind a decision is whether it keeps the organization from being held responsible: “If I do it this way, I won’t be blamed when something goes wrong.” That’s understandable, but it’s a factor I rarely felt at American IT companies.

What Japan is good at

Now look at what Japan does well. In its creative industries, what one person contributes counts for a great deal. In heavy industry, cars for one, what counts is how rigorously a team follows the rules. Software engineering needs a third thing: discussion among equals, and fast iteration.

Japanese IT has its own individual and small-team breakthroughs too. Chainer, the deep learning framework from Preferred Networks, introduced the define-by-run style in 2015, an approach the research community went on to adopt widely. But for the industry as a whole to move forward, the way its teams think has to change.

What Japanese IT companies should do

First, learn from the top American tech companies: build engineering teams of your own and outsource less. Your systems are a core capability, and they shouldn’t all be in someone else’s hands. Only engineers inside the company can be close to the business and the product, and answer for the results.

Second, bring in highly skilled people from abroad, and understand why: you don’t pay top salaries to smart people to tell them what to do. You pay them so they can tell you what to do.

Third, and most important, embrace an open, equal workplace and weaken the hierarchy of power. At work, focus on the work itself, not, as GMO’s chief did, on a button at a press conference held to apologize.

Fourth, let people who understand the technology make the technical decisions. Give engineers a career path parallel to management, so senior engineers can reach the matching level and pay without becoming managers. The people who lead engineering teams should ideally be engineers too.

Fifth, when something goes wrong, review it without blaming anyone. Google’s SRE practice calls this a blameless postmortem: the review looks only for what caused the incident, accuses no person or team, and assumes everyone acted in good faith and did the right thing with what they knew at the time. What gets fixed is the system and the process, not people. Once people stop fearing blame, decisions go back to asking what is best, not what is least likely to go wrong.

I’m not writing this to point fingers. I want to show where there is room to improve.