Why Babylist still uses a no-AI coding screen

We're not yet at the point where engineers don't need to understand code. To understand code well, you need to be able to write it. That's the main reason Babylist's first coding screen is done without AI, even though our engineers use AI to write code every day and we interview candidates separately on how they work with it. This post covers our reasoning, what we look for, and what we're still figuring out.

How we use AI today

We've invested heavily in AI coding tools, and using them well is a basic expectation of the job. We've also invested in automated code review, including agentic reviewers that we keep improving.

The way our industry builds software is changing quickly, and so are the practices around it. Our hiring process reflects where we are now. It isn't a fixed position.

Why we still do a screen

Given all that, it's fair to ask why we still have candidates write code without AI. That isn't how they'll work once they join.

Our answer is that most of what we learn in a screen comes from how you work, not from the code you end up with. That was arguably true before AI, too. In an hour without AI, we look for:

• How you approach and break down a problem
• What questions you ask and what assumptions you make
• How you communicate while you work
• How you debug and adapt when something doesn't work
• How you organize your code
• Whether you know common patterns and when to apply them
• Whether you can explain tradeoffs clearly
• Whether you're comfortable with basic data structure operations, such as working with arrays and objects

Engineers are still responsible for the code

Most changes at Babylist are still reviewed by an engineer before they ship. In some areas we've fully automated review, and we expect that to grow. Either way, engineers are accountable for what we ship, whether a person or an AI agent wrote it.

That accountability depends on reading code critically: recognizing sound structure, catching a missed edge case, and knowing when a simpler approach exists. Those are the skills the screen looks for.

We assess AI-assisted coding separately

The no-AI screen isn't our only technical interview. In a separate round, candidates work in their own GitHub Codespace with the AI coding agent of their choice, extending a small app with new features. There we look at how they plan and direct the work, how they review what the agent produces, and whether they understand the code they ship.

Keeping the two apart gives us a clearer signal on each. When a candidate uses AI during the screen, it's hard to tell which parts reflect their own reasoning.

What the screen looks for

The screen isn't a trick question. It doesn't test memorized algorithms or obscure language features. We want a sense of how you approach problems, how you communicate, and how you design systems. A working solution helps, but your reasoning along the way matters more.

We also separate style from design. Your style is your own, and your naming and conventions probably won't match ours. That's fine. What we pay attention to is the design: which pattern you reach for, and why. Explain your decisions, back them with sound judgment, and you'll give us what we're looking for.

Advice for candidates

• Ask questions. Clarifying the problem is part of the work.
• Build what the problem needs. If you aren't sure whether something is in scope, ask instead of building extra.
• Talk through your thinking, including approaches you consider and reject (and why!).
• If you get stuck, say what you're trying and why. We care about how you debug, not only whether you finish.

What we're still working out

We want to be direct about the limits of this approach.

A coding screen is an artificial exercise. The problem is contrived, and so are the environment and the constraints. Our engineers rarely write code by hand anymore, and nobody works day to day in a live, timed interview. We accept that tradeoff because we haven't found a better way to see how someone reasons through a problem on their own.

A live interview also depends on the interviewer's judgment, and no process removes bias entirely. We keep working on making our evaluations more consistent across interviewers.

And we expect this process to change. No one knows yet what engineering hiring should look like as AI tools improve, or what predicts success in roles that are changing this fast. This is how we're handling that uncertainty today, and we'll change our approach as we learn more.