Humans are the slowest part of the debugging loop

I recently set up an SMB share for Time Machine backups on my homelab. It’s a fiddly task fighting networking and permissions issues, and Docker configuration. I thought an agent would help speed things up.

It turned out to echo something I keep running into at work with ML models: to get the best out of an agent, you need to relinquish control in ways that feel uncomfortable, but only with the right guardrails in place.

Setting up Time Machine over SMB

#

I wanted to set up an SMB network share on my Ubuntu homelab to use as a Time Machine backup destination¹. The requirements were:

• Containerised within Docker
• Configurable enough to build a runtipi app later
• Modern Samba version (more efficient streaming for virtual APFS)

Definition of done: The Mac can see the SMB share AND successfully read and write to it.

The naive approach: using myself as a middleman

#

Initially, I ran Claude Code on the homelab server and told it to set up the share. Then I’d manually test the connection from my Mac and report back any errors.

This was slow. I had to:

• Describe what wasn’t working with enough context
• Remember to test all functionality (I kept forgetting to check write access)
• Go back and forth between machines

I knew the agent could test the connection itself, and that it could SSH between the two machines. But I kept doing it manually anyway. Partly because I’m used to being hands-on, but more fundamentally, it felt risky to give an agent full SSH access to both machines for something this simple; like using a sledgehammer to crack a nut.

The shift: let the agent own everything

#

Then I realised: why am I doing half the work when the agent can do all of it?

I changed the workflow:

1. Run Claude Code on my Mac (not the server)
2. Have it SSH into the homelab server
3. Now it can configure the server AND test the connection from the Mac
4. It sees full error context from both sides and can iterate on its own

The agent would:

• SSH in, adjust the smb.conf
• Test the mount from the Mac side
• See the error message
• SSH back in, fix the config
• Repeat until it worked

Then I just watched it debug itself. It was much faster.

However, this only worked because I had tight constraints beforehand. I’d explicitly told it: modern Samba, must be containerised, needs to work with runtipi. Without those guardrails, it would have taken shortcuts that technically worked but didn’t meet my actual requirements.

Building models is similar

#

At work, I’ve started to use agents to iterate on machine learning models for time-series forecasting², and I’ve noticed the same pattern.

If you give the agent free rein to optimise a forecasting model without a well-defined evaluation methodology, it will confidently report unbelievable metrics. This is because the agent can cheat.

Data leakage in time-series is a classic failure mode. The model uses future data to predict the past, so the results look amazing, but it has cheated. The agent doesn’t know this is wrong because it just sees the metrics improving.

For ML work, the guardrail is evaluation. You build a thorough evaluation pipeline which uses proper train/test splits, validation that catches temporal leakage and metrics that actually measure what you care about. Once you’ve defined that and tested it properly, the agent can iterate through dozens of model configurations in minutes.

For sysadmin work, the guardrail is end-to-end testing. Not “config looks right”, but “does it actually work when I try to use it?”

The pattern

#

Relinquishing control is hard, especially if you’re used to being hands-on. But agents can iterate far more quickly, and are far more patient, than we will ever be.

It already knows smb.conf and PyTorch syntax better than I do. What it cannot do is decide what “correct” means, or how to test for it. So the paradox is that you have to be more rigorous about defining success, not less, to be hands-off about how you (or the agent) get there.

In practice that has meant:

1. Defining requirements clearly beforehand: otherwise the agent takes shortcuts around them
2. Making “done” testable: giving the agent something it can verify itself
3. Giving it access to test its own work: getting myself out of the iteration loop
4. Intervening when it gets stuck: asking it to summarise root cause and suggest alternatives

Where domain expertise still matters

#

Our role has shifted: we set the problem up so the agent can explore it, rather than doing the iteration ourselves.

For time-series forecasting: “Does this evaluation catch the ways my model could be wrong?”

For SMB configuration: “Does this test prove the system actually works end-to-end?”

If you define it correctly, the agent can move fast. If not, we’re just automating the production of plausible-looking slop.

I’m still not entirely sure what the best way to enforce these guardrails is. Is it through writing skills that the agent can reference? Is it just careful prompting? I suspect it’s a mix of both, but I haven’t figured out the right balance yet.