A call for --help got out of hand

I leave small maintenance tasks for AI coding assistants to work on while I sleep. A program reads the waiting tasks and starts assistant sessions to read code, make changes and run checks. Their work is left for me to review in the morning. I’ve described the full overnight arrangement before, but this story concerns how that program gets started each night.

On my Mac, the operating system’s service manager, launchd, starts that program on a schedule. A small installation script writes the configuration that tells launchd what to run and when. Re-running that script applies the configuration by removing the old scheduled job and registering it again. If the second step fails, the configuration file can still exist while the job is no longer registered to run.

On 12 September, one of the overnight assistants was reading the code for this setup. It wanted to understand the installation script. At 03:50, it asked for help. With the private directory removed, the command was:

sh bootstrap/install-queue-drain.sh --help 2>/dev/null | head -3

Command-line programs usually treat --help as a request for usage instructions. This script didn’t handle arguments at all, so it ignored the question and ran its normal installation steps.

An assistant trying to read about its own setup had just reconfigured the job that started the night shift. That job was subsequently found missing from launchd. The investigation identified the still-running overnight process as the likely obstacle to registering the job again. The error output was discarded, so the precise reason the reload failed remains unconfirmed.

That discarded output is the part I keep coming back to.

The installer already contained a check for whether launchd had accepted the job. If the check found it missing, the script was written to refuse success, give a command for investigating and exit with a failure status. I would have been pleased to see that error handling in a review.

But the assistant’s command included 2>/dev/null. Command-line programs have a separate output stream for errors, called stderr; that redirection throws it away. The | passes ordinary output to head -3, which keeps only the first three lines. The assistant had asked for a short preview, with error messages excluded.

Any complaint from the attempt to register the job, or from the installer’s own check, would disappear before the assistant could read it. I can’t say which complaint ran: the transcript never received it. The script contained a useful error path, and its caller had removed the channel intended to explain the failure.

All eighteen installation scripts in that directory ignored arguments. They were careful in other respects, including explicit paths and installation checks. Asking for help also looked like an ordinary way to learn about an unfamiliar program. The failure sat between what the caller expected and what the script actually did, with the explanation filtered out on the way back.

I eventually found the missing overnight reports while asking whether the assistants were hitting their session limits. I was trying to understand how much work they could do. The program that would have started them wasn’t being started itself.

A separate nightly health check should have helped here. It looked for expected scheduled jobs and complained if any were missing. Its list contained eleven jobs. I also had a daytime run that worked through the same task list, but neither that job nor the overnight one was on the health check’s list. The check could find everything it expected while the night shift was absent.

So I had two opportunities to learn what happened: the installation command’s error report and a separate check of the resulting state. The first was filtered out. The second didn’t know to look.

“It logs an error” now feels like an unfinished answer to me. Failure handling that depends on the caller happening to listen has left part of the system unbuilt. The caller’s output limits and redirections belong in the review alongside the message the script prints.

- The caller is part of the error path.

There is a small, harmless way to see how a caller can lose a failure. This example needs Bash and the standard head command. Its pretend installer only prints messages and returns a failure status; it writes no files and touches no scheduled jobs. Paste the whole block into a terminal:

bash --noprofile --norc <<'SH'
set +o pipefail
install_demo() {
  printf '%s\n' 'configuration written'
  printf '%s\n' 'job NOT loaded' >&2
  return 1
}

install_demo 2>/dev/null | head -3
printf 'pipeline status: %s\n' "$?"

if captured=$(install_demo 2>&1); then
  printf '%s\n' "$captured" | head -3
else
  rc=$?
  printf 'installer status: %s\n%s\n' "$rc" "$captured"
fi
SH

An exit status is the number a command returns: 0 conventionally means success, and a nonzero value means failure. The shell keeps the most recent status in $?. With pipefail disabled here, a pipeline (commands joined by |) reports the last command’s status. So the first call prints configuration written and reports status 0: head succeeded, even though the pretend installer failed.

The second call captures both output streams and checks the function’s own status before shortening anything. It reports status 1 and includes job NOT loaded. An unattended caller would still need to put that failure somewhere its operator checks. Preserving it makes that possible. These are the example’s shell settings; the historical caller’s settings aren’t established. Enabling pipefail can preserve a failure status, but it cannot recover error text already thrown away.

The repair to my installation scripts was small. A shared argument check now answers help requests before installation starts and rejects unknown arguments. A test asks every installer for help, then compares configuration-file sizes and timestamps and the set of registered jobs before and afterwards. The health check now includes the overnight and daytime jobs.

A manually maintained list can still miss the next job, though. And being registered to run doesn’t establish that a program finished its work. For these unattended runs, I want a check that asks whether the expected result arrived on time, running somewhere other than inside the worker whose silence it needs to notice.

I had put care into explaining failure. I hadn’t followed the explanation all the way to the person who needed it.