Skip to content

test: stop the suite from writing skill files into the real $HOME#527

Open
LucasRollo wants to merge 1 commit into
Panniantong:mainfrom
LucasRollo:fix/doctor-test-writes-to-real-home
Open

test: stop the suite from writing skill files into the real $HOME#527
LucasRollo wants to merge 1 commit into
Panniantong:mainfrom
LucasRollo:fix/doctor-test-writes-to-real-home

Conversation

@LucasRollo

Copy link
Copy Markdown

Problem

Running the test suite mutates the developer's real home directory.

tests/test_cli.py::test_doctor_runs invokes the real agent-reach doctor:

def test_doctor_runs(self, capsys):
    with patch("sys.argv", ["agent-reach", "doctor"]):
        main()
    ...

_cmd_doctor calls _install_skill(force=False) ("Auto-install skill if not already present"), which installs SKILL.md into every agent directory it finds on the machine — ~/.claude/skills/agent-reach/, ~/.agents/skills/agent-reach/, ~/.openclaw/skills/agent-reach/ — and Config() creates ~/.agent-reach/. The test patches only sys.argv, so all of that lands in the developer's real $HOME.

That means simply running pytest silently installs (or overwrites) skill files for the developer's live AI agents. A SKILL.md is standing instructions for an agent, so this isn't just stray files — it can change how the developer's Claude Code / OpenClaw behaves afterward. It's a surprising side effect, and the same pattern a malicious dependency could use to inject agent instructions via a test run.

The sibling test test_doctor_preserves_existing_skill_install already carefully sandboxes expanduser and the Config paths, so the need for isolation is understood — test_doctor_runs just lacked it, and nothing enforced it suite-wide.

Fix

Rather than patch a single test, isolate $HOME for the whole suite so no test can leak into the real home:

  • tests/conftest.py — autouse fixture redirecting HOME/USERPROFILE to a throwaway tmp dir. It also re-points Config.CONFIG_DIR / Config.CONFIG_FILE, which are class attributes bound to Path.home() at import time (before any fixture runs), so an env-var redirect alone wouldn't contain them.
  • tests/test_home_isolation.py — asserts the isolation is actually in effect (expanduser + Config resolve under the sandbox; doctor's writes stay under it). If the fixture is ever removed or stops redirecting $HOME, these fail instead of silently leaking again.

Verification

Full suite passes, and after running it the real ~/.claude/skills/agent-reach and ~/.agent-reach are no longer created (previously they were).

Note (out of scope for this PR)

Arguably a diagnostic command (doctor) shouldn't install skills into the system at all as a side effect — that's a surprising mutation from a read-only-sounding command. Happy to follow up separately if the maintainers want to reconsider that behavior; this PR just stops the test suite from being affected by it.

Running `pytest` mutated the developer's actual home directory.
`test_doctor_runs` calls the real `agent-reach doctor`, which as a side
effect auto-installs SKILL.md into every agent directory it finds
(~/.claude/skills, ~/.agents/skills, ~/.openclaw/skills) and creates
~/.agent-reach/ via Config(). The test patched nothing, so just running
the suite silently installed/overwrote skill files for the developer's
live AI agents — a surprising and potentially destructive side effect,
and the kind of thing a malicious dependency could weaponise.

- add tests/conftest.py: an autouse fixture redirecting HOME/USERPROFILE
  to a throwaway tmp dir, and re-pointing Config.CONFIG_DIR/CONFIG_FILE
  (class attributes bound to Path.home() at import time, before any
  fixture runs) at that dir. Contains every home-directed write for the
  whole suite, not just the one test that exposed it.
- add tests/test_home_isolation.py: asserts the isolation is in effect
  so the leak can't regress unnoticed — if the fixture stops redirecting
  HOME, expanduser("~") reverts to the real home and these fail.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant