Skip to content

fix(windows): slash-form logical paths and unix-scoped ownership tests - #548

Merged
David-Noble-at-work merged 6 commits into
developfrom
windows-separator-and-ownership
Aug 18, 2026
Merged

David-Noble-at-work merged 6 commits into
developfrom
windows-separator-and-ownership

Conversation

@David-Noble-at-work

@David-Noble-at-work David-Noble-at-work commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Windows baseline reduction, plus the CI fix this PR's own verification needed. The first pass
over-reached and CI caught it
; this carries the fixes that held, the reverts of the ones that did not, and
a workflow repair.

What CI said about the first attempt

Nine cleared, six regressed — net 28 → 25. Three changes were wrong, all the same way: slash form
applied to access rather than identity.

Over-reach How it broke
Join → slash builds a path for use; Join(a,b,c) gave a/b/c where the platform wants a\b\c
commonAncestor → slash its result becomes the run's confinement Root; fsroot panicked relating C:\… to /
identity URI → slash four consumers key on the native form, including drift detection, which stopped seeing a stale file

The third did not merely fail a comparison — it silently changed behavior. That is the "along the way"
fix #547 exists to prevent: the transform belongs at resource construction, with the resource owning
both forms.

What stands

Fix Why it is identity, not access
Provider.Name / Parent → slash they answer questions about a path as a value, on a projected Starlark surface
mem SourcePath test → slashpath asserted backslashes against a Rel() that is canonically slash-form by design
ownership tests → //go:build unix os.Getuid() returns -1 on Windows; the platform has no such concept
commonAncestor test → //go:build unix it feeds /home/user, a drive-relative string with no meaning on Windows
git argv assertions exec.Cmd.String() renders the resolved binary; a leading "git " matched only because /usr/bin/git ends with git, and .exe breaks it

And the CI repair (#439)

quality-gate hung 20+ minutes installing lint tools, blocking this PR's own measurement. Two defects:

  1. No timeouts on the curl — a stalled connection hangs forever. Retries handle failures, not hangs.
    Now --connect-timeout 15 --max-time 120.
  2. The retry loop had never retried. quality-gate declared no shell, so it ran bash -e {0} without
    pipefail
    . In curl … | sh, a failed curl left sh reading empty input and exiting 0 — the
    pipeline succeeded and && break fired on attempt one. The job now declares shell: bash like its
    test and scenario siblings.
  3. The other two network operations had neither. shfmt is pinned to v3.13.1 — @latest would
    change formatting verdicts with no commit on our side, and 3.13.1 is what developers run locally — and
    wrapped in timeout 180. apt gets Acquire::Retries=3 with 30s http/https timeouts, because a mirror
    that accepts the connection then stalls is the same hang, and a bare timeout would retry nothing.

The curl is the only pipeline in that job, so pipefail changes nothing else. Validated with actionlint,
which also shellchecks the run blocks.

Expected windows result

28 → 18, no regressions. Held loosely — the last confident number was wrong by six.

Verified

make check and make lint-all (linux, darwin, windows) both exit 0; the workflow validates under
actionlint.

David Noble added 2 commits August 18, 2026 00:54
Seven of the 28 windows failures were one defect at four sites: a path used as a VALUE was computed with
filepath, so a pure string operation answered differently per platform.

Provider.Name/Parent/Join answered a backslash for "/" on Windows. These are a projected Starlark surface,
where a path is a slash-form language on every platform -- the same contract as io/fs and the canonical
fsroot.Path rel form.

commonAncestor split on filepath.Separator, so two /home/user inputs produced a backslash-joined ancestor.

The file resource identity was "file://" + Abs(), putting backslashes INSIDE a URI. Identity travels into
serialized documents and the checksums over them, so this made identity platform-dependent -- the defect
#377 removed from Rel(), reintroduced one layer up. Now ToSlash.

The mem SourcePath test built its expectation with filepath.Join and asserted backslashes against a Rel()
that is canonically slash-form. That test was wrong, not the code.

The root cause -- no mechanism separating platform-neutral identity from platform-native access -- is filed
separately; this fixes the instances.
os.Getuid and os.Getgid return -1 on Windows, so tests that format them build the string "-1" --
resolveOwnership reads it as a numeric id and then rejects it, because both sides land on the -1
"leave unchanged" sentinel.

The failure is the platform having no such concept, not the code being wrong, so the two uid-dependent
tests move behind //go:build unix in helpers_unix_test.go, matching provider_unix_test.go. The rest of the
ownership tests are platform-neutral and stay.
@David-Noble-at-work David-Noble-at-work self-assigned this Aug 18, 2026
David Noble added 2 commits August 18, 2026 11:20
…t access

The first pass applied slash form to three things that are ACCESS, not identity, and windows CI caught all
three -- nine cleared, six regressed.

Join builds a path FOR USE and its result goes to the filesystem: Join(a,b,c) answered "a/b/c" where the
platform wants "a\b\c". Reverted to native, with the distinction from Name/Parent stated at the site --
those answer questions ABOUT a path as a value.

commonAncestor's result becomes the run's confinement Root, so it must be a path the platform can resolve.
A slash-form answer made fsroot panic trying to relate C:\... to /. Reverted to native; its test feeds Unix
absolute paths that mean nothing on Windows, so it moves behind //go:build unix -- the Windows behavior
that matters is covered by TestRegisterLayer_Link, which uses the result as a real root.

The file resource identity stays OS-native for now. Normalizing it at that one site fixed the identity
string and broke four consumers keyed on the native form -- including drift detection, which stopped seeing
a stale file. That is the "along the way" fix #547 exists to prevent; the transform belongs at construction
with the resource owning both forms.
exec.Cmd.String() renders the LookPath-RESOLVED executable, so the narration reads "/usr/bin/git -C ..." on
Unix and "C:\...\git.exe -C ..." on Windows.

Asserting a leading "git " only ever worked because "/usr/bin/git" ends with "git"; ".exe" breaks the same
substring. The narration is correct -- it shows what actually ran -- so the assertion moves to the argv
these tests are about.
David Noble added 2 commits August 18, 2026 11:45
quality-gate sat 20+ minutes in "Install lint tools" while every other leg finished. Two defects, and the
second is why the first was never survivable.

The curl had no timeouts, so a stalled connection hung indefinitely -- retries handle failures, not hangs.
It now carries --connect-timeout 15 --max-time 120, which turns a stall into a failure the loop can see.

And the loop could not have seen it anyway. quality-gate declared no shell, so it ran GitHub's default
`bash -e {0}` -- WITHOUT pipefail. In `curl ... | sh`, a failed curl left sh reading empty input and exiting
0, so the pipeline succeeded, `&& break` fired on the first attempt, and the three-attempt loop had never
once retried. The job now declares `shell: bash` like its test and scenario siblings, which supplies
pipefail.

The curl is the only pipeline in the job, so nothing else changes behavior under pipefail.

The other two network operations in the step had neither retries nor timeouts. shfmt is pinned to v3.13.1
-- @latest would change formatting verdicts with no commit on our side, and 3.13.1 is what developers run
locally -- and wrapped in `timeout 180`. apt gets Acquire::Retries=3 with 30s http and https timeouts,
because a mirror that accepts the connection and then stalls is the same hang as the curl, and a bare
`timeout` would retry nothing.

Validated with actionlint, which also shellchecks the run blocks.
TestWalkTree_ReducerError_StackHoldsCompletedReceipts built its map with
p.Name(receipt.Result().(Entry).Path().Abs()).

Abs() is OS-native. p.Name now speaks the slash-form language of the projected Starlark surface, so on
Windows it finds no separator and returns the WHOLE path -- the map came out keyed by absolute paths rather
than "a.txt" and "b.txt".

The test is not testing Name; it wants a file name from a native path, which is filepath.Base. It worked
before only because Name happened to be filepath.Base too.

This is the mismatch #547 exists to remove: a value crossing between the native and neutral worlds with
nothing to mark the boundary. Swept the other callers -- the provider's Name and Parent have four Go
callers, all feeding slash-form literals, so this was the only one.

Found by windows CI, which is the only place it can fail.
@David-Noble-at-work
David-Noble-at-work merged commit f8c9f75 into develop Aug 18, 2026
12 of 15 checks passed
@David-Noble-at-work
David-Noble-at-work deleted the windows-separator-and-ownership branch August 18, 2026 22:11
David-Noble-at-work pushed a commit that referenced this pull request Aug 28, 2026
… proves it

The Windows legs of this PR failed every knowledge-indexer test. The cause is
older than the PR, and this is its fourth appearance in seventeen days -- #395,
#548, #600, and now.

The provider speaks two path dialects. file.join, file.glob, file.resolve and
file.walk_tree build paths FOR USE, so they return OS-native strings. file.name
and file.parent answer questions ABOUT a path, so they are slash-form. On Windows
that means Name is handed a backslash path, slashpath.Base finds no separator in
it, and the whole path comes back as the "last element":

    file.name(file.join("knowledge", "packages", "slots"))
      darwin   -> "slots"
      windows  -> "knowledge\packages\slots"

The knowledge indexer therefore saw every asset directory on Windows as an
unrecognized type. The previous indexer made the same call and had the same bug;
it simply never ran there, because devlore-registry's workflow is ubuntu-only.

Name and Parent now convert with filepath.ToSlash before Base and Dir: native
input, slash-form output, which is what their documentation already claimed.
ToSlash rather than a blind ReplaceAll, and the distinction matters -- a
backslash is a legal character in a Unix filename, so it must not be read as a
separator off Windows.

WHY THIS KEEPS HAPPENING, and what stops it

Every existing assertion writes its input in the dialect of the function under
test. test_imm_file.star checks file.name against the literal "/some/dir/file.txt";
provider_test.go checks it against "/a/b/c.txt". Those are already slash-form, so
they behave identically on every platform and the mismatch between the dialects is
never exercised. Running the matrix on Windows buys nothing when the inputs cannot
vary by platform.

test_imm_file.star declines multi-part join explicitly: "it builds a path FOR USE,
so its result is OS-native and a fixture cannot assert it without encoding a
platform". True of join's RESULT -- but not of the composition. A fixture can
assert that taking the last element of a joined path returns the last element,
which names no separator and no platform, holds everywhere, and is false on
Windows whenever the seam is broken.

test_imm_file_path_seam.star does exactly that, over file.name(file.join(...)),
file.parent(file.join(...)), and file.name(<glob match>) -- the last being the
composition the indexer performs on every directory entry.

PROVEN ON THE HARDWARE, NOT ARGUED

Built for windows/arm64 and run on danoble-wd11-3. Without the fix:

    FAIL: equal("knowledge\packages\slots", "slots")
    FAIL: equal("a\b", "b")
    FAIL: equal(".", "knowledge/packages")
    FAIL: equal(["C:\...\seam_dir\slots"], ["slots"])

With it: PASS. The parent case is a second breakage nobody had noticed, for the
same reason -- nothing composed those two either.

scripts/Invoke-WindowsDevloreTest.sh is that loop, packaged: cross-compile here
(the box carries Go 1.26.7 against a go.mod needing 1.27), ship the binary and
data/, run over SSH. About a minute, against real Windows, instead of a fortnight
of waiting for a CI leg to notice.

One change was needed to make the binary portable: testdataDir resolved the
fixture directory through runtime.Caller, which bakes in the BUILD host's source
path. DEVLORE_TESTDATA overrides it.
David-Noble-at-work added a commit that referenced this pull request Aug 28, 2026
* fix(star): the knowledge indexer preserves what it did not write

star devlore knowledge index rebuilt each domain's index.yaml from a directory
listing and wrote {name: filename} per entry, discarding everything else. That
was invisible while devlore-registry's update-indexes workflow was failing. Its
first successful run after the unfreeze deleted the concepts section from two
domains, providers from a third, and every slot's package/description/platforms/
install_types -- an entire ollama slot with them.

It now UPDATES. Each existing entry is kept whole when its file is present; a new
file gets {name}. Entry metadata is hand-written, and tracking which files exist
is the only job this command has.

The rule the rest follows from: anything the indexer cannot classify is an ERROR,
not an omission. Its silence used to be indistinguishable from success -- the
output still parsed, so validate-yaml-schemas stayed green while content was
deleted. Five refusals now:

  - a directory that is not a known asset type
  - a file loose in a domain root, under no asset type
  - a section naming an unknown type
  - a section whose directory is gone
  - an indexed entry whose file is absent

The last is not pedantry. signatures/dotfile-systems.yaml is indexed while the
file sits at the domain root: it moved rather than vanished, and an indexer that
merely drops entries without files would have deleted it silently.

ASSET_TYPES grows from six to nine. concepts (migration, packages) and providers
(shared) are sections in use that the indexer did not recognize; bindings
(package-authoring) is a directory with three files that appears in no index at
all. cmd/devlore-index, the uninvoked Go tool that looked like the fix, carries
the same six in its struct and would have destroyed concepts and providers too.

Two things surfaced while building this, both the same shape as the bug:

yaml.parse returns a yaml.Resource whose only attributes are equal, resource_base
and validate -- the document's data is unreachable through it. The first version
of load_existing used it, silently received nothing, and looked like it worked
while treating every index as empty. yaml.decode returns a dict.

The audit's findings went to note() on stdout but never into the error, so a
caller seeing the failure without stdout was told something was wrong but not
what. They are in the error now.

Nine integration tests cover both halves: metadata survives, a new file is added
without disturbing its neighbours, a second run is byte-identical, and each of
the five refusals fires for its own reason. The refusal tests assert the reason
rather than merely that an error occurred -- asserting only err != nil passed on
an unrelated failure, which is the same indistinguishable-from-success problem
these tests exist to prevent.

Also: the package indexer stamped its output "generated: star devlore knowledge
index packages". It is the package indexer. Fixed.

Phase 2 of docs/plans/knowledge-index-preserves-metadata.md. The registry side --
the schema, resolving the five real problems, and re-enabling the workflow -- is
phase 3 and lands separately.

* fix(file): name and parent accept OS-native paths, and a fixture that proves it

The Windows legs of this PR failed every knowledge-indexer test. The cause is
older than the PR, and this is its fourth appearance in seventeen days -- #395,
#548, #600, and now.

The provider speaks two path dialects. file.join, file.glob, file.resolve and
file.walk_tree build paths FOR USE, so they return OS-native strings. file.name
and file.parent answer questions ABOUT a path, so they are slash-form. On Windows
that means Name is handed a backslash path, slashpath.Base finds no separator in
it, and the whole path comes back as the "last element":

    file.name(file.join("knowledge", "packages", "slots"))
      darwin   -> "slots"
      windows  -> "knowledge\packages\slots"

The knowledge indexer therefore saw every asset directory on Windows as an
unrecognized type. The previous indexer made the same call and had the same bug;
it simply never ran there, because devlore-registry's workflow is ubuntu-only.

Name and Parent now convert with filepath.ToSlash before Base and Dir: native
input, slash-form output, which is what their documentation already claimed.
ToSlash rather than a blind ReplaceAll, and the distinction matters -- a
backslash is a legal character in a Unix filename, so it must not be read as a
separator off Windows.

WHY THIS KEEPS HAPPENING, and what stops it

Every existing assertion writes its input in the dialect of the function under
test. test_imm_file.star checks file.name against the literal "/some/dir/file.txt";
provider_test.go checks it against "/a/b/c.txt". Those are already slash-form, so
they behave identically on every platform and the mismatch between the dialects is
never exercised. Running the matrix on Windows buys nothing when the inputs cannot
vary by platform.

test_imm_file.star declines multi-part join explicitly: "it builds a path FOR USE,
so its result is OS-native and a fixture cannot assert it without encoding a
platform". True of join's RESULT -- but not of the composition. A fixture can
assert that taking the last element of a joined path returns the last element,
which names no separator and no platform, holds everywhere, and is false on
Windows whenever the seam is broken.

test_imm_file_path_seam.star does exactly that, over file.name(file.join(...)),
file.parent(file.join(...)), and file.name(<glob match>) -- the last being the
composition the indexer performs on every directory entry.

PROVEN ON THE HARDWARE, NOT ARGUED

Built for windows/arm64 and run on danoble-wd11-3. Without the fix:

    FAIL: equal("knowledge\packages\slots", "slots")
    FAIL: equal("a\b", "b")
    FAIL: equal(".", "knowledge/packages")
    FAIL: equal(["C:\...\seam_dir\slots"], ["slots"])

With it: PASS. The parent case is a second breakage nobody had noticed, for the
same reason -- nothing composed those two either.

scripts/Invoke-WindowsDevloreTest.sh is that loop, packaged: cross-compile here
(the box carries Go 1.26.7 against a go.mod needing 1.27), ship the binary and
data/, run over SSH. About a minute, against real Windows, instead of a fortnight
of waiting for a CI leg to notice.

One change was needed to make the binary portable: testdataDir resolved the
fixture directory through runtime.Caller, which bakes in the BUILD host's source
path. DEVLORE_TESTDATA overrides it.

* test(file): state the parent assertion plainly, and drive Windows through bash

Three small corrections to the seam work.

The `and` in the file.parent case was a no-op. The expression evaluated to the
string literal whatever the left side returned, so it read as though it compared
two producer outputs while comparing nothing. Split into two direct assertions,
which also adds a second parent case. Both fail on unfixed Windows with got "."
-- slash-form Dir finds no separator in a backslash path and reports the current
directory -- for seven failures without the fix and PASS with it, verified on
danoble-wd11-3.

Invoke-WindowsDevloreTest.sh is formatted to shfmt -i 4 -ci, which is what
lint.shell runs. That was quality-gate's only complaint on the previous push;
every test leg passed, both Windows legs included.

The runner now drives the far side through Git Bash rather than PowerShell.
sshd's DefaultShell on that host is pwsh, so a bare ssh command lands there --
but ci.yaml sets `shell: bash` for the test job, and a local loop that uses a
different shell from CI reproduces the platform without reproducing the
environment. The defect this exists to catch is in Go path handling, where the
shell is irrelevant; anything quoting- or PATH-sensitive would have diverged, and
a green here would not have meant a green there.

It also drops two pwsh irritations: it splits `-test.run` at the dot unless every
argument is quoted, and it prints profile-not-found noise on each invocation.

---------

Co-authored-by: David Noble <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant