Skip to content

One selector API: writ and lore detect the host and match platform selectors through the same code, along os-release's lineage #944

Closed

Description

@David-Noble-at-work

writ and lore each have their own selector grammar, and they disagree. Neither follows a distribution's lineage. This issue is the one selector API, and it absorbs #860, the lineage chain (merged 2026-09-28, ruled "Yes, merge them").

Today

Verified against the code at 9cd1e95d on 2026-09-28. The full findings, with path:line, are in this comment. On an Ubuntu host:

Unix Linux Debian Ubuntu
writ ✔ ✔ ✘ ✘ (#959: detect.go:130 closes os-release before reading it)
lore ✔ ✔ ✔ only as Linux.Debian, from a hard-coded family list ✘ (no Ubuntu selector exists)
  • Two os-release readers. writ's cmd/writ/writ/segment/detect.go and pkg/platform, which lore uses. Neither reads ID_LIKE, and pkg/platform refuses any ID outside its ten.
  • Three spellings. writ's suffix Debian, lore's token Linux.Debian, and Starlark's ubuntu.
  • writ's precedence ranks by suffix count, so ties break by lexical order and Unix wins.

The rulings

  • 2026-09-07, the lineage (was The Linux suffix chain follows os-release's lineage: what deploys to Debian deploys to Ubuntu, what deploys to Fedora deploys to RHEL #860): "Debian is the base for Ubuntu so what deploys to Debian should deploy to Ubuntu. Similarly, Fedora is the base for RHEL so what deploys to Fedora should deploy to RHEL."
  • 2026-09-24, one API: there is one selector grammar and one API for it. Every tool calls it: writ's layer matching, lore's script-directory lookup and release selection, and anything else that renders, parses, expands or matches a platform selector. No tool keeps its own os-release reader or its own code for capitalizing OS names.
  • 2026-09-28, the owner's rule: "if i'm on an unbuntu system, i will match linux, debian, and unbuntu. rationale: ubuntu is like debian, and both are linux platforms."
  • 2026-09-28, precedence: "the most specific link in the chain wins."
  • 2026-09-28, grammar: "everyone knows, for example, the lineage of Debian. there is no need to force folks to spell that out completely. the .Debian form becomes the permanent form." And, for every distribution: ".Debian is just one example, and assuming I can say the same about any distro, we are good."

The rule

The host's chain is most general first, and follows the lineage the distribution states in /etc/os-release:

Unix → Linux → <ID_LIKE reversed> → <ID>
Host ID ID_LIKE Chain after Linux
Debian debian — Debian
Ubuntu ubuntu debian Debian → Ubuntu
Linux Mint linuxmint ubuntu debian Debian → Ubuntu → Mint
Fedora fedora — Fedora
RHEL rhel fedora Fedora → RHEL
CentOS Stream centos rhel fedora Fedora → RHEL → CentOS
Rocky rocky rhel centos fedora Fedora → CentOS → RHEL → Rocky
no os-release — — nothing: the chain stops at Linux
  • A selector word names one link (Debian, Ubuntu, Fedora), for every distribution. Dots separate independent dimensions (common.Debian.arm64), not a path through the lineage. lore's Linux.Debian and Linux.Fedora directories become Debian and Fedora.
  • Every link contributes. For one file at one target the most specific link wins: <ID> beats each ID_LIKE ancestor, closest first, which beats Linux, which beats Unix.
  • Manifests union across the chain (packages-manifest.yaml files collide like deployed files: the most specific one shadows the rest, so a layer's general packages are never planned #814).
  • Detection falls back along ID_LIKE. A distribution pkg/platform doesn't list, such as Pop!_OS (ID=pop, ID_LIKE="ubuntu debian"), takes its package manager from its closest listed ancestor rather than failing.
  • Open for the design: the spelling of an ID as a selector word. That covers the capitalization table (rhel → RHEL), IDs with hyphens (opensuse-leap), and an unknown ID, which gets its first letter uppercased as today.

The os-release facts behind this are in this comment.

Acceptance

  • One package detects the host and produces the chain; writ's segment.DetectSegments and lore's token both call it, and detect.go's own reader and capitalization table are gone
  • A unit test per row of the table above, each driven by a synthetic os-release
  • writ ranks matched directories by chain depth; a test pins Ubuntu > Debian > Linux > Unix for one file
  • lore's phase-script chain is the host's chain; Linux.Debian and Linux.Fedora script directories are renamed in the registry and the tests
  • The tests that pin today's behavior are corrected: segment_test.go:107 (Debian must match on Ubuntu) and adopt_integration_test.go:341-343 (Ubuntu is a valid platform)
  • The layer-journey scenario's step 2.3b deploys common.Debian on ubuntu-latest and ubuntu-24.04-arm. It skips by issueLineageChain = 860 (cmd/writ/scenario_layer_journey_test.go:80) until then, and that constant becomes this issue's number
  • docs/guides/writ/platform-awareness.md states the chain, the precedence and the grammar, and loses its uname and "matched in order" claims; the lore guide renames its selectors; the design record (The writ design record states the layer overlay, the implicit common project, the suffix chain, and the manifest union in one place #849) carries the chain beside the suffix grammar

Related

#959 (the file-closing bug; it lands first), #968 (writ's nil-platform panic on an unknown distribution), #969 (lore's unread platform declarations), #970 (segments that upgrade and reconcile don't see), #369 (the suffix grammar: order, dimensions, ties), #551 (the same principle for package-manager vocabulary), #814 (manifests union across the chain), #855 (where the lineage gap was measured).

Plan and design documents

Kind Document
Plan docs/plans/feature/944-one-selector-api-writ-and-lore.md
Design docs/guides/selectors.md

Epic: #447 · Feature: #463

Activity

  1. added
    taskImplementation work item
    Epic:WritDeploymentwrit deployment: layers, adoption, and drift
    and removed
    Epic:WritDeploymentwrit deployment: layers, adoption, and drift
    on Sep 24, 2026
  2. David-Noble-at-work commented on Sep 28, 2026

    @David-Noble-at-work
    ContributorAuthor

    Design input (2026-09-28): what the one selector API reads on Linux, and the matching the owner wants. The owner: "i want all this info in the design." When this issue's branch opens, the design document carries what follows.

    The owner's rule

    In the owner's words: "It should be true that if i'm on an unbuntu system, i will match linux, debian, and unbuntu. rationale: ubuntu is like debian, and both are linux platforms."

    /etc/os-release: the source

    • Its origin and reach. It's the freedesktop.org os-release specification (systemd, 2012), and virtually every current distribution ships it, including those without systemd: Debian, Ubuntu, Fedora, RHEL and its rebuilds, SUSE, Arch, Alpine, Gentoo and Void.

    • Where it lives. A reader takes /etc/os-release, and falls back to /usr/lib/os-release only when the first is missing. On most systems /etc/os-release is a symlink to the second.

    • ID is always present: lowercase, such as ubuntu, debian or rhel, defaulting to linux when absent.

    • ID_LIKE is optional: a space-separated list of related distributions, ordered from closest to most distant. It's the distribution's own statement of lineage, exactly the relation in the owner's rule:

      Distribution ID ID_LIKE
      Ubuntu ubuntu debian
      Linux Mint linuxmint ubuntu debian
      Rocky Linux rocky rhel centos fedora
      Debian, Fedora, Arch debian, fedora, arch none: they're the roots
    • VERSION_ID and VERSION_CODENAME are also optional.

    • Where it's missing: distributions from before about 2012 (RHEL/CentOS 6), some minimal containers (busybox, scratch), Android and some embedded systems. A detector treats a missing file as "Linux, distribution unknown", not as an error.

    Seen on DANOBLE-UD24-1: ID=ubuntu, ID_LIKE=debian, VERSION_ID="26.04", VERSION_CODENAME=resolute.

    To follow

    What writ and lore do today, checked against the code at 9cd1e95d rather than the table above, which dates from 2026-09-24. Then the matching this implies.

    The candidate homes for the design, for the owner to choose when the plan is written:

    • a new section of docs/architecture/3.4-platform-package-managers.md, beside "The platform capability — platform.Platform";
    • a document of its own under docs/architecture/.

    docs/guides/writ/platform-awareness.md (§Segment detection, §OS family matching) is the user-facing half, and changes with it.

  3. David-Noble-at-work commented on Sep 28, 2026

    @David-Noble-at-work
    ContributorAuthor

    Design input (2026-09-28): what writ and lore actually do today. Checked against the code at 9cd1e95d by two readers, each re-checked line by line by an adversarial verifier, with probes run on DANOBLE-UD24-1 (Ubuntu 26.04, ID=ubuntu, ID_LIKE=debian, aarch64). This supersedes the "Today" table in the issue body.

    The owner's rule against today

    "If i'm on an unbuntu system, i will match linux, debian, and unbuntu."

    On this Ubuntu host Unix Linux Debian Ubuntu
    writ today ✔ ✔ ✘ ✘ (#959)
    writ with only #959 fixed ✔ ✔ ✘ (nothing reads ID_LIKE) ✔
    lore today ✔ ✔ ✔ as Linux.Debian, through a hard-coded list, not ID_LIKE ✘ (no Ubuntu selector exists)
    #860's ruled chain ✔ ✔ ✔ ✔

    Neither tool meets the rule. #860's chain, Unix → Linux → <ID_LIKE reversed> → <ID> (ruled 2026-09-07, "Debian is the base for Ubuntu so what deploys to Debian should deploy to Ubuntu"), meets it exactly. Nothing implements it; it exists only in a scenario test that skips by #860's number (cmd/writ/scenario_layer_journey_test.go:466-504, :973-986).

    Two os-release readers, neither reads ID_LIKE

    • writ, cmd/writ/writ/segment/detect.go:119-172. Reads ID only and capitalizes it through its own table (Ubuntu, RHEL, CentOS, OpenSUSE; otherwise it uppercases the first letter). :130 closes the file before scanning it (writ never detects the Linux distribution: detectDistro closes /etc/os-release before reading it, so no .Debian or .Ubuntu directory deploys anywhere #959, a regression from 57a9fd20), so DISTRO is "" on every Linux host: a probe here gives [Linux Unix arm64]. ARCH is Go's GOARCH (arm64, amd64), not uname -m (aarch64).
    • pkg/platform, detect_linux.go:92-119. lore uses it, and so does writ for package managers. Reads ID, VERSION_ID and VARIANT_ID, lowercase, with the aliases linuxmint→mint and centos→centos-stream. It refuses any ID outside ten (defaults.go:31-42), so a derivative such as Pop!_OS (ID=pop, ID_LIKE="ubuntu debian") fails detection outright. The ten are listed in three places that must agree: defaults.go:31-42, platform.go:39-53 (knownLinuxDistros) and token.go:30-33.

    Three grammars

    • writ splits a directory name on ., and every suffix must be one of the host's values: an unordered AND over a flat set (segment.go:84-108), case-sensitive. Linux.Debian means OS=Linux AND DISTRO=Debian.
    • lore treats a selector as one whole token naming one directory, <pkg>/<Selector>/<Action>/<phase>.star, over a fixed chain Common → Unix → Linux → <token> (cmd/internal/lorepackage/lifecycle.go:160-181). The token groups by package-manager family (pkg/platform/token.go:21-40): debian and ubuntu give Linux.Debian; fedora, rhel, centos, rocky and alma give Linux.Fedora; anything else gives Linux.
    • Starlark scripts see a third form: platform.distro is the raw lowercase ubuntu, and platform.version is 26.04 (pkg/op/provider/platform/provider.go:47-58).

    The comments at token.go:11-13 and cmd/lore/lore/builder.go:658-659 claim writ shares lore's vocabulary; it doesn't. #931 ruled that writ adopt --platform uses writ's segment vocabulary, not lore's token (cmd/writ/writ/adopt/batch.go:332-335).

    Precedence has no rule for the chain

    • writ ranks a directory by its suffix count (segment.go:119-121), so common.Unix, common.Linux, common.Debian and common.Ubuntu all tie at 1.
    • Ties fall to an unstable sort.Slice and last-walked-wins (matcher.go:61-66, tree/builder.go:271-280, :357-361). With 12 or fewer matches that means lexical order, where the last name wins. So today common.Unix beats common.Linux on a colliding file, and with lineage Unix would beat Ubuntu, the reverse of the chain.
    • The design must rank by chain depth, most specific (ID) winning.

    Tests that pin the wrong behavior

    Documents that contradict the code

    • docs/guides/writ/platform-awareness.md:
      • :25 and :33 say Linux.Debian deploys on Debian and Ubuntu. It deploys on neither today, and on Ubuntu never, by design and by test.
      • :42 and :44 say detection uses uname -s and uname -m. It uses GOOS and GOARCH.
      • :46 says segments match "in order". Matching is unordered.
    • cmd/writ/writ/segment/segment.go:14 gives a lowercase debian as its example; detected values are capitalized.
    • docs/plans/writ-deploy-scenario.md:73-80 records DISTRO as ID only.

    Dead or duplicate selector code the one API replaces

    • writ's MatchAllProjects, GroupByProject and ProjectNames (matcher.go:71-140) have no callers, and SetValues and UndefinedSegmentError are used only by tests.
    • lore's GetPhaseScript (lifecycle.go:227-242) has no callers.
    • A symlinked variant directory (such as common.Ubuntu → common.Debian) is never matched (matcher.go:30-32), so symlinks are no workaround.

    Workarounds that exist today, for deploy only

    • WRIT_SEGMENT_DISTRO=Ubuntu restores Ubuntu.
    • writ deploy -s LIKE=Debian appends a custom segment, which makes Debian match.
    • A probe with both makes common.Debian, common.Linux.Debian and common.Debian.Ubuntu all match.
    • Neither reaches writ adopt --platform. writ upgrade doesn't take -s at all.

    Filed separately (not selector design)

    • A nil-platform panic in writ's manifest merge on an unrecognized distribution: pkg/op/runtime_environment.go:223-227 swallows the detection failure, and pkg/platform/identifier.go:58 calls a method on the nil platform.
    • lore's declared-but-unread platform surfaces: the lifecycle schema is never enforced, platforms: is never read, lore search --platform is ignored, and the guide documents flags that don't exist.
    • writ upgrade doesn't reproduce deploy -s segments, and reconcile's Config.Segments is never read.

    The issue numbers follow in a comment below.

  4. David-Noble-at-work commented on Sep 28, 2026

    @David-Noble-at-work
    ContributorAuthor

    Filed separately, from the findings above: #968 (writ deploy panics on a distribution pkg/platform does not know), #969 (lore's platform declarations are never read), #970 (writ upgrade and reconcile do not see the segments deploy was given). The test that pins #959 is noted on #959.

  5. David-Noble-at-work commented on Sep 28, 2026

    @David-Noble-at-work
    ContributorAuthor

    Ruled 2026-09-28: precedence follows the chain. In the owner's words: "the most specific link in the chain wins." For a file present in several matching directories, <ID> beats each ID_LIKE ancestor (closest first), which beats Linux, which beats Unix: on Ubuntu, Ubuntu > Debian > Linux > Unix. That replaces today's rank by suffix count, whose ties fall to lexical order (Unix wins).

  6. David-Noble-at-work commented on Sep 28, 2026

    @David-Noble-at-work
    ContributorAuthor

    Ruled 2026-09-28: one grammar, a selector names one link. In the owner's words: "everyone knows, for example, the lineage of Debian. there is no need to force folks to spell that out completely. the .Debian form becomes the permanent form." Every selector word is a single link of the chain (Unix, Linux, an ID_LIKE member, the ID), and the chain supplies the lineage. Dots separate independent dimensions (common.Debian.arm64: Debian lineage AND arm64), not a path through the lineage. lore's script directories Linux.Debian and Linux.Fedora become Debian and Fedora, and Ubuntu, RHEL and the rest become possible. In writ, common.Linux.Debian still matches but is redundant. The interim .Debian of 2026-09-24 is permanent.

  7. changed the title [-]One selector API: writ and lore read and match platform selectors through the same code[/-] [+]One selector API: writ and lore detect the host and match platform selectors through the same code, along os-release's lineage[/+] on Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Epic:WritDeploymentwrit deployment: layers, adoption, and driftThread:WritDeploydeploying converges the machinetaskImplementation work item

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions