Skip to content

Allow same root_fs for different jobs: sinks and so on - #752

Merged
problame merged 5 commits into
zrepl:masterfrom
dsh2dsh:fix-overlapping-root-fs
Oct 31, 2023
Merged

problame merged 5 commits into
zrepl:masterfrom
dsh2dsh:fix-overlapping-root-fs

Conversation

@dsh2dsh

@dsh2dsh dsh2dsh commented Oct 19, 2023 •

Copy link
Copy Markdown
Contributor

Because some jobs add client identity to root_fs and other jobs don't do that,
we can't reliable detect overlapping of filesystems. And and the same time we
need an ability to use equal or overlapped root_fs for different jobs. For
instance see this config:

  - name: "zdisk"
    type: "sink"
    root_fs: "zdisk/zrepl"
    serve:
      type: "local"
      listener_name: "zdisk"

and

  - name: "remote-to-zdisk"
    type: "pull"
    connect:
      type: "tls"
    root_fs: "zdisk/zrepl/remote"

As you can see, two jobs have overlapped root_fs, but actually datasets are not
overlapped, because job zdisk save everything under zdisk/zrepl/localhost,
because it adds client identity. So they actually use two different filesystems:
zdisk/zrepl/localhost and zdisk/zrepl/remote. And we can't detect this
situation during config check. So let's just remove this check, because it's
admin's duty to configure correct root_fs's.

The idea of this check is
```
// idea:
//   no path in rfss must be prefix of another
```

Technically, with two equal root_fs, one of them is really prefix of other. But
logically, and I think we need exactly this, a prefix is a part of some string.
So two equal root_fs are not a prefix of each other.

And I don't see any reason why we can't have jobs with same root_fs. Anyway we
are root_fs + client's name, so names of datasets never are never overlapping.
It isn't compatible with pull jobs, because puul jobs don't add client
identities to their root_fs.

This reverts commit 3a4d546.
Because some jobs add client identity to root_fs and other jobs don't do that,
we can't reliable detect overlapping of filesystems. And and the same time we
need an ability to use equal or overlapped root_fs for different jobs. For
instance see this config:

```
  - name: "zdisk"
    type: "sink"
    root_fs: "zdisk/zrepl"
    serve:
      type: "local"
      listener_name: "zdisk"
```
and
```
  - name: "remote-to-zdisk"
    type: "pull"
    connect:
      type: "tls"
    root_fs: "zdisk/zrepl/remote"
```

As you can see, two jobs have overlapped root_fs, but actually datasets are not
overlapped, because job `zdisk` save everything under `zdisk/zrepl/localhost`,
because it adds client identity. So they actually use two different filesystems:
`zdisk/zrepl/localhost` and `zdisk/zrepl/remote`. And we can't detect this
situation during config check. So let's just remove this check, because it's
admin's duty to configure correct root_fs's.
@problame
problame disabled auto-merge October 31, 2023 23:05
@problame

Copy link
Copy Markdown
Member

Will merge this after #763

@problame
problame merged commit 27012e5 into zrepl:master Oct 31, 2023
@dsh2dsh
dsh2dsh deleted the fix-overlapping-root-fs branch November 1, 2023 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants