Skip to content

updatedAt filter over list servers endpoint #291

Description

@domdomegg

From #50 (comment)

@domdomegg:

Thinking about this (and the data replication model in general), maybe it would be useful to have an endpoint like:

/v0/servers?changedSince=2025-08-07T13:15:04.280Z (or maybe just takes unix timestamp in seconds or something)

That would basically just do a filter over updatedAt. Then package registries would only fetch what they need to update.

And malicious packages would be kept in the registry, but marked as 'malicious/archived' or something (maybe also important to make sure they don't get resubmitted!).

@.tadasant:

This makes sense to me!

And malicious packages would be kept in the registry, but marked as 'malicious/archived' or something (maybe also important to make sure they don't get resubmitted!).

I think it'd be reasonable to set their status as such and basically exclude them from results unless some filter to explicitly include their status is invoked.

Related: #135

Activity

  1. domdomegg commented on Aug 21, 2025

    @domdomegg
    MemberAuthor

    Is this a go-live blocker?

    Not strictly: downstream registries could iterate through everything.

    As someone planning to also build a downstream registry, I do think this would make it a lot more efficient though so I'd like it relatively soon! It would also mean we're able to scan more frequently, which means servers percolate through faster (which I think is good for new servers being able to be distributed quickly, and malicious/dangerous servers being removed quickly)

  2. changed the title [-]updatedAt filter over servers?[/-] [+]updatedAt filter over list servers endpoint[/+] on Aug 21, 2025
  3. domdomegg commented on Aug 21, 2025

    @domdomegg
    MemberAuthor

    Similar to #135, I would propose that if we did this:

    For now, keep it out of the registry API spec, so that...

    • subregistries don't need to implement it
    • easier to make breaking changes/change how search works in future to support more capable forms of search

    I suspect we would want it to be supported in the registry API spec eventually given the tree nature of subregistries, and this being pretty key to being able to pull from higher registries.

  4. tadasant commented on Aug 21, 2025

    @tadasant
    Member

    This all makes sense to me 👍

  5. self-assigned this
    on Sep 1, 2025
  6. added a commit that references this issue on Sep 8, 2025
    30c37ef
  7. added a commit that references this issue on Sep 8, 2025
    a85456a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions