What happened
Nothing in the Qdrant integration ever reclaims a resource, and two kinds of garbage accumulate permanently.
Native collections. The native name is {namespace}__{sha256(config)}, so a new collection appears for every distinct config, including every revision of the server-global properties_schema. They are never deleted, nothing enumerates them, and nothing can tell whether one is still referenced. Qdrant Cloud caps a cluster at 1000 collections.
Records under dead partition keys. After delete_collection, records written through a handle held across the deletion land under the old partition key. With #1527 they are correctly invisible and can never be resurrected — the code comment says so — but they are also unreclaimable, because nothing records that the partition key ever existed. Payload values cannot be enumerated exhaustively either: facet returns top-N approximate counts, not a listing.
A failed creation leaves a third, milder kind: an empty native collection, and in distributed mode an unused shard key. That one is self-healing when the same config recurs, since creation is idempotent and content-addressed.
Expected
Every resource that exists should be either referenced by a live registry entry or recorded somewhere that a reclaimer can find it. Leakage should be bounded and, ideally, actively reclaimed.
Suggested direction
Two mechanisms, because the two kinds of garbage are discoverable in different ways:
- Container-level garbage is enumerable — Qdrant can list its collections — so a periodic reconcile that removes collections matching no live entry handles it, with a grace period so an in-flight creation is not mistaken for garbage.
- Record-level garbage is not discoverable at any price, so it has to be written down when it is created: a queue row committed in the same transaction as the registry deregistration, drained by a purger that filter-deletes by partition key. That also makes
delete_collection O(1) for the caller instead of O(rows), which matters since deleting a large tenant is proportional to its record count.
The reclaim verb belongs on the store, since only it knows what deleting means for its backend; the queue belongs with the registry, which already has the transaction.
Notes
Not addressed by any open PR. #1526's design doc records the absence of enumeration as a dated omission.
What happened
Nothing in the Qdrant integration ever reclaims a resource, and two kinds of garbage accumulate permanently.
Native collections. The native name is
{namespace}__{sha256(config)}, so a new collection appears for every distinct config, including every revision of the server-globalproperties_schema. They are never deleted, nothing enumerates them, and nothing can tell whether one is still referenced. Qdrant Cloud caps a cluster at 1000 collections.Records under dead partition keys. After
delete_collection, records written through a handle held across the deletion land under the old partition key. With #1527 they are correctly invisible and can never be resurrected — the code comment says so — but they are also unreclaimable, because nothing records that the partition key ever existed. Payload values cannot be enumerated exhaustively either:facetreturns top-N approximate counts, not a listing.A failed creation leaves a third, milder kind: an empty native collection, and in distributed mode an unused shard key. That one is self-healing when the same config recurs, since creation is idempotent and content-addressed.
Expected
Every resource that exists should be either referenced by a live registry entry or recorded somewhere that a reclaimer can find it. Leakage should be bounded and, ideally, actively reclaimed.
Suggested direction
Two mechanisms, because the two kinds of garbage are discoverable in different ways:
delete_collectionO(1) for the caller instead of O(rows), which matters since deleting a large tenant is proportional to its record count.The reclaim verb belongs on the store, since only it knows what deleting means for its backend; the queue belongs with the registry, which already has the transaction.
Notes
Not addressed by any open PR. #1526's design doc records the absence of enumeration as a dated omission.