Repository navigation
Conversation
Bound search and ingestion embedding calls across batches and retries when configured. Signed-off-by: xp <[email protected]>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose of the change
Applications that retrieve memory before replying need to limit the time spent waiting for embeddings. An HTTP request timeout alone does not bound an embedding operation containing batches, SDK retries, and application retry backoff.
Description
Add optional
search_timeout_secondsandingest_timeout_secondsto OpenAI-compatible embedder configuration. Each applies to the entire public embedding call, including its batches, chunk requests, retries, and dimensions fallback. Expiry cancels pending work and raisesExternalServiceAPIErrorwith the timeout as its cause; caller cancellation still propagates and partial vectors are not returned.Both settings default to
Nonefor compatibility with existing deployments and slower local providers. The event-memory sample explicitly enables 5 seconds for search and 60 seconds for ingestion. SDK request timeouts and retry settings are unchanged. These are cooperative embedding-stage budgets, not deadlines for the full memory API.Move operation latency tracking to the public call boundary so expiry is recorded as an error. Batched calls now produce one operation observation covering all batches.
Fixes/Closes
Fixes #1771. Scope proposed in the issue discussion; the compatibility defaults have not yet received maintainer feedback.
Type of change
How Has This Been Tested?
On Windows, Python 3.12, with no paid model calls:
74 passed, 3 skipped, 27 deselected; three existing testcontainers deprecation warnings. Tests cover cancellation of pending batches/clusters, partial-result rejection, retry/backoff, dimensions fallback, concurrent search/ingestion, caller cancellation, configuration round-tripping and manager wiring, and timeout metrics. A real
AsyncOpenAIclient withhttpx.MockTransportexercises SDK retry cancellation without network access.ruff check,ruff format --check, andgit diff --checkpassed. Targetedtychecks passed for the embedder, configuration, and changed tests. Checking the manager reports missing optionalsentence_transformers, also reproduced on the base version. Fullty check packagesreports 25 diagnostics from missing optional packages and Windowstime.tzset; it did not pass in this environment. Live-provider/database integration tests were not run.Checklist