[Feat]: Server tech debt resolution wishlist #1297

❯ Rate horizontal scalability of each component.

⏺ Based on what I've read in the codebase:

  ┌──────────────────────────────────┬────────┬───────────────────────────────────────────────────────────────────────────────────┐
  │            Component             │ Rating │                                    Bottleneck                                     │
  ├──────────────────────────────────┼────────┼───────────────────────────────────────────────────────────────────────────────────┤
  │ MemMachine                       │ B      │ Mostly stateless orchestrator, but holds a singleton _retrieval_agent with a      │
  │                                  │        │ process-local lock. Multiple instances work, minor duplication.                   │
  ├──────────────────────────────────┼────────┼───────────────────────────────────────────────────────────────────────────────────┤
  │ EpisodeStorage                   │ B+     │ Postgres via SQLAlchemy. Reads scale with replicas. Writes are single-primary.    │
  │                                  │        │ Session-scoped data is naturally shardable.                                       │
  ├──────────────────────────────────┼────────┼───────────────────────────────────────────────────────────────────────────────────┤
  │ SessionDataManager               │ B+     │ Same as EpisodeStorage — Postgres-backed, shardable by session_key.               │
  ├──────────────────────────────────┼────────┼───────────────────────────────────────────────────────────────────────────────────┤
  │                                  │        │ Process-local LRU cache with ref counting. No cross-instance cache coherence.     │
  │ EpisodicMemoryManager            │ D      │ Each server instance builds its own cache, wastes memory, risks stale state. This │
  │                                  │        │  is the main scaling bottleneck.                                                  │
  ├──────────────────────────────────┼────────┼───────────────────────────────────────────────────────────────────────────────────┤
  │ ShortTermMemory                  │ C      │ Summary state persisted in DB (good), but the in-memory message buffer is         │
  │                                  │        │ process-local. Requires the same instance to serve a session's lifetime.          │
  ├──────────────────────────────────┼────────┼───────────────────────────────────────────────────────────────────────────────────┤
  │ LongTermMemory/DeclarativeMemory │ A-     │ Stateless facade. All state in external VectorGraphStore. Scales as well as the   │
  │                                  │        │ backing store does.                                                               │
  ├──────────────────────────────────┼────────┼───────────────────────────────────────────────────────────────────────────────────┤
  │ EventMemory                      │ A-     │ Stateless. All state in external SegmentStore + vector Collection. Same story.    │
  ├──────────────────────────────────┼────────┼───────────────────────────────────────────────────────────────────────────────────┤
  │ SemanticService                  │ A-     │ Stateless compute (LLM calls). Backed by external SemanticStorage                 │
  │                                  │        │ (Neo4j/PgVector). Scales well.                                                    │
  ├──────────────────────────────────┼────────┼───────────────────────────────────────────────────────────────────────────────────┤
  │ RetrievalAgent                   │ A      │ Stateless LLM-based query refinement. Fully parallelizable.                       │
  └──────────────────────────────────┴────────┴───────────────────────────────────────────────────────────────────────────────────┘

❯ Rate the configuration system.

⏺ Honestly? It's overcomplicated.

  What it does well:
  - Session-scoped config stored as JSON in Postgres — durable, queryable
  - Resource IDs (string → object) via resource manager — clean indirection

  What's unnecessarily complex:

  1. Partial/Full config duplication — every config has two classes (LongTermMemoryConf + LongTermMemoryConfPartial,
  ShortTermMemoryConf + ShortTermMemoryConfPartial, EpisodicMemoryConf + EpisodicMemoryConfPartial). The partial exists only for the
  merge system.
  2. The merge system — EpisodicMemoryConfPartial.merge() merges user-provided config with server defaults. It has nested merge
  logic, null coalescing, special cases for session_key/session_id propagation. This is a lot of machinery to do what Pydantic's
  model_copy(update=...) or a simple dict merge would handle.
  3. Config flows through too many layers — Configuration.episodic_memory (partial) → MemMachine._with_default_episodic_memory_conf()
   (merges) → EpisodicMemoryConf (full) → stored in param_data → loaded back → service_locator → Params → constructor. That's 6+
  hops.
  4. _enabled flags alongside nullable fields — long_term_memory: LongTermMemoryConf | None AND long_term_memory_enabled: bool. The
  nullability already signals "not configured"; the bool adds a separate "configured but disabled" state that complicates every
  check.