MemAdapter

Official implementation of MemAdapter: Counterfactual Adaptation Against Memory-induced Sycophancy. MemAdapter evaluates how retrieved memories are used during language-model generation, with shared retrieval, generation, and judging pipelines for memory-system comparisons.

A retrieved memory may be useful, stale, too broad, or irrelevant to the current request. MemAdapter handles this in three stages:

1. Counterfactual boundary induction identifies when each retrieved memory should and should not influence a response.
2. Context-aware reflection turns those boundaries into task-specific instructions for the current request.
3. Memory-use-guided generation produces the final response using the validated instructions.

The repository supports A-MEM, Mem0, naiveRAG, MemoryBank, and LightMem. It also includes direct generation and four post-retrieval comparison methods: Anti-Sycophancy, Self-ReCheck, Dynamic Partition, and MemGate.

external_benchmarks/       Retrieval, evaluation protocol, judges, and validation
  launcher/                Retrieval and generation/judging launchers
  judges/                  Dataset-specific structured judge parsers
  rubrics/                 Task-specific prompt templates
generation/systems/        Experiment matrix for each memory system
methods/
  memadapter/              Three-stage method, model clients, and ablations
  baseline/                Direct generation baseline
  shared/                  Post-retrieval comparison methods
config/models.example.env  Local configuration template

external_benchmarks/memory_systems/ contains the bundled retrieval adapters for MemoryBank and LightMem. Per-system datasets, backbones, and ablations are recorded in generation/systems/<system>/protocol.json.

python -m pip install -r requirements.txt

Copy config/models.example.env to a local environment file, or export its values through your shell or secret manager. For local Qwen3-8B inference, set QWEN_LOCAL_MODEL_PATH to the model directory. Obtain each benchmark from its official distribution and provide its local location through the documented environment variables or command-line arguments.

Inspect the available options for the main components:

python methods/memadapter/run_memadapter.py --help
python -m external_benchmarks.build_retrieval --help
python -m external_benchmarks.run_external_baseline --help
python -m external_benchmarks.run_external_judge --help

The pipeline runs in four steps:

1. Build and freeze retrieval for each (dataset, task, memory system) cell.
2. Run Baseline, MemAdapter, or a post-retrieval comparison method on that retrieval file.
3. Run the task-specific judge.
4. Validate coverage and judge parsing.

Each experimental cell uses one frozen retrieval record, shared by every compared generation method. Methods receive the current request and the frozen memories; they do not receive dialogue context, query-session history, or task evidence.

Within a setting, all methods use the same generation backbone and decoding configuration:

Judges use deterministic decoding and task-specific rubrics. Metrics are computed from their structured outputs. Retrieval embeddings use BGE-M3.

Two ablations are included:

Use methods/memadapter/ablations/run_ablation.ps1 for one ablation cell, or the batch launchers in the same directory for all registered systems.

@article{ningmemadapter,
  title={Memadapter: Counterfactual Adaptation Against Memory-induced Sycophancy},
  author={Ning, Ruqing and Meng, Haibo and Xiang, Zhishang and Chen, Zerui and Su, Jinsong and Wang, Xin and Zhang, Qinggang},
  journal={arXiv preprint arXiv:2610.05162},
  year={2026}
}