MobQA: A Benchmark Dataset for Semantic Understanding of Human Mobility Data through Question Answering

This repository provides tools and data for building a mobility question answering dataset. The dataset combines trajectory data with question-answer pairs covering factual retrieval, multiple choice, and free-form questions about mobility patterns and behaviors.

• macOS: This project has been tested and confirmed to work on macOS
• uv: Required for Python package management and environment setup
• OpenAI API: Required when using OpenAI models
• Google Cloud Platform (GCP): Required when using Gemini models

First, install dependencies and set up your environment:

# Install dependencies
uv sync

Configure your environment by creating a .env file with your API keys and project settings.

1. Copy the example file:

   cp .env.example .env

2. Open .env in your editor and replace the placeholder values with your actual credentials. Example .env content:

   OPENAI_API_KEY=sk-your_actual_api_key_here
   # The following GCP settings are only required if you plan to use Google Gemini models.
   GCP_PROJECT=your-gcp-project-id
   GCP_LOCATION=your-gcp-region

Required Environment Variables:

• OPENAI_API_KEY: Your OpenAI API key
• GCP_PROJECT: Google Cloud Platform project ID
• GCP_LOCATION: GCP region (e.g., us-central1, asia-northeast1)

Before building the dataset, you need to download the GeoLife dataset:

1. 📥 Download: Visit Microsoft Research GeoLife and download the GPS trajectory dataset
2. 📂 Extract and Place: Extract the downloaded archive and place the contents in data/geolife/
3. ✅ Verify: Ensure the directory structure matches the expected format below

Expected directory structure after setup:

data/
└── geolife/
    └── Data/
        ├── 000/
        │   └── Trajectory/
        │       ├── 20081023025304.plt
        │       └── ...
        ├── 001/
        ├── 002/
        └── ... (up to 181)

To build the complete dataset, run the following command:

# Build the complete Mobility QA dataset
bash scripts/build_dataset.sh

The script will automatically execute the preprocessing pipeline and generate the final dataset in the mobility_qa_dataset/ directory.

For detailed information about each preprocessing step, see src/mobility_qa/preprocess/README.md.

The final dataset is organized as follows:

mobility_qa_dataset/
├── factual_retrieval/
│   ├── one_day/                    # 📅 Single-day factual retrieval questions
│   │   ├── place_to_time/          # 📍 "When was the person at location X?"
│   │   │   ├── 1/qa.json
│   │   │   ├── 2/qa.json
│   │   │   └── ...
│   │   └── ... (other question types)
│   └── weekly/                     # 📈 Weekly pattern questions
├── free_form/
│   ├── one_day/                    # 💭 Single-day open-ended questions
│   └── weekly/                     # 📝 Weekly analysis questions
└── multiple_choice/
    ├── one_day/                    # ✅ Single-day multiple choice questions
    └── weekly/                     # 📋 Weekly multiple choice questions

Each QA file contains:

• ❓ Question text
• 🏷️ Question type and metadata
• ✅ Ground truth answers
• 🛣️ Integrated trajectory data (filtered and normalized)
• 👤 User ID and original day information

The repository includes three completion scripts for generating answers using large language models.

1. 🔍 Factual Retrieval: scripts/fact_retrieval.py - Extract specific facts from trajectory data
2. ✅ Multiple Choice: scripts/multi_choice.py - Answer multiple-choice questions about mobility patterns
3. 💭 Free Form: scripts/free_form.py - Generate open-ended responses with automatic evaluation

Run each script with default settings (no annotations, 10 questions per script):

# Run factual retrieval (extracts specific facts)
uv run scripts/fact_retrieval.py

# Run multiple choice (selects from given options)
uv run scripts/multi_choice.py

# Run free form (generates open-ended answers)
uv run scripts/free_form.py

# Specify model and parameters
uv run scripts/fact_retrieval.py \
  --model_name gpt-4o \
  --temperature 0.5 \
  --num_questions 20

# Include all annotation types (mobility_mode, poi, reason)
uv run scripts/multi_choice.py \
  --with_annotation

# Run free form with evaluation model
uv run scripts/free_form.py \
  --model_name gpt-4o-mini \
  --evaluation_model_name gpt-4o \
  --with_annotation

# Basic usage with default settings (no annotations, 10 questions)
uv run scripts/fact_retrieval.py

# Process more questions with a different model
uv run scripts/multi_choice.py \
  --model_name gpt-4 \
  --num_questions 50

# Include all annotations for comprehensive analysis
uv run scripts/free_form.py \
  --with_annotation \
  --temperature 0.3

# Large-scale evaluation with custom retry settings
uv run scripts/fact_retrieval.py \
  --with_annotation \
  --model_name gpt-4 \
  --num_questions 100 \
  --max_retries 10 \
  --initial_wait_time 60 \
  --seed 123

# Reproducible experiments with specific seed
uv run scripts/multi_choice.py \
  --model_name gpt-4 \
  --seed 12345

{
  "question": "When was the person at (10.54, 3.78)?",
  "question_type": "place_to_time",
  "original_days": [6],
  "user_id": 115,
  "answer": ["01:30"],
  "trajectory": [
    ["day", "time", "x", "y"],
    [1, "02:19:16", -0.15, 0.4],
    [1, "02:20:01", -0.17, 0.33],
    ...
  ]
}

The trajectory data in each QA file follows a consistent format with different annotation levels:

Used in factual_retrieval and multiple_choice datasets:

[
  ["day", "time", "x", "y"],
  [1, "00:00:00", -0.05, -3.52],
  [1, "00:01:00", -0.04, -3.78],
  ...
]

• day: Normalized day number (starts from 1)
• time: Time in HH:MM:SS format
• x, y: Normalized spatial coordinates in kilometers, relative to a reference point, with precision up to two decimal places.

Used in free_form dataset with additional annotations in Japanese:

[
  ["day", "time", "x", "y", "mobility_mode", "reason", "poi"],
  [1, "00:00:00", -0.05, -3.52, "バス", "職場への移動", "通勤途中のバス停"],
  [1, "00:01:00", -0.04, -3.78, "バス", "職場への移動", NaN],
  ...
]

• mobility_mode: Transportation mode (e.g., "バス", "徒歩", "車")
• reason: Purpose of movement (e.g., "職場への移動", "余暇", "買い物")
• poi: Point of interest category (e.g., "自宅", "オフィス", "レストラン")

• Day numbers are normalized to start from 1 based on original_days field
• Spatial coordinates (x, y) are normalized and may not represent actual GPS coordinates
• Time format is consistent across all trajectory points

This dataset is built upon the GeoLife GPS Trajectory Dataset. We have processed and transformed the original GeoLife data to create question-answer pairs for mobility analysis tasks, including additional semantic annotations.

If you use this dataset, please cite the original GeoLife papers

@inproceedings{zheng2010geolife,
  title={GeoLife: A collaborative social networking service among user, location and trajectory},
  author={Zheng, Yu and Xie, Xing and Ma, Wei-Ying},
  booktitle={IEEE Data Eng. Bull.},
  volume={33},
  number={2},
  pages={32--39},
  year={2010}
}

This project is licensed under the MIT License.

@article{asano2025mobqa,
  title         = "{MobQA}: A benchmark dataset for semantic understanding of
                   human mobility data through question answering",
  author        = "Hikaru, Asano and Hiroki, Ouchi and Akira, Kasuga and Ryo,
                   Yonetani",
  journal       = "arXiv preprint arXiv:2508.11163",
  year          =  2025,
}