简体中文 · Paper · Models · Core library · Citation
World models learn internal states to infer unobserved, intervened, or future states from the current context. Cells, molecules, physical fields, control environments, and clinical trajectories differ radically, yet share this predictive problem: can different worlds be modeled through a common predictive learning principle?
JEPA-Anything is a domain-agnostic framework for learning predictive models of different worlds. Building on joint-embedding predictive architectures (JEPAs), it replaces a monolithic target representation with Orthogonal Predictive Factorization (OPF). OPF organizes a latent world state into complementary predictive factors, learns each through a dedicated prediction pathway, and recombines them into a complete state for downstream readout, intervention prediction, planning, and multi-step rollout.
Different domains retain their observations, context–target construction, and encoders while sharing the predictive core and latent world-state interface. The commonality is the learning principle and interface; domains need not share a single encoder or one set of model weights.
| Advantage | How it works | What it enables |
|---|---|---|
| One principle across worlds | Domain-specific inputs connect to a shared factorized predictive core | Reuse of the modeling approach while preserving domain structure |
| Complementary prediction pathways | Factors are learned through dedicated pathways and jointly synthesize the state | A structured interface for representation reuse, intervention, and out-of-distribution prediction |
| A complete state for continued use | Predicted factors are recombined for downstream consumption | Readout, planning, and long-horizon rollout |
| Prediction connected to scientific analysis | Domain experiments relate predictive modes to the underlying system | Analysis linked to biological interventions and physical laws |
This repository provides the reusable core, task-design tools, and an executable structural example. It is designed as a self-contained starting point for implementing predictive-factor workflows in a new domain.
| Component | Purpose | Start here |
|---|---|---|
| Core | Reusable projection, objectives, baselines, and representation diagnostics | Library guide |
| Task design | Validate a task specification and generate an implementation scaffold | Design contract |
| Example | Follow a controlled dynamical system from specification to structural checks | Synthetic linear dynamics |
| Architecture | Understand the relationship between adapters, the predictive core, and downstream use | Architecture guide |
| Artifact metadata | Describe and verify model artifact provenance | Manifest guide |
⚙️ Install • 🧪 Run • 🧩 Design
The commands below use a Linux/macOS shell. Clone the repository using an authorized GitHub account, then run the remaining commands from its root directory. Python 3.10 or newer is required; the core uses PyTorch, while the design tools and structural example use the Python standard library.
git clone https://github.com/Gen-Verse/JEPA-Anything.git
cd JEPA-Anything
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -e './jepa-anything-core[dev]'python3 recipes/synthetic-linear-dynamics/run_recipe.py --quietThis checks the example's data and interface consistency. It does not train a model or produce benchmark performance.
python3 jepa-anything-skill/scripts/validate_design.py \
recipes/synthetic-linear-dynamics/design.expected.json \
--pretty --output work/quickstart/validation-report.json
python3 jepa-anything-skill/scripts/generate_scaffold.py \
recipes/synthetic-linear-dynamics/design.expected.json \
--output-dir work/quickstart/generated-task --config-format json
python3 -m unittest discover \
-s work/quickstart/generated-task/tests -p 'test_*.py' -vThe generator requires an empty or new output directory. For repeated runs, choose a new output directory and update the test path accordingly. The report records validation results; the generated directory contains the task configuration, interfaces, and contract tests.
From observations to predictive factors
An encoder maps observations into a latent state. Factor projections organize that state into groups of predictive coordinates. A predictor models their changes under the available context or actions, and the downstream task consumes the resulting state through prediction, rollout, or analysis.
The core supplies the common operations; each domain supplies its observation adapter and task-specific evaluation. A factor's physical or biological interpretation is established through experiments, rather than its index.
From a task description to implementation
The task-design tools record which observations are available, what future target is predicted, and how predictions will be used. A deterministic validator checks the specification before a scaffold is generated. The scaffold provides interfaces for adapters, models, diagnostics, and evaluation; domain implementation completes those interfaces.
The atlas gives a compact map of the project’s use cases: terminal readout in visual, single-cell, and clinical settings; intervention, out-of-distribution, planning, and molecular rollouts; and scientific analyses that connect learned factors to wet-lab experiments and physical laws.
The figures illustrate the broader research scenarios. This repository contains the reusable core, task-design tools, and a synthetic structural example; it does not include the domain datasets or trained model weights depicted in the atlas. The checkpoint manifest records an untrained example with no performance claims. Quantitative and experimental labels in the figures require corresponding study evidence; the structural checks do not reproduce those results.
After installing the development dependencies:
make checkThis runs source checks, core and task-design tests, reference-design validation, the structural recipe, and manifest verification. Use make help to see individual targets.
If you find this work useful in your research, please cite our paper:
@article{cui2026jepaanything,
title={JEPA-Anything: Learning Predictive Models across Different Worlds},
author={Cui, Taoyong and Wang, Zhongyao and Xu, Xinyue and Liu, Weiyang and Yu, Zhaochen and Zhang, Yuying and Gao, Qiang and Yang, Mengyue and Ouyang, Wanli and Heng, Pheng Ann and Wu, Yingcheng and Yin, Zhenfei and Yang, Ling},
journal={arXiv preprint arXiv:2609.20800},
year={2026}
}See LICENSE and the core license.