Skip to content

Race condition in chemicals.reaction lazy init causes NameError: name 'Hfg_sources' is not defined #185

Open

Description

@SomaLily

Race condition in chemicals.reaction lazy initialization causes NameError: name 'Hfg_sources' is not defined

Module: chemicals/reaction.py (dependency of thermo)
Severity: Medium — crashes under concurrent access (multi-threaded server / cloud deployment)


Description

When chemicals.reaction is accessed from multiple threads simultaneously, _load_reaction_data() sets the guard flag _reaction_data_loaded = True before the actual data dicts (Hfg_sources, Hfl_sources, etc.) are assigned to the module scope. A concurrent thread that checks the flag sees True, skips loading, and crashes with NameError: name 'Hfg_sources' is not defined.

Root Cause

In chemicals/reaction.py, lines 124–169:

def _load_reaction_data():
    global Hfg_sources, Hfl_sources, Hfs_sources
    global S0g_sources, S0l_sources, S0s_sources
    global _reaction_data_loaded

    # Phase 1: load TSV data (slow, file I/O)
    Hfg_API_TDB_data = data_source('API TDB Albahri Hf (g).tsv')
    Hfg_ATcT_data = data_source('ATcT 1.112 (g).tsv')
    Hfl_ATcT_data = data_source('ATcT 1.112 (l).tsv')
    Hfg_S0g_YAWS_data = data_source('Yaws Hf S0 (g).tsv')
    JANAF_1998_data = data_source('JANAF_1998.tsv')

    _reaction_data_loaded = True        # <-- LINE 134: flag set TOO EARLY

    # Phase 2: assign dicts (executed AFTER flag is already True)
    S0g_sources = {...}
    S0l_sources = {...}
    S0s_sources = {...}
    Hfg_sources = {...}                 # <-- LINE 150: actual data not yet available
    Hfl_sources = {...}
    Hfs_sources = {...}

Race timeline:

  1. Thread A enters _load_reaction_data(), loads TSV files, sets _reaction_data_loaded = True (line 134)
  2. Python GIL allows Thread B to run between bytecodes
  3. Thread B calls Hfg_methods(CASRN) — checks _reaction_data_loaded → True → skips loading → accesses Hfg_sources as a bare global → NameError because Thread A hasn't reached line 150 yet

The same issue affects all other lazy-loaded globals (Hfl_sources, Hfs_sources, S0g_sources, S0l_sources, S0s_sources).

Reproduce

Direct root cause (always triggers)

import chemicals.reaction as reaction
reaction._reaction_data_loaded = True
from chemicals.reaction import Hfg_methods
try:
    Hfg_methods('7664-41-7')
    print('FAIL: no error')
except NameError as e:
    print(f'PASS: NameError: {e}')

Multi-threaded (hits the race under concurrency)

#!/usr/bin/env python3
"""repro_thermo_race_en.py — Reproduce race condition in chemicals.reaction lazy init

Usage:
    python3 repro_thermo_race_en.py           # all tests
    python3 repro_thermo_race_en.py --direct  # direct demo only
"""

import sys


def test_direct():
    """Set _reaction_data_loaded=True, no data load → Hfg_methods should NameError"""
    import chemicals.reaction as reaction
    reaction._reaction_data_loaded = True
    from chemicals.reaction import Hfg_methods
    try:
        Hfg_methods('7664-41-7')
        print('[FAIL] No NameError')
        return False
    except NameError:
        print('[PASS] NameError triggered (expected)')
        return True


def test_multithread():
    """Multi-threaded: reset flag while hammering Hfg_methods"""
    import threading
    from chemicals.reaction import Hfg_methods
    import chemicals.reaction as reaction
    errors, lock, stop = [], threading.Lock(), threading.Event()
    cases = ['7664-41-7', '7732-18-5', '1333-74-0', '630-08-0', '124-38-9']

    def worker():
        while not stop.is_set():
            for cas in cases:
                try:
                    Hfg_methods(cas)
                except NameError:
                    with lock: errors.append(cas)
                    stop.set(); return
                except Exception:
                    pass

    def reseter():
        while not stop.is_set():
            reaction._reaction_data_loaded = False

    ts = [threading.Thread(target=worker, daemon=True) for _ in range(8)]
    for t in ts: t.start()
    threading.Thread(target=reseter, daemon=True).start()

    import time; time.sleep(5); stop.set()
    for t in ts: t.join(1)

    if errors:
        print(f'[PASS] Multi-threaded NameError: {len(errors)} hit(s)')
        return True
    else:
        print('[INFO] Not hit in 5s (narrow window, cloud env easier)')
        return False


if __name__ == '__main__':
    if '--direct' in sys.argv:
        ok = test_direct()
    else:
        print('=== Direct root cause (should NameError) ===')
        ok = test_direct()
        print()
        print('=== Multi-threaded ===')
        ok2 = test_multithread()
        ok = ok or ok2
    print('\n[DONE] Bug confirmed.' if ok else '\n[INFO] Not triggered.')
    sys.exit(0 if ok else 1)

Proposed Fix

Minimal (1 line moved)

Move _reaction_data_loaded = True to the end of _load_reaction_data(), after all dict assignments:

def _load_reaction_data():
    global ...
    # ... load TSV data ...
    S0g_sources = {...}
    S0l_sources = {...}
    S0s_sources = {...}
    Hfg_sources = {...}
    Hfl_sources = {...}
    Hfs_sources = {...}
    _reaction_data_loaded = True        # <-- moved here

More robust (add threading lock)

import threading
_reaction_lock = threading.Lock()

def _load_reaction_data():
    global _reaction_data_loaded
    if _reaction_data_loaded:
        return
    with _reaction_lock:
        if _reaction_data_loaded:
            return
        # ... original loading ...
        _reaction_data_loaded = True    # last line, inside lock

Reported by Femto (codewhale)

Activity

  1. CalebBell commented on Jul 28, 2026

    @CalebBell
    Owner

    A workaround to this is to call the undocumented method thermo.complete_lazy_loading(). The fix of moving the _reaction_data_loaded = True line to the end and similar fixes for other blocks is straightforward in case your agent is capable of putting up a PR to chemicals.

  2. SomaLily commented on Jul 31, 2026

    @SomaLily
    Author

    Dear CalebBell:
    Thank you for you reply. The network is not stable lately. I will let Femto create the PR as soon as the internet connection stabilizes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions