FIX: Read Nihon Kohden annotation file accurately by myd7349 · Pull Request #13251 · mne-tools/mne-python

myd7349 · 2025-05-17T09:20:09Z

Reference issue (if any)

Fix #11267.

What does this implement/fix?

This PR adds support for reading sub event log blocks in Nihon Kohden EEG annotation files (.LOG).

In certain versions of the .LOG files, in addition to the standard event log blocks, there are sub event log blocks.

The event log block contains timestamps in HHMMSS format.
The sub event log block provides additional millisecond and microsecond precision in the form of cccuuu.
If the event text is too long, it is split into two parts and stored separately in the event log block and the sub event log block.

Additional information

I noticed that @jacobshaw42 also attempted something similar in #11431, but for some reason, the PR was closed.

This PR differs from #11431 in the following ways:

Sub event log blocks are not limited to 'EEG-1200A V01.00'

For example, in MB0400FU.EEG, the device type is EEG-1100C V01.00, yet it does contain sub event log blocks.

Nihon Kohden does not clearly specify which device types or software versions generate .LOG files that include sub event log blocks. I previously tried to implement a function to determine whether sub event log blocks are present, based on the device type:
```
def contains_sub_event_blocks(device_type: str) -> bool:
    device_types_with_sub_events = (
        "EEG-1100A V01.00",
        "EEG-1100A V02.0",
        "EEG-1100B V01.00",
        "EEG-1100C V01.00",
        "EEG-2100  V01.00",
        "EEG-2100  V02.00",
        "EEG-1100A V02.00",
        "EEG-1100B V02.00",
        "EEG-1100C V02.00",
    )
    device_types_without_sub_events = (
        "QI-403A   V01.00",
        "QI-403A   V02.00",
    )

    if (
        device_type.startswith("EEG-2110")
        or device_type in device_types_without_sub_events
    ):
        return False
    elif device_type in device_types_with_sub_events:
        return True

    raise NotImplementedError(f"Unsupported device type: {device_type}.")
```
However, I found this approach overly complicated, so I switched to a more general strategy.

In Nihon Kohden .LOG files, the control block can define up to 43 event log blocks. When sub event blocks are present:
- Blocks 1–21 define the offsets for standard event log blocks,
- Block 22 may be unused,
- Blocks 23–43 define the offsets for the corresponding sub event log blocks (matching 1–21 one-to-one).
Therefore, this PR assumes that sub event log blocks are present when the number of log blocks (n_logblocks) parsed from the control block does not exceed 21.

BTW, in nk2edf, the presence of sub event log blocks is assumed.

Since the logic for reading event blocks and sub event blocks is largely similar, I refactored the relevant code in _read_nihon_annotations into a helper function _read_event_log_block. Two conditions are used to ensure a sub event block is valid:
- The block offset in the control block must be greater than zero.
- The data name inside the block must match the device type from the device block.
Decode event description at last

Because event text can be split across the event log block and the sub event log block, this PR concatenates the byte strings from both blocks before decoding. This affects the following logic:
- Since the event log block is no longer decoded directly, the strptime method can no longer be used to parse the HHMMSS time(which is a byte string, not a str). Instead, the time is parsed using int for each component.

myd7349 · 2025-05-17T13:00:12Z

The tests failed because:

mne/io/nihon/tests/test_nihon.py:41: in test_nihon_eeg
    assert an1["onset"] == an2["onset"]
E   assert np.float64(1.14) == np.float64(1.0)

I saw a note here:

mne-python/mne/io/nihon/tests/test_nihon.py

Line 39 in dda454d

# EDF has some weird annotations, which are not in the LOG file

There are only two events in the .LOG file, but four annotations in the .EDF file. So I wrote a test script:

# encoding: utf-8

import os.path
import urllib.request

import edfio
from mne.io.nihon import read_raw_nihon


def download_file(url: str, output_path: str):
    try:
        urllib.request.urlretrieve(url, output_path)
        return True
    except Exception as e:
        print(e)
        return False

def test_edf():
    file = r"MB0400FU.EDF"
    if not os.path.exists(file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.EDF",
                             file):
            return

    edf = edfio.read_edf(file)
    for annotation in edf.get_annotations():
        print(annotation)


def test_eeg():
    eeg_file = "MB0400FU.EEG"
    elec_file = "MB0400FU.21E"
    pnt_file = "MB0400FU.PNT"
    log_file = "MB0400FU.LOG"

    if not os.path.exists(eeg_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.EEG",
                             eeg_file):
            return

    if not os.path.exists(elec_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.21E",
                             elec_file):
            return

    if not os.path.exists(pnt_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.PNT",
                             pnt_file):
            return

    if not os.path.exists(log_file):
        if not download_file("https://raw.githubusercontent.com/mne-tools/mne-testing-data/refs/heads/master/NihonKohden/MB0400FU.LOG",
                             log_file):
            return

    raw = read_raw_nihon(eeg_file)

    for onset, duration, description in zip(
        raw.annotations.onset,
        raw.annotations.duration,
        raw.annotations.description,
    ):
        print(onset, description)


if __name__ == "__main__":
    test_edf()
    test_eeg()

Output:

EdfAnnotation(onset=0.0, duration=None, text='+0.000000')
EdfAnnotation(onset=0.0, duration=None, text='Segment: REC START ALLE EEG')
EdfAnnotation(onset=1.0, duration=None, text='+1.140000')
EdfAnnotation(onset=1.0, duration=None, text='A1+A2 OFF')
Loading MB0400FU.EEG
Reading header from D:\edf_demo\MB0400FU.EEG
Found PNT file, reading metadata.
Found LOG file, reading events.
0.0 REC START ALLE EEG
1.0 A1+A2 OFF

myd7349 · 2025-05-17T13:36:32Z

It appears that MB0400FU.EDF is suspicious. Therefore, I used nk2edf to convert MB0400FU.EEG to EDF format: MB0400FU_1-1+.zip.

By the way, the RESET condition shown above is not an actual event or annotation — it’s just a trigger label parsed from the Events/Markers channel:

https://gitlab.com/Teuniz/EDFbrowser/-/blob/master/edf_annotations.cpp?ref_type=heads#L157

When reading MB0400FU_1-1+.zip using edfio or mne.io.read_raw_edf, it returns two annotations.

myd7349 · 2025-05-17T14:09:40Z

I also noticed that Nihon Kohden's software supports a type of annotation called P_COMMENT. When using P_COMMENT, the event text stored in the .LOG file is simply "P_COMMENT", while the actual comment appears to be stored elsewhere.

myd7349 · 2025-05-17T14:18:26Z

Therefore, I believe this test failure should be addressed by updating MB0400FU.EDF in the following way:

Convert MB0400FU.EEG to EDF using nk2edf, and replace the current file with the newly generated EDF(MB0400FU_1-1%2B.zip, for example); or
~~Re-export a new EDF using Nihon Kohden's software.~~Change test code
Testing revealed that the file at https://github.com/mne-tools/mne-testing-data/blob/master/NihonKohden/MB0400FU.EDF appears to be an EDF file exported using Nihon Kohden's Neuro Workbench software. This export process is actually carried out by invoking a program called BESA EEG Converter. On my machine, after converting MB0400FU.EEG with BESA EEG Converter version 1.0.0.25, the resulting EDF file seems to have the same issue as the one at https://github.com/mne-tools/mne-testing-data/blob/master/NihonKohden/MB0400FU.EDF.

Fix mne-tools#11267.

* upstream/main: (46 commits) MAINT: Restore edfio git install (mne-tools#13421) Support preload=False for the new EEGLAB single .set format (mne-tools#13096) [pre-commit.ci] pre-commit autoupdate (mne-tools#13453) MAINT: Restore PySide6 6.10.0 testing (mne-tools#13446) MAINT: Auth [skip azp] [skip actions] MAINT: Deploy [circle deploy] [skip azp] [skip actions] Bump github/codeql-action from 3 to 4 in the actions group (mne-tools#13442) ENH: Dont constrain fiducial clicks to mesh vertices (mne-tools#13445) Use timezone-aware ISO 8601 for website timestamp (mne-tools#13347) [pre-commit.ci] pre-commit autoupdate (mne-tools#13443) FIX: Update osf.io links to new format (mne-tools#13440) MAINT: Ensure full checkout is used (mne-tools#13439) Add BDF export (mne-tools#13435) [pre-commit.ci] pre-commit autoupdate (mne-tools#13434) [pre-commit.ci] pre-commit autoupdate (mne-tools#13431) MAINT: Update code credit (mne-tools#13432) FIX, TST: Try to get test_export_epochs_eeeglab passing again (mne-tools#13428) FIX: Add on_few_samples parameter to core rank estimation (mne-tools#13350) MAINT: Reenable mpl nightly (mne-tools#13426) [pre-commit.ci] pre-commit autoupdate (mne-tools#13427) ...

larsoner

Sorry we missed this one @myd7349 ! In the future if we don't respond in a week or so feel free to ping us again for review, this one just slipped through the cracks

I pushed a couple tiny commits to add your test file from option (1) above, bump testing version, adjust the test appropriately, and adjust a couple tiny style things. Marking for merge-when-green, thanks in advance @myd7349 !

* upstream/main: (23 commits) ENH: Add on_missing for combine_channels (mne-tools#13463) Bump the actions group with 2 updates (mne-tools#13464) Move development dependencies into a dependency group (no more extra) (mne-tools#13452) ENH: add on_missing for rename_channels (mne-tools#13456) add advisory board to website (mne-tools#13462) ENH: Support Nihon Kohden EEG-1200A V01.00 (mne-tools#13448) MAINT: Update dependency specifiers (mne-tools#13459) ENH: Add encoding parameter to Nihon Kohden reader (mne-tools#13458) [MAINT] Automatic SPEC0 dependency version management (mne-tools#13451) FIX: Read Nihon Kohden annotation file accurately (mne-tools#13251) MAINT: Restore edfio git install (mne-tools#13421) Support preload=False for the new EEGLAB single .set format (mne-tools#13096) [pre-commit.ci] pre-commit autoupdate (mne-tools#13453) MAINT: Restore PySide6 6.10.0 testing (mne-tools#13446) MAINT: Auth [skip azp] [skip actions] MAINT: Deploy [circle deploy] [skip azp] [skip actions] Bump github/codeql-action from 3 to 4 in the actions group (mne-tools#13442) ENH: Dont constrain fiducial clicks to mesh vertices (mne-tools#13445) Use timezone-aware ISO 8601 for website timestamp (mne-tools#13347) [pre-commit.ci] pre-commit autoupdate (mne-tools#13443) ...

Co-authored-by: Eric Larson <[email protected]>

eulerleibniz · 2026-05-23T12:12:45Z

Hello there @larsoner . Not sure if i need to create a new issue for this, but the new code has a bug here:
in line 341 here:

            for li, t_log in enumerate(t_logs): # Chek if t_logs Is None
                t_desc, t_onset = _parse_event_log(t_log)
                if t_sub_logs is not None and t_sub_logs.size == t_logs.size:
                    t_sub_desc, t_sub_onset = _parse_sub_event_log(t_sub_logs[li])
                    t_desc += t_sub_desc
                    t_onset += t_sub_onset

You did not check if t_logs is None, so we get errors on enumerate.

larsoner · 2026-05-23T18:38:47Z

@eulerleibniz do you want to open a PR with the suggested change?

eulerleibniz · 2026-05-24T06:42:28Z

@larsoner , unfortunately i live in iran and our internet access is not great thanks to ongoing war, so i prefer to leave it to you lovely people. Sorry cant help ☹️

myd7349 · 2026-05-24T08:23:53Z

Hi @eulerleibniz, thanks for your report. I will do some test.
If this is confirmed to be a bug, I'd be happy to fix it.

myd7349 · 2026-05-24T08:43:18Z

Old implementation(before this PR):

def _read_nihon_annotations(fname):
        ...
        for t_block in range(n_logblocks):
            fid.seek(0x92 + t_block * 20)
            t_blk_address = np.fromfile(fid, np.uint32, 1)[0]
            fid.seek(t_blk_address + 0x12)
            n_logs = np.fromfile(fid, np.uint8, 1)[0]
            fid.seek(t_blk_address + 0x14)
            t_logs = np.fromfile(fid, "|S45", n_logs)
            for t_log in t_logs:
                ...

New implementation:

def _read_event_log_block(fid, t_block, version):
    fid.seek(0x92 + t_block * 20)
    data = np.fromfile(fid, np.uint32, 1)
    if data.size == 0 or data[0] == 0:
        return
    t_blk_address = data[0]

    fid.seek(t_blk_address + 0x1)
    data = np.fromfile(fid, "|S16", 1).astype("U16")
    if data.size == 0 or data[0] != version:
        return

    fid.seek(t_blk_address + 0x12)
    data = np.fromfile(fid, np.uint8, 1)
    if data.size == 0:
        return
    n_logs = data[0]

    fid.seek(t_blk_address + 0x14)
    return np.fromfile(fid, "|S45", n_logs)


def _read_nihon_annotations(fname):
        ...
        for t_block in range(n_logblocks):
            t_logs = _read_event_log_block(fid, t_block, version)

As you can see, in the new implementation, _read_event_log_block can indeed return None, and there are several places where that may happen.

Let’s compare the behavior step by step:

Reading t_blk_address

Old code:

fid.seek(0x92 + t_block * 20)
t_blk_address = np.fromfile(fid, np.uint32, 1)[0]

New code:

fid.seek(0x92 + t_block * 20)
data = np.fromfile(fid, np.uint32, 1)
if data.size == 0 or data[0] == 0:
    return
t_blk_address = data[0]

If np.fromfile(fid, np.uint32, 1) returns an empty array, the old code would raise an exception immediately, whereas the new code returns None.

Checking version to ensure the data format is correct

fid.seek(t_blk_address + 0x1)
data = np.fromfile(fid, "|S16", 1).astype("U16")
if data.size == 0 or data[0] != version:
    return

This check was added in the new implementation.

Reading n_logs

Old code:

fid.seek(t_blk_address + 0x12)
n_logs = np.fromfile(fid, np.uint8, 1)[0]

New code:

fid.seek(t_blk_address + 0x12)
data = np.fromfile(fid, np.uint8, 1)
if data.size == 0:
    return
n_logs = data[0]

If np.fromfile(fid, np.uint8, 1) returns an empty array, the old code would raise an exception immediately, whereas the new code returns None.

Reading t_logs

The behavior is the same in both implementations. If n_logs == 0, t_logs will be an empty array.

So, in summary, if you are getting None, it could be coming from any of the three return points above, namely steps 1, 2, or 3.

Could you try adding a few debug prints like this so we can see exactly where None is being returned?

def _read_event_log_block(fid, t_block, version):
    fid.seek(0x92 + t_block * 20)
    data = np.fromfile(fid, np.uint32, 1)
    if data.size == 0 or data[0] == 0:
        print(f"Failed to read t_blk_address: {data}.")
        return
    t_blk_address = data[0]

    fid.seek(t_blk_address + 0x1)
    data = np.fromfile(fid, "|S16", 1).astype("U16")
    if data.size == 0 or data[0] != version:
        print(f"Failed to read version: {data}, expected: {version}.")
        return

    fid.seek(t_blk_address + 0x12)
    data = np.fromfile(fid, np.uint8, 1)
    if data.size == 0:
        print(f"Failed to read n_logs: {data}.")
        return
    n_logs = data[0]

    fid.seek(t_blk_address + 0x14)
    return np.fromfile(fid, "|S45", n_logs)

eulerleibniz · 2026-05-24T09:14:57Z

@myd7349 Thank you . I am not sure if you saw this other issue i opened some time ago:

Issue: #13633 -> Support reading comment text/content from Nihon Kohden EEG files

I also provided some codes there for reading the CMT files and later this issue was created based on that:
Issue: #13642 -> Implement reading .CMT comment files in our Nihon Kohden reader

Here is a standalone code i use currently to read my annotations completely, some of it is possibly unique to how our Nihon devices are setup, but the read_comments_from_nihon_kohden_cmt_file function might be of interest to you. All the constatns in the begining of the file are from inspection of the files using Hex editor.
nihon copy.py

myd7349 · 2026-05-24T09:39:33Z

Hi @eulerleibniz,

Yes, I saw it. I’m also interested in how P_COMMENT is read, and I’m glad to see that @wmvanvliet is working on it.

To fix the new issue you identified, I need your help. Could you patch nihon.py with the code below and share the output with me? That will help me fix this newly discovered issue.

def _read_event_log_block(fid, t_block, version):
    fid.seek(0x92 + t_block * 20)
    data = np.fromfile(fid, np.uint32, 1)
    if data.size == 0 or data[0] == 0:
        print(f"Failed to read t_blk_address: {data}.")
        return
    t_blk_address = data[0]

    fid.seek(t_blk_address + 0x1)
    data = np.fromfile(fid, "|S16", 1).astype("U16")
    if data.size == 0 or data[0] != version:
        print(f"Failed to read version: {data}, expected: {version}.")
        return

    fid.seek(t_blk_address + 0x12)
    data = np.fromfile(fid, np.uint8, 1)
    if data.size == 0:
        print(f"Failed to read n_logs: {data}.")
        return
    n_logs = data[0]

    fid.seek(t_blk_address + 0x14)
    return np.fromfile(fid, "|S45", n_logs)

Many thanks.

eulerleibniz · 2026-05-24T10:17:11Z

@myd7349 I just did what you asked. using mne.version = 1.12.1

The error doesn't happen with my files which have EEG-1100A V01.00 in their header.
It only happens with EEG files which have header EEG-1200A V01.00.
I also checked the mne version=1.10.0 of the mne which also had issues with this old format and they clearly wrote this in nihon file in line 50:

_valid_headers = [
    "EEG-1100A V01.00",
    "EEG-1100B V01.00",
    "EEG-1100C V01.00",
    "QI-403A   V01.00",
    "QI-403A   V02.00",
    "EEG-2100  V01.00",
    "EEG-2100  V02.00",
    "DAE-2100D V01.30",
    "DAE-2100D V02.00",
    # 'EEG-1200A V01.00',  # Not working for the moment.
]

where it was clearly mentioned that they don't support that specific version yet.
here is the output of the prints you asked on these files:

🪲 Failed to read version: [''], expected: EEG-1200A V01.00.
🪲 Failed to read version: [''], expected: EEG-1200A V01.00.
Traceback (most recent call last):
  File "c:\GitHub\Vigilant\vigilant\emu\sop_v0_1_0\nihon.py", line 460, in <module>
    raw_mne = mne.io.read_raw_nihon(path, preload=False)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Titania\anaconda3\envs\devenv\Lib\site-packages\mne\io\nihon\nihon.py", line 52, in read_raw_nihon
    return RawNihon(fname, preload, encoding=encoding, verbose=verbose)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<decorator-gen-207>", line 12, in __init__
  File "C:\Users\Titania\anaconda3\envs\devenv\Lib\site-packages\mne\io\nihon\nihon.py", line 506, in __init__
    annots = _read_nihon_annotations(fname, encoding)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Titania\anaconda3\envs\devenv\Lib\site-packages\mne\io\nihon\nihon.py", line 375, in _read_nihon_annotations
    for li, t_log in enumerate(t_logs):
                     ^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable

….LOG files This is a follow-up to mne-tools#13251. In mne-tools#13251, some additional checks were added to `_read_event_log_block` (including a version check), which could cause `_read_event_log_block` to return `None`, leading to iteration errors. This change avoids that issue by returning an empty array instead of `None` when reading fails. In addition, when the version does not match, a warning is now issued instead of returning `None`.

myd7349 · 2026-05-24T16:50:16Z

Hello @eulerleibniz I have just created a PR #13915 to address this issue. Could you try this change and see whether it resolves your problem?

myd7349 marked this pull request as ready for review May 17, 2025 10:23

myd7349 requested review from agramfort, dengemann, drammock and larsoner as code owners May 17, 2025 10:23

myd7349 force-pushed the fix-issue-11267 branch 4 times, most recently from fe60298 to 2265c3d Compare May 23, 2025 13:08

myd7349 force-pushed the fix-issue-11267 branch from 0a160a3 to df1d550 Compare July 13, 2025 03:28

myd7349 added 2 commits August 17, 2025 23:38

FIX: Read Nihon Kohden annotation file accurately

f663681

Fix mne-tools#11267.

Check for sub event more carefully

9805b69

myd7349 force-pushed the fix-issue-11267 branch from df1d550 to 9805b69 Compare August 17, 2025 15:40

myd7349 mentioned this pull request Oct 22, 2025

ENH: Support Nihon Kohden EEG-1200A V01.00 #13448

Merged

2 of 3 tasks

larsoner added 3 commits October 22, 2025 09:59

FIX: Tests

80ca18c

FIX: Double

2f3f4c7

larsoner approved these changes Oct 22, 2025

View reviewed changes

larsoner enabled auto-merge (squash) October 22, 2025 14:11

larsoner added 4 commits October 22, 2025 10:16

TST: Ping

673b558

FIX: MD5

8a6e709

FIX: Maybe

01b0a20

FIX: Maybe

e8569c3

larsoner merged commit 181fea1 into mne-tools:main Oct 22, 2025
32 checks passed

myd7349 deleted the fix-issue-11267 branch October 23, 2025 00:21

myd7349 mentioned this pull request Oct 23, 2025

ENH: Better handling of annotation encoding in Nihon Kohden reader #13457

Closed

sseth pushed a commit to xannnimal/mne-python that referenced this pull request Mar 25, 2026

FIX: Read Nihon Kohden annotation file accurately (mne-tools#13251)

4393799

Co-authored-by: Eric Larson <[email protected]>

myd7349 mentioned this pull request May 24, 2026

FIX: Avoid iteration errors caused by None when reading Nihon Kohden .LOG files #13915

Merged

Conversation

myd7349 commented May 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reference issue (if any)

What does this implement/fix?

Additional information

Uh oh!

myd7349 commented May 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

myd7349 commented May 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

myd7349 commented May 17, 2025

Uh oh!

myd7349 commented May 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

larsoner left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

eulerleibniz commented May 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

larsoner commented May 23, 2026

Uh oh!

eulerleibniz commented May 24, 2026

Uh oh!

myd7349 commented May 24, 2026

Uh oh!

myd7349 commented May 24, 2026

Uh oh!

eulerleibniz commented May 24, 2026

Uh oh!

myd7349 commented May 24, 2026

Uh oh!

eulerleibniz commented May 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

myd7349 commented May 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

myd7349 commented May 17, 2025 •

edited

Loading

myd7349 commented May 17, 2025 •

edited

Loading

myd7349 commented May 17, 2025 •

edited

Loading

myd7349 commented May 17, 2025 •

edited

Loading

larsoner left a comment •

edited

Loading

eulerleibniz commented May 23, 2026 •

edited

Loading

eulerleibniz commented May 24, 2026 •

edited

Loading