Skip to content

Inconsistent raising of IndentationError on for and if blocks after mismatching indented comment  #14242

Closed
#14721
Closed
Inconsistent raising of IndentationError on for and if blocks after mismatching indented comment #14242
#14721

Description

@JonathanGrant92
JonathanGrant92
opened on Nov 12, 2023

Hi, I was directed here by the Jupyter Notebook team, so I'm copying everything over from jupyter/notebook/issues/7140

Description

Similar to issue #11594, which was initially reported at jupyter/notebook/issues/171 and fixed by #11718, whitespace of single-line comments at the top scope causes inconsistent raising of exceptions in code blocks like for and if blocks defined at the top scope anywhere after the indented comment in the same cell. Not reproducible with the same CPython version in IDLE Shell.

Reproduce

  1. Add an indented single-line comment at the top level, for example, not matching expected indentation of a scope by 4 whitespaces.
  2. Add an if or for code block in the same cell anywhere after indented single-line comment
  3. Copy code to CPython's IDLE for a control trial
  4. Compare test and control trials for discrepancies and inconsistency in output.

Expected behavior

Consistent raising and handling of IndentationError

Context

  • Operating System and version: Windows 11 Home 10.0.22621
  • Python: 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
  • VSCode and version:
    Version: 1.84.0 (user setup)
    Commit: d037ac076cee195194f93ce6fe2bdfe2969cc82d
    Date: 2023-11-01T11:29:04.398Z
    Electron: 25.9.2
    ElectronBuildId: 24603566
    Chromium: 114.0.5735.289
    Node.js: 18.15.0
    V8: 11.4.183.29-electron.0
    OS: Windows_NT x64 10.0.22621
  • Jupyter: v2023.10.1003070148

Troubleshoot Output

indented_comments_break_jupyter.pdf

no-indentation-errors-in-idle
Command Line Output 
18:31:21.496 [debug] notebook communications already initialized for editor ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:27.370 [info] Handle Execution of Cells 0 for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:27.370 [debug] start the kernel, options.disableUI=false for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:27.371 [debug] Execution Request Sent to Kernel for cell 0
18:31:27.375 [debug] Start cell 0 execution @ 1699057887375 (clear output)
18:31:27.375 [info] Kernel acknowledged execution of cell 0 @ 1699057887375
18:31:28.594 [info] End cell 0 execution @ 1699057888594, started @ 1699057887375, elapsed time = 1.219s
18:31:28.594 [debug] Cell 0 executed with state Success
18:31:29.156 [info] Handle Execution of Cells 1 for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:29.156 [debug] start the kernel, options.disableUI=false for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:29.157 [debug] Execution Request Sent to Kernel for cell 1
18:31:29.161 [debug] Start cell 1 execution @ 1699057889161 (clear output)
18:31:29.161 [info] Kernel acknowledged execution of cell 1 @ 1699057889161
18:31:29.165 [info] End cell 1 execution @ 1699057889165, started @ 1699057889161, elapsed time = 0.004s
18:31:29.165 [warn] Cancel all remaining cells due to cancellation or failure in execution
18:31:29.165 [debug] Cancel pending cells
18:31:29.166 [debug] Cell 1 executed with state Error
18:31:31.026 [info] Handle Execution of Cells 2 for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:31.026 [debug] start the kernel, options.disableUI=false for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:31.027 [debug] Execution Request Sent to Kernel for cell 2
18:31:31.031 [debug] Start cell 2 execution @ 1699057891031 (clear output)
18:31:31.031 [info] Kernel acknowledged execution of cell 2 @ 1699057891031
18:31:31.035 [info] End cell 2 execution @ 1699057891035, started @ 1699057891031, elapsed time = 0.004s
18:31:31.035 [warn] Cancel all remaining cells due to cancellation or failure in execution
18:31:31.035 [debug] Cancel pending cells
18:31:31.035 [debug] Cell 2 executed with state Error
18:31:32.607 [info] Handle Execution of Cells 3 for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:32.607 [debug] start the kernel, options.disableUI=false for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:32.608 [debug] Execution Request Sent to Kernel for cell 3
18:31:32.613 [debug] Start cell 3 execution @ 1699057892613 (clear output)
18:31:32.613 [info] Kernel acknowledged execution of cell 3 @ 1699057892613
18:31:32.616 [info] End cell 3 execution @ 1699057892616, started @ 1699057892613, elapsed time = 0.003s
18:31:32.616 [debug] Cell 3 executed with state Success
18:31:34.053 [info] Handle Execution of Cells 4 for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:34.054 [debug] start the kernel, options.disableUI=false for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:34.054 [debug] Execution Request Sent to Kernel for cell 4
18:31:34.059 [debug] Start cell 4 execution @ 1699057894058 (clear output)
18:31:34.059 [info] Kernel acknowledged execution of cell 4 @ 1699057894058
18:31:34.061 [info] End cell 4 execution @ 1699057894061, started @ 1699057894058, elapsed time = 0.003s
18:31:34.062 [debug] Cell 4 executed with state Success
18:31:35.183 [info] Handle Execution of Cells 5 for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:35.183 [debug] start the kernel, options.disableUI=false for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:35.184 [debug] Execution Request Sent to Kernel for cell 5
18:31:35.188 [debug] Start cell 5 execution @ 1699057895188 (clear output)
18:31:35.188 [info] Kernel acknowledged execution of cell 5 @ 1699057895188
18:31:35.191 [info] End cell 5 execution @ 1699057895191, started @ 1699057895188, elapsed time = 0.003s
18:31:35.192 [warn] Cancel all remaining cells due to cancellation or failure in execution
18:31:35.192 [debug] Cancel pending cells
18:31:35.192 [debug] Cell 5 executed with state Error
18:31:36.340 [info] Handle Execution of Cells 6 for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:36.341 [debug] start the kernel, options.disableUI=false for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:36.341 [debug] Execution Request Sent to Kernel for cell 6
18:31:36.345 [debug] Start cell 6 execution @ 1699057896345 (clear output)
18:31:36.345 [info] Kernel acknowledged execution of cell 6 @ 1699057896345
18:31:36.349 [info] End cell 6 execution @ 1699057896349, started @ 1699057896345, elapsed time = 0.004s
18:31:36.349 [warn] Cancel all remaining cells due to cancellation or failure in execution
18:31:36.349 [debug] Cancel pending cells
18:31:36.349 [debug] Cell 6 executed with state Error
18:31:37.474 [info] Handle Execution of Cells 7 for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:37.475 [debug] start the kernel, options.disableUI=false for ~\Documents\python\datasci\helloworld\indented_comments_break_jupyter.ipynb
18:31:37.476 [debug] Execution Request Sent to Kernel for cell 7
18:31:37.481 [debug] Start cell 7 execution @ 1699057897480 (clear output)
18:31:37.481 [info] Kernel acknowledged execution of cell 7 @ 1699057897480
18:31:37.484 [info] End cell 7 execution @ 1699057897484, started @ 1699057897480, elapsed time = 0.004s
18:31:37.484 [debug] Cell 7 executed with state Success

ipynb cell samples exported to python

# %%
!jupyter --version

# %%
    # 4 whitespace indented comment throws IndentationError
print('This doesn\'t affect me!')
for x in range(0, 3):
    print("We're on time %d" % (x))

# %%
    # 4 whitespace indented comment throws IndentationError
if 1:
    print("We're here!")

# %%
 # 1 whitespace indented comment throws no error
  # 2 whitespace indented comment throws no error
   # 3 whitespace indented comment throws no error
     # 5 whitespace indented comment throws no error
      # 6 whitespace indented comment throws no error
       # 7 whitespace indented comment throws no error
        # 8 whitespace indented comment throws no error
print('This doesn\'t affect me!')
for x in range(0, 3):
    print("We're on time %d" % (x))

# %%
# 4 whitespace indent on next line throws no error; related to issue jupyter/notebook#171 reported on Jun 25, 2015
     
if 1:
    print(0)

    for b in range(2):
        print(b)

# %%
     # 5 whitespace indented comment throws IndentationError
if 1:
    print(0)

    for b in range(2):
        print(b)

# %%
       # 7 whitespace indented comment throws IndentationError
if 1:
    print(0)

    for b in range(2):
        print(b)

# %%
   # 3 whitespace indented comment throws no error
if 1:
    print(0)

    for b in range(2):
        print(b)

# %%
import sys
sys.version

Sample where I first encountered this issue:

    # Create DataFrame for cat plot using `pd.melt` using just the values from 
    # 'cholesterol', 'gluc', 'smoke', 'alco', 'active', and 'overweight'.
# https://pandas.pydata.org/docs/reference/api/pandas.melt.html
df_cat = df.loc[:,'cholesterol':'overweight'].melt(id_vars=['cardio'])

# Group and reformat the data to split it by 'cardio'. Show the counts of each feature. 
# You will have to rename one of the columns for the catplot to work correctly.
df_cat = df.groupby('cardio')
for i in range(0, 10):
    print(i)

Output is unexpected:

{
	"name": "IndentationError",
	"message": "expected an indented block after 'for' statement on line 9 (638309045.py, line 10)",
	"stack": "  Cell In[7], line 10
    print(i)
    ^
IndentationError: expected an indented block after 'for' statement on line 9
"
}

In IDLE, the code was minimally modified to not require pandas import and output is expected; indented comments are ignored.
image

This isn't a critical issue, so even if we agree this is a bug, we can also agree to triage this as a maintenance issue, because we can work around the issue by:

  • using multi-line comment syntax to preserve desired indentation
  • changing indentation of single line comments that cause the exception

Ideally, Jupyter's implementation should ignore indentation of single line comments like the CPython implementation, albeit, afaik Jupyter is using the same implementation as IDLE, yet I can't reproduce this unexpected behavior in IDLE Shell.
image

Activity

  1. maurerle commented on Feb 4, 2025

    @maurerle
    maurerle
    on Feb 4, 2025
    Contributor

    Had the same problem today and wondered quite a while what went wrong until I got this MWE:

    working:

    print("hello")
    # hello

for i in range(8):
    print(i)

    broken:

        # hello

for i in range(8):
    print(i)

    returns: Indentation error: expected an indented block after for

    Image

    Of course this happens for IPython, VSCode and Jupyter

  2. maurerle commented on Feb 4, 2025

    @maurerle
    maurerle
    on Feb 4, 2025
    Contributor

    Maybe @takluyver can take a look at this please? :)
    You seem to have written and reviewed parts of the other fix mentioned in the initial issue here :)

  3. maurerle
    added 3 commits that reference this issue on Feb 4, 2025

    fix indentation error with starting comment

    2fdb5a7

    fix indentation error with starting comment

    b0d90d4

    fix indentation error with starting comment

    2d3f2c6
  4. Carreau
    closed this as completedin #14721on Feb 5, 2025
  5. Carreau
    closed this as completedin 2d3f2c6on Feb 5, 2025
  6. Carreau
    added a commit that references this issue on Feb 5, 2025

    fix indentation error with starting comment (#14721)

    0a2e90f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions